Why don't cameras capture dynamic range as our eyes do?

Question

When sitting in a room with no lights on, and I look out the window, I can easily see the interior of the room even if I focus on a tree outside.

Why can a camera not capture a similar image to what my eyes can see? I would think that newer cameras should be able to capture this much dynamic range easily. I do not believe that display is a problem if this much dynamic range is captured, because it can be normalized. In a digital camera I have to set exposure which will only capture outer scene or inside scene correctly.

Is this only an issue with digital cameras or is it same for film cameras?

A similar question is already discussed here How to capture the scene exactly as my eyes can see?. I am not talking about resolution, focusing or detail. I am interested in exposure or dynamic range similar to when we fix our eyes on a single scene.

Answer 1

The reason you can see such a large dynamic range isn't that the eye, as an optical device, can actually capture such a range - the reason is that your brain can combine information from lots and lots of "exposures" from the eyes and create an HDR panorama of the scene in front of you.

The eye is pretty poor from an image quality standpoint but it has a very high "frame rate" and can change sensitivity, direction and focus very quickly.

The brain takes all those images from the eye and create the image you think you see - this includes details from images at different sensitivity and even details that are completely made up based on what you expected to see. (This is one reason why there are optical illusions - the brain can be fooled into "seeing" things that aren't really there).

So, you can see with your camera just like with your eye, just take lots of exposures at different settings then load everything into Photoshop, create an HDR panorama and use "content aware fill" to fill the gaps.

By the way, why cameras "should" be able to capture that range but monitors shouldn't be able to reproduce it? If technology that doesn't exist should exist then monitors should be able to reproduce anything we can see (and I should be able to take a vacation at a low gravity hotel on the moon)

Answer 2

You may have a slight advantage in sensor dynamic range over a camera, but most of what makes the difference is having a sophisticated autoexposure system, saccades, HDR processing, and a scene recognition system that persists across multiple exposures. The human brain is at least as important to the visual system as the eye is.

Presented with a scene having a very high dynamic range, the human visual system takes some time to adapt. That's not because we have to adjust a dynamic range setting, but because we need to analyse the very bright and very dark parts of the scene separately, then glue the important parts of the image together. An awful lot of what we "see" actually depends on already knowing what's there; we can use a very few indications of real detail to fill in the blanks (and when we don't have enough real information, we can interpolate — but not always correctly).

Getting a camera — any camera — to operate at that level will mean designing a system that "knows" what it is looking at. We can already do the "dumb" version of that using various HDR techniques (in your specific example, usually by simple masking where the doorway would be cut out of the darkness exposure and a version from the bright exposure inserted in its place). Current automated process are based entirely on brightness (since they can't analyze for meaning or importance), and tend to produce obvious artifacts. And if you've ever seen a raw 32-bit HDR-combined image that has not yet been tonemapped (which is essentially the sort of thing you'd get solely by increasing the dynamic range of the sensor), you will probably have noticed that the image is very "flat" and lacking in both local and global contrast. It's knowing what the scene is that allows us to do the mapping, to decide where contrast is locally important. Until the camera can make the same sort of decisions, it won't be able to produce an image that looks anything like what your brain sees.

Answer 3

It's to do with the way the brain interprets the information provided by the eyes (or to put it another way, it's the software not the hardware).

We only see colour and detail within a very narrow field in the centre of our vision. To build up the detailed colourful image we perceive, the brain moves this central spot around without us knowing.

I'm not a neurobiologist but it stands to reason that as the brain is making up this wider picture from lots of tiny snapshots it also does some normalisation on the brightness yielding an image that appears roughly the same brightness everywhere, despite some areas being much brighter in reality. Basically the ability to see dark and bright things at the same time is an illusion.

There's no reason this behaviour can't be imitated by digital cameras, nor is there any reason we can't make sensors capable of much greater dynamic range in a single exposure. In fact Fuji manufactured a sensor with extra low sensitivity photosites to capture extra highlight detail.

The problem comes down to the inability to display high dynamic range images. In order to display such images on a standard low dynamic range monitor you need to do some special processing called tonemapping, which has it's own set of disadvantages. To most consumers high dynamic range cameras would simply be more hassle.

Answer 4

Summary:

• God made our eyes.

• We make cameras.

• We haven't caught up with God yet.

• BUT the best camera available are about up to the requirement that you describe.

• There are ways of achieving what you want. You have simply decided to define them as not what you want. That is your choice.

Light level in a darkened room with a window open to an exterior scene may be as low as about 0.1 lux (0.1 lumen per square metre.) The outside scene light level may be anything from 10's to thousands of lux in the situation you describe.

At 100 lux external and 0.1 lux internal the ratio is 1000:1 or just under 10 bits of dynamic range. Many modern cameras could differentiate tonal differences at both ends of this range is set correctly. If the tree light level was just saturating the sensor then you'd have about 4 bits of level available inside the room = 16 levels of lighting. so you could see some degree of detail with brightest level EXCEPT THAT theat level of light is so low that eyes would have trouble with it.

If the tree light level was 1000 lux (= 1% of full sunlight) you'd need about 13 bits of dynamic range. The very best 35mm full frame cameras available would handle this. Camera adjustment would need to be spot-on and you would have about zero tonal information inside the room. This level of external lighting is higher than you would get in other than a flood-lit night time situation.

Many modern medium to top end DSLRs have inbuilt HDR processing that allows far greater dynamic ranges to be obtained by combining multiple images. Even a 2 image HDR photo would easily accommodate your scene. My Sony A77 offers up to +/- 6 EV 3 frame HDR. That will give well over 20 bits of dynamic range - allowing very adequate tonal variations at top and bottom ends in your example.

Answer 5

Is it the problem of digital cameras only or is it same for film cameras?

None of the answers have touched this yet, directly at least... yes, it is very much an issue with film, too. The famous Fuji Velvia colour transparency film, for example, has a truly rotten dynamic range (great colour though!) Transparency film in general suffers from this. On the other hand, negative films can have very good dynamic range, about as good as the best current digital cameras. It is handled a bit differently, though - while digital has a linear response to light, film tends to have a marked "S" contrast curve built-in. The blacks and almost-blacks, and whites and almost-whites, are bunched up more than the middle tones.

Keep in mind that as film photos will generally end up printed in ink on a white paper background, there is a not too generous limit on how much dynamic range one would want it to capture in the first place! Capturing, say, a thirty-stop dynamic range and then outputting it to a... what is the ballpark DR of a photographic print anyway? Five stops? Six? ...output medium would look... odd, to say the least. I suspect that it is this factor more than any unsurmountable hurdles with the chemistry that has limited photographic film dynamic range. It is not so much that we cannot do it, it is more that we actively don't want to do it.

Answer 6

Enough stuff to fill a book - but the simple gist of it is, that human eyes see brightness logarithmically while cameras "see" brightness linearly.

So if you assume a condition where the brightness goes from 1 to 10000 (randomly chosen number), in log base 10, the human eye would see the brightness as 0 to 5 while the camera, linearly, sees it as 1 to 10000. Building a sensor that can cover such a large range is difficult as you have noise interfering with low measurements and overspill interfering with higher brightness measurements. Having said that, I believe there is a RED camera that can record 18 stops of dynamical range - not sure if it is only a prototype or production model though.

By the way the logarithmic vs. linear difference is also why brightness doubles or halves per one stop difference.

But this is enough for a research topic - so this is just a brief pointer.

Answer 7

The eye doesn't capture dynamic range. It compresses dynamic range, and then the "post processing" in the brain creates the illusion of dynamic range. A compressed dynamic range is why you can see into shadows and lit areas at the same time. The "gain", so to speak, is automatically cranked up in the parts of the retina that is sensing the shadows, making them brighter, and reduced where the retina is seeing lit areas. The brain still knows that it's looking into a shadow so it creates a sensation that it is dark there. A kind of expansion over the compressed data is going on, so to speak, so that you're not aware that the dynamic range has been compressed.

The sensors in digital cameras could easily outperform the retina in raw dynamic range. The problem is that you don't control the exposure on a per-area basis. Cameras have gain settings (usually presented in film terminology as ISO settings) which are global.

What the eye does, so to speak, is somethign like using "ISO 100" for a bright area and "ISO 800" for a dark area at the same time.

If the camera could adjust gain for specific areas of pixels based on brightness, that would be undoubtedly useful, but we know from applying such gain-leveling effects in post-processing that the brain is not really fooled by them. It does not look natural. It looks natural only when your own eye is doing it in coordination with your own brain.

Answer 8

This is sort of an interesting question if you give it chance instead of bringing up the obvious reasons why cameras are already made the way they are made.

Let's consider the closest option. Tone Mapping is a method in which a low-pass filter is applied on the exponent values of the RGBe image. That plays a large part in how eyes see something. But let's consider that our eyes are taking in lengthy steams of imagery. They work a lot more like video cameras than photo cameras.

Tone mapping could be greatly improved if it was built like a GLSL shader that ran in real-time with a specialized video camera that could capture a constant stream of HDR images.

In a much more simplified example, the iPhone's "HDR" photos are composites of a low and high exposure image pushed through a tone-mapping process that works fairly well if you haven't tried it. Many other consumer-grade cameras do similar things.

There is also the fascinating subject of how intuition/intention/free-will plays into how your eyes are being calibrated along the stream of time. If you're looking at a dark wall and think about turning your head towards a window that is brightly lit your brain can tell your eyes to go ahead and start closing up your pupils. A camera with automatic exposure can do the same thing but only after there's too much light coming in. People who work in cinema spend a lot of time getting the timing of movie cameras' settings to flow smoothly so that they feel natural in a complicated shot (or lighting a scene in such a way that the cameras' settings don't actually have to be adjusted) But again, the only reason those sorts of things work is because the director knows what's going to happen to the camera before it happens.

Answer 9

The biggest problem would be reproducing the captured image.

It's not outside the realm of technology to create an image sensor and configuration that would capture an extremely wide range of brightness levels in a single image. In the end it's just a matter of photon-counting, which is a technology that does scale to the necessary levels. Current cameras primarily use exposure settings to modulate the amount of brightness that the sensor sees, though more of this work could be done in the sensor, perhaps resulting in greater error noise, but you could certainly get a wider range out of a photo sensor than what is currently available on the market.

But the problem is this: once you have that picture, what do you do with it? Even high-end displays still use 24-bit color, meaning only 256 shades per color channel allowed. Current printers are similarly limited, if not more so. So nothing could actually be done with such an image without some processing first to reduce the range down to what existing cameras produce.

You've probably seen this problem before: most current RAW formats already store a wider range than can be reproduced, and the color range already has to be compressed or clipped before you can look at the picture. Adding even more range to the RAW output would just be more of the same. The camera would likely be dramatically more expensive but the pictures wouldn't be significantly better because you still have to chop the range down to 24-bit color before you can look at it.

Still, perhaps with the right software and the right kind of user, you may be able to get something wonderful out of it. It'd probably be not very unlike current HDR photography, but you wouldn't have to snap multiple images.