Why do cameras use a single exposure rather than integrating across many very quick reads?

Question

I have never understood why cameras need a shutter with a specific speed and why this cannot be adjusted in postprocessing. I think that present sensor works in an integral way: they save the amount of light reaching them during all the time the shutter is open. But why can't they work in a differential way?

In my mind I have this idea: set the shutter speed to be open for a long time, more that the one you need... for example in daylight set it to 1 second, press your button, the shutter opens and the sensor starts to record, but in a differential way: it would save the amount of light reaching it every 0.001 second for 1 second. In this way I have more information, actually I have 1000 frames recorded in 1 second and in postprocessing I can choose to integrate only the first ten, to simulate a shot with 0.01 second, or the first hundred, to simulate a shot with a 0.1 second exposure

Using either sophisticated processing or by manually selecting areas, I could even decide to use a different exposure for different parts of the final image, for example an exposure of 0.1 second for the ground and 0.03 for the sky, using 100 frames for the sky and 30 frames for the sky.

Does it make sense? Why don't cameras work in this way?

Answer 1

The problem is the read-out speed. You cannot read the entire sensor fast enough for this to work in the general case. Even if you could. there would also be a detrimental effect on image quality as you would be applying read-out noise over and over again.

At least with CMOS sensors you can read at random locations but with CCD sensors each row is shifted into the next to perform the read-out. This is why when lights are too bright you get vertical streaks in the preview of cameras using CCDs.

Then there are reasons why photographers choose a shutter-speed: to freeze a slice of time. People would almost always be blurred if you did not stop the shutter fast enough.

Answer 2

I think they could work that way, but there's two big problems:

1. As sensors work now, read noise only happens once. With many very short reads, read noise would be compounded.
2. There's just more data than we can possibly handle. This is both a problem in actually reading the sensor (too much data to read) and resulting RAW files would be orders of magnitude larger than they are today.

In your original question, you used ¹⁄₁₀₀th of a second as an example. That's actually not a very long shutter speed at all, and is already 10× where we're at with current cameras. For this to be really meaningful, we'd need the technology to the point where the sensor is read 10 to 100 times faster than that — which compounds the problems significantly.

The second problem will be solved by Moore's law in a decade or two, and the first may be as well, as electronics get smaller and more precise. For the present, it's not practical.

The Olympus OM-D E-M5 actually has a feature where it shows the exposure "developing" in long exposures, but that avoids the problems above by requiring a minimum of half a second between reads: it's only useful for long exposures.

But, in the future, with better electronics, all cameras will probably work this way, and with much more storage available, they might as well record continuously. The "shutter" button would simply serve to mark a part of the stream as interesting, for later development. And while we're at it, ditch the lens and make this a 360º light-field camera; framing, aperture, and focus can be selected after the fact as well, throwing the conventional wisdom about camera basics completely out the window

Answer 3

As lots of people have mentioned, there is the read speed aspect of this. The imaging circuitry cannot simply "acquire" the sensor pixel values instantaneously, they must be read line by line.

Then, you also have to think about where these pixel update values go. Even with some compression, assuming low entropy of later frames relative to earlier frames they will take considerable amount of space. And also, data that happens not to very compressible will slow the entire process down - since the compressibility of a new frame relative to the old cannot be guaranteed, the system would still need to have enough bandwidth to cover the worst case scenario without degrading.

Finally, (and this is somewhat corny), the effect of the Heisenberg inequality, or uncertainty principle, needs to be considered. At every sample, we have a level of uncertainty about the measurement. By taking very many samples (each with which we apparently have low confidence, otherwise we could just select a single frame of the thousand or so), we are obtaining uncertainty on every one of those frames, rather than just once. This would be compounded when combining multiple frames, and now the maximum uncertainty is multiplied by the number of frames you are composing the final image with. In the worst case scenario, this could degrade images considerably.

Answer 4

While your idea is intriguing, there are several reasons that it wouldn't work.

File size: Think about how large the resulting image file would be. My 8mp SLR gives files around 3mb in size. If it was taking 100 images every time I pressed the shutter, I would have 300mb taken up on my card. A card would fill up too fast. Also, these are just numbers for my camera, which has a relatively low resolution. For pro cameras the size could easily double or triple that. In Raw, the size could increase another three times or so. In the end, I pro shooting Raw could end up with over 3gb per shot.

Exposure: Sometimes, a camera needs to expose for longer than 1/100 sec. to get the proper amount of light. If you shot in too low of light, the resulting images would be underexposed and potentially unusable. You can't just combine images to make up for this, as you can't find data that isn't there.

Noise: As sensors heat up, they display a phenomenon known as noise. This is the speckling of randomly colored pixels apparent in some photos. If the sensor was constantly working for a second at a time, the noise levels would quickly rise, potentially leading to unuseable images.

Card Write Speed: Memory cards are limited in how fast they can have information added. This is known as their write speed. To handle files of this size, cards would need a fast write speed. These cards can be extremely expensive.

So, to recap, this is an interesting idea, but is one with several large hurdles in its way.

I hope that this helped.

Answer 5

It might be noted that since this question has been asked, this idea has been somewhat implemented in (at least some) smartphones.

As the technological limits did not improved that much, it is of course restricted to very long exposures in very low light (e.g., astrophotography). The fact that the software stacks multiple n-second exposures is visible if there is movement in the frame (They will have some slight discontinuities on readout).

The stacking can even be viewed in real time on the screen as the 'result' image appear increasingly brighter.

Answer 6

There are cameras that essentially work that way. Search for High Dynamic Range (HDR) cameras. The least expensive ones work by bracketing the exposure, essentially taking two or four different exposures, then combining them in their internal software into a unified image. You can actually do this with any camera, using external software.

My understanding of how the higher-end HDR cameras work is they basically dynamically calculate a separate auto-exposure time for each individual pixel, and use that to determine the actual brightness of the scene. That solves the issue of having tons and tons of data for each pixel, you basically just store the exposure time and the color values when it crosses a certain brightness threshold. Obviously, this requires much more complex hardware and software to get right.

Answer 7

There are a few different types of shutters but the mains ones are mechanical and electronic shutters (the sensor is turned on/off).

Most DSLRs use the focal plane shutter while video cameras use electronic shutters. So while it sounds like what you want to do in integrate them, there are other ways of achieving that.

The main reason for a shutter is for exposure, a fundamental principle of photography.

You could probably set something up for your example. You might want to look at bracketing or if that's not enough, tethering.

0.01 = 1/100 and is fine shutter speed. Most cameras support speeds up to 1/8000 seconds.

Just on your last part:

let say that it saves the amount of light reaching it every 0.01 second for 1 second.

You'd still need to think about the exposure for that shot, every 0.01 second.

Answer 8

Aside from the points everyone else made - whats going to happen to exposure ?

e.g. with my conventional camera a 1 second exposure is what I need for a given shot. Essentially this means 1 seconds worth of light recorded by the sensor in one go.

Ok, lets go to the suggestion you made, the camera records 0.001 seconds worth of light 1000 times. All of these images are underexposed and therefore useless. If I stack them I do not get the same image as my conventional camera because the stacking doesn't fix the underexposure. To get anything near correct exposure I'd need a very wide aperture, which (even if you could find a suitable lens) would have other issues - depth of field etc. If you mean that the sensor would be sensitive enough to actually capture enough light in 0.001 of a second so that the image wouldn't be underexposed we'd have a severe problem with noise. The idea you mention of being to include/exclude frames to get a different exposure is already handled in PP with the exposure slider.

You could try out your idea by taking shots with a short shutter (as you suggest) and then stacking them in either HDR software or startrail software.

The post processing stuff you mention about taking the ground and sky with different exposures to balance the image can already be rectified in PP with shadow/highlight controls and/or split toning.

Also, think about action photography like motorsport where I need a long shutter whilst panning to give a sense of speed and movement. If I have say 200 images with a very small shutter time I won't catch the blur and the effect of the shot doesn't work.