25
\$\begingroup\$

I have never understood why cameras need a shutter with a specific speed and why this cannot be adjusted in postprocessing. I think that present sensor works in an integral way: they save the amount of light reaching them during all the time the shutter is open. But why can't they work in a differential way?

In my mind I have this idea: set the shutter speed to be open for a long time, more that the one you need... for example in daylight set it to 1 second, press your button, the shutter opens and the sensor starts to record, but in a differential way: it would save the amount of light reaching it every 0.001 second for 1 second. In this way I have more information, actually I have 1000 frames recorded in 1 second and in postprocessing I can choose to integrate only the first ten, to simulate a shot with 0.01 second, or the first hundred, to simulate a shot with a 0.1 second exposure

Using either sophisticated processing or by manually selecting areas, I could even decide to use a different exposure for different parts of the final image, for example an exposure of 0.1 second for the ground and 0.03 for the sky, using 100 frames for the sky and 30 frames for the sky.

Does it make sense? Why don't cameras work in this way?

\$\endgroup\$
12
  • 1
    \$\begingroup\$ Related to Why do digital cameras need an exposure time at all? \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 1:16
  • \$\begingroup\$ I had flagged this as a duplicate on first read, but it's not, at least not of the question I linked, which makes different (and incorrect) assumptions. \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 4:29
  • 1
    \$\begingroup\$ Have you looked at high speed digital video cameras before? \$\endgroup\$
    – Cascabel
    Jan 23, 2013 at 7:02
  • \$\begingroup\$ Regarding your edit.... how would the camera tell what is ground vs sky and what about people standing on the ground covering some of the sky and moving within your frame? This would require processing way beyond a different sensor design. \$\endgroup\$
    – Itai
    Jan 23, 2013 at 12:46
  • 1
    \$\begingroup\$ Guess you've never tried to achieve specific effects caused by specific combinations of shutter speed, lighting, and aperture. And you've also never had to deal with the need for very fast exposure to capture moving objects. \$\endgroup\$
    – jwenting
    Jan 23, 2013 at 20:33

8 Answers 8

17
\$\begingroup\$

The problem is the read-out speed. You cannot read the entire sensor fast enough for this to work in the general case. Even if you could. there would also be a detrimental effect on image quality as you would be applying read-out noise over and over again.

At least with CMOS sensors you can read at random locations but with CCD sensors each row is shifted into the next to perform the read-out. This is why when lights are too bright you get vertical streaks in the preview of cameras using CCDs.

Then there are reasons why photographers choose a shutter-speed: to freeze a slice of time. People would almost always be blurred if you did not stop the shutter fast enough.

\$\endgroup\$
10
  • \$\begingroup\$ what is the present read-out speed? 1 s, 0.1 s, ... ? Since normal cameras are able to record 25fps video I would say ~0.04s. It seems too slow for normal photos. \$\endgroup\$ Jan 23, 2013 at 9:33
  • 9
    \$\begingroup\$ @wiso Normal cameras are able to record 25fps video by subsampling the sensor. The best readout->process speed for the whole sensor is the 18MP Canon 1DX which can do 1/14s in JPEG and 1/12s RAW. \$\endgroup\$
    – Matt Grum
    Jan 23, 2013 at 12:29
  • \$\begingroup\$ There are also artistic reasons to leave the shutter open for a long time, such as to soften water, blur the motion of a baseball player running for a base, etc. \$\endgroup\$
    – jrista
    Jan 23, 2013 at 21:16
  • 2
    \$\begingroup\$ @wiso: No, you can't superimpose fast frames that entirely freeze motion to create the apparent sense of motion. If we take a waterfall as an example, freezing the waterfall motion at a high speed captures a lot of detail. Superimposing multiple images might increase the amount of "detail noise" in the waterfall itself, but you will never be able to create that soft flowing water effect...you quite explicitly need the shutter to be open for a longer duration of time to achieve that. The baseball player would be similar. Superimposing multiple shots won't so it, especially if you want... \$\endgroup\$
    – jrista
    Jan 24, 2013 at 4:45
  • 1
    \$\begingroup\$ @jrista: If one had a sensor which could instantaneously copy its integrated readings to an integrated readout register and zero them, while having almost no time at which the sensor was not sensitive to light, combining multiple short exposures could be better than using a single long-duration exposure (among other things, it would allow much greater flexibility in post). The biggest problem is the dead time between exposures when using present technologies. \$\endgroup\$
    – supercat
    Apr 28, 2014 at 21:19
11
\$\begingroup\$

I think they could work that way, but there's two big problems:

  1. As sensors work now, read noise only happens once. With many very short reads, read noise would be compounded.
  2. There's just more data than we can possibly handle. This is both a problem in actually reading the sensor (too much data to read) and resulting RAW files would be orders of magnitude larger than they are today.

In your original question, you used ¹⁄₁₀₀th of a second as an example. That's actually not a very long shutter speed at all, and is already 10× where we're at with current cameras. For this to be really meaningful, we'd need the technology to the point where the sensor is read 10 to 100 times faster than that — which compounds the problems significantly.

The second problem will be solved by Moore's law in a decade or two, and the first may be as well, as electronics get smaller and more precise. For the present, it's not practical.

The Olympus OM-D E-M5 actually has a feature where it shows the exposure "developing" in long exposures, but that avoids the problems above by requiring a minimum of half a second between reads: it's only useful for long exposures.

But, in the future, with better electronics, all cameras will probably work this way, and with much more storage available, they might as well record continuously. The "shutter" button would simply serve to mark a part of the stream as interesting, for later development. And while we're at it, ditch the lens and make this a 360º light-field camera; framing, aperture, and focus can be selected after the fact as well, throwing the conventional wisdom about camera basics completely out the window

\$\endgroup\$
9
  • \$\begingroup\$ Outside of formatting, I cannot really distinguish you answer and mine :) \$\endgroup\$
    – Itai
    Jan 23, 2013 at 1:52
  • \$\begingroup\$ As an interesting (to me) thought- You'd only have to store the difference between successive frames, so the amount of data wouldn't necessarily be so much greater than a single frame. Processing time might be an issue with today's technology, though. \$\endgroup\$
    – BobT
    Jan 23, 2013 at 2:47
  • \$\begingroup\$ @BobT Not even... you just need a high-bit depth accumulator or even floating point. You read the increment and add it. You only need one image in the end, not a video! HDR images are often 32-bit floating point and that is enough range for pretty much any practical use. \$\endgroup\$
    – Itai
    Jan 23, 2013 at 2:49
  • \$\begingroup\$ @Itai: yeah I noticed yours come in while I was writing and figured eh, I'll finish anyway. \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 3:08
  • \$\begingroup\$ @BobT: you'd still have to read it, and given noise (not just read noise) I'm not sure how much compression would buy you. But sure, let's do that in the theoretical camera, especially because for the most use we do want to save as much as possible. \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 4:41
4
\$\begingroup\$

As lots of people have mentioned, there is the read speed aspect of this. The imaging circuitry cannot simply "acquire" the sensor pixel values instantaneously, they must be read line by line.

Then, you also have to think about where these pixel update values go. Even with some compression, assuming low entropy of later frames relative to earlier frames they will take considerable amount of space. And also, data that happens not to very compressible will slow the entire process down - since the compressibility of a new frame relative to the old cannot be guaranteed, the system would still need to have enough bandwidth to cover the worst case scenario without degrading.

Finally, (and this is somewhat corny), the effect of the Heisenberg inequality, or uncertainty principle, needs to be considered. At every sample, we have a level of uncertainty about the measurement. By taking very many samples (each with which we apparently have low confidence, otherwise we could just select a single frame of the thousand or so), we are obtaining uncertainty on every one of those frames, rather than just once. This would be compounded when combining multiple frames, and now the maximum uncertainty is multiplied by the number of frames you are composing the final image with. In the worst case scenario, this could degrade images considerably.

\$\endgroup\$
3
  • 1
    \$\begingroup\$ dear user1207217, I am a particle physicist, let me say that the Heisenberg inequality really doesn't fit with this case. Since the sum over many frames is linear and since they are uncorrelated the noise you obtain integrating many frame is exactly the same with the one using an equivalent single shot. \$\endgroup\$ Jan 24, 2013 at 1:12
  • \$\begingroup\$ Measurement uncertainty =\= noise. I am well aware of the effect of averaging noise tends to the noise mean. In the case of 0 mean noise, this is simply 0 \$\endgroup\$ Jan 24, 2013 at 10:54
  • \$\begingroup\$ @wiso I would add, the uncertainty principle is very much applicable here. Just as it is applicable to time-frequency resolution, it is also applicable to space-time resolution as well (which is what we have in this case). We might not currently be near to the limits with current technology, but the tradeoff will certainly exist and will become a limiting factor in this type of imaging process. \$\endgroup\$ Jan 24, 2013 at 11:44
2
\$\begingroup\$

While your idea is intriguing, there are several reasons that it wouldn't work.

File size: Think about how large the resulting image file would be. My 8mp SLR gives files around 3mb in size. If it was taking 100 images every time I pressed the shutter, I would have 300mb taken up on my card. A card would fill up too fast. Also, these are just numbers for my camera, which has a relatively low resolution. For pro cameras the size could easily double or triple that. In Raw, the size could increase another three times or so. In the end, I pro shooting Raw could end up with over 3gb per shot.

Exposure: Sometimes, a camera needs to expose for longer than 1/100 sec. to get the proper amount of light. If you shot in too low of light, the resulting images would be underexposed and potentially unusable. You can't just combine images to make up for this, as you can't find data that isn't there.

Noise: As sensors heat up, they display a phenomenon known as noise. This is the speckling of randomly colored pixels apparent in some photos. If the sensor was constantly working for a second at a time, the noise levels would quickly rise, potentially leading to unuseable images.

Card Write Speed: Memory cards are limited in how fast they can have information added. This is known as their write speed. To handle files of this size, cards would need a fast write speed. These cards can be extremely expensive.

So, to recap, this is an interesting idea, but is one with several large hurdles in its way.

I hope that this helped.

\$\endgroup\$
12
  • \$\begingroup\$ Except for read noise, you can make the effect of a longer exposure by adding multiple quick frames. In fact, it's often done, but not usually for such short individual exposures as to apply to "normal" photography, and usually not in-camera (although Pentax DSLRs can do it in a limited way). \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 4:39
  • 1
    \$\begingroup\$ To build on what mattdm said: You can theoretically combine exposures, the data is there. The problem is that for very short exposures, with real sensors, the data is masked by an enormous amount of noise relative to the signal, and your resulting picture will be very meh. Better sensors with a more suitable SNR at the short exposure times would make the concept in the OP feasible. \$\endgroup\$ Jan 23, 2013 at 4:44
  • \$\begingroup\$ I think File size is not a big issue, in my mind you can choose to shoot in integral mode or in differential mode. Timelapse need a lot of memory, this doesn't mean people are not doing it. For the Exposure I think you are wrong as mattdm said and I think the noise is the same in the case you do only one shoot or summing more shoots, the only difference is that you open and close your shutter. When summing different frames you are summing the signal and the noise. I think the proof is that the sum of two independent Poissonian random variable is a still Poissonian distributed. \$\endgroup\$ Jan 23, 2013 at 15:26
  • \$\begingroup\$ The amount of noise will be roughly the same for a long or a short exposure, which for the multiple-exposure, will be integrated again and again resulting in a poor, noisy final result. \$\endgroup\$ Jan 23, 2013 at 16:38
  • 1
    \$\begingroup\$ @wiso: temperature related noise is only one source of noise. The significant factor of noise for typical digital cameras is in reading the sensor and signal amplification, which don't depend on shutter speed. \$\endgroup\$ Jan 23, 2013 at 19:26
2
\$\begingroup\$

It might be noted that since this question has been asked, this idea has been somewhat implemented in (at least some) smartphones.

As the technological limits did not improved that much, it is of course restricted to very long exposures in very low light (e.g., astrophotography). The fact that the software stacks multiple n-second exposures is visible if there is movement in the frame (They will have some slight discontinuities on readout).

The stacking can even be viewed in real time on the screen as the 'result' image appear increasingly brighter.

\$\endgroup\$
1
\$\begingroup\$

There are cameras that essentially work that way. Search for High Dynamic Range (HDR) cameras. The least expensive ones work by bracketing the exposure, essentially taking two or four different exposures, then combining them in their internal software into a unified image. You can actually do this with any camera, using external software.

My understanding of how the higher-end HDR cameras work is they basically dynamically calculate a separate auto-exposure time for each individual pixel, and use that to determine the actual brightness of the scene. That solves the issue of having tons and tons of data for each pixel, you basically just store the exposure time and the color values when it crosses a certain brightness threshold. Obviously, this requires much more complex hardware and software to get right.

\$\endgroup\$
2
  • \$\begingroup\$ interesting, I have found only this surveillance camera using a double exposure: video.boschsecurity.com/video/Introducing-the-new-HDR-camera/…, do you know specific model? \$\endgroup\$ Jan 23, 2013 at 23:58
  • \$\begingroup\$ HDR isn't really the same as what the OP is asking about, because HDR is selective in how it combines the information, as opposed to a straight integration. A camera as the OP is describing it would essentially be a device that does a 2D measure of the scene's radiant flux, a measuring of rate, and a photograph from it would be an integration of a set of its data, a display of quantity. \$\endgroup\$ Jan 24, 2013 at 0:11
0
\$\begingroup\$

There are a few different types of shutters but the mains ones are mechanical and electronic shutters (the sensor is turned on/off).

Most DSLRs use the focal plane shutter while video cameras use electronic shutters. So while it sounds like what you want to do in integrate them, there are other ways of achieving that.

The main reason for a shutter is for exposure, a fundamental principle of photography.

You could probably set something up for your example. You might want to look at bracketing or if that's not enough, tethering.

0.01 = 1/100 and is fine shutter speed. Most cameras support speeds up to 1/8000 seconds.

Just on your last part:

let say that it saves the amount of light reaching it every 0.01 second for 1 second.

You'd still need to think about the exposure for that shot, every 0.01 second.

\$\endgroup\$
9
  • \$\begingroup\$ I think this answer misses the idea behind the question. The fractional captures would be underexposed, often greatly. You'd choose the desired exposure level for a final image by deciding how many frames to integrate. \$\endgroup\$
    – mattdm
    Jan 23, 2013 at 4:22
  • \$\begingroup\$ Isn't the "idea" behind the question about having a mechanical shutter open for 1 second, then using the electronic shutter expose 100 times within that time frame? 1/100 doesn't necessarily mean underexposed at all, it depends on the lighting. And yes, that's exactly my point! You'd have to think about frames per second. And generally, you want to layer differently exposed images, not the same. Question: Why do I need a shutter. Answer: Exposure. \$\endgroup\$
    – BBking
    Jan 23, 2013 at 5:17
  • \$\begingroup\$ I'm also adding that the OP ultimately is talking about shutter speeds and is suggesting a 1/10 second result. However, that can be done with an electronic shutter that would still need a shutter speed. I feel the OP is asking about the actual shutter rather than simply shutter speeds. \$\endgroup\$
    – BBking
    Jan 23, 2013 at 5:39
  • \$\begingroup\$ @BBKing: my numbers were just an example, probably you should use the faster speed as you can. Each frame should be underexposed to be able to get te correct exposure summing different frames. Probably I have to change the title of my question from "why do I need shutter speed?" to "why do I need mechanical shutter speed". \$\endgroup\$ Jan 23, 2013 at 12:21
  • \$\begingroup\$ Sure, I know they were just examples. The problem I have with the question is that "Why do I need a (mechanical) shutter" and "Why do I need a shutter speed" (regardless of mechanical or electronic) are very different questions. If you're proposing to sample different areas of the sensor at different times, that's an interesting idea! And I believe it's doable (while the other answers argue why it's not). As to why? No one has developed it yet, that's all. You might be interested in a Lytro camera. You can choose different focus points in Post Processing. \$\endgroup\$
    – BBking
    Jan 23, 2013 at 21:09
-1
\$\begingroup\$

Aside from the points everyone else made - whats going to happen to exposure ?

e.g. with my conventional camera a 1 second exposure is what I need for a given shot. Essentially this means 1 seconds worth of light recorded by the sensor in one go.

Ok, lets go to the suggestion you made, the camera records 0.001 seconds worth of light 1000 times. All of these images are underexposed and therefore useless. If I stack them I do not get the same image as my conventional camera because the stacking doesn't fix the underexposure. To get anything near correct exposure I'd need a very wide aperture, which (even if you could find a suitable lens) would have other issues - depth of field etc. If you mean that the sensor would be sensitive enough to actually capture enough light in 0.001 of a second so that the image wouldn't be underexposed we'd have a severe problem with noise. The idea you mention of being to include/exclude frames to get a different exposure is already handled in PP with the exposure slider.

You could try out your idea by taking shots with a short shutter (as you suggest) and then stacking them in either HDR software or startrail software.

The post processing stuff you mention about taking the ground and sky with different exposures to balance the image can already be rectified in PP with shadow/highlight controls and/or split toning.

Also, think about action photography like motorsport where I need a long shutter whilst panning to give a sense of speed and movement. If I have say 200 images with a very small shutter time I won't catch the blur and the effect of the shot doesn't work.

\$\endgroup\$
2
  • 1
    \$\begingroup\$ "the stacking doesn't fix the underexposure", why not? The amount of light entering the camera in 1000 x 0.001s is the same in 1 x 1s \$\endgroup\$ Feb 2, 2013 at 22:13
  • \$\begingroup\$ wiso: try it. Lets say we have a shot that needs a 5 second exposure. Take 5 1 second exposures, stack them and see what you get. You'd have to open up the aperture (which might be impossible or undesirable). \$\endgroup\$
    – Mark
    Feb 6, 2013 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.