Why does Image Stabilization have a Limit?

Question

Now that there is a CIPA standard for measuring image stabilization, more and more manufacturers are quoting the efficiency of their stabilization in stops or half-stops. Yesterday, for example, Olympus launched their M.Zuiko 12-100mm F/4 IS PRO which has built-in image stabilization and, combined with 5-axis in-body stabilization present in high-end Olympus mirrorless such as the OM-D E-M5 Mark II gives 6.5 stops of stabilization according the the CIPA standard.

That seems like an incredible amount of stabilization. Understanding the meaning of Stop that would mean it is possible to shoot at 12mm with shutter-speeds of up to 2.6s and at 100mm with speeds of 1/3s! This is calculated using the 1/effective-focal-length rule-of-thumb. Still, even if this is off by an entire stop, it would remain extremely impressive.

The question is though, if a stabilization can stabilize for that long, why does it stop there? Why can't it just keep doing what it's doing and stabilize for 5 or 10s or longer? What makes it stop working after a while?

Answer 1

What makes it stop working after a while?

Educated guess: Error.

An image stabilization system is like navigation by dead reckoning, in which you figure out where you are based on what you know about where you were, your speed, and changes in direction.

If you're in a car traveling at 60mph for 5 minutes, you know you're going to be about 5 miles from where you started. You might be off a little bit if the car is actually moving at 59 or 61 mph, but you'll end up within easy walking distance of your predicted location, so close enough. But, if you try to predict where the car will be after an hour instead of just 5 minutes, that same small 1 mph error will accumulate over that longer time period, and you'll end up a full mile from your expected location. That may be a larger error than you're willing to accept.

It's the same thing with an image stabilization system. The camera doesn't have an absolute point of reference in space — its accelerometers and gyros can only measure relative displacement and rotation, and although they're very accurate they're not perfect. Moreover, the hardware that moves the sensor or lease element that keep the image stable will have some error of its own. Some error is also inherent in active IS systems due to the fact that the system has to sense movement before it can react, so there's bound to be a delay that causes the system not to track the camera's movement perfectly. Finally, it's likely that no IS system can ensure perfect corner-to-corner image registration while it's compensating for camera motion.

All these errors will accumulate over time. A good IS system might be able to make a handheld 10 s shot better than what you'd get without IS, but not so much better that the manufacturers are willing to claim that it's useful at such a long exposure setting.

In other words: It doesn't stop working; it just reaches a point where it's not sufficiently helpful.

Answer 2

I suspect that one main problem is accumulated error.

No measurement is perfect. There's always an error. The image stabilisation has to measure the relative movement of the camera and counteract it.

During the exposure, many measurements occur. Each one builds on the result of the previous one. This means that the error also builds up. At some point the total error is considered to be too big. I guess the standard specifies that with some threshold for the total error and the probability at which it is reached after a certain amount of time.

Answer 3

You are correct that if the motion were cyclical and never exceeded the limits of the stabilization systems maximum travel then it should be able to last indefinitely. But if motion is in the same direction along an axis eventually the system reaches the limit of its travel.

The main limit is with regard to the extent of the range of motion that can be accommodated before the stabilization system reaches the edge of its travel. If a compensating system can keep up with a motion in the same direction for only 3° before it reaches the end of its travel then any movement in excess of 1° per second means the system can only maintain compensation for 3 seconds at the most.

The limits of the sensors travel are eventually determined by the limits of the image circle cast by the lens. Even if the sensor could move twice as far, it would not help if that means part of the sensor is now outside the edge of the image circle cast by the lens.

With sensor based stabilization the problem is compounded when using longer lenses because it takes less angular movement of a longer focal length lens to produce the same blur as a shorter focal length lens. A 600mm lens with a full frame system has a diagonal FoV of only about 4°. A 1° angular movement is equivalent to 1/4 (25%) of the entire frame! In contrast, a 35mm lens has a diagonal FoV of 63°. A 1° movement is only equivalent to 1/63 or less than 1.6% of the entire frame.

That is the main reason that as they have begun offering longer focal length lenses the makers that use camera based stabilization have also begun to support it with lens based compensation as well. Lens based stabilization systems are usually very near the center of the lens, where a very small movement can affect a much larger shift in the spot the projected cone of light moves where it strikes the sensor.

Answer 4

According to Olympus themselves, the rotation of the earth is stopping them going beyond 6.5 stops (and then something to do with the gyroscope).

I read this on an article today on PetaPixel, who themselves lifted it from Amateur Photographic where they had an interview with Olympus Deputy Division Manager Setsuya Kataoka:

The in-body stabilization itself gives 5.5 stops, and the Sync IS gives 6.5 stops with OIS lenses. 6.5 stops is actually a theoretical limitation at the moment due to rotation of the earth interfering with gyro sensors.

Answer 5

The numbers don't really reflect any kind of hard limit, they reflect a probability. We can consider camera shake effectively random, so any shot has a chance of being blurred by camera shake. The longer the exposure, the higher that chance is that the shake will add up to enough to spoil the image. Image stabilization can cancel out most of the shake under reasonable conditions, but not all of it, for reasons that others have explained — the acceleration sensors aren't perfect, the motors don't react instantly, there are physical limits to the motion, etc. The left-over bit of camera shake still contributes to the probability of a blurred image, it just does so more slowly because there's less of it. If they're claiming 6 stops of improvement, it means that the shake-induced blur accumulates 1/64th as fast on average with IS on as it does with IS off, but every shot is different. You can have good luck without IS, and bad luck with it. Actual testing for IS involves taking a large number of shots at varying shutter speeds with IS on and off, and either comparing the fraction of acceptable images or the average amount of blur between the two populations. If a certain camera/lens combo gets an acceptable image 90% of the time at 1/30s with IS off, but can still get an acceptable image 90% of the time at 1s with IS on, then that's a data point showing 5 stops of improvement. With a lot of data points like that, we can summarize the performance (or, if we're the marketing department, choose the best ones).

Answer 6

The photographer and the camera are essentially open-loop system. The photographer gives the input by pointing the camera on the subject, and the camera has no means to influence this input. Because of this, accumulated error soon overwhelms the useful picture data if stabilisation over longer period is attempted.

Note that in other applications like astronomy, positioning systems are directly controlled by the imaging process, making the system closed-loop: the telescope follows the object being shot. As a result, stabilisation periods of several seconds or even minutes are not unheard of. Here's an example of a telescope designed to take picture of objects as faint as magnitude 24, which stabilises the picture for up to 1 minute:

There is a grain of truth in Paul's answer after all, but those techniques are unlikely to be applied to photography any time soon. Perhaps some day cameras will have neuro-nterfaces to take control over photographer's hands, but lenses with stabilisation times of many seconds will have to wait until then.

Answer 7

The question is though, if a stabilization can stabilize for that long, why does it stop there? Why can't it just keep doing what it's doing and stabilize for 5 or 10s or longer? What makes it stop working after a while?

The various image stabilized Canon lenses I had did not stop the motion entirely. They only slowed it down. From observing the effect in the viewfinder it was clear that the exposures can't be infinite. All my IS lenses were in the 70-300mm range, the effect is possibly not so obvious with short lenses that allow really low exposures, but I suspect the outcome is similar.

Answer 8

It's probably somewhat doubtful that the 2+ second exposure (even with a short lens) will come out very well very often.

When a person is holding a camera, you have a number of fundamentally different movements involved. They differ in both frequency and magnitude. Image stabilizers work well with movements caused by muscle tremor, which are (relatively speaking) high in frequency and small in magnitude. That works well for exposures up to, say, a tenth of a second or so.

With exposures of multiple seconds, you have entirely different sorts of movements to deal with. For example, most of your upper body moves somewhat as you breathe. This movement is much slower, but also (in a lot of cases) much larger. This leads to two problems. First of all, it's slow enough that most accelerometers aren't calibrated to measure them very well. Second (and harder to deal with) typical stabilization systems can only move a few millimeters or so. Movement from breathing can be much larger than that.

Even just standing completely still for multiple seconds at a time becomes difficult. This becomes particularly obvious if you try to do hand-held macro photography. If you're very close (with minimal depth of field) it's often difficult to stand still enough to just keep a subject well focused. Again, the movements here are often on the order of (for example) centimeters instead of the millimeters for which stabilization systems can typically compensate well.

Answer 9

In practice when extreme precision is required, one resorts to nested systems, where within a reasonably accurate stabilized system that is optimized to damp out large movements, you put a more sophisticated system that can compensate for tiny fluctuations in movements that are the residuals of the first system. And within that system you can put another one etc. etc. Camera stabilization systems use one layer, so there is a lot of room for improvement (but the costs would likely be prohibitive).

Such systems typically use both passive and active damping mechanisms. You want the second layer to be isolated from the first layer, so there is a passive damping system that links the layers. There is also an active system to compensate for movements. In a layered system this is best done by measuring the movement of the previous layer and then calculating the propagation through the damping mechanism to get to the required compensation.

The LIGO experiment is a good example where such methods are used to get to extremely accurate compensation of vibrations.

Answer 10

Interesting question, but I think some premises are wrong.

it is possible to shoot at 12mm with shutter-speeds of up to 2.6s

Really? Will the photographer will be standing still for 2.6 seconds?

A physical image stabilisation system rely on one physical property of the matter: inertia.

It is like the trick of pulling the cloth on the table and leaving the dishes alone.

If it is somehow loose one from another you can move one piece at some extent without moving the other piece.

They also are designed to some type of frequencies.

A pendulum has a frequency to resonate. If you create some equilibrium with a broom upside down you are applying this same principle. But you need to compensate at the proper speed.

Imagine now that you want to reframe an image and the image stabilization system prevents doing that. "Oh no, that is a shake, I will stay in place!".

Yes. A big telescope has more mass and I am sure that reframing take some more time than a hand held camera. But on a hand held camera you have some limits on the stabilization.

By the way, the other device that provides longer stabilization is called a tripod. And rely on the mass of the Earth.

Answer 11

I'll probably get a nice number of downvotes again...but all the above answers are wrong from the beginning to the end. And the answer is already in your question:

there is a CIPA standard for measuring image stabilization

That's all. The concept here is "frame of reference": as there is a standard, there must be a way to test all cameras in the same way and produce a number that is a valid indicator, i.e. it is "comparable" across cameras.

---

CIPA Test: how it works

(and probably in-house tests before CIPA standardization, too)

As "there is a CIPA standard for measuring image stabilization", 5 stop (e.g.) of stabilization is the result of a standard test that measure under specific conditions how much the camera can be pushed before a certain set of things happens (namely, bokeh degradation and motion blur).

_{Note: there are -at least- 50 pages in the CIPA image stabilization test procedures manual. And I don't remember them all, nor I've got the brain to understand every aspect of them (even if I produce software for vibrations testing platforms :-D); following explanation is a large oversimplification, if someone wants to go into fine details he can just read the procedure by himself, it's publicy available}

The CIPA standard use a vibratory platform to test the camera. That's the magic.

The camera is put on a platform that produce vibrations and aimed to a "standard image"; the platform is powered off and a reference shot is taken. Then the platform is powered on, a set of vibrations are produced, a lot of shots are taken at different shutter speeds, and the moment the camera start producing bad photos is the moment the IS is not able to correct the exposition. Then just imagine that difference between the initial shutter speed and the last good one, expressed in stop, it's the amount of stops the camera stabilization system is able to manage.

---

Moreover, there is a problem with the question you posed:

it is possible to shoot at 12mm with shutter-speeds of up to 2.6s and at 100mm with speeds of 1/3s! This is calculated using the 1/effective-focal-length rule-of-thumb

Why is not possible to shot at 100mm with shutter speed longer than 1/3? Simple because you have imposed it yourself in the example! :-)

If you establish that handeld you can shot 100mm at maximum 1/100s, and you then apply 5 stops and it results in 1/3s at max...it's because you made the math, not because the image stabilization system will shut down after 1/3 of a seconds, nor because it will start to work badly after that time! Indeed, image stabilization systems are tested (if I remember correctly) with exposures up to 32 secs :-D

You set the frame of reference here, saying "I take the 1/mm rule and apply the stop factor", so you forced yourself in the corner. What if someone with a really steady hand can shot handeld 100mm@1sec? Does the system stop working after 1/3s even for him because you can't go more than 100mm@1/100th of second?

Answer 12

The image stabilization is controlled by MEMS gyroscopes. While I do not have full information on the usage in cameras, I can work backward. Starting with the fact that MEMS gyroscopes are used to measure the rotation of the earth at man y universities and research centers. These gyroscopes are used in sensors. When a gyroscope is pushed off of its axis, it exerts a force to maintain its position. This force can then be measured. Processing of this measurement can then be used to determine the movement force exerted against that. In a stabilization system, this would then lead to a counter force to maintain the position with the measurements from the gyroscope controlling controlling the counter force. As the earth rotates, its force pressure on the gyroscope allows it to be measured. I notice he said a theoretical limitation of 6.5 stops. A theoretical limitation means the maximum that can be achieved with no errors and everything perfect. I question his statement that their camera is at the theoretical limit as that is never achieved. There are always physical limitations. I do not have his math for this statement. It must involve the minimum force his camera system responds to. After 6.5 stops, the force from the rotation of the earth then is greater than this minimum movement at which point the system not knowing the object the camera was pointed at had also moved, would then attempt to aim the camera where it thought the object still was. Then math for when this occurred would involve the pixel size, minimum and maximum limits it can correct for and much more involved with optics and the dampening built in the system. Which includes the human holding it. As a camera dropped from a plane and remotely triggered would not give a clear image at 1 sec much less at longer times. For cameras, I would suggest the solution for this would be an oversized sensor in the camera to move the part of the sensor the image comes from as well as the optics and physical movement of the sensor. To do this, they then need a storage area and to continuously read the sensor storing the image in the storage area and adding to what is already there. I feel this is possible with a dedicated processor and allowing a longer time the image can be stabilized for. However there is still a limit. BTW, this type of system is in use in some places where expense does not matter. Coming back to original question, it does not state where on earth this is the limit. The limit may be less on the equator and more on the poles. Also most cameras today give more stabilization with longer lens and less stops with shorter. Which again comes back to his 6.5 stops comment with no reference to the focal length nor the actual time. I would tend to think that this is more a limit of the multiple gyroscopes operating on different planes and the interaction between them as it is easy enough to have a gyroscope to determine the orientation of the camera in relation to the earths rotation and then program that into the stabilization processor. There is plenty math on this on the internet in articles about measuring the earths rotation. I hope this is a plain English explanation of why there are limitations beyond which the gyroscope system cannot go.

Answer 13

I would suggest that you are correct and that there is no absolute limit. You should be able to stabilise for 10 minutes or two hours.

Mention has been made of accumulated error in an open loop control system that is the stabilisation mechanism. Open ended control systems can drift beyond what can be compensated out. This is kids control systems 101 and the problem was solved centuries ago in mechanical engineering. Simply close the loop with feedback.

If you think of the two parts of a camera, you have a lens and a sensor. The (stabilised) lens moves to change what the sensor sees, and the sensor sees what the lens is pointing at. Connect the two with a feedback loop. A digital signal processor should be able to lock onto an image target (we have basic face tracking after all) and detect if the image has shifted. The shift is then fed back to the lens motion control and the lens shifted in the opposite direction. The trick would be in detecting the pixel level shifts. That's why we don't have these yet, but nothing I've outlined seems physically impossible. As long as the lens points at the target with some sufficient accuracy, you'll be able to expose all day.

The reason I'm confident that this will work is that it's already done sort of. Telescopes these days have active /flexible mirrors that constantly adjust their geometries to stabilise out atmospheric turbulence and self weight distortions. They also lock onto a target and track it.

Can't wait to buy a lens that can stabilise for a full day.