3

One of the Samsung Galaxy S6's new camera feature is to use its IR-based heart-rate sensor to aid in white balancing. Some of the write-ups on tech websites briefly mention how the IR sensor can 'detect the light around' or 'if a photo is being taken outdoors or indoors' for this feature to work. Are there more scientific explanations to how this works, or will we have to wait for actual reviews to be out?

The general photography angle to this is: are existing cameras DSLRs using a similar method to figure out white balance too?

  • 1
    existing cameras is an awful broad category. Some may, but many use more sophisticated and accurate methods. Others use much cruder methods. – Michael C Mar 2 '15 at 4:55
  • 1
    By comparing the ratio of visible to IR light in a scene, the camera can probably make an educated guess about the color/temperature of the light illuminating the scene. – Michael C Mar 2 '15 at 5:04
  • @MichaelClark thanks for your comment, I've restricted to DSLRs just to narrow the scope down... – h.j.k. Mar 2 '15 at 6:31
  • 2
    Same as before. DSLRs is awful broad and includes many different approaches to determining white balance. But when saving files in RAW format (something many DSLR users do) the camera doesn't figure out WB at all - the converting software does in post processing. – Michael C Mar 3 '15 at 0:54
  • @MichaelClark doesn't the camera record a white balance setting in the RAW file which the software is free to use or ignore as desired? – Mark Ransom Mar 1 '18 at 22:10
3

I know some samsung phones used MAX30102 hear rate monitor chips (datasheet: https://datasheets.maximintegrated.com/en/ds/MAX30102.pdf) and I speculate this one does as well.

These chips measure red and near infra red (NIR) light. Based on the diference of the red vs NIR intensities I guess one could estimate the warmth of the light and set the balance accordingly, eg. fluorescent, daylight, indoors.

I do not think DSLRs use this approach, I think they just check the RGB values in an image and do some smart stuff with that :). I think using a red & NIR & other colours sensor could be an interesting feature for DSLRs, althoug this just means another sensor. I think DSLRs set WB good enough as it is.

I just noticed this is a rather old question. Still it pays to try and answer it.

  • 1
    Many current DSLRs have RGB+IR light meters, rather than the older monochromatic kind. – Michael C Mar 4 '18 at 5:07
  • interesting! If you have a remote functionality you most likely already have an IR photo-diode on camera to receive the remote signal, maybe it could be used for other stuff as well. Is RGB an external light meter or is it just the cameras sensor? – Mecgrad Mar 6 '18 at 10:00
  • 1
    Not necessarily. Many remotes operate via wires (remote cable releases) or radio (anything with a range of more than 15-20 feet), rather than infrared. In general only the lower end DSLRs have IR receivers for IR remotes. The pro bodies do not. The RGB+IR light meters in DSLRs are in the same place the old monochrome light meters were: Between the focusing screen/viewscreen and the viewfinder. – Michael C Mar 7 '18 at 2:22
1

The general photography angle to this is: are existing cameras DSLRs using a similar method to figure out white balance too?

The number of methods DSLRs use to calculate white balance are probably more varied than the number of models on the market. This is because some models may use more than one method or algorithm depending on user selected settings and other conditions. For most DSLRs, if not all of them, the user can also manually select a color temperature (along the blue←→amber axis) and WB correction (along the green←→magenta axis) themselves rather than let the camera attempt to calculate white balance.

There have been three basic types of in-camera light meters popularized over the years.

Monochrome

For many years cameras did not even have built-in light meters. When they first started appearing, they were simple photovoltaic sensors that measured the intensity of the light striking them. These tended to be more sensitive to the red end of the visible spectrum than the blue end, so the same amount of low temperature reddish tinted light (say, around 2800K) would read brighter than the same amount of higher temperature blue tinted light (like around 8000K).

Over the years monochrome sensors were improved a bit, but not until some cameras started introducing multi-zone sensors for their light meters did things really start to improve. Multi-zone metering divides the frame into separate zones and measures each zone separately. This also led to different metering patterns such as spot, center weighted average, etc. by giving each zone (or only a few or even only one zone) a different weight or bias in the overall calculation.

Beginning in the late 1980s and early 1990s, some cameras that used monochrome metering also began to use 'library based' algorithms to match the information from the light meter to different lighting scenarios contained in a library in the camera's firmware. They then set exposure based on the instructions provided by the library for the closest match. The earliest such systems were very simple and rudimentary.

One very simplistic example: If the top third of the frame is very bright and the bottom two thirds much darker, the camera might assume the photographer shooting a landscape photo desires the darker landscape to be properly exposed. If the upper two thirds of the frame is very bright and only the lower one-third is dark, the camera might assume the photographer cares more about properly exposing the bright sky. Library based metering algorithms are named using brand-specific monikers such as 'Evaluative' or 'Matrix' metering modes.

Processing power and speed available to camera designers increased dramatically over time. So did the number of zones meters had. This increased data was used by ever more complex and sophisticated library based algorithms to improve metering under a wider and wider variety of lighting scenarios.

Dual layer meters

Dual layer sensors measure the intensity of light at two different areas of the spectrum and compare the results to give a reading based on the predicted overall color of the light.

Here's a detailed example that explains how the 'iFCL 63-zone Dual Layer Metering Sensor' used by the Canon EOS 7D, 60D, and 5D Mark III works:

The metering sensor has 63 measurement zones and is a Dual-layer design with each layer sensitive to different wavelengths of light. Electronic sensors in general are more sensitive to red light. This means when photographing subjects with lots of red in them – skin tones for example – the sensor receives a stronger signal as it only detects brightness levels. This can lead to the wrong assumption that there is more light than there really is.

The Dual-layer system overcomes this by having one layer sensitive to red/green light and one layer sensitive to blue/green light. Both these layers measure the light in their respective spectra and the metering algorithm then combines the two to provide an accurate light reading. In this way, accurate exposures can be attained in a wide range of shooting situations and irrespective of the colour of the subject being metered.

RGB and RGB+IR

(R=red, G=green, B=blue, IR=infrared)

Even more recently, many advanced bodies from Nikon, Canon, and others include light meters that are basically small Bayer-masked imaging sensors contained in the optical path of the viewfinder, the same location most monochromatic light meters built into cameras are found.

Depending on the camera and the amount of processing power available, these RGB+IR color light meters can not only more accurately measure the brightness of light, they can also aid in other ways. The use of increasingly sophisticated library based evaluation of the information collected by RGB+IR light meters can give such meters enhanced roles. Fifty years ago no one could have dreamed those early single cell photovoltaic meters would one day evolve into the 300,000+ pixel RGB+IR light meters we have today. They can serve to allow the camera to:

  • Adjust for unique lighting scenarios for which the color of the light can affect exposure slightly by setting WB predictively and adjusting the recommended exposure parameters before the shutter opens and the main imaging sensor is exposed to light
  • Process exposed images faster because the WB can already be analyzed and set before the data from the main imaging sensor is read
  • Assist in predictive autofocus ('Servo' or 'AF-C') by tracking specifically colored objects as they move around the frame
  • Assist in automatic selection of autofocus areas using facial recognition algorithms without requiring the use of Live view
  • Adjust exposure parameters used to capture the image and adjust contrast during in-camera processing after the image is captured to optimize exposure and skin tones for human faces detected by the RGB+IR light meter, again without requiring Live View

Here's another example that shows how the 100,000 pixel RGB+IR light meter introduced to the EOS system with the Canon EOS 1D X works:

You can think of it as a miniature version of the imaging sensor in a digital camera. Covered with Red, Green or Blue filtration over each pixel, it can read that elusive combination of brightness and color. And, because it has over 100,000 pixels, it can also read enough detail at the subject to be able to recognize many types of subjects and scenes – without resorting to Live View, and before a picture is taken.

and

The new RGB sensor for light metering is located up in the prism area of the camera, above the eyepiece optics. It reads most of the image area, looking down on the focus screen. This is typical of nearly all current digital SLRs. But extremely atypical is this sensor's ability to read color information. Because it uses the combined results from Red, Green and Blue pixels, it can ascertain subject colors, distribution of color over the frame, and therefore do a good job of recognizing many types of scenes.

Armed with this information, the metering system can apply subtle corrections when colors such as yellow or green are detected in a scene. (Green subjects, for instance, normally tend to cause slight over-exposure in an image, if they dominate a scene.) Because the RGB metering system can account for color throughout the frame, exposure is less likely to be influenced by the color of background objects, or the color of parts of a scene like a subject's clothes. The result of reading not only brightness but color information provides great exposure stability, an important consideration for pros and serious enthusiasts who are using automatic exposure along with Evaluative metering.

The number of pixels in an RGB+IR light meter can vary greatly. The Nikon D3/D700/D300 series of cameras had 1,005. The 1D X had 100,000. The Canon 80D has 7,560. The Canon 7D Mark II, 5Ds/5Ds R, and 5D Mark IV have 150,000 pixel RGB light meters. The Canon 1D X Mark II has 300,000+ pixels. The Nikon D5 has 180,000, the D4 had 91,000.

-1

As Michael Clark hints at in the comments, there must exist reliable, purely software based methods to determine how to set the white balance, since e. g. Adobe's Camera Raw can suggest its own white balance. (However, there might also be some additional data stored in the RAW files which comes from such dedicated sensors.)

I'm no expert, but I would be suprised, if there were sophisticated measuring devices. What we see as white just is not at all white. The camera has to make adjustments to match our complex perception. Therefore there is no one way to white balance a shot.

You probably have come accross the black-blue/white-gold dress. That illusion most likely is caused by your brain compensating for missing information. Your mind might suppose it's a warm, bright environment tinting everything orange and brightening the dress. When compensating for the tint and brightness the dress appears blue and black.
If on the other hand your brain thinks the scene is dark and cold, it makes the dress appear white and gold (that even rhymes).
Since the "appropriate" white balance is quite subjective, a machine can only guess.

IR-sensors only assist that guessing. As Michael Clark pointed out, they might interpret high amounts of IR light as being outside under the sun and adjusting the WB accordingly.
But the camera still is only guessing, so if you want a perfect result (i. e. what you find most pleasing) you should - as with every single aspect of photography - control it manually and not let some machine guess what you're thinking.

  • 2
    this does not appear to answer any of the three questions. it's some guesswork and tangents about manual whitebalancing which is irrelevant to the questions. – ths Mar 6 '15 at 11:33
  • 1
    Yes, I understand. I hoped, it would give some insight into white balancing and why I think those techniques are mostly chatchphrases. In retrospect, it doesn't really adress the question. – J0hj0h Mar 7 '15 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.