12
\$\begingroup\$

It seems like the Foveon sensor should be able to produce better images, because it's not dependent on the separate red, green, and blue pixels as exist on most digital cameras. However, cameras equipped with Foveon sensors are pretty much nonexistant. Why?

(Side note: This question was inspired by Bayer Filter answer where the Bayer filter potentially caused problems...)

\$\endgroup\$
1

7 Answers 7

10
\$\begingroup\$

What happened is that Sigma bought Foveon and put a lot of pressure on them to produce a sensor that is actually capable of competing with standard DSLR sensors. Now that Sigma is building the whole camera and sensor, there is a lot more focus on producing a compelling end-product.

Last year Sigma announced the SD1 which uses an APS-C (1.5X crop) sensor with 15 million photosites. They way they count Sigma calls it a 46 megapixels sensor. They have not released many details to members of the press (me at least) but is expected to be available by this summer.

There are still several Sigma cameras (DP1x, DP2s, SD15) in production which use the 1.7X Foveon sensor with 4.5 million photosites (aka 14 megapixels).

\$\endgroup\$
9
  • 6
    \$\begingroup\$ It should be noted that the use of megapixels here cannot be used in direct comparison to megapixels of bayer-type sensors. While there may be 46 million distinct photo-sensitive elements in the sensor, the image produced is a 15 megapixel image. The benefits of Foveon are lower color moire and better color definition at each image pixel. \$\endgroup\$
    – jrista
    Commented Apr 9, 2011 at 19:28
  • 5
    \$\begingroup\$ It should be noted that bayer-type sensors also have no real relation from the MP ratings they use to the final output image, because MP gives you a photosite count, three of which are required for any one output pixel. In addition any bayer sensor may have a different strength of AA filter which further impairs image clarity, while still producing the same pixel count in output. Foveon sensors do not use AA filters. \$\endgroup\$ Commented Apr 10, 2011 at 6:58
  • \$\begingroup\$ @Kendall: Bayer sensors would be most accurately described as having XYmp pixel "intersections". Bayer sensors and their image processors produce images by interpolating all the neighboring sensor photosites at each intersection to produce an RGB image pixel. That means four (not three) bayer photosites are interpolated to produce a single RGB pixel. In a 15mp bayer sensor, there are indeed 15mp "RGB pixel intersections", due to the way interpolation is performed. Just multiply the width and height of bayer image sizes to see how real bayer MP ratings are. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:59
  • 1
    \$\begingroup\$ As for AA filters, it depends on the filter whether it impairs image clarity or not. The purpose of the filter (which I believe are better described as low-pass filters) is to filter out spatial frequencies below the spatial resolution of the sensor. When a sensor does try to resolve spatial frequencies below its "nyquist limit", the resulting artifacts have a far greater detrimental effect on the image than anything else. The low-pass filter, when designed properly, will only filter out frequencies that can't be resolved to start with...thus, they don't "further" impair anything. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 6:03
  • \$\begingroup\$ Some DSLR's have low pass filters that are too strong. In the general case, however (Canon and Nikon), they seem to be just right (which one would expect, after more than a decade of manufacturing and using bayer sensors.) The current generation of CMOS bayer sensors seem to properly resolve or out-resolve all but the absolute best lenses, so any complaints about low-pass filters only apply to fringe cases (or in the case where the filter is improperly designed and too strong.) \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 6:08
7
\$\begingroup\$

It comes down to this: at least for most people, spatial resolution (especially in green range of colors) is much more important than color resolution, especially in the reds and blues. The color response curve I included in a previous answer gives at least some notion of the reason for this.

This is particularly relevant when the vast majority of pictures stored/displayed electronically are in JPEG or MPEG formats. These formats support down-sampling the chroma channels to half resolution anyway -- and (especially in the case of MPEG) that's how most pictures are stored. As such, converting data from a Foveon sensor to JPEG or MPEG format typically throws away quite a bit of the extra information you collected.

Though the benefit isn't necessarily huge, some Bayer-sensor cameras (e.g., the high-end Leaf/Phase One's) support sensor-shifting to take a series of four pictures (of a fixed subject) with the sensor shifted to different positions, so each pixel in the final picture has full color information (and still has twice as many bits for green as for red or blue, so it still fits reasonably well with normal vision).

\$\endgroup\$
7
  • \$\begingroup\$ Early Sigma cameras used JPEG compression settings (subsampling) that didn't show their sensor to the best advantage, but they fixed this. I wish I could remember where I had seen a quite graphic demonstration of the problem. \$\endgroup\$ Commented Apr 9, 2011 at 23:45
  • \$\begingroup\$ Note that the phase-shifting approach is really only practical for still subjects. There is a lot of value in gathering all data at once. \$\endgroup\$ Commented Apr 10, 2011 at 6:29
  • 1
    \$\begingroup\$ I don't really think it's relevant to compare a medium format body in any with a 35mm body, they would be used in wholly different ways anyway... I just wanted to note that while the sensor shifting is one way to potentially address the issue even for smaller cameras, that it has real drawbacks. \$\endgroup\$ Commented Apr 10, 2011 at 7:43
  • 1
    \$\begingroup\$ Also of note is that relying heavily on the observed theory that green spatial resolution is more important than blue/red resolution, leads to the generation of images that appear sharper but are less accurate. There is a tradeoff in any kind of compression of data, and throwing away 2/3 of the visible wavelengths for any given spatial location in an output image is most definitely a form of pre-image-compression not even the use of RAW formats can work around. \$\endgroup\$ Commented Apr 10, 2011 at 7:47
  • 4
    \$\begingroup\$ @Kendall: but calling it "2/3rds" is a little deceptive. Clearly, we're not recording all of the electromagnetic spectrum no matter what. So, focusing on percentage of the human vision color space covered seems much more realistic. \$\endgroup\$
    – mattdm
    Commented Apr 10, 2011 at 11:59
3
\$\begingroup\$

Foveon sensors are great in theory, but in practice they aren't a compelling choice. They're generally much lower resolution and can only compete by counting the 3 sensors at each pixel position to be individual pixels.

Sigma still produces cameras with Foveon sensors: http://blog.sigmaphoto.com/2011/faqs-the-sigma-camera-and-its-foveon-x3-direct-image-sensor/

\$\endgroup\$
15
  • \$\begingroup\$ +1 -- Does that loss of resolution affect the image quality? Sure, you've got fewer pixels, but you're getting all 24 bits per pixel, rather than 8. (No, I don't work for foveon, I'm just trying to understand ;) ) \$\endgroup\$ Commented Apr 9, 2011 at 19:05
  • \$\begingroup\$ True. It turns out that most people live better with color accuracy they get our of ~14MPix bayer-interpolated sensor, that true 24bit colors out of Foveon DSLR that has only 5Mpix resolution. \$\endgroup\$
    – che
    Commented Apr 9, 2011 at 19:06
  • 2
    \$\begingroup\$ Come to think of it, your statement about counting pixels seems kind of backwards. A 15 MP bayer camera has exactly one photosite (either red, green, or blue) at any location, yet counts a total of three of them at each location (the combination of red, green, blue) to give you that 15MP output number. You seem to be saying Foveon is misleading you while not acknowledging Bayer is doing the same thing from the other end, pretending they have 15MP of data when they really have less. How much resolution has a 15MP bayer camera got when you put on a red filter? 3.75MP of data recorded. \$\endgroup\$ Commented Apr 10, 2011 at 6:56
  • 1
    \$\begingroup\$ @Kendall: Technically speaking, a 15mp bayer sensor counts INTERSECTIONS between quads of pixels, in terms of the image produced. Bayer doesn't have less than 15mp, it simply interprets the information at each point that represents an image pixel in a certain way. All things being equal, the human eye works more like a bayer array than a Foveon, and our visual acuity/color perception is superb. I think you put too much negative weight on bayer sampling than it deserves, and too much bonus on foveon sampling. Both technologies have their pros and cons, foveons are just different than bayers. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:17
  • 1
    \$\begingroup\$ @Kendall, although each pixel of a Bayer array has a filter in front of it, they are still individual pixels with their own spatial characteristics. Sophisticated interpolation allows the red channel to incorporate information from the green and blue channels as well. \$\endgroup\$ Commented Apr 11, 2011 at 16:43
3
\$\begingroup\$

What happened to the Foveon sensor is that Sigma adopted the technology early on, but other camera companies were reluctant to do so.

That state continues to this day. Sigma continues to evolve cameras, currently offering an SD-15 DSLR, and the fixed-focal length large sensor compact cameras DP-1 and DP-2.

However recently Foveon technology seems to have been on the upswing. As another post mentioned, Sigma seems close to releasing a greatly improved Foveon sensor in the SD-1 with even better noise handling, and resolution that exceeds pretty much any consumer DSLR today (though not medium format systems). The new sensor is known to be roughly 46MP, which translated into Bayer equivalence means around 30MP of roughly equal detail to a Bayer image - that is to say, if you took the 15 million pixel output image from a RAW converted from an SD-1, and upsampled it to 30MP it would look identical to a 30MP bayer image. Only it would also lack color pattern issues a Bayer sensor might have, and have better falloff in detail. Foveon sensors have traditionally held a large dynamic range, and also very low noise at lower ISOs, but since the new sensor seems so different we need to wait to see what the characteristics are like going forward.

So what has changed for the better that allows for such advances? It' partly because we are seeing the result of steady R&D work at Foveon, but also because Sigma bought Foveon and have them focused now wholly on producing better large camera sensors. Before Foveon was trying to see what segment of the photographic market might make a good customer for the technology and as a result was a lot more scattered in goals.

Not only are the results of this focus seen in really significant resolution increases from the sensor over previous generations, but also that they technology was selected to go to Mars by the ESA:

http://translate.google.com/translate?hl=da&sl=ko&tl=en&u=http%3A%2F%2Fwww.styledb.com%2Fbbs%2Fboard.php%3Fbo_table%3DB08_news%26wr_id%3D102

Sorry for the rough translation, I cannot find a single other source for that news.

So basically what's happening for Foveon technology is that it's still evolving, just at what was seemingly a slower pace than other sensor technologies but what may end up being a leap ahead of them. We need to see what the new sensor can do to see where the state of Foveon technology really sits these days, so really this is probably a great question to review in three months time.

If you really want more information on just how it is a 15 million Foveon output image can contain as much more more detail than a 30 MP bayer output image, read this article comparing a 4.7MP Foveon sensor to a 12MP Bayer one (the Canon 5D):

http://www.ddisoftware.com/sd14-5d/

Especially note color chart resolution and ponder this interesting question - a 15MP bayer camera has only 3.75 million photosites detecting red. So if you put a traditional red filter like B&W photographers like to use, all the other sensors are blacked out and you are now shooting with a 3.75MP camera. Meanwhile a 46MP Foveon sensor with three layers of 15 million photosites detecting red/green/blue (roughly) does not care what filter you put in front of it, every pixel of output will hold data from 15 million different red sensors.

That might seem an arbitrary case, but what about tone shifts in something like a red car - or a blue sky.

For those REALLY wondering where Foveon is going at a technical level, read the latest patent from Foveon basically covering the fundamentals of what is probably the SD-1 sensor:

http://www.freepatentsonline.com/y2010/0155576.html

One last thing of note is that some form of the Foveon technology, even if not the Foveon design exactly does seem to be the future of imaging - patents have started to arrive from Sony and other companies also looking at ways to layer sensors.

\$\endgroup\$
23
  • \$\begingroup\$ See comments on my answer. The linked-to patent covers a scheme for linking multiple "pixel sensors" so they can be read in groups, reducing the need for wiring. The need for more wiring in a smaller space is a natural problem when you stack the sensors on top of each other, so this is a solution for that. It does not, unfortunately, provide a further description of the fundamentals of the SD-1 sensor. \$\endgroup\$
    – mattdm
    Commented Apr 11, 2011 at 3:45
  • \$\begingroup\$ @Kendall: I think you seriously need to reconsider the statement "a resolution that exceeds pretty much any consumer DSLR today". The 46mp spec of the SD1 is NOT the same in terms of image RESOLUTION as many DSLR's on the market today. Resolution refers to detail resolvability, and Sigma's misleading use of MP in their sensor leads people to make the very grave mistake you just have. The SD1 resolves 3200 lines, while the Canon 5D II resolves 3744 and the Sony A900 resolves 4032. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:08
  • 1
    \$\begingroup\$ Resolution and MP need to be treated distinctly when talking about the SD-1 since Sigma counts all three LAYERS of sensels at each photosite to arrive at the number 46mp. Your upsampling comment is also very subjective, and not based on all the facts. The 15mp image produced by a Foveon sensor will exhibit lower moire, particularly color moire, but it most certainly has not RESOLVED greater detail. Simply put, 3200 lines of resolution is 3200 lines of resolution, and 4032 lines of resolution is 4032 lines of resolution...the latter has more detail. Upsampling never improves resolvability. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:12
  • \$\begingroup\$ It should also be noted that human perception is most sensitive to green, less sensitive to red, and least sensitive to blue. The fact that there are half as many red/blue sensing pixels in a bayer design needs to be weighted with the simple facts of human perception. It also needs to be noted that the deficiencies of bayer interpolation used to create images is only really a problem when photographing objects of high spatial frequency that exhibit moire, and at all other times, the resulting image is plenty sufficient for the vast majority of photographs. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:42
  • 2
    \$\begingroup\$ Finally, it should also be noted that with modern Canon cameras, the use of sRAW and mRAW can produce lower resolution images that make full use of all four bayer pixels for each image pixel. No interpolation occurs when using sRAW/mRAW, however image resolution is lower (closer to Foveon image sizes). Bayer interpolation is only used when using full RAW. I think this is a great testament to bayer's versatility, and a good indication of why Canon has not yet moved to Foveon. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:45
3
\$\begingroup\$

There are two issues which have been problematic for Foveon sensors other than the problem of spatial resolution. These are both inherent to Foveon's key concept: using the spectral absorption of different depths of silicon to separate colors.

With a Bayer array, the different filters are created with dyes carefully selected to match the chosen red, green, and blue primaries. With Foveon, the distinction is entirely based on the physics of silicon, which isn't as neat a match as the marketing materials typically show. This results in the two problems.

First, the three primary colors recorded by Foveon sensors are further from the primary wavelengths that the human eye's cone cells respond to, and in fact the shape of the wavelength curve to which each depth responds is very different from that of our vision. That means the native color space of the device is a different, shifted shape from sRGB and other typical output color spaces — or from human vision. The sensor records "imaginary colors" — ones we can't really see — in some part of its color range, and other parts of the color range aren't covered perfectly. This doesn't show up as missing colors, but as a sort of color-blindness (the analogy there is actually quite good, since it's effectively the same problem), where colors which should be distinct are represented similarly.

Second, lower-frequency red light is absorbed at the deepest level, which unavoidably results in some attenuation — which means more noise in the red channel. As I understand it, noise reduction in Sigma cameras deals with this by blurring the red channel more strongly. I know that my Bayer-sensor camera exhibits, by a wide margin, more noise in the blue channel. I'm not sure if that's an inherent problem with Bayer or CMOS sensors, or if it's double problem on Foveon. (I made that its own question.)

None of this is to say that the widespread Bayer technology is perfect, or even absolutely better than Foveon. It's just that everything has its compromises, and Foveon actually turns out to have some tough ones. The big issues with Bayer (aliasing, color resolution) can be solved by throwing more pixels at the problem, given corresponding increases in noise handling. This has worked out very successfully so far, and of course it's no accident that it corresponds well to megapixel-based marketing.

Update (May 2011): Sigma has just announced the new "SD1" model, priced at around $9,700 — comparable in cost to something like the Pentax 645D medium-format camera, but with an APS-C sized sensor. It'll be interesting to see if they have, indeed, been able to address some of these issues. My speculation is that they probably have, but at the sort of cost that led them to change the target market. But even then, I'm not so sure — the maximum ISO is still 6400, which is two stops behind the current crop of Bayer sensors. (Remains to be seen, of course, if they simply decided on a more conservative limit. Without staring too much harder at the crystal ball, there's no way to tell; I'll update this again when the reviews are in, and if I'm very lucky after I get a chance to play with the camera — unfortunately unlikely at that price!)

Disclaimer: I don't have a Foveon-sensor camera (although I've used one, and it was cool!). I don't follow the technology very closely. Sigma is putting lot of research into working around or solving these problems.

\$\endgroup\$
19
  • \$\begingroup\$ All of what you say seems to be addressed in the latest sensor design if you look at the patent. In real world shooting I have found the color data to be more accurate, sometimes a lot more accurate, on average than other people I have shot the same subjects with in a group. As for resolution, bayer has been able to keep ahead with higher resolution counts but with the SD-1 sensor the bayer sensors are not at all ahead in resolution anymore. \$\endgroup\$ Commented Apr 10, 2011 at 23:39
  • \$\begingroup\$ Can you summarize the improvements? Are they basically work-arounds or is it something more clever than that? \$\endgroup\$
    – mattdm
    Commented Apr 10, 2011 at 23:42
  • \$\begingroup\$ If you read through the patent link I posted in my response it may help. But one of them seems to be slightly different pairs (perhaps more than pairs) of blue sensors per underlying red/green photosite, that does a better job of separating out the wavelengths and possibly moving the range covered to better match the visible spectrum. Also the design supposedly reduces read noise considerably, and we have read in interviews from Sigma that the "native" ISO is now 200, where it used to be 100. \$\endgroup\$ Commented Apr 10, 2011 at 23:54
  • \$\begingroup\$ Hmmm. Patents are mind-numbing to read, since they're legal documents, but on quick skim, the one you link to seems to be concerned with a more efficient means of wiring the sensor to reduce read noise, not the issues I describe. \$\endgroup\$
    – mattdm
    Commented Apr 11, 2011 at 0:03
  • \$\begingroup\$ The extra blue sensors totally change everything you were talking about. Remember that today the Foveon sensors as they are already do an excellent job rendering colors in real-world use. \$\endgroup\$ Commented Apr 11, 2011 at 3:08
1
\$\begingroup\$

The biggest reason "nobody" uses Foveon, I think, has little to do with Foveon and a lot to do with Sigma. Had Canon or Sony bought up the tech instead of Sigma, it would be mainstream by now, the basic idea is a good one. Sigma is a bit-player in this field, too small to do it all by themselves, and Sigma cameras are something of an acquired taste.

\$\endgroup\$
3
  • 1
    \$\begingroup\$ Okay then; why didn't Canon or Nikon jump on it then? I'm sure it was pitched to them; they must have had some problem with it in order to reject it... \$\endgroup\$ Commented Apr 10, 2011 at 16:36
  • \$\begingroup\$ This is very true but one part of the core question is why a larger camera manufacturer is not and has not tried using the Foveon technology in a camera. \$\endgroup\$ Commented Apr 10, 2011 at 23:42
  • \$\begingroup\$ I would say the reason is a base of investment. Other sensor manufacturers have an extensive existing base of design, infrastructure, manufacturing, and support for bayer-type sensors. It can cost hundreds of millions to even billions to invest in new CMOS design and manufacturing. Despite Kendall's admirable dedication to Foveon, the differences between the two technologies are not nearly as large as they are often made out to be. Canon and Sony (as Nikon currently uses Sony sensors) have little reason to change yet. \$\endgroup\$
    – jrista
    Commented Apr 11, 2011 at 5:38
0
\$\begingroup\$

The sensor is fine ... or at least it was up to the 45Mp Merrill version. With the later Quattro version Sigma has abandoned the "pure" approach of capturing three colours at each location for a compromise, with fewer sensors in the lower layers.

But the sensor is not the problem. Anyone using it knows that it excels at low ISO, but is inferior to Bayer sensors with comparable REAL resolution at high ISO.

The real problem is that Sigma cameras are frustratingly slow and inconvenient to use, especially because of the absurdly slow write times. In the early days of affordable digital cameras we'd have been delighted with the SD1, but once you have got used to the speed of a good DSLR from Nikon or Canon it is hard to go back to waiting for two minutes for a burst of 7 shots to write to the card, and until that completes you cannot check your exposures, and you do not have full use of the camera's controls.

What is more, the camera makers continue to wring more and more performance out of the Bayer technology. It reminds me of the Porsche 911. The engine is in the wrong place, but with enough clever engineering the car can be made to handle as well as many better balanced front or mid-engined machines.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.