Time to be with your loved ones

Time to be with loved ones

by sat

submit your photo


Hall of Fame
View past winners from this year

Please participate in Meta
and help us grow.

Take the 2-minute tour ×
Photography Stack Exchange is a question and answer site for professional, enthusiast and amateur photographers. It's 100% free, no registration required.

From a technical perspective, what exactly is RAW and how does it differ from JPG or bitmap images? I'm not asking about the pros/cons versus another format, but rather what constitutes RAW format, how is the data stored, how does it differ from JPG (other than lack of compression), and how is it different than bitmap?

share|improve this question
add comment

5 Answers

RAW is data that is extracted from the camera at an earlier stage of processing.

This allows further processing to be carried out later on a computer at will. Notably, it avoids the lossy processing steps of demosaicing, sharpening/noise reduction, and JPEG compression being permanently applied to the resulting image file before it leaves the camera.

When you take a photo:

Here's what happens when you take a photo, and at what stages of processing the RAW and JPEG images are extracted.

  1. Bayer filter

    The shutter opens and light enters the camera. The light is filtered through a bayer color filter so each pixel on the sensor only receives either red, green or blue light. This is required because without it the sensor is monochrome.

  2. Sensor

    The light hits the sensor (often through tiny microlenses which help gather more light into the sensitive part of the pixel). It's then converted into millions of analogue impulses, each one corresponding to a pixel.

  3. Analogue to digital conversion

    The analogue impulses are amplified to the correct amount according to the sensitivity (ISO rating) selected and then they are converted into digital values using an analogue to digital converter, at a certain bit depth. After this stage you have millions of (typically 12- or 14-bit) digital values in a stream.

    Note that the method for analogue to digital conversion is very different in CMOS and CCD sensors. For instance, a CMOS sensor contains a transistor in each pixel to assist with this, but suffice to say that at the end, you still get a bunch of digital values.

  4. Black level adjustment

    The camera needs to determine the correct black level, since the sensor will not necessarily return zero for pixels that are black. For example, the ambient heat of the sensor may cause some signal. Correcting for this is often done by reading sensor data from some edge pixels which are permanently covered so light can't reach them. Then it subtracts an appropriate value from the other results. It may do other corrections on the digital data - it may try to reduce some types of noise, such as background noise caused by the sensor.

    Sometimes the digital values may then further be multiplied or divided in order to reach the right sensitivity (ISO) value, if you are using an ISO value that your sensor doesn't do natively.

    This is where RAW data is extracted.

    If you shoot RAW, none of the steps below here are permanently applied to the image file as it comes out of your camera, allowing you to apply them at will in your RAW software instead.

  5. Demosaicing

    Each value is in fact either a red, green or blue value. However, you need each pixel to have all three colours - red, green and blue - for the final image. Therefore, a demosaicing algorithm has to guess the other two colour parts for each pixels, and it does this based on knowledge of surrounding pixels. There are a variety of different demosaicing algorithms with varying qualities, and it is a lossy process. If this occurs in-camera, then you are stuck with the camera's built-in algorithm.

  6. Colour space conversion

    Red, green and blue in the Bayer filter are not necessarily the same hue as red, green and blue in the standard sRGB colour space. The camera then does colour correction to convert the colours into the desired colour space, which is usually sRGB.

  7. White balance

    White balance correction is applied to correct for different colour temperatures of light sources while taking the picture.

  8. Gamma correction

    Gamma correction is applied which converts from the linear values to gamma corrected values as required by digital image files. This correction is not a straight gamma correction; a contrast curve is applied to ensure that highlights and blacks curve off nicely.

  9. Sharpening and noise reduction

    An appropriate amount of sharpening and noise reduction are applied to enhance the image and try to suppress annoying noise. There are different sharpening and noise reduction algorithms, and this is a lossy procedure. If this is done in-camera, then you are stuck with whatever sharpening and noise reduction was applied by the camera.

  10. JPEG compression

    The resulting image data is compressed as a JPEG. This is also, obviously, a lossy procedure.

    This is where a JPEG file would be generated.

More about the RAW format

To create the RAW file, raw data from steps 1 to 4 above is assembled into a file format, which is usually a proprietary file format based on the TIFF format. The data is sometimes compressed using a simple lossless compression algorithm and sometimes part of it is also encrypted.

Within the RAW file, the camera embeds a whole lot of metadata, with all the information necessary to do the extra processing steps later according to the camera's settings. This includes information like what white balance, sharpening and contrast settings are selected in the camera. RAW software can then choose to follow this advice or ignore it.

The camera also embeds a small JPEG file into the RAW file, which it can then use when playing back the image in the in-camera "play" mode. This embedded JPEG adds a little to the size of RAW file but it means the camera doesn't have to apply all that processing every time the image is viewed in play mode. RAW processing software on computers largely ignores the embedded JPEG, though some software may use it for a fast preview mode or for generating fast thumbnails. It is generally a fairly low quality JPEG, good enough for previewing only.

share|improve this answer
    
Note that some cameras actually do apply some noise reduction at step 4. (Sony A850, for example.) –  mattdm Feb 7 '11 at 14:21
9  
+1 for the "timeline view", btw. –  mattdm Feb 7 '11 at 14:21
1  
Yeah. It's to keep people from editing their posts gratuitously to bump them to the front page. I think it's not fair when you really work on a post to make it better (and mods can reverse it on a case-by-case basis). –  mattdm Jan 31 '13 at 0:50
1  
Note: generally, demosaicing algorithms (step 5) will also do some sharpening as part of the demosaicing. This will often be in addition to further sharpening done at step 9. –  thomasrutter Apr 22 '13 at 2:48
1  
Thomas, in case it's converted to community wiki you can flag to a moderator to revert it, don't worry! And I'd say, keep on improving it! Side note: does the in-camera raw processor apply lens correction? –  clabacchio Jan 15 at 23:17
show 5 more comments

RAW is a copy of the data from the sensor. JPEG has been refined.

Summary

A RAW file is a direct dump of the data which the camera captures.
A JPEG file has been derived from that RAW data by a process of sharpenning, colour correction and compression.

Detail

The heart of a digital camera is a light-sensitive chip known as a sensor.
A RAW file is pretty much a copy of the data directly from this chip when it an image is captured. As such, the image file is not readily recognisable as an image because:

  • it is not in any widely recognised file format
  • it has not been refined in any way

A JPEG file, on the other hand, is both a standard file format, and has had some refinement applied to the image to make it look better. These refinements include correcting for white balance and sharpenning. Also, a JPEG has typically had some compression appiled. So the process of creating a JPEG discards data which was originally present on the sensor, and which will be preserved in a RAW file.

Both a RAW file and a JPEG are kinds of bitmaps (they both represent individual pixels).

Many cameras are capable of generating both a JPEG and a RAW file for each image.

When to use each

JPEG is better if

  • you want a usable file straight from your camera
  • space on your memory card is at a premium
  • you want to shoot images in very quick succession

RAW is better if

  • you want to post-process the file
  • you want to keep your options open for exposure or white balance
  • you want the highest possible level of detail (I could be wrong about this - depends on the camera)

disclaimer

I have skipped some of the details here (see comments) in order to make the basics clear.

share|improve this answer
1  
The camera still applies all the post-processing to the RAW file, it just saves it in a reversible way. To demonstrate this, you can put your camera (in RAW) in black and white mode, take a photo, and you'd get a black and white image even though your camera took a color photo, then converted it. This common demo is used to show the flexibility of RAW because you can open the B&W RAW file in software and change the camera mode to "Normal" and get the color version back. If it was shot in JPG, you'd lose the color version. –  Erica Marshall Jul 26 '10 at 13:57
1  
Full disclosure: I didn't remember in time to edit: In RAW not all post-processing is applied, but color enhancement and correction like white balance is. Things like sharpness and noise reduction are not applied. Sorry for the double comment. –  Erica Marshall Jul 26 '10 at 14:09
    
+1 this answer doesn't deserve a demotion at all. The camera only processes a small jpeg as a thumbnail to display to the user, it doesn't process the original information when using RAW. –  Rezlaj Jul 26 '10 at 15:03
    
Thanks, @Rezlaj ;) –  AJ Finch Jul 27 '10 at 9:00
1  
@Erica: I think what you are trying to say is that the camera hardware applies a tone curve to raw image data, which they do. Each channel is processed by applying a tone curve, which effectively "adds" the proper white balance and contrast. In reality, however, no real processing is being done, the tone curve just attenuates the image when it is rendered to a screen. If a linear tone curve is applied to each channel, the original raw state can still be viewed (although the image would look rather drab.) –  jrista Jul 30 '10 at 0:27
show 1 more comment

Raw files aren't really images as such, but the data straight off of the camera's sensor. I think of it as freezing the state of the camera immediately after the exposure has finished, and bundling it up into a file, normally with some kind of lossless compression. The wikipedia article provides a fair amount of detail about the sort of thing that's recorded, but as a guide I'd say:

  • All the levels readings from the sensor
  • Some information about the sensor type (so the Raw converter knows how to interpret the sensor data)
  • All of the settings currently applied to the camera
  • The time and other EXIF-type meta-data associated with the image
  • Possibly a JPEG thumbnail image

Bear in mind that there is no single Raw format, and all camera manufacturers are free to bundle up this sort of data in their own way. Adobe is trying to promote its DNG (Digital Negative) format, which I personally use, as a standard RAW format. A few cameras can output DNG, but most have to be converted. I do this on import into Lightroom, as one of the advantages is that it can store Lr edit data within the file, and not require xml sidecars. Another advantage of DNG is that it tends to result in smaller file sizes than native Raw files.

I would imagine that because of how most sensors are built with a Bayer filter-type design, the data from the sensor just contains levels from each photo-site on the sensor. A bitmap, for example, would have combined the RGB sites to form a single pixel.

share|improve this answer
3  
Adobe's attempt to standardize RAW formats: en.wikipedia.org/wiki/Digital_Negative_(file_format) Also, RAW files are often loslessly compressed. –  Eruditass Jul 26 '10 at 15:13
3  
The DNG format is often compressed better than a camera's native RAW format, so if storage space is a consideration then that's worth bearing in mind. Anecdotally, this seems to be less of an issue with newer camera models. –  Nick Miners Jul 26 '10 at 15:42
1  
I don't think DNG is the topic of discussion in this question. –  Dave Van den Eynde Jul 26 '10 at 19:33
1  
@DaveVandenEynde - DNG is a type of RAW, so it is at least related. –  Fake Name Jan 17 '13 at 4:55
1  
@DaveVandenEynde - I don't know where you're getting the idea that a RAW image in any way has to be proprietary or not open, but it's absolutely not true. Furthermore, there is no reason there should be any loss of information when converting to DNG. It may happen in some situations, but that's more indicative that the conversion from .CR2/.NEF/whatever to DNG isn't very good. –  Fake Name Jan 17 '13 at 22:55
show 12 more comments

If you took a look at the image that your camera initially takes, you'd be disappointed in it. After an image is taken, the photo is run through what you can think of as post-processing within the camera to boost the color, contrast, sharpness, etc.

When you shoot in RAW you get roughly the same image as you would with JPG, but you get all the data necessary to "undo" all that in-camera post-processing so that you can do it differently yourself if you want to.

When you shoot in JPG, the camera records the data, processes the image, then deletes that "undo" data to save room on your memory card.

More technical info can be found here: http://en.wikipedia.org/wiki/Raw_image_format

Or if you want to qualify in a degree in RAW and camera processing check out these videos from the Google PhotoTechEDU talk: http://www.youtube.com/watch#!v=7SuDOMhUUMg&feature=PlayList&p=F7C5C8C217CF2E13&index=1&playnext=1

share|improve this answer
    
Thinking of it as "undo" data works conceptually but isn't technically accurate. And since the question says "technically".... –  mattdm Feb 7 '11 at 16:52
add comment

I think this question can better be answered by wikipedia for the most technical aspects: http://en.wikipedia.org/wiki/Raw_image_format.

In general, RAW is the exact output of the sensor. It can be compressed, or not, but the main difference between JPEG and RAW is that RAW is a loss-less format. JPEG has an 8-bit per pixel depth, while RAW can have as many bits as the sensor has. So, in order to get from RAW to JPEG you have to lose a LOT of information to get from, let's say, 12-bit per pixel to 8-bit per pixel.

RAW formats are what cameras use. After that, when you are post-processing you would use a TIF format to maintain as much information as possible. TIF is also a loss-less format and it will keep the same bit depth as RAW.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.