What is RAW, technically?

Question

From a technical perspective, what exactly is RAW and how does it differ from JPG or bitmap images? I'm not asking about the pros/cons versus another format, but rather what constitutes RAW format, how is the data stored, how does it differ from JPG (other than lack of compression), and how is it different than bitmap?

Answer 1

RAW is data that is extracted from the camera at an earlier stage of processing.

This allows further processing to be carried out later on a computer at will. Notably, it avoids the lossy processing steps of demosaicing, sharpening/noise reduction, and JPEG compression being permanently applied to the resulting image file before it leaves the camera.

When you take a photo:

Here's what happens when you take a photo, and at what stages of processing the RAW and JPEG images are extracted.

1. Bayer filter

   The shutter opens and light enters the camera. The light is filtered through a bayer color filter so each pixel on the sensor only receives either red, green or blue light. This is required because without it the sensor is monochrome.

2. Sensor

   The light hits the sensor (often through tiny microlenses which help gather more light into the sensitive part of the pixel). It's then converted into millions of analogue impulses, each one corresponding to a pixel.

3. Analogue to digital conversion

   The analogue impulses are amplified to the correct amount according to the sensitivity (ISO rating) selected and then they are converted into digital values using an analogue to digital converter, at a certain bit depth. After this stage you have millions of (typically 12- or 14-bit) digital values in a stream.

   Note that the method for analogue to digital conversion is very different in CMOS and CCD sensors. For instance, a CMOS sensor contains a transistor in each pixel to assist with this, but suffice to say that at the end, you still get a bunch of digital values.

4. Black level adjustment

   The camera needs to determine the correct black level, since the sensor will not necessarily return zero for pixels that are black. For example, the ambient heat of the sensor may cause some signal. Correcting for this is often done by reading sensor data from some edge pixels which are permanently covered so light can't reach them. Then it subtracts an appropriate value from the other results. It may do other corrections on the digital data - it may try to reduce some types of noise, such as background noise caused by the sensor.

   Sometimes the digital values may then further be multiplied or divided in order to reach the right sensitivity (ISO) value, if you are using an ISO value that your sensor doesn't do natively.

   This is where RAW data is extracted.

   If you shoot RAW, none of the steps below here are permanently applied to the image file as it comes out of your camera, allowing you to apply them at will in your RAW software instead.

5. Demosaicing

   Each value is in fact either a red, green or blue value. However, you need each pixel to have all three colours - red, green and blue - for the final image. Therefore, a demosaicing algorithm has to guess the other two colour parts for each pixels, and it does this based on knowledge of surrounding pixels. There are a variety of different demosaicing algorithms with varying qualities, and it is a lossy process. If this occurs in-camera, then you are stuck with the camera's built-in algorithm.

6. Colour space conversion

   Red, green and blue in the Bayer filter are not necessarily the same hue as red, green and blue in the standard sRGB colour space. The camera then does colour correction to convert the colours into the desired colour space, which is usually sRGB.

7. White balance

   White balance correction is applied to correct for different colour temperatures of light sources while taking the picture.

8. Gamma correction

   Gamma correction is applied which converts from the linear values to gamma corrected values as required by digital image files. This correction is not a straight gamma correction; a contrast curve is applied to ensure that highlights and blacks curve off nicely giving it a nice film-like appearance.

9. Sharpening and noise reduction

   An appropriate amount of sharpening and noise reduction are applied to enhance the image and try to suppress annoying noise. There are different sharpening and noise reduction algorithms, and this is a lossy procedure. If this is done in-camera, then you are stuck with whatever sharpening and noise reduction was applied by the camera.

10. JPEG compression

    The resulting image data is compressed as a JPEG. This is also, obviously, a lossy procedure.

    This is where a JPEG file would be generated.

More about the RAW format

To create the RAW file, raw data from steps 1 to 4 above is assembled into a file format, which is usually a proprietary, manufacturer-specific file format based on the TIFF format. The data is sometimes compressed using a simple lossless compression algorithm and sometimes part of it is also encrypted.

Within the RAW file, the camera embeds a whole lot of metadata, with all the information necessary to do the extra processing steps later according to the camera's settings. This includes information like what white balance, sharpening and contrast settings are selected in the camera. RAW software can then choose to follow this advice or ignore it.

The camera also embeds a JPEG file into the RAW file, which it can then use when playing back the image in the in-camera "play" mode. This embedded JPEG adds a little to the size of RAW file but it means the camera doesn't have to apply all that processing every time the image is viewed in play mode. RAW processing software on computers largely ignores the embedded JPEG, though some software may use it for a fast preview mode or for generating fast thumbnails. It is generally a fairly low quality JPEG, good enough for previewing only.

Answer 2

Raw files aren't really images as such, but the data straight off of the camera's sensor. I think of it as freezing the state of the camera immediately after the exposure has finished, and bundling it up into a file, normally with some kind of lossless compression. The wikipedia article provides a fair amount of detail about the sort of thing that's recorded, but as a guide I'd say:

• All the levels readings from the sensor
• Some information about the sensor type (so the Raw converter knows how to interpret the sensor data)
• All of the settings currently applied to the camera
• The time and other EXIF-type meta-data associated with the image
• Possibly a JPEG thumbnail image

Bear in mind that there is no single Raw format, and all camera manufacturers are free to bundle up this sort of data in their own way. Adobe is trying to promote its DNG (Digital Negative) format, which I personally use, as a standard RAW format. A few cameras can output DNG, but most have to be converted. I do this on import into Lightroom, as one of the advantages is that it can store Lr edit data within the file, and not require xml sidecars. Another advantage of DNG is that it tends to result in smaller file sizes than native Raw files.

I would imagine that because of how most sensors are built with a Bayer filter-type design, the data from the sensor just contains levels from each photo-site on the sensor. A bitmap, for example, would have combined the RGB sites to form a single pixel.

Answer 3

RAW is a copy of the data from the sensor. JPEG has been refined.

Summary

A RAW file is a direct dump of the data which the camera captures.
A JPEG file has been derived from that RAW data by a process of sharpenning, colour correction and compression.

Detail

The heart of a digital camera is a light-sensitive chip known as a sensor.
A RAW file is pretty much a copy of the data directly from this chip when it an image is captured. As such, the image file is not readily recognisable as an image because:

• it is not in any widely recognised file format
• it has not been refined in any way

A JPEG file, on the other hand, is both a standard file format, and has had some refinement applied to the image to make it look better. These refinements include correcting for white balance and sharpenning. Also, a JPEG has typically had some compression appiled. So the process of creating a JPEG discards data which was originally present on the sensor, and which will be preserved in a RAW file.

Both a RAW file and a JPEG are kinds of bitmaps (they both represent individual pixels).

Many cameras are capable of generating both a JPEG and a RAW file for each image.

When to use each

JPEG is better if

• you want a usable file straight from your camera
• space on your memory card is at a premium
• you want to shoot images in very quick succession

RAW is better if

• you want to post-process the file
• you want to keep your options open for exposure or white balance
• you want the highest possible level of detail (I could be wrong about this - depends on the camera)

disclaimer

I have skipped some of the details here (see comments) in order to make the basics clear.

Answer 4

I think this question can better be answered by wikipedia for the most technical aspects: http://en.wikipedia.org/wiki/Raw_image_format.

In general, RAW is the exact output of the sensor. It can be compressed, or not, but the main difference between JPEG and RAW is that RAW is a loss-less format. JPEG has an 8-bit per pixel depth, while RAW can have as many bits as the sensor has. So, in order to get from RAW to JPEG you have to lose a LOT of information to get from, let's say, 12-bit per pixel to 8-bit per pixel.

RAW formats are what cameras use. After that, when you are post-processing you would use a TIF format to maintain as much information as possible. TIF is also a loss-less format and it will keep the same bit depth as RAW.

Answer 5

If you took a look at the image that your camera initially takes, you'd be disappointed in it. After an image is taken, the photo is run through what you can think of as post-processing within the camera to boost the color, contrast, sharpness, etc.

When you shoot in RAW you get roughly the same image as you would with JPG, but you get all the data necessary to "undo" all that in-camera post-processing so that you can do it differently yourself if you want to.

When you shoot in JPG, the camera records the data, processes the image, then deletes that "undo" data to save room on your memory card.

More technical info can be found here: http://en.wikipedia.org/wiki/Raw_image_format

Or if you want to qualify in a degree in RAW and camera processing check out these videos from the Google PhotoTechEDU talk: http://www.youtube.com/watch#!v=7SuDOMhUUMg&feature=PlayList&p=F7C5C8C217CF2E13&index=1&playnext=1