0

Ok, I have in mind that a picture P can be represented by a two dimensional matrix A where each element of A (aka every pixel) is a vector with four coordinates (Red,Green,Blue). What extra stuff is encoded in a RAW file except this matrix?

1
3

A RAW file does not consist of pixels with multiple color coordinates. It consists of a single value from each sensor well on the sensor. Those values must be demosaiced by an algorithm that has some knowledge of the Bayer or other filter that sits in front of the sensor wells. These algorithms take the value of each well and those from some small neighborhood of other wells, and generates an RGB pixel value for that location.

RAW files also typically include one or more pre-processed JPEG preview images, EXIF and other metadata, and maybe some other stuff, all packaged into some sort of "archive" file (some use TIFF containers with different sections for the different types of data, but that's not universal), and probably with some level of compression applied.

1

A raw image file includes single luminance values measured at each photosite (a/k/a "sensel" or "pixel well"). Each single value tells how much light energy was absorbed by each photosite. These values are listed in a specific sequence.

At this point there is no "color" information in the way you seem to expect. There is only a single luminance value for each photosite. There is no "red", "green", or "blue" value assigned to each photosite. It is true that in a color sensor, there are color filters in front of each photosite that affect how much different wavelengths of light are allowed to pass through and strike the sensor, but the colors used in Bayer masks do not correlate on a one-to-one basis to the colors we use in our color reproduction systems, such as RGB or CMYK.

The three color filters for most Bayer masked "RGB" cameras are really 'blue-with a touch of violet', 'Green with a touch of yellow', and somewhere between 'Yellow with a touch of green' (which mimics the human eye the most) and 'Yellow with a lot of orange' (which seems to be easier to implement for a CMOS sensor). Each filter does not pass "only blue" or "only green" or "only red" light. They each pass "more blue than green or red", "more green than blue or red" or "more red than blue or green", but some of all wavelengths will pass through each color of the filter array. Just as with the cones in the human retina, there's a lot of overlap between what each filter allows through.

enter image description here
The color sensitivity of a typical Bayer masked sensor, showing the response curve to wavelengths of light across the visible spectrum buy photosites filtered by each color of the bayer filter array. The four vertical lines show the wavelengths typically emitted by our RGB color reproduction systems, including those which also use a fourth Yellow channel. Please notice that the peak sensitivity for each color filter does not correspond to the wavelengths used by out color reproduction systems.

enter image description here
The sensitivities of the different cones and rods in the human retina. Notice that our "red" cones are most sensitive to 564nm light, which is more of a slightly green-tinted yellow than red.

To assign color values to each pixel of an image created using the raw information from the sensor, the relative brightnesses from adjoining photosites filtered with different colors are compared and "Red", "Green", and "Blue" values are assigned to each pixel of the resulting image. This closely mimics the way our human brains process the information from our retinas to create a perception of color from wavelengths and combinations of wavelengths of light.

In addition to the actual measurements of luminance at each photosite, a raw image file contains preview image(s) and metadata that allows a raw processing application to know some things it need to process that data. These would include (but not be limited to):

  • Preview and/or thumbnail images. Theses are basically in camera generated JPEGs at various resolutions that can be anywhere from very small thumbnail images to full resolution images. They're generated by the camera's internal JPEG processing engine based on the in-camera settings at the time the photo was taken. These preview images are almost always what one sees when they look at a photo on their camera's LCD screen. They're typically what is displayed by an operating system or image viewing application that shows "icons" or "thumbnails" of each image file in a folder or other collection of images. They're also what is often first displayed by a raw image processing application when a file is initially opened.
  • Identifying information about the sensor. In general, raw files do not tell how many lines/columns the sensor contains. It is expected that the processing application is able to access that in its database from reading the sensors ID.
  • Similarly, information about the specific colors used in the sensor's color filter array (Bayer mask), assuming it has one, are not spelled out in the raw metadata. It is expected that the raw processing application can access that information from its database.
  • Information about when the photo was taken (timestamp), what exposure settings were used (aperture/exposure duration/ISO (analog signal amplification)), what focal length, etc. These are all encoded using the EXIF standard and are (supposed to be) standardized across all camera makers and models. There are many "standard" EXIF fields.
  • Beyond the "standard" EXIF fields for metadata, cameras can also include "maker notes" in the EXIF information. This may include whatever the camera's designer wishes to include. Many camera makers include information such as the specific lens used (not only the focal length and model number, but also down to the specific serial number in some instances), values of "masked pixels" on the sensor used to determine the black level included in the standard EXIF info, etc.
  • How much of the "maker notes" section of the EXIF info is used by a raw processing application depends on the raw processing application. Adobe products, for example, ignore almost all of the "maker notes" information. If a raw image file is converted from the camera's native raw format to Adobe's DNG format, most of the "maker notes" information is stripped and discarded. Other raw processing applications may or may not use anywhere from none of the information contained in the maker notes to significant portions of that information.
  • Almost all of the "maker notes" data is coded to keep it as compact as possible. An "EF 70-200mm f/2.8 L IS II" lens may be encoded in the "lens ID" field as something like "57c3bff8" (in binary). Each camera maker, and in some instances each specific camera, will use their own proprietary codes because the EXIF standard does not designate "standard" fields or encoding schemes for such information.

Not the answer you're looking for? Browse other questions tagged or ask your own question.