Aren't all digital images ultimately just pixel values between 0 - 255?

Question

I have a few incredibly basic (stupid?) questions about images; specifically, image formats and pixel values.

Forgive me, I'm not a photographer. I'm just someone that works with images, and to me, they are just rows and columns of numbers.

My questions are:

If at the core, photos are just 3 channels of pixel values [0, 255] X RBG, then how could there possibly be any difference between any two images formats? I mean, what makes a RAW different than a TIFF -- aren't these all limited to values between 0 - 255? A number is a number -- shouldn't there just be one set format possible? Or, shouldn't any two images with the same height and width be locked into having the same file size?

Further, from a numeric standpoint, what makes something like a 16-bit images different than 32-bit images? Again, an image is just an array with integer values between 0 -255.

Continuing with this perspective that a image on a computer's filesystem is just a 3-channel array of integers between 0 - 255, what is the point of compressing an image into, a lossy format like, for example, JPG? Say the compression algo changes some pixel values from 254 to 255 or whatever. So? How does that provide any savings in file size or make any impact on visual quality?

I know that there are lots of different ways to store image data. But I'm not asking about anything other than a basic 3-channel RBC image. All I know is that if someone hands me one of these, I now have an array of numbers. I have no reason to know why one array of numbers could possibly be any different than some other array of numbers from 0 to 255. I hope this makes sense. This question is not limited to the RAW format! Rather, it is about any array of pixel values

Answer 1

Sorry, but your basic premise is wrong: an image can be encoded as an array of RBG pixels with 8 bits per value, but there are a lot of other ways:

• one channel with one bit/channel (pure black and white),
• one channel with x bit/channel (grayscale formats, x will usually be 8 or 16, giving 256 or 65536 values),
• various palette-based formats (cf.GIF)
• full-colour with (at least in theory) as many channels as you wish with any required bit depth.

And that's for the image as stored in the computer's RAM during editing/viewing. I'm ignoring the various RAW image formats that exist (here and in the rest of this post).

For photography, most common are 3 channels with 8, 16 or 32 bit/channel (usually integer, but at least some programs work internally with 32-bit floating point numbers). Often there's a 4th channel (alpha), especially when the program allows the use of layers. And somewhere, the dimensions of the image array need to be stored.

There are various reasons for these different formats. For the in-memory format, an important consideration used to be the size of the data, and the speed (much faster to manipulate one 8-bit channel than 4 32-bit channels). Those are less important nowadays, but we got full colour management with various colour spaces. Some of those (e.g. prophoto RGB) need at least 16 bits/channel to keep differences between neighbouring colours small enough to avoid visible banding. And as treatments get more complicated, there are advantages to using 32-bit floating point numbers (where colours are encoded with values between 0.0 and 1.0, and the treatment allows intermediate values outside this range).

If you want to be able to store the image to file, and reload it to the same in-memory data, you'll need to use at least as many bits per channel as the im-memory format, and you must store information about image dimensions, bit depth and colour space.

Users of those images also like to store some additional information about the image (caption, title, who took the image, etc...). Again various ways to store this information.

Then there are different ways of compressing the image data for file storage. One of the simpler ones is RLE (Run Length Encoding), where you store a count and a pixel value whenever you encounter a repeated pixel value. Others, like jpeg, are a lot more complicated, but also give a lot more compression. E.g. jpeg uses a cosine transform, and throws away the (less visible) high-frequency information, giving high compression rates at the cost of information loss (there's more to it, but this is getting too long as it is).

This already gives a lot of ways to store the information on disk, but whatever way you pick, the format must be well specified to allow correct interpretation on loading the image.

Then there is a constant development in e.g. lossless compression techniques, which existing formats can't always handle.

So we end up with a variety of file formats, with various trade-offs between fidelity of the stored information, disk space occupied and speed of reading, writing and transmitting (compare the size of a non-compressed TIFF and a decent quality jpg).

---

After seeing the edited question, some additional aspects:

If you get handled an in-memory image, it will be in the form of one or more arrays. At that point, the original file format shouldn't play a role anymore. I'll assume you get handled your data with 8 bits/channel.

But you will have to know if you have a processed image or a raw image, as there are two important differences between those:

• raw images typically have 1 colour per pixel, and the pixels are usually arranged in a Bayer array with 2 green, 1 red and 1 blue pixel per square of 4 pixels. The values are proportional with the scene intensity (except very low and very high values).
• processed images can be arranged as a 2D array of records containing 3 numerical values, or as colour planes (3 2D arrays, one for each of R, G, B). In addition, the values usually are not proportional with the scene intensities. Worse, the exact relation between pixel values and scene intensities depends on the processing the image has had. And the balance between the colours has been adjusted to correspond to the response of the human eye (White Balance, red and blue are amplified relative to the green).

So if you get a raw image with 3 colour values per pixel, that raw image has had some treatment already (at least either demosaicing, or simple binning of 4 raw pixels to 1 image pixel). Whether that's acceptable, will depend on your application.

Answer 2

If at the core, photos are just 3 channels of pixel values [0, 255] X RBG,

But photos are not "just 3 channels of pixel values" even "at the core." Computer screens are typically made up of an array of RGB pixels, so if you want to display an image on a computer screen you must, at some point, map whatever image data you have into an array of RGB pixels, but that data is only a particular rendering of the image data. The data in the image might not consist of a stream of pixel values at all. In order to get pixel values from an image, you must know how how the data is formatted.

then how could there possibly be any difference between any two images formats? I mean, what makes a RAW different than a TIFF -- aren't these all limited to values between 0 - 255?

Those are two good examples, because neither one of those formats necessarily holds a rectangular array of RGB values.

RAW isn't a single format at all -- it's a sort of catch-all name for files that contain data recorded straight from an image sensor. So, a RAW file might contain a sequence of values that represent voltages read from the various sensor sites. Those sites are like image pixels, but they're not RGB pixels. In order to get RGB pixels from a RAW file, you have to interpret that data in the context of information about the sensor, the camera settings at the time, etc. In other words, you can open up a RAW file in a hex editor and look all you want, but you won't find a single RGB value.

TIFF stands for tagged image file format, and it's a very interesting format because it can contain many different representations of an image. A single TIFF file could contain the "same" image in several sizes, like a thumbnail, screen resolution image, and print resolution image, and it might also have color and grayscale versions. Did you know that fax machines typically send their data as TIFF files? In order to get RGB pixels out of a TIFF file, you need to understand not only the TIFF format, but also the format of the particular image representation within that file.

A number is a number -- shouldn't there just be one set format possible?

No. There are lots of different image formats because people each one serves a different set of needs. The lossy compression of JPEG is great for getting very small image files, but it's no good for images that will have to be edited several times. Some formats use interlacing, which makes it very fast to read the image at several different resolutions. And so on... each format offers its own mix of advantages and compromises.

Or, shouldn't any two images with the same height and width be locked into having the same file size?

No, that'd be terrible. If the size of every image file had to be essentially width * height * 3 (assuming 24-bit color), then you'd waste a lot of storage space. Most photos contain a lot of redundancy, i.e. regions where the same color is repeated many times. To save storage space, it often makes sense to eliminate that redundant information. One way to do that, for example, is run length encoding, or RLE. For example, if you have a region of 4195 consecutive pixels that are all white, it's a lot more efficient to encode that as "the next 4195 pixels are all {255, 255, 255}" instead of simply storing that many white pixels in the file. RLE is actually used in some image formats, but many formats have much more sophisticated schemes that save a lot more space, and that means that you can store many more images on a hard drive or memory card. It also makes it much faster to send the image to someone else.

Continuing with this perspective that a image on a computer's filesystem is just a 3-channel array of integers between 0 - 255, what is the point of compressing an image into, a lossy format like, for example, JPG?

The point is that it makes the file much smaller. JPEG compression frequently reduces the size of a file by a factor of 10 or more. That means that you can fit more images on a given storage device, you can copy them faster, you can open them faster, and you can upload and download them faster. Storing the same image (or very nearly so) in a much smaller space uses resources more efficiently, and therefore reduces cost. Think about that on a large scale: it's likely that a very large percentage of the information available on the Internet consists of images and movies, and without compression we'd need more or larger data centers and consume much more energy.

Say the compression algo changes some pixel values from 254 to 255 or whatever. So? How does that provide any savings in file size or make any impact on visual quality?

Consider my RLE example above. Let's say you have a photo that includes a large blank wall, so large areas of your photo are all the same color, except that there are a scattering of slightly darker pixels, barely even noticeable in the image. Those pixels reduce the effectiveness of the compression. Instead of being able to just say "the next 500,000 pixels are all {243, 251, 227}," you have to run length encode many more much smaller chunks, because every so often you run into one of those slightly different pixels. If you allow the compression algorithm to make small changes, perhaps only changing any pixel by no more than 1% or 2%, then you can get a much higher compression ratio without perceptibly changing the image. It's a trade-off: you're giving up a small amount of the information in the original image in return for a big reduction in file size. Exactly where you want to draw that line may change, so lossy formats like JPEG let the user choose what level of compression he/she wants.

Answer 3

In addition to @remco's fantastic answer, I want to add why there are different codecs for (roughly) the same purpose.

Codecs are designed to:

• Be lossless vs. lossy
• Encode fast vs. reduce filesize
• Asymmetric vs. Symmetric en-/decoding
• Be compatible with software
• Be perceptionally almost lossless in different compression levels / situations
• Have features that other codecs do not offer, including:
  • being royalty-free
  • support for layers
  • support for alpha-channel (e.g. RGBA) / transparrency
  • offer fast web view
  • support high(er) bit depth
  • support multiple color spaces (RGB/CMYK)
  • support for metadata / versioning / ...

Some of those things are mutually exclusive. And because of that, we are left with a multitude of codecs.

^{Obligatory xkcd}

---

A few examples

Note: Neither is the list of codecs complete, nor are all of their features (or the lack of it) mentioned. If this answer proves to be useful to somebody, I might add some more information (and be a bit more precise).

Perhaps the most commonly known format is JPEG. It is a very broadly supported, but old format. It uses DCT (Discrete Cosine Transformation), so while it offers quite good quality at its highest quality settings, blocking will appear with the lower ones.

Then JPEG 2000 came along to replace JPEG: It is based on the Wavelet-Transformation, so while it offers roughly the same quality as JPEG in the higher quality settings, it offers much better quality in the lower quality settings (blocks are a bit blurry). Also, JPEG 2000 offers regions of interest (high quality at one area of the picture, lower quality somewhere else) and 16bit support. (Also, some other things.) Unfortunately(?), because it is more computational expensive than JPEG and because of some licensing concerns, JPEG 2000 is not as broadly supported as JPEG.

PNG is another broadly known format - it is lossless and supports alpha-channels, but it does not offer support for non-RGB color spaces (like CMYK). Therefore, it is an "online only"-format.

Then there are the VFX formats like OpenEXR. They all revolve around quality and speed: OpenEXR is lossless, supports up to 64bit, and encodes/decodes fast. It is mainly used in the VFX industry as intermediate format.

TIFF is another lossless format that is quite popular with photographers. For compression, it offers none/ZIP/RLE/LZW/JPEG. It supports up to 32bit. With its selectable compression, it is quite adaptive, yet because of its losslessness, it is more of a offline-format.

HEIF is one of the latest image codecs. It uses the same compression as HEVC/h.265 and is therefore expected to give a better compression ratio than JPEG. However, because it is quite new and because it is subject to patents, it is not as broadly supported as any of the above.

RAW images^{See also} are not real pictures, really: They are more of a container for the raw (hence the name) sensor readout data. Only with software that knows how to interpret the data it is possible to get a picture. That also is why RAW converters like Lightroom / Capture One / DarkTable / ... need updates to support new cameras that use already specified containers like *.CR2 for Canon. It is also the reason why a 14bit RAW offers more editing options than a 32bit TIFF you exported out of the same RAW.

---

Intermisision: Lossless vs. lossy

I am still not sure what you are really asking, so I thought that it would not hurt to add a small explanation about lossless vs. lossy.

Lossless compression works by doing run-length encoding (RLE) / Huffman coding / ... to compress the data. The data itself is not altered, but saved in a smaller package. For example, take RLE: Say, we have a R-channel bitstream (from pixel 0,0 to pixel 0,11) of 255,255,255,255,255,215,215,235,100,000,000,000 - RLE would encode this as 52552215123511003000 - this is much smaller, and since we know that it is saved in groups of 4 digits and that the first digit is the counter and the last three digits are the value, then we can reconstruct the full 255,255,255,255,255,215,215,235,100,000,000,000.

Lossy compression, on the other hand, tries to compress even further than lossless can do. To do this, lossy codecs usually try to remove things our perception does not get. Take, for example, the YUV (YCbCr, really) model JPEG (and almost every video codec) uses: Y = Luminance, Cb = Chrominance Blue, Cr = Chrominance Red. A human cannot make out the difference between a 4:2:0 (every pixel has a luminance value, but colors are saved in blocks of 2x2 alternatingly) and a 4:4:4 (every pixel has luminance and both color channels) encoded picture. This is due to the physiology of our eye: We cannot see differences in color as well as we can see differences in luminance.

This works well most of the time, but compare it with an MP3 file: Almost noone can make out differences between 192kbps and 320kbps, but go below 64kbps and things get ugly quickly. Also, re-encoding will further reduce the quality, as unwanted artifacts might appear (e.g. in JPEG, small blocks from high quality encodings will be considered as details of the picture in further encodings).

---

Bottom line

If you do not care about image formats or their features, either one will be okay. With high enough quality settings, it is possible and expectable that you will not even see a difference between them.

If, however, you need any specific feature, there might (and almost certainly: will) be a codec that has that covered.

Answer 4

There are several reasons why this assumption is incorrect, and they all come down to one thing:

What scale are you actually using?

And that can be broken down a little further:

What is 255?

"Color" is not a property of the physical universe. It is a sensation that arises in the mind. And, that includes things like "blue", "green", and "red". A scale from 0 meaning "no blue at all" to 255 meaning "all the blue!" can't actually have 255 represent the platonic ideal of blue, because... there is no such perfect thing in the real world. So, does it mean:

• the bluest kind of thing you can make on the device in front of you?
• as close to the ideal match to pure blue from a human vision system point of view, even if most screens and printer/ink/paper combinations can't represent it?
• a pretty good blue that's likely to be reasonably represented on a wide variety of devices?
• a blue that is outside of the range of human vision, but which allows your RGB triple cover most colors which are in range?

Sound contrived? Nope! These are actually real examples. Check out these representations of each choice. The curved area is a 2D slice of the human vision color space, and the triangle shows the area which can be represented given a particular choice for red, green, or blue.

First, here's the profile for my laptop screen, which is pretty representative of current mid-range devices:

Now, here's the Adobe RGB space. Notice how much bigger this is than what my screen can show!

So, here's sRGB — the defacto standard and default space usually assumed when nothing is specified. It's meant to be "good enough" in most situations.

And finally, ProPhoto RGB, which use imaginary colors as primaries, in order to make the triangle big enough to fit almost all of human vision.

Now throw in the color of light itself, and chromatic adaptation — the human vision system's ability to adjust perception to the environment. In fact, not just ability: thing that happens whether you want it to or not. Does "pure blue" mean that thing looks as blue as it can possibly be under this incandescent light? What should the value be if we instead photograph in sunlight?

So "255" can mean a lot of different things.

What is 0?

This is fairly simple — how black do you need 0 to be? Is it vantablack black? If it is, but all of the actual shades in your scene are much less extreme, do you really want to "waste" a bunch of potential values for a dynamic range which isn't in your scene — and which, like color, can't even be represented by any device or printer you have access to?

What's your curve?

So, once you have your endpoints, how do you get from one to another? The human perception of brightness is decidedly non-linear. In your 0-255 scale, should 100 be twice as bright as 50, or should it be some greater factor? Should the perceptual difference between, say, 3 and 4 be the same as that between 203 and 204?

If you decide to use a log storage system, should that curve be optimized to match human vision, or for data optimization, or for something else?

There are many possibilities, for many different needs.

On compression

You ask.

Say the compression algo changes some pixel values from 254 to 255 or whatever. So? How does that provide any savings in file size or make any impact on visual quality?

Modern compression algorithms are more complicated than this, but this provides a good example. I'm going to use hexadecimal FF to represent 255 and FE to represent 254, and imagine we are using run length encoding as a form of compression. And for simplicity, let's assume black and white instead of color. With that, if we have a row of data that looks like this:

FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 

we can compress that to a very simple

16×FF 

... which is a pretty obvious savings. We can basically store 16 bytes in two (one for the count, two for the data). But lets say we have:

FF FF FE FF FE FF FF FF FF FF FE FE FE FF FE FE

Now, run-length encoding gives us:

2×FF 1×FE 1×FF 1×FE 5×FF 3×FE 1×FF 2×FE

... which is no savings at all, and in fact could have increased file size. But if we round all FE values to FF, we're back to the first case, with a significant size reduction, with a small but probably hard to notice impact on file quality.

Of course that's a trivial, contrived example, but all lossy compression algorithms share this basic trait: the loss of data makes it easier to use a more compact storage format, with, hopefully, not too much perceived change.

On bit depth

Further, from a numeric standpoint, what makes something like a 16-bit images different than 32-bit images? Again, an image is just an array with integer values between 0-255.

So..... an array of integer values between 0-255 is an eight bit array. (2⁸ = 256.) With three channels, this is a 24-bit image; some formats have a transparency ("alpha") channel as well, for 32 bits. One can also use a higher value per channel, which is usually what we mean when we say a "16 bit depth". That means the array goes from 0-65535 (2¹⁶ = 65536) rather than 0-255. Generally in such a scheme, this is basically just a multiplier where the highest value represents the same thing on each scale, but the higher bit depth gives more possible nuance. (See this answer for more on this.) There are also some specialized file formats which use 64-bit floats (!) instead of integers for the values, or other data types depending on the use case, but the basic concept is the same.

Answer 5

If at the core, photos are just 3 channels of pixel values [0, 255] X RBG

That is a seriously broken assumption and the rest of your question is simply not answerable without breaking away from it.

I mean, what makes a RAW different than a TIFF -- aren't these all limited to values between 0 - 255?

The term "raw" can refer to two different things, a "camera raw" image or a file that contains raw image data with no headers.

A "camera raw" image stores the raw data as it comes out of the sensor. Most modern camera sensors have ADCs with more than 8 bits, but they also only gather intensity data for one color compoenent at each location. The geometry may be distorted by the lens, the intensity values from the ADC may not do a good job of reflecting a humans perception of intensity, the color components may not map exactly to those used by your monitor and so-on.

A complicated mapping process involving interpolation is needed to turn the raw sensor data into a good quality RGB image and there is no one right way of doing it. Furthermore due to the need to interpolate color components the RGB image may end up larger than the raw data.

The conversion can be (and often is) done in the camera but many photographers perffer to save the raw data so that they can tweak the processing after the fact.

Tiff is a complex file format that can store images in a wide variety of different formats with a wide variety of metadata. In practice though it is usually used to store uncompressed or losslessly compressed RGB or CMYK images.

Files that contain raw image data with no headers are rarely used because you have to know their format and dimensions before you can read them. Some image processing tools support them though.

Further, from a numeric standpoint, what makes something like a 16-bit images different than 32-bit images?

Unfortunately "n bit" can mean two different things. It can mean that all color components are crammed into a n bit number (e.g. 5 bits for red, 5 bits for blue and 6 bits for green for 16 bit or 8 bits of red, 8 bits of green, 8 bits of blue and 8 bits of alpha for 32 bit) or at can mean that each color component has n bits of information at each pixel location.

Continuing with this perspective that a image on a computer's filesystem is just a 3-channel array of integers between 0 - 255

Again this perspective is just plain wrong.

A file is a sequence of bytes, but those bytes are almost never "just a 3-channel array of integers between 0 - 255"

You could store an image like that. Some tools even support reading and writing such files but the problem is that it means you have to know about the file before you can read it. Suppose you had such a file that was 3000 bytes in size, do you have 1000 24 bit RGB pixels? 3000 8 bit greyscale pixels? 3000 8 bit pixels from a pallete? What order are the color components in? what shape is the image? are the color components in the order RGB or BGR? Unless you know the answers to these questions you can't meaningfully read such a file.

So practical image formats typically start out with one or more headers which identify the type of file, the dimensions of the image and how the actual image data is stored. They may also contain optional metadata.

what is the point of compressing an image into, a lossy format like, for example, JPG? Say the compression algo changes some pixel values from 254 to 255 or whatever. So? How does that provide any savings in file size or make any impact on visual quality?

Compression algorithms don't merely "change values", they encode the information in a totally different manner, for example JPEG can be roughly described as

• Convert the data from RGB to YUV
• (optionally) reduce the resoloution of the chroma channels by a factor of 2 in one or both dimensions
• Split the data for each channel into 8x8 blocks.
• Convert the blocks to the frequency domain using a discrete cosine transform
• Quantise the results, preserving low frequency information while reducing the precision of high frequency information.
• Encode the resulting numbers as a sequence of bytes using a variable length encoding scheme (either huffman coding or arithmetic coding)
• Save those bytes in the file along with appropriate headers.

Losslessly compressed formats on the other hand often build on general purpose data compression algorithsm but sometimes supplement then with image-specific pre-processing, for example PNG looks like.

• Convert the data to one of the supported formats (e.g. a bits each for Red, green and blue in that order)
• For each line of the image perform a "filtering" processes, there are serveral filtering options (including no filtering at all) but the general aim is to take the image-specific information that a pixel is likely to be similar to it's neighbours and encode it in a way that "deflate" can deal with.
• Compress the filtered data using the "deflate" general purpose compression algorithm.
• Save those bytes in the file along with appropriate headers.

Answer 6

No, an image is not just RGB values in the range 0-255. Even if you ignore storage formats, there are many ways to describe color. Here are some examples:

• Red, green and blue components (RGB)
• Cyan, magenta, yellow and black components (CMYK)
• Hue, saturation and lightness/value (HSL/HSV)
• The amount of light that hit a group of sensors in a camera
• The amount of light and its direction when it hit sensors (in a light-field camera)

The first two are the most commonly used for displaying on monitors and for printing, respectively.

Additionally, an image is not just pixels, but also metadata. It could be things such as the width in number of pixels, the physical width if you were to print it, a thumbnail image, or even the geographical location of the camera when the image was taken.

Answer 7

Your premise is not wrong: any image can be represented using an N-dimensional array of finite values. Personally, I generalize that using discrete geometry instead of a matrix, but the essence is the same. But that's the content, not the file.

However, the file formats are different. Basically, there are several different ways to represent that same image, like people mentioned: bmp, png, jpg, etc. Of course, once you decode them, two lossless encoded version of the same image will lead to the same matrices.
Think of it as a .txt file that you compressed with zip. With the added weirdness that a non-lossless encoding would return a text that is not the same as the original, but really close, almost like a dumbed down version of the text.

Staying with the text analogy, let's say you have the same text, saved as .txt, .docx, .pdf, etc. Why aren't all the files exactly the same, if the content is the same? (Ok, txt doesn't have formatting, but the others do).

By the way, check out how the Netpbm encoding is really different from JPEG.

Answer 8

Bitmaps

A bitmap (BMP) is essentially what you describe, an array of numbers that represent pixel colors. E.g. something like

1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1

Lossless compression

Now, let's define a compression scheme. In our compression scheme, we will have an array of pairs of numbers. E.g.

3, 1, 1, 0, 7, 1

Now, the first thing I want to point out is that this compression scheme represents the same pixels as the first array. The first array has three 1s followed by a single 0 and then seven 1s. And that's what we're representing here. This format is shorter, as it represents multiple pixels with two numbers. The bitmap format has to use one number for each pixel.

Obviously this is a somewhat simplified view of an image (e.g. it's just one row) and a compression scheme. But hopefully this allows you to see how a compression scheme changes the format of an image. This is how a GIF relates to a BMP. GIF uses a compression scheme called Lempel-Ziv-Welch instead of this simplistic one.

What we've described here is a lossless compression scheme. A problem with lossless compression schemes is that for some inputs, the encoded form may be longer than the original. E.g. for

1, 0, 1, 0, 1

The encoding is

1, 1, 1, 0, 1, 1, 1, 0, 1, 1

Well, that was useless. We made the input twice as long.

Another lossless compression

Now, let's consider a different compression scheme. In this one, we will represent the image as overlaid circles. For each circle, we will define a center, a radius, and a color.

Our first bitmap would become

5, 5, 1, 3, 0, 0

This is the same length as our first compression method.

And our second could be either

2, 2, 1, 2, 1, 0, 2, 0, 1

This is three circles centered at the middle element (which in computer counting is number 2, as computers start counting at 0). One circle has radius 2 and color 1. Then we add a circle of color 0 and radius 1. Finally, we have a circle of color 1 and radius 0. In steps, this would be

1, 1, 1, 1, 1
1, 0, 0, 0, 1
1, 0, 1, 0, 1

Or

2, 2, 1, 1, 0, 0, 3, 0, 0

This is the same initial circle but covered by two point circles. In steps, it would be

1, 1, 1, 1, 1
1, 0, 1, 1, 1
1, 0, 1, 0, 1

These are both one shorter than the first encoded version but still longer than the original.

You may wonder why I'm talking about circles and not ranges. The main reason is that circles are closer to what real two dimensional images use.

Lossy compression

We also have the concept of lossy compression schemes. These lossless compression schemes can be turned back into the original bitmap array. Lossy compression schemes may not be reversible.

Let's consider a lossy version of our circles method. In this, we will use a simple rule. We won't store any circles with a radius less than 1. So in our last two encodings, we would instead have

2, 2, 1, 2, 1, 0

and

2, 2, 1

which converted to pixels again are

1, 0, 0, 0, 1

and

1, 1, 1, 1, 1

The first version is only one element longer than the original. The second version is shorter. Both are valid, so the algorithm is free to develop both and pick the shorter one.

We describe images with more restrictive rules as being of lower quality.

This representation of images as overlaid collections of circular shapes is similar to how the Joint Photographic Experts Group or JPEG format works. Its shapes are ellipses rather than circles, but the idea is similar. Rather than our simplistic method, it uses the discrete cosine transform to encode images.

Unlike GIF, JPEG is actually a different way of representing the image. GIF is still pixels. They are just stored in a different way. JPEG is shapes. To view a JPEG, we then convert the shapes into pixels because that's how screens work. In theory, we could develop a screen that did not work this way. Instead of pixels, it could produce shapes so as to better match the JPEG format. Of course, that screen wouldn't be able to show bitmaps. To display a BMP or GIF, we'd have to convert to JPEG.

If you convert a standard GIF, say 300x300 pixels, convert it into a JPEG, and crank the quality way down, the base shapes that it uses should be visible. Many JPEGs avoid these artifacts by starting with a much higher resolution image.

JPEGs scale well because they are shapes rather than pixels. So if you start with an 8000x8000 image, convert it to JPEG, and display it as a 300x300 image, much of the detail that was lost would have been lost anyway. If you converted the 8000x8000 bitmap to a 300x300 bitmap first and then to JPEG, the results will often be of lower quality.

MPEG

We've been talking about still images. The Moving Picture Experts Group or MPEG format uses the same kind of compression as JPEG, but it also does something else. While a simple way of doing video is to send a sequence of still images, MPEG actually sends a frame, followed by some number of frames listing changes, and finishing with an end frame. Because most frames are similar to the previous frame, the list of changes is often smaller than a second image would be.

The sequence normally isn't that long, say five frames. But it helps make the stream smaller than it otherwise would be.

Simplifications

I've ignored a lot. My images only have two colors (1-bit), not the 256 of an 8-bit image and certainly not the 4,294,967,296 of a 32-bit image. Even with 8-bit images, note that you can often choose different palettes for the image. So two 8-bit bitmaps with the same sequences may represent images that look different (same shape but different colors).

My images are single rows, not two dimensional. Most images will have a specific row size stored, making the arrays two-dimensional.

I haven't tried to represent the actual encodings at all. They are much more complex than the simple ones that I used. I did this because I wanted to be able to describe the encodings in this post. I'm not convinced that I could explain Lempel-Ziv much less the more complex Lempel-Ziv-Welch refinement in a single answer. And I don't understand Fourier transforms well enough to explain them at any length.

This is very much a simplified version of actual image handling. However, I feel that for didactic purposes, it is easier to understand than the more complex reality while still hitting the essential points.

Answer 9

Let's say it was true, that every pixel was just three numbers (red, green and blue) each in the range 0-255. Other answerers have started by (correctly) challenging that assumption, but for simplicity let's just say it's true.

I remember (but sadly cannot find online) a cartoon from a linguistics textbook: two ancient Egyptian stone carvers are sitting exhausted at the bottom of a massive wall on which they have carved a very large number of marching figures. One is saying to the other: "Surely there's got to be an easier way to write, 'The Pharaoh had 100,000 soldiers?'". Keep that idea in mind.

Now, suppose the first row of your image contains 1800 black pixels. How would that be represented?

0 0 0    0 0 0     0 0 0   ....

So how much storage space would that require? Each value is a byte. Three bytes per pixel, 1800 pixels in the row, so already 5400 bytes per row. So an image with dimensions 1800 x 1200 must take up 1200 times that much, which is over 6 megabytes. So now let's go and do a Google image search and download a couple of 1800x1200 images—let's say, one .png image and one .jpg image. Look at the file size: is it 6 MB? No way, it's usually much smaller than that. And that's a desirable thing, of course, all that space saved, and shorter download time....

So what's going on? The key is that, even if you have that many numbers to store, there are different ways to represent those numbers in the file. There's an example of a more efficient representation right here in my answer, two paragraphs ago. I wrote the words "1800 black pixels". That's 17 characters, and so doesn't need to take up any more than 17 bytes, yet it perfectly describes the exact same information for which we thought we needed 5400 bytes. And you could certainly do better than 17 bytes (and also save a lot of effort in the encoding/decoding implementation) if you didn't use the English language to encode this information, but rather a more special-purpose language. So now, already, we've posited more than one image compression format: one that uses English words, and one that's more efficient than that. See where this is going?

OK, you say, that works if a whole bunch of adjacent pixels happen to have the same colour. But what if they don't? Well, sure, it's dependent on the content of the particular image: the more redundancy there is, the easier it is to compress the information. Redundancy means that parts of the image can be predicted pretty well if you already know other parts. Compression means only writing down the bare minimum necessary to reconstruct the information. Not every possible image has redundancy, but any real image that has meaning to the human eye and brain, despite being more complex than my pure-black example, will still tend to have quite a lot of redundancy. And there are many different ways of compressing. Some compression methods are lossless, meaning that the information can be reconstructed to be mathematically identical to the original, as in my black-row-of-pixels example. Most .png files use a lossless compression method. Some methods are lossy: the reconstruction is not perfect, but the errors are hidden in ways such that the human eye and brain hardly notice them. Most .jpg files are lossy.

The details of how you recognize complicated patterns of redundancy, and how you write efficient compressed descriptions of them, are highly mathematical—and non-trivial, which is why there is room for so many different formats out there, corresponding to different compression strategies. But hopefully you get the principle.

A couple of commenters above have made reasonable guesses as to where your misconception might have arisen. In your question, you seem to think that compression just changes the pixel values a little (and sure, lossy compression methods do that in places, but only as an unwanted side-effect) without changing the information layout. When you open the file and look at the image content (for example, as an array of numbers in Matlab or as an image on screen in Photoshop) you are not looking at the compressed file content, but rather at the reconstruction, which has the same layout as the original (it wouldn't be much of a reconstruction if it didn't recreate the layout correctly). The file-opening procedure has de-compressed the information from the file into a full uncompressed representation in memory. If you compare two uncompressed reconstructions, then indeed there is nothing to distinguish between the two different image formats they came from (except for the small reconstruction errors, if any, that you might get if the compression was lossy).

Answer 10

For RAW and TIFF formats, as far as I can tell, the answer (as others have said) is that they do not actually always use the same colour spaces (e.g. RAW files might use more bits per pixel so can store finer colour information).

But to get to the crux of your question - sometimes there are images which are stored in different formats, but each ultimately represents exactly the same array of numbers.

A good example of a reason for this is the differences in compression between a PNG file and a TIFF file.

PNG files use one particular compression algorithm. That means an image won't just be stored as a big list of numbers for each pixel. Simplified example: it might store something that says "in this 10x10 block of pixels, all pixels are colour XYZ". Then instead of storing that information 100 times over, it stores it once, plus a bit of information about the region that information applies to.

The issue is then to get the original array of numbers (representing colours) back, so you can show it or edit it or whatever, you need software that knows how to interpret that compressed info.

PNG files always use the same compression algorithm, so it is easy for software to support all valid PNG files. On the other hand, some images have a structure that doesn't lend itself to the compression algorithm of PNG, so some of your PNG files might end up being quite large.

TIFF files, on the other hand, support many different compression algorithms. In fact, it can even store different parts of the image compressed differently. AND it supports 'extensions', so you can compress images using proprietary ways. So maybe the top half of your image will be compressed using a similar method to PNG, but this won't compress the bottom half very well, so the bottom half is compressed using a different method.

So TIFF files are more flexible - you might be able to store the exact same array of numbers using less bytes. But the software needed to decode the image will be more complicated, and might not work consistently with every TIFF file you throw at it, e.g. you might save a TIFF file in one software and be unable to open it using a different software, although it still works in the original.

So you ask

But I'm not asking about anything other than a basic 3-channel RBC image. All I know is that if someone hands me one of these, I now have an array of numbers. I have no reason to know why one array of numbers could possibly be any different than some other array of numbers from 0 to 255.

In order to hand it to you, someone had to know how the image was stored and how to translate that into an array of numbers. (Or possibly some software is doing that translation for you unbeknownst to you).

You could try saving an image as PNG and again as TIFF or GIF and look at it in a hexadecimal viewer to see how they each represent the same array of numbers differently. Or read up on the details of how PNG files and TIFF files are internally represented to give you an idea of what needs to be built into software to read identical arrays of numbers differently.

Answer 11

Yes, but how you get to those 1s and 0s is very different.

I will lay out an example, but it's fake and is suppose to illustrate more than be accurate. Keep in mind that all digital images are represented in binary at some level.

To complicate matters, there are different channels. CMYK, RGB, B&W, just to name a few. We're not going to be going into that. There are also different stages, like capture, storage, and display. We will be going into that, though again the example is supposed to demonstrate not be accurate. If you want accurate examples you will need to look up a ton of technical documents.

So in our sample, we are going to be looking at a black and white image.

00067000
00067000
00567800
04056090
40056009

The numbers represent how strong the "Black" is. This is how the camera captured the image. It's a decent camera so it's also how it stores the image.

Now it's store the image on a computer, but takes up a lot of space so we're going to compress it. In addition to mashing it up, we also know that most people can't detect a difference of 1 black level so we're going to smooth it out some.

302730
302730
204820
*04056090
1420262019

Now that's how we store the image on disk. It takes up less space and lets us produce much of the original image.

Now let's say we want to print it on a printer. The printer only prints one level of black, so a computer translates the stored, compressed image into printer speak.

00011000
00011000
00111100
01011010
10011001

This prints out a reasonable looking image, but you can see, even in the example an extream lack of quality. But hey it's the printer's fault.

Finally, you go to print the image on a good printer with 10 levels of black. Same as your camera. So you use the stored and compressed image.

00077000
00077000
00888800
04056090
40066009

As you can see the image is "better" but has been altered a bit from the original.

At any given time your correct that it's all just strength of a channel. And other then the compressed image, that has to be decompressed anyway, it stays pretty true to that.

However, the compressed format loses a lot of "information". Is that information important? Well, that's up to the artist, and audience. There are several trade-offs between saving space, processing time, quality of the final/stored image, and need. I scan most of my documents in one color black because that's all I need. However, my wedding photos are in the HUGE RAW format because I never know when I am going to want a great reprint of those. That said, when I transfer them (photos) to a digital picture frame I convert them to JPEG to save space. DIfferent channels, different filters, and different compression methods are all a series of trade-offs. It's like a digital version of the printers triangle.

Answer 12

I'll chime in with a bit of supplementary info as I've worked with image sensing and encoding/compression, albeit mostly moving images.

In its basic form, an image (ANY image) displayed on a particular screen IS indeed just an identical array of numbers. Those numbers may all be 0-255 or 0-65535 or 0-whatever-32-bits-is-I-forgot-go-google-it.

BUT there are so very many ways to STORE and TRANSPORT that information, a lot of them are simply products of technologies lost to the mists of time.

Also, one detail that I haven't seen any of the other pedants here mention is that truly RAW image sensor data from a digital camera may well be RGrGbB in a bayer pattern or somesuch which needs to be processed at least a little bit to make any sense to the Mk.1 human eyeball. Chances are you never get that even in a RAW format saved by your DSLR because it's useless until you convert it to a nice grid of RGB or YUV pixels, be they 8, 16, 32 or eleventy-squillion bits deep.

The stuff I've worked on uses YUV internally for whatever reason, I assume it's more easily processed by the codecs as humans perceive brightness with a lot more sensitivity than colour.

For some light bedtime reading, see the "frame image format" section: http://focus.ti.com/lit/ug/sprufg8b/sprufg8b.pdf

Anyway... back to your original question about the difference between uncompressed image files such as TIFF / RAW / IFF / PNG.

Generally the reason these exist is that, many moons ago, each computer / OS / printer manufacturer came up with their own slightly different set of requirements for some way of storing/sending images.

So, RAW as discussed by others in this thread is a generic term for several different things saved by different digital cameras, using whatever load of data that camera's manufacturer thought was important, based on the features their camera has or might have in future. So, although the main picture data bit might be very similar, the "packaging" around it that describes the image and all the camera settings etc. so one file would not be understood by a different manufacturer.

Traditionally this is so they can make you (or, more likely, professional photographers) use their proprietary (and sometimes expensive) software to process these higher quality images, otherwise you might start using other people's expensive software. Also, maybe Adobe Photoshop want to support their format, so maybe they can charge Adobe $$$ for that information so that more professional photographers will buy PS and maybe buy that make of camera because PS supports it now. Cosy!

RAW also stores information about how to turn that particular bundle of data back into a human-viewable picture, put simply all the tweaks you need to make to the data get the image to look "right".

TIFF was an early image format that was, among other things, used to send graphical data to printers (when graphics-capable printers started to get affordable). It was fairly basic so easy to process on the small cheap microprocessor inside the printer.

IFF (yeah, that's a thing) was a similar format used on Amiga computers, I believe invented by them or one of the popular paint packages. But, I'm using it here as an example because although it stores bit-map picture data like the others, it supported uncompressed or RLE data, variable bit-depths from 1-bit mono to 8-bit 256-colour (but with a 3x8-bit RGB palette to choose from for each of the colours) as well as special modes called Halftone and Hold-And-Modify allowing for many more colours than other machines of the era could manage. Oh, and it supported animation as well (like GIF) so an IFF file could store any number of frames, with variable delays between frames, and each frame could have its own palette. So, IFF would include extra data to handle all this compared to, say, a TIFF file.

PNG is another lossless image format, again storing bitmap data, but supporting some funky features such as an 8-bit alpha channel for variable transparency across an image (useful on web pages), so again the picture data "payload" might look very similar but the wrapper around it is different, and the payload might contain RGBA rather than just RGB data per-pixel.

So, that's 4 different image file formats described - you could store a sample full-colour HD picture of a cat in any of the 4 and it would LOOK identical, every pixel on your screen would have the EXACT SAME value and there would be NO difference in quality between the 4... but the 4 files would likely be different in size, layout, and be easier or harder for software to load & process.

Hope that helps!

Answer 13

Just thought I'd chime in here with the information that should have been in the very first answer to this question.

Pixels in an image are not stored in a byte - unless the image is monochrome, i.e. black and white only.

If you have a truecolor image, then each pixel is represented by 16 bits, or 2 bytes - as one value. If you have a 32bit image, then each pixel requires 32 bits or 4 bytes, again as a single value.

interestingly enough, image and sound files and every other data type in a computer boils down to bits of 1s and 0's. It is only by interpreting them in the correct sized chunks that meaning is extracted from them.

For example, an image and a word document and an mp3 file all have the same basic data content (a bunch of bytes), and any of them could be interpreted as one of the other types - you could interpret a word doc as a sound file and you would hear something, but it wouldn't be music. You could definitely interpret a sound file as an image, and it would display something, but it wouldn't be a cohesive image.

So, to summarize, a computer only knows about bits - a bit is either 1 or 0. All images, sounds, documents, movies, videos, recordings, games, phone calls, text messages and anything else labeled as digital have the same exact content - a bunch of 1's and 0's. The 1's and 0's become images, sounds and documents and everything else because the code reading them knows to read those bits in groups and process them accordingly.

That's why we have things like 16 bit and 32 bit images, and 16 bit and 24 bit audio files. The more bits you use for a pixel or a sound sample, the more expressive you can be - 16 bits can only define 64k unique colors, but 32 bits can define over 4 million unique colors. A monochrome image uses 1 bit per pixel - it's either on or off.

With audio files, the more bits you use per sample, the more detailed and nuanced the recording can be.

Answer 14

I haven't read the whole thread but it seems to me many people are forgetting about vectorized image formats. Those aren't arrays of pixels, because the concept of a pixel doesn't even exist in such a format. It's up to the renderer to figure out how to produce the image on a screen or any other medium.

Even without mentioning color domains, compression, bit sizes and channel format, there is a set of file formats that are totally unlike pixel maps. And yet vector formats are also much "better" at representing certain types of images, typically produced by a computer and not a camera.

Answer 15

This question was answered quite detailed before. However despite there is a lot of theory presented into the answers, i feel there are some basic subjects, typically related to computer programming that require more clarification. I must state i'm a software engineer. After i read the question i realized there is a completely misunderstanding of the basic programming data types that generated this question.

The first question here is:

Further, from a numeric standpoint, what makes something like a 16-bit images different than 32-bit images? Again, an image is just an array with integer values between 0 -255.

As presented before: No it's not. An image is not just an array of integer values between 0-255. Actually it may be an single or multidimensional array of 0 to 65535 values, an array of 0 to 4294967295 or even an array of bits (a bit can hold 0 or 1 values, that's all) that is converted by the software that is able to read the image files into integers numbers according to various encoding rules.

To understand this further, as stated before, I think a discussion on basic programming data types is necessary. I will try to explain them as simple as possible so anybody understand the problems involved with storing integer values in computers files.

In computer programming we use some basic primitive data types to write values into files, read them from files into computer memory, manipulate those values using various specific programming languages data types and eventually save them back to files. Integers in computer programming are not just integer. There are all kind of integers, depends on the programming language we are using and how much memory we need for each one. Typically, in most programming languages we have the following data types (and ways to manipulate them):

• BIT - holding 0 or 1
• UINT8 - 8bit unsigned integer - they can hold values between [0 to 255] interval.
• INT8 - 8bit signed integer - they can hold values between [-126 to 127] interval.
• UINT16 - 16bit unsigned integer - they can hold values between [0 to 65535] interval.
• INT16 - 16bit unsigned integer - they can hold values between [−32768 to 32767] interval.
• UINT32 - 32bit unsigned integer - they can hold values between [0 to 4294967295] interval.
• INT32 - 32bit unsigned integer - they can hold values between [−2147483648 to 2147483647] interval.
• OR a combination of all those data type in a more complex format. For example a UINT16 (16 BIT) holding 3 different values, first 4 BIT holding values between 0 to 127, next BIT holding 0 or 1 and so on.

Further MORE there is something programmers have to deal when reading or writing integer data type from files. The endianess. Endianness refers to the sequential order in which bytes (UINT8 from our table) are arranged into larger numerical values when stored in memory or files. Endianness is of interest in computer science because two conflicting and incompatible formats are in common use: values may be represented in big-endian or little-endian format, depending on whether bits or bytes or other components are ordered from the big end (most significant bit) or the little end (least significant bit). Simple put you can store a value like this 0000000011011111 or ... like this 1101111100000000 depending or the endian order you chose. And you are free to chose any order that fit your purpose. There are no rules others that the ones you make when you design an image file format.

Please notice in computer programming integers are using more or less space, depends on the value. Like you need more paper to write 255255255 you need more BITs to write a bigger value. Then later when you want to read the value you must know exactly the rules you created when you wrote it. Otherwise it's impossible for you to figure our how to read just just an array with integer values between 0 -255 because you simply don't know where those numbers are stored and how those numbers are stored given so many choices you have (BIT, UINT8, UINT16, UINT32 or a combination of all those computer data types). And don't forget, Endianness. If you don't know the data was written using either big-endian or little-endian order you are unable to read the proper value.

Because of this images are NEVER just a just an array with integer values between 0 - 255. Some of them are arrays of UINT16 (16bit images) others are arrays of UINT32 (32bit images) or others are arrays of UINT8 (8bit images). Some very creative computer programmer can even use signed types that live you with arrays of INT8, that means array of values between -126 and 127.

Actually when you read an image file, one of the first data you encounter is usually some BITs representing the image width and height. And those are not just some 0-255 values. Those are also some data types chosed by the programmer. Some programmers will think 16 BITs are enogh for storing a maximum image width of 65535 pixels, because they are designing an image format used in a game to keep some little buttons images. Some other programmer may use a 32bit value here allowing you to store images up to a width & height of 4294967295. Some crazy NASA programmer may even use 64bit for storing a huge photo of the galaxy up to 18446744073709551615 pixels. If you don't know the rules, you can not read those "values" as you call them. Because you don't know where they start in the image file and where they end. So you end up with a bunch of BITs you don't understand nothing about.

That's why the universe is full with so many different images formats. Because there is no standard solution to write some integer values into a file. It's the programmer choice entirely based on many factors like the Endianess of the machine you are working on, the programming language you are using to design the original file format implementation and many other things like the purpose of the image format (as clearly stated before by other answers).

A practical simple file format of an black & white image that holds only one single value 166 to represent a 4x2 pixels image:

The image (1 - black pixel, 0 - white pixel):

1010 
0110

This file format use 1 BIT per PIXEL stored as a SINGLE 8bit integer value 166 (10100110). That's all. No array of 0-255 values are used but 8 different 0 or 1 values stored as value 166.

If you used an array of 0-255 values for each pixels * 3 times for RGB you will end up with an image 24 time bigger. This file format just saved 24 times the disk space you need to save an image like this or 24 times less the computer memory needed to read and keep this image into computer RAM when you use this image for example in your high performance 3D game engine to draw something on the screen with it (texturing thousands of dust particles flying around could be a good candidate :)).