13

I don't understand how frequencies are defined in images/photographs. As far as I understand it now, high frequencies are like sharp things in images, like edges or so, and low frequencies are kind of the opposite?

I also would like to understand the outcome of Discrete Fourier Transformations, like how to read them properly.

It would be cool if somebody could explain to me the following:

  1. What are frequencies in pictures and how are they defined?

  2. How do you read the outcome of a Discrete Fourier Transformation?

  • Related: photo.stackexchange.com/questions/21762/… – coneslayer Jun 25 '13 at 16:25
  • thanks, I already read this, it helped me, but im still a little clueless. – Jakob Abfalter Jun 25 '13 at 16:29
  • The second part of your question is covered by the link @coneslayer provides. It could probably use better answers, but it's still the same question. :) – mattdm Jun 25 '13 at 16:30
26

I will only answer the first question: What are frequencies in images?

Fourier Transform is a mathematical technique where the same image information is represented not for each pixel separately but rather for each frequency. Think about it this way. The sea has waves some of which are very slow moving (like tides), others are medium in size and still some others are tiny like the ripples formed from a gust. You can think of them as three separate waves but at each point on the surface of the sea and a moment in time, you get just one height of water.

The same applies to images. You can think of the image being made up of various waves or frequencies. To create your image, start with the average colour (actually thinking of gray scale images is easier). Then add waves of different wave lengths and strength to slowly build up details in the picture.

Source Image:

Source Image

First Frequency (Average):

Average

The second frequency along the vertical dimension is a wave starting at zero at the bottom of the image, rising, becoming zero again along the centred horizon and falling below zero to finally become zero at the top of the image. (I described a Fourier Series without phase shift, but the analogy still holds.)

Here you can see the second frequency along the horizontal and vertical. Notice that you can make out where the mountain will be (dark) and where the sky and lake will be (lighter).

Second Frequency:

First Component

Each additional wave or frequency brings along more ripples and as such, more detail. To get different images, the wave height/amplitude can be changed as well as the starting point of the wave, also called the Phase.

Third Frequency:

Third

Interestingly, the information amount is the same in this representation and one can go back and forth between normal images (spatial domain) and Fourier Transformed images (frequency domain). In the frequency domain we need to keep information of all frequencies along with the amplitude and the phase information.

Here it is using 50% of the frequencies:

50%

There are variants of all this, with distinctions to be made among Fourier Series, Fourier Transform and Discrete Fourier Transform and Discrete Cosine Transform (DCT).

One interesting application is in the use of compression algorithms like JPEG. Here the DCT is used to save more of the important parts of the image (the low frequencies) and less of the high frequencies.

I wrote this in the hope that novice readers can get a basic understanding of the idea of Fourier Transforms. For that I made some simplifications that I hope the more advanced readers will forgive me.

Animated

Video generated by Thomas Devoogdt can be viewed at Vimeo.

animated


Frequencies in Post-Processing

There are numerous methods that rely on frequencies for post processing, mostly because we never look at single pixels individually. Many algorithms work on frequency because it is more natural to think about them this way. But also because the Fourier Transform contains the same information we can express any mathematical operation (or post processing step) in the frequency and the spatial domains! Sometimes the pixel-wise description is better but often the frequency description is better. (Better primarily means faster in this context.)

One technique I would like to point for no particular reason except that it is artists working directly with frequencies and that is *frequency separation *. I am not going to describe it but you can see how it works on YouTube for both Photoshop and GIMP.

You create two layers one with the low frequencies and one with the high frequencies. For portraits you can do skin smoothing on the high frequencies without affecting the skin tones in the low frequencies.

Code

This is some code to generate the above examples. It can be run as a simple Python program.

from PIL import Image
from numpy.fft import rfft2, irfft2
import numpy as np

def save_dims(ft, low, high, name):
    ft2 = np.zeros_like(ft)
    # copy the frequencies from low to high but all others stay zero.
    ft2[low:high, low:high] = ft[low:high, low:high]
    save(ft2, name)

def save(ft, name):
    rft = irfft2(ft)
    img = Image.fromarray(rft)
    img = img.convert('L')
    img.save(name)

def main():
    # Convert input into grayscale and save.
    img = Image.open("input.jpg")
    img = img.convert('L')
    img.save('input_gray.png')
    # Do Fourier Transform on image.
    ft = rfft2(img)
    # Take only zeroth frequency and do Inverse FT and save.
    save_dims(ft, 0, 1, 'output_0.png')
    # Take first two frequencies in both directions.
    save_dims(ft, 0, 2, 'output_1.png')
    save_dims(ft, 0, 3, 'output_2.png')
    # Take first 50% of frequencies.
    x = min(ft.shape)
    save_dims(ft, 0, x/2, 'output_50p.png')

def generateGif():
    ''' Generates images to be later converted to a gif.
    This requires ImageMagick:
    convert -delay 100 -loop 0 output_*.png animation.gif
    '''
    # Requires images2gif from code.google.com/p/visvis/source/browse/vvmovie/images2gif.py 
    # from images2gif import writeGif

    img = Image.open('input.jpg')
    img = img.convert('L')
    # Resize image before any calculation.
    size = (640,480)
    img.thumbnail(size, Image.ANTIALIAS)
    ft = rfft2(img)

    images = []
    for x in range(0, max(ft.shape)):
        ft2 = np.zeros_like(ft)
        ft2[0:x, 0:x] = ft[0:x,0:x]
        rft = irfft2(ft2)
        img_out = Image.fromarray(rft).convert('L')
        fname = 'animation/output_%05d.jpg' %(x, )
        img_out.save(fname, quality=60, optimize=True)

    #writeGif('animation.gif', images, duration=0.2)


if __name__=='__main__':
    main()
    #generateGif()
  • It should be clarified that, while theoretically we could, assuming we had infinite knowledge of the image at hand, decompose it to component frequencies and recompose it with no loss...in the real world we can't. Convolution of a real world image, which occurs at each and every "interface" along the optical pipeline, is effectively an irreversible process. We can't ever know all convolution factors, and therefor reconstruction of an FFT back into an image is difficult, and extreme modifications usually result in artifacts and data loss. – jrista Jun 26 '13 at 18:21
  • @jrista I think the point Unapiedra was making about reversability was that once you're working with a digital image (an array of pixels on a computer), you can go to frequency space and back, and get the same image you started with. You're looking at a bigger picture of the physical imaging system (lenses and such), where real-world limitations intrude. – coneslayer Jun 26 '13 at 19:25
  • 3
    jrista's comment is misleading in that FT is blamed for information loss. Of course, photography is a lossy process and so is post-processing. If I convert a discrete image to Fourier Space, do some lossy processing there, and then convert back, of course I loose information. But it happens in the processing step and not in the conversion step. True, because of machine precision every mathematical operation looses information but if we are talking about 8 bit per channel images, we won't notice machine precision errors. – Unapiedra Jun 27 '13 at 18:05
  • 1
    @Turkeyphant , I don’t remember why I mention diagonal in that context. You can see that the principal direction of the second frequency seems to be that particular diagonal. Maybe that’s why. To answer your question, you only ever need two axes to represent a 2D image. It’s important that the two axes are orthogonal. Horizontal and vertical axis fulfill that criterium. (Also they are practical.) With discrete images (i.e. composed of pixels), aliasing will make all other angles worse. – Unapiedra Aug 5 at 7:15
  • 1
    @Turkeyphant correction, Unapiedra described the second frequency, not the first, as 0, -1, 0, +1, 0. The paragraphs describing the 2nd frequency are immediately after the 1st frequency image (the uniform gray image), and I can see how it might be tempting to read that paragraph as a description of the preceding image (articles often show an image, then describe it in text following the image), but not in this case. =) – scottbb Aug 7 at 12:31
2

Briefly, frequency refers to the rate of change. More precisely, the frequency is the inverse of the period of the change—that is, the amount of time it takes to cycle from one brightness (or whatever) to a different brightness and back again. The faster then change (e.g. from light to dark), the higher the visual "frequency" required to represent that part of the image.

In other words, you can think of frequency in an image as the rate of change. Parts of the image that change rapidly from one color to another (e.g. sharp edges) contain high frequencies, and parts that change gradually (e.g. large surfaces with solid colors) contain only low frequencies.

When we talk about DCT and FFT and other similar transforms, we're usually doing them on a portion of an image (e.g. for JPEG compression, edge detection, and so on). It makes the most sense to talk about the transforms, then, in the context of a transform block of a given size.

Imagine, if you will, a 32 pixel x 32 pixel block of image data. (This number is arbitrary.) Suppose that the image is a simple gradient that is white on the left side, black in the middle, and white on right side. We would say that this signal has a period that is roughly one wavelength per 32 pixels of width, because it goes through a complete cycle from white to black to white again every 32 pixels.

We might arbitrarily call this frequency "1" — 1 cycle per 32 pixels, that is. I vaguely recall that this is commonly called θ in transform textbooks, or maybe θ/2, but I could be remembering wrong. Either way, we'll call it 1 for now, because this truly is arbitrary in an absolute sense; what matters is the relationship between frequencies in a relative sense. :-)

Suppose you have a second image that is white at one edge, then faded twice as quickly so that it went from white to black, to white, to black, and to white again at the other edge. We would then call that frequency "2" because it changes twice as often over the width of that 32 pixel block.

If we wanted to reproduce those simple images, we could literally say that every row consists of a signal with a frequency of 1 or 2, and you would know what the images look like. If the images went from black to 50% grey, you could do the same thing, but you'd have to say that they had a frequency of 1 or 2 at an intensity of 50%.

Real-world images, of course, aren't just a simple gradient. The image changes frequently and not periodically as you scan from left to right. However, within a small enough block (e.g. 8 pixels, 16 pixels) you can approximate that row of pixels as the sum of a series of signals, starting with the average of the pixel values in the row, followed by the amount of the "frequency 0.5" signal (black on one side, fading to white) to blend in (or with a negative amount, the amount of that signal to subtract), followed by the amount of frequency 1, frequency 2, frequency 4, and so on.

Now an image is unique in that it has frequency in both directions; it can get lighter and darker when moving both horizontally and vertically. For this reason, we use 2D DCT or FFT transforms instead of 1D. But the principle is still basically the same. You can precisely represent an 8x8 image with an 8x8 grid of similarly sized buckets.

Images are also more complex because of colors, but we'll ignore that for now, and assume that we're looking only at a single greyscale image as you might get by looking at the red channel of a photograph in isolation.

As for how to read the results of a transform, that depends on whether you're looking at a 1D transform or a 2D transform. For a 1D transform, you have a series of bins. The first is the average of all the input values. The second is the amount of the frequency 1 signal to add, the third is the amount of the frequency 2 signal to add, etc.

For a 2D transform, you have an n x n grid of values. The upper left is typically that average, and as you go in the horizontal direction, each bucket contains the amount of signal to mix in with a horizontal frequency of 1, 2, 4, etc. and as you go in the vertical direction, it is the amount of signal to mix in with a vertical frequency of 1, 2, 4, etc.

That is, of course, the complete story if you're talking about a DCT; by contrast, each bin for an FFT contains real and imaginary parts. The FFT is still based on the same basic idea (sort of), except that the way the frequencies are mapped onto bins is different and the math is hairier. :-)

Of course, the most common reason to generate these sorts of transforms is to then go one step further and throw some of the data away. For example, the DCT is used in JPEG compression. By reading the values in a zig-zag pattern starting with the upper left (the average) and moving towards the lower right, the most important data (the average and low-frequency information) gets recorded first, followed by progressively higher frequency data. At some point, you basically say "this is good enough" and throw away the highest-frequency data. This essentially smooths the image by throwing away its fine detail, but still gives you approximately the correct image.

And IIRC, FFTs are also sometimes used for edge detection, where you throw away all but the high frequency components as a means of detecting the areas of high contrast at sharp edges.

National Instruments has a nice article that explains this with pictures. :-)

0

Reading the outcome of a bidimensional FFT is... Well, tricky. Basically because the output is in the complex numbers domain.

I cannot give you a very detailed description because I have forgotten most of it myself, but basically you have a decomposition expressed as the sum of a series of exponentials of imaginary numbers.

There are frequency transforms that are more useful for image processing than the DFT, like the Cosine Transform (used in JPEG encoding for example, which has only real coefficients) and the different kinds of Wavelets.

0

Imagine scanning the image line by line with a photocell, and feeding the results to a plotter (these flat machines that make black waves on paper), oscilloscope (these boxes that make flickery green waves on a screen) or spectrum analyzer(bigger boxes that make green or multi colored picket fences). Or a loudspeaker even. The finer the structures in an image, the higher the frequencies (the pitch in the loudspeaker) of the signal shown/heard will be. The more contrast there is in the fine structures, the higher the amplitude of the high-frequency parts of the signal will be.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.