Is there already implementation of the algorithm the removes reflections on windows from photos like described here: http://petapixel.com/2015/05/11/new-algorithm-can-automatically-remove-window-reflections-from-photos/?

  • 1
    It's based on machine learning, so it isn't a simple algorithm with a simple formal description. The problem is to get it pretty much exactly right, otherwise you'll get artifacts after almost removing the reflections. An image with reflections may still look better than an image where you've removed almost all of it at the price of some ugly artifacts here and there. Oct 13 '15 at 20:00
  • 1
    Have you tried using a polarizing filter on your camera to avoid reflections in the first place?
    – Ben
    Oct 14 '15 at 8:50
  • Note the comment in the article: "The algorithm doesn’t work on single reflection photos". So it works on some but not necessarily on one you pick.
    – TFuto
    Nov 2 '15 at 17:40

This does not require machine learning. It requires however a 2D deconvolution. An image with double reflection P is nothing else but reflected single image R convolved with displacement matrix M superimposed with original intended image O. So: P = R*M + O, where the * is the 2D convolution symbol.

Only P is known. However, you can select manually a segment on P when you see no reflection, in that area, P-Q==O. Knowing this, R*M=P-Q == 0 in those areas.

The displacement matrix is unknown. In its ideal state it is a completely black image with only two white pixel at a distance that creates the separation between the double reflection. The task ahead is to extract that matrix from the current image. It is a simple Least Error fitting with some brute force effort on sharpness (acuteness). The result is a matrix with two white spots.

From this, and R*M== 0 in certain areas, calculate R, the reflected image.

From this, calculate O = P-R*M, the image without reflection.

The drawback is that this entire operation can add considerable noise and artifacts.

  • Least Error fitting has some residual error (will create edgy artifacts after deconvolution),
  • The displacement matrix will not be ideal (introduces blur and noise),
  • The deconvolution is not a trivial operation by itself, prone to numerical noise and division by almost-zero problems.
  • Area masking in 2D operations will obviously show up in the 2D FFT and when deconvoluting, this will create larger spots (changes in intensity and color).
  • Not to mention that if the image is not in super high resolution, quality will degrade quickly.
  • And so on...

I suggest using a filter ;-). Or using reflection as an artistic effect. Or avoiding reflection.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.