12

I am attempting to write a script that loops through each pixel in a photograph and applies a bokeh to the image as a whole.

I built a script based off this link, however that seems to be a hack.

I have three input images: a black and white depth map, a photograph, and a bokeh "brush" image (which is currently a hexagon). For every pixel of the photograph, I stamp the bokeh brush so it is centered on that pixel and in the color of that pixel.

It looks... okay on tiny bokeh brushes, but once I increase the bokeh brush size at all it ends up looking like gaussian blur. Here's a picture of times square blurred with my algorithm:

enter image description here

Nevermind the dark edges, I can fix that.

You can sorta tell it's different from gaussian, but it's still a long way away from what could respectably be called bokeh, with crisp edges:

enter image description here

I understand why my algorithm does what it does... how can I more accurately simulate a bokeh?

  • Perhaps this would help: What's the difference between Bokeh and Gaussian Blur? – Imre Jun 18 '15 at 6:36
  • @Imre I understand the differences, and that Bokeh is typically caused by a lens, and Gaussian by post-processing, but I want to simulate Bokeh. – Entity Jun 18 '15 at 6:45
  • @Entity: How do I look for sample images with an accurate depth map? Is yours accurate? I would like to try this myself (after finals). Maybe I'll come up with an answer in a couple of weeks then. If yours is public, can I have a link to it? – Martijn Courteaux Jun 18 '15 at 21:38
  • @MartijnCourteaux For my initial testing I'm just using a flat depth map so that everything is blurry. For simple geometry (e.g. a cup on a table) I think it would be fairly simple to make a good looking depth map. For more complex images, you would probably need a real depth map. This can be calculated from two images, or even from just your single image. – Entity Jun 22 '15 at 17:25
19

I think the main problem is one of dynamic range, your algorithm is probably right but you're working on the wrong type of data.

A point lightsource that would otherwise clip and go pure white gets spread over a larger area by a defocussed lens, so that it forms a disc that isn't as bright and therefore doesn't clip.

That's why you get those nice circles in your real bokeh image. If you clip the signal (making it less bright than it otherwise would be and then spread it out with your bokeh simulation you get a dim circle (or hexagon, or whatever) that doesn't stand out and thus doesn't look realistic.

What you have in a real image chain is:

bokeh (from the lens) -> digitisation (clipping) -> gamma correction & dynamic range compression

What you are doing is

sharp image -> digitisation (clipping) -> gamma correction & dynamic range compression -> bokeh simulation

You wont get the correct result because you're not working with linear data.

What you can do is attempt to linearise the data, replace any dynamic range that has been lost to clipping, perform your bokeh simulation, and then redo the nonlinear operations!

Here's an example. I've started with an HDR image that has been tonemapped, giving a highly nonlinear result. This is the worst type of image to attempt bokeh simulation with!

http://mattgrum.com/photo_se/hdr-original.jpg

Doing a standard convolution operation to simulate bokeh (using photoshop's lens blur tool) yields this result, which is very similar to what you are getting:

http://mattgrum.com/photo_se/non-linear-bokeh.jpg

To get a better result, I applied an extreme curve to try and get the image back to roughly what it would have been before tonemapping, where the highlights are much, much brighter than the rest of the image. I did this by with the levels tool, pushing the centre input a long way to the right, from 1.0 to about 0.2). I then applied the lens blur tool, just as before. Finally I applied an extreme curve in the opposite direction to the first curve. The result, whilst a long way from perfect, looks much more like real lens bokeh:

http://mattgrum.com/photo_se/linear-hdr-bokeh2.jpg

 

If you're doing this in code, then try cubing each value, then applying your bokeh simulation routine, then take the cube root of each value. You should see an improvement. It might take some tweaking.

tl;dr even if you have implemented a perfect mathematical model of bokeh, it must be applied on unclipped linear data. If you apply the same calculations to heavily modified data (even a standard in camera JPEG is heavily modified from a mathematical point of view) you will get a very different result.

12

First of all, in optics, only light adds up and darkness does not. Make sure that your algorithm does not bleed dark pixels outwards their original location. Resulting pixels should rather resemble maximum of nearby source pixels than average. Or, to be even more exact, you'd be summing up logarithms of affecting source pixels.

Another possible cause why your edges might not be sharp would be if the edges of your mask are not sharp. The animation in page you've provided as reference could be misunderstood so that in a mask, original pixel is bright and others are gradually darker. This would translate into muddy edges in calculated bokeh, too. In photography, apertures have definite edges, not gradual ones. So actually most of the pixels in a mask should be of equal brightness and only edges (where less than a pixel should be colored for a smooth line) can be some shade of gray.

You also mention having a depth map, but no word on using it. Your bokeh mask size should be correlate with pixel depth and focal plane depth difference - the further a pixel is from focal plane (in either direction), the larger its mask should be. At focal plane, the mask size should be 1×1 pixel.

  • Provided the operations are all additive you wont get the problem of dark areas spreading out. It's true you need a sharp mask to get sharp circles, but the major problem is that his bokeh operation is being applied to non-linear data. – Matt Grum Jun 18 '15 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.