So I've been on this problem for about a year now and haven't been able to solve it. I gave up on it a while back, but now I'm researched up and ready to get an answer.
I'm trying to fake a long exposure by taking multiple RAW photos and doing something with the Bayer filter values in order to get the effect of having left the shutter open for that long.
The idea I have in my mind is that, intuitively, long exposure works by leaving the shutter open for longer, and so the light sensor values (that RAW pictures store) are simply an aggregation of multiple shorter photos' values.
For example, if I have three 1/3 second-long photos taken without moving the camera, they should store the same information as a 1 second-long photo.
The problem I'm getting is with how I can quantify that aggregation. I found that many different methods fail, and I'm not sure why. If I simply average out the multiple photos, I get a heck of a lot of noise reduction. So that part is nice to have. I can later use Camera Raw to brighten up the photo and I'll get a close approximation to what I would have gotten with a 1 sec long exposure. The problem is, I want to do that on-device. I'm currently using an iPhone to test, but I don't think that should matter.
Current Setup & Formula
I'm using three 1/3 sec exposures to try and fake a 1 sec photo. My current formula is simple:
out_picture[x][y] = picture1[x][y] + picture2[x][y] + picture3[x][y]
The problem is that that gives me really pink pictures like so:
I tried multiple different variations of the above formula. One that I thought was promising was the derivative of the sigmoid function. The reason why I used the derivative of the sigmoid is because I did a case-study where I studied the effect of final RGB outputs as a function of the "EV" slider value on Photoshop Camera Raw, and found a sigmoid-like pattern. Here is what that looks like when I plot it for some pixels:
So I thought maybe the derivative of the sigmoid (high changes for middle values, low changes for low and high filter values) would fix things. I normalized the filter values to a 0-1 range by using the camera's black level and max bit value. I made some modifications to the derivative so that it would fit within the range of values 0-1, this is the function I ended up with. I then multiplied by number of pictures taken, to emulate the
EV slider in Camera Raw.
Here is the formula for that (in python):
def exposureAdjust(x): black_level = 528 ## iPhone RAW Photos have a black_level of 528 max_pixel_value = 2**14 - 1 ## 14-bit depth normalized = (x - black_level)/max_pixel_value ## now x is between [0,1] EV = num_pics ## faking EV value adjustment = EV * (np.exp(-10 *(x - 0.25)))/(np.exp(-10*(x - 0.25)) + 1)^2 return min(x * (1+adjustment), max_pixel_value) ## Cap at 2**14 num_pics = 3 out_picture[x][y] = picture1[x][y] + picture2[x][y] + picture3[x][y] out_picture[x][y] = exposureAdjust(out_picture[x][y]/3)
Running this formula, I get a similar result:
Conclusion & Observations
As you can see, it's still pinkish. The pics above are JPEG's yes, but I'm actually working with the RAW's. I just had to upload them as JPEG's so they can be attached inline.
Pink happens when there is a lack of green. The camera has a bayer filter layout of RGGB, but I really don't see how I could be getting this effect. Anyone have any idea what is to blame here and how I can adjust it programmatically. I have the entire pipeline set up. I only need the formula, but somehow it's not working. I've coded this up in Python and Swift, and the results are the same. There's definitely an issue with the formula using the RAW bayer filter values. I think this is a cool challenge, but I've reached the end of my wits on this.