I recently thought of a time lapse technique and want to know if there is there a name for it or if it has been widely used before.
I want to take a time lapse of my garden at 1 frame per day for a long time, say one year (similar to this post) : Doing a time lapse of my backyard for 1 year
Obviously the large problem with this method is that there are many things that will cause flicker or jitter in the image like differing brightness levels from clouds, sunrise and sunset patterns, and wind blowing the trees.
My idea was to use a raspberry pi to take thousands of pictures per day, say one every few seconds, and delete all of them in any given day apart from one. The one which is kept is the one which is closest to the previous day's frame.
I wrote a simple script to select the best frame for the day by comparing pixel by pixel and selecting the frame with the minimum sum of squared differences.
Obviously the first frame needs to be chosen manually.
Initial results after a week look promising, it's been able to get frames for each day with minimal difference to each other.
I want to know if this is a novel technique, or if it's been investigated before and there are any extra hints or tips other people have discovered about this particular method.
Edit: This received a lot more attention than I expected so I'll make a few notes here about things frequently raised in the coments.
- Shadows: If there are strong shadows in the scene then this will mean that images need to be selected at almost the exact same time each day to avoid jerky shadows caused by the sun at different angles. I found that I can easily avoid this by picking the first frame when the scene isn't being lit by direct sunlight, such as at dawn or dusk or when there is light cloud cover. There are generally a few hours each day where there is adequate light for a good photo but no direct shadows, and the lighting is mid-level so it's fairly easy to find a frame with matching light levels the next day.
- Snowfall, or other objects that are not in the scene for very long. It's quite clear that only comparing against the previous day's frame in the cost function could cause some pretty big problems. So to counteract this it would definitely help to use more than one previous frame in the cost function. What's not clear is exactly how to do this, since I can think of a few different techniques, but I'll experiment with a few. These could include making the cost function the sum of the difference between the previous frame and also the first frame, or constructing an artificial frame for today's cost function using the modal, or average pixels from the last 5 day's frames (amount not certain).
- "A one-year timelapse of most-similar frames might look a bit dull and boring, since there'd be barely any "change" in the sequence". Of course, the algorithm will try to hide changes as far as possible, such as changes in brightness or changes in the position of the trees when blown by the wind, but there are things that will change in the sequence, that the algorithm can't hide, such as plants growing gradually over time, which is what I'm interested to see.
- "keep in mind that 365 pictures give a very short video at 30 frames per second" : That's very true, I'll probably go for something like 10fps. Also, there's no reason why the same algorithm can't be applied to any time span. For example: for every hour, keep the picture that's most similar to the picture kept on the hour before, for a 1 frame per hour video.