I used a 18mm-50mm lens at f/3.5, with a card with a cut-out heart shape of about 4mm.
I got the shaped bokeh effect, but the outer side of image is dark due to the card.
How do I avoid this and achieve a full image?
From my brief experience testing the 'Bokeh Master's Kit' I learnt these things:-
a) As you probably know a wider camera aperture allows the shaped background blur to be more pronounced. On the opposite side, a smaller aperture will not only reduce the bokeh shaping but increases the vignetting *darkness at the edges)
b) A longer focal length on the lens with the wider aperture helps increase the shaped bokeh effect. As well as this, having a longer focal length (50mm in your case) will cause less vignetting as the lens is looking at less of the cut-out and more of the view through it. So using it at the 18mm end of the lends would not only decrease any shaped bokeh effect, but also cause more dark edges
c maybe) Longer exposures may cause more vignetting to be visable as the lighter areas are exposed more and the darker ones less so causing a greater difference between. This may have more of an effect on film that digital sensors but that's purely my speculation, and any difference I saw when testing was more likely because of the small apertures I used to get that longer exposure time.
So in summary, widest apertures, longer focal lengths and maybe less exposure time should reduce any vignetting you get on photographs as well as make the shaped bokeh more pronounced. I know images without vignetting can be achieved as I took test videos too and there was none on there (taken on a manual 50mm f1.8).
It's amazing how small the cut-out hole can be in front of a lens without it blocking out any image
It's a lot harder to do after the fact, but the key is to take another picture, of a plain grey or white card, using the same vignetting matte. It would help if it were at the same focal length, focal distance and aperture, and if the matte is in the same location in front of the lens. Don't worry about getting the card in focus; you only want a plain center and the right shape and gradient of vignette.
You can then use that second image as an overlay layer in your favorite image-editing program. You may have to resize it a bit and move it or rotate it to make the vignettes match. Then make the layer a negative. You may want to create a layer below your plain vignette layer, filled with a neutral grey, then merge the negative vignette down. With the combined layer set to overlay, it should brighten up the vignetted area. Keep in mind, though, that any detail that was lost to darkness in the original image is gone forever—you may find that you have to clone in the missing bits as well as you can, then settle for cropping out the irretrievable parts. A more-appealing vignette shape (like an oval) for the final image might just be the best you can do.