I was thinking of using public domain images as backgrounds to my photos. Do I still technically "own" the resulting work?
Your best bet is to consult a legal expert with experience in Copyright law for your country as the laws vary from place to place.
In the US, you are free to use images in the public domain without permission from the original copyright owner (as the owner is the public). As such, you are free to construct derivative works.
You of course will need to ensure that the images you do use, are indeed part of the public domain.
If you build on an image that someone else has generously placed in the public domain (assuming that it is really public domain), why would you want to own it? Surely you should equal the original act of generosity and make your derivative work public domain as well.
It is this spirit of generosity and sharing that created the Internet which is based on open source software. As an example, the intent of the GPL software licence is to perpetuate the original licensing conditions to maintain the chain of sharing and prevent freeloading by derivative authors.
I am aware that one cannot really equate software with photos and naturally I respect the right of original authors to choose the licensing conditions that suit their particular circumstances.
Nevertheless it is worth remembering that the spirit of sharing, exemplified by the open source software movement has had an extraordinarily transformative effect.
In the USA:
As others have explained, you can do this legally. You'll create something called a "derived work". You own the copyright to the derived work, but not the public domain portion.
For example, this is why you can't legally copy CDs of classical music performances. While the music itself is in the public domain, the particular performance is copyrighted by the performer.