Think of it not from a technophobe's perspective, but from an accessibility perspective.
An optical viewfinder, reflex viewfinder or EVF will be far preferable to a bare LCD on the camera back. Not only is it easier to hold the camera steadily when it's in tight (and probably getting some extra support from the cheek), the image presented in an up-to-the-eye viewfinder will be at a virtual distance that means you're not going to need reading glasses (or bi/trifocal head gymnastics) to look at the finder.
Make sure the camera is big enough to be easy to handle -- even if the camera only has one button, that one button is useless if the user can't easily hold the camera and press the button. (I don't know about your grandfather, but many men of that generation actually worked for a living, and have hands roughly the size of Rhode Island with, um... let's say limited dexterity, shall we?)
A whole lot of buttons and dials and doodads really isn't a problem if you can say, "just put it on 'P' and fire away."
I like the EyeFi idea, but most in-the-box software will pop up an import utility the moment the camera is plugged into the computer (Canon's utilities for its P&S cameras are pretty much idiot-proof, and I assume most included software these days is similar).
And yes, batteries can be an issue. I have no idea who thought it would be a good idea to force rechargeables (we always had that option with AAs), particularly non-replaceable lithiums, but they're the next best thing to useless for somebody who hauls the camera out every few months when it's time to take a picture since the batteries will be flat until after the moment has passed.