I read that lens makers used to use radioactive glass to increase the refractive index property of their lenses.
How radioactive are they?
Here are some example dosages; where would looking through the viewfinder for an hour fit?
The article in rfusca's answer includes some references: The Aero-Ektars, by NASA scientist Michael Briggs; Radioactive Materials in Camera Lenses, from the Health Physics Society (an organization focused on radiation safety); and Thoriated Camera Lens (ca. 1970s), from Oak Ridge Associated Universities's professional training on radiation safety.
From the ORAU PTP article:
Measurements have indicated that the exposure rate at a depth of 10 cm in the body of an individual carrying a camera containing 0.36 uCi of thorium would be approximately 0.01 mrem/hr. Based on this value, NUREG-1717 calculated that a serious photographer might receive an annual exposure of 2 mrem. This assumed that the photographer carried the camera 30 days per year and for 6 hours per day. They also estimated an exposure of 0.7 mrem per year for an average photographer. If the camera lens contained the maximum permitted concentration of thorium (30%), NUREG-1717 estimated that the aforementioned annual doses could triple.
This puts the "6hrs/day for a month" usage at about the same as getting a chest X-ray — or, one little green square on the xkcd chart. Or to put it another way, using the lens six hours a day for a year would be the same as taking three round-trip flights from one US coast to the other in that year. Not completely trivial, but not something people normally stress about. And that'd be really heavy usage.
The articles indicate that exposure to the eye might be a greater concern than overall dosage, particularly if you happen to have thorium in an eyepiece (unlikely for general photo equipment). So you might decide to spend a little less time holding the camera right to your eye than you might otherwise.
Assuming (based on the reading) that looking through the viewfinder is very roughly an order of magnitude greater exposure than the general usage, looking through the viewfinder for an hour is about 1µSv — equivalent to getting an arm x-ray.
Typical lens radiation was apparently approaching 1 mR/hr at the surface of the lens and tapers off rapidly with distance. I'm not sure exactly where it lands in your chart, but the same source states that a chest xray is about 10 mR.
It should also be noted that the conversion rate between Rems and Seiverts is 1 mR = 10 µSv. So if a chest x-ray is 10mR (according to the article @rfusca linked), thats about 100µSv. According to the chart, that is equivalent to the approximate total dose received at Fukishima Town Hall over a full two weeks, and just shy of half what those two Fukishima workers received that will likely make them a little sick for a while, but is easily treatable.
If you examine the full XKCD radiation chart, an x-ray dose is about 1000 times more than the amount of radiation you get off of a radioactive lens at its surface. The lowest dose of possibly cancer-causing ionizing radiation is 10,000 times more, and the lowest possibly deadly dose of ionizing radiation is about 2 million times more, than the amount of radiation you get off of a radioactive lens at its surface. You would need to encounter 4 million - 8 million times more radiation than a radioactive lens releases, in a single dose or continual ongoing short-term exposure, to encounter truly life-ending radiation.
You probably get more radiation from the sun in a day than you get from a radioactive lens. ;)
One issue here is that some lenses are more radioactive than others. Without a geiger counter, it's hard to know what you're up against. Next, photographers are creatures of habit. Some wear their lenses next to their body, always in the same place. My lenses for instance are on a shelf next to a couch where I read for hours and sometimes fall asleep. People make light of this exposure – and for some lenses for good reason – but there is a significant element of risk. With lots of these lenses around, your body will definitely be absorbing more radiation every day.
Another substantial risk is dropping one of them and ending up with particles in the air from the lens and breathing some of them in.
Probably better to pass on them. Particularly if you have children around the house (could break the lens, more sensitive to radiation).