Irrational fear of contact with mercury

The fear of even small amounts mercury is confusing to me. I can
remember playing with the mercury that was in a broken thermometer
several times when I was a child. And I'm not dead, or sick because of
it. Neither is my mother, who remembers playing with a good size ball
of the material back when she was in high school.

Now, the media and the EPA are terrifying people about a "trace" of mercury in
the new compact fluorescent lights that we're supposed to be using
instead of the incandescent bulbs that we've used for as long as most
people can remember - because the new CFLs are more energy efficient
(not to mention a lot more expensive).

And the EPA's 'disposal instructions' in the event of a CFL breaking are
just silly, imho. They say that when one breaks, you should clear the
area of all people, then turn off the air circulation systems. Then,
instead of using a broom or a vacuum, you're supposed to carefully,
while wearing heavy gloves, use two peices of cardboard to scoop the
broken material up and put it into a metal container to be taken to a
recycling center. Trouble is, those centers aren't everywhere - and
they won't always take it when you bring it in. So what then?

All of this for an admitted trace amount of mercury. That's insane.

I have to wonder if my mom and I are the only ones who remember playing
with mercury and don't have any ill effects from having done so. Anyone
else out there remember playing with it as a child? And if so, do you
have any health problems that you can trace directly back to mercury contact?