There are toxins in art materials and I strongly recommend all artists to research the materials they use and please take all precautions necessary while painting.
This issue was the last thing in my mind when I started painting. I've always been concerned about the oils, but I never thought much about charcoal and pastels. The dust from the pastels can be harmful. Please use a mask, wash your hands frequently and if possible wear gloves.
I have to admit at times I can get careless about handling my medium. When I'm so involved with my art... my priority changes and nothing else seems to matter.
Keep painting, be safe, but more importantly... enjoy your art :)