There are a number of good brands that have gloves and hats. Gloves are essential, and contrary to the way many weekend runners dress, if you find it necessary to wear pants or tights, you should certainly be wearing gloves. When you're running, your hands will get cold long before your legs do.
If you deal with bitter cold, mittens provide more protection than gloves (reduced surface area means less heat escapes)
Good gloves/mittens are often expensive ($20-$30US), and cheapo gloves sold on the street are often of poor quality. A good middle-road is to try to find a generic brand made of Polypro or Polartec 200/300 at an online outdoor goods store (for example, campmor).
Hats are optional, some may find them irritating. For people who don't like hats, ear warmers are also a good way to protect the ears. Unlike a har, an ear warmer can easily be wrapped around your wrist later on if your head gets itchy due to heat buildup. However, I find the Sporthill Infuzion hat surprisingly unintrusive. It is lighter than most ``store'' hats. Other polyester hats from ``running clothes brands'' (such as Hind) are similar.
The hat is made of lightweight material, but good enough to keep the head warm on pretty cold days, and people who usually find hats irritating will be able to stand it.