I have used my compost straight to start my seeds for the last 10 years without any problems. I have not experienced "damping off" in my hothouse (which can be a common occurrence). This is probably due to not having to water as often because the castings are hydroscopic; they hold moisture incredibly well.
Worm compost will not burn, therefore it is the perfect growing medium. However, if you need to stretch your worm compost, adding 20-40% to your potting mix will provide plenty of disease inhibitors and growth enhancers.
Additionally, adding a handful of worm compost while transplanting will aid in water retention and disease suppression for your plants.