From that silent place fear flows in unseen waves, like white fog. The shadows are many, and the wind breathes cold through the broken battlements and casements. Through it's frowning walls and dark window openings there's a lantern of the spirit which none see by but those who bear it.
When was the last time a character caught a cold or flu? Real people get them every now and again even with solid heating and dry interiors. Your characters are out in the cold and wet, and even inside is not always dry and warm. Imagine having to stop your dungeon crawl, sitting in a tight defensive position, while you all are feverish.