Plagues and other great diseases are important parts of history and some settings. They defy man and may even defy God in their swift and terrible punishment upon the body of society.
My issue with modifying/removing dwarves and elves is that the PC’s may want to play them.
One of the most powerful tools found in the World Building 102b article is the Worksheet. Here is a place to put the settings you might make with them.
Diseased people, (leprosy or any other such fear inducing disease) when traveling, will often times wear a cloak that entirely covers their body and ring a bell as they travel, used as a warning for any others to stay away. Could be used for a disguise or safe passage. ... you meet strangely cloaked people, sometimes ringing a bell. "Please give us something to eat, mighty heroes!" This is time to show how good they are.