Plagues and other great diseases are important parts of history and some settings. They defy man and may even defy God in their swift and terrible punishment upon the body of society.
My issue with modifying/removing dwarves and elves is that the PC’s may want to play them.
One of the most powerful tools found in the World Building 102b article is the Worksheet. Here is a place to put the settings you might make with them.
The nations of the Kolm surpasses all other barbarians in their wilderness of life. Thoug they do just bear the likeness of men, of a very ugly pattern, they are so little advanced in civilization that they make no use of fire, nor any kind of relish, in the preparation of their food, but feed upon the roots which they find in the fields, and the half-raw flesh of any sort of animal. I say half-raw, because they give it a kind of cooking by placing it between their own thighs and the back of their horses. They fight in no regular order of battle but by being extremely swift and sudden in their movements, they disperse, and hen rapidly come toghether in loose array. They spread havoc over the vast plains and flying over the ramparts, they pillage the camp of their enemy almost before he has become aware of their approach. They are the most terrible warriors for when in close combat with swords and flails they fight without regard to their own safety, and while their enemy is intent upon parrying the thrust of the swords, they will entangle him with their chains so that he loses all power of walking or riding.
Excerpt from "The peoples of the world" By Taklamarian court-scholar Guliman Amon.