/u/MrApophenia on /r/DnD (on Reddit) posted "Goblin Variant" in mid-November, 2014. The basic idea is developing Goblins, and the goblinoid family, as a better enemy. I took this idea further for my own purposes, making them into a very different kind of "better" enemy.
The basis is in cannibalism. When a goblin dies, other goblins will eat it. This places the dead goblin's memory and mind into the living goblin's mind. With a little nest of minds inside each goblin, they are sort of "immortal", and this is why they would act psychotically in battle, since they don't fear death.
Hobgoblins, when they die, have a similar form of immortality. They keep goblins around, because if a hobgoblin dies, they can make the goblin eat it, and they will turn into a hobgoblin. They will turn into that very SAME hobgoblin, in fact.
However, hobgoblins will not eat each other, because when they do, they turn into the monstrous bugbear, which is a solitary, violent creature.
That all roughly describes what /u/MrApophenia set out in his original post. My changes came over the course of these past few months.
First, the reason goblins eat each other is to preserve their culture. When Raksanti, goddess-queen of Goblinkind, created the goblin race, she created them to oppose her ancient enemy, Voorm-Sotha, an entity of the underworld. It should be noted that Raksanti is a goddess who refuses, and has always refused, to contact any other entity, living or dead, if it is not a goblinoid. She has been this way forever. It is assumed by the "civilised" races that she does this because she, like her creations, is "evil". Because of this, she's never been able to set the record straight about their nature, and the genocide continues.
When a goblin dies, it is usually of old age. Any and all of its children then partake equally of its corpse, and gain their parent's memories and its mind. A goblin who dies of old age keeps a peaceful mind. It exists in the living goblins' minds as a friendly reminder and guide, full of advice, to make sure their daily rituals are done properly.
These rituals, over thousands of years since Raksanti first set them into motion, have not changed. And it is these little daily rituals (which can be anything you want) which keep Voorm-Sotha at bay in the underworld.
Hobgoblins are the physical protectors of goblinkind. In their very regimented society, cannibalism is considered incredibly taboo, since, as in the original example, it will transform them into a monstrous bugbear, a mindless killing machine whose original intended purpose has been lost.
However, hobgoblins have a sort of "martial law" over goblins, allowing them to command them innately. And when a hobgoblin falls, a goblin will be commanded to eat them, which transforms them into an exact copy of that hobgoblin, down to the memories and personality. The goblin they once were is thrown into the backseat and often forgotten entirely.
A hobgoblin who was previously a goblin will be referred to as a "goblinish hobgoblin".
One of the traits which is pulled over from their goblinish lives is their physical compulsion to consume their dead brethren. They do, in fact, recognise that they are now hobgoblins, and have an internal battle upon sighting a fallen hobgoblin, since they know it is wrong, but will consume them anyway.
This transforms them into a "goblinish bugbear". A true "bugbear" is only created by two bugbears of any type breeding.
DM's PLOTLINE OVERVIEW
When implementing this storyline, first inform players that goblins are well-known to be psychotic killing machines. Inform them that they are physically compelled to consume their dead, which is known to pass on the dead goblin's mind. This is the reason they are to be slaughtered.
Let them know that hobgoblins seem to hate this behaviour, but will protect a goblin, even to their own death.
Bugbears are rare. Your players do not need to know of them. They also do not need to know that a goblin can become a hobgoblin.
Begin with a few fights with goblins. Let your characters practice their murderhoboing. Knowing players, they'll be happy to go out of their way to murder the faces of goblins without much provocation. However, the goblins are psychotic, and will chase the characters down if they do try to run.
The only thing they would rather do is consume their dead, and they'll only stop chasing if the dead in their immediate vicinity outnumber them.
Throughout the course of months worth of sessions, introduce your characters to the concept that goblins are held in the lowest possible regard by the "civilised" races. Even those who hate violence will wish them dead.
This is because of an old story from around a thousand years ago.
Goblinkind lives far to the east somewhere. They arrived around a thousand years ago to the eastern edge of civilisation. Many were slaughtered on sight, and the leftovers were captured while they attempted to consume their dead. Scholars tortured them for weeks in an attempt to learn their purpose, for they were detected as evilly aligned creatures.
What they said was translated as "we are the shadows of beneath and we'll destroy your world above".
For a thousand years, goblins have grown only more psychotic.
For a fairly long while, your aim should be to make your players uncomfortable with the way you're treating these goblins (NPC torture, wholesale slaughter, kicking/burning goblin babies, etc), while putting them in positions where they must also take part in the slaughter.
If they encounter a bugbear, it should be a violent, but poignantly sad creature. Due to its history, it will be detected as evil.
Because they believe they are doing it for the right reason, it will not change their alignment to evil.
Eventually, you should set about the motions of a war effort to journey into goblin lands and put an end to their blight.
Armies should march into goblin camps, finding them to be living peacefully, while submitting themselves to the deaths NPCs and PCs present to them. Hobgoblins should fight back in force.
This is the time to reveal that goblins are always detected as goodly aligned. The reason for their usual "evil" alignment is that the memory of brutal slaughter will radiate pure evil. Killing a goblin is considered a sin against the world above, and, again, the only thing allowing the players to remain "good" or "lawful" if that is what they are, is that they believed they were doing right.
Finally, a great goblin stronghold should be encountered.
Two things should happen. First, the hobgoblins will be witnessed biting into each other, while they live, each of them transforming into a horde of bugbears which will tear through the armies. It should be stressed how taboo this is in hobgoblin society, especially since there will now essentially be no hobgoblins left in the peaceful goblin lands.
Second, a goblin leader should come out, translating him- or herself to Common for the PCs to speak with. It should be revealed that the translation was mistranslated. What they were trying to say is that "we protect the world above from the shadows of beneath and they'll destroy us all". Or some such.
Essentially, this last genocide of all goblinkind will result in Voorm-Sotha's forces breaking free of the underworld, plunging the surface world into darkness.
Consider letting your players choose new characters to play against this darkness if they want. Maybe their new characters won't be genocidal sinners.