If one travels, you might notice it. Each one has a blue orb on their signage or somewhere in the shop. The merchants seem to specialize in foreign goods, but often buy local produce and products, to trade in other places. They seem totally unrelated to the others. It seems to be a secret organization, but nobody knows for sure.
Want to avoid fighter types focusing on swords? Make Longswords and Bastardswords the weapon of the Noble class, with a death penalty for other to carry. These weapons will be ornate and finally crafted. Adventurers will be stuck with cheaply made broad and shortswords, while professional fighters might use two handed swords.