Most towers and baileys are defensive, protecting defenders from attackers. This keep tower and bailey is used to protect everyone else from what is within.
Some time magic users are less magical than one expects.
The district gets its name by the good sized walls that are painted a bright blue that mark the magic district. That wall is watched by the Witchhunter guard.
Helioglyph (his magical name) is a Talismonger, a maker of charms, talismans, holy items, and so on. This is the place where those seeking fine craftsmanship and good symbolism come. If you are looking for enchanted items, go elsewhere.
Diseased people, (leprosy or any other such fear inducing disease) when traveling, will often times wear a cloak that entirely covers their body and ring a bell as they travel, used as a warning for any others to stay away. Could be used for a disguise or safe passage.