No building is allowed to be within a weapon’s throw of the city wall, that is the law. When the city was properly rebuilt, after being sacked, there was a paved area 12 Imperial strides (1 meter each) between the wall and the buildings. In fact, most buildings along that line have their “back” to the wall. The Good People were once again safe.
A century or so has passed since then. The undesirable peoples (be they merely poor or of the wrong species/ ethnic group) have found their way into the wall line. They have found that the “good people” turn a blind eye to them being there. They make the Guard run them out of their neighborhoods. The “sad people” have made the Wall Line their home.
All along the edge of the city (except for the area around the city gate and the docks) there is a pleathora of temporary places. They are tents mostly, but some others are crates and merely bits of cloth stretched to protect from rain. The sad people have their own community there; resting between their menial or scavanging jobs, cooking the captured ratlins, larger grubs, weed greens, and scavanged left overs from the good folks, and looking for a way away from the wall.
As long as all the buildings are temporary, and there is an easy to navagate passage along the wall (the wall road is a corridor between the tents), the guards leave them alone.