Stronghold has a third kind of unit. Most of them can't really be directed (like the military units), nor do they have a fixed task (like the walkers). Most of them are only there to enhance the flavour of your game. Some however, are pretty important.
He is, without a doubt, the most important of your wanderers. He's the only one that can be pointed to go and attack something, and the only that has hitpoints. He's normally seen wandering about your castle, visiting outlying buildings and watching over his people. If you manage to get him killed, you've lost the game.
When your Lord is married, this charming woman of impeccable lineage shows up wandering across the castle grounds. Once she's
moved in it seems your peasants work a lot harder, but that could just be my imagination.
Note: Getting married is an event that is hardcoded in a scenario, alternatively the scenario can start with the Lord already married.
Women will show up carrying babies once you have a hovel or two. They just wander
around and chase after the kids.
We have gathered a list of all the names showing up in the game, which can be found here
Once you have hovels up you will also see children, boys and girls, running around the castle and in the surrounding area. You won't believe where they will go...
The people in your castle are very civilised. You will only see one drunken sod per inn, no more, no less. The worst part about him, is that he always seems to be walking where you want to build a new structure. If you want to see quite how drunk he can get, have a look here
The trader and his cart will show you your market is active. He doesn't seem to be bothered about closed gates, nor do the enemy troops worry about him bringing you a fresh supply of weapons.