Well actually I just read sephi's ("master of mana" /successor of Fall from heaven Mod) comment when he actually played Master of Magic.
In Master of Magic a lot of Barbarian Units running around, even much stronger.
In FE we've the situation to make this barbarians completely dump and only acting in Wilderness or let them attack cities which leads to the problem that often too much defense is needed and means either too much resources go to defense or a unlucky person/AI just get overrunned.
The solution is to let barbarians do some damage to the city but not destroying it.
It can be something like destroyed building or stolen resources or even some funny stuff like a giant spider uses town houses as new home and add "negative building" to the city - which may be handled in another way somehow.
Bigger Barbarian Forces like fire demons coming from quests/dungeons may still destroy/conquer cities but the usual bear/spider/what-ever-beast should not but still need to influences cities!
And don't get me wrong. I know about the bug in 0.77 (forces tries to double kill pretenders and kill AI nations very frequently). Actually I WANT the wilderness creators acting in a way with cities near by just not as harsh as destroying the whole city.
In moment the lairs/walking monsters are there to get destroyed. they try to catch stupid heroes at best. Not that thrilling for me.