I can see how this might appear too random for some but the setting of a city lends itself to random encounters and in truth they are not totally random as they are often pre-generated before the session meaning the encounter table has been created with GM guidance based on plot and session objective. Im running a long running campaign based in an off the shelf setting complete with well established NPCs and about 10-12 detailed location maps which is pretty handy when the party revisit locations. New locations such as 3 new enemy strongholds have been added over time (50+ rpg sessions)
I think because its really a sandbox campaign (based in and around a single city) thats well laid out with players able to choose where they go and what order to complete adventures in the random elements help keep them guessing and keeps the setting developing. I also use plenty of intrigue, misinformation, spies, traitors and shifty protagonists like crime lords, master assassins leaders, a thieves guild, an evil cult and the odd scheming evil wizard so the overall feel might appear random but everything has its place and makes sense, its just very layered and frequently changing as various protagonists work towards their various goals. Each of the factions have allies and common goals but also their own agenda.
Randomness like this perhaps works best when you know your players goals, the party goals and have a linear plan mapped out already - which I do. NPCs encountered can also be known to the players from previous adventures or by reputation and usually will be linked to the larger story in some meaningful way. You can be sure your players will mess up your carefully laid plans often. Thinking on your feet and reacting to your players actions are just as important as having a plan for the session.
In the example above I knew the party were travelling to a set destination in the city, I knew they would be followed by spies and I knew their goal was to investigate a known enemy stronghold just outside the city.
The dice decided the order of the encounters along the way and the level of skill of the spies tailing them.
As GM your random dice rolling is always balanced by asking yourself "How would this NPC normally behave?" "What is their likely demeanour/attitude/purpose" "How friendly are they towards the party" etc
In summary, you still need a plan, an overall plot and consistent themes in place. This lets you hand the reigns for the direction of the story over to the players a little more and let them experience the world/setting . Your job as GM is to keep them engaged in the story and having fun. How you do this will vary from game to game. rpgs have 3 story tellers. GM - Players - The Dice. Find the best balance for your game and use what works for you. You know when you have got things right when your players are excited to come back for more adventures.