How unified are the Quaidu? Would it make sense to have Quaidu deserters? They might be more interested in disappearing from the rest of the Quaidu forces than in conquering (though people may not trust that). A small, inconsequential town suits that purpose. Not sure if it makes sense for them, just tossing it out there.
OK, not letting real life be injected here, but my experience in the Middle East would show that bands of armed fighters are a real issue. In Bosnia, Afghanistan and Iraq I saw small groups of armed thugs holding villages and "consuming" natural resources. The fact that they had weapons and some training made them more than enough to terrorize the locals.
Those villages that successfully resisted had weapons (usually taken from previous bands) and the training of resiting in the past. They also knew the area better than the thugs. As mention previously I see them as a small band, maybe as smalls 4-5 Quaidu. They are just fleeing the fighting in the South and trying to find a place to hide from Ulor. Only a few would be hardened and what we see as evil. The later would not be satisfied with being paid off.
As for racial make-up I can only go with the Xa-ar sourcebook. On page 23 TKA mentions the two places what we are using as the linchpins of the area, Tepentor to the west and Rapata to the East. In Tepentor the Lotana are not even mentioned. While in the larger town, Ratapa, the Lotana are mentioned. I am assuming American product listing methods here so I might be wrong, but he mentions Jameri as the majority (barely), followed bu Haid, and then the Lotana.
I am not against having the Lotana in the area, I am just not understanding the need to have them as a majority in the town. If it is to have a priest of the Blue Dragon that can be handled by having several of the farmhold on the eastern side of the towns influence being Lotana in make-up. They have moved out of Rapata in the last few generations. The Priest has come her as a basis of gathering supplies, i.e. Herbs from the Innkeepers wife. Since we have determined the area of influence of the town to be about 20 miles, this could be explained.
Is the Cult of the Blue Dragon necessary for an adventure hook you have in mind?
-BP