Generally for the lower levels* the pure-arms users come out a bit ahead, the others take more time to "mature" so to speak. So, if you are talking about pure ability, then the non-spell users would be the way to go.
But, as you said "if you had" then I would go with the character I "liked" - be he arms, semi, or pure. (Really, I would go with a mix as that covers more of bases of capability.)
*Yes, I consider levels 1-10 the "lower" levels.