If a weapon were to be a holy symbol, I might consider that it would have to be VERY ornate, and look unique enough to be able to identify it AS a holy symbol. A sword would have a jewel-encrusted cross-piece with a colorfully painted scabbard.
I think it depends on how fleshed out the religion is, and in what way. I wouldn't lay down any hard and fast aesthetic rules on weapons as Holy Symbols.
My dwarf cleric, for instance, as a condition of being a cleric of his particular forge god, is required to take a specific training package every time I level him. This does give him some ranks on combat, but it also keeps his smithing skills maxed out, along with various other lore and other non-combat skills. I've also made him unable to claim treasure from adventuring, he's only allowed to keep money he makes from weaponsmithing, his particular chosen profession.
As for his holy symbol, his warhammer is it by virtue of the fact that he has cast the spell Holy Symbol on it. It is not particularly ornate, and if it were, that would get in the way. It is his hammer that he uses for combat and for smithing. It is utilitarian, not ornate, and aside from its uses as a tool and weapon, it represents the hard work and dedication that his god requires of his clerics. Having it be particularly decorative wouldn't fit in with the religion.