It's all down to personal judgement here, but it's a complex issue, for the most part down to the player making the attack.
However, it's always been a personal bugbear of mine. Players, specifically in 5th D&D (but equally relevant here in RM, which hopefully players are a little more "real-life friendly"), always seem to think that a target can be hit or not, and that misses simply miss. It's never as simple as that, right?
1. Use your common sense as GM. If a character, with a bow (or whatever missile weapon you have).. and they are firing at a TARGET who is already actively in melee with the characters allies, then automatically give that target a minor bonus to it's defense for each ally it's in melee with.. regardless of whether the allies in melee are actually interposing themselves between the target or not. This reflects not only the characters reduced difficulty of hitting the target but also the character's intention of NOT hitting an ally! I'd suggest a +10 DB if equal sized, or +5 DB if the target is larger...PER ALLY.
2. A target that is already actively in combat with Allies will already blocking, dodging or avoiding the blows of your allies. However, if it's also aware of the shooter, i.e. Is roughly facing the shooter and knows their location, may also additionally be able to deliberately use your ally as an unwitting shield (remember it's actually more difficult, as an ally, to know where NOT to be, when an ally fires from behind at the same target. It gets an additional bonus as 1. (but only one)
3. Treat as a "Miss" might also be simply a "Missed Opportunity to safely fire, rather than a chance to hit an ally".
Basically, if the "Miss" would have hit without the inclusion of any of the additional defense bonus (given in 1. and 2. ) then the player has simply missed. However, if the player would have hit if the DB bonuses not being applied, then they may be given to option not to shoot at all (simply wasting an action) or run the risk of hitting an ally but potentially still hitting the Target... the chance being the defense bonus given in 1 & 2. If the character hits regardless of the bonuses applied then they have still hit the target as normal.
Example. Three characters are fighting an Orc. Two characters are in Melee with the Orc (one in front one behind) the third an archer tries to take a shot.
Give the Orc, already has a DB of 20, an additional +30 to it's DB (only +10 for each ally in melee with it, but an additional +10, because the orc is capable of deliberately using the character in front of it as cover.)
If the Archer rolls so badly they would miss regardless of the additional DB it got for being in melee, it simply misses.
If he "misses" because of the additional +30 DB applied, then he has the option to "Not Shoot" or "Shoot and still hope to hit"... in this case the chance of hitting an ally is 01-30%, but he still hits the Orc on 31-00% doing the relevant damage for that attack roll.
If he does hit an ally, then re-roll the attack on an ally (randomly determined), but with the DB applied to the original shot as a bonus to the OB.