The Fear spells specify that those who fail a RR flee for 1 min per ten points of failure.
Companion II section 5.1 about Handling Fear gives a set of results that can be used for different types of non-spell (demon, dragon, giant, undead, etc.) fear effects. Failure by 1-10 is -25 to all actions, 11-25 is -50 to all actions, 25-50 is break and run, 51-99 is paralysed, and 100+ is special (possibly a heart attack, etc.) There's also a Fear section of an expanded static actions table 11.2.
I don't know of any other references, although there might be guidelines for specific creatures and monsters.
