The sorts of rolls I mean by basic rolls are not skill rolls, but the common sorts of rolls that people make for things like remembering a person's name. There's no real skill for that: it's usually just a function of natural ability. But if you say, 'Ok, it's a medium maneuver to remember that guy's name' (and note that players do tend to default to a medium maneuver), and 'You can add your Memory bonus to the roll', the average person is going to fail that most of the time.
An average person rolling an average roll only gets 40% success (roll of 50). Furthermore, a person with an amazing memory of 100 (+25) still only gets a 50% chance of success (50+25 = 75 on medium maneuver). This is the sort of thing I'm talking about. The success rate is simply too low, and even exceptionally high bonuses don't seem to help sufficiently. This contributes to the frustration and sense of the system as too difficult, especially at lower levels.
If, however, you said, 'add your memory stat', the average person would succeed 50% of the time (which I think should be the baseline) and the person with the 100 Memory is going to succeed 95% of the time.