I'm thinking in terms of risk increasing throughout the day, as you use more and more power between rest periods. As to the consequences of failure, stipulate for the sake of argument that as risk rises, direness of consequences also rises. So the "light the candle" cantrip in the predawn when you wake up has a tiny chance of failure, and about the worst that can possibly happen is that it consumes the candle all at once and leaves you searching for another one in the dark... and that's a "catastrophic failure" of the casting. But after you've been chased by orcs through Moria for 2 straight days without any rest, by the time you meet the balrog you're so wiped out that just believing it's possible for you to do magic is hazardous. Not just because of the fatigue of the 2 days, but because of 2 spellcasting days, your fatigue and mental sloppiness and therefore your risk is well beyond that of the fighters who stayed up that same 2 days. Even lighting the candle might kill you. After all, enough energy to power a lighter flame passed through your brain at some point in the process, no?
Yes, I realize there are a lot of variables, and how much you're willing to bet will vary not only with the odds of winning but with the size of the pot. I'm just trying to get a feel for what gamers think of as "high risk" in spell failures/fumbles.