I don't expect perfect performance; I run things at 512 with 2 buffer settings, keeping latencies kinda high. Theres a clear difference though between "normal" dropouts due to lots of activity and these random freezes, which seem to happen regardless of load, latency, activity, etc.
So bad RAM? That's always a tough one to diagnose - the last time this problem plagued me I ran several RAM tests but wasn't sure I was getting accurate results. Perhaps I'll just swap it all out and replace it with new memory, to at least ID the problem, before diving in and testing my existing RAM.