your second diagram strengthens my suspicion this is a memory management issue (of the operating system) ...
i would try to watch changes to size of memory types and disk-I/O (read/write) during the first 5-6 minutes.
loading of samples (into RAM) seems to collide with prefetch technology - data/code which is assumed to be used gets loaded into memory beforehand, after system realizes it doesn't need the data it gets moved to the pagefile (which is on disk) to provide more space for samples (which are marked as not to be unloaded into the pagefile, even if not currently needed).
my assumption: in the beginning there is more unused memory available and the read process is fast, after a certain point prefetch data gets unloaded, then loading of sample data decreases until a second border is reached and behind that constant memory re-organization happens.
it would be interesting to see a second diagram what happens on the system volume (where the pagefile resides) during the process and how amount and relation of different memory types change over the time-axis.
and remember: only a CRAY can run an endless loop in just three seconds.