VI, Mac and memory question

[musos]

Hi there,

I just added 4GB of RAM to my G5 to get a total of 6GB
Previously, the system was struggling with "heavy" VI Logic songs.

Okay, the whole system is MUCH better. With the previous 2GB RAM, screen redraws became jerky and the finder was slow and unresponsive. Everything is vastly improved. [+o(]

Looking at Activity Monitor now (with a big song loaded), I see that VSL-Server is using 1.93 GB of Real Memory but also 2.3 GB of Virtual Memory. There is 712 MB Free and 4.78 GB Inactive

(Before the new RAM, VSL-server used only 117.5 MB of Real Memory plus the same 2.3 GB of VM, while Free memory was down to 22 MB and Inactive memory down to 614 MB)

How come VSL-Server still uses so much Virtual memory? (The total VM size is now 15.69 GB which is more than before.)

Maybe some tech-maestro here can explain this to me.....

Regards - Colin

[JWL]

Hey Colin--

Maybe I've missed a post elsewhere, but yours is the first I've seen that proclaimed improvements with RAM above 4 GB. I've been wondering what good it does to have more RAM than the DAW and VI's actually seem to use.

VM vs real RAM remains a question for me as well. Someone else posted a report (DG?) of seeing physical RAM dedicated to VI cap off at 1GB with everything else delegated to VM. I don't think the question was fully answered, but I'm certain I'll be upping my RAM by at least 2GB, and will probably go further to max it out unless there's a reason why I shouldn't.

I had 1.5 GB in my old G4 (max'd out) and it's almost funny to think that three times as much in my current G5 is hardly enough. But then again, I'm reminded of the Commodore 64 shipping with a whopping 64k and maxing out at 128k.

Okay, so that's like comparing apples to wax friut, but you get my point.

[cm]

if a statement is true for something, than for GB RAM and GHz CPU - much helps much.
in any case no application related to (sample-)streaming is using virtual memory (a swap file) - wouldn't make sense to outsource data from disk to disk, only physical memory is fast enough to fill the buffers.
displaying of the various memory types is a little bit *exotic* though ...
christian

[musos]

Thanks for the replies...but I'm still confused.
With 2GB RAM, VSL-server used 2.3 GB VM
With 6GB RAM and lots of free memory, it still uses 2.3 GB VM
???

Christian: why "exotic"? I often look at Activity Monitor (ships with Mac OSX) to see what processes are running. It shows everything I mentioned without me even asking! [[;)]]
If "no application related to (sample-)streaming is using virtual memory", then why does VSL-Server use so much VM?

So, my system feels much better, I don't feel like I'm running at the "limit" anymore - but I still want to understand this whole thing.

Anyone?

[JWL]

I've been researching this topic, because I'm curious about it, too. One of the confusing things is how RAM is used on dual processors. If you have 4GB installed, for example, that's only 2 GB per processor-- so when, how, and why VM kicks in must be related to this.

Here's and article from Ohio State University-- in plain-speak:

How is Virtual Memory Handled in Mac OS X?


Memory, or RAM, is handled differently in Mac OS X than it was in earlier versions of the Mac OS. In earlier versions of the Mac OS, each program had assigned to it an amount of RAM the program could use. Users could turn on Virtual Memory, which uses part of the system's hard drive as extra RAM, if the system needed it.
In contrast, Mac OS X uses a completely different memory management system. All programs can use an almost unlimited amount of memory, which is allocated to the application on an as-needed basis. Mac OS X will generously load as much of a program into RAM as they can, even parts that may not currently be in use. This may inflate the amount of actual RAM being used by the system. When RAM is needed, the system will swap or page out those pieces not needed or not currently in use. It is important to bear this in mind because a casual examination of memory usage with the top command via the Terminal application will reveal large amounts of RAM being used by applications. (The Terminal application allows users to access the UNIX operating system which is the foundation of Mac OS X.) When needed, the system will dynamically allocate additional virtual memory so there is no need for users try to tamper with how the system handles additional memory needs. However, there is no substitute for having additional physical RAM.

It seems to fit that the long load times for VI are due to the large amounts of data trying to be loaded into RAM until it hits a brick wall, at which point OSX determines what the pre-load "ceiling" will be and how much RAM "headroom" it will need. Crunching numbers for a huge orch. library then, understandably, takes a bit of time for the "page-out-swap" process between RAM and VM. Load times seems to be most prevelant on the first boot, as many VI users have reported, but subsequent load times are shortened after whatever "page-swap" data is been written into whatever directory management utilities for each application. This of course further supports the notion that performance is still enhanaced by having more RAM than the host app will accommodate. The "as-needed" statement also confirms a lot of my suspicions based upon behaviors other Mac users have reported.

The most confusing part of this is that RAM allocation is no longer assigned by the user. I'm still weening myself from old OS9 thinking.

Here are some geek-speak links:

OSX and Virtual Memory

Memory Management and Enhancing OSX Performance

10 Things Apple Did To Make OSX Faster

[musos]

Thanks for that into JWL!
As a lawyer once said: "If by now you are not confused, you haven't been properly advised!" [:D]
I'll need a strong cup of coffee before reading the "geek" links....

[Nick Batzdorf]

I didn't get a chance to read those links, but are you 100% the memory is tied to each processor? Intuitively, that sounds *extremely* unlikely.

[JWL]

@Nick Batzdorf said:

I didn't get a chance to read those links, but are you 100% the memory is tied to each processor? Intuitively, that sounds *extremely* unlikely.

Well, Nick-- it only makes visceral sense. I'm only 1/8 geek by ancestry, and 1/16 nerd! However, the statement was made with respect to the notion that RAM must be installed in even quantities for each processor. If you install 4 GB, then 2 GB goes on the first four slots (pending 8 total GB capacity) and the other two GB goes to the second four slots. One cannot install 4 GB only in the first four slots, leaving the second four slots empty. This was the first big chide I was given before I even paid for my G5-- and it was pointed out that there were two RAM sticks included-- one in slot 1 and the other in slot 5-- and that RAM upgrades must follow the same method in even quantities.

There must be a reason for this-- and it's reasonable to think (accuracy aside) that it is related *somehow* to how the processors use RAM in these respective slots.

But I'm almost at the point now, thanks to Colin, of just buying more RAM and not asking too many questions in exchange for improved performance.

[michael.matthews]

So do we have a bottom line here? Will VI perform better (more instances) on a G% with 6GB than on one with 4GB?

Is this a question with a yes/no answer?

Michael

[JWL]

@michael.matthews said:

So do we have a bottom line here? Will VI perform better (more instances) on a G% with 6GB than on one with 4GB?

Is this a question with a yes/no answer?

Michael

Yes, according to Colin's report. The only question remaining seems to be just how OSX handles the RAM in conjunction with Virtual Memory.

[Nick Batzdorf]

I'm not an EE either, but - very politely, of course [:)] - I suspect that's a serious load of bollox. Sorry.

The reason you need RAM in pairs is because of the way the RAM works. DIMMs - *dual* inline memory modules - are there because of the 64-bit memory bus (2x32 bits), not with one set of memory being dedicated to one processor.

[Nick Batzdorf]

Here you go:

http://en.wikipedia.org/wiki/DIMM

[JWL]

Thanks for the link, Nick.... another piece of the puzzle.

The following is clear:

1. ...that I need to spend much more time researching how dual cpu's really interact with RAM and virtual memory-- and I need to do so before the technoogy becomes entirely antiquated-- could be a moot point in a matter of months. It was because of the misleading explanations I've gotten (from Apple, no less) that I've not put more than 4.5 GB of RAM in my system. Even on this forum, there have been discussions without resolution on the RAM limits of Logic and DP being less than 4 GB, and with OSX magically handling RAM management that there was no real need to add more RAM than your applications will allow.

I'm also wondering when/if the VSL team will continue its PPC Mac tests, knowing that their present stress tests were all done with less than 4GB of RAM installed (and knowing that part of their focus for the rest of the year will be dedicated to the IntelMacs). I'd be most interested in what performance benefits there may be when using VI on a Quad loaded with 16GB RAM... or if there is a point of diminishing returns.

2. ...that further study of how to accurately read the Activity Monitor is needed-- or even a more accurate Activity Monitor is needed. For someone who has no idea other than common visceral sense, there would be no reason to believe that more RAM would improve performance as long as the AM indicates that only half your current RAM is in use. Would be off the mark to now suspect that when AM says that 2GB is left unused that a more accurate reading would be that all or part of the "unused" RAM has actually been "reserved for use according to demands of the applications running"? Could the point at which virtual memory kicks in actually be the point at which OSX sums its used RAM plus its reserved RAM? (if there is a such thing as RAM in reserve)

Understanding how to read this will do much to tell the user what demands one's applications will place on a system, how quickly these demands reach their limit, and at what point more RAM needs to be installed-- until such time one realises that they're overloading their system and should probably get a new computer!

One of the things I appreciated about OS8-9 was that it would tell you when an application needed to have more RAM allocated. It was much easier to know in an instant when to make the adjustments in the Extensions Manager or when to add additional physical RAM.

3. ...that I need to thank the Brits for the word "bollox". Makes me laugh every time!

[musos]

bollocks (also ballocks or bollix): vulgar slang chiefly Brit. noun
1 [in pl. ] the testicles.
2 used to express contempt, annoyance, or defiance.

Thought you'd like to know..... [[;)]]

Getting back on topic, there has been some discussion over at Logic-users about virtual memory and someone posted this interesting titbit:

@Another User said:

VM is only expensive if swapping *out* is actually
happening, and swapping out only happens if there are more *active*
processes than there is memory to hold them all. Further, the only
bits that swap in are the ones being accessed, so even if a process
is waking up periodically and looking for something to do (and there
isn't anything) only a small bit of the process memory will stay in RAM.

Beyond that, all code associated with a process is mapped into VM
when it is launched. This just means that the program as it sits in
the .app package on disk is considered to be swapped out; if code
actually needs to execute, it will be loaded in by the VM system.
This code does not take up swap space, and most of it never ends up
in RAM (since most of it is never executed.)

Beyond that, code never swaps out, since it is marked as read-only
(and a copy is known to be on disk already); only the data swaps
out. So the RAM taken up by the code of formerly active processes is
gained back essentially for free.

All of this is why the swap in count is always nonzero (it's how all
programs load). If the swap out count is nonzero, then swapping has
taken place at some point.

All of this aside, if you launch Logic and the system does need to
swap other stuff out to make enough room for it, you may have
performance problems at first as everything gets pushed out to disk,
but then things should mellow out.

Regards - Colin

[Nick Batzdorf]

http://dictionary.reference.com/search?q=bollox

You can also spell it the way I did. [:)]

[JWL]

So, 'bollox' or 'bollix' can be used as a noun or a verb! I've also heard it used as the adjective "bollicky" (sp?). "Chuffed" is another favorite-- different meaning, of course.

Back OT--

Thanks Colin for the quote-explanation. It confirms my second round of suspicions that once the potentially flagged data is identified, subsequent loads of this data tends to be faster. It also supports Nick's observation of the 2 x 32-bit RAM process, making the benefits of having more RAM less fuzzy-- which itself supports your recent experience of performance improvements.

So, while I continue to kick and scream about adding some sort VI farm, the most effective thing I can do in the meantime is to load up my current system with 8GB.

[julian]

[quote=JWL]So, 'bollox' or 'bollix' can be used as a noun or a verb! I've also heard it used as the adjective "bollicky" (sp?). "Chuffed" is another favorite-- different meaning, of course.

Bollocks... as in the dog's is what us Limey's regard as "Oxford English" !

Regarding RAM I have often appeared to have more performance on my G5 when running 6.5GB than say 4GB. And when I was doing a large project across 4 G5's a couple of years ago we had to reduce 1 to 2GB for a while and performance definately suffered - it was only really running 1 app (Logic) with mainly a lot of audio files and tracks. So there is something beyond the single app limit that makes a difference.

Also I was able to get over 5GB of VI instruments by using a plug-in within Logic and a standalone - you can get even more if you re-name the stand alone app and open a second instance.

Julian

[JWL]

@julian said:

...Also I was able to get over 5GB of VI instruments by using a plug-in within Logic and a standalone - you can get even more if you re-name the stand alone app and open a second instance.

Julian

Renaming the standalone for a second instance? Interesting.

Hmm... alias or copy?

I'm amazed that this even works with app/file associations. Are there any potential problems with directories?

[julian]

Copy app, re-name, and off you go

julian

[Nick Batzdorf]

As far as I know you can't run multiple instances of Logic 7. You could in Logic 6, but my experience is that this isn't a great loss.