(wrote this before the post above but will leave it here anyway)

Well, I think the point is that we like the direction of VI and the interface, but unfortunately the hardware is not there yet to load and stream the entire cube on a single box with the full sample quality. Nor probably will it be for a few years yet.
We also like the fact that VI sounds near-real when exported.
So we are happy with the interface and happy with the end result.
What we are not yet happy with is the efficiency of the work-flow that sits between the two. Granted VSL are working hard on this as you can see from other posts and that is great! But there is more we can do to make the VI engine even smarter.
We don't need a different product like OPUS. The product and direction are correct.
What we do want is the ability to be able to hear our pieces live as we compose them, live, with no pops of clicks and have a workflow that can run on a single box.
What I have suggested above would achieve this. When you export/mix-down, of course, you export down at full quality... you don't care that it takes longer at that point... it is off-line... you can go and have a coffee...
The bottom-line is that I believe an awful lot of the composers here would jump at the chance of trading real-time quality (ONLY DURING COMPOSING remember) for the ability to load the entire orchestral template in one DAW project.
I can understand that with the care and attention VSL go to ensure we have a high quality, well-recorded product, that the notion of somehow making that product sound 'worse' than when it was recorded is sacrasanct.
Of course, I would prefer everything now... but we are not going to get that.
In terms of how easy this would to implement, I imagine pretty easy. The only issues I can think that would get in the way would be
a) If the encryption of the files means that you need to have access to all data.
b) how to make the DAW know you are exporting and therefore turn off the draft mode for that.
We should try an experiment... export a track at the full bandwidth and trying dithering it down to different sample rates. I think we would find that we could quite happily live with much lower sampler rates during composition.

@cm said:

ok, i was understanding the term *sample* in the sense it is used in the VSL product description (violin staccato has xy samples, ect).

if we look at it in a digital manner we have to clear definitions first:
- the headers of a PCM/RIFF file are using a 32bit data format (this is the reason why a RIFF file is limited to 4 GB maximum, usually even 2 GB because of the difference between signed and unsigned integer)
- the sampling bit-depth can be 8/16/24/32 bit (VSL is 16 bit currently, of course theoretically it can be lower than 8 bit but i'm sure we wouldn't like that, additionally 8 bit is stored as unsigned integer, wheras 16 bit as signed integer, 32bit would be floating point)
- the sampling rate can be something (VSL is using 44100 Hz currently - /2 = 22050, /4=11025, /8=5512,5 ... oops, here we would leave the integer and we had to start rounding because i can't display a half sample)
- PCM data ist stored interleaved in frames and each frame consists of sequential data for the used channels (2 in case of VSL because of stereo)
i'm basically referring to this description
- the Nyquist-Shannon-Theorem says that unambiguous reconstruction is possible if the signal is bandlimited and the sampling frequency is greater than twice the signal bandwidth. that's why telephone lines don't sound too good and i think a sampling frequency of 11050 would leave us with an unacceptable *quality*.
one could possibly now *invent* (=interpolate) the missing samples to stay with 44100 or repeat the *reference sample* but i've never tried that and probably i wouldn't like to hear the result. do you have any examples for that to compare the quality using a human ear?

based on this - where would you cut the amount of information into half (and further /4, /8, ect) without loosing an acceptable sound?

another issue is the needed amount of preloaded samples. this is basically only neccessary to fill the various bufffers (soundcard, audio application, operating system, harddisk I/O) and *hide* the various latencies.
128 kB (=32.768 samples 16 bit stereo) is a common value for most samling engines - ViennaInstruments needs far less (and is 24 bit!).
256 Byte (= 64 samples 16bit stereo) is a common value for soundcards - many users need to use a higher value because their system cannot deliver such a data stream continuosly.

any efforts in this direction (reducing amount of preloaded data) are useless IMO and stay theoretical unless we get storage media with an almost-zero seektime like compact-flash or solid state disks with enough capacity for a reasonable price.
christian

Thanks for the detailed reply Christian. I think we're getting to an interesting place now...
This is how I envisaged this working (and I'm fully prepared to be told that it doesn't work - it is just an idea)
1) You have a 'quality' value that can be controlled on the VI which is defaulted to 1, but can be changed to 2 or 4. As you say, smaller values than 4 would not only probably sound bad, but would also start to cause rounding issues. as 44100/8 is not a round number. That isn't to say it isn't possible of course.
2) When pre-loading or streaming samples from disk, you use this value to only load every nth sample value in the file, skipping the samples in between. In other words with a value set to 4, you take sample 0,4,8,12,16 etc.
This in turn will reduce the pre-load buffer by a factor of n. In other words, with a value of 4, we could theoretically load 4x as many instruments into memory. What you end up loading is a less accurate sample waveform, but one that is much smaller.
3) When playing back the sample (at 44100), you repeat the same sample n times so that it appears to be a sample recorded at 44100. At no point does the VI change its sample rate. It always remain at 44100.

There are improvements on the above. As you say, you could interpolate the samples rather than playing 4 identical samples thus you still save on memory but synthetically create slighly different values for each played sample so that it sound closer to the original.
After all, this is how computer animation works. Quite often frames are interpolated from the frames around it to speed up workflow.

I would like to experiment with this... although I'll have to read up on how to read a wav file programmatically... what would be an interesting test would be to read a 44100 wav file into memory and write out different versions of the same file using different settings as above. Also try out the interpolation. I may look into this and get back to the forum and post the results for people's assessment of the quality.

I don't think working with 11khz samplerate would be fun, but I think most composers would be happy composing with 22kHz samplerate as "draft" mode if they'd get more bang out of the hardware. 11k audio bandwidth is not too bad, but 5.5k is really bad.

@cm said:

any efforts in this direction (reducing amount of preloaded data) are useless IMO and stay theoretical unless we get storage media with an almost-zero seektime like compact-flash or solid state disks with enough capacity for a reasonable price.
christian

Nobody responded to my idea if USB flash drives wouldn't be suitable for sample preload buffering. Are they too slow?

for another reason i've done some research on such storage-types and found the ultimative values at 8 GB with 40 MB/s.
whereas 40 MB/s is not so much more what modern harddisks can deliver 8 GB will be definately too little - a rough math would result in the need of about 24 GB for the current symphonic cube.
not having spent a thought so far how such an additional buffer would fit in any sampling system ....

there are of course a new type of solid state disks (also using NAND technology) with more storage space for datacenter and military use, but they are not faster and the *old type* using battery-powered RAM are really overkill.
christian

@cm said:

another issue is the needed amount of preloaded samples. this is basically only neccessary to fill the various bufffers (soundcard, audio application, operating system, harddisk I/O) and *hide* the various latencies.
128 kB (=32.768 samples 16 bit stereo) is a common value for most samling engines - ViennaInstruments needs far less (and is 24 bit!).
256 Byte (= 64 samples 16bit stereo) is a common value for soundcards - many users need to use a higher value because their system cannot deliver such a data stream continuosly.

any efforts in this direction (reducing amount of preloaded data) are useless IMO and stay theoretical unless we get storage media with an almost-zero seektime like compact-flash or solid state disks with enough capacity for a reasonable price.
christian

Just reading this again... not sure if I mis-read your post, but you seem to be indicating that you would still need a pre-load of the original size?

That isn't so.

The approach I am suggesting would reduce the preload buffer size by the factor of 'n' (which is where the memory savings occur). The buffer would be a 'virtual' buffer in that it would only hold some of the data and synthesise the missing data that lies between the data points. Synthesise meaning that it would either repeat the same sample point 'n' times or interpolate from the points either side.

I am presuming that the pre-load buffer is entirely controlled inside the C++ VI application to make this possible?

Just in case this wasn't clear...

I'm still looking into trying this out... I want to write a Java app that reads a 44100 .wav file and creates a new .wav file using this approach. I want to produce .wav files at different freq and using different 'draft' settings and see what they sound like.

The proof of the pudding - in this case - is in the hearing...

Stay tuned... (if you've not already tuned out)

Wtih a PCMCIA card plug in to an external box housing memory and/or hard drives, and with future laptops hopefully housing 16gigs plus of RAM - the technolog is almost there to easily run the full cube on a laptop, it must be less than 2 years away.

I just had this idea to circumvent the 2GB RAM software limitation when having more of RAM installed: Why can't the sampler developers use a virtual disc, a RAMDisc? Or several of them?

VSL forum is where the engeneering lives ... [:P]
not a bad idea, although this is something the user had to configure in his operating system.
AFAIK also a RAM-disk is limited to 2GB. to use several ones would need to have the available physical memory in your computer.
so 1 RAM-disk 2 GB, kernel memory 1 GB (for windows, drivers, ect), user memory 2 GB (for samples) makes 5 GB RAM in total on your computer and you'd need an operating system which can use more than 4 GB (eg. windows 2003 server enterprise edition - happy downsizing, btw.)

somehow it looks as if we'd need a vista64bit version to get somewhere, and then a 64bit VI could run on it and - we would not need any RAM-disk .... seems cat bytes tail
christian

Engineering is fun! A lot of engineers are musicians funnily enough...

The other thing that struck me with reference to the 'draft' mode idea is that its primary aim is to reduce the memory used by pre-loading the samples.
Perhaps, if that is the case, the draft quality need only apply to the pre-loaded samples. All subsequently streamed samples would be played as they currently are, at full quality?
The pre-load size is so small and most notes have a slow(ish) attack, that you may not notice the difference...
The utopia here is to not have any pre-load. It is merely there, as Christian has pointed out, to circumvent the latency in your system.
In my humble opinion, the only real barrier to having the entire cube loaded is the issue of the RAM used by the pre-load. If we can intelligently solve that problem...

I agree, with what little I know, it seems to me that the fact is that the best solution here is to wait a short while for technology to catch up. Spending money on software development to compensate for the lack of capable hardware will only result in that software being redundant because by the time it's ready and usable, the hardware availability will negate the need for the compensatory software development in the first place. At best there will be a 6 month to 1 year period when the software would be worthwhile having been developed and usable (and then you have to ask how many would use it?). Of course I'm guessing at technology changes over that time, but generally speaking, the investment of time and money probably wouldn't be worth it, so it's probably better to wait for the hardware to catch up. I think Mac laptops even now *could* hold 16gigs(???), if the chips were available in that size ie 2X 8gig simms. PC's I don't know about but as cm says, a 64bit OS will yield results in the hardware department - better to let MS do the work there and again the hardware will catch up quickly thereafter (straight away for towers, a bit later perhaps for laptops)

Miklos.

The solution should be solved both by hardware and by software. There is bound to be a need to load and play more samples in the future and you'll find yourself right back where you started... software always lags behind hardware.
What has been spoken about on this forum shouldn't technically be that difficult to implement. In fact, I am developing a proto-type of this in software as we speak to prove whether it will work or not.
I'll make my findings available to the technical team of VI
Fingers crossed...

@cm said:

VSL forum is where the engeneering lives ... [:P]
not a bad idea, although this is something the user had to configure in his operating system.
AFAIK also a RAM-disk is limited to 2GB. to use several ones would need to have the available physical memory in your computer.
so 1 RAM-disk 2 GB, kernel memory 1 GB (for windows, drivers, ect), user memory 2 GB (for samples) makes 5 GB RAM in total on your computer and you'd need an operating system which can use more than 4 GB (eg. windows 2003 server enterprise edition - happy downsizing, btw.)

somehow it looks as if we'd need a vista64bit version to get somewhere, and then a 64bit VI could run on it and - we would not need any RAM-disk .... seems cat bytes tail
christian

Well, for people with a 4GB machine that still could be a temporary solution for the 2GB limitation until Vista is out.
Setting up a RAMdisc is not too hard and the Sampler just needed another option in the preferences dialog...

BTW, I'm an engineer's son... [[;)]]

@cm said:

VSL forum is where the engeneering lives ... [:P]
not a bad idea, although this is something the user had to configure in his operating system.
AFAIK also a RAM-disk is limited to 2GB. to use several ones would need to have the available physical memory in your computer.
so 1 RAM-disk 2 GB, kernel memory 1 GB (for windows, drivers, ect), user memory 2 GB (for samples) makes 5 GB RAM in total on your computer and you'd need an operating system which can use more than 4 GB (eg. windows 2003 server enterprise edition - happy downsizing, btw.)

somehow it looks as if we'd need a vista64bit version to get somewhere, and then a 64bit VI could run on it and - we would not need any RAM-disk .... seems cat bytes tail
christian

Actually, upon further investigation, this is a real possibility. If you have a look at the following link:
http://www.gigabyte.com.tw/Products/Storage/Products_Overview.aspx?ProductID=2180&ProductName=GC-RAMDISK
... you'll see that the RAMDisk here is 4gb... in fact, if you think about it, there is no reason why it could not be bigger. After all, it is merely using RAM to store what is otherwise stored on magnetic media. The index into the RAM isn't address space as in memory, but is file system based therefore isn't subject to the same 32bit addressing limitations. Obviously magnetic drives can be 100s of GB, therefore in theory so can RAMDisk.
Therefore, if you could somehow separate the preloaded files from the streamed files, you could place those on RAMDisk for fast access.
Playing back samples would therefore mean that you would need NO pre-load at all in your memory space. When you hit a note, the VI player would simply load the 'pre-load' file on the RAMdisk, then subsequently start loading the non 'pre-load' file on the slower drive.
Of course, once harddrives become totally RAM based and memory and disk converge... [:)]
Christian... does that sound at all possible?
You will also notice that the above product has a back-up battery so data remains on the RAMDisk after shut-down... In other words the pre-load files could be loaded onto the RAMDisk and left there. This would mean ZERO load times for VI. Just bring up the VI VST and go...

i'm leaving aside now considerations if it would be even possible to *split* the preload-portion from the rest of the data (as paynterr assumes) to be stored on an extra drive, this goes too deep into the design of the VI.

but there is something moving http://techreport.com/onearticle.x/10116 and from what i've heard 16 GB is on their roadmap although i found no information about needed RAM-sticks (4GB?) and pricing.
common server chipsets can hold 32 GB RAM in 16 slots and need ECC memory, so following this idea could easily get very expensive quickly.

a zero-loadtime would even then be not possible considering the read-cycles for RAM and a rough calculation turns up that max. available throughput would be limited by several components to ~250 MB/s (where capital B means Bytes). not a bad value, but still *only* 1/6 load time than with a very fast harddrive (~40 MB/s).

so from a value-for-money pont of view (and to avoid complexity of a setup) i'd say it's more reasonable to invest in a high-end raid system, which (correctly configured) can give you 110-150 MB/s.

besides it is of course interesting to think about such optimizations - isn't it a little bit frightening that we are talking about components usually found in supercomputers and number-crunchers when the aim is just to make music?
christian

Rather interesting, I would say! I found an unofficial announcement that Gigabyte wants to offer the i-RAM 2 with up to 16GB RAM which is not fitted on a PCI card but in a 5 1/4" housing.

Although the price for the i-ram seems to be somewhere between 120 and 150 $ it is cheaper than a second computer with all the other problems.

The RAM disk would seem to only be a viable possibility if:
a) It was so large that you could load a significant number of the VI on there (unlikely in the near future)
or
b) The VI samples were split into "pre-load" and "stream" sample files, the pre-load being placed - presumably - on the ram disk. The slower stream files placed on your normal disks.