@arne said:

The only drawback I can see is that you have to wait 300ms after hitting 'play' before playback starts.

this would be unacceptable for a large portion of our userbase, at least for those who need to add tracks using a keyboard ...

christian

Hi Christian,

I admit meanwhile reading this thread very intrested, I am of course not that far inside software optimization. But as far I understand Arne right, he seems to propose, we wont need no exagerated RAM-Hype if we accept the 300ms latency for loading the Samples totally from HDD.

Does'nt that sound promissing: working with half a terabyte samples on one machine without being limited by any RAM-capacity just admitting that one can't use the midikeyboard in realtime.

Why not allowing the software to do that optionwise for those, who work mainly with large scores in Sequencers. For now, working with large sophisticated scores, which VSL is already well prepared to cope with muscally, still forces you to more or less extensive use of the RAM-optimization function.

I admit the current VI-RAM-Optimization is - compared to former days - an ingenius feature, But still it would be more comfortable to keep all necessary Level II articulations, velocitys available for minor tweaks and adjustments than to optimize other Patches to reload those you want to change, just because there are some users that sometimes like to programm their midievents with realtime playing.

OK, nobody should be forced to accept 300ms Latency if he preferres to boost his Hardwaresetup to play all his midievents in real time, but why should all need little servers to run VSL just because some like to have that realtime option, (No one would play a whole Mahlerscore or anything like this in realtime ...at least that would presumably neither really speed up nor ameilorate the workflow in any way 😉

VSL already seem to be mentaly prepared to sacrifice that realtime-principle in order to get MIR-work for your Userbase,

So I suppose the necessary latency for MIR would be even easier accepted, the earlier one can already optionwise experience its real advantages for the efficiency of Hardware-usage and impacts on the workflow. 

Meanwhile I wont be forced to sell any car to upgrade my hardware in anyway and actually just wait for the moment that I can rely on some more experiences which would be the suitable Hardwaresetup to make the best use of 64-Systems. But since you opend up the VSL-Userbase for those who are lucky that they can afford the SE, this option might make live easier, for those who wont buy server-hardware to use their SE or Download-VI's.

In short a total Direct-from-Disc as Arne proposed it, still seems to me an interesting additional option if ever possible.

best

Steffen

steffen, maybe my statement it would be *worth thinking about* has been drowned out by the amount of involved details ...

direct from disk is also called (patented?) the technology from NI which also cannot cut out buffers for sample headers .. such a *read ahead* technology is a totally other approach to sample based arrangement - we could do a lot of midi detection and processing during such 300 ms which would finally result in a kind of *sample rendering* engine and might also be worth to think about it.

the neccessary latency for the MIR is caused by the nature of convolving and i'd say there is nothing to be sacrificed ... compare with CGI (Computer Generated Images) ... to draw a simple shadow works easily realtime, to show even the first reflections (from objects within a scene) works too, but to render a realistic movie containing several layers, a dozend light sources and reflections (meaning the depth of reflecting light on surfaces, not the number of reflections) is impossible realtime - you would either need incredible CPU power and a high delay or accept offline rendering.

AFAIK convolving in sequoia adds at least 1024 or even 2048 to the latency, and the impulses are not very much and long in this case ....

christian

@arne said:

You should ask your developers if they thought about this. If I hadn't already have a good job I'd build a business case around these two ideas... 

If you actually really consider this idea I'd make my userbase a lot if not indefinitely larger by providing SSD benefits for all possible applications - you'd need to buffer/copy the first portion of every file to SSD to overcome HDD seektime and have a little RAID-like controller manage your files and managing read and write operations, e.g. in an encapsulated external bay. I think a 64GB SSD drive should be enough for buffering 1million files, typical on a 500GB drive should be around 300k at max. and less.

Actually the drawback here is, that it wouldn't really work with monolithic files 😉

PolarBear 

btw ... this reminds me to something else (related to my initial misunderstanding of the three-tier-model above) ....

AFAIK all notebooks with the VISTA logo need to have hybrid-disks (normal harddrive + some flash memory to hold often used data) since june 2007 ....

now lately i've read a report about an extensive test how much such hybrid-disks in fact do speed up system boot / application start / loading data ...

guess what the result has been ... a shattering one .. no or almost no effect ... the difference between *slower* and *faster* drives was much more sigmificant than any performance gain using the hybrid-thingies ....

a not so shattering but nevertheless relatively poor result for waking up from the hibernate mode (memory written to disk when computer *sleeps*) if using flash instead of normal harddrives.

christian

@arne said:

The only drawback I can see is that you have to wait 300ms after hitting 'play' before playback starts.

this would be unacceptable for a large portion of our userbase, at least for those who need to add tracks using a keyboard ...

Of course this wouldn't be a global option but a per-instrument option, so the user will be able to add tracks in the usual low-latency way.

@PolarBear said:

If you actually really consider this idea I'd make my userbase a lot if not indefinitely larger by providing SSD benefits for all possible applications - you'd need to buffer/copy the first portion of every file to SSD to overcome HDD seektime and have a little RAID-like controller manage your files and managing read and write operations, e.g. in an encapsulated external bay.

Actually the drawback here is, that it wouldn't really work with monolithic files 😉

This is a nice idea, but hard to implement on device level. Normally the device doesn't know anything about files, it only knows about sectors. Of course you can teach your device how FAT32, NTFS, UFS, ZFS, but you can imagine how error prone that is. Also, if the user builds a RAID with these device, each device only sees partial filesystems.

A different approach would be to add a 'learn' button. As long as this button is held down, every read access is mirrored to flash. Otherwise the content is untouched.

This would lead to the following usage szenario: the user installs his libs to our drive. Then he starts his VST host and loads one instrument after the other, always pressing the button while the application reads the sample headers. This will give the drive exactly the information it needs, regardless if the data is organized in single or monolithic files.

Cheers,

Arne

@cm said:

now lately i've read a report about an extensive test how much such hybrid-disks in fact do speed up system boot / application start / loading data ...

guess what the result has been ... a shattering one .. no or almost no effect ...

I could only guess, but so far the SSD managment is purely done by Vista and not implemented at hardware level. Cache sizes usually are 256MB which is really small for OS plus apps. And then again - Vista would have to know which files should be loaded to manage to get them to the SSD portion before it really requests them. It would also have to know which application you were to launch next or which application data to precache when booted. So I don't think it's a problem that SSD is involve, but morea structural one, that it's not supported or used to maximum (or in that case: any noticeable) effect.

PolarBear 

Arne, that "learn on startup" idea is nice, but that would only work for monolithic sample files where we know we will have 100% of all possible read accesses done in the loading phase... anyway... this thing is a task of its own and I seriously doubt I'm gonna go and do it.

PolarBear 

So how are those flash disks ging on with VSL on the 2009 ?

can't find modern posts about it ?

will the vsl instrument be able to playback sounds on these disks without loading the all sample in ram ?

mmm maybe i'm wrong but ... when you load an instrument all the sounds are going in the ram right ?

that's why you need huge amout of ram.

with flash disks maybe it would be possible to imagine that the sounds stays and the drive until you play it. Then loaded very fast to ram or direct to sound card (or not i dunno i'm not engeenier).

anyway i can't find recent posts about flash disks and vsl ... i imagine you can load sounds very fast from those drives but about the sreaming, what does it bring.

i'm about to buy a new computer for my samples. about 10Go of ram. or maybe there's something new coming with the flash disks..

hell i understand :P thx for info.

ok so flashdisk is a good option then in addition to good ram...(?)

i only have the strings 1&2 full but i plan to buy woodwinds 1 full. And of course i have other librarys (tonehammer, synthogy pianos, superior drums etc...)
i plan to buy a :

1st solution (low budget)
ASUS P5QL Pro
INTEL Core2 Duo E8400 (3.0 DualCore LGA775)
Nvidia 7200 GS 256 Mo Fanless
DDR2 4x4Go
Vertex SSD SATA (flashdrive)

2nd solution high price (for me of course)
ASUS P6T (Corei7 + DDR3 + PCIE)
INTEL Core2 Duo E8400 (3.0 DualCore LGA775)
Nvidia 7200 GS 256 Mo Fanless (PCI-E)
DDR3 SDRAM 3x4Go PC10600
Vertex SSD SATA (flashdrive)

any advice ?

 it would be nice to have sample data streamed *directly* from a flash disk but the *chain of buffers and latencies* does not allow that. actually reducing the preload buffer for Vienna Instruments from 64 to 8 KB per sample does not work successfully with current flash disks, because strange latency spikes (with the disk) occurre ...

christian

sry i'm no english and not sur to understand 100%

i understood your last post was considering a "directl streaming" from flash drive that lead to strange problems. But does it mean that using flash drives for vsl doesn't bring much more than classical fast drives ?

when you hear little drops in sounds when lots of instrument. Does it come from slow disk or slow cpu ?

 what i'm saying is that also with flash disks it would not be possible to stream everything directly even from a flash drive (RAM needs to be involved anyway)

currently i can't find any benefit streaming sample data from a flash drive justifying the price difference to *normal* drives ... what makes flash drives so tempting is the incredibly low seek time, data throughput is not so much more (especially with the MLC type)

also a german computer magazine ran some real-world tests with operating system on flash drives (XP, VISTA, OS X) vs. normal sATA drives and the difference for startup time and launching applications was not measurable or a few %

i think getting 4 1TB sATA drives for the price of 1 64GB SSD makes any decision easy ...

christian

@cm said:

 i think getting 4 1TB sATA drives for the price of 1 64GB SSD makes any decision easy ...

i see 😊 thx for info