I'll try to sort it out a bit. 8-)

Just to make one thing very clear: Dolby (the company) has once again managed to make one of their proprietary products synonymous with a certain way to handle audio. People were constantly talking about "Dolby Surround" when they simply meant "multichannel", and they say "Atmos" when they are just talking about 3D audio. But there are quite a few "best practice" methods to deal with 3D audio - open source or closed. Atmos is only one of the latter, and quite frankly, it's not even the best choice for making music (in my very personal opinion).

3D audio is basically very simple to achieve and straight forward to handle. A tried-and-tested setup is a so-called 5.1.4 array, which is just the typical circular left/center/right/left surr/right surr speaker/(LFE) in the lower plain, and a more or less identical circle somewhere 30° to 45° above the listener's head (minus Center and of course minus the LFE, which is useless for music in 99% of all cases anyway). The beauty of this format is that you can actually record in it (e.g. Synchron Instruments!), and that you don't need any overly fancy devices to create it from individually recoded tracks either. It's just routing and/or panning between speakers. (Forget about Dolby's moving "Objects" for now, they are just causing trouble in our little world of music creation). And room, of course. 😉

Enter "Binauralisation": A setup like the one described above can be virtually reproduced on conventional stereo headphones to a certain degree, using clever psycho-acoustic tricks and something called "Head-related transfer function" (HRTF). This "model" of a human head is the decisive part: The better it matches your physical appearance, the more convincing the perceived effect will be. Implementing a "binaural encoder" to your DAW couldn't be simpler: You will most likely have a "3D Mix Bus" created somewhere in your project, like you would use a "Stereo Bus". Here you will insert the binauralisation processor (i.e. a plug-in), which will then output the virtual 3D (for headphones) like described above. Maybe you will find the possibility to load a personalised HRTF (e.g. Genelec's "Aural ID"), or special linearisation-EQs for certain headphone models (e.g. Dear VR Monitor), but that's about it.

Sidenote: The available binauralisation plug-ins mostly differ in the quality of their "generalised" HRTF. We at VSL just happen to like the one by Dear VR Monitor a lot, but there's nothing wrong when you prefer the one that comes with your DAW or freeware offerings which come mostly from an academical background.

... now you just have to route this signal to your cans, and you will listen to your mix in 3D! 😊 BTW: This of course is also valid for plain surround or even stereo mixes. Binauralisation first and foremost tries to get rid of the dreaded "in-head-imaging", and of course could be useful for trusty old stereo, too.

A final hint: Be aware that you have to mark any mix you print _with_ that processing as "Headphones Only" (or something like that), because it will sound pretty strange on speakers. *yikes*

Oh, and BTW: As you will understand now, it makes little sense to have binauralisation available _within_ MIR 3D directly, when it is your final mix that has to go binaural. 😉

Enjoy!

PS: VSL's support team is working on a series of tutorials for MIR 3D (videos as well as exemplary DAW projects) that will become available by and by during the next weeks and months.

Thanks Dietz for a detailed reply its much appreciated.  I think MirPro3D is a remarkable update and can't wait to hear immersive sound.  MirPro3D is really the ultimate 3D panner plugin!

I have been wanting to setup surround setup in my home studio, not because I need to ship surround product.  I am just a humble hobbiest I don't have to ship ANY product.  Its only about pleasing my own senses really.  I was simply going to add two rear speakers to my humble home studio so that I could at least have some 2D sound.  

Alright I am getting an Atmos Sonos system in the living room, so maybe playing something back on my TV in the living room with full atmos could be enjoyable eventually too, impress some friends and family.  or perhaps eventually ship some kind of encoded MP4 that friends and family could playback on their home sonos system (or playback through Apple AirPods).  Like it or not, that final output format is likely going to be Dolby Atmos.

I think the vast majority of hobbyists will not have even 5.1 monitoring systems in their home studios, much less 5.1.4.   Even many working composer pros, will simply not have actual 3D speaker configurations in their home studio.  It will have to be binaural monitoring for us all the way if we want to hear an orchestra played back with 3D reverberation from MirPro3D.

In some way it would be hypothetically superior to translate directly from 3rd order ambisonics direct to binaural encoding, with full spherical accuracy...if you know what I mean.  not to mention that my daw, Digital Performer, does not currently have any mixing formats that really support 3D sound.  It only has 5.1, 6.1, 7.1 and 10.2 whatever that is.  They are behind the times it seems.  The only reason LogicPro is coming up is because it does support 7.1.2 (or maybe 7.1.4?) as a mixing format, not sure right now about 5.1.4, I will have to look into that.  But for myself I will never have 5.1.4 speakers in my studio.  At best I will have 4 speakers and many times not even that.

But anyway, I also now realize that its pointless to bring binaural back to the daw from MirPro3D if you intend to do any more mixing with it..as I think any further processing of those binaural tracks would probably destroy the binaural encoding itself...  So maybe that was a dumb idea after all.  

Ideally we could have the entire DAW in ambisonics mode from end to end...and then convert to the final output format from that...whether that be atmos, or some specific speaker configuration or binaural.  That way the full spherical information would be retained all the way through the mixing process and carried into the final output encoding.  

 But its also pointless now in 2022 to hypothetically consider DAW's being full ambisonics all the way through the mixing chain like that.  Maybe in 10-20 years eh, but not now.  So I can see why from a practical standpoint we have to mix in some kind of non-encoded multi-channel format that a DAW can make sense out of it for plugins to work properly, for panning tools, etc..  then re-encode it back to either binaural or atmos....(or ship some other format as requested by a client).  if you think about it even a direct ambisonics-to-atmos encoder would be a much better thing to ship if the client ultimately wants atmos.  The selection of a speaker system is determined at playback time as I understood atmos to be.  But the problem is that our DAWs cannot handle mixing ambisonics, nor atmos nor binaural...they fundamentally need to be mixing in one of the folded down speaker configurations.

So anyway, my DAW doesn't support any 3D speaker configurations yet.  Maybe 10.2 is?  I'm not sure because my current computer doesn't have enough actual audio interface outputs and DP won't even let me setup a mix at 10.2 unless I have enough physical audio outputs to represent 10.2, which I currently don't.  So I can't even try it.  I am going to try LogicPro later though, because with the Atmos addition they also made sure it can support 7.1.2 or 7.1.4 mix down, even without actual speakers connected in the studio.  Then a built in plugin that can re-encode it on the master bus back to binaural so that I can monitor it in my humble home studio.

I actually don't know of any other DAW that provides the capability to encode to binaural on the master bus without a third party plugin...I will look at Cubase12 tomorrow to see what it can do in terms of mixing formats and maybe a binaural output plugin.  

Anyway I was also curious and confused a bit about MirPro3D's ambisonics order blending control and what that does exactly and how the virtual loudspeakers are used in MirPro3D while translating back to an actual mixable output format such as 7.1, 10.2...the best that are currently provided by Digital Performer.

Even "planar" surround is much more fun than stereo, believe me. :-) 

The format 10.2 you mentioned is most likely just that: 10 speakers in the circle, and two LFEs. I suggest that you start with a less demanding setup like 7.1 and take it from there. As long as you instantiate MIR 3D in this channel format, the engine should select a proper Output Format all by itself. If you want to test different approaches to this format then there are alternatives in the pull-down in the Main Mic Selection panel on the right. Don't feel obliged to "roll your own" Output Format in MIR 3D's respective editor. As long as you don't really know what you're actually longing for, chances are that you're overcomplicating things quickly. As soon as you know what you miss, you're ready to work on the finer details here!

3D is great, but you won't miss the top layer speakers as long as you didn't get used to them. ;-)

HTH,

I'm going to be hitting on MOTU to update their software, what mixing formats in particular should I try to make sure they support to best s support the built in output formats of MirPro3D?  I presume 5.1.4 since you mentioned it, or perhaps 7.1.4 since LogicPro is using that one and MirPro3D already has some preset output formats in 5.14, 7.1.2 and 7.1.4.   I guess if I ultimately export a Dolby Atmos creation to play back on my home Sonos system, its probably 5.1.2 or 7.1.2 that would best represent what is actually going to happen at playback time through a typical home system which has a sonos bar and sub in front and two surround speakers in the back.  The sonos bar has a few speakers angled up I believe to mimic having front speakers up high...something like that.  But in the rear, most people including myself, if anything just have a couple of rear speakers...often times in the ceiling.  I do think that more and more people in the future are going to listen to tunes on Spotify using Apple Air Pods..that will be able to translate Atmos into the AirPod spatial thing.  I think probably using LogicPro's 7.1.4 or 7.1.2 format makes a lot of sense..regardless of the fact that I will have to monitor at home using binaural headphones while producing it.

Another question is, what it is the reason for the stereo and planar downmix output formats?

@Dewdman42 said:

[...] Another question is, what it is the reason for the stereo and planar downmix output formats?

I'm not sure that I really get the question, sorry ...?

why do those down mix output formats exist, and when should we use them?

I just read up a bit about what Cubase12 can do.  Holy crap that must be the 3D king of DAW's...its support fro 3D audio is very extensive and its going to take me quite a long time to figure out exactly what it can do...but among other things, besides supporting all the mixing formats we have talked about so far like 5.1.4, 7.1.2, 7.1.4, etc..  it also supports mixing directly in Ambisonics!!!  It will take me quite a while to figure out how it all works before I can say anymore about how that works, but I did try already to at least insert MirPro3D plugin onto a 3rd order ambisonic track in Cubase..  the plugin showed 16 meters.  Fine so far!  I opened up MirPro3D and there is an output format called RAW 3rd order ambisonics.  I selected it and the mic array became some kind of non-mic sphere..it skips any mic virtualization I guess and just passes the raw ambisonics from any direction directly back out to Cubase!  

Cubase also provides numerous ways to monitor including built in binaural encoding.

Alright conceptually that looks interesting, but like I said, I have a lot more to figure out before its going to be working right to say any more, but that looks to me like I could literally use Cubase's binaural encoding to basically skip any virtual speaker fold downs and hear exactly the ambisonic representation of MirPro3D's rooms...in binaural headphones as best as they are capable of translating.  This may or may not  cause my CPU's to explode, so we shall see...

But Anyway, what I can say is between Cubase, LogicPro and DP....Cubase has WAYY more stuff related to 3D audio, Dolby Atmos, Binaural monitoring and even ambisonic mixing.  You can even mix directly in Dolby Atmos, dealing with Atmos objects instead of conceptual virtual loudspeakers.  I think I would have to take a class to understand how all that works though...so we'll see if I get anywhere with this, but just wanted to report what I found.

@Dewdman42 said:

why do those down mix output formats exist, and when should we use them

I'm still not sure I really understand the question, but I will try to answer it to the best of my abilities:

I found out that even though full "spherical" decoding from Higher Order Ambisonics is extremely realistic in 3D, it can also be seen as some kind of glorified "multi-mic" recording. The ruthless sound engineer in my head thought, "Well, we do great-sounding downmixes from real surround- and 3D-recordings, why not from spherical decodings, too?" ... and after a few seriously failed attempts 8-) I developed several approaches that made a full 7.(1*).4 sphere sound great in simple 5.1 or even stereo, too. The trick is to use the simplistic, built-in Matrix Mixer in MIR 3D's Output Format Editor. 

... these settings will give you very different results than our "old-school" capsule-based decodings. The most obvious difference ist that you can make good use of stage positions on the hard left and right sides of the Main Mic without sounding strange (even positions in the back will work surprisingly good, sometimes). Their downside is that the frontal center position seems to lack a bit of "grip" in these setups, that's why I often mixed in an additional capsule-array for the dry signals as support.

EDIT: Funny sidenote: Our ingenious Ambisonics development partners from IEM at the University of Graz were pretty sure that no one in their academic circles had done this "downmixing" before - this concept lacks any kind of scientific purism. But you know - if it sounds right, it is right. 😄

Does this answer your questions ...?

*) PS: The brackets around the LFE channel in formats like 7.(1).2 stands for the simple fact that it takes care for the routing of the LFE without actually using it. Makes the integration in the DAW a bit easier than the seemingly more "logical" 7.0.2 in some cases.

@Another User said:

I will explore that later when I have an actual way to hear 3D monitoring somehow

Just to avoid possible misunderstandings: You don't need anything fancy for a stereo downmix of a spherical decoding from HOA. It's just that: stereo. 😊

Enjoy!

@Dewdman42 said:

[...] I totally get it. I will have to explore all of these later and read carefully the notes Dietz added to each one.

Thanks for the friendly words! You're definitely on the right track. 

Just to keep things in perspective: I may know a thing or two about audio engineering and music mixing, that's true. But please keep in mind that HOA and its intricacies are themselves an area where even I don't have "many years" of experience, just a little over two. MIR 3D as a production-ready application is even younger than that. So we're all here really just scratching the surface so far.

The best is yet to come, I'm sure. 😊

@Dietz said:

You don't need anything fancy for a stereo downmix of a spherical decoding from HOA. It's just that: stereo. 😊

One strange question that occurred to me this morning.  So basically you're saying that the "sphere" of ambisonic information is there in the MirPro3D and we are pretty much always folding audio down to some number of outputs....could be stereo or surround or even 7.1.2 immersive...  But its always being folded down..its just HOW its folded down that will differ in the virtual mic array techniques employed.

But here's the new question...  While MirPro3D has encoded a 360 degree sphere of information for any XYZ point in space where the mic array is placed...  When we play it back eventually on some system that supports say 5.1.2, we don't have any speakers to represent the bottom half of the sphere.  We have ear level, and we have above the head level...but unless I'm missing something, we don't have anything to playback sounds below us.  

How does MirPro3D translate sounds from below the mic array into playback systems, which really are only half of a sphere if you think about it?  If I'm understanding that right, which I very well might not be.

I guess one could assume that it gets folded into the ear level speakers in some way, but it makes an interesting case for not putting the mic array too high in the air as that would squash a large amount of sonic information Ito the ear level rather then below the mics as it actually is.  Am I making sense?

Very good question! This is where the so-called "coefficients" come into play, i.e. the data we we refer to and load in the lower half of the Output Format Editor. Call them "virtual speakers", if you like, which can, but don't have to relate to physical speakers.

To avoid even more complexity in an application that might be overwhelming already we decided to leave at least _that_ part of the equation to the specialists of our academic development partners at IEM Graz. They offer a fantastic, Ambisonics-focused software suite (freeware!) that contains the tool we use to create our sets of coefficients. 

... explaining the whole concept is too much for a little forum positing. Please continue reading here: 

-> https://plugins.iem.at/docs/allradecoder/

All of this is ongoing recent development, so we might see/hear even better "coefficients" in the not-so-distant future. :-.)

HTH,

Thanks so much Dietz & Dewdman for this very instructive conversation. The fog has cleared. You have addressed the struggles I was having (in a previous thread) until I realized the Dolby-Apple Logic side of the equation was causing all the confusion in converting legacy MIR projects to the new 3D and, like Dewdman, was anxious to “hear” the new MIR itself – without having to assign near, mid & far in the Atmos plugin…I have no clue what that would do to the MIR audio. The coming tutorial videos will no doubt make things even clearer. But for now, I have opted for the Dear VR Monitor solution which seems to be able to render MIR Prod 3D’s environment as intended. Thanks again for your thoroughness.

PS. And Deadman, I have appreciated & well used your VSL AU3 Logic templates. Many thanks.

@jimwine said:

But for now, I have opted for the Dear VR Monitor solution which seems to be able to render MIR Prod 3D’s environment as intended. Thanks again for your thoroughness.

 

Oh that is interesting.  dearVR is handling binaural encoding differently then Dolby?  That kind of makes sense though...so something for me to investigate further.  

I was just reading an article yesterday about how Apple's Binaural encoder is different from Dolby's..  Apple uses theirs for their so called "Spatial Audio" feature on Apple Music.  A lot of people had problems though because Atmos productions did not translate well after having been produced and presumably monitored using Atmos system...or perhaps Dolby's monitoring...then on Apple Music as so called "Spatial audio" it would sound totally wrong...in particular it seems to do with how center channel is handled and Apple's version tends to make everything wider then life.....or something to that extent.  In any case, last year Apple added actual Dolby Atmos as also being playable from AppleMusic now...perhaps because of these binaural compatibility problems.  

So yea I'm curious how dearVR's binaural would differ from Dolby's.

Apple's also does not have near, mid far stuff.  But when I have listened to some test binaural recordings, everything sounded like it was just barely outside my skull.  I could hear things moving around in front and around the sides and the back of my head..just barely.  I think Dolby is adding, perhaps, some processing to get things to sound further away from our head.  But then, that is what MirPro3D is supposed to do...so I guess in LogicPro you would want to monitor MirPro3D with the Dolby renderer, and make sure to set the speakers to OFF (not near, mid or far).  That way it won't be interjecting any more distance related processing that might interfere with what MirPro3D is doing.

Well I'm the kind of person that when I listen to headphones it generally sounds like its inside my head....so having something put the sounds outside my head is a welcome improvement, but its not the same as listening to loudspeakers that are surrounding you

@Dietz said:

Very good question! This is where the so-called "coefficients" come into play, i.e. the data we we refer to and load in the lower half of the Output Format Editor. Call them "virtual speakers", if you like, which can, but don't have to relate to physical speakers.

-> https://plugins.iem.at/docs/allradecoder/

That is very interesting.  Imaginary loudspeakers eh.  Is there any way I can view the actual JSON for the factory presets that are using custom coefficients?  Are they found anywhere after install or just buried inside the MirPro3D binary?

So another thought/question I have is that in a real world situation, SOME of the sound coming "from below", would make it into the mics in a real recording scenario.  They wouldn't be spatially represented during playback as being from below; in a playback system that only has speakers at or above ear level...but still some of that reflected sound off the ground, etc..would go into the mics and be present in any kind of recording.  yea?

I notice that the factory preset output formats only include virtual loudspeakers (and their coefficient parameters), for the 3D presets.  The other presets don't seem to have them for some reason.  Why is that?  If nothing else, don't they also need to handle the "below head level" sounds in some way?

I also noticed in the iem.at docs a comment about 1st vs 3rd order ambisonics...and they made a comment that 1st order are smoother panning, while 3rd order are bumpier panning.  Which might explain what the "Ambisonics Order Blend" control is for?   There are only a couple of the factory presets that actually have that control set to anything other than 100% first order.  So I'm curious what the implications of that are, and when or why some of the presets have sometimes nudged it halfway towards 3rd order ambisonics?

You see the used coefficients in the left part of the lower half of the Output Format Editor. The list is a bit ugly (... waiting for a GUI update ;-) ...), but you see the speaker index and its coordinates (angles, actually).

Yes, an ideal Ambisonics reproduction system would be spherical, too, with the listener in the center. This is not easy to get in Real Life, that's why this genius IEM AllRADecoder offers so-called "imaginary speakers". They aren't visible in MIR 3D and serve no direct reproduction tasks, but they allow for balancing the geometry (and thus the energy dispersion) within an "imperfect" sphere. ... this topic is not my forte, I have to learn a lot myself in this field.

Re: Capsules in 3rd Order: Just try it yourself. Push the blender towards 100% 3rd order and watch the shape of a beautiful Fig-8 capsule derail completely. ;-D ... this is what the guys at IEM describe as "bumpy". There is no real-world equivalent for these shapes, and they are quite hard to handle. The Blender (invented by VSL's chief software developer, Martin Saleteg, IIRC) tries to offer workarounds, so we can have it all: Better imaging, embracing space _and_ smooth panning.

@Dietz said:

Re: Capsules in 3rd Order: Just try it yourself. Push the blender towards 100% 3rd order and watch the shape of a beautiful Fig-8 capsule derail completely. ;-D ... this is what the guys at IEM describe as "bumpy". There is no real-world equivalent for these shapes, and they are quite hard to handle. The Blender (invented by VSL's chief software developer, Martin Saleteg, IIRC) tries to offer workarounds, so we can have it all: Better imaging, embracing space _and_ smooth panning.

I guess 1st order is not as precise..the big bubble is somewhat ambiguous about the precise imaging of any given sound within that area...while 3rd order is more precise.....but 3rd order has some dead spaces...  So its only more precise for capturing the sound that is not in a dead spot.  The 1st order will not have so many dead spots...but then you lose exact spatialization...  something like that.  Obviously 7th order would hypothetically have even more precision and perhaps less dead spots, but would be unusable on my 2010 MacPro.  The blender lets us get a little in-between in some way, but when I drag it around I see the 3D photo distort in ways that don't make any sense to me to understand what it might be doing to the sound.  hehe.  

Well Aside from tweaking around with it and listening, I don't think I would be able to consciously do anything with that blender control or even the coefficients, to make my own user presets...  Maybe someday.  For now I will stick to using Factory presets...once I figure out which ones are the ones I prefer.