Wow, this thread is alive, with the sound of musings :)
First, Jason, thank you for waking up my baby half an hour early this morning, with your comment about the page turner in my video I shared. I read it to Becky and she burst in to laughter so loud that the Queen of the Night has nothing on her...and that laughter woke my little girl preventing me from enjoying my morning coffee. Seriously, that was the funniest thing I've read in a long time! By the way, that page turner is available for hire, just contact me for details and rates.
To add to the discussion, I think what it comes down to is composers and performers both want their music/ideas to be understood and appreciated by an audience. As was alluded to by Seventh Sam, people go to watch performers and often, the main way in which an audience interprets the music, is actually through the visual cues on stage, such as grand, sweeping gestures, facial expressions, audience reaction and more. The general audience member will instinctively react to the music and determine whether or not it sounds good to them, but won't be able to delve deeper in to the actual substance of what they are hearing, especially upon a first listen of the work, which is typical for most classical/new music atendees.
The composer, on the other hand, works in isolation, creating music that will largely be experienced on a very superficial level by most listeners, whether they consume a virtual recording, or attend a live performance. I think this becomes the issue that alienates composers from their audience. Music is a language of communication, and when the message is not fully received by the listener, through no fault of their own, the composer ends up internalizing the meaning of their work, and over time becomes more and more resistant to sharing outwardly with others. After all, what's the point? A live performance will potentially resonate with some audience members, but most will be, as has been mentioned, there to be seen. Most will take away trivial meaning from the performance, talking over drinks afterward about what the pianist was wearing, or the hillarious scowl faces the conductor made as he stared a hole through the orchestra.
Beyond this, the composer's relationship with the musicians is also volatile. The composer knowing that the musicians are the conduit between their ideas and the communication of those ideas to an audience, rely on the musicians to essentially "speak on the composer's behalf." Since musicians themselves often see a performance as "just another job" (especially work for hire musicians, or orchestras in which individual musicians have no ownership over what they must play), the passion that was poured in to the composition, is often lacking in the musicians, who of course, want to devote as little energy/time to learning a piece as possible (this goes back to Paul's well-founded argument that the music should be easy enough to motivate performers).
As William said with Glenn Gould, one-of-a-kind performers suffer the same isolation over time. As the recognition and fame develops, audiences want to come out and see the genius in action. Despite the fact that the house may be full, someone like Gould is all too aware of how few in attendance actually give a damn about the music. They're not married to it. They're not obsessed with every intricate detail. They don't listen to partitas with headphones in a dark room (love that, William!) And so, someone like Gould, eventually sick of hearing another candy unwrapped, another whisper, another cough, and another meaningless standing ovation from an audience filled with only a handful of actually appreciative/knowledgeable listeners, decides it's not worth putting himself out there for the lemmings that fill the concert hall because it's the in thing to do. Of course, with Gould, there's also the desire to explore recording technology and create the perfect technical masterpiece...even with his prodigious technique, he was known for exploring multi-tracking and other techniques to bring out exactly what he wanted in a given performance. Again, though, the vast majority of listeners wouldn't identify the genius, and so his work belongs to the few that have the ability to interact deeply with it.
Bottom line, from my perspective, the composer/performer that wants to be understood by his/her audience, will have to communicate to them on a level that resonates with that audience. If one decides to explore music in a way that challenges one's own abilities and helps them grow musically, they likely will suffer the apathy/indifference of the vast majority of people. There's a reason that a youtube video in which a person fills a hotel room with balloons gets 4 million views, and a performance of Mahler gets 100 views. The best musicians can do, is find a small, like-minded community and share their work amongst each other, knowing that the feedback of one person that actually was interested in/"got" the music, means more than a concert hall filled with people looking to be seen by others.
Dave