Have you ever wanted to get a taste of what it might be like to conduct your own Prom? To sit in the middle of the 主播大秀 Philharmonic orchestra? Or to listen in really closely to what just one of the musicians is playing? We're using our audio orchestration technology to give listeners that opportunity.
This week, we鈥檝e launched Pick A Part on 主播大秀 Taster in collaboration with the 主播大秀 Philharmonic. With Pick A Part, you can connect your devices (laptops, phones, and tablets) together and choose which instruments they play. Everything鈥檚 synchronised as closely as possible using our audio orchestration software.
Pick A Part is the latest in a series of orchestrated trial productions that started with audio drama The Vostok-K Incident in 2018 and recently saw the release of an orchestrated episode of 1927's Decameron Nights. You can read more about how we worked with 1927 to make Decameron Nights.
Adapting to lockdown
We've been discussing audio orchestration with the 主播大秀 Philharmonic for a while now - it's something that we've been interested in the potential of, but it's been hard to actually put together. One of the challenges is that orchestral music is normally recorded with all of the musicians playing together at the same time, whilst audio orchestration required object-based audio - that's having access to all of the individual parts that make up the piece.
That all changed when the country went into lockdown earlier this year, and musicians at the 主播大秀 Philharmonic had to adapt to making music from their homes. Suddenly, we had unprecedented access to recordings of individual musicians playing their parts to be edited together later. So we jumped on the opportunity to see if our idea of applying device orchestration to orchestral music would work.
Simon Highfield, a technical producer at the 主播大秀 Philharmonic, sent over the recordings from the musicians so we could start experimenting. And we were immediately very excited by how well it worked. Not only was this an immersive listening experience just by connecting one or two phones, but it was also novel to be able to explore a piece of music in such detail.
Simon Webb, director of the 主播大秀 Philharmonic:
"There are so many potential applications for Pick-a-Part; this is the 主播大秀 at its very best, collaborating, innovating and sharing. Whether in an educational setting exploring the inner workings of a string quartet, exploring different audio mixes of a recording, or playing along with professional musicians in your own home, this is a brilliant piece of kit."
How we made Pick A Part
Pick A Part is the latest production that we've made with our new production tool, Audio Orchestrator. With Audio Orchestrator, you can import your audio files, set up rules that determine how they're assigned to connected devices, and export a prototype application.
To make Pick A Part, we modified the template application that is produced by Audio Orchestrator to add some new features, including:
- showing a picture of the selected instrument on the connected devices;
- automatically assigning an instrument once a device has been connected; and
- displaying a representation of the instruments assigned to each connected device.
We're using trial productions like this to experiment with new features and see what we might want to feed back into the Audio Orchestrator software.
If you'd like to find out more about Audio Orchestrator, you can request access through 主播大秀 MakerBox, read our blog post, or have a look at the .
Keeping instruments in sync
I mentioned above that one barrier to using audio orchestration with classical music was that we needed access to the individual instruments. The second challenge that we needed to address was that music requires very accurate synchronisation between devices. The synchronisation framework that we use is accurate to within around 10鈥�20 milliseconds, but there's sometimes additional delay added by devices, which can reach up to 200 milliseconds. That's OK for the types of sounds in the drama productions that we'd previously released, but not good enough for music.
To get around this, we developed a manual calibration stage so that listeners can easily correct the time-alignment for their own devices. Danial Haddadi, an , ran an experiment to determine how well listeners can align two devices, and what method is easiest for doing this. Based on the experimental results, he implemented the calibration tool that we used in Pick A Part. The same speech content is played from the main device and an aux device, and the listener is asked to adjust the time delay on the aux device until the two devices are in sync.
Pick A Part is the first orchestrated music experience that we've released, so we're interested to find out about how well this works and how easy it is to use the calibration tool.
What's next?
We've got a few more trial productions made with Audio Orchestrator in the pipeline, including a spooky Hallowe'en drama with the 主播大秀 Writersroom. Keep an eye on 主播大秀 Taster if you'd like to hear more. Hundreds of creators have requested access to Audio Orchestrator, and we're really looking forward to see the range of ideas that they develop.
This will help us to evaluate the production tool, decide on new features to implement and assess the potential for audio device orchestration. We're also involved in research collaborations at the universities of and looking at technical and creative aspects of device orchestration.
- -
- 主播大秀 MakerBox - Audio Orchestrator
- 主播大秀 R&D - Vostok K Incident - Immersive Spatial Sound Using Personal Audio Devices
- 主播大秀 R&D - Vostok-K Incident: Immersive Audio Drama on Personal Devices
- 主播大秀 R&D - Evaluation of an immersive audio experience
- 主播大秀 R&D - Exploring audio device orchestration with audio professionals
- 主播大秀 R&D - Framework for web delivery of immersive audio experiences using device orchestration
- 主播大秀 R&D - The Mermaid's Tears
- 主播大秀 R&D - Talking with Machines
- 主播大秀 R&D - Responsive Radio
-
Immersive and Interactive Content section
IIC section is a group of around 25 researchers, investigating ways of capturing and creating new kinds of audio-visual content, with a particular focus on immersion and interactivity.