VR gigs point to the future of live music | Industry Trends
One of the leaders in the field, MelodyVR, has partnered with Live Nation to deliver 360 ° virtual reality gigs from London’s iconic O2 Academy Brixton.
The shows feature 90-minute appearances by Tom Grennan, Frank Carter & The Rattlesnakes, Burner Boy, Fontaines DC, Kaiser Chiefs and Blossoms, with fans being able to choose from multiple camera angles – including on stage – that can be viewed via the MelodyVR app Smartphones and VR headphones can be viewed.
“We want the end user to be able to direct their own live music video by being able to switch between camera positions,” said Matthew Williams, MelodyVR’s technical director of production. “It has never been more important to connect fans with the artists they love using immersive video than now that they are prevented from attending in person.”
Five 360º virtual reality camera rigs designed and built in-house with multiple Sony 4K sensors were placed in front of the house, to the side of the stage and on stage with the drummer and keyboards (or saxophone or bass guitar, depending on the Act) and the lead singer at the O2 Academy. A director’s cut offers a sixth feed that can be selected by the viewer.
As the signal leaves each camera, it travels over an extensive network made up of multiple Blackmagic Design devices. Banks of more than 30 Teranex Minis convert HDMI, SDI and fiber with all 4K feeds managed by three Smart Videohub 12G 40×40 that act as the main matrix for production.
Each of the 4K lens feeds are routed to PCs containing DeckLink cards (4K and 8K) for stitching, playback and encoding of the high definition 360 ° video. The stitching software comes from the Swedish developer Voysys.
Williams explains, “The stitched 360-degree video is a 16×9 rectangular image with a top and bottom added. The viewer effectively sticks his head into a 3D video sphere. “
The stitching is processed in just 400ms and the feed switching via an ATEM Constellation 8K, which also generates four multiviews for the MelodyVR production team on site and remotely.
vMix is the software solution for live video production with TouchDesigner for graphic overlay and VFX animation. The live streams are fed into the cloud by vMix, where MelodyVR has its own user-defined cloud distribution network. It is capable of streaming at multiple bitrates to end users with tickets accessing the MelodyVR app or web player.
All feeds from the live shows, including individual lenses and equirectangular outputs, are recorded in Blackmagic Design HyperDeck Studio 12G in ProRes HQ.
“This will give us the highest bit rate and the greatest scope for grading in the future,” says Williams. “Should we have to go back and put everything back together, we can go straight to the source and do that.”
As with an audience concert, there are IMAG screens with 2D live content of the show captured by a variety of Blackmagic PCC 6K, Micro Studio cameras, URSA Mini Pro 4.6K and PTZ robotic cameras that do not show operators can bring the stage.
“360 immersive video is something that users haven’t had much experience with before, and where they have it wasn’t necessarily the most reliable or enjoyable experience with out of sync audio and / or a technical problem,” says Williams. “It’s really important for us to focus on the reliability of the stream.
“We do several dress rehearsals before going live and several internal tests for different types of lighting and LED screens. We’re constantly moving cameras into positions we’ve never tried before. “
Due to the social distancing restrictions, this set up was harder than usual. The producers had to reduce the number of crews they would normally have on site.
“Most of what we call our ‘war room’ – our operating room – is out of the way,” says Williams. “All of our seams are remote. We only have one core team on site. We had to take it back and use multi-skills to be as far away as possible. Our shelving systems are set up in such a way that social distancing is possible. All of our operators are two meters apart. We have a clear separation on site so that artists never have to come into contact with the crew. We use QR codes at the entrances. We log everyone, we trace and trace. PPE is mandatory. “
“We do several dress rehearsals before going live and several internal tests for different types of lighting and LED screens. We’re constantly putting cameras in positions we’ve never tried before, ”Matthew Williams, MelodyVR
He continues, “Remote workflows have put much more pressure on our internet lines. We have three lines in the academy; one satellite connection and two hardwired from different providers, so that we have a backup immediately in case of problems.
“We have several operators working remotely on these shows, some as far away as in the US, others in Hastings and the rest of the UK.”
Virtual stage invasion
Together with Frank Carter and his team, MelodyVR and Innovate Live developed a “stage invasion” in which selected fans had the opportunity to virtually stand on stage with the artist.
“Imagine a zoom call with 150 fans on a huge multiview grid displayed on LED screens in the back of the stage,” says Williams. “At home there were people in front of the webcam. We could see them dancing, with their pets, drinking, playing the guitar and having fun.
“On any other route, we could choose someone who had a ‘hot seat’. The idea was to have someone who could show up in the middle of the screen and have a face-to-face conversation with Frank. They could request songs. So it was a massive zoom call on stage. This even once included Frank Carter’s mother on screen.
“We had to configure the audio delays between the home users and Frank so they could interact properly and keep everything in sync. It took a lot of trial and error, but we managed to pull it off for the show. “
Each of the six “jump points” or camera positions that can be selected by the viewer is streamed in 4K. MelodyVR develops the tile coding so that only the part of the stream that the viewer sees is streamed.
“We want the lowest bit rate with the highest user experience so that users with the lowest internet bandwidth can access 4K content and beyond,” says Williams. “We’re trying to push the boundaries of what is possible with immersive videos.”
It explores depth mapping and six degrees of freedom (6DoF) – where the user can virtually walk across the stage during a live performance.
“We want the lowest bit rate with the highest user experience so that the users with the lowest internet bandwidth can access 4K content and beyond. We’re trying to push the boundaries of what’s possible with immersive video, ”Matthew Williams, MelodyVR
Elements of the workflow will allow them to produce 8K VR across the board. “When our camera systems upgrade to 8K resolution, we will already have 8K-compatible hardware.”
The reviews for the shows that took place before Christmas were decent. “While seeing the band in person never does justice to the raw energy,” wrote one, “the set has been an excellent placeholder until we can all get back into the pit.”
MelodyVR records Spotify
MelodyVR produced a three-day festival back in July that delivered 130,000 live music streams from Alexandra Palace. Over 100,000 live streams were recorded during a virtual reality broadcast of Good Morning America’s Summer Concert Series in September.
The five-year-old AIM-listed company has a market value of £ 74 million and claims to be the only licensed VR music platform. It also claims to own the world’s largest library of immersive and interactive music experiences (around 1,800 tracks) by Post Malone, Kelly Clarkson, Fall Out Boy, and The Chainsmokers, and has partnerships with O2 Telefónica and Facebook.
Continue reading Shoot the big weekend of Radio 1
That was now compounded by last August’s $ 70 million acquisition of Rhapsody International, the US company that owns the original groundbreaking digital music brand Napster. Rhapsody recorded 10.8 billion streams in 2019 with revenue of $ 113 million based on an ad-free model that charges users $ 9.99 per month for its service. It has 3 million users worldwide and 90 million licensed titles in its catalog.
It’s a move that puts MelodyVR in a more direct competition with Spotify and Apple iTunes.
“MelodyVR’s acquisition of Napster will lead to the development of the very first music entertainment platform that combines immersive visual content and music streaming,” said Anthony Matchett, CEO and Executive Chairman of MelodyVR, in a statement. “Our acquisition of Napster, one of the original disruptors of the music industry, was born out of our desire to deliver the world’s leading music experience, seamlessly available through audio and visual media, and a true next-generation music service.”
As with sports, streaming live appearances in VR effectively removes the capacity constraints for any concert or venue and allows an unlimited number of fans to purchase VR tickets for an otherwise sold out show. VR could prove to be a supplementary source of income to the live experience if gig visitors were allowed to mosh again.
Continue reading The outlook for virtual reality after the pandemic