Video Transport powers a Unity-based virtual theatre project
Studio Biscoe is a creative studio run by visual and performance artist, director, producer, entrepreneur, designer and engineer Ian Biscoe. The studio creates artistic experiences across theatre, dance, music and installation bringing together audio & video streams, projection mapping, 3D engines, animation, analogue and digital electronics — often in transdisciplinary collaborations with musicians, composers, choreographers, dancers, stage directors & scientists.
On this particular occasion, Video Transport was used to connect remote participants — 200 university students working in groups during a week-long project — to a virtual theatre hosted in the cloud.
The project was created as a replacement for a planned physical activity due to the current pandemic.
Individual students working from home with different qualities of broadband connections used VT’s browser interface to stream VP9-encoded WebRTC feeds of their individual performances into the virtual theatre, as well as to receive streams from different virtual cameras. Selected students in each performance were provided with a separate Node.js application developed by Studio Biscoe, which allowed them to control lights and cameras in the virtual theatre.
On this occasion, we mostly used USB webcams for the web-based contribution, but one node used Blackmagic’s SDI Studio 4K cameras connected via DeckLink 4K PCIe cards to the client computer running the web contribution link.
Two Windows Server 2019 AWS G4 instances were deployed in the project: 32-core machines with Nvidia T4 GPUs. One machine was used for audio mixing, composition and synchronisation. The second machine hosted the virtual theatre in Unity. Other applications ingesting or publishing NDI streams in the solution included Resolume Arena, vMix, Newtek’s NDI Studio Monitor and NDI Virtual Input.
The VT Publisher, running in the cloud, was picking up an NDI stream and publishing it to all users of the web contribution link. Two copies of VT Receiver received up to 10 WebRTC streams each from the contribution links and published those as NDI streams on the two cloud-based servers.
We had two key requirements in this project: 1) a low-latency browser application that could be accessed by non-technical users to send and receive audio/video streams and for which the server application could ingest and publish those streams via NDI; 2) a tool that was easy to integrate as a component into our system architecture in a short time with minimum overhead.
Several alternatives were considered. vMix Call required the use of vMix as the NDI publisher on the server with too much GPU overhead for multiple instances. Both Sienna Cloud and BirdDog NDI Cloud required buying into a larger ecosystem when only specific components were needed.
We also considered developing our own WebRTC solution, but there wasn’t enough time; and, as an arts studio and system integrator, we prefer to source commercially available components and use those if the cost is not prohibitive. VT worked reliably, and for us it was the best solution to easily integrate with our workflow and current system architecture in the short time available for the project.