~ Joint research project, awarded an R&D grant from XR Network+, will see AI researchers from the university work with virtual production experts from the Salford facility ~
14 December 2023: dock10, the UK’s leading television facility, has teamed with the University of York for a joint research project to explore how AI can tackle a major challenge for programmes filming in virtual studios – simulating realistic light interactions.
The project, which brings together leading AI researchers from the university and virtual production experts from dock10, has been awarded an embedded R&D grant from XR Network+. Led by XR Stories at the University of York, the XR Network+ project provides funding and support to researchers working in virtual production technologies, facilitating collaboration between academia and industry.
It is currently difficult to achieve realistic lighting interactions for live or ‘as live’ TV productions – including sports programmes, game shows and music performances – that film in multi-camera green screen studios. For example, an actor will not automatically cast a shadow or pick up reflections when placed into a computer-generated world. These have to be added in post-production.
In single-camera film or drama productions, these effects can be achieved with an LED volume, but this technique is not suitable for multi-camera productions which need to capture multiple angles simultaneously.
dock10 and University of York’s research project explores how AI can help simulate accurate light interactions between physical actors and virtual environments in multi-camera virtual studios. Senior University of York academics and PhD students with a track record of research in AI and digital creativity will work on the project alongside dock10’s virtual studio experts who will provide industry expertise and production workflows.
The research project aims to lay the foundations for the creation of innovative AI-powered compositing tools that ultimately will be applicable in real-time, making them suitable for live or ‘as live’ productions.
Dr. Florian Block, R&D Lead, AI & Immersive at dock10 studios and a Reader in Digital Creativity at the University of York, said: “Lighting is one of the core elements of creative expression in any TV show, but the lighting options for virtual studio productions are currently extremely constrained. That’s what makes this foundational research project so exciting. We are aiming to create a proof of concept that will ultimately lead to the development of a bespoke AI tool for delivering realistic lighting interactions in virtual studio productions in real-time.”
Richard Wormwell, dock10 Head of Production Innovation, said: “dock10 is the UK leader in virtual studios production and the University of York is the UK leader for research into digital creativity. We are combining our industry and academic expertise to try to crack one of the great challenges for virtual studio productions. To do so, we are exploring innovative, AI driven solutions that have the potential to radically improve virtual production workflows. We are hugely excited to see where this research project might lead.”
About XR Network+
XR Network+ Virtual Production in the Digital Economy provides funding and support to researchers working in virtual production technologies. Through our networks of partners, we are setting the research agenda for virtual production (VP), supporting growth, and facilitating collaboration between academia and industry on a national stage.
XR Network+ is funded by the Engineering and Physical Sciences Research Council (EPSRC). It will establish a ten-year research agenda for VP related content creation and consumption for the creative and digital economies.
Led by XR Stories at the University of York, XR Network+ is conducted in collaboration with the University of Edinburgh, University of the Arts London, Cardiff University and Ulster University. Grant funding is administered by the University of York. https://xrnetworkplus.xrstories.co.uk/
About dock10
dock10 is the UK’s leading television facility where some of the nation’s best loved programmes are made. Based at the heart of MediaCityUK, our studios and specialist post production services are used to make everything from popular prime time shows to outstanding commercials and corporate films for leading brands. We offer broadcasters, production companies and creative agencies the world-class facilities they need to produce award-winning content. https://www.dock10.co.uk/
About The University of York
The University of York is at the forefront of digital creativities research. We’re exploring new worlds and new developments to understand where there is exciting new potential. Our projects share a common purpose – to ask challenging questions and explore how cutting-edge digital technology can support and develop creativity. Departments across all faculties are involved in high-profile, funded creative projects. Researchers at York have expertise in a number of areas including VR, AR, XR, AI and ethics, gaming, sound, immersive storytelling and virtual production. https://www.york.ac.uk/research/themes/creativity/digital-creativities/