Final Pixel Publishes In-depth R&D Case Study on the Use of Mocap in Virtual Production

Final Pixel, the global creative studio specializing in end-to-end virtual production for film, TV and advertising, has published an in-depth case study on the use of motion capture in virtual production. The case study, “At the edge of the Metaverse: Live Body and Facial Motion Capture for LED Wall Virtual Production, with Rendering of High-Quality Digital Characters in Real-time”, is available for free download from our website. A showreel summarizing the project can be watched here.

The research project was devised by Final Pixel and incorporated live-action body and facial motion capture of a detailed creature animation into Final Pixel’s current successful virtual production workflow. This industry-first enabled the team at Final Pixel to understand the limits of the software and workflow to give future clients the opportunity to incorporate detailed motion capture digital characters in their virtual productions.

Michael McKenna, CEO of Final Pixel, said: “As a company specialising in virtual production for film, tv and advertising, we are excited by the opportunities working in real-time game engines can provide for the creative process when everything can be captured in-camera while shooting live-action.  The next evolution of this technology is to look at the elements which are still considered too heavy or complex to move out of the post production workflow.  Digital characters and creature work is a big area for this, and also extremely important for storytelling narrative. The recent Unreal Engine 5 release from Epic games showcasing Keanu Reeves in the Matrix is a great example of the powerful technology we can now use in virtual production.  We are excited to share our findings with the rest of the industry to help us collectively move the use of virtual production forward.”

The team successfully achieved live facial and body motion capture streamed to Unreal Engine and played through Disguise, using cluster rendering to render a high-quality bespoke 3D character built using a traditional CG pipeline with an extremely high level of detail. The team was able to create real-time interactions between the characters in-camera with no noticeable latency for the viewer.

Potential uses of this approach are many and significant:

• The virtual production pipeline for film, TV and advertising allows for live interactions between digital and human characters, all filmed in real-time and in-camera.
• Live-action mocap with creatures and characters which can then be replaced by full-scale CG in post - thereby capturing more ‘natural’ actor reactions and engagement verses use of green screen.
• To create increased fidelity augmented reality plates, in particular for live broadcasts and when using the enhanced stage management provided by Disguise.
  

The project was shot at the Digital Catapult’s Virtual Production Test Stage (VPTS), a joint venture with Target3D. More information on the facility can be found at https://www.digicatapult.org.uk.