Have you ever wondered how Star Wars Mandalorian was made? What the deal is with modern CGI and how the technology, traditionally centred around the motion picture and gaming industries, is crossing over into broadcast and commercial filmmaking? Well, the answers could all lie within Europe’s state-of-the-art Mixed Augmented Reality Studio, MARS, which promises to offer a ‘technology canvas’ for the filmmakers, entertainers, content designers and event organisers of tomorrow.

The ambitious project, named from the abbreviation of its purpose, i.e. Mixed Augmented Reality Studio, [MARS] opened its doors recently, demonstrating virtual 3D environments which are mapped onto an LED stage, allowing performers and/or products to be combined into a photo-real or imaginary virtual environment. All in real-time.

The studio blends state-of-the-art video game engines with camera-tracking technology and XR, [extended reality] to form an immersive and varied studio space. Natural light that’s produced by the LED panelled wall gives the ability to update any scene in real-time and the set itself gives performers a perfect visual reference to orient themselves. This lifts the technology into a new realm if compared with the alternative and more widely-known green screen method, where visual elements are combined during post-production.

The MARS studio space has been developed as the perfect setting to give filmmakers, broadcasters and those working across a multitude of creative disciplines, exceptional access to high-quality XR, MR, AR, along with Virtual Production studio facilities. 

To demonstrate the possibilities of shooting at MARS, British art historian, BBC presenter and BAFTA-nominated broadcaster, Dr James Fox worked closely with the team at the event and commercial, experiential gurus, The Experience Machine, and the technical and production teams at MARS. The resulting piece, entitled, ‘Masterpiece: Nighthawks’ positions James Fox within Edward Hopper’s Nighthawks, the well-known 1942 oil on canvas painting that depicts four people in a downtown diner late at night as viewed through the diner’s window. 

In the piece the 3D representation of Hopper’s painting was extracted as 3D geometry with intricate layers of texturing added; partly handmade, partly AI-generated, that would resemble the brush strokes and fine crafting of Edward Hopper’s unique painting style. Ultimately, the 3D scene was imported into the Unreal Engine, best known as a gaming engine, and rendered in real-time based on the position of the physical camera on the MARS stage.

Commenting on ‘Masterpiece’, Christopher Pearson, founder and director of The Experience Machine, Told PhotoBite: “Access to high-end XR platforms such as MARS alongside advances in game engine software has opened the opportunity to craft narrative journeys from an entirely new perspective; the inside of an artwork. Considering the painted canvas as an environment that a viewer can walk into has never been more possible. Working closely with Dr James Fox we have been able to reimagine how a painting could exist as a 3-dimensional world. This process has allowed the technology to reveal insights into the painting that have never been considered before. Our working practice focuses on the conception and creation of genuine digital and physical experiences. We approach this by merging real world phenomena with their digital counterparts, while engaging the participant within an immersive narrative journey. MARS supports this approach with the integration of real-world effects within the virtual world of Masterpiece, combining virtual and physical camera depth of field, dynamic environmental lighting and live properties controlled by the director via a mobile tablet. This unique stage allows us a more tactile connection with the scene and has given us a new perspective on directing within a virtual set.”

The technology that’s being demonstrated at the MARS studio could well provide many answers for filmmakers and broadcasters in a post-COVID-19 age. What the studio demonstrates is that visual content can be produced without the need to travel to remote locations and that work can be carried out while maintaining the social distancing practices that have become so important in our lives recently.

Watch MARS’ short intro video below.

See more HERE.