Cinema 4D Makes Game Characters Part of Virtual Reality SEHSUCHT uses Cinema 4D and the Unreal engine to create one of the first-ever interactive VR music videos.

Virtual reality is a hot topic in 2016. Various hardware manufacturers are introducing headsets that let enthusiasts immerse themselves in 3D worlds. The games industry in particular is creating products specifically for use with head-mounted displays that give players the feeling of being part of the world in which they are playing.

Berlin, Germany-based, animation studio SEHSUCHT is at the forefront with its first VR project: For its single ‘Reminder’, the band Moderat had director Mate Steinforth produce a VR video that takes the viewer on a spectacular trip through a gloomy, sinister world.

What is unique about this project is that the interactive VR video runs with the Unreal engine, which means that head tracking by headsets such as Oculus Rift is supported. The viewer can move freely in 360° throughout their journey through the surreal, bleak gray landscape while experiencing the virtual world with their own eyes. The story is about two humanoid beings who have to mine crystals in back-breaking work that they then present to a divine being as an offering.

The song’s lyrics and the visual interpretation are a perfect amalgamation. Mate and his team decided to use an abstract look because of the song’s severity and the fact that it left room for political interpretation. The minimalistic graphics, which were developed together with SEHSUCHT and Moderat, were also easier for the small design team to handle. The monochrome textures were used to emphasize the gloomy environment. Numerous independent games that had been produced in the past served as inspiration for the design team.

The production of 3D assets for use in a game engine was a new challenge for SEHSUCHT. Since the Unreal engine renders the video in real-time, the scenes’ performance had to be optimized greatly. VR headsets place high demands on hardware and also require consistently high refresh rates of 75 to 120 fps in order to offer the viewer the best possible experience. When creating the 3D models in Cinema 4D, the SEHSUCHT team made sure to keep the scene’s complexity and polygon count to a minimum.
In addition to affecting the technical realization, the VR production also influenced the project’s design and conception, as Mate describes: “In short, VR means that you have to work without cuts and the viewers themselves determine how the scene is viewed. You can’t frame a scene as you usually would. The viewer’s attention has to be drawn by properly setting up the scene and giving the viewer more time to explore their environment.”

After SEHSUCHT had completed the video’s conceptual phase, they started designing the characters. The characters were modeled in Cinema 4D and loaded into the online auto-rigger Mixamo in whose animation library SEHSUCHT found the movement sequences they were looking for. The project required a video game feel and the Mixamo tool offered 80% of what the team was looking for as presets.

Cinema 4D’s character tools were used for fine-tuning. The animations were exported from Mixamo and converted to Motion Clips in Cinema 4D. These Motion Clips were then combined and mixed to create seamless transitions between the sequences. SEHSUCHT used the Motion Layers to animate details such as special hand gestures, which weren’t included in the motion capture clips. “The Motion Layers were a great help for the character animations because they made it possible to modify and relocate the motion capture clips like you would in a video editing program,” explained Mate. SEHSUCHT used Microsoft Kinect in conjunction with iPi Soft Motion Capture to create additional motion capture sequences.

Before the 3D assets could be exported from Cinema 4D to the Unreal engine, the Bake Object feature had to be applied so the animations would run smoothly in the game engine.

With their interactive VR music video, SEHSUCHT impressively demonstrates just how fascinating VR can be, which is why this project was selected to participate in the Kaleidoscope VR World Tour, which showcases the best VR experiences in cities around the world. This nomination shows that Cinema 4D is an ideal tool for creating 3D VR content!

Note: The interactive VR version for Oculus Rift is available for free at the wearvr website. Visit YouTube to see the non-interactive video version.

Credits
MODERAT „Reminder“
Produced by SEHSUCHT Berlin
Screenplay and direction: Mate Steinforth
Concept, co-art direction and graphics by PFADFINDEREI