Reimagining Characters with Unreal Engine's MetaHuman Creator: Elevate your films with cinema-quality character designs and motion capture animation

BIRMINGHAM—MUMBAI

Reimagining Characters with Unreal Engine’s MetaHuman Creator

Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Rohit Rajkumar

Publishing Product Manager: Vaideeshwari Muralikrishnan

Senior Editor: Hayden Edwards

Technical Editor: Simran Ali

Copy Editor: Safis Editing

Project Coordinator: Sonam Pandey

Proofreader: Safis Editing

Indexer: Manju Arasan

Production Designer: Joshua Misquitta

Marketing Coordinators: Teny Thomas and Nivedita Pandey

First published: January 2023

Production reference: 1071222

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80181-772-1

www.packt.com

To my mother, Gaby, for her relentless inspiration. To my wife, Kenia, and my two children, Eoin and Isabella, for their encouragement, support, and understanding. To all the folks at Media House for their never-ending motivation. And, last but not least, to all my great friends who have been checking in on the book’s progress and spoiling me with their encouraging words.

– Brian Rossney

Foreword

I have known and worked with Brian Rossney for the best part of a decade.

Our paths crossed when Brian and I worked on TV commercials, with Brian as the Director and myself as Director of Photography. What was unusual about Brian was that he had an innate ability to direct, that was equally complemented by a comprehensive grasp of all aspects of post-production and visual effects.

Brian’s hard-won experience and passion in this field goes back over 20 years. Primarily working in commercials and dramas, he has produced countless excellent visual effects, 3D environments, and digital doubles. These skills, along with the ability to transfer information about the subject to others, was quickly recognized, and it came as no surprise that he was hired to teach advanced video production at Dublin City University.

As a Director of Photography on feature films and episodic television, I’ve grown accustomed to seeing pre-visualization. But seeing what was possible with Brian’s guidance and the Unreal Engine, particularly in terms of lighting and animation, was an eye opener. On one film, Brian created pre-visualizations in 3D for every shot. Then he progressed to using the Unreal Engine for his own work, creating an impressive short film titled Apetito which maximized the technology.

Simultaneously ILM and Disney began utilizing the Unreal Engine for their The Mandalorian series!

The Unreal Engine is most definitely a filmmaker’s tool, with the required skills seemingly out of reach for most. However, Reimagining Characters with Unreal Engine’s MetaHuman Creator allows artists to animate realistic characters in an easy-to-follow fashion, without being bogged down by boring techno garble.

The book is designed for professionals and hobbyists who don’t have time to learn about all the technology behind the magic, but want to get to work on their projects quicky. This is why Brian’s book is an effective reference – it gradually introduces you to key concepts surrounding digital characters, along with how to use and navigate around Unreal Engine’s MetaHuman Creator.

On top of this, he collates the most efficient ways of working, particularly in terms of getting MetaHumans to work with Mixamo animation libraries or bespoke body motion capture through DeepMotion.

Users who have an iPhone can also create highly realistic facial capture to use on their characters, or alternatively can use their webcam and Faceware technology. Both methods are provided with step-by-step breakdowns of how they are accomplished.

Brian never intended to write the definitive guide to every aspect of MetaHumans or all things relating to Motion Capture. However, he did intend to write a book where readers can get results fast, without having to rely on multiple sources of information. Having completed all of the research for you, the outcome is within these pages.

So, get up and running with creating your own highly realistic digital characters and animations for your film project – something that, until now, was only imaginable for huge animation and film studios - without all the stress and heartache.

Happy filmmaking!

Ciaran Kavanagh

ISC IATSE 669

Contributors

About the author

Brian Rossney is a creative director and VFX supervisor at Rossney Pictures. With 22 years of experience working in the film and TV industry, Brian specializes in post-production and digital visual effects. He has produced commercials and TV opening sequences using 3D animation tools and 3D node-based compositing applications. With a thorough understanding of the VFX and 3D animation pipeline and live-action film producing and directing, Brian has lectured at Dublin City University and Galway Film Centre, and has worked as a VFX supervisor for Hallmark and Maravista. He is also the creator of two short films – they are called Apetito, using Unreal Engine, and 1976, using Unreal Engine and MetaHuman Creator.

I’d like to thank my very patient editor, Hayden Edwards.

About the reviewer

Emilio Ferrari is a motion capture supervisor and the CEO of Raised by Monsters (a preproduction studio based on Unreal Engine). He works closely with the Unreal Engine department to develop tools that facilitate a smooth transfer between the mocap stage and engine. In addition to managing the mocap stage, he also leads research and development efforts focused on finding ways to blend VFX, film, and Unreal Engine through playable tools.

Emilio is currently working on the development of the book Digital Filmmaking with Unreal Engine 5, which focuses on cinematography and storytelling with Unreal Engine.

Table of Contents

Preface

Part 1: Creating a Character

1

Getting Started with Unreal

Technical requirements

What is Unreal?

What are MetaHumans?

Setting up Unreal and the MetaHuman Creator

Creating an Epic account

Downloading and installing Unreal Engine 5

Launching UE5

Installing Quixel Bridge

Booting up MetaHuman Creator

Summary

2

Creating Characters in the MetaHuman Interface

Technical requirements

Starting up Quixel Bridge

Editing your MetaHuman

Face

Hair

Body

Using the Move and Sculpt tools

Downloading and exporting your character

Resolution

Download considerations

Export considerations

Processing in the background

Summary

Part 2: Exploring Blueprints, Body Motion Capture, and Retargeting

3

Diving into the MetaHuman Blueprint

Technical requirements

What are Blueprints?

Opening a Blueprint

Using the folder structure to navigate around the Blueprint

Navigating around the Blueprint without using folders

Importing and editing skeletons

Adding the Mannequin character

Editing the Mannequin skeleton's retargeting options

Editing the MetaHuman skeleton's retargeting options

Summary

4

Retargeting Animations

Technical requirements

What is an IK Rig?

What is a rig?

What is IK?

Creating an IK Rig

Creating the IK chains

Creating an IK Retargeter

Importing more animation data

Summary

5

Retargeting Animations with Mixamo

Technical requirements

Introducing Mixamo

Preparing and uploading the MetaHuman to Mixamo

Orienting your character in Mixamo

Exploring animation in Mixamo

Downloading the Mixamo animation

Importing the Mixamo animation into Unreal

Working with subsequent animations

Summary

6

Adding Motion Capture with DeepMotion

Technical requirements

Introducing DeepMotion

Preparing our video footage

Uploading our video to DeepMotion

Exploring DeepMotion’s animation settings

Downloading the DeepMotion motion capture file

Importing the DeepMotion animation into Unreal

Retargeting the DeepMotion motion capture

Fixing position misalignment issues

Summary

Part 3: Exploring the Level Sequencer, Facial Motion Capture, and Rendering

7

Using the Level Sequencer

Technical requirements

Introducing the Level Sequencer

Creating a Level Sequencer and importing our character Blueprint

Adding the retargeted animation to the character Blueprint

Adding and editing the Control Rig

Adding a camera to the Level Sequencer

Rendering a test animation from the Level Sequencer

Summary

8

Using an iPhone for Facial Motion Capture

Technical requirements

Installing the Live Link Face app

Installing Unreal Engine plugins (including Take Recorder)

Live Link, Live Link Control Rig, and Live Link Curve Debug UI

ARKit and ARKit Face Support

Take Recorder

Connecting and configuring the Live Link Face app to Unreal Engine

Configuring and testing the MetaHuman Blueprint

Calibrating and capturing live data

Summary

9

Using Faceware for Facial Motion Capture

Technical requirements

Installing Faceware Studio on a Windows PC

Installing Unreal’s Faceware plugin and the MetaHuman sample

Setting up a webcam and the streaming function

Realtime Setup panel

Streaming panel

Status bar

Enabling Live Link to receive Faceware data

Editing the MetaHuman Blueprint

Recording a session with a Take Recorder

Importing a take into the Level Sequencer

Baking and editing facial animation in Unreal

Summary

10

Blending Animations and Advanced Rendering with the Level Sequencer

Technical requirements

Adding the MetaHuman Blueprint and body mocap data to the Level Sequencer

Adding the MetaHuman Blueprint

Adding previously retargeted body mocap data

Adding additional body mocap data and merging mocap clips

Adding facial mocap data to the Level Sequencer

Adding a recorded Faceware take

Editing the facial mocap

Exploring advanced rendering features

Adding and animating a camera

Adding and animating light

Using Post Process Volumes

Using the Movie Render Queue

ACES and color grading

Summary

11

Using the Mesh to MetaHuman Plugin

Technical requirements

Installing the Mesh to MetaHuman plugin for Unreal Engine

Introducing and installing KIRI Engine on your smartphone

Importing your scanned mesh into Unreal Engine

Modifying the face mesh inside the MetaHuman Creator online tool

Summary

Index

Other Books You May Enjoy

Preface

Technology and art have always gone hand in hand. In recent years, we have seen technology evolve rapidly, and for many of us, the advancements are hard to keep track of. The technology behind movies is no different.

I grew up in the 1980s when all film technology was analog. An incredibly inspiring film for me was Clash of the Titans (1981). Learning that the visual effects animation was created by just one person, Ray Harryhausen, amazed me. Fast-forward to the 2000s, and there are now teams consisting of hundreds of visual effects artists and animators, with several companies working on one single film title.

With Unreal Engine and MetaHuman Creator, things have seemingly gone full circle. Gone are the days when cinema-quality animation and VFX tools were exclusive to large and wealthy studios. With a reasonably powered computer, even a laptop, people can now make their own films. There are no boundaries and no excuses. We are at the exciting dawn of a new era, where we will see new Ray Harryhausens emerge.

Who this book is for

This book is written by a filmmaker for filmmakers. To understand some of the more difficult concepts introduced in the book, you should have a reasonable understanding of the filmmaking process, particularly 3D animation. If you’ve dabbled with 3D character animation and have read at least one book on the subject, you should have no problem understanding anything written in the book. If you don’t have any prior learning, don’t worry – you can still progress to the end without getting stuck.

Ultimately, this is a step-by-step guide on how to create an animated MetaHuman with Unreal. It covers various techniques that you can apply to your own ideas and designs.

What this book covers

In Chapter 1, Getting Started with Unreal, you will learn what Unreal is, what a MetaHuman is, and how to set up Unreal Engine on your computer.

In Chapter 2, Creating Characters in the MetaHuman Interface, we will examine all the major features of the Metahuman Creator Interface, how to edit a MetaHuman, and also how to download and export your own MetaHuman.

In Chapter 3, Diving into the MetaHuman Blueprint, we will learn what a Blueprint is and how to use them in relation to MetaHumans.

In Chapter 4, Retargeting Animations, we will learn all about IK Rigs, IK Chains, and the IK Retargeter.

In Chapter 5, Retargeting Animations with Mixamo, we learn how to retarget animation with Mixamo animation assets.

In Chapter 6, Adding Motion Capture with DeepMotion, similar to the previous chapter, we will learn about markerless motion capture using DeepMotion and how we can retarget DeepMotion-acquired motion capture data onto a MetaHuman.

In Chapter 7, Using the Level Sequencer, we will learn about the Level Sequencer and how we can use it to control our MetaHuman character.

In Chapter 8, Using an iPhone for Facial Motion Capture, we will learn how to install the Live Link app and use an iPhone to capture facial capture live and directly into Unreal Engine.

In Chapter 9, Using Faceware for Facial Motion Capture, as an alternative to the last chapter, we will look at how we can use a simple webcam to get an equally high standard of facial motion capture utilizing the Faceware software.

In Chapter 10, Blending Animations and Advanced Rendering with the Level Sequencer, we will look at how we can use multiple motion capture clips inside a Level Sequencer. This chapter will also introduce some advanced rendering concepts using the Level Sequencer, such as Post Process Volumes.

In Chapter 11, Using the Mesh to MetaHuman Plugin, as a bonus, you will learn about a new plugin that allows users to adapt their own 3D scans for their MetaHumans to be based on.

To get the most out of this book

To get the most out of this book, a basic understanding of concepts from the 3D animation industry such as meshes, video timelines, motion capture, and keyframes is required. Some experience with 3D applications such as Maya, 3ds Max, Houdini, Blender, Cinema 4D, or Modo would be an advantage.

It is recommended that you familiarize yourself with the results of motion capture and 3D animation by watching behind-the-scenes clips from VFX movies such as Avatar, Pirates of the Caribbean, and The Avengers.

You will need a fast internet connection and a lot of hard drive space. A MetaHuman character when downloaded will easily take up about 4 GB of hard drive space, and an Unreal Engine project with a simple MetaHuman scene could take up to around 50 GB of hard drive space. Additionally, a high-spec graphics card such as an NVIDIA RTX 3000 series is recommended.

Also, take a look at the writer’s YouTube channel for tips and tricks regarding all aspects of Unreal Engine and MetaHumans relating to this book as soon as they become available: https://bit.ly/3Fv9u18.

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/5RXwo.

Conventions used

Bold: Indicates a new term, an important word, or words that you see on screen. For instance, words in menus or dialog boxes appear in bold. Here is an example: Select System info from the Administration panel.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com

Share your thoughts

Once you’ve read Reimagining Characters with Unreal Engine’s MetaHuman Creator, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily!

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below:

https://packt.link/free-ebook/9781801817721

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Creating a Character

In Part 1 of this book, you will learn about the key prerequisites you need to know, such as installing Unreal Engine and Quixel Bridge. Then, you will learn all there is to know about the MetaHuman Creator, including how to create your character and how to bring it into Unreal Engine.

This part includes the following chapters:

Chapter 1, Getting Started with Unreal

Chapter 2, Creating Characters in the MetaHuman Interface

1

Getting Started with Unreal

In the first chapter of this book, you will learn about what Unreal Engine is, what it is used for, and how you can start working with it. You will then learn how to set up an account with Epic Games and download Unreal Engine, before using the same account details to download Bridge, which is the application needed to start creating MetaHuman characters.

Overall, you will become proficient in installing and setting up these applications on a Windows machine.

So, we will cover the following topics:

What is Unreal Engine?

What are MetaHumans?

Setting up Unreal Engine and the MetaHuman Creator

Technical requirements

Before we get started, I must remind you that this book is about extremely high-quality 3D characters so it’s important you understand this before setting up Unreal and risk being disappointed because of technical issues. This is because Unreal Engine, like any 3D graphics tool, requires a powerful