Unreal Engine 4 Virtual Reality Projects Build immersive, real-world VR applications using UE4, C++, and Unreal Blueprints

Kevin Mack and Robert Ruud FOR SALE IN INDIA ONLY

www.packt.com

Unreal Engine 4 Virtual Reality Projects

Build immersive, real-world VR applications using UE4, C++, and Unreal Blueprints

Kevin Mack Robert Ruud

BIRMINGHAM - MUMBAI

Unreal Engine 4 Virtual Reality Projects Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Kunal Chaudhari Acquisition Editor: Karan Gupta Content Development Editor: Arun Nadar Technical Editor: Rutuja Vaze Copy Editor: Safis Editing Project Coordinator: Kinjal Bari Proofreader: Safis Editing Indexer: Priyanka Dhadke Graphics: Alishon Mendonsa Production Coordinator: Arvindkumar Gupta First published: April 2019 Production reference: 1300419 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78913-287-8

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content

Packt.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors About the authors Kevin Mack is a co-founder of Manic Machine, a Los Angeles-based development studio specializing in VR and virtual production development using Unreal Engine. Manic Machine designs and builds games in VR and provides development services to clients and partners in the film and visual effects industries. Prior to this, he co-founded WhiteMoon Dreams, which developed traditional and VR games and experiences. Earlier work includes design on the Medal of Honor series for EA, Fear Effect series for Kronos Digital Entertainment, and several titles for Disney Interactive. Kevin holds a BFA in film production from New York University and an MFA in film directing from the American Film Institute. I am deeply grateful to Lorrie for her endless patience and understanding as I spent numerous days and evenings glued to a screen or inside a VR headset. I would like to thank my parents as well for encouraging me always to do what I love and supporting the journey. And for that TRS-80 Color Computer that started it all. Finally, thank you Rob for joining me on this bizarre adventure and being such a fantastic partner through it.

Robert Ruud is a co-founder of Manic Machine, where he focuses primarily on the design and development of Manic Machine's proprietary tech and gameplay experiences. Prior to this, he spent six years at Whitemoon Dreams, where he designed and engineered gameplay for the successfully kickstarted game, Warmachine: Tactics, which was one of the first games to be released to market using Unreal Engine 4, and where he also led the design exploration for the company's location-based VR experiences. Robert holds a BA in philosophy from California State Polytechnic University, Pomona, where his studies focused on cognitive science and philosophy of the mind. I would like to thank my beautiful, intelligent, and caring girlfriend Hannah for being so incredibly supportive throughout this entire process and life in general. My parents for always believing in me and helping me however they could. My friends for everything they have taught me and the adventures they have joined me in. Finally, I would like to thank Kevin for being an astounding business partner as we explore this new and wonderful medium.

About the reviewer Deepak Jadhav is a game developer based in Pune, India. Deepak holds a bachelor's degree in computer technology and a master's degree in game programming and project management. Currently, he is working as a game developer at a leading game development company in India. He has been involved in developing games on multiple platforms, such as PC, macOS, and mobile. With years of experience in game development, he has a strong background in C# and C++, and has also refined his skills in platforms including Unity, Unreal Engine, Augmented and Virtual Reality. I would like to thank the authors, as well as the Packt Publishing team, for giving me the opportunity to review this book.

Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents Preface Chapter 1: Thinking in VR What is virtual reality?

1

9 10 VR hardware 11 VR isn't just about hardware though 12 Presence is tough to achieve 14 What can we do in VR? 15 Games in VR 16 Interactive VR 17 VR cinema – movies, documentary, and journalism 18 Architecture, Engineering, and Construction (AEC) and real estate 20 Engineering and design 21 Education and training 21 Commerce, advertising, and retail 22 Medicine and mental health 23 So much else 23 Immersion and presence 23 Immersion 24 Using all the senses 24 Make sure sensory inputs match one another and match the user's expectations 24 Keep latency as low as possible 25 Make sure interactions with the world make sense 26 Build a consistent world 27 Be careful of contradicting the user's body awareness 28 Decide how immersive you intend your application to be and design accordingly 28 Presence 29 Simulator sickness 29 Safety 30 Best practices for VR 31 Maintain framerate 31 Tethered headsets 31 Standalone Headsets 32 Never take control of the user's head 32 Do not put acceleration or deceleration on your camera 34 Do not override the field of view, manipulate depth of field, or use motion blur 34 Minimize vection 35 Avoid stairs 36 Use more dimmer lights and colors than you normally would 37 Keep the scale of the world accurate 37 Be conscious of physical actions 38

Table of Contents

Manage eyestrain Make conscious choices about the content and intensity of your experience Let players manage their own session duration Keep load times short Question everything we just told you

Planning your VR project

Clarify what you're trying to do Is it a good fit for VR? Why? What's important – what has to exist in this project for it to work? (MVP) Break it down Tackle things in the right order Test early and often Design is iterative

Summary Chapter 2: Setting Up Your Development Environment Prerequisite – VR hardware Setting up Unreal Engine What it costs Creating an Epic Games account The Epic Games launcher Installing the engine Editting your vault cache location Setting up a Derived Data Cache (DDC) Setting up a local DDC

Launching the engine

Setting up for mobile VR

Creating or joining an Oculus developer organization Setting your VR headset to developer mode in Oculus Go Installing Android Debug Bridge (ADB) Setting up NVIDIA CodeWorks for Android Verifying that the HMD can communicate with your PC Generating a signature file for Samsung Gear Deploying a test project to the device Setting up a test project

Checking that your OculusVR plugin is enabled Setting a default map Clearing the default mobile touch interface Setting your Android SDK project settings Setting your Android SDK locations

Launching the test project

Using the Epic Games launcher The Unreal Engine Tab Learn

The content examples project Gameplay concepts and example games

Marketplace

[ ii ]

38 41 41 41 42 42 42 43 44 45 46 47 48 48 51 52 52 52 53 53 55 56 57 58 60 63 64 64 64 65 65 66 67 67 68 68 68 68 70 70 72 73 73 75 78 78

Table of Contents

Library

Setting up for C++ development

Installing Microsoft Visual Studio Community Recommended settings The UnrealVS plugin Installing the UnrealVS plugin Turning on the UnrealVS toolbar

Unreal debugging support Test everything out

Building Unreal from source code

Setting up a GitHub account and installing Git

Setting up or logging into your GitHub account Installing Git for Windows Installing Git Large File Storage Installing a Git GUI Connecting your GitHub account to your Epic Games account

Downloading the Unreal Engine source code Choosing your source branch Forking the repository Cloning the repository to your local machine Option 1 – Cloning using GitHub Desktop Option 2 – Cloning from the command line

Downloading engine binary content Generating project files

Opening and building the solution Updating your fork with new changes from Epic

Option – Using the command line to sync changes Setting the upstream repository Syncing the fork Reviewing the Git commands we just used

Option – Using the web GUI to sync changes Creating a pull request Merging the pull request Pulling the origin to your local machine

Re-synchronizing your engine content and regenerating project files Going further with source code on GitHub

Additional useful tools

A good robust text editor 3D modeling software Image-editing software Audio-editing software

Summary Chapter 3: Hello World - Your First VR Project Creating a new project Setting your hardware target Setting your graphics target Settings summary

[ iii ]

79 81 82 83 84 84 85 85 86 93 93 93 94 95 95 96 97 98 98 99 99 100 100 100 100 102 102 103 104 104 105 107 108 108 109 110 110 110 111 113 113 114 115 116 117 118 118

Table of Contents

Taking a quick look at your project's structure The Content directory The Config directory The Source directory The Project file A summary of an Unreal project structure

Setting your project's settings for VR Instanced Stereo Round Robin Occlusions Forward and deferred shading

Choosing the right rendering method for your project

Choosing your anti-aliasing method Modifying MSAA settings

Starting in VR Turning off other stray settings you don't need Turning off default touch interface (Oculus Go/Samsung Gear) Configuring your project for Android (Oculus Go/Samsung Gear) Verifying your SDK locations

Making sure Mobile HDR is turned off (Oculus Go/Samsung Gear) Mobile Multi-View (Oculus Go/Samsung Gear) Monoscopic Far Field Rendering (Oculus Go / Samsung Gear) Project Settings cheat-sheet

Decorating our project

Migrating content into a project Cleaning up migrated content

Deleting assets safely Moving assets and fixing up redirectors

Setting a default map Testing our map on desktop Testing our map on mobile (Oculus Go/Samsung Gear)

Setting up a game mode and player pawn Creating a VR pawn Creating a game mode

Assigning the game mode Overriding a GameMode for a specific map

Placing a pawn directly in the world Setting up the VR pawn Adding a camera Adding motion controllers Setting our tracking origin.

Adjusting our Player Start location to the map. Testing in the headset. Packaging a standalone build

Summary Chapter 4: Getting Around the Virtual World Teleport locomotion Creating a navigation mesh

Moving and scaling the Navmesh Bounds volume Fixing collision problems

[ iv ]

119 119 120 121 121 121 122 123 124 125 128 129 132 133 134 134 135 136 136 136 137 138 138 139 142 143 143 148 148 149 151 151 154 155 156 158 159 159 160 162 163 163 164 165 167 168 168 170 175

Table of Contents

Excluding areas from the navmesh Modifying your navmesh properties

Setting up the pawn Blueprint Iterative development Make it work Make it right Make it fast Do things in order

Setting up a line trace from the right motion controller Improving our Trace Hit Result Using navmesh data

Changing from line trace to parabolic trace

Drawing the curved path Drawing the endpoint after all the line segments have been drawn

Teleporting the player

Creating Input Mappings Caching our teleport destination Executing the teleport

Allowing the player to choose their landing orientation Mapping axis inputs Cleaning up our Tick event Using thumbstick input to orient the player Creating a teleport destination indicator Giving it a material Adding the teleport indicator to the pawn

Optimizing and refining our teleport

Displaying UI only when teleport input is pressed Creating a deadzone for our input Fading out and in on teleport

Teleport locomotion summary

Seamless locomotion

Setting up inputs for seamless locomotion Changing the pawn's parent class Fixing the collision component

Handling movement input

Fixing movement speed Letting the player look around without constantly steering Implementing snap-turning Setting up inputs for snap turning Executing the snap turn

Going further

Snap turn using analog input

Summary Chapter 5: Interacting with the Virtual World - Part I Starting a new project from existing work Migrating Blueprints to a new project Copying input bindings

Setting up new project to use the migrated game mode Additional project settings for VR

[v]

178 180 180 181 181 181 182 182 183 188 190 191 192 196 198 198 200 203 204 204 205 207 216 216 218 220 220 222 224 228 229 229 230 233 235 238 238 239 239 240 242 243 246

247 248 248 250 250 251

Table of Contents

Testing our migrated game mode and pawn Adding scenery Adding a NavMesh Testing the map

Creating hands

Migrating hand meshes and animations from the VR Template project Adding hand meshes to our motion controllers Creating a new Blueprint Actor class

Adding motion controller and mesh components Adding a Hand variable Using a Construction Script to handle updates to the Hand variable

Adding BP_VRHand child actor components to your pawn Fixing issues with Hand meshes Replacing references to our old motion controller components in blueprints Creating a function to get our hand mesh

Animating our hands

A quick word about access specifiers Calling our grab functions from the pawn

Creating new input action mappings Adding handlers for new action mappings

Implementing grab animations in the Hand blueprints

Creating an Animation Blueprint for the hand Creating a blend space for our hand animations Wiring the blend space into the animation blueprint Connecting the animation blueprint to our hand blueprint Creating a new enumerator for our grip Smoothing out our grip animation

Summary Chapter 6: Interacting with the Virtual World - Part II Creating an object we can pick up Creating a Blueprint Interface for pickup objects Implementing the Pickup and Drop functions Setting up VRHand to pick up objects Creating a function to find the nearest pickup object Calling Find Nearest Pickup Object on the Tick event Picking up an actor Releasing an actor Test grabbing and releasing

Fixing cube collision Letting players know when they can pick something up

Adding haptic feedback

Creating a Haptic Feedback Effect Curve Playing the haptic effect on command

Going further Summary Chapter 7: Creating User Interfaces in VR Getting started [ vi ]

251 252 252 254 254 255 256 257 257 258 260 262 264 266 268 277 278 278 279 280 283 284 286 288 291 292 296 297

299 299 302 304 306 306 311 312 314 316 317 318 321 322 324 326 327 329 330

Table of Contents

Creating a new Unreal project from an existing project

We’re not alone – adding an AI character

Migrating the third-person character blueprint Cleaning up the third-person character blueprint Examining the animation blueprint Creating a companion character subclass Adding a follow behavior to our companion character Examining the AI controller Improving the companion's follow behavior

Adding a UI indicator to the companion pawn Creating a UI widget using UMG Adding a UI widget to an actor Orienting the indicator widget to face the player Implementing the Align UI function Calling Align UI from the Tick event

Adding a new AI state to the companion pawn Implementing a simple AI state Indicating AI states using the UI indicator Using events to update, rather than polling Being careful of circular references

Ensuring that UI is updated when our state is changed

Adding an interactive UI

Adjusting the button colors Adding event handlers to our buttons Attaching the UI element to the player pawn Using widget interaction components

Sending input through widget interaction components Making a better pointer for our interaction component Creating an interaction beam material Creating an impact effect

Summary Chapter 8: Building the World and Optimizing for VR Setting up the project and collecting assets Migrating blueprints into the new project Verifying the migrated content

Using the VR editor

Entering and exiting VR Mode Navigating in VR Mode

Moving through the world Teleporting through the world Rotating the world Scaling the world Practicing movement

Modifying the world in VR Mode

Moving, rotating, and scaling objects Using both controllers to rotate and scale objects

[ vii ]

331 332 332 332 333 334 336 336 339 342 342 347 349 349 350 352 353 353 357 357 358 360 364 365 366 369 373 374 375 377 382 383 384 385 385 386 387 388 389 390 391 392 393 393 393 396

Table of Contents

Practicing moving objects

Composing a new scene in VR Mode Navigating the radial menu Gizmo Snapping Windows Edit Tools Modes Actions and System

Making changes to our scene

Optimizing scenes for VR

Testing your current performance Stat FPS

Determining your frame time budget

Warnings about performance profiling Stat unit Profiling the GPU Stat scenerendering Draw calls

Stat RHI Stat memory Optimization view modes CPU profiling Turning things on and off

Addressing frame rate problems Cleaning up Blueprint Tick events Managing skeletal animations Merging actors Using mesh LODs Static mesh instancing Nativizing Blueprints

Summary Chapter 9: Displaying Media in VR Setting up the project Playing movies in Unreal Engine

Understanding containers and codecs Finding a video file to test with Adding a video file to an Unreal project Creating a File Media Source asset Creating a Media Player Using Media Textures Testing your Media Player Adding video to an object in the world Using a media playback material Adding sound to our media playback Playing media

[ viii ]

397 397 399 400 400 401 404 405 406 406 406 411 411 411 412 412 414 417 420 422 422 424 425 426 428 428 428 429 429 431 433 433 435

437 438 438 438 440 441 442 444 446 447 448 448 449 451

Table of Contents

Going deeper with the playback material Adding additional controls to our video appearance

Displaying stereo video

Displaying half of the video Displaying a different half of the video to each eye Displaying over/under stereo video

Displaying 360 degree spherical media in VR Finding 360 degree video Creating a spherical movie screen

Playing stereoscopic 360 degree video Controlling your Media Player Creating a Media Manager Adding a Pause and Resume function Assigning events to a media player

Summary Chapter 10: Creating a Multiplayer Experience in VR Testing multiplayer sessions Testing multiplayer from the editor

Understanding the client-server model

The server Listen servers, dedicated dervers, and clients Listen servers Dedicated servers Clients

Testing multiplayer VR

Setting up our own test project Adding an environment Creating a network Game Mode Objects on the network

Server-only objects Server and client objects Server and owning client objects Owning client only objects

Creating our network game mode

Creating a network client HUD

Creating a widget for our HUD Adding a widget to our HUD

Network replication

Creating a replicated actor

Spawning an actor on the server only Replicating the actor to the client Replicating a variable Notifying clients that a value has changed using RepNotify

Creating network-aware pawns for multiplayer Adding a first-person pawn

Setting collision response presets

[ ix ]

454 457 459 461 463 465 466 467 467 469 471 472 474 476 478 479 480 480 484 487 489 489 490 490 491 492 493 493 494 496 497 497 498 498 500 501 504 505 506 507 508 509 511 513 513 514

Unreal Engine 4 Virtual Reality Projects Unreal Engine 4 (UE4) is a powerful tool for developing VR games and applications. With its visual scripting language, Blueprint, and built-in support for all major VR headsets, it's a perfect tool for designers, artists, and engineers to realize their visions in VR. This book will guide you step-by-step through a series of projects that teach essential concepts and techniques for VR development in UE4. You will begin by learning how to think about (and design for) VR and then proceed to set up a development environment. A series of practical projects follows, taking you through essential VR concepts. Through these exercises, you'll learn how to set up UE4 projects that run effectively in VR, how

to build player locomotion schemes, and how to use hand controllers to interact with the world. You'll then move on to create user interfaces in 3D space, use the editor's VR mode to build environments directly in VR, and profile/optimize worlds you've built. Finally, you'll explore more advanced topics, such as displaying stereo media in VR, networking in Unreal, and using plugins to extend the engine. Throughout, this book focuses on creating a deeper understanding of why the relevant tools and techniques work as they do, so you can use the techniques and concepts learned here as a springboard for further learning and exploration in VR.

Things you will learn: •

Understand design principles and concepts for building VR applications

•

Evaluate and solve performance problems in VR to maintain high frame rates

•

Set up your development environment with Unreal Blueprints and C++

•

Display mono and stereo videos in VR

•

Create a player character with several locomotion schemes

•

Extend Unreal Engine's capabilities using various plugins

www.packt.com

FOR SALE IN INDIA ONLY