Unreal Engine 4 Virtual Reality Projects Build immersive, real-world VR applications using UE4, C++, and Unreal Blueprints
Kevin Mack and Robert Ruud FOR SALE IN INDIA ONLY
www.packt.com
Unreal Engine 4 Virtual Reality Projects
Build immersive, real-world VR applications using UE4, C++, and Unreal Blueprints
Kevin Mack Robert Ruud
BIRMINGHAM - MUMBAI
Unreal Engine 4 Virtual Reality Projects Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Kunal Chaudhari Acquisition Editor: Karan Gupta Content Development Editor: Arun Nadar Technical Editor: Rutuja Vaze Copy Editor: Safis Editing Project Coordinator: Kinjal Bari Proofreader: Safis Editing Indexer: Priyanka Dhadke Graphics: Alishon Mendonsa Production Coordinator: Arvindkumar Gupta First published: April 2019 Production reference: 1300419 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78913-287-8
www.packtpub.com
mapt.io
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content
Packt.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at
[email protected] for more details. At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors About the authors Kevin Mack is a co-founder of Manic Machine, a Los Angeles-based development studio specializing in VR and virtual production development using Unreal Engine. Manic Machine designs and builds games in VR and provides development services to clients and partners in the film and visual effects industries. Prior to this, he co-founded WhiteMoon Dreams, which developed traditional and VR games and experiences. Earlier work includes design on the Medal of Honor series for EA, Fear Effect series for Kronos Digital Entertainment, and several titles for Disney Interactive. Kevin holds a BFA in film production from New York University and an MFA in film directing from the American Film Institute. I am deeply grateful to Lorrie for her endless patience and understanding as I spent numerous days and evenings glued to a screen or inside a VR headset. I would like to thank my parents as well for encouraging me always to do what I love and supporting the journey. And for that TRS-80 Color Computer that started it all. Finally, thank you Rob for joining me on this bizarre adventure and being such a fantastic partner through it.
Robert Ruud is a co-founder of Manic Machine, where he focuses primarily on the design and development of Manic Machine's proprietary tech and gameplay experiences. Prior to this, he spent six years at Whitemoon Dreams, where he designed and engineered gameplay for the successfully kickstarted game, Warmachine: Tactics, which was one of the first games to be released to market using Unreal Engine 4, and where he also led the design exploration for the company's location-based VR experiences. Robert holds a BA in philosophy from California State Polytechnic University, Pomona, where his studies focused on cognitive science and philosophy of the mind. I would like to thank my beautiful, intelligent, and caring girlfriend Hannah for being so incredibly supportive throughout this entire process and life in general. My parents for always believing in me and helping me however they could. My friends for everything they have taught me and the adventures they have joined me in. Finally, I would like to thank Kevin for being an astounding business partner as we explore this new and wonderful medium.
About the reviewer Deepak Jadhav is a game developer based in Pune, India. Deepak holds a bachelor's degree in computer technology and a master's degree in game programming and project management. Currently, he is working as a game developer at a leading game development company in India. He has been involved in developing games on multiple platforms, such as PC, macOS, and mobile. With years of experience in game development, he has a strong background in C# and C++, and has also refined his skills in platforms including Unity, Unreal Engine, Augmented and Virtual Reality. I would like to thank the authors, as well as the Packt Publishing team, for giving me the opportunity to review this book.
Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents Preface Chapter 1: Thinking in VR What is virtual reality?
1
9 10 VR hardware 11 VR isn't just about hardware though 12 Presence is tough to achieve 14 What can we do in VR? 15 Games in VR 16 Interactive VR 17 VR cinema – movies, documentary, and journalism 18 Architecture, Engineering, and Construction (AEC) and real estate 20 Engineering and design 21 Education and training 21 Commerce, advertising, and retail 22 Medicine and mental health 23 So much else 23 Immersion and presence 23 Immersion 24 Using all the senses 24 Make sure sensory inputs match one another and match the user's expectations 24 Keep latency as low as possible 25 Make sure interactions with the world make sense 26 Build a consistent world 27 Be careful of contradicting the user's body awareness 28 Decide how immersive you intend your application to be and design accordingly 28 Presence 29 Simulator sickness 29 Safety 30 Best practices for VR 31 Maintain framerate 31 Tethered headsets 31 Standalone Headsets 32 Never take control of the user's head 32 Do not put acceleration or deceleration on your camera 34 Do not override the field of view, manipulate depth of field, or use motion blur 34 Minimize vection 35 Avoid stairs 36 Use more dimmer lights and colors than you normally would 37 Keep the scale of the world accurate 37 Be conscious of physical actions 38
Table of Contents
Manage eyestrain Make conscious choices about the content and intensity of your experience Let players manage their own session duration Keep load times short Question everything we just told you
Planning your VR project
Clarify what you're trying to do Is it a good fit for VR? Why? What's important – what has to exist in this project for it to work? (MVP) Break it down Tackle things in the right order Test early and often Design is iterative
Summary Chapter 2: Setting Up Your Development Environment Prerequisite – VR hardware Setting up Unreal Engine What it costs Creating an Epic Games account The Epic Games launcher Installing the engine Editting your vault cache location Setting up a Derived Data Cache (DDC) Setting up a local DDC
Launching the engine
Setting up for mobile VR
Creating or joining an Oculus developer organization Setting your VR headset to developer mode in Oculus Go Installing Android Debug Bridge (ADB) Setting up NVIDIA CodeWorks for Android Verifying that the HMD can communicate with your PC Generating a signature file for Samsung Gear Deploying a test project to the device Setting up a test project
Checking that your OculusVR plugin is enabled Setting a default map Clearing the default mobile touch interface Setting your Android SDK project settings Setting your Android SDK locations
Launching the test project
Using the Epic Games launcher The Unreal Engine Tab Learn
The content examples project Gameplay concepts and example games
Marketplace
[ ii ]
38 41 41 41 42 42 42 43 44 45 46 47 48 48 51 52 52 52 53 53 55 56 57 58 60 63 64 64 64 65 65 66 67 67 68 68 68 68 70 70 72 73 73 75 78 78
Table of Contents
Library
Setting up for C++ development
Installing Microsoft Visual Studio Community Recommended settings The UnrealVS plugin Installing the UnrealVS plugin Turning on the UnrealVS toolbar
Unreal debugging support Test everything out
Building Unreal from source code
Setting up a GitHub account and installing Git
Setting up or logging into your GitHub account Installing Git for Windows Installing Git Large File Storage Installing a Git GUI Connecting your GitHub account to your Epic Games account
Downloading the Unreal Engine source code Choosing your source branch Forking the repository Cloning the repository to your local machine Option 1 – Cloning using GitHub Desktop Option 2 – Cloning from the command line
Downloading engine binary content Generating project files
Opening and building the solution Updating your fork with new changes from Epic
Option – Using the command line to sync changes Setting the upstream repository Syncing the fork Reviewing the Git commands we just used
Option – Using the web GUI to sync changes Creating a pull request Merging the pull request Pulling the origin to your local machine
Re-synchronizing your engine content and regenerating project files Going further with source code on GitHub
Additional useful tools
A good robust text editor 3D modeling software Image-editing software Audio-editing software
Summary Chapter 3: Hello World - Your First VR Project Creating a new project Setting your hardware target Setting your graphics target Settings summary
[ iii ]
79 81 82 83 84 84 85 85 86 93 93 93 94 95 95 96 97 98 98 99 99 100 100 100 100 102 102 103 104 104 105 107 108 108 109 110 110 110 111 113 113 114 115 116 117 118 118
Table of Contents
Taking a quick look at your project's structure The Content directory The Config directory The Source directory The Project file A summary of an Unreal project structure
Setting your project's settings for VR Instanced Stereo Round Robin Occlusions Forward and deferred shading
Choosing the right rendering method for your project
Choosing your anti-aliasing method Modifying MSAA settings
Starting in VR Turning off other stray settings you don't need Turning off default touch interface (Oculus Go/Samsung Gear) Configuring your project for Android (Oculus Go/Samsung Gear) Verifying your SDK locations
Making sure Mobile HDR is turned off (Oculus Go/Samsung Gear) Mobile Multi-View (Oculus Go/Samsung Gear) Monoscopic Far Field Rendering (Oculus Go / Samsung Gear) Project Settings cheat-sheet
Decorating our project
Migrating content into a project Cleaning up migrated content
Deleting assets safely Moving assets and fixing up redirectors
Setting a default map Testing our map on desktop Testing our map on mobile (Oculus Go/Samsung Gear)
Setting up a game mode and player pawn Creating a VR pawn Creating a game mode
Assigning the game mode Overriding a GameMode for a specific map
Placing a pawn directly in the world Setting up the VR pawn Adding a camera Adding motion controllers Setting our tracking origin.
Adjusting our Player Start location to the map. Testing in the headset. Packaging a standalone build
Summary Chapter 4: Getting Around the Virtual World Teleport locomotion Creating a navigation mesh
Moving and scaling the Navmesh Bounds volume Fixing collision problems
[ iv ]
119 119 120 121 121 121 122 123 124 125 128 129 132 133 134 134 135 136 136 136 137 138 138 139 142 143 143 148 148 149 151 151 154 155 156 158 159 159 160 162 163 163 164 165 167 168 168 170 175
Table of Contents
Excluding areas from the navmesh Modifying your navmesh properties
Setting up the pawn Blueprint Iterative development Make it work Make it right Make it fast Do things in order
Setting up a line trace from the right motion controller Improving our Trace Hit Result Using navmesh data
Changing from line trace to parabolic trace
Drawing the curved path Drawing the endpoint after all the line segments have been drawn
Teleporting the player
Creating Input Mappings Caching our teleport destination Executing the teleport
Allowing the player to choose their landing orientation Mapping axis inputs Cleaning up our Tick event Using thumbstick input to orient the player Creating a teleport destination indicator Giving it a material Adding the teleport indicator to the pawn
Optimizing and refining our teleport
Displaying UI only when teleport input is pressed Creating a deadzone for our input Fading out and in on teleport
Teleport locomotion summary
Seamless locomotion
Setting up inputs for seamless locomotion Changing the pawn's parent class Fixing the collision component
Handling movement input
Fixing movement speed Letting the player look around without constantly steering Implementing snap-turning Setting up inputs for snap turning Executing the snap turn
Going further
Snap turn using analog input
Summary Chapter 5: Interacting with the Virtual World - Part I Starting a new project from existing work Migrating Blueprints to a new project Copying input bindings
Setting up new project to use the migrated game mode Additional project settings for VR
[v]
178 180 180 181 181 181 182 182 183 188 190 191 192 196 198 198 200 203 204 204 205 207 216 216 218 220 220 222 224 228 229 229 230 233 235 238 238 239 239 240 242 243 246
247 248 248 250 250 251
Table of Contents
Testing our migrated game mode and pawn Adding scenery Adding a NavMesh Testing the map
Creating hands
Migrating hand meshes and animations from the VR Template project Adding hand meshes to our motion controllers Creating a new Blueprint Actor class
Adding motion controller and mesh components Adding a Hand variable Using a Construction Script to handle updates to the Hand variable
Adding BP_VRHand child actor components to your pawn Fixing issues with Hand meshes Replacing references to our old motion controller components in blueprints Creating a function to get our hand mesh
Animating our hands
A quick word about access specifiers Calling our grab functions from the pawn
Creating new input action mappings Adding handlers for new action mappings
Implementing grab animations in the Hand blueprints
Creating an Animation Blueprint for the hand Creating a blend space for our hand animations Wiring the blend space into the animation blueprint Connecting the animation blueprint to our hand blueprint Creating a new enumerator for our grip Smoothing out our grip animation
Summary Chapter 6: Interacting with the Virtual World - Part II Creating an object we can pick up Creating a Blueprint Interface for pickup objects Implementing the Pickup and Drop functions Setting up VRHand to pick up objects Creating a function to find the nearest pickup object Calling Find Nearest Pickup Object on the Tick event Picking up an actor Releasing an actor Test grabbing and releasing
Fixing cube collision Letting players know when they can pick something up
Adding haptic feedback
Creating a Haptic Feedback Effect Curve Playing the haptic effect on command
Going further Summary Chapter 7: Creating User Interfaces in VR Getting started [ vi ]
251 252 252 254 254 255 256 257 257 258 260 262 264 266 268 277 278 278 279 280 283 284 286 288 291 292 296 297
299 299 302 304 306 306 311 312 314 316 317 318 321 322 324 326 327 329 330
Table of Contents
Creating a new Unreal project from an existing project
We’re not alone – adding an AI character
Migrating the third-person character blueprint Cleaning up the third-person character blueprint Examining the animation blueprint Creating a companion character subclass Adding a follow behavior to our companion character Examining the AI controller Improving the companion's follow behavior
Adding a UI indicator to the companion pawn Creating a UI widget using UMG Adding a UI widget to an actor Orienting the indicator widget to face the player Implementing the Align UI function Calling Align UI from the Tick event
Adding a new AI state to the companion pawn Implementing a simple AI state Indicating AI states using the UI indicator Using events to update, rather than polling Being careful of circular references
Ensuring that UI is updated when our state is changed
Adding an interactive UI
Adjusting the button colors Adding event handlers to our buttons Attaching the UI element to the player pawn Using widget interaction components
Sending input through widget interaction components Making a better pointer for our interaction component Creating an interaction beam material Creating an impact effect
Summary Chapter 8: Building the World and Optimizing for VR Setting up the project and collecting assets Migrating blueprints into the new project Verifying the migrated content
Using the VR editor
Entering and exiting VR Mode Navigating in VR Mode
Moving through the world Teleporting through the world Rotating the world Scaling the world Practicing movement
Modifying the world in VR Mode
Moving, rotating, and scaling objects Using both controllers to rotate and scale objects
[ vii ]
331 332 332 332 333 334 336 336 339 342 342 347 349 349 350 352 353 353 357 357 358 360 364 365 366 369 373 374 375 377 382 383 384 385 385 386 387 388 389 390 391 392 393 393 393 396
Table of Contents
Practicing moving objects
Composing a new scene in VR Mode Navigating the radial menu Gizmo Snapping Windows Edit Tools Modes Actions and System
Making changes to our scene
Optimizing scenes for VR
Testing your current performance Stat FPS
Determining your frame time budget
Warnings about performance profiling Stat unit Profiling the GPU Stat scenerendering Draw calls
Stat RHI Stat memory Optimization view modes CPU profiling Turning things on and off
Addressing frame rate problems Cleaning up Blueprint Tick events Managing skeletal animations Merging actors Using mesh LODs Static mesh instancing Nativizing Blueprints
Summary Chapter 9: Displaying Media in VR Setting up the project Playing movies in Unreal Engine
Understanding containers and codecs Finding a video file to test with Adding a video file to an Unreal project Creating a File Media Source asset Creating a Media Player Using Media Textures Testing your Media Player Adding video to an object in the world Using a media playback material Adding sound to our media playback Playing media
[ viii ]
397 397 399 400 400 401 404 405 406 406 406 411 411 411 412 412 414 417 420 422 422 424 425 426 428 428 428 429 429 431 433 433 435
437 438 438 438 440 441 442 444 446 447 448 448 449 451
Table of Contents
Going deeper with the playback material Adding additional controls to our video appearance
Displaying stereo video
Displaying half of the video Displaying a different half of the video to each eye Displaying over/under stereo video
Displaying 360 degree spherical media in VR Finding 360 degree video Creating a spherical movie screen
Playing stereoscopic 360 degree video Controlling your Media Player Creating a Media Manager Adding a Pause and Resume function Assigning events to a media player
Summary Chapter 10: Creating a Multiplayer Experience in VR Testing multiplayer sessions Testing multiplayer from the editor
Understanding the client-server model
The server Listen servers, dedicated dervers, and clients Listen servers Dedicated servers Clients
Testing multiplayer VR
Setting up our own test project Adding an environment Creating a network Game Mode Objects on the network
Server-only objects Server and client objects Server and owning client objects Owning client only objects
Creating our network game mode
Creating a network client HUD
Creating a widget for our HUD Adding a widget to our HUD
Network replication
Creating a replicated actor
Spawning an actor on the server only Replicating the actor to the client Replicating a variable Notifying clients that a value has changed using RepNotify
Creating network-aware pawns for multiplayer Adding a first-person pawn
Setting collision response presets
[ ix ]
454 457 459 461 463 465 466 467 467 469 471 472 474 476 478 479 480 480 484 487 489 489 490 490 491 492 493 493 494 496 497 497 498 498 500 501 504 505 506 507 508 509 511 513 513 514
Unreal Engine 4 Virtual Reality Projects Unreal Engine 4 (UE4) is a powerful tool for developing VR games and applications. With its visual scripting language, Blueprint, and built-in support for all major VR headsets, it's a perfect tool for designers, artists, and engineers to realize their visions in VR. This book will guide you step-by-step through a series of projects that teach essential concepts and techniques for VR development in UE4. You will begin by learning how to think about (and design for) VR and then proceed to set up a development environment. A series of practical projects follows, taking you through essential VR concepts. Through these exercises, you'll learn how to set up UE4 projects that run effectively in VR, how
to build player locomotion schemes, and how to use hand controllers to interact with the world. You'll then move on to create user interfaces in 3D space, use the editor's VR mode to build environments directly in VR, and profile/optimize worlds you've built. Finally, you'll explore more advanced topics, such as displaying stereo media in VR, networking in Unreal, and using plugins to extend the engine. Throughout, this book focuses on creating a deeper understanding of why the relevant tools and techniques work as they do, so you can use the techniques and concepts learned here as a springboard for further learning and exploration in VR.
Things you will learn: •
Understand design principles and concepts for building VR applications
•
Evaluate and solve performance problems in VR to maintain high frame rates
•
Set up your development environment with Unreal Blueprints and C++
•
Display mono and stereo videos in VR
•
Create a player character with several locomotion schemes
•
Extend Unreal Engine's capabilities using various plugins
www.packt.com
FOR SALE IN INDIA ONLY