Artificial Intelligence for Robotics Build intelligent robots that perform human tasks using AI techniques

By Francis X. Govers

www.packt.com

FOR SALE IN INDIA ONLY

Artificial Intelligence for Robotics

Build intelligent robots that perform human tasks using AI techniques

Francis X. Govers

BIRMINGHAM - MUMBAI

Artificial Intelligence for Robotics Copyright © 2018 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Vijin Boricha Acquisition Editor: Namrata Patil Content Development Editor: Sharon Raj Technical Editor: Mohit Hassija Copy Editor: Safis Editing Project Coordinator: Drashti Panchal Proofreader: Safis Editing Indexer: Pratik Shirodkar Graphics: Tom Scaria Production Coordinator: Arvindkumar Gupta First published: August 2018 Production reference: 1290818 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78883-544-2

www.packtpub.com

I would like to dedicate this book to the memory of my father, Francis X. Govers II, who taught me from a young age to dream big and never stop reading and learning.

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content

PacktPub.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors About the author Francis X. Govers is an autonomous vehicle designer at Bell Helicopter Textron. He is the designer of 28 unmanned vehicles or robots for land, sea, air, and space, including RAMSEE, the autonomous security guard robot. Francis helped design the International Space Station, the F-35 JSF Fighter, the US Army Future Combat Systems, and systems for NASCAR and IndyCar. He is an engineer, pilot, author, musician, artist, and maker. He received five outstanding achievement awards from NASA and recognition from Scientific American for World Changing Ideas. I want to thank everyone who supported me in the writing of this book, starting with my wife, Carol, who cheered me on; our adult children, Jessica, Peter, Corbin, and Ammie, for their encouragement. To my grandchildren, William, Oliver, Amelia, and Henry, for their inspiration. To my editors at Packt Publishing, Namrata Patil and Sharon Raj, for encouragement and tireless dedication. I also have to thank Dr. Bob Finkelstein, my robotics guru and mentor. Finally, to Jay Blanchard, my oldest and dearest friend and fellow author.

About the reviewers Nikhil Borkar holds a CQF designation and a post-graduate degree in quantitative finance. He also holds the Certified Financial Crime examiner and Certified Anti-Money Laundering Professional qualifications. He is a registered research analyst with the Securities and Exchange Board of India (SEBI) and has a keen grasp of the Indian regulatory landscape pertaining to securities and investments. He is currently working as an independent FinTech and legal consultant. Prior to this, the worked with Morgan Stanley Capital International (MSCI) as a global RFP project manager. Lentin Joseph is an author and robotics entrepreneur from India. He runs a robotics software company called Qbotics Labs in India. He has 8 years of experience in the robotics domain, primarily in ROS, OpenCV, and PCL. He has authored seven books on ROS, which include Learning Robotics Using Python, both the first and second edition, Mastering ROS for Robotics Programming, first and second edition. He has a master's in Robotics and Automation and also did research at Robotics Institute, CMU, USA.

Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents Preface

1

Chapter 1: Foundation for Advanced Robotics and AI Technical requirements The basic principle of robotics and AI

What is AI (and what is it not)? There is nothing new under the sun The example problem – clean up this room! What you will learn Artificial intelligence and advanced robotics techniques Introducing the robot and our development environment Software components (ROS, Python, and Linux) Robot control systems and a decision-making framework Soft real-time control Control loops

The robot control system – a control loop with soft real-time control Reading serial ports in a real-time manner

Summary Questions Further reading Chapter 2: Setting Up Your Robot Technical requirements What is a robot?

Robot anatomy – what are robots made of?

Subsumption architecture Software setup

Laptop preparation Installing Python Installing ROS on the laptop Setup of Raspberry Pi 3

VNC Setting up catkin workspaces

Hardware

Beginning at the beginning – knolling Assembling the tracks Mounting the tracks Arm base assembly (turntable) Arm assembly Wiring

7 8 8 9 11 12 14 15 17 19 21 21 22 33 36 40 40 41 43 43 44 44 48 51 52 52 53 56 57 58 59 59 60 62 64 66 69

Table of Contents

Summary Questions Further reading Chapter 3: A Concept for a Practical Robot Design Process A systems engineering-based approach to robotics Our task – cleaning up the playroom

Use cases

The problem – part 1

Who – the robot What – pick up toys and put them in the toy box What – pick up and put away in the toy box the items that were not previously in the room When – after the grandchildren have visited and they have left, continue to pick up toys until there are none left When – when I (the user) tell you to, and don’t stop until there are no more toys to be found Where – the game room upstairs

The problem – part 2

Who – the robot, the user (granddad), and the grandchildren What – receive commands and verbally interact (hold a conversation) with children, which must include knock-knock jokes When – as requested by the robot controller, then when the child speaks to the robot Where – in the game room, within about six feet of the robot

What is our robot to do?

Storyboards

Storyboard – put away the toys Project goals

Decomposing hardware needs Breaking down software needs Writing a specification

Summary Questions Further reading Chapter 4: Object Recognition Using Neural Networks and Supervised Learning Technical requirements The image recognition process The image recognition training and deployment process – step by step Image processing Convolution Artificial neurons The convolution neural network process Build the toy/not toy detector Using the neural network

[ ii ]

70 71 71 73 73 74 75 75 77 77 78 78 78 79 80 80 81 82 82 82 86 86 98 98 99 102 104 104 105

107 108 108 110 111 112 114 116 126 135

Table of Contents

Summary Questions Further reading Chapter 5: Picking up the Toys Technical requirements Task analysis

138 138 139

Setting up the solution

How do we pick actions?

Summary of robot arm learning process

Teaching the robot arm

Version one – action state reinforcement learning Adaptive learning rate

Q-learning implementation Version 2 – indexed states and actions Genetic algorithms

Other robot arm machine-learning approaches Google’s SAC-X Amazon Robotics Challenge

Summary Questions Further reading Chapter 6: Teaching a Robot to Listen Technical requirements Robot speech recognition What are we doing? Speech to text Intent Mycroft Hardware Mycroft software Skills Dialogs

Telling jokes – knock, knock Receiving jokes – who’s there?

Summary Questions Further reading Chapter 7: Avoiding the Stairs Technical requirements Task analysis

141 142 143 144 146 147 148 148 153 154 159 163 171 171 172 172 173 173 175 176 176 176 177 181 182 182 185 187 187 191 195 199 199 200 201 202 202 204 206 213

What is SLAM? Alternatives for navigation Neural networks

[ iii ]

Table of Contents

Processing the image Training the neural network for navigation Convolutional neural network robot control implementation

Summary Questions Further reading Chapter 8: Putting Things Away Technical requirements Task analysis Decision trees

What do we mean by pruning? Self-classifying decision trees and AI tools

Entropy One hot encoding

Random forests

Grid searching and A* (A-Star) The A* algorithm D* (D-Star or Dynamic A*) GPS path finding does not use a map!

Summary Questions Further reading Chapter 9: Giving the Robot an Artificial Personality Technical requirements What is an artificial personality? The Turing test The art and science of simulation An emotion state machine Playing the emotion game Creating a model of human behavior Integrating artificial personality into our robot Personality construction – building blocks Context

Under construction The robot emotion engine The human emotion model

Human information storage Context memory

Questions Further reading Chapter 10: Conclusions and Reflections Conclusions about our journey Careers in robotics

[ iv ]

215 218 223 227 228 228 229 230 231 231 233 236 244 245 250 251 257 260 261 263 264 264 265 265 266 268 269 273 276 279 279 280 284 285 290 292 293 294 295 296 297 300 300

Table of Contents

Issues in AI – real and not real

Some very brief words about robots and AI phobia

Understanding risk in AI Final words

Summary Questions Further reading Assessments

302 304 307 309 309 310 310 311

Other Books You May Enjoy

323

Index

327

[v]

Preface The objective of this book is to deliver exactly what is on the cover – Artificial Intelligence for Robotics. The emphasis is on machine learning techniques applied to ground mobile robots. The book starts with professional robot design principles that have been scaled down for smaller robot projects. The AI section begins with convolutional neural networks for object recognition and continues with reinforcement learning and genetic algorithms. The robot gets a voice and learns to tell jokes using AI-based voice recognition that can discern user intent. The book introduces a novel way to navigate without a map using a literal divide and conquer program that uses the upper part of the room to remember paths, and the lower part of avoid obstacles. The book demonstrates how path planning, decision trees, object classification, and navigation are all part of the same problem set. We finish by giving the robot an artificial personality. The final chapter concludes with thoughts on the future of robots and gives advice on robotics as a career. The entire book is built around a single fun example task, which is to design and build a robot that can pick up toys in an indoor, unstructured environment. As you will learn, this project is anything but easy.

Who this book is for This book is designed for intermediate to advanced robotics researchers, professionals, and hobbyists, as well as students who have worked past the basics of robotics and are looking for the next step in their education and skill set. Readers should be familiar with Python and the Robotics Operating System (ROS), as well as Linux. Advanced math is most definitely not required to get a lot out of this book.

What this book covers Chapter 1, Foundation for Robotics and AI, introduces artificial intelligence (AI) and covers

the basics of robotics as applied in this book. The chapter also introduces the AI framework used, which is the Observe-Orient-Decide-Act (OODA) model, and soft real-time control.

Preface Chapter 2, Setting Up Your Robot, covers the robot architecture, ROS, and setting up the

software and hardware, including the construction of the robot example for the book.

Chapter 3, A Concept for a Practical Robot Design Process, introduces a simplified systems

approach to robot design that combines use cases (from systems engineering) and storyboards (from Agile development) to give the reader a structure and a process to use when solving problems with robots and AI. Chapter 4, Object Recognition Using Neural Networks and Supervised Learning, teaches how to

build an artificial neural network. Readers will learn the basics of image recognition as well as the training and evaluation of neural networks using Keras and Python. Chapter 5, Picking Up the Toys, introduces techniques that allow the robot to learn for itself

how to user its robot arm. The key technique is to have a mechanism for the robot to score how well it does. We explore reinforcement learning and dive into Genetic Algorithms. Chapter 6, Teaching the Robot to Listen, We develop on top of a voice-based command

system, a type of digital assistant that uses AI techniques to understand words and divine the intent of the speaker. Basic concepts of speech recognition and natural language processing are introduced, such as context, knowledge bases, intent recognition, and sentence reconstruction. We teach the robot to both tell and understand knock-knock jokes. Chapter 7, Avoiding the Stairs, helps the readers understand robot navigation, including

SLAM. It will help you navigate the robot using a combination of two techniques: Floor Finding for obstacle avoidance, and Neural Network Image recognition for learned navigation without a map. Chapter 8, Putting Things Away, covers path planning, decision trees, classification

techniques, wave front, the A* (A star) and D* (D star) algorithms, and node-based planners.

Chapter 9, Giving the Robot an Artificial Personality, describes simulation and Monte Carlo

modeling, the Robot Emotion Engine, the Human Emotion Model, and integrating personality rules into a chatbot-based conversation engine.

Chapter 10, Conclusions and Remarks, has some words about the future of AI and robotics,

as well as advice about robotics as a career.

[2]

Preface

To get the most out of this book The reader should have a good grasp of programming with Python, either 2.7 or 3.6. Most of the examples are written in 2.7, but conversion to 3.6 or above is fairly simple – it is mostly changing the print statements. The ROS works only in Linux. The reader can use a virtual machine to emulate Linux on a Windows computer. This was done by the author when writing this book. Packt Publishing has several excellent books that explain how to use ROS, if you need more detailed instruction. No special math skills are required past high-school level. If you want to follow the robot's construction, then basic hand tools (screwdrivers, wrenches, Allen keys, and a soldering iron) will be required. All the other installation instructions are in the appropriate chapters as the book progresses.

Download the example code files You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you. You can download the code files by following these steps: 1. 2. 3. 4.

Log in or register at www.packtpub.com. Select the SUPPORT tab. Click on Code Downloads & Errata. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of: WinRAR/7-Zip for Windows Zipeg/iZip/UnRarX for Mac 7-Zip/PeaZip for Linux The code bundle for the book is also hosted on GitHub at https:/​/​github.​com/ PacktPublishing/​Artificial-​Intelligence-​for-​Robotics. In case there's an update to the code, it will be updated on the existing GitHub repository. We also have other code bundles from our rich catalog of books and videos available at https:/​/​github.​com/​PacktPublishing/​. Check them out!

[3]

Artificial Intelligence for Robotics Artificial Intelligence for Robotics starts with an introduction to Robot Operating Systems (ROS), Python, robotic fundamentals, and the software and tools that are required to start out with robotics. You will learn robotics concepts that will be useful for making decisions, along with basic navigation skills.

object. With plenty of use cases throughout, you will explore natural language processing (NLP) and machine learning techniques to further enhance your robot. In the concluding chapters, you will learn about path planning and goal-oriented programming, which will help your robot prioritize tasks.

As you make your way through the chapters, you will learn about object recognition and genetic algorithms, which will teach your robot to identify and pick up an irregular

By the end of this book, you will have learned to give your robot an artificial personality using simulated intelligence.

Things you will learn: •

Get started with robotics and artificial intelligence

•

Teach your robot to listen using NLP via an expert system

•

Apply simulation techniques to give your robot an artificial personality

•

Use machine learning and computer vision to teach your robot how to avoid obstacles

•

Understand object recognition using neural networks and supervised learning techniques

•

Understand path planning, decision trees, and search algorithms in order to enhance your robot

•

Pick up objects using genetic algorithms for manipulation

www.packt.com

FOR SALE IN INDIA ONLY