Learn Docker – Fundamentals of Docker 19.x Second Edition

Build, test, ship, and run containers with Docker and Kubernetes

Gabriel N. Schenker

www.packt.com

FOR SALE IN INDIA ONLY

Learn Docker – Fundamentals of Docker 19.x Second Edition Build, test, ship, and run containers with Docker and Kubernetes

Gabriel N. Schenker

BIRMINGHAM - MUMBAI

Learn Docker – Fundamentals of Docker 19.x Second Edition Copyright © 2020 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Vijin Boricha Acquisition Editor: Shrilekha Inani Content Development Editor: Ronn Kurien Senior Editor: Richard Brookes-Bland Technical Editor: Sarvesh Jaywant Copy Editor: Safis Editing Project Coordinator: Neil Dmello Proofreader: Safis Editing Indexer: Tejal Daruwale Soni Production Designer: Deepika Naik First published: April 2018 Second edition: March 2020 Production reference: 1130320 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-83882-747-2

www.packt.com

Packt.com

Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Fully searchable for easy access to vital information Copy and paste, print, and bookmark content Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors About the author Gabriel N. Schenker has more than 25 years of experience as an independent consultant, architect, leader, trainer, mentor, and developer. Currently, Gabriel works as Lead Solution Architect at Techgroup Switzerland. Prior to that, Gabriel worked as Lead Curriculum Developer at Docker and at Confluent. Gabriel has a Ph.D. in Physics, and he is a Docker Captain, a Certified Docker Associate, a Certified Kafka Developer and Operator, and an ASP Insider. When not working, Gabriel enjoys time with his wonderful wife Veronicah and his children. I want to give special thanks to my editors, Ronn Kurien and Suzanne Coutinho, who patiently helped me to get this book done and get it done right.

About the reviewer Francisco Javier Ramírez Urea is a technology enthusiast and professional, Docker Captain, casual developer, open source advocate, a certified trainer and solutions architect at HoplaSoftware, and a technical book writer and reviewer. He is also a Kubernetes Certified Administrator, a Docker Certified Associate, a Docker Certified Instructor, and a Docker MTA program Consultant, as well as a Docker/Kubernetes and NGINX expert and a DevOps/CI-CD solutions integrator. He currently works as a solutions architect focused on containers and microservices technologies. He is passionate to teach his students everything he know. Continuous learning is the main motivation of his career.

Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents Preface

1

Section 1: Motivation and Getting Started Chapter 1: What Are Containers and Why Should I Use Them? What are containers? Why are containers important? What's the benefit for me or for my company? The Moby project Docker products Docker CE Docker EE

Container architecture Summary Questions Further reading Chapter 2: Setting Up a Working Environment Technical requirements The Linux command shell PowerShell for Windows Using a package manager Installing Homebrew on macOS Installing Chocolatey on Windows

Installing Git Choosing a code editor

Installing VS Code on macOS Installing VS Code on Windows Installing VS Code on Linux Installing VS Code extensions

Installing Docker for Desktop

Installing Docker for Desktop on macOS Installing Docker for Desktop on Windows Installing Docker CE on Linux

Installing Docker Toolbox

Installing Docker Toolbox on macOS Installing Docker Toolbox on Windows Setting up Docker Toolbox

Installing Minikube

11 12 15 17 17 18 18 19 19 20 21 22 23 24 24 25 25 25 26 27 28 28 29 29 30 30 32 34 35 35 36 37 38 42

Table of Contents

Installing Minikube on macOS and Windows Testing Minikube and kubectl

Summary Questions Further reading

42 42 44 44 45

Section 2: Containerization, from Beginner to Black Belt Chapter 3: Mastering Containers Technical requirements Running the first container Starting, stopping, and removing containers Running a random trivia question container Listing containers Stopping and starting containers Removing containers

Inspecting containers Exec into a running container Attaching to a running container Retrieving container logs

Logging drivers Using a container-specific logging driver Advanced topic – changing the default logging driver

Anatomy of containers

Architecture Namespaces Control groups (cgroups) Union filesystem (Unionfs) Container plumbing runC Containerd

Summary Questions Further reading Chapter 4: Creating and Managing Container Images What are images? The layered filesystem The writable container layer Copy-on-write Graph drivers

Creating images

Interactive image creation Using Dockerfiles

[ ii ]

49 50 50 52 54 56 58 60 60 62 63 65 66 67 68 69 70 70 72 73 73 73 74 74 74 74 77 78 78 80 81 82 82 82 86

Table of Contents

The FROM keyword The RUN keyword The COPY and ADD keywords The WORKDIR keyword The CMD and ENTRYPOINT keywords A complex Dockerfile Building an image Multi-step builds Dockerfile best practices

Saving and loading images

Lift and shift: Containerizing a legacy app Analysis of external dependencies Source code and build instructions Configuration Secrets Authoring the Dockerfile The base image Assembling the sources Building the application Defining the start command

Why bother?

Sharing or shipping images Tagging an image Image namespaces Official images Pushing images to a registry

Summary Questions Further reading Chapter 5: Data Volumes and Configuration Technical requirements Creating and mounting data volumes

Modifying the container layer Creating volumes Mounting a volume Removing volumes Accessing volumes created with Docker for Desktop

Sharing data between containers Using host volumes Defining volumes in images Configuring containers

Defining environment variables for containers Using configuration files Defining environment variables in container images Environment variables at build time

[ iii ]

87 88 89 90 91 93 94 98 100 102 103 104 104 105 105 106 106 106 107 107 108 109 109 109 112 112 113 113 114

115 116 116 116 117 120 122 123 126 128 130 133 134 135 137 138

Table of Contents

Summary Questions Further reading Chapter 6: Debugging Code Running in Containers Technical requirements Evolving and testing code running in a container Mounting evolving code into the running container

Auto restarting code upon changes Auto-restarting for Node.js Auto-restarting for Python Auto-restarting for .NET

Line-by-line code debugging inside a container Debugging a Node.js application Debugging a .NET application

Instrumenting your code to produce meaningful logging information Instrumenting a Python application Instrumenting a .NET C# application

Using Jaeger to monitor and troubleshoot Summary Questions Further reading Chapter 7: Using Docker to Supercharge Automation Technical requirements Executing simple admin tasks in a container Using test containers Integration tests for a Node.js application The Testcontainers project

Using Docker to power a CI/CD pipeline Summary Questions Further reading Chapter 8: Advanced Docker Usage Scenarios Technical requirements All of the tips and tricks of a Docker pro

Keeping your Docker environment clean Running Docker in Docker Formatting the output of common Docker commands Filtering the output of common Docker commands Optimizing your build process Limiting resources consumed by a container Read-only filesystem

[ iv ]

139 139 140 141 142 142 146 149 149 151 154 159 160 165 169 170 172 176 181 181 182 183 184 184 187 190 200 207 221 222 222 223 224 224 224 225 228 229 229 231 232

Table of Contents

Avoid running a containerized app as root

Running your Terminal in a remote container and accessing it via HTTPS Running your development environment inside a container Running your code editor in a remote container and accessing it via HTTPS Summary Questions Further reading

232 234 238 243 247 247 247

Section 3: Orchestration Fundamentals and Docker Swarm Chapter 9: Distributed Application Architecture Understanding the distributed application architecture Defining the terminology

Patterns and best practices

Loosely coupled components Stateful versus stateless Service discovery Routing Load balancing Defensive programming Retries Logging Error handling

Redundancy Health checks Circuit breaker pattern

Running in production Logging Tracing Monitoring Application updates

Rolling updates Blue-green deployments Canary releases Irreversible data changes Rollback

Summary Questions Further reading Chapter 10: Single-Host Networking Technical requirements Dissecting the container network model [v]

251 252 252 254 254 255 255 257 258 259 259 259 260 260 261 261 263 263 263 264 264 264 265 266 266 267 268 268 268 269 270 270

Table of Contents

Network firewalling Working with the bridge network The host and null network The host network The null network

Running in an existing network namespace Managing container ports HTTP-level routing using a reverse proxy Containerizing the monolith Extracting the first microservice Using Traefik to reroute traffic

Summary Questions Further reading Chapter 11: Docker Compose Technical requirements Demystifying declarative versus imperative Running a multi-service app Building images with Docker Compose Running an application with Docker Compose

Scaling a service Building and pushing an application Using Docker Compose overrides Summary Questions Further reading Chapter 12: Orchestrators What are orchestrators and why do we need them? The tasks of an orchestrator Reconciling the desired state Replicated and global services Service discovery Routing Load balancing Scaling Self-healing Zero downtime deployments Affinity and location awareness Security

Secure communication and cryptographic node identity Secure networks and network policies Role-based access control (RBAC) Secrets

[ vi ]

272 273 283 283 284 285 287 289 290 293 294 296 296 297 299 300 300 301 302 305 309 311 313 315 315 315 317 317 319 319 320 321 322 322 323 324 325 326 327 327 328 328 329

Table of Contents

Content trust Reverse uptime

Introspection

Overview of popular orchestrators Kubernetes Docker Swarm Apache Mesos and Marathon Amazon ECS Microsoft ACS

Summary Questions Further reading Chapter 13: Introduction to Docker Swarm The Docker Swarm architecture Swarm nodes Swarm managers Swarm workers

Stacks, services, and tasks Services Task Stack

Multi-host networking Creating a Docker Swarm

Creating a local single node swarm Creating a local Swarm in VirtualBox or Hyper-V Using Play with Docker to generate a Swarm Creating a Docker Swarm in the cloud

Deploying a first application

Creating a service Inspecting the service and its tasks Logs of a service Reconciling the desired state Deleting a service or a stack Deploying a multi-service stack

The swarm routing mesh Summary Questions Further reading Chapter 14: Zero-Downtime Deployments and Secrets Technical requirements Zero-downtime deployment Popular deployment strategies Rolling updates

[ vii ]

330 330 331 332 332 333 334 335 336 336 337 338

339 340 341 342 343 345 346 346 347 348 349 350 352 356 360 366 366 368 370 371 372 373 375 376 376 377 379 380 380 380 380

Table of Contents

Health checks Rollback Blue–green deployments Canary releases

Storing configuration data in the swarm Protecting sensitive data with Docker secrets Creating secrets Using a secret Simulating secrets in a development environment Secrets and legacy applications Updating secrets

Summary Questions Further reading

387 392 393 394 395 399 400 401 403 404 407 408 408 409

Section 4: Docker, Kubernetes, and the Cloud Chapter 15: Introduction to Kubernetes Technical requirements Kubernetes architecture Kubernetes master nodes Cluster nodes Introduction to Minikube Kubernetes support in Docker for Desktop Introduction to pods

Comparing Docker container and Kubernetes pod networking Sharing the network namespace Pod life cycle Pod specifications Pods and volumes

Kubernetes ReplicaSet

ReplicaSet specification Self-healing

Kubernetes deployment Kubernetes service Context-based routing Comparing SwarmKit with Kubernetes Summary Questions Further reading Chapter 16: Deploying, Updating, and Securing an Application with Kubernetes Technical requirements Deploying a first application [ viii ]

413 414 414 417 418 420 422 428 429 431 434 435 437 439 440 442 443 444 446 447 448 448 450 451 452 452

Table of Contents

Deploying the web component Deploying the database Streamlining the deployment

Defining liveness and readiness Kubernetes liveness probe Kubernetes readiness probe Kubernetes startup probe

Zero downtime deployments Rolling updates Blue-green deployment

Kubernetes secrets

Manually defining secrets Creating secrets with kubectl Using secrets in a pod Secret values in environment variables

Summary Questions Further reading Chapter 17: Monitoring and Troubleshooting an App Running in Production Technical requirements Monitoring an individual service Instrumenting a Node.js-based service Instrumenting a .NET Core-based service

Using Prometheus to monitor a distributed application Architecture Deploying Prometheus to Kubernetes Deploying our application services to Kubernetes Deploying Grafana to Kubernetes

Troubleshooting a service running in production The netshoot container

Summary Questions Further reading Chapter 18: Running a Containerized App in the Cloud Technical requirements Deploying and using Docker EE on AWS Provisioning the infrastructure Installing Docker Installing Docker UCP Using remote admin for the UCP cluster Deploying to Docker Swarm Deploying to Kubernetes

[ ix ]

452 456 460 462 463 466 467 468 468 472 477 477 479 480 482 484 485 485 487 487 488 489 492 495 496 496 503 507 511 512 514 514 515 517 518 518 518 520 521 524 525 527

Learn Docker – Fundamentals of Docker 19.x – Second Edition Containers enable you to package an application with all the components it needs, such as libraries and other dependencies, and ship it as one package. Docker containers have revolutionized the software supply chain in both small and large enterprises. Starting with an introduction to Docker fundamentals and setting up an environment to work with it, you'll delve into concepts such as Docker containers, Docker images, and Docker Compose. As you progress, the book will help you explore deployment, orchestration, networking, and security.

Finally, you'll get to grips with Docker functionalities on public clouds such as Amazon Web Services (AWS), Azure, and Google Cloud Platform (GCP), and learn about Docker Enterprise Edition features. Additionally, you'll also discover the benefits of increased security with the use of containers. By the end of this Docker book, you'll be able to build, ship, and run a containerized, highly distributed application on Docker Swarm or Kubernetes, running on-premises or in the cloud.

Things you will learn: •

Containerize your traditional or microservice-based applications

•

Run a highly distributed application using Docker Swarm or Kubernetes

•

Develop, modify, debug, and test an application running inside a container

•

Update or rollback a distributed application with zero downtime

•

Share or ship your application as an immutable container image

•

Secure your applications with encapsulation, networks, and secrets

•

Build a Docker Swarm and a Kubernetes cluster in the cloud

•

Troubleshoot a containerized, highly distributed application in the cloud

www.packt.com

FOR SALE IN INDIA ONLY