OpenStack Cookbook

OpenStack Cookbook

Manage Compute, Storage and Networking through Single Interface

Jorven Halquin

Preface

OpenStack Cookbook is specifically tailored for cloud professionals, network administrators, and engineers who express a desire to augment their proficiency in effectively managing OpenStack environments. The book provides a pragmatic and interactive methodology with recipes that specifically target solutions to tackle the actual difficulties encountered in implementing and overseeing cloud infrastructure. Every chapter presents systematic solutions, enabling you to develop proficiency in designated OpenStack components.

These recipes provide instructions on how to install, configure, and optimize OpenStack services such as Keystone for identity management, Glance for image processing, Neutron for networking, Nova for computing management, Cinder for block storage, and Octavia for load balancing. Some of the more advanced topics covered in the book include how to use Heat for Infrastructure as Code (IaC), how to automate and repeat infrastructure deployments, and how to use Heat and Ceilometer to create auto-scaling solutions that dynamically adjust resources according to demand. Typical problems with stack creation, resource management, and orchestration tasks will be explained and solved. The book encompasses a broad spectrum of situations, ranging from the prevention of stack creation failures and template validation errors to the effective management of resource dependencies and performance issues.

With the knowledge you gain from this book, you will be able to set up, administer, and fix OpenStack environments with ease. No matter your level of expertise with OpenStack, this cookbook will equip you with the practical knowledge and solutions you need to use OpenStack efficiently in real-world scenarios.

In this book you will learn how to:

Get the environment quickly settlled with all the OpenStack services, including Nova, Keystone, Glance, and Neutron.

Make easy management of OpenStack cloud environments a reality with these real-world solutions.

Get your cloud infrastructure up and running consistently and reliably every time with the help of Heat templates.

Improve your resource allocation skills with practical experience in auto-scaling with Heat and Ceilometer.

Keep operations stable by troubleshooting common OpenStack issues and providing step-by-step solutions.

Vertigo, XLAN, and security group configurations are just a few of the advanced networking techniques you can learn about in Neutron.

Secure your cloud with SSL termination and volume encryption.

Optimize traffic management and guarantee high availability by deploying Octavia-based load balancing solutions.

Build automated cloud environment management with Infrastructure as Code (IaC).

Get to the heart of practical problems by following recipes developed specifically for the demands of enterprise cloud infrastructure.

Prologue

You are here: OpenStack Cookbook. Hello, I'm Jorven Halquin, and I've been an expert in cloud infrastructures, particularly OpenStack, for many years. I wrote this book to assist you, network administrators, engineers, and cloud specialists, in making your work with OpenStack more efficient. My goal is to share my knowledge and insights through a series of well-crafted recipes so you can have the most seamless and effective OpenStack experience possible.

Throughout this book, we will be working with an imaginary but relatable tech company called GitforGits. Initially, GitforGits operated as a small on-premise server-based software development company. Their needs for infrastructure expanded along with their business. To handle their growing number of projects and clients, they discovered that they needed better storage options, more processing power, and a more adaptable networking configuration. They decided to switch to OpenStack at that point. GitforGits selected OpenStack because of its open-source architecture and the complete control it provides over their cloud environment. But they quickly discovered, as many other organizations have, that OpenStack management can be daunting without the proper resources and expertise. This book fills that need. We meticulously crafted every recipe and chapter in this cookbook. We'll go over their issues and investigate how OpenStack can help, whether it's using Keystone for identity management, Nova for virtual machine deployment, or Neutron for complex networking configuration.

I aimed to make practicality a central theme in this book. I didn't want to limit my advice to abstract ideas or theoretical knowledge. Instead, we organize the content based on real-world issues and their corresponding solutions. We considered using OpenStack's Load Balancer as a Service, Octavia, when GitforGits needed to guarantee that their applications would always be accessible and responsive. We looked into Cinder to handle their block storage needs when they needed safe and effective storage solutions. And Heat became our go-to tool when they wanted automation to handle resource scaling because it allowed us to define infrastructure as code, guaranteeing repeatable and consistent deployments.

This book aims to empower you to take advantage of OpenStack's full potential in your own environments, not just teach you how to use it. The recipes here are as versatile as the platform itself; both are meant to be easily adjusted to suit your needs. You can use them directly in your scenarios or modify them to suit your specific requirements. I've also included troubleshooting techniques to assist you in overcoming the unavoidable hiccups on the road. I derived these solutions from my own experiences and lessons learned, addressing issues like resource allocation optimization and stack creation failures.

I hope you'll see the tremendous value that OpenStack can add to your infrastructure as we go through each chapter. Although the GitforGits story is fictional, the problems they encounter and the fixes we provide are very real. By the time this book ends, I want you to feel comfortable taking care of your OpenStack environment and have the know-how to handle any challenge that comes your way.

Copyright © 2024 by GitforGits

All rights reserved. This book is protected under copyright laws and no part of it may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without the prior written permission of the publisher. Any unauthorized reproduction, distribution, or transmission of this work may result in civil and criminal penalties and will be dealt with in the respective jurisdiction at anywhere in India, in accordance with the applicable copyright laws.

Published by: GitforGits

Publisher: Sonal Dhandre

www.gitforgits.com

[email protected]

Printed in India

First Printing: August 2024

Cover Design by: Kitten Publishing

For permission to use material from this book, please contact GitforGits at [email protected].

Content

Content

Preface

GitforGits

Acknowledgement

Chapter 1: Setting up OpenStack Infrastructure

Introduction

Recipe 1: Preparing Hardware for OpenStack Deployment

Background

Hardware Requirements

Controller Node

Compute Node

Network Configuration

BIOS and Firmware Configuration

Operating System Installation

Post-Installation Configuration

Recipe 2: Setting up Ubuntu for OpenStack Installation

Preparing Ubuntu Environment

Installing OpenStack Client

Adding OpenStack Bobcat Repository

Installing OpenStack Packages

Configuring Database and Messaging Services

Configuring OpenStack Services

Starting the Services

Recipe 3: Installing and Configuring Keystone

OpenStack Components Overview

Introduction to Keystone

Install Keystone Packages

Configure Keystone Database

Configure Keystone

Initialize Fernet Keys

Bootstrap Keystone

Configure Apache HTTP Server

Verify Keystone Installation

Create Keystone Domain, Projects, Users, and Roles

Populate the Service Catalog

Recipe 4: Installing and Configuring Glance

Glance Overview

Install Glance Packages

Configure Glance Database

Configure Glance

Register Glance with Keystone

Start the Glance Services

Uploading an Image to Glance

Managing Image Metadata

Recipe 5: Installing and Configuring Neutron

Purpose of Neutron

Install Neutron Packages

Configure the Neutron Database

Configure Neutron

Configure Linux Bridge Agent

Configure the DHCP Agent

Configure the Metadata Agent

Configure the L3 Agent

Register Neutron with Keystone

Start the Neutron Services

Creating a Network and Subnet

Recipe 6: Installing and Configuring Nova

Nova’s Characteristics

Installing and Configuring Nova

Configure the Nova Database

Configure Nova

Configure Nova Compute Service

Register Nova with Keystone

Start Nova Services

Create a Flavor

Launch an Instance

Accessing Instance Console

Recipe 7: Setting up Cinder for Block Storage

Block Storage and Volume Service

Introduction to Cinder

Installing and Configuring Cinder

Configure Cinder Database

Configure Cinder

Prepare LVM Backend

Register Cinder with Keystone

Start Cinder Services

Creating and Attaching a Volume

Accessing the Volume from the Instance

Recipe 8: Defining Network Topology for OpenStack

Understanding Network Topology

Choosing the Right Network Topology

Management Network

Tenant Network

External (Public) Network

Storage Network

Define Management Network

Create Management Network

Create Management Subnet

Define Tenant Network

Define External Network

Assign Floating IPs to Instances

Security Groups and Firewall Rules

Summary

Chapter 2: Identity Management with Keystone

Introduction

Recipe 1: Managing Keystone Tokens

Understanding Keystone Tokens

Viewing Token Information

Validating a Token

Revoking a Token

Managing Fernet Keys

Checking Token Expiration Settings

Auditing and Token Usage

Recipe 2: Setting up RBAC Policies

RBAC in OpenStack

Defining Roles

Assigning Roles to Users

Creating and Applying RBAC Policies

Applying RBAC to Specific Projects

Verifying RBAC Implementation

Recipe 3: Configuring and Utilizing Service Catalog

Understanding the Service Catalog

Define Services in the Catalog

Define Endpoints for each Service

Verifying the Service Catalog

Testing Endpoint Accessibility

Recipe 4: Integrating Keystone with LDAP

Role of LDAP

Benefits of LDAP Integration

Configuring LDAP in Keystone

Configuring Keystone to use LDAP Backend

Creating LDAP Users and Assigning Roles

Verifying LDAP Integration

Recipe 5: Implementing Multi-Domain Support

Need for Multi-Domain Support

Enabling Multi-Domain Support

Creating a New Domain

Creating Users and Projects within New Domain

Managing and Accessing Multiple Domains

Managing Roles Across Domains

Recipe 6: Using Keystone Federation for Multi-Cloud Authentication

Understanding Keystone Federation

Necessity of Multi-Cloud Authentication

Setting up the Identity Provider (IdP)

Setting up Service Provider (SP)

Federating Users Across Clouds

Recipe 7: Troubleshooting Common Keystone Issues

Understanding Common Keystone Issues

Keystone Service Fails to Start

Troubleshooting Tips

Authentication Failures

Troubleshooting Tips

Token Validation Issues

Troubleshooting Tips

Federation Configuration Issues

Troubleshooting Tips

LDAP Integration Issues

Troubleshooting Tips

RBAC Issues

Troubleshooting Tips

Issues with Multi-Domain Support

Troubleshooting Tips

Service Catalog Issues

Troubleshooting Tips

Summary

Chapter 3: Image Management with Glance

Introduction

Recipe 1: Creating and Registering Images in Glance

Introduction to Image Management

Installing Glance CLI

Creating an Image

Obtain or Build an Image

Create an Image in Glance

Verify Image Creation

Registering Image in Glance

Registering an External Image

Updating Image Metadata

Deleting an Image

Recipe 2: Building Custom Images with Cloud-Init

Getting Started with Cloud-Init

Installing Cloud-Init

Understanding Cloud-Init Configuration

Sample Cloud-Init Configuration

Building a Custom Image with Cloud-Init

Prepare a Base Image

Customize the Image with Cloud-Init

Verify the Cloud-Init Configuration

Upload the Custom Image to Glance

Launch an Instance using Custom Image

Recipe 3: Managing Image Metadata and Properties

Introduction to Image Metadata and Properties

Understanding Image Metadata and Properties

What Are Metadata and Properties?

Why Manage Metadata and Properties?

Viewing Image Metadata and Properties

Adding and Updating Image Metadata Properties

Add Custom Properties to an Image

Update Existing Properties

Verify Added or Updated Properties

Deleting Image Properties

Delete a Specific Property

Verify Property Deletion

Using Image Metadata for Enhanced Management

Organizing Images by Tags

Searching for Tagged Images

Implementing Image Policies Based on Properties

Automating Metadata Management

Using Scripts to Automate Metadata Tasks

Automated Reporting on Image Metadata

Best Practices for Managing Image Metadata and Properties

Recipe 4: Configuring Glance to Use Different Backends (Swift, Ceph)

Introduction to Object Storage Systems

Swift Object Storage

Ceph Object Storage

Choosing the Right Storage System

Configuring Glance to use Ceph

Prepare the Ceph Cluster

Create a Ceph Pool for Glance

Create a Client Keyring

Install and Configure Ceph on the Glance Node

Install Ceph Client Packages

Configure Ceph for Glance

Configure Glance to use Ceph

Verify the Configuration

Managing and Monitoring the Ceph Backend

Monitor Ceph Health

Expanding Storage

Backup and Recovery

Recipe 5: Automating Image Uploads with Glance CLI

Introduction to Automating Image Uploads

Creating the Automation Script

Scheduling the Script with Cron

Multiple Image Uploads

Custom Metadata and Tags

Fetching Images from Remote Sources

Testing and Monitoring

Recipe 6: Managing Image Versions and Updates

Introduction to Image Versioning and Updates

Creating and Tagging Image Versions

Creating a New Image Version

Tagging Images

Viewing Image Versions

Updating Images

Uploading an Updated Image

Deprecating Older Versions

Replacing Instances with Updated Images

Rolling Back to Previous Versions

Automating Image Version Management

Automated Image Upload Script

Automating Version Tagging

Best Practices for Managing Image Versions

Recipe 7: Securing and Encrypting Images in Glance

Recent Insights on Image Vulnerabilities

Enforcing Secure Access Controls

Encrypting Images

Secure Image Distribution

Summary

Chapter 4: Networking with Neutron

Introduction

Recipe 1: Understanding OpenStack Network Types

Introduction to OpenStack Network Types

Exploring Neutron Network Types

Flat Network

VLAN (Virtual LAN)

VXLAN (Virtual Extensible LAN)

GRE (Generic Routing Encapsulation)

Configuring Neutron for GitforGits

Recipe 2: Configuring Neutron Plugins and Agents

Introduction to Neutron Plugins and Agents

Configuring Neutron Plugins and Agents

Configure the ML2 Plugin

Configure the OVS Agent

Configure the L3 Agent

Configure the DHCP Agent

Configure the Metadata Agent

Starting and Verifying Neutron Services

Start the Neutron Services

Verify the Configuration

Test Network Functionality

Recipe 3: Setting up Neutron L2 Networking (VLAN, VXLAN)

Setting up VLAN Networking

Create a VLAN Network

Create a Subnet for the VLAN Network

Launch an Instance on the VLAN Network

Verify VLAN Connectivity

Setting up VXLAN Networking

Create a VXLAN Network

Create a Subnet for the VXLAN Network

Launch an Instance on the VXLAN Network

Verify VXLAN Connectivity

Recipe 4: Implementing Neutron L3 Routing

Introduction to Neutron L3 Routing

Create the External Network

Create a Subnet for the External Network

Creating and Configuring Router

Create a Router

Set the External Gateway for the Router

Connect Internal Networks to the Router

Assigning Floating IPs to Instances

Allocate a Floating IP

Associate the Floating IP with an Instance

Verify External Connectivity

Implementing Advanced L3 Features

Enable DVR in the ML2 Configuration

Enable DVR for the Router

Configuring High Availability (HA) Routers

Recipe 5: Configuring Neutron Security Groups and Firewall Rules

Security Groups and Firewall Rules

Understanding Neutron Security Groups

Configuring Neutron Security Groups

Configuring Firewall Rules

Best Practices for Security Groups and Firewall Rules

Recipe 6: Troubleshooting Neutron Networking Issues

Troubleshooting L2 Networking Issues (VLAN, VXLAN)

Instances on the Same VLAN/VXLAN Cannot Communicate

High Latency or Packet Loss in VXLAN Networks

Troubleshooting L3 Routing Issues

Instances Cannot Reach External Networks

Intermittent Connectivity Between Internal Networks

Troubleshooting Security Groups and Firewall Rules

Instances Not Accessible Despite Correct Security Group Rules

Firewall Rules Blocking Legitimate Traffic

Recipe 7: Integrating Neutron with SDN Controllers

Overview of OpenDaylight

Preparing Integration

Configuring Neutron to use OpenDaylight

Modify the Neutron ML2 Configuration

Modify the Neutron Server Configuration

Verifying Integration

Summary

Chapter 5: Compute Resources with Nova

Introduction

Recipe 1: Launching Virtual Machine Instances with Nova

Nova Overview

Selecting Flavor

Launching VM Instance

Accessing the Instance

Verifying Instance Functionality

Recipe 2: Managing Instance Placement and Affinity Rules

Affinity and Anti-Affinity Rules

Defining Instance Placement Strategies

Implementing Affinity Rules

Creating an Affinity Group

Launching Instances with Affinity

Verifying Affinity Placement

Implementing Anti-Affinity Rules

Creating an Anti-Affinity Group

Launching Instances with Anti-Affinity

Verifying Anti-Affinity Placement

Managing and Modifying Affinity Rules

Revising Server Group Policies

Adding an Instance

Removing an Instance

Deleting a Server Group

Advanced Placement Strategies

Creating a Host Aggregate

Using Availability Zones

Recipe 3: Deploying Instances with SSH Key Injection

SSH Keys for Deploying Instances

Creating an SSH Key Pair

Generate a New SSH Key Pair

Upload the Public Key to OpenStack

Launching an Instance with SSH Key Injection

Launch the Instance

Monitor the Instance Launch

Accessing the Instance via SSH

Retrieve the Instance’s IP Address

SSH into the Instance

Managing SSH Keys

Listing SSH Keys

Deleting an SSH Key

Updating an SSH Key

SSH Key Management Best Practices

Recipe 4: Using Nova Hypervisors (KVM, QEMU)

Nova Hypervisors: KVM and QEMU

Verifying Hypervisor Support

Configuring Nova to Use KVM and QEMU

Launching Instances with KVM or QEMU

Launch an Instance

Verify Hypervisor in Use

Access the Instance

Monitoring and Managing Hypervisors

Monitor Hypervisor Performance

List Hypervisors

Migrate Instances

Recipe 5: Configuring Nova for Resource Quotas and Limits

Resource Quotas and Limits in Nova

Understanding Default Quotas

Configuring Custom Quotas

Managing Quota Allocations Across Multiple Projects

Monitoring and Managing Quota Usage

Setting Global Quotas

Recipe 6: Automating Instance Deployments

Instance Deployment Automation

Scripting a Single Instance Deployment

Automating the Deployment of Multiple Instances

Automating Post-Deployment Tasks

Scaling Deployment with Nova CLI

Monitoring and Managing Automated Deployments

Summary

Chapter 6: Block Storage with Cinder

Introduction

Recipe 1: Creating and Attaching Volumes to Instances

Overview

Creating a Volume in Cinder

Attaching Volumes to an Instance

Verifying Attached Volumes

Automating Volume Management

Recipe 2: Managing Volume Snapshots for Data