OpenStack
OpenStack is a middleware that enables cloud-computing according to the following generally agreed on definition:
Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model is composed of five essential characteristics, three service models, and four deployment models.
Mell, Peter, and Tim Grance. "The NIST definition of cloud computing." (2011).
Abstract Workflow
From the users perspective it acts as a blackbox. In the following we iteratively dive deeper into this box.
Inside The box
OpenStack consists of a multitude of different components.
Technically the virtualization technologies reside within these components. The following is an incomplete list with widely applied OpenStack components. The actual list way more exhaustive
- Nova (Compute)
- Neutron (Network)
- Cinder (Storage)
- Glance (Images)
- Designate (DNS)
- Keystone (AuthX)
- Horizon (Dashboard)
The actual components share a lot of properties:
- Distributed loosely coupled system
- Slightly similar to microservices or SOA
- Every component has a strictly defined and versioned API
- They communicate directly, as well as decoupled via a message queue
They are developed individually by different teams
The following gives an overview over the components with examples of their respective duties. This list ist not exhaustive.
Nova
Nova acts as a hypervisor. It therefore
- Handles physical resources (CPU and Memory) of a VM
- Enact VM lifecycle management
- Handles console access
Nova is Pluggable with arbitrary compute virtualization technologies (we use KVM and Android emulator)
Neutron
Neutron is responsible for networking. Typical tasks are:
- Internal and external network access to a VM and across VMs
- Management of networks, subnetworks, routers, etc
Neutron is pluggable with arbitrary network virtualization technologies (we use Linux kernel features like namespaces and bridges with VXLAN)
Cinder
Cinder is the storage engine for Openstack. It provides
- Storage for system state
- Configuration and user-data virtually attached to a VM
- Snapshots
Cinder is pluggable with arbitrary storage virtualization technologies (we use ceph and LVM)
Glance
Glance is OpenStacks Image registry and tightly integrates with Nova and Cinder. It can
- Proide initial bootable root disks containing the extracted root filesystem of the designated VM
- Contain arbitrary OS like Ubuntu and Android
- Extract to cinder volumes (cinder/ceph acts as vessel)
- Create snapshots of VMs
Designate
Designate implements Name resolution. It therefore
- Extends Neutron via name resolution based on DNS
- Integrates with existing DNS and other directory services (Active Directory)
- Provides the resolution as a service
Keystone
Keystone is the central Authentication and Authorization Service. It integrades with every other OpenStack component and thus provides
- Role Based Access Control (RBAC) to enable multi tenancy
- Fine grained access control on projects, resources and services
- An AuthX layer that is user and backend facing
Designate is pluggable with many AutX management backends (we use ActiveDirectory and a local database)
Horizon
Horizon provides the OpenStack Dashboard as a webinterface. It is decoupled from other OpenStack services as such that no other component depends on it. It acs as
- Alternative to CLI
- Web based user interface to OpenStack services
Horizon itself is Not mandatory for operations. Everything Horizon can do, can be done via APIs and/or the CLI.
Takeaway
- OpenStack components are strictly decoupled
- Components interact with each other
- Every component utilizes lower level (virtualization) technologies and provides an API
OpenStack technology options are interchangeable
All components are developed individually by different teams. This is clearly perceivable in minor difference like code architecture and configuration options.
Improved Workflow
Extending the workflow from above and extending it with the components just discussed, this workflow graph can be extended as follows.
The wokflow highlights the interactions between components and gives a simplified overview of the lifecycle of a VM creation. We can see that keystone acts as entrypoint to every other component. With the key provided by keystone subsequent mechanisms can be trigger.
In this worklflow first nova is asked to create a VM. Nova subsequently interacts with glance to get an appropriate image. Glance possibly interacts with Ceph as it acts as storage backend for the images. Once the VM including the image is started, neutron create the necessary networking components. This typically includes a service for name resolution and ip address distribution. Once this was successfull, neutron notifies designate so that it can create the respective name entries for DNS resolution. Finally possibly configured disks as ceph volumes are attached to VM via cinder to be available as additional block device. Theoretically speaking, this step could occur further, if the user decides to have his root filesystem on a cinder volume too. Finally, a VM is started and ready to serve as the user requests.
However, this workflow still too superficial, so let's get into detail
- What about the actual compute services?
- How are compute and control services coupled?
- How do cinder and ceph relate?
Thus the following briefly reiterates some of the OpenStack services above, to talk more about the specifics:
Nova
Nova separates into the following nested services:
- Nova Control Service
- Scheduler: Implements scheduling decisions. TODO: scheduling docs
- Conductor: Offloads database operations for nova-compute
- Nova Compute Service
- Scaled Out
- Compute: The API component on the computenodes
- Libvirt: The actual Hypervisor
Neutron
Neutron separates into the following nested services:
- Neutron Control Service
- Metadata: Provides cloud information of the VM from inside of it
- DHCP: IP distribution service for subnets
- Bridge & L3 Agent: Routing and switching for VMs
- Neutron Compute Service
- Scaled Out
- Compute: The API component on the computenodes
Cinder
Neutron separates into the following nested services:
- Cinder Control Service
- Scheduler: Schedules software defined storage on physical disks
- Cinder Volume Service
- Scaled Out
- Compute / Storage
Architecture Overview
Finally, the following image gives a more detailled view over OpenStack and its involved components. Of course, the granuality can never be fine enough, however, more details would be lost if taking this approach further. We recommend to study the public OpenStack documentation if wanting to dive deeper into OpenStack specifics.
What is missing?
The following incomplete list briefly discusses some of the auxillary services necessary to operate OpenStack, but are actually not part of OpenStack itself. Please refer to
Message Queue?
TODO: Message Queue Docs
- Responsible for compute & control communication
- Publish and subscribe model to scale according to demand
- Decoupling enables scaling without reconfiguring control
- Acts as compute node registry
Ceph?
TODO: Message Queue Docs
- Not directly part of OpenStack
- Software defined storage via API
- OpenStack cinder has a volume driver for the ceph API
- Ceph slices individual block devices or filesystems out of a bigger block device which are mountable in a target system
State?
- Most components have state induced by users or operators
- OpenStack components have a "companion" database colocated. Each service has its own database
- Horizon is special as it only has a cache attached
Takeaway
- OpenStack services are subdivided
- Nova, Neutron and Cinder consist of control & compute pairs
- Compute parts are scaled across compute, storage or other special nodes
- External services are pluggable (ceph)
Full VM spawn happy path