OpenStack Demonstrates Harmony of Stability and Innovation

The OpenStack community released this week Yoga, the 25th version of the world’s most widely deployed open source cloud infrastructure software. Yoga highlights include support for advanced hardware features such as SmartNIC DPUs, improved integration with cloud-native software such as Kubernetes and Prometheus and reduction of technical debt to maintain a stable and reliable OpenStack core.

OpenStack, the open infrastructure-as-a-service standard, is the one infrastructure platform for deployments of diverse architectures — bare metal, virtual machines (VMs), graphics processing units (GPUs) and containers. Over 12 years, the OpenStack project has maintained a consistent harmony between release stability and a steady pace of new use cases and technical innovation. OpenStack is also more and more adopted by cloud providers with a special focus on digital sovereignty and highest security requirements.

With more than 25 million cores in production and over 180 public cloud data centers worldwide running OpenStack, the community has steadily integrated new technologies like Kubernetes over the project’s history, with more than 560,000 changes from over 8,700 contributors from more than 180 countries merged since 2012.

Ninety percent of the world’s largest telcos run OpenStack, and established users continue growing their deployments while new contributors like NVIDIA, the BBC and the European Centre for Medium-Range Weather Forecasts bring new use cases and technologies to the community. All this has happened against a backdrop of consistent usability improvements, enabling deployments sizes that range from a few dozen to millions of cores.

The OpenStack Yoga release is available for download.

Yoga Release Highlights

Over the span of just 25 weeks, almost 13,500 changes authored by over 680 contributors from more than 125 organizations and 44 countries were merged into the Yoga release. Feature advancements in Yoga include:

  • Hardware enablement extended, specifically for SmartNIC DPUs. Neutron adds support for a remote-managed VNIC type, enabling port binding to SmartNIC DPUs. In addition, Nova now offers support for network backends that leverage SmartNICs to offload the controlplane from the host server. This enables increased security by removing the control plane from the host server and reduced overhead by leveraging the CPU and RAM resources on modern SmartNIC DPUs.
  • Local IP added to Neutron. This feature is primarily focused on high efficiency and performance of the networking data plane for very large-scale clouds, or clouds with high network throughput demands. Local IP is a virtual IP which can be shared across multiple ports or VMs, and guaranteed to only be reachable within the same physical server or node boundaries.
  • Soft delete scheme offered in Manila. File system shares can now be soft-deleted into a recycle bin where they can stay for a configurable amount of time before being purged. While they are in the recycle bin, shares can be viewed and restored on demand.
  • Cloud-native compatibility expanded for Prometheus and Kubernetes.
    • Prometheus integration: Octavia load balancers now support deep observability by adding listeners that expose a Prometheus exporter endpoint. The Octavia amphora provider exposes over 150 unique metrics. Kolla adds support for deploying Prometheus Libvirt exporter.
    • Kubernetes integration: Kuryr adds enhanced debugging capabilities by including Kubernetes events to resources managed by Kuryr. Tacker introduces several new features to its Kubernetes Virtualized Infrastructure Manager (VIM), including using Docker private registry images or Helm charts to deploy Container Network Functions (CNFs).

Routine maintenance and updates in Yoga includes:

  • In Ironic, the default deployment boot mode has changed from Legacy BIOS to UEFI.
  • Cinder adds new backend drivers: Lightbits LightOS for NVMe/TCP, a TOYOU NetStor Fibre Channel driver and NEC V Series Storage drivers (FC and iSCSI). Current backend storage drivers now have added support for features exceeding the required driver functions, for example, Active/Active replication.
  • In Kolla, binary images are deprecated, and any support for them will be removed in the next release. Users are requested to migrate to source based images.

For a more detailed list of Yoga release features, please see the release notes.

Open Infrastructure Summit is Live and In Person This Year

Held in person for the first time since November 2019, the Summit takes place June 7-9, 2022, at the Berlin Congress Center. Attendees will collaborate directly with the people building and running open source infrastructure using OpenStack, Kubernetes and 30+ other technologies. Speakers will include Bloomberg, BMW, Volvo, Adobe and more. The full schedule is live, and sponsorships and registration have limited availability.

 

As mentioned, the BBC adopted OpenStack. They have built and deployed two OpenStack cloud environments based in on-premise data centers, which staff in the department can use for research purposes.

Background: Data centers are widely known to be energy-intensive and were estimated to form around 1% of global electricity consumption in 2018. The first step in reducing the environmental impact of data centers and cloud deployments is to measure it – and increasingly, cloud users want to understand their individual impact. Providing this information to users can motivate them to reduce inefficient usage – for example, by avoiding unnecessary computations.

Therefor they used Ansible to deploy the Scaphandre package to all compute hosts and ran Scaphandre’s Prometheus exporter. This time series data provides the power consumption of the server’s CPU and the CPU power consumption of each process.

In the time series data, the CPU power consumption of VM processes is labelled with the VM UUID. BBC R&D enriched this data using the OpenStack API to add labels for the VM name, project name and flavour details, creating a new Prometheus exporter. Finally, the carbon emissions per VM are calculated by multiplying the power consumption data by the Carbon Intensity data. They now have a dataset that has appropriate metrics and labelling to allow reporting queries to be performed.

They then visualized this data using Grafana dashboards, accessible to users of their cloud deployment. These visualizations included graphs of CPU power usage and carbon dioxide emissions for each VM over time. It is clear that carbon dioxide emissions are not directly proportional to power usage; this is because the carbon intensity of the UK power grid varies significantly according to the time of day.

BBC created leaderboards to allow users to compare their VMs to others doing similar tasks but achieving significantly lower emissions. For example, one team discovered high carbon emissions from a test Kubernetes cluster hosting a simple web service. The energy overhead of running the Kubernetes cluster was far greater than that of the application itself and seeing this data prompted them to decommission the test cluster.

More Information here.

Leave a Reply