Kubernetes is an opensource platform for automating deployment, scaling, and management of containerized applications. According to the data from GitHub, Kubernetes represents one of the largest opensource projects worldwide, and the development and deployment of tools around Kubernetes is rapidly and continuously growing. This rapid and continuous evolution makes Kubernetes an unsteady, fastmoving target, in particular regarding security.
Understanding and implementing security in such a distributed, highly scalable environment is challenging, because it requires a holistic and thorough analysis in different domains, like infrastructure, platform, networking, containers (applications), and governance. This article aims to address security challenges regarding Kubernetes in the domains of platformhardening and networking, by describing the interplay of three different and independent Cloud Native solutions, namely Cilium, Istio and Octarine. The first part of this article introduces and describes the puzzle pieces, i.e. Cilium, Istio and Octarine, and identifies their main security features. The second part of this article outlines the interplay of these solutions, i.e. it shows how these solutions complement each other, and how this improves security of Kubernetes clusters. Last, but not least, the article provides an outlook with respect to next steps and further security considerations.
The puzzle pieces
This section introduces and describes the puzzle pieces, namely Cilium, Istio and Octraine. This introduction provides a general overview and serves as background. For more specific information regarding the presented solutions, the reader is encouraged to consult the corresponding project website and documentation.
This section provides a general overview on Cilium.
What is Cilium?
Cilium is an opensource project for securing the network connectivity between applications deployed as Linux containers, for instance by using Kubernetes. At the foundation of Cilium lies a technology rooted in the Linux kernel, namely the extended Berkley Packet Filter (eBPF). This enables Cilium to enforce security policies without the necessity to change the application code or container configuration.
Fig. 1: Cilium components (source: https://cilium.readthedocs.io/en/stable/concepts/overview/)
Cilium consist of different components, shown in Figure 1, that run in the cluster:
– Agent: The agent is a userspace daemon that interacts with the container management platform, e.g. Kubernetes, to setup security on the network layer for all containers. The agent provides an API, which among others it enables the configuration of network security policies, and the visibility of network traffic.
– CLI Client: A command line tool installed along the Cilium Agent, enabling the interaction with the Cilium Agent API. This includes, checking the state of each network endpoint,
i.e. container, configuring and visualizing network security policies, and configuring network monitoring behaviour.
– Linux Kernel BPF: Berkeley Packet Filter (BPF) is bytecode interpreter rooted in the Linux kernel that was originally introduced to filter network packets. Since its introduction BPF has been extended with additional data structures, e.g. arrays, as well as additional functionality to support packet mangling, forwarding, encapsulation etc. BPF programs can be run at various hooking points in the kernel such as for incoming packets, outgoing packets, system calls, and kprobes. Cilium leverages BPF to provide visibility and control over all inbound/outbound network traffic for all containers.
– Container Platform Network Plugin: Each container management platform, e.g. Kubernetes or Docker, provides a plugin inter face for how external networking platforms integrate. Thus, this component enables the integration of Cilium in Kubernetes.
– Key-Value Store: Cilium shares data between the Cilium Agents via a keyvalue store, e.g. etcd in Kubernetes.
– Operator: This is a daemon which enables cluster management tasks to be handled once per cluster, rather than once per node.
Cilium provides security on multiple layers, namely 3, 4 and 7. The security features provided by Cilium, which can be used individually or combined, can be classified as follows:
– Securing APIs based on Layer 3, 4 and 7 net work security policies.
– Securing servicetoservice communication based on identities.
– Securing access to and from external services based on CIDR (Layer 3) network security policies.
– Securing inbound/outbound network traffic based on TLS.
This section provides a general overview on Istio.
What is Istio?
Istio is an opensource project providing service mesh capabilities. The term service mesh refers to a network of microservices and their interactions. The requirements on a service mesh usually include dynamic service discovery, load balancing, failure recovery, metrics, and monitoring. In addition, more complex requirements include A/B testing, canary rollouts, rate limiting, access control, and endtoend authentication. Istio provides runtime metrics and operational control over a service mesh, and with its capabilities it addresses the diverse requirements of microservice architectures.
Fig. 2: Istio Components (source: https://istio.io/docs/ops/deployment/architecture/arch.svg)
Istio consists of different components, shown in Figure 2, that run in the cluster:
– Proxy: The proxy is based on an extended version of the Envoy proxy. It mediates the inbound/outbound network traffic for all services in the service mesh. Proxies are deployed as sidecars to services, logically extending them with capabilities, like service discovery, TLS termination, and metrics. This model enables deployment without the necessity to change the application code or container configuration.
– Mixer: A modular and extensible component, responsible for providing network policy control and telemetry collection. Mixer includes a flexible plugin model, enabling Istio to interface with different host environments and infrastructure backends, like access control, telemetry capturing, quota enforcement, and billing.
– Pilot: Pilot provides service discovery for the Envoy sidecars, traffic management capabilities for intelligent routing, and resiliency. Pilot translates highlevel routing rules into Envoyspecific configurations, and propagates these to the sidecars at runtime. In addition, Pilot abstracts platformspecific service discovery for the sidecars.
– Citadel: Provides key and certificate management. It enables servicetoservice and enduser authentication with builtin identity and credential management. Thus, network policies can be enforced based on service identity rather than on layer 3/4 network identifiers.
– Galley: This component enables the rest of the Istio components to abstract from the details of the underlying platform, e.g. Kubernetes, and obtaining corresponding configurations. It provides configuration validation, ingestion, processing and distribution.
Istio provides security features to address challenges in microservice architectures, such as protection against ManInTheMiddle attacks, flexible service access control, and auditing. The security features provided by Istio, which can be used individually or combined, can be classified as follows:
– Securing identities for human users, services or group of services based on X.509 certificates.
– Securing servicetoservice and enduserto service communication, i.e. providing encryption in transport and origin authentication, based on mutual TLS.
– Securing access control on meshlevel, namespacelevel, and workloadlevel based on its authorisation mechanism.
– Securing inbound/outbound traffic via TLS.
– Securing operations by using the corresponding adapters in the Mixer component.
This section provides a general overview on Octarine.
What is Octarine?
Octarine is a Cloud Native security platform that secures containers (applications) through out their lifecycle, meaning from build to run. Octarine provides runtime visibility, policy management, threat prevention, and container security across virtualized and containerized environments. Octarine can be deployed onpremises or as SaaS.
Fig. 3: Octarine Components (product overview, source: https://www.octarinesec.com/resources/#data-sheet)
Octarine consists of different components, shown in Figure 3:
– Controller: The controller based on the capabilities of its subcomponents, namely Visibility, Security and Policy engine, provides visibility, monitors and highlights changes to the security configuration for all clusters. It can also enforce security policies and validate changes to ensure compliance.
– Visibility Engine: Collects and analyses metadata from the OctaGuard giving insight regarding container activity. The collected data is stored in a time series database. In addition, it audits the Kubernetes and container configurations, and highlights potential vulnerabilities based on security best practices from the Center of Internet Security benchmarks.
– Security Engine: Monitors the Kubernetes security configurations across clusters and validates changes to prevent malicious actions. In addition, it checks for patterns to identify anomalies, e.g. zeroday attacks. The collected threat intelligence is shared with the Policy Engine to ensure continuous security and compliance.
– Policy Engine: Manages and recommends policies. The recommended policies are automatically generated based on the learned, realtime behaviour. This proactively protects the environment from new emerging threats. Policies are continuously distributed to the OctaGuard.
– GuardRail: This component provides policybased reports and enforces the corresponding security related configurations across all workloads deployed in Kubernetes clusters.
– OctaGuard: This component, which is based on Envoy proxy, collects metadata and enforces policies generated by the Controller to ensure that all activities are authenticated
and authorized, and thus potential threats are mitigated.
The security features provided by Octarine address security challenges at the network, as well as at the platformhardening domain. As Octarine supports Istio natively (the core of Istio, namely the Envoy proxy), only additional security features are considered here:
– Securing network activity with an anomalybased Intrusion Detection System.
– Securing Kubernetes and Istio clusters by managing policies in one place.
– Securing runtime by automated policy and configuration enforcement, as well as by realtime, recommended improvements.
Putting the puzzles together
Based on the identified and provided security features, and independent of their different focus domain these solutions seem to share similar security features. Nevertheless, the following comparison shows how they differ:
– Cilium, in comparison to Istio, secures nonIPv4/TCP protocols.
– Cilium, in comparison to Istio, protects against compromised sidecars (Envoy proxies), as these are not subject of the Istio security policies.
– Cilium, in comparison to Istio, by using its flexible BPF, can enforce application traffic through sidecars, and also the principle of least privilege at the container and/or process level inside the Pod.
– Istio, in comparison to Cilium, provides strong authentication and encryption based on X.509 certificates, and mutual TLS for servicetoservice and endusertoservice communication.
– Octarine, in comparison to Cilium and Istio, provides security at the platformhardening domain by providing runtime insights and enforcing secure configurations based on security best practices.
– Octarine, in comparison to Cilium and Istio, provides threat detection at the network domain through an anomalybased Intrusion Detection System.
However, it is important to emphasize that these differences represent a significant addedvalue regarding security. When combined these solutions provide complementary security features, and hence a greater set of such features. Consequently, their interplay improves the security of Kubernetes clusters at the network and platformhardening domain.
Even though the combination of these solutions enhances the security for Kubernetes clusters, further security challenges, for instance security of container images, need to be considered.
Based on the presented findings, next steps include the implementation of use cases that demonstrate the security improvements of combining Cilium, Istio and Octarine in practice.
– https://kubernetes.io, last accessed 20.02.2020
– https://github.com, last accessed 20.02.2020
– https://cilium.io, last accessed 20.02.2020
– https://istio.io, last accessed 20.02.2020
– https://www.envoyproxy.io, last accessed 20.02.2020
– https://tools.ietf.org/html/rfc5280, last accessed 20.02.2020
– https://tools.ietf.org/html/rfc8446, last accessed 20.02.2020
– https://www.octarinesec.com, last accessed 20.02.2020
– https://www.cisecurity.org, last accessed 20.02.2020
Dr. Jurlind Budurushi
Chief Cyber Security Officer