Kubernetes' emergence as the de facto operating system of the cloud is upending conventional roles and processes, and it has also introduced new risks. However, I've come to realize many CISOs have limited experiences with Kubernetes and don't understand that securing these environments requires a different approach.
Kubernetes makes it easier to create and manage container components, link services, automate scale and failover, and tear down when containers outlive their usefulness. These are great benefits for developers, but they create challenges for security and operations. Containers are black boxes that work well as application building blocks but block visibility.
I consistently speak to the Global 2000 as they transition to the cloud, and from my experience, many cloud teams address operations challenges as they architect these environments but delay bringing in security until workloads are ready for production. They assume security teams will slow their march to cloud, but in reality, addressing security late delays scaling production.
Who owns Kubernetes' security?
Historically, enterprise infrastructure introduction involved lengthy discussions and analysis, but containerization started differently.
Container adoption typically involves an advanced cloud or platform team experimenting with containers and cloud. They get a small budget and build an environment, and other developers and app teams eventually join in.
The group that brings in Kubernetes ends up tending to overall health, including performance and security. As they move critical applications to production, specialists are brought in to develop compliance and security processes.
The initial thought is often to apply legacy tools or acquire a collection of random tools that address a single use case. To realize the agility benefits of the cloud, they need an approach that integrates security and compliance across the build, run and respond workflows.
Because CISOs are brought in at a later stage, they may not understand how Kubernetes' security is fundamentally different and how these differences shift responsibilities. Specific security functions require a deep understanding of service interactions.
In the build stage, the development and operations workflow should include resolving vulnerabilities, checking compliance and validating security policies. The DevOps team should triage alerts to assess if there are configuration or security issues and work with the security team on forensics. Security teams should provide guidance as these processes are defined and ensure proper controls are in place. If DevOps and security teams work together to architect the approach, organizations will be able to confidently deploy production workloads.
For organizations beginning their Kubernetes journey, it's not too late. CISOs need to understand where and how Kubernetes is used within their company, who has assumed ad hoc security roles, what new security tools have been acquired and how they are being used, and how risk is being managed.
What can CISOs do?
• Get involved early. Get involved during the planning stage. Don't wait until developers are ready to deploy in production. By understanding how securing Kubernetes is different, CISOs can ensure the integrity of their environment by properly overseeing workflow development and tool evaluation.
• Embed security into the DevOps life cycle. A secure DevOps approach is the only way to manage risk without slowing the CI/CD pipeline. With cloud computing, everything is in software. Developers building cloud applications access resources and services via APIs, including security controls. As a result, security must be addressed earlier in the development cycle and integrated into DevOps.
A secure DevOps approach includes:
1. Predeployment vulnerability scanning and configuration validation. The majority of breaches, such as the Ecuador and Malindo Air breaches, are due to misconfigurations or vulnerabilities not discovered in a timely fashion. The scale, complexity and elasticity of these environments require tools that enable teams to prioritize vulnerabilities and confirm that configurations are compliant.
2. Runtime protection. Many edge container security tools are focused on the integrity of container images. While important, when scaling Kubernetes environments, you'll need to spend more time managing runtime policies and responding to alerts. Container environments grow and change quickly, which requires updating and tuning policies and addressing issues. DevOps teams have an understanding of intended service interactions that are required to triage alerts.
3. Capturing events and context for forensics and incident response. The ephemeral nature of containers can be a challenge when reconstructing events to conduct forensics and incident response. If something occurs inside a container, there's a good chance Kubernetes already killed off the container without that data. How do you conduct forensics, a postmortem audit or incident response? How do you ensure it won't happen again?
4. Upfront compliance validation and streamlined reporting. It's important to validate compliance predeployment, which includes baking compliance into the CI/CD pipeline, validating CIS compliance during the build and enforcing NIST and PCI compliance at runtime. There are tools that provide out-of-the-box compliance policies, noncompliance alerts, detailed remediation information and compliance records for reporting.
• Don't assume existing tools can adapt. Enterprises operating cloud-native need tools that fit with the secure DevOps workflows and scale for production. You can't defend what you can't see, and existing tools don't provide visibility into containers, nor can they provide context to help understand what's going on. Whether there is a vulnerability in the container image, a misconfiguration or suspicious activity that requires investigation, without visibility into containers, you can't secure them, which is a major barrier to scaling.
Many security teams try to retrofit existing tools, using, say, virtual firewalls in the cloud to safeguard containers. But edge firewalls can't see interactions between services, making it impossible to detect potentially malicious activity.
The good news is that Kubernetes simplifies shifting security left, building more security checks into development so that the final code is more secure. However, Kubernetes on its own doesn't protect against vulnerabilities, misconfigurations and runtime attacks without outside help. Security teams need to embrace Kubernetes and understand process, workflow and tooling changes.
There's no going back. Adoption of Kubernetes means DevOps teams need to shoulder security responsibilities, and that’s a good thing. The more people across organizations who care about security, the better.