[Container & K8s security]Containerize Applications and Deploy them using Kubernetes with security best practices

Overview

In this article, We will containerize the applications, run and verify the stack, and then tag and push the built images to a registry. Then, The containerized application will be deployed and ran in Kubernetes.

Then, we will cover the security best practices.

Note

This is for beginner and some security things such as hardcorded password are used. But keep in mind that this is just a demo and not for the security purpose. All the contents come from

Interactive Introduction to Containers and Kubernetes

What are cloud native principles

• Container packaged (isolated unit of work that does not require OS dependencies)
• Dynamically managed (actively scheduled and managed by an orchestration process)
• Microservice oriented (loosely coupled with dependencies explicitly described)

deploying web app using Docker

Docker security best practices

• create minimal images to reduce attack surface
• use multi-stage builds to reduce size
• run static scan for vulnerabilities (output of static scan)

Build Docker image for frontend application

Step 1: Analyze Dockerfile for your frontend application

Let’s inpsect the Dockerfile

cd gowebapp/gowebapp
vi Dockerfile

The inside

FROM ubuntu
COPY ./code /opt/gowebapp
COPY ./config /opt/gowebapp/config
EXPOSE 8080
USER 1000
WORKDIR /opt/gowebapp/
ENTRYPOINT ["/opt/gowebapp/gowebapp"]

Build the gowebapp image locally.

docker build -t gowebapp:v1 .

Build Docker image for backend application

Step 1: Analyze Dockerfile for your backend application

This Dockerfile contains the instructions to tell the Docker build engine how to create an image for the backend MySQL database.

vi Dockerfile
FROM mysql:5.6

USER 1000

COPY gowebapp.sql /docker-entrypoint-initdb.d/

Step 2: Build gowebapp-mysql Docker image locally

docker build -t gowebapp-mysql:v1 .

Run and test Docker images locally

Before deploying to Kubernetes, let’s run and test the Docker images locally, to ensure that the frontend and backend containers run and integrate properly.

Step 1: Create Docker user-defined network

Facilitate cross-container communication, let’s first define a user-defined network in which to run the frontend and backend containers

~] $ docker network create gowebapp
285d7ea367b39fb5b27b4c759ce9a236ec2183f5245bee5f5dd5205f911241ed

Step 2: Launch frontend and backend containers

Next, let’s launch a frontend and backend container using the Docker CLI.

backend

>launched the database container, as it will take a bit longer to startup, and the frontend container depends on it. Notice how we are injecting the database password into the MySQL configuration as an environment variable:

[~] $ docker run --net gowebapp --name gowebapp-mysql --hostname gowebapp-mysql -d -e MYSQL_ROOT_PASSWORD=mypassword gowebapp-mysql:v1

2a7fe4653046a5d51679e7a8219708582e27b01ab88035c3d030d0265c340f59

frondend
>launched a frontend container, mapping the container port 8080 - where the web application is exposed - to port 8080 on the host machine:

docker run -p 8080:8080 --net gowebapp -d --name gowebapp --hostname gowebapp gowebapp:v1

e01a5f6eaa8721bf35cb05e0dbd57686b0598de68a20682e4dcbcb8fafbeffd6

Step 3: Test the application locally

Now that we’ve launched the application containers, let’s try to test the web application locally.

We should be able to access the application by running the following command in the terminal and then clicking the URL it produces.

echo "http://$SESSION_NAME-gowebapp-docker.$INGRESS_DOMAIN"

Create an account and login. Write something on your Notepad and save it. This will verify that the application is working and properly integrates with the backend database container.

Step 4: Inspect the MySQL database

Let’s connect to the backend MySQL database container and run some queries to ensure that application persistence is working properly:

docker exec -it gowebapp-mysql mysql -u root -pmypassword gowebapp

Once connected, run some simple SQL commands to inspect the database tables and persistence:

SHOW DATABASES;
USE gowebapp;
SHOW TABLES;
SELECT * FROM <table_name>;
exit;

Step 5: Cleanup application containers

When we’re finished testing, we can terminate and remove the currently running frontend and backend containers from our local machine:

docker rm -f gowebapp gowebapp-mysql

1.6. Create and push Docker images to Docker registry

Step 1: Tag images to target another registry

We are finished testing our images. We now need to push our images to an image registry so our Kubernetes cluster can access them. First, we need to tag our Docker images to use the registry in your lab environment:

docker tag gowebapp:v1 $REGISTRY_HOST/gowebapp:v1

docker tag gowebapp-mysql:v1 $REGISTRY_HOST/gowebapp-mysql:v1

Step 2: publish images to the registry

docker push $REGISTRY_HOST/gowebapp:v1
docker push $REGISTRY_HOST/gowebapp-mysql:v1

Deploying web app to k8s

K8s security best practices

• run latest K8s version

[~] $ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.11+vmware.1", GitCommit:"581335ed759cada5037cc08960ce95d73170087f", GitTreeState:"clean", BuildDate:"2021-09-17T00:48:38Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}

• run kube-bench (scan) for CIS benchmark periodically

The output of kube-bench

remediations

• Enable audit logs

k8s application security best practice

• use namespaces to divide the cluster (by default, all pods can talk each other)
• separate resource quota and access for each namespace
• use network policy to control pod traffic by IP, label or namespace
• Implement RBAC(Role Based access control) such as admin, developer
• Do not allow privileged escalation

2.1. Getting Started with kubectl

Step 1: use kubectl to understand an object

Use explain to get documentation of various resources. For instance pods

[~] $ kubectl explain pods
KIND:     Pod
VERSION:  v1
DESCRIPTION:
     Pod is a collection of containers that can run on a host. This resource is
     created by clients and scheduled onto hosts.
FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

Step 2: get more information on an object

kubectl describe <resource name>

2.2. Create Service Object for MySQL

Step 1: Analyze a Kubernetes Service object for the backend MySQL database

Let’s inpsect the gowebapp-mysql-service.yaml file.

[~/gowebapp/gowebapp-mysql] $ cat gowebapp-mysql-service.yaml 
apiVersion: v1
kind: Service
metadata: 
  name: gowebapp-mysql
  labels:
    run: gowebapp-mysql
    tier: backend
spec:
  type: ClusterIP
  ports:
  - port: 3306
    targetPort: 3306
  selector:
    run: gowebapp-mysql
    tier: backend

Step 2: create a Service defined above

Use kubectl to create the Service defined above

[~/gowebapp/gowebapp-mysql] $ kubectl apply -f gowebapp-mysql-service.yaml
service/gowebapp-mysql created

Step 3: test to make sure the Service was created

~/gowebapp/gowebapp-mysql] $ kubectl get service 
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
gowebapp-mysql   ClusterIP   10.101.167.242   <none>        3306/TCP   63s

2.3. Create Deployment Object for MySQL

Step 1: Analyze a Kubernetes Deployment object for the backend MySQL database

[~/gowebapp/gowebapp-mysql] $ cat gowebapp-mysql-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: gowebapp-mysql
  labels:
    run: gowebapp-mysql
    tier: backend
spec:
  replicas: 1
  strategy: 
    type: Recreate
  selector:
    matchLabels:
      run: gowebapp-mysql
      tier: backend
  template:
    metadata:
      labels:
        run: gowebapp-mysql
        tier: backend
    spec:
      containers:
      - name: gowebapp-mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: mypassword
        image: gowebapp-mysql:v1 # here
        ports:
        - containerPort: 3306

We are using a custom image that we created in the previous section. Therefore we need to add the registry server to the image: line in the YAML so that Kubernetes knows which registry to pull the image from. Otherwise it will try to find the image on the public/default configured registry server.

[~/gowebapp/gowebapp-mysql] $ sed -i s/"image: gowebapp"/"image: $REGISTRY_HOST\/gowebapp"/g gowebapp-mysql-deployment.yaml

After

[~/gowebapp/gowebapp-mysql] $ cat gowebapp-mysql-deployment.yaml apiVersion: apps/v1
kind: Deployment
metadata: 
  name: gowebapp-mysql
  labels:
    run: gowebapp-mysql
    tier: backend
spec:
  replicas: 1
  strategy: 
    type: Recreate
  selector:
    matchLabels:
      run: gowebapp-mysql
      tier: backend
  template:
    metadata:
      labels:
        run: gowebapp-mysql
        tier: backend
    spec:
      containers:
      - name: gowebapp-mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: mypassword
        image: eduk8s-labs-w10-s119-registry.kube-prod-blue-524328a.kubeacademy.esp.vmware.com/gowebapp-mysql:v1
        ports:
        - containerPort: 3306

Step 2: create the Deployment defined above

Use kubectl to create the Deployment defined above

kubectl apply -f gowebapp-mysql-deployment.yaml
deployment.apps/gowebapp-mysql created

Step 3: test to make sure the Deployment was created

[~/gowebapp/gowebapp-mysql] $ kubectl get deployment
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
gowebapp-mysql   1/1     1            1           45s

2.4. Create Service Object for frontend application: gowebapp

Step 1: Analyze a Kubernetes Service object for the frontend gowebapp

Let’s inpsect the gowebapp-service.yaml file.

[~/gowebapp/gowebapp] $ cat gowebapp-service.yaml
apiVersion: v1
kind: Service
metadata: 
  name: gowebapp
  labels:
    run: gowebapp
    tier: frontend
spec:
  type: NodePort
  ports:
  - port: 8080
  selector:
    run: gowebapp
    tier: frontend

Step 2: create a Service defined above

[~/gowebapp/gowebapp] $ kubectl apply -f gowebapp-service.yaml 
service/gowebapp created

Step 3: test to make sure the Service was created

[~/gowebapp/gowebapp] $ kubectl get service
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
gowebapp         NodePort    10.109.99.19     <none>        8080:30636/TCP   31s
gowebapp-mysql   ClusterIP   10.101.167.242   <none>        3306/TCP         12m

2.5. Create Deployment Object for gowebapp

Step 1: Analyze a Kubernetes Deployment object for the frontend gowebapp

Let’s inpsect the gowebapp-deployment.yaml file

~/gowebapp/gowebapp] $ cat gowebapp-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: gowebapp
  labels:
    run: gowebapp
    tier: frontend
spec:
  replicas: 2
  selector:
    matchLabels:
      run: gowebapp
      tier: frontend
  template:
    metadata:
      labels:
        run: gowebapp
        tier: frontend
    spec:
      containers:
      - name: gowebapp
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: mypassword
        image: gowebapp:v1       
        ports:
        - containerPort: 80

same action in the previous step

[~/gowebapp/gowebapp] $ sed -i s/"image: gowebapp"/"image: $REGISTRY_HOST\/gowebapp"/g gowebapp-deployment.yaml

After

[~/gowebapp/gowebapp] $ cat gowebapp-deployment.yaml apiVersion: apps/v1
kind: Deployment
metadata: 
  name: gowebapp
  labels:
    run: gowebapp
    tier: frontend
spec:
  replicas: 2
  selector:
    matchLabels:
      run: gowebapp
      tier: frontend
  template:
    metadata:
      labels:
        run: gowebapp
        tier: frontend
    spec:
      containers:
      - name: gowebapp
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: mypassword
        image: eduk8s-labs-w10-s119-registry.kube-prod-blue-524328a.kubeacademy.esp.vmware.com/gowebapp:v1       
        ports:
        - containerPort: 80

Step 2: create the Deployment defined above

[~/gowebapp/gowebapp] $ kubectl apply -f gowebapp-deployment.yaml
deployment.apps/gowebapp created

Step3: check

[~/gowebapp/gowebapp] $ kubectl get deployment
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
gowebapp         2/2     2            2           32s
gowebapp-mysql   1/1     1            1           9m7s
[~/gowebapp/gowebapp] $

2.6. Test Your Application

Step 1: Access gowebapp through the Service

we should be able to access the application

[~/gowebapp/gowebapp] $ echo "http://$SESSION_NAME-gowebapp-k8s.$INGRESS_DOMAIN"
http://eduk8s-labs-w10-s119-gowebapp-k8s.kube-prod-blue-524328a.kubeacademy.esp.vmware.com