Everything Kubernetes

EVERYTHING KUBERNETES: A P R AC ACT TICAL GUIDE

CONTENTS 3

INTRODUCTION

4

KUBERNETES — BIRD’S EYE VIEW HIGH LEVEL ARCHITECTURE

6

KUBERNETES BUILDING BLOCKS

6

THE BASICS BLOCKS

8

USING LABELS AND SELECTORS FOR FINE-GRAINED CONTROL

8

SERVICE DISCOVERY

9

3 STORAGE BUILDING BLOCKS

10

CHOOSING THE RIGHT BLOCK FOR THE JOB

12

IMPERATIVE VS. DECLARATIVE ORCHESTRATION

13

HANDS-ON: GETTING STARTED STARTED

14

INSTALLATION

18

LOGGING

19

MONITORING

19

WORKING WITH MULTIPLE CLUSTERS

21

HANDS-ON: DEPLOYING AN APPLICATION

33

DIY CLUSTER CONSIDERATIONS

35

SUMMARY

36 36

ABOUT STRATOSCALE STRATOSCALE USING KUBECTL CLI

CONTENTS 3

INTRODUCTION

4

KUBERNETES — BIRD’S EYE VIEW HIGH LEVEL ARCHITECTURE

6

KUBERNETES BUILDING BLOCKS

6

THE BASICS BLOCKS

8

USING LABELS AND SELECTORS FOR FINE-GRAINED CONTROL

8

SERVICE DISCOVERY

9


10

CHOOSING THE RIGHT BLOCK FOR THE JOB

12

IMPERATIVE VS. DECLARATIVE ORCHESTRATION

13

HANDS-ON: GETTING STARTED STARTED

14

INSTALLATION

18

LOGGING

19

MONITORING

19

WORKING WITH MULTIPLE CLUSTERS

21

HANDS-ON: DEPLOYING AN APPLICATION

33

DIY CLUSTER CONSIDERATIONS

35

SUMMARY

36 36

ABOUT STRATOSCALE STRATOSCALE USING KUBECTL CLI

INTRODUCTION Kubernetes is an open-source, container management

deployment, but also for managing mulple containers

soluon originally announced by Google in 2014.

as a single enty for the purposes of scaling,

Aer its inial release in July 2015, Google donated

availability, and so on.

Kubernetes to the Cloud Nave Compung Foundaon. Since then, several stable versions have been released

Being infrastructure agnosc, Kubernetes clusters can

under Apache License.

be installed on a variety of public and private clouds (AWS, Google Cloud, Azure, OpenStack) OpenStack) and on bare

For a developer, Kubernetes provides a manageable

metal servers. Addionally, Google Container Engine

execuon environment for deploying, running,

can provide a deployed Kubernetes cluster. This makes

managing, and orchestrang containers across clusters

Kubernetes similar to Linux kernel, which provides

or clusters of hosts. For devops and administrators,

consistency across dierent hardware plaorms, or

Kubernetes provides a complete set of building

Java, which runs on almost any operang system.

blocks that allow the automaon of many operaons for managing development, test, and producon environments. environment s. Container orchestraon enables coordinang containers containers in clusters consisng of mulple nodes when complex containerized applicaons are deployed. This is relevant not only for the inial

3

Everything Kubernetes: A Praccal Guide

Stratoscale

KUBERNETES — HIGH LEVEL ARCHITECTURE NODE A Kubernetes cluster consists of one or more nodes managed by Kubernetes. The nodes are bare-metal servers, on-premises VMs, or VMs on a cloud provider. Every node contains a container runme (for example, Docker Engine), kubelet (responsible for starng, stopping, and managing individual containers by requests from the Kubernetes control plane), and kube-proxy (responsible for networking and load balancing).

MASTER NODE A Kubernetes cluster also contains one or more master nodes that run the Kubernetes control plane. The control plane consists of dierent processes, such as an API server (provides JSON over HTTP API), scheduler (selects nodes to run containers), controller manager (runs controllers, see below), and etcd (a globally available conguraon store).

DASHBOARD AND CLI A Kubernetes cluster can be managed via the Kubernetes Dashboard, a web UI running on the master node. The cluster can also be managed via the command line tool kubectl, which can be installed on any machine able to access the API server, running on the master node. This tool can be used to manage several Kubernetes clusters by specifying a context dened in a conguraon le.

4


Stratoscale

Internet

Node

Kublet

Proxy

Kubec(CLI)

Docker

Master Node

Pod

Pod

Authencaon Authorizaon

API's

Container

Scheduler

Controller Manager

Distributed Storage

Container

Node

Scheduler Kublet

Proxy

Docker Pod

Pod

Container

5


Container

Stratoscale

KUBERNETES

BUILDING BLOCKS Kubernetes provides basic mechanisms for the deployment, maintenance, and scaling of containerized applicaons. It uses declarave primives, or building blocks, to maintain the state requested by the user, implemenng the transion from the current observable state to the requested state.

THE BASICS POD

A pod is the smallest deployable unit that can be managed by Kubernetes. A pod is a logical group of one or more containers that share the same IP address and port space. The main purpose of a pod is to support co-located processes, such as an applicaon server and its local cache. Containers within a pod can nd each other via localhost, and can also communicate with each other using standard inter-process communicaons like SystemV semaphores or POSIX shared memory. In other words, a pod represents a “logical host”. Pods are not durable; they will not survive scheduling failures or node failures. If a node where the pod is running dies, the pod is deleted. It can then be replaced by an idencal pod, with even the same name, but with a new unique idener (UID).

LABEL

SELECTOR

A label is a key/value pair that is aached to

A label selector can be used to organize Kubernetes

Kubernetes resource, for example, a pod. Labels

resources that have labels. An equality-based selector

can be aached to resources at creaon me, as

denes a condion for selecng resources that have

well as added and modied at any later me.

the specied label value. A set-based selector denes a condion for selecng resources that have a label value within the specied set of values.

6


Stratoscale

CONTROLLER

REPLICATION CONTROLLER

A controller manages a set of pods and ensures that

A replicaon controller is responsible for running the

the cluster is in the specied state. Unlike manually

specied number of pod copies (replicas) across the

created pods, the pods maintained by a replicaon

cluster.

controller are automacally replaced if they fail, get deleted, or are terminated. There are several controller types, such as replicaon controllers or deployment controllers.

DEPLOYMENT CONTROLLER

REPLICA SET

A deployment denes a desired state for logical

A replica set is the next-generaon replicaon

group of pods and replica sets. It creates new

controller. A replicaon controller supports only

resources or replaces the exisng resources, if

equality-based selectors, while a replica set supports

necessary. A deployment can be updated, rolled

set-based selectors.

out, or rolled back. A praccal use case for a deployment is to bring up a replica set and pods, then update the deployment to re-create the pods (for example, to use a new image). Later, the deployment can be rolled back to an earlier revision if the current deployment is not stable.

SERVICE A service uses a selector to dene a logical group of pods and denes a policy to access such logical groups. Because pods are not durable, the actual pods that are running may change. A client that uses one or more containers within a pod should not need to be aware of which specic pod it works with, especially if there are several pods (replicas). There are several types of services in Kubernetes, including ClusterIP, NodePort, LoadBalancer. A ClusterIP service exposes pods to connecons from inside the cluster. A NodePort service exposes pods to external trac by forwarding trac from a port on each node of the cluster to the container port. A LoadBalancer service also exposes pods to external trac, as NodePort service does, however it also provides a load balancer.

7


Stratoscale

USING LABELS AND SELECTORS FOR FINE-GRAINED CONTROL A Kubernetes controller, for example, uses a selector to dene a set of managed pods so that pods in that set have the corresponding label. A label is just a key/value pair that is aached to Kubernetes resources such as pods. Labels can be aached to resources when they are created, or added and modied at any me. Each resource can have mulple labels. For example: release: stable environment: dev

A label selector denes a set of resources by specifying a requirements for their labels. For example: environment = dev environment != live environment in (dev, test) environment notin (live) release = stable, environment = dev

The rst two selectors have an equality-based requirement, the third and fourth selectors have a set-based requirement. The last selector contains the comma separator, which acts as a logical “AND” operator, so the selector denes a set of resources where the label “release” equals “stable” and the label “environment” equals “dev.”

SERVICE DISCOVERY Kubernetes supports nding a service in two ways: through environment variables and using DNS.

ENVIRONMENT VARIABLES Kubernetes injects a set of environment variables into pods for each acve service. Such environment variables contain the service host and port, for example: MYSQL_SERVICE_HOST=10.0.150.150 MYSQL_SERVICE_PORT=3306

An applicaon in the pod can use these variables to establish a connecon to the service. The service should be created before the replicaon controller or replica set creates a pod’s replicas. Changes made to an acve service are not reected in a previously created replica. DNS

Kubernetes automacally assigns DNS names to services. A special DNS record can be used to specify port numbers as well. To use DNS for service discovery, a Kubernetes cluster should be properly congured to support it.

8


Stratoscale


VOLUME A container le system is ephemeral: if a container crashes, the changes to its le system are lost. A volume is dened at the pod level, and is used to preserve data across container crashes. A volume can be also used to share data between containers in a pod. A volume has the same lifecycle as the the pod that encloses it— when a pod is deleted, the volume is deleted as well. Kubernetes supports dierent volume types, which are implemented as plugins.

PERSISTENT VOLUME A persistent volume represents a real networked storage unit in a cluster that has been provisioned by an administrator. Persistent storage has a lifecycle independent of any individual pod. It supports dierent access modes, such as mounng as read-write by a single node, mounng as read-only by many nodes, and mounng as read-write by many nodes. Kubernetes supports dierent persistent volume types, which are implemented as plugins. Examples of persistent volume types include AWS EBS, vSphere volume, Azure File, GCE Persistent Disk, CephFS, Ceph RBD, GlusterFS, iSCSI, NFS, and Host Path.

PERSISTENT VOLUME CLAIM A persistent volume claim denes a specic amount of storage requested and specic access modes. Kubernetes nds a matching persistent volume and binds it with the persistent volume claim. If a matching volume does not exist, a persistent volume claim will remain unbound indenitely. It will be bound as soon as a matching volume become available.

9


Stratoscale

CHOOSING THE RIGHT BLOCK FOR THE JOB Designed as a simple building block; a replicaon

selectors. From this perspecve, a replica set is just

controller’s only responsibility is to maintain the specied

a more advanced version of a replicaon controller.

number of replicas. A replicaon controller counts only live pods;, terminated pods are excluded. Other

Using only pods and replicaon controllers

Kubernetes building blocks should be used together

to deploy an applicaon is, at least in part, an

with replicaon controllers for more advanced tasks.

imperave form of managing soware, because

For example, an autoscaler can monitor applicaon-

it usually requires manual steps. A Kubernetes

specic metrics and dynamically change the number of

deployment is an alternave that enables

replicas in the exisng replicaon controller. In addion,

completely declarave applicaon deployment.

a replicaon controller does not support scheduling policies, meaning you cannot provide rules for choosing cluster nodes to run pods from the managed set. A replica set is another Kubernetes building block. The major dierence between it and a replicaon controller is that replicaon controllers do not support selectors with set-based requirements, while replica sets support such

SECRET

CONFIG MAP

A Kubernetes secret allows users to pass sensive

A Kubernetes cong map allows users to

informaon, such as passwords, authencaon

externalize applicaon conguraon parameters

tokens, SSH keys, and database credenals, to

from a container image and dene applicaon

containers. A secret can then be referenced when

conguraon details, such as key/value pairs,

declaring a container denion, and read from

directory content, or le content. Cong map

within containers as environment variables or

values can be consumed by applicaons through

from a local disk.

environment variables, local disks, or command line arguments.

10


Stratoscale

JOB A job is used to create one or more pods and ensure that a specied number of them successfully terminate. It tracks the successful compleons, and when a specied number of successful compleons is reached, the job itself is complete. There are several types of jobs, including non-parallel jobs, parallel jobs with a xed compleon count, and parallel jobs with a work queue. A job should be used instead of a replicaon controller if you need to spread pods across cluster nodes and ensure, for example, so that each node has only one running pod of the specied type.

DAEMON SET

NAMESPACE

A daemon set ensures that all or some nodes

A namespace provides a logical paron of the

run a copy of a pod. A daemon set tracks the

cluster’s resources. Kubernetes resources can

addional and removal of cluster nodes and adds

use the same name when found in dierent

pods for nodes that are added to the cluster,

namespaces. Dierent namespaces can be

terminates pods on nodes that are being removed

assigned dierent quotas for resource limitaons.

from a cluster. Deleng a daemon set will clean up the pods it created. A typical use case for a daemon set is running a log collecon daemon or a monitoring daemon on each node of a cluster.

QUOTA

A quota sets resource limitaons, such as CPU, memory, number of pods or services, for a given namespace. It also forces users to explicitly request resource allotment for their pods.

11


Stratoscale

IMPERATIVE VS. DECLARATIVE ORCHESTRATION Before geng to the praccal steps of the Kubernetes

A declarave approach for administrave tasks is

deployment, it’s important to understand the key

intended to solve such challenges. With a declarave

approaches to orchestraon.

approach, an administrator denes a target state for a system (applicaon, server, or cluster). Typically, a

The classic imperave approach for managing soware

domain-specic language (DSL) is used to describe

involves several steps or tasks, some of which are manual.

the target state. An administrave tool, such as

When working in a team, it is usually required that these

Kubernetes, takes this denion as an input and

steps be documented, and, in an ideal case, automated.

takes care of how to achieve the target state from the

Preparing good documentaon for a classic imperave

current observable state.

administrave procedure and automang these steps can be non-trivial tasks, even if each of the steps is simple.

12


Stratoscale

HANDS-ON: GETTING STARTED Minikube is an ideal tool for geng started with

In the following instrucons, Minikube is used

Kubernetes on a single computer. It enables running of

to install a single-node Kubernetes cluster on a

a single-node Kubernetes cluster in a virtual machine.

machine with 64 bit GNU/Linux (Debian or Ubuntu)

It can be used on GNU/Linux or OS X and requires

and KVM. Refer to the Minikube documentaon if

VirtualBox, KVM (for Linux), xhyve (OS X), or VMware

you want to use an alternave conguraon.

Fusion (OS X) to be installed on your computer. Minikube creates a new virtual machine with GNU/Linux, installs and congures Docker and Kubernetes, and nally runs a Kubernetes cluster.

13


Stratoscale

INSTALLATION 1. Install the kubectl command line tool locally:

$ curl -Lo kubectl \ http://storage.googleapis.com/kubernetes-release/\ release/v1.3.0/bin/linux/amd64/kubectl \ && chmod +x kubectl \ && sudo mv kubectl /usr/local/bin/

2. Next, install the KVM driver:

$ sudo curl -L \ https://github.com/dhiltgen/docker-machine-kvm/\ releases/download/v0.7.0/docker-machine-driver-kvm \ -o /usr/local/bin/docker-machine-driver-kvm $ sudo chmod +x /usr/local/bin/docker-machine-driver-kvm

3. Install Minikube:

$ curl -Lo minikube \ https://storage.googleapis.com/minikube/\ releases/v0.6.0/minikube-linux-amd64 \ && chmod +x minikube \ && sudo mv minikube /usr/local/bin/

14


Stratoscale

4. Start the Minikube cluster:

$ minikube start --vm-driver=kvm Starting local Kubernetes cluster... Kubernetes is available at https://192.168.42.213:8443. Kubectl is now congured to use the cluster.

The Kubernetes cluster is up and running. Let’s start with a simple deployment using an exisng image:

$ kubectl run hello-minikube \ --image=gcr.io/google_containers/echoserver:1.4 \ --port=8080 deployment “hello-minikube” created $ kubectl expose deployment hello-minikube --type=NodePort service “hello-minikube” exposed

5. Check that the pod is up and running: $ kubectl get pod NAME

READY

STATUS

hello-minikube-2433534028-ouxw8

RESTARTS 1/1

AGE

Running

0

4m

“Running” should appear in the STATUS eld. If “ContainerCreang” appears instead, wait a few moments, then repeat the last command.

15


Stratoscale

6. Check that the service works: $ curl $(minikube service hello-minikube --url) CLIENT VALUES: client_address=172.17.0.1 command=GET real path=/ query=nil request_version=1.1 request_uri=http://192.168.42.213:8080/ SERVER VALUES: server_version=nginx: 1.10.0 - lua: 10001 HEADERS RECEIVED: accept=*/* host=192.168.42.213:31759 user-agent=curl/7.35.0 BODY: -no body in request-

7. Execute the following command to open the Kubernetes Dashboard in your web browser: $ minikube dashboard

16


Stratoscale

8. To stop the cluster (shut down the virtual machine and preserve its state), execute the following command:

$ minikube stop Stopping local Kubernetes cluster... Stopping “minikubeVM”...

9. To start the cluster again and restore it to the previous state, execute the following command:

$ minikube start --vm-driver=kvm

10. To delete the cluster (delete the virtual machine and its state), execute the following command:

$ minikube delete

Note: There are other open-source tools, such as kubeadm, that simplify installaon of Kubernetes cluster in public clouds, on-premises virtual machines, and bare-metal servers. However, there are sll many things that are out of scope. For example, you sll need a reliable distributed block or le storage, you sll need to think about HA, scalability, networking and security. Somemes, it is simpler to use Kubernetes as a Service.

17


Stratoscale

LOGGING Basic logging in Kubernetes behaves much like logging in Docker. There is a kubectl logs command that will show you all the informaon wrien to stdout and stderr for a given container in a pod. If a pod has only one container, there is no need to specify it explicitly, however when a pod has several containers we need to add -c container to the end of the command. As with Docker, we can opt to follow logs, to reduce the number of recent lines with --tail and we can lter them by date. Unlike Docker, Kubernetes enables us to check the logs of a container that crashed using the --previous opon. The ability to keep the logs of a previous container is available as long as the pod it was run in remains available. When a pod is removed, so are its logs. Log rotaon is performed by Kubernetes. The default values are daily rotaon or 10 MB to avoid log les taking up all the available disk space. Up to ve rotaons are kept for historical evidence. Remember that only the last rotaon is displayed with kubectl logs; if you want to access an earlier one, you must do so manually. Per our example with the Hello Minikube service, we can use kubectl logs hello-minikube2433534028-ouxw8 to view the access log containing our curl request. All informaon so far concerns per-node log les. There are no cluster-level logging capabilies built into Kubernetes, but there are some common methods that can be implemented for this purpose.

DEDICATED AGENT RUNNING ON EVERY NODE In this approach, a logging agent is run on every node, preferably through a DeamonSet replica. A popular choice in this space is uentd. It can be congured with various backends among which are Google Cloud Plaorm and Elascsearch. Fluentd even goes as far as to provide a ready to use DaemonSet YAML on GitHub. All that needs to be done is edit its conguraon to point at our logging backend.

DEDICATED CONTAINER INSIDE A POD There are several use cases for this approach. The general idea is that a dedicated logging component in each pod either writes logs to its stdout and stderr or delivers the logs directly to a logging backend. With such a sidecar approach, we can aggregate all logs from dierent containers in a pod to a single stream that can be accessed with kubectl logs or we can split logs of one applicaon into dierent logical streams. For example, Webservers’ access.log and error.log can each be streamed by a dedicated container to its stdout, so we can check them separately with kubectl logs $podname -c access-log and kubectl logs $podname -c error-log.

DIRECT LOGGING FROM AN APPLICATION If the applicaon running in a pod can already communicate with a logging backend, it is possible to skip the Kubernetes logging opons altogether. While direct logging may oer some performance advantages and provide slightly beer security (all data stored in just one place), it also prevents us from using kubectl logs. In most cases, this approach is discouraged.

18


Stratoscale

MONITORING As Kubernetes containers are actually Linux processes, we can use our favourite tools to monitor cluster performance. Basic tools, such as top or kubectl top, will behave as expected. It’s also possible to use soluons that are dedicated to Kubernetes. One such soluon is Heapster. Heapster aggregates events and data from across the cluster. It runs as a pod in system namespace. Discovery and querying of resources is done automacally with data coming from a kubelet managing node. Storage is congurable with InuxDB and Google Cloud Monitoring being the most popular choices. When building a DIY cluster InuxDB with Grafana for visualizaon is the preferred choice. Using it with Minikube requires two easy steps. First, enable the Heapster addon ( minikube addons enable heapster) by opening the dashboard with Minikube addons. Then open Heapster. By default, dashboards are available for Cluster and for Pods, one for monitoring running nodes and overall cluster ulizaon, the other for running pods. The informaon presented for pods includes CPU usage, memory usage, network usage, and lesystem usage. It is possible to edit exisng graphs or add custom ones based on data exported by Heapster.

WORKING WITH MULTIPLE CLUSTERS So far, we have used kubectl to connect to only one cluster created by Minikube. But kubectl can be congured to use mulple clusters and mulple contexts to connect to them. To check available contexts, we use kubectl cong get-contexts. We can conrm that only one context and only one cluster is dened by kubectl cong view. It should look like this: apiVersion: v1 clusters: - cluster: certicate-authority: $HOME/.minikube/ca.crt server: https://192.168.99.100:8443 name: minikube contexts: - context: cluster: minikube user: minikube name: minikube kind: Cong preferences: {} users: - name: minikube user: client-certicate: $HOME/.minikube/apiserver.crt client-key: $HOME/.minikube/apiserver.key

19


Stratoscale

The cong le used by kubectl is stored at ~/.kube/cong. We can edit it with a text editor and add another cluster, context and user. When ready, kubectl cong get-contexts should show our newly added context without marking it as current. This is the desired state:

apiVersion: v1 clusters: - cluster: certicate-authority: $HOME/.minikube/ca.crt server: https://192.168.99.100:8443 name: minikube - cluster: certicate-authority: $HOME/.minikube/ca.crt server: https://192.168.99.100:8443 name: secondkube contexts: - context: cluster: minikube user: minikube name: minikube - context: cluster: secondkube user: secondkube name: secondkube current-context: secondkube kind: Cong preferences: {} users: - name: minikube user: client-certicate: $HOME/.minikube/apiserver.crt client-key: $HOME/.minikube/apiserver.key - name: secondkube user: client-certicate: $HOME/.minikube/apiserver.crt client-key: $HOME/.minikube/apiserver.key

To switch context, we use kubectl cong use-context secondkube . We can verify the switch was successful again with kubectl cong get-contexts . The marker for current should have moved to the new context. All kubectl commands from now on will be executed in a selected context (which in our example is exactly the same as the rst one).

20


Stratoscale

HANDS-ON: DEPLOYING AN APPLICATION In this example, we deploy the WordPress content management system with a MySQL backend. It is a classic two-er applicaon, where the rst er is the applicaon server (WordPress) that uses the second er for data persistence (MySQL).

STEP 1. CREATE A KUBERNETES SECRET As discussed above, Kubernetes secrets allow users to pass sensive informaon, such as passwords, database credenals, to containers. In the rst step, we need to dene a Kubernetes secret that contains a database name, user, and password. It should also contain the root password for MySQL. Before creang a secret, we need to encode such informaon in Base64 format. Let’s assume we want to use the following values in our applicaon: •

“app-db” as a database name

•

“app-user” as a database user name

•

“app-pass” as a database password

•

“app-rootpass” as a database root password

Note that we need to provide a database root password to allow WordPress to create the required database. To encode these values to Base64 format, we can use the standard base64 ulity that is available in almost all Linux distribuons:

$ echo -n “app-db” | base64 YXBwLWRi $ echo -n “app-user” | base64 YXBwLXVzZXI= $ echo -n “app-pass” | base64 YXBwLXBhc3M= $ echo -n “app-rootpass” | base64 YXBwLXJvb3RwYXNz

21


Stratoscale

We use the “-n” opon to make sure that the new line symbol (“\n”) is not included in the encoded value. To dene a new Kubernetes secret, create a new le, app-secret.yaml, with the following content:

apiVersion: v1 kind: Secret metadata: name: app-secret type: Opaque data: dbname: YXBwLWRi dbuser: YXBwLXVzZXI= dbpass: YXBwLXBhc3M= dbrootpass: YXBwLXJvb3RwYXNz

Note:Kubernetes allows its building blocks to be dened using JSON and YAML formats. These formats are quite popular now as a more lightweight alternave to XML. The idea behind XML, JSON, and YAML is to provide a universal text notaon to serialize data in both machine- and human-readable form. In this example, we will use YAML. In the app-secret.yaml le, we specied the required Kubernetes API version and the data type to let Kubernetes know that we are dening a secret. In addion, the le denes four keys (dbname, dbuser, dbpass, dbrootpass ) with the corresponding values we encoded above. Now we can create our Kubernetes secret using its denion in the app-secret.yaml le:

$ kubectl create -f app-secret.yaml secret “app-secret” created

Let’s verify the secret creaon:

$ kubectl get secrets

22

NAME

TYPE

app-secrets

Opaque


DATA 4

AGE 1m

Stratoscale

STEP 2. CREATE A PERSISTENT VOLUME Next, we will create a Kubernetes persistent volume to provide the underlying storage for our MySQL database. To dene a new persistent volume, create a new le, app-pv.yam, with the following content:

apiVersion: v1 kind: PersistentVolume metadata: name: app-pv labels: vol: mysql spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce hostPath: path: /data/app

In this le, we use a HostPath volume, which will just pass a directory from the host into a container for consumpon. To specify that the persistent volume will be used for our applicaon, we added a label ( vol: mysql) that can be used later in a selector. Before creang the actual persistent volume, verify that the directory /data/app exists:

$ sudo mkdir -p /data/app

Now we can create our persistent volume:

$ kubectl create -f app-pv.yaml persistentvolume “app-pv” created

23


Stratoscale

Let’s verify that the persistent volume is available:

$ kubectl describe pv/app-pv Name:

app-pv

Labels:

vol=mysql

Status:

Available

Claim: Reclaim Policy: Retain Access Modes:

RWO

Capacity:

1Gi

Message: Source: Type:

HostPath (bare host directory volume)

Path:

/data/app

No events.

STEP 3. CLAIM A PERSISTENT VOLUME For MySQL, we need to claim our previously created persistent volume. Create a new le, apppvc.yaml, with the following content:

kind: PersistentVolumeClaim apiVersion: v1 metadata: name: app-pvc spec: accessModes: - ReadWriteOnce resources:

requests: storage: 1Gi

selector:

matchLabels: vol: mysql

The label selector “matchLabels” is used to make the associaon to the persistent volume that we created early. To create the persistent volume claim using its denion execute the following:

$ kubectl create -f app-pvc.yaml persistentvolumeclaim “app-pvc” created

24


Stratoscale

STEP 4. DEPLOY MYSQL Now we will create a new deployment for MySQL using the exisng Docker image. Create a new le, mysql-deployment.yaml, with the following content:

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: mysql-deployment spec: replicas: 1 template:

metadata:

labels: app: mysql

spec: containers: - name: mysql image: “mysql:5.6”

ports: - containerPort: 3306

volumeMounts: - mountPath: “/var/lib/mysql” name: mysql-pd

env: - name: MYSQL_ROOT_PASSWORD

valueFrom:

secretKeyRef: name: app-secret key: dbrootpass - name: MYSQL_USER

valueFrom:

secretKeyRef: name: app-secret key: dbuser - name: MYSQL_PASSWORD

valueFrom:

secretKeyRef: name: app-secret key: dbpass - name: MYSQL_DATABASE

valueFrom:

secretKeyRef: name: app-secret key: dbname

volumes: - name: mysql-pd

persistentVolumeClaim: claimName: app-pvc

25


Stratoscale

There is quite a lot of stu happening in the above YAML. Let’s break it down: •

We are specifying a standard pod’s informaon, such as container name (mysql), image to use (mysql:5.6), and exposed port to access (3306).

•

We are specifying the number of replicas (1) and aaching a label ( app: mysql).

•

We are then mounng a volume to the “/var/lib/mysql” directory in the container, where the volume to mount is named “mysql-pd” and is declared at the boom of this document.

•

We are also declaring environment variables to inialize. The MySQL image we are using that is available on Docker Hub supports environment variable injecon. The four environment variables we are inializing are dened and used within the Docker image itself. The environment variables set all reference dierent keys we dened in our secret earlier. When this container starts up, we will automacally have MySQL congured with the desired root user password. We will also have the database for WordPress created with appropriate access granted for our WordPress user.

To create the deployment using its denion, execute the following:

$ kubectl create -f mysql-deployment.yaml deployment “mysql-deployment” created

Let’s verify that everything was created successfully:

$ kubectl get pv NAME REASON app-pv

CAPACITY

ACCESSMODES

STATUS

CLAIM

RWO

Bound

default/app-pvc

AGE 1Gi

10m $ kubectl get pvc NAME

STATUS

VOLUME

CAPACITY

app-pvc

Bound

app-pv

0

ACCESSMODES

AGE 5m

$ kubectl get deployments NAME

DESIRED

CURRENT

UP-TO-DATE

AVAILABLE

1

1

1

1

AGE mysql-deployment 2m

26


Stratoscale

STEP 5. CREATE A SERVICE FOR MYSQL As we know, pods are ephemeral. They come and go, with each newly created pod receiving a new and dierent IP address. To connect to a database, the WordPress applicaon should know its IP address. If the database container is ephemeral, then how should our applicaon keep track of the database server’s IP addresses? We need an IP address that is decoupled from that pod and that never changes, and this is exactly what Kubernetes Services oer. To dene a service for MySQL, create a new le, mysql-service.yaml, with the following content:

apiVersion: v1 kind: Service metadata: name: mysql-service spec:

ports:

- port: 3306 protocol: TCP targetPort: 3306 selector: app: mysql

To create the actual service execute, the following command:

$ kubectl create -f mysql-service.yaml service “mysql-service” created

Let’s verify that the service is created and correctly mapped: $ kubectl describe svc/mysql-service Name:

mysql-service

Namespace:

default

Labels:

Selector:

app=mysql

Type:

ClusterIP

IP:

...

Port:

3306/TCP

Endpoints:

172.17.0.3:3306

Session Afnity:

None

No events. $ kubectl get pods -o wide NAME

READY

mysql-deployment-... 1/1

STATUS

RESTARTS

AGE

RUNNING

0

30m

IP

172.17.0.3

27


Stratoscale

From the output above, we can verify the service was correctly mapped to the pod for our MySQL deployment in that the Endpoints IP address for the service aligns with the IP address for the MySQL Pod.

STEP 6. DEPLOY WORDPRESS To dene a deployment for WordPress, create a new le, wordpress-deployment.yaml, with the following content:

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: wordpress-deployment spec: replicas: 2 strategy: type: RollingUpdate template:

metadata:

labels: app: wordpress

spec: containers: - name: “wordpress” image: “wordpress:4.5-apache”

ports: - containerPort: 80

env: - name: WORDPRESS_DB_HOST value: mysql-service - name: WORDPRESS_DB_USER

valueFrom:

secretKeyRef: name: app-secrets key: dbuser - name: WORDPRESS_DB_PASSWORD

valueFrom:

secretKeyRef: name: app-secret key: dbpass - name: WORDPRESS_DB_NAME

valueFrom:

secretKeyRef: name: app-secret key: dbname

28


Stratoscale

In this le, we are specifying standard pod informaon such as container name (wordpress), image to use (wordpress: 4.5-apache), exposed port to access (80), and number of replicas (2). We are also aaching a label (app: wordpress) to the pod’s replicas. One of the key things we accomplish in the YAML above is the inializaon of the environment variable ”WORDPRESS_

DB_HOST” to a value of “mysql-service”. This is how we are tell the WordPress applicaon to

access its database through the Kubernetes service we created in the previous step. To create the actual deployment, execute the following command:

$ kubectl create -f wordpress-deployment.yaml deployment “wordpress-deployment” created

Verify the deployment:

$ kubectl get deployments NAME

DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

mysql-deployment

1

1

1

1

45m

wordpress-deployment 2

2

2

2

5m

Get a list of created pods:

$ kubectl get pods NAME

READY

STATUS

RESTARTS

AGE

mysql-deployment-...

1/1

Running

0

45m

wordpress-deployment-...

1/1

Running

0

6m

wordpress-deployment-...

1/1

Running

0

6m

Make note of the name of one of the WordPress pods from the output above. Execute an interacve shell within that pod:

$ kubectl exec -it wordpress-deployment-... bash

Let’s check that the MySQL service can be resolved within the pod using the service’s name:

root@wordpress# getent hosts mysql-service 10.0.0.248

29


mysql-service.default.svc.cluster.local

Stratoscale

The above output veries that mysql-service can be resolved through DNS to the ClusterIP address that was assigned to the MySQL service (your IP address may be dierent). Now let’s verify that WordPress is properly congured:

root@wordpress# grep -i db /var/www/html/wp-cong.php dene(‘DB_NAME’, ‘app-db’); dene(‘DB_USER’, ‘app-user’); dene(‘DB_PASSWORD’, ‘app-pass’); dene(‘DB_HOST’, ‘mysql-service’); ...

The WordPress pod has congured itself using the environment environments “injected” into container using the values from the Kubernetes secret we dened earlier.

STEP 7. CREATE A SERVICE FOR WORDPRESS The nal step is to expose the WordPress applicaon to external users. For this, we again need a service. In this step, we expose a port on the node running our applicaon, and forward it to port 80 of our container. This allows us to access the applicaon, but it probably is not the approach one would take in producon, especially if Kubernetes is hosted by a service provider. Kubernetes can integrate with load balancing services oered by plaorms such as GCE and AWS. If you are using either of those, then that would be an approach to take for using the load balancing funconality oered by those plaorms. To dene a service for the WordPress applicaon, create a new le, wordpress-service.yaml, with the following content:

apiVersion: v1 kind: Service metadata: name: wordpress-service labels: app: wordpress spec: type: NodePort ports: - port: 80 nodePort: 30080 selector: app: wordpress

30


Stratoscale

To create the actual service using the denion from the wordpress-service.yaml le, execute the following command:

$ kubectl create -f wordpress-service.yaml service “wordpress-service” created

Verify its status:

$ kubectl describe svc/wordpress-service Name:

wordpress-service

Namespace:

default

Labels:

app=wordpress

Selector:

app=wordpress

Type:

NodePort

IP:

...

Port:

80/TCP

NodePort:

30080/TCP

Endpoints:

...:80,...:80,...:80

Session Afnity:

None

No events.

31


Stratoscale

STEP 8. TEST THE WORDPRESS APPLICATION Open your browser and navigate to hp://:30080, where is the address of your Kubernetes cluster node. You can follow the installaon wizard to get WordPress up and running through the browser. Congratulaons! The following diagram shows all of the Kubernetes building blocks we dened and created for our applicaon:

Service Selector app=wordpress

Pod

Secret

Container wordpress

Label app: wordpress

Pod Container wordpress

Label app: wordpress

Service

Volume Claim

Selector app=mysql

Label vol=mysql

Pod Container

Volume mysql /var/lib/mysql

Label

Label

app: mysql

vol: mysql

In producon, we may want to update the WordPress deployment increasing the number of replicas to handle a high load for the applicaon. We can easily do it manually, but the preferred way in Kubernetes is to use auto-scaling feature. For the MySQL deployment in our example, there is no simple way to increase the number of MySQL pods, because we cannot share the same persistent volume between several MySQL processes: MySQL does not support that. To scale the database and make it highly available we can use, for example, a Galera cluster in the mul-master node. In this case, each Galera pod will use its own persistent volume and Galera cluster will replicate its data between pods using its own protocol.

32


Stratoscale

DIY CLUSTER CONSIDERATIONS What if you need a producon ready Kubernetes

When deploying a Kubernetes cluster in producon,

cluster, but for some reason you cannot use the

one should take care of several things to ensure the

exisng cloud oerings, such as Google Container

best possible outcome. High Availability of master

Engine? Kubernetes can be installed on a variety

nodes helps minimize downme and data loss as

plaorms, including on-premises VMs, VMs on a

well as eliminates single point of failure. Networking

cloud provider, and bare-metal servers. There are

should be both robust and scalable to handle growing

several tools that allow installing producon ready

needs (e.g., The number of nodes in a cluster to handle

Kubernetes cluster on a variety of targets:

more replicas). Finally, some users may want to take

•

kargo

advantage of mul-site support to uniformly handle

•

kube-deploy

geographically dispersed data centers.

•

kube-admin

HIGH AVAILABILITY Possible cases of failure in Kubernetes clusters usually point to pods, nodes, and master nodes. Pod failures can be handled by built-in Kubernetes features, so the main concern here is to provide persistent storage if needed. Node failures can be handled by master nodes and require use of services outside of Kubernetes. For example, kubelet talks to an external load-balancer rather than directly to clients; if the enre node fails, trac can be load balanced to the node with corresponding pods. Finally, the master controller can fail, or one of its services can die. We need to replicate the master controller and its components for a Highly Available environment. Fortunately, mulple master nodes are also accounted for in Kubernetes. Furthermore, when it comes to monitoring the deployment, It is advisable that process watchers be implemented to “watch” the services that exist on the master node. For example, the API service can be monitored by a kubelet. It can be congured with less aggressive security sengs to monitor non-Kubernetes components such as privileged containers. On a dierent level is the issue of what happens if a kubelet dies. Monitoring processes can be deployed to ensure that the kubelet can be restarted. Finally, redundant storage service can be achieved with clustered etcd.

33


Stratoscale

SECURITY First of all, direct access to cluster nodes (either physical or through SSH) should be restricted. kubectl exec allows access to containers—this should be enough. Use Security Contexts to segregate privileges. Dening quotas for resources helps prevent DoS aacks. Selecvely grant users permissions according to their business needs. Finally, consider separang network trac that is not related (e.g., The load balancer only needs to see a front-end service, while the back-end service has no need to contact the load balancer).

SCALE Kubernetes allows for adding and removing nodes dynamically. Each new node has to be congured appropriately and pointed at the master node. The main processes of interest are kubelet and kube-proxy. For larger scale clusters, a means of automaon is preferred, such as Ansible or Salt. If the cluster is running on one of supported cloud providers, there is also an opon to try the Cluster Autoscaler.

34


Stratoscale

SUMMARY Kubernetes is an open-source project that is well supported by community. It allows applicaon development to be completely infrastructure-agnosc and avoids vendor lock-in. Installing, maintaining, and manually monitoring a producon-ready Kubernetes cluster on premises is a dicult task. For the installaon, high availability and networking models should be chosen and properly implemented. A tool to manage nodes in the cluster or to monitor the cluster’s and nodes’ health is also handy. Keeping previous suggesons in mind, you should be able to roll out your own HA cluster for Kubernetes. It takes some work, both in terms of planning and actual execuon. Planning carefully should save you much me later on when you start to scale your services. However if you want to take advantage of Kubernetes features but are not keen on maintaining your own cluster, Stratoscale’s Kubernetes-as-a-Service may suit your needs perfectly.

35


Stratoscale

ABOUT STRATOSCALE SYMPHONY Stratoscale is the cloud infrastructure company, providing comprehensive cloud infrastructure soware soluons for service providers, enterprise IT and development teams. The company’s comprehensive cloud data center soware, Stratoscale Symphony, can be deployed in minutes on commodity x86 servers, providing an Amazon Web Services (AWS) experience with the ability to augment aging VMware infrastructure. Stratoscale was named a “Cool Vendor in Servers and Virtualizaon” by Gartner and is backed by over $70M from leading investors including: Baery Ventures, Bessemer Venture Partners, Cisco, Intel, Qualcomm Ventures, SanDisk and Leslie Ventures.

SIMPLE CREATION

ONGOING MONITORING

Admins can create private clusters via

Easily monitor cluster health and usage.

intuive GUI or API.

MINIMAL LEARNING CURVE

MULTI-TENANCY

Leverage Kubernetes GUI and maintain

Kubernetes is transformed to oer a true

exisng pracces.

mul-tenant service.

USING KUBECTL CLI FOR SINGLE-CLICK MANAGEMENT OF KUBERNETES CLUSTERS Symphony’s fully managed Kubernetes-as-a-Service removes the operaonal barriers of adopng a container-based strategy. Admins can leverage KubeCtl CLI or Symphony’s intuive GUI to create Kubernetes clusters and easily monitor cluster health and usage. Symphony keeps the clusters up and running according to dened sizes and provides network access to cluster endpoints and node maintenance.

SYMPHONY: YOUR ON-PREM AWS-REGION This simple example demonstrates how you can leverage the agility and simplicity of cloud method ologies within your on-prem environment. Symphony’s service oers easy and single-click creaon, monitoring and management of Kubernetes clusters to ensure a smooth transion towards a con tainers-driven applicaon strategy.

STEP 1.

STEP 2.

ASSIGN STORAGE AND NETWORK FOR THE CLUSTER

CREATE A NEW KUBERNETES CLUSTER

Use KubeCtl CLI commands to allocate a

Use KubeCtl CLI commands to create the

storage pool, network (VPC) and oang IP

new cluster based on the required number

to the new Kubernetes cluster.

of nodes and the parameters for each node, including number of CPUs and memory. The assigned oang IP will be used as the cluster’s end-point.

STEP 3.

STEP 4.

MONITOR KUBERNETES

CLUSTERS

MANAGE AND EXTEND KUBERNETES CLUSTERS

Use KubeCtl CLI commands to connuously

As needs and requirements evolve, use

monitor the clusters and check their status.

KubeCtl CLI commands to easily expand the cluster and increase the number of Kubernetes nodes.

Everything Kubernetes

Recommend Documents