In part 1 you started a kind cluster, declared “three copies of nginx” and watched Kubernetes replace a Pod you deleted. This part opens the box. By the end you will be able to name every component of a cluster, say what each one does and where it runs, and follow a single kubectl apply from your terminal to a running container, using events and verbose logs as evidence. You need the kind cluster from part 1 (kind create cluster --name k8s-from-zero), kubectl v1.37 and about 30 minutes. Part 3 is the install lesson, with checksums and the other ways to get a local cluster. The short setup in part 1 is enough to follow this tour.
What you will build
You will not build an app today. You will take a guided tour of a live cluster: list the control-plane components, find them as files and containers on the node, watch each component leave its fingerprint in the event log while a Deployment rolls out, peek into the database that stores it all, and then switch the scheduler off to see what breaks. The map below is the one to keep in your head for the rest of the series.
The control plane is the cluster’s brain
The architecture page splits a cluster into a control plane and a set of worker nodes. The control plane makes global decisions (for example, scheduling) and reacts to events (for example, starting a new Pod when a Deployment has too few). In production it usually runs on several machines for fault tolerance; in kind it all lives on one node. It has four core parts and one optional part:
- kube-apiserver is the front door. It exposes the Kubernetes HTTP API, checks who you are and whether you are allowed (controlling access), validates what you send and saves it. It is designed to scale horizontally: you can run several copies behind a load balancer.
- etcd is a consistent, highly available key-value store and the backing store for all cluster data. If etcd is lost without a backup, the cluster forgets everything it was told. Part 27 covers etcd backups.
- kube-scheduler watches for Pods that have no node yet and picks one for each. It weighs resource requests, hardware and policy constraints, affinity rules, data locality and more (scheduler page). It only decides; it does not start anything.
- kube-controller-manager runs the built-in controllers, the control loops from part 1. Each controller is logically separate, but they are compiled into one binary and run as one process. Examples: the Node controller (notices when nodes go down), the Job controller, the EndpointSlice controller, the ServiceAccount controller, and the Deployment and ReplicaSet controllers you will see in the lab.
- cloud-controller-manager (optional) links the cluster to a cloud provider’s API, for example to create a cloud load balancer. A cluster on your own laptop does not have one.
Nodes are where your containers run
Every worker node runs a small set of components (components page):
- kubelet is the node agent. It receives the Pods assigned to its node and makes sure their containers are running and healthy. It does not manage containers that Kubernetes did not create.
- A container runtime actually runs containers. The kubelet talks to it through the Container Runtime Interface (CRI), a gRPC API (CRI page), so the runtime is swappable. kind nodes use containerd.
- kube-proxy (optional) maintains network rules on the node so that Services (a stable address in front of a group of Pods, part 8) work. Some network plugins replace it with their own implementation.
On top of these sit add-ons, which run as ordinary Pods: cluster DNS (CoreDNS), the network plugin that gives every Pod an IP (kindnet in kind; part 61 explains network plugins), and in kind a small storage provisioner.
The rule that ties the control plane together: control-plane components and kubelets coordinate through the API server, mostly by watching it for changes and then writing back what they did (node communication). The kubelet also talks to the container runtime over the Container Runtime Interface, a gRPC API. On this cluster that runtime is containerd (CRI page). The API server connects to each kubelet for Pod logs, for exec and attach, and for port-forward. Only the API server reads and writes etcd.
That hub is why Kubernetes survives partial failures well. If the scheduler is down, nothing new gets a node, but everything already running keeps running. You will test exactly that.
Life of a kubectl apply
Here is what happens, in order, when you run kubectl apply -f web.yaml for a Deployment with two replicas. Each step is a separate component reacting to a change it saw through the API server.
- kubectl reads your file, asks the API server whether a Deployment called
webexists, and since it does not, sends an HTTPPOST. The API server authenticates you, validates the object and stores it in etcd. - The Deployment controller (inside kube-controller-manager) is watching Deployments. It sees the new one and creates a ReplicaSet for it.
- The ReplicaSet controller sees a ReplicaSet that wants 2 Pods and has 0, so it creates two Pod objects. They exist only as records: no node, no container.
- The scheduler sees two Pods with an empty
spec.nodeName, scores the nodes and binds each Pod by writing a node name into it. - The kubelet on that node sees Pods assigned to it, asks containerd to pull the image (or finds it cached), create the container and start it.
- The kubelet reports the Pod’s status,
Running, back to the API server.kubectl get podsnow shows it.
Every arrow in this apply goes through the API server, and each of those steps leaves an event you can read. A Pod that never receives a node, as in the scheduler lab below, can sit in Pending with no events. Ownership is recorded too: each Pod carries an ownerReference to its ReplicaSet, and the ReplicaSet one to its Deployment (owners and dependents). That chain is how deleting a Deployment cleans up everything below it.
Try it: tour your own cluster
All commands below ran on our kind cluster k8s-from-zero. Names with random suffixes and port numbers will differ on yours.
1. Look at the node.
kubectl get nodes -o wideExplain this command
kubectl get nodes -o widekubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.nodesThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.wideAn output format: the default table plus more columns. For nodes the lesson's output adds INTERNAL-IP, OS-IMAGE, KERNEL-VERSION and CONTAINER-RUNTIME, among others.
NAME STATUS ROLES AGE VERSION INTERNAL-IP ... OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-from-zero-control-plane Ready control-plane 2m39s v1.37.0 172.18.0.2 ... Debian GNU/Linux 13 (trixie) 6.12.113+ (amd64) containerd://2.3.4
CONTAINER-RUNTIME confirms containerd. Our single node is the control plane and also runs your workloads, which kind allows.
2. Find the control plane. The components run as Pods in the kube-system namespace (a namespace is a folder for objects; part 6 covers them):
kubectl get pods -n kube-system -o wideExplain this command
kubectl get pods -n kube-system -o widekubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.podsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-nShort form of --namespace. When you set it, this request is limited to that namespace.kube-systemThe namespace name passed to --namespace.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.wideAn output format: the default table plus more columns. The get examples say it adds more information, such as the node name; the lesson's output shows IP and NODE.
NAME READY STATUS RESTARTS AGE IP NODE ...
coredns-559f6c778d-m82pm 1/1 Running 0 2m29s 10.244.0.4 ...
coredns-559f6c778d-n6kz5 1/1 Running 0 2m29s 10.244.0.2 ...
etcd-k8s-from-zero-control-plane 1/1 Running 0 2m37s 172.18.0.2 ...
kindnet-hczvf 1/1 Running 0 2m29s 172.18.0.2 ...
kube-apiserver-k8s-from-zero-control-plane 1/1 Running 0 2m37s 172.18.0.2 ...
kube-controller-manager-k8s-from-zero-control-plane 1/1 Running 0 2m37s 172.18.0.2 ...
kube-proxy-t6cgl 1/1 Running 0 2m29s 172.18.0.2 ...
kube-scheduler-k8s-from-zero-control-plane 1/1 Running 0 2m37s 172.18.0.2 ...
Everything from the map is here: etcd, kube-apiserver, kube-controller-manager and kube-scheduler (each with the node name as a suffix), plus kube-proxy, kindnet and CoreDNS.
3. See how the control plane starts. kind bootstraps each node with kubeadm, which runs the control-plane components as static Pods: Pod files that the kubelet reads straight from a folder on the node, without asking the API server (static Pods). A kind node is a Docker container, so you can look inside with docker exec:
docker exec k8s-from-zero-control-plane ls /etc/kubernetes/manifests
docker exec k8s-from-zero-control-plane systemctl is-active kubelet containerd
etcd.yaml
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml
active
active
The kubelet and containerd are ordinary system services on the node. The four control-plane parts are files in a folder. Part 30 goes deeper into the kubelet.
4. Talk to the runtime directly. crictl is a debugging client for any CRI runtime:
docker exec k8s-from-zero-control-plane crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD NAMESPACE
8210829884fa8 520212b8b0fcd 2 minutes ago Running coredns 0 b03643ff49a4e coredns-559f6c778d-m82pm kube-system
...
64b8dfaa414e7 270fbeb697171 2 minutes ago Running etcd 0 7b32bee57a197 etcd-k8s-from-zero-control-plane kube-system
9aedd1c85f871 364b3c3d9ec19 2 minutes ago Running kube-controller-manager 0 be8231196ab41 kube-controller-manager-k8s-from-zero-control-plane kube-system
541b0a5b9cd28 1fabf80a1273a 2 minutes ago Running kube-scheduler 0 a98ceb696a15a kube-scheduler-k8s-from-zero-control-plane kube-system
08f713ee1ecf4 bec5f0e1e2eeb 2 minutes ago Running kube-apiserver 0 d3e52fb566d4d kube-apiserver-k8s-from-zero-control-plane kube-system
Same components, seen from below: containers that containerd runs.
5. Ask the API server how it feels. It has health endpoints; /readyz?verbose lists every internal check:
kubectl get --raw='/readyz?verbose' | tail -3Explain this command
kubectl get --raw='/readyz?verbose' | tail -3kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.--rawRequests a raw URI from the API server, using the transport from the kubeconfig file.'/readyz?verbose'The URI. The API health endpoints page says /readyz reports whether the API server is ready to accept traffic, and that the verbose parameter prints each check and its status for a human operator. That page shows this kubectl command.|Shell. Sends the previous command's standard output to the next command.tailShell. Prints the end of its input.-3Shell. How many lines tail prints. Here, the last 3.
[+]poststarthook/apiservice-openapiv3-controller ok
[+]shutdown ok
readyz check passed
6. See what the API offers. Every kind of object is an API resource:
kubectl api-resources | head -6Explain this command
kubectl api-resources | head -6kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.api-resourcesPrint the supported API resources on the server.|Shell. Sends the previous command's standard output to the next command.headShell. Prints the beginning of its input.-6Shell. How many lines head prints. Here, the first 6.
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
Our cluster listed 71 resource types (kubectl api-resources --no-headers | wc -l).
7. See who is in charge. In a cluster with several control-plane machines, only one scheduler and one controller manager act at a time; they hold a Lease object to prove it (leases):
kubectl get leases -n kube-systemExplain this command
kubectl get leases -n kube-systemkubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.leasesThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-nShort form of --namespace. When you set it, this request is limited to that namespace.kube-systemThe namespace name passed to --namespace.
NAME HOLDER AGE
apiserver-uwnlkcfd6lspuhs4nbdjmcuiy4 apiserver-uwnlkcfd6lspuhs4nbdjmcuiy4_8248f3ae-eac1-41ac-af27-67f6b0cfba7b 2m39s
kube-controller-manager k8s-from-zero-control-plane_ccba66fc-e889-452c-b239-090af9dc7a32 2m36s
kube-scheduler k8s-from-zero-control-plane_4a40fecb-4367-4589-8f16-70f0c542327b 2m35s
8. Follow one apply. Save this as web.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
labels:
app.kubernetes.io/name: web
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: web
template:
metadata:
labels:
app.kubernetes.io/name: web
spec:
containers:
- name: nginx
image: nginx:1.29-alpine
ports:
- containerPort: 80
In a second terminal, start watching events before you apply, so you see them in the order they happen:
kubectl get events --watch-only -o custom-columns=FROM:.source.component,REASON:.reason,KIND:.involvedObject.kind,NAME:.involvedObject.nameExplain this command
kubectl get events --watch-only -o custom-columns=FROM:.source.component,REASON:.reason,KIND:.involvedObject.kind,NAME:.involvedObject.namekubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.eventsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.--watch-onlyWatch for changes to the requested objects. The get docs say this flag skips the initial list.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.custom-columns=FROM:.source.component,REASON:.reason,KIND:.involvedObject.kind,NAME:.involvedObject.namecustom-columns is an output format. The get examples write each column as a heading, a colon, and a JSONPath. The headings here are FROM, REASON, KIND and NAME. The paths are .source.component, .reason, .involvedObject.kind and .involvedObject.name.
Then, in the first terminal, apply with -v=6, which makes kubectl log every API request it sends with its response code (we cut the log prefix from each line):
kubectl apply -f web.yaml -v=6 2>&1 | grep 'verb='Explain this command
kubectl apply -f web.yaml -v=6 2>&1 | grep 'verb='kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.applyApply a configuration to a resource by file name or stdin. JSON and YAML are accepted. The apply docs say the resource is created if it does not exist yet.-fShort form of --filename. For apply, the file, directory or URL that contains the configuration.web.yamlThe file named by --filename.-v=6Sets kubectl's log level. The quick reference says level 6 displays requested resources. The same table says level 8 displays HTTP request contents.2>&1Shell. Sends standard error to the same place as standard output, so the next command can read kubectl's log lines.|Shell. Sends the previous command's standard output to the next command.grepShell. Prints input lines that match a pattern.'verb='Shell. The pattern grep looks for.
"Response" verb="GET" url="https://127.0.0.1:45267/openapi/v3?timeout=32s" status="200 OK" milliseconds=4
"Response" verb="GET" url="https://127.0.0.1:45267/openapi/v3/apis/apps/v1?...&timeout=32s" status="200 OK" milliseconds=1
"Response" verb="GET" url="https://127.0.0.1:45267/apis/apps/v1/namespaces/default/deployments/web" status="404 Not Found" milliseconds=2
"Response" verb="POST" url="https://127.0.0.1:45267/apis/apps/v1/namespaces/default/deployments?fieldManager=kubectl-client-side-apply&fieldValidation=Strict" status="201 Created" milliseconds=4
That is step 1 of the sequence in plain HTTP: kubectl fetched the API schema, checked for an existing web (404), then created it (201 Created). The second terminal shows steps 2 to 6:
FROM REASON KIND NAME
deployment-controller ScalingReplicaSet Deployment web
replicaset-controller SuccessfulCreate ReplicaSet web-69977c9c79
replicaset-controller SuccessfulCreate ReplicaSet web-69977c9c79
default-scheduler Scheduled Pod web-69977c9c79-pwjwm
default-scheduler Scheduled Pod web-69977c9c79-vfjt9
kubelet Pulled Pod web-69977c9c79-vfjt9
kubelet Pulled Pod web-69977c9c79-pwjwm
kubelet Created Pod web-69977c9c79-vfjt9
kubelet Created Pod web-69977c9c79-pwjwm
kubelet Started Pod web-69977c9c79-vfjt9
kubelet Started Pod web-69977c9c79-pwjwm
Read it top to bottom: the Deployment controller scaled up a ReplicaSet, the ReplicaSet controller created two Pods, the scheduler placed them, and the kubelet pulled, created and started each container. Press Ctrl+C to stop the watch.
9. Check the ownership chain and the binding. The first line looks up one Pod from this Deployment. The sample event log shows web-69977c9c79-pwjwm and web-69977c9c79-vfjt9. Your suffixes will differ.
POD=$(kubectl get pods -l app.kubernetes.io/name=web -o name | head -1)
kubectl get "$POD" -o jsonpath='{.spec.nodeName}{"\n"}{.metadata.ownerReferences[0].kind}/{.metadata.ownerReferences[0].name}{"\n"}'
kubectl get replicaset -l app.kubernetes.io/name=web -o jsonpath='{.items[0].metadata.ownerReferences[0].kind}/{.items[0].metadata.ownerReferences[0].name}{"\n"}'Explain this command
POD=$(kubectl get pods -l app.kubernetes.io/name=web -o name | head -1)
kubectl get "$POD" -o jsonpath='{.spec.nodeName}{"\n"}{.metadata.ownerReferences[0].kind}/{.metadata.ownerReferences[0].name}{"\n"}'
kubectl get replicaset -l app.kubernetes.io/name=web -o jsonpath='{.items[0].metadata.ownerReferences[0].kind}/{.items[0].metadata.ownerReferences[0].name}{"\n"}'POD=Shell. The name before the equals sign is a variable. The shell stores the next command's output in POD.$(Shell command substitution. The shell runs the command inside $(...) and pastes its output into the line.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.podsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-lShort form of --selector. A label query. The get docs say it supports '=', '==', '!=', 'in' and 'notin', and that matching objects must satisfy every constraint.app.kubernetes.io/name=webThe label query. It asks for objects whose app.kubernetes.io/name label equals web, the label this lesson's Deployment sets.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.nameAn output format. kubectl get -o name prints each object as type/name, one per line, such as pod/web-69977c9c79-pwjwm.|Shell. Sends the previous command's standard output to the next command.headhead, a standard utility. It prints the first lines of its input (10 by default).-1head's option: print only the first line (the older form of -n 1), so POD holds a single Pod name.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name."$POD"The shell variable from the previous line, in quotes. It holds one Pod from this Deployment, printed as type/name. The sample event log shows web-69977c9c79-pwjwm and web-69977c9c79-vfjt9. Your suffix will differ.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.jsonpath='{.spec.nodeName}{"\n"}{.metadata.ownerReferences[0].kind}/{.metadata.ownerReferences[0].name}{"\n"}'jsonpath is an output format. A template is JSONPath expressions in curly braces. {.spec.nodeName} reads that field. The JSONPath page says a double-quoted string inserts text, and its example uses {"\n"} for a newline. The slash between the owner-reference expressions is text outside the braces, which that page prints as plain text. [0] is the first item, the same subscript the page uses in {.users[0].name}. The two fields are the owner reference kind and name.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.replicasetThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-lShort form of --selector. A label query. The get docs say it supports '=', '==', '!=', 'in' and 'notin', and that matching objects must satisfy every constraint.app.kubernetes.io/name=webThe label query. It asks for objects whose app.kubernetes.io/name label equals web.-oShort form of --output. It selects the output format. For get, the formats include json, yaml, name, jsonpath, custom-columns and wide.jsonpath='{.items[0].metadata.ownerReferences[0].kind}/{.items[0].metadata.ownerReferences[0].name}{"\n"}'jsonpath output for the first list item. {.items[0]} is the subscript the JSONPath page shows. The template prints that item's first owner-reference kind, a slash, its name, and a newline from {"\n"}.
k8s-from-zero-control-plane
ReplicaSet/web-69977c9c79
Deployment/web
spec.nodeName is the field the scheduler wrote. The Pod is owned by the ReplicaSet, which is owned by the Deployment.
Safety for steps 10 and 11. Run these two steps only on the local kind cluster from this lesson, on your own machine. Leave a shared cluster and a production cluster alone. The etcd command only reads one key. To restore the scheduler, move kube-scheduler.yaml back into /etc/kubernetes/manifests/. Step 11 ends with that move. If you stop midway, run that move before you close the terminal.
10. Peek into etcd. This is for learning only; in real life you never touch etcd by hand. The etcd Pod includes etcdctl, and the certificate paths are kubeadm’s defaults:
kubectl -n kube-system exec etcd-k8s-from-zero-control-plane -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
get /registry/deployments/default/web --keys-onlyExplain this command
kubectl -n kube-system exec etcd-k8s-from-zero-control-plane -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
get /registry/deployments/default/web --keys-onlykubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.-nShort form of --namespace. When you set it, this request is limited to that namespace.kube-systemThe namespace name passed to --namespace.execExecute a command in a container. The synopsis is kubectl exec (POD | TYPE/NAME) [-c CONTAINER] [flags] -- COMMAND [args...].etcd-k8s-from-zero-control-planeThe Pod name passed to exec.--Ends kubectl's flags. The exec examples say to use two dashes so the command's own flags are read as that command's flags.etcdctlThe command run in the container. The etcdctl README calls it a command line client for etcd.\Shell. Continues the command on the next line.--endpoints=https://127.0.0.1:2379etcdctl flag. The etcdctl README uses --endpoints to name the etcd members a command talks to. The address here is https://127.0.0.1:2379.--cacert=/etc/kubernetes/pki/etcd/ca.crtetcdctl global flag for the TLS certificate authority file. The README lists --cacert among global flags and shows ETCDCTL_CACERT pointing at a certificate file. The path is the one in this command.--cert=/etc/kubernetes/pki/etcd/server.crtetcdctl global flag for the client certificate. The README lists --cert and ETCDCTL_CERT.--key=/etc/kubernetes/pki/etcd/server.keyetcdctl global flag for the client key. The README lists --key and ETCDCTL_KEY.getetcdctl GET gets the key, or a range of keys when a range end is given./registry/deployments/default/webThe key get reads.--keys-onlyetcdctl GET option. The README says it gets only the keys.
/registry/deployments/default/web
Your Deployment is a key in etcd, written there by the API server.
11. Switch the scheduler off. Because the scheduler is a static Pod, moving its file out of the manifests folder makes the kubelet stop it. Then ask for two more replicas:
docker exec k8s-from-zero-control-plane mv /etc/kubernetes/manifests/kube-scheduler.yaml /root/
kubectl get pods -n kube-system -l component=kube-scheduler
kubectl scale deployment web --replicas=4
kubectl get pods -l app.kubernetes.io/name=webExplain this command
docker exec k8s-from-zero-control-plane mv /etc/kubernetes/manifests/kube-scheduler.yaml /root/
kubectl get pods -n kube-system -l component=kube-scheduler
kubectl scale deployment web --replicas=4
kubectl get pods -l app.kubernetes.io/name=webdockerThe docker client. The lesson uses it because a kind node is a Docker container.execdocker exec runs the rest of the line in the named container. This line has no docker flags.k8s-from-zero-control-planeThe container name.mvThe command inside the container. It moves the first path to the second./etc/kubernetes/manifests/kube-scheduler.yamlThe source path./root/The destination path.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.podsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-nShort form of --namespace. When you set it, this request is limited to that namespace.kube-systemThe namespace name passed to --namespace.-lShort form of --selector. A label query. The get docs say it supports '=', '==', '!=', 'in' and 'notin', and that matching objects must satisfy every constraint.component=kube-schedulerThe label query. It asks for objects whose component label equals kube-scheduler.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.scaleSet a new size for a deployment, replica set, replication controller, or stateful set.deploymentThe resource type, the TYPE in the scale synopsis TYPE NAME.webThe object name.--replicas=4The new desired number of replicas. The scale command says --replicas is required.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.podsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-lShort form of --selector. A label query. The get docs say it supports '=', '==', '!=', 'in' and 'notin', and that matching objects must satisfy every constraint.app.kubernetes.io/name=webThe label query. It asks for objects whose app.kubernetes.io/name label equals web.
No resources found in kube-system namespace.
deployment.apps/web scaled
NAME READY STATUS RESTARTS AGE
web-69977c9c79-pjmll 0/1 Pending 0 10s
web-69977c9c79-pwjwm 1/1 Running 0 50s
web-69977c9c79-qd4cv 0/1 Pending 0 10s
web-69977c9c79-vfjt9 1/1 Running 0 50s
The controllers still did their part: the two new Pods exist. Nobody gave them a node, so they wait in Pending, and the two old Pods keep serving. kubectl describe on a new Pod shows Node: <none> and no events at all. Put the file back and the scheduler catches up:
docker exec k8s-from-zero-control-plane mv /root/kube-scheduler.yaml /etc/kubernetes/manifests/
kubectl get pods -l app.kubernetes.io/name=webExplain this command
docker exec k8s-from-zero-control-plane mv /root/kube-scheduler.yaml /etc/kubernetes/manifests/
kubectl get pods -l app.kubernetes.io/name=webdockerThe docker client. The lesson uses it because a kind node is a Docker container.execdocker exec runs the rest of the line in the named container. This line has no docker flags.k8s-from-zero-control-planeThe container name.mvThe command inside the container. It moves the first path to the second./root/kube-scheduler.yamlThe source path./etc/kubernetes/manifests/The destination path.kubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.getDisplay one or many resources. kubectl get prints a table of the most important information about the resources you name.podsThe resource type to display. kubectl get points you to kubectl api-resources for the full list of types the server supports.-lShort form of --selector. A label query. The get docs say it supports '=', '==', '!=', 'in' and 'notin', and that matching objects must satisfy every constraint.app.kubernetes.io/name=webThe label query. It asks for objects whose app.kubernetes.io/name label equals web.
NAME READY STATUS RESTARTS AGE
web-69977c9c79-pjmll 1/1 Running 0 35s
web-69977c9c79-pwjwm 1/1 Running 0 75s
web-69977c9c79-qd4cv 1/1 Running 0 35s
web-69977c9c79-vfjt9 1/1 Running 0 75s
12. Clean up.
kubectl delete -f web.yamlExplain this command
kubectl delete -f web.yamlkubectlThe Kubernetes command-line client. Subcommands and flags are documented under kubernetes.io/docs/reference/kubectl/.deleteDelete resources by file names, stdin, resources and names, or by resources and label selector.-fShort form of --filename. For delete, the file that contains the resource to delete. The delete examples include kubectl delete -f ./pod.json.web.yamlThe file named by --filename.
Keep the cluster for part 3, or remove it with kind delete cluster --name k8s-from-zero. As in part 1, our lab had no internet route inside the cluster, so we preloaded nginx:1.29-alpine with kind load docker-image; you do not need to.
When it goes wrong
Pods stay Pending with no events. Nothing scheduled them. Check that the scheduler runs: kubectl get pods -n kube-system -l component=kube-scheduler. If it does, kubectl describe pod usually shows a FailedScheduling event with the reason, such as not enough CPU; part 17 covers scheduling.
A node turns NotReady. Its kubelet stopped reporting. We stopped the kubelet with docker exec k8s-from-zero-control-plane systemctl stop kubelet; 47 seconds later:
NAME STATUS ROLES AGE VERSION
k8s-from-zero-control-plane NotReady control-plane 28m v1.37.0
kubectl describe node showed every condition as Unknown with the message Kubelet stopped posting node status. Containers that were already running kept running, which is why kubectl still worked. systemctl start kubelet brought the node back to Ready within 15 seconds. On a real node, check the kubelet’s service and logs (journalctl -u kubelet) and the machine itself. The nodes page explains heartbeats.
The connection to the server ... was refused. kubectl cannot reach the API server: the cluster is stopped, or kubectl points at the wrong cluster. Part 3 shows how kubectl decides where to connect.
You want to know what kubectl is really doing. Add -v=6 to any command to see each request and its response code; -v=8 also shows the request contents (kubectl quick reference).
Cheat sheet
| Task | Command or field | Note |
|---|---|---|
| Nodes with runtime and IP | kubectl get nodes -o wide |
CONTAINER-RUNTIME column |
| Control-plane and add-on Pods | kubectl get pods -n kube-system |
names end in the node name for static Pods |
| Static Pod files on a kubeadm node | /etc/kubernetes/manifests/ |
the kubelet runs whatever is here |
| Containers as the runtime sees them | crictl ps (on the node) |
in kind: docker exec <node> crictl ps |
| API server health | kubectl get --raw='/readyz?verbose' |
also /livez |
| All resource types | kubectl api-resources |
shows short names and API groups |
| Who leads the scheduler and controllers | kubectl get leases -n kube-system |
one holder at a time |
| Watch events as they happen | kubectl get events --watch-only |
add -o custom-columns=... |
| See kubectl’s HTTP calls | kubectl <command> -v=6 |
-v=8 adds request contents |
| Which node a Pod landed on | .spec.nodeName |
written by the scheduler |
| Who owns an object | .metadata.ownerReferences |
Pod to ReplicaSet to Deployment |
| Why a node is unhappy | kubectl describe node <name> |
read Conditions |
# The fields each component writes on a running Pod (trimmed)
apiVersion: v1
kind: Pod
metadata:
name: web-69977c9c79-pwjwm # one name from the sample event log; yours will differ
ownerReferences: # set by the replicaset controller
- kind: ReplicaSet
name: web-69977c9c79
spec:
nodeName: k8s-from-zero-control-plane # set by the scheduler
status:
phase: Running # reported by the kubelet
Recap and what is next
- The control plane is kube-apiserver, etcd, kube-scheduler, kube-controller-manager and, in clouds, cloud-controller-manager.
- Every node runs a kubelet and a container runtime (containerd here), and usually kube-proxy; DNS and networking come as add-ons.
- Control-plane components and kubelets coordinate through the API server. The kubelet also talks to the container runtime over CRI, and the API server connects to kubelets for logs, exec and port-forward. Only the API server talks to etcd.
- One
kubectl applybecomes a chain: Deployment, ReplicaSet, Pods, a scheduler binding, a kubelet start. Events and-v=6let you watch it. - When one component stops, only its job stops: no scheduler means new Pods wait, and running Pods keep running.
Next: Kubernetes from Zero, part 3: your local lab with kind, kubectl and kubeconfig.
Command explanations and quoted passages in this lesson come from or adapt the kubectl reference and the Kubernetes documentation, © The Kubernetes Authors, licensed under CC BY 4.0. We shortened and edited some passages. Kubernetes® is a registered trademark of The Linux Foundation. aitamer.news is not affiliated with or endorsed by The Linux Foundation or the CNCF.

The Campfire
CommentsThanks. Your comment is waiting for the desk. Once it is approved it appears here within about a minute.
Add a comment