This is Part 4 of my "VMware vSphere with Kubernetes" Guide. In the last article, I've explained how to install and configure the Kubernetes CLI Tool kubectl and how to deploy the first pod. In this article, I'm taking a deeper look at kubectl.
This is a part of a series. If you do not have a Kubernetes activated vSphere Cluster, refer to Part 1 to get started with the deployment.
Configuration and Contexts
To recap from previous articles, I've used the following commands to login using the vSphere Plugin and worked with Namespaces. There is also a command to view the configuration which is saved in the .kube directory in your user's home directory.
## Login # kubectl vsphere login --server=<CLUSTER> -u <USER> ## View a list of Contexts # kubectl config get-contexts ## Switch to context # kubectl config use-context playground ## View active context # kubectl config current-context ## View Config # kubectl config view
View Object Information
The 3-pod Nginx deployment from Part 3 is still running. Use kubectl get to view the state of all components.
# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE webserver 3/3 3 3 2d # kubectl get pods NAME READY STATUS RESTARTS AGE webserver-d698d7bd6-h495m 1/1 Running 0 2d webserver-d698d7bd6-w86zd 1/1 Running 0 2d webserver-d698d7bd6-wxlx6 1/1 Running 0 2d # kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE webserver LoadBalancer 10.96.0.45 192.168.250.132 80:30733/TCP 2d # kubectl get nodes NAME STATUS ROLES AGE VERSION 4217339ad6df885b6121ac551ba3db2d Ready master 9d v1.17.4-2+a00aae1e6a4a69 4217931b9612c01232e50fe1eaa8cfb2 Ready master 9d v1.17.4-2+a00aae1e6a4a69 4217a39f121857906d9364ae52e9d91d Ready master 9d v1.17.4-2+a00aae1e6a4a69 esx2.virten.lab Ready agent 9d v1.17.4-sph-c4c19c8 esx3.virten.lab Ready agent 9d v1.17.4-sph-c4c19c8 esx4.virten.lab Ready agent 9d v1.17.4-sph-c4c19c8 esx5.virten.lab Ready agent 9d v1.17.4-sph-c4c19c8
Many commands can be used with the option -o wide to get additional information.
# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES webserver-d698d7bd6-h495m 1/1 Running 0 2d 10.244.0.244 esx5.virten.lab <none> <none> webserver-d698d7bd6-w86zd 1/1 Running 0 2d 10.244.0.242 esx5.virten.lab <none> <none> webserver-d698d7bd6-wxlx6 1/1 Running 0 2d 10.244.0.243 esx5.virten.lab <none> <none>
You can also use the --all-namespaces option to list objects in all namespaces. There are about 60 pods just from enabling vSphere with Kubernetes.
# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-5cd65bfbdc-5sckk 1/1 Running 0 9d kube-system coredns-5cd65bfbdc-gqs6b 1/1 Running 0 9d kube-system coredns-5cd65bfbdc-prrq8 1/1 Running 0 9d kube-system csr-signer-4217339ad6df885b6121ac551ba3db2d 1/1 Running 0 9d kube-system csr-signer-4217931b9612c01232e50fe1eaa8cfb2 1/1 Running 0 9d kube-system csr-signer-4217a39f121857906d9364ae52e9d91d 1/1 Running 0 9d kube-system docker-registry-4217339ad6df885b6121ac551ba3db2d 1/1 Running 0 9d kube-system docker-registry-4217931b9612c01232e50fe1eaa8cfb2 1/1 Running 0 9d kube-system docker-registry-4217a39f121857906d9364ae52e9d91d 1/1 Running 0 9d [...]
To get the object's status and configuration in YAML, use the -o yaml option.
# kubectl get deployment webserver -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"webserver","namespace":"playground"},"spec":{"replicas":3,"selector":{"matchLabels":{"app":"webserver"}},"template":{"metadata":{"labels":{"app":"webserver"}},"spec":{"containers":[{"image":"nginx","name":"nginx"}]}}}}
creationTimestamp: "2020-07-31T19:13:39Z"
generation: 2
name: webserver
namespace: playground
resourceVersion: "4222386"
selfLink: /apis/apps/v1/namespaces/playground/deployments/webserver
uid: bb3b8e09-4626-4802-a900-5352f914880b
spec:
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
app: webserver
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: webserver
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
availableReplicas: 3
conditions:
- lastTransitionTime: "2020-07-31T19:13:39Z"
lastUpdateTime: "2020-07-31T19:13:47Z"
message: ReplicaSet "webserver-d698d7bd6" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2020-07-31T19:51:32Z"
lastUpdateTime: "2020-07-31T19:51:32Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 3
replicas: 3
updatedReplicas: 3Describe Objects
The describe command can be used to get additional object information.
# kubectl describe deployment webserver
Name: webserver
Namespace: playground
CreationTimestamp: Fri, 31 Jul 2020 21:13:39 +0200
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=webserver
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=webserver
Containers:
nginx:
Image: nginx
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: webserver-d698d7bd6 (3/3 replicas created)
Events: <none>
# kubectl describe service webserver
Name: webserver
Namespace: playground
Labels: app=webserver
Annotations: ncp/internal_ip_for_policy: 100.64.0.1
Selector: app=webserver
Type: LoadBalancer
IP: 10.96.0.45
LoadBalancer Ingress: 192.168.250.132
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 30733/TCP
Endpoints: 10.244.0.242:80,10.244.0.243:80,10.244.0.244:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>Events and Logs
When you create a deployment, you do not see much output. To see what happens under the hood, use the kubectl get events command. This command is very important when pods can not be deployed as it will give hints where to start troubleshooting. Events are not displayed in chronological order, but they can be sorted with the --sort-by option.
# kubectl get events --sort-by=.metadata.creationTimestamp 63s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-wxlx6 Successfully realized NSX resource for Pod 63s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-h495m Successfully realized NSX resource for Pod 63s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-w86zd Successfully realized NSX resource for Pod 40s Normal ScalingReplicaSet deployment/webserver Scaled up replica set webserver-d698d7bd6 to 3 40s Normal SuccessfulCreate replicaset/webserver-d698d7bd6 Created pod: webserver-d698d7bd6-9nlbx <unknown> Normal Scheduled pod/webserver-d698d7bd6-jwjjm Successfully assigned playground/webserver-d698d7bd6-jwjjm to esx5.virten.lab 40s Normal SuccessfulCreate replicaset/webserver-d698d7bd6 Created pod: webserver-d698d7bd6-jwjjm 40s Normal SuccessfulCreate replicaset/webserver-d698d7bd6 Created pod: webserver-d698d7bd6-csxkb 31s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-jwjjm Successfully realized NSX resource for Pod 39s Normal Image pod/webserver-d698d7bd6-jwjjm Image nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 bound successfully 39s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 esx3.virten.lab: Image status changed to Resolving 39s Normal Image pod/webserver-d698d7bd6-csxkb Image nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 bound successfully 31s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-9nlbx Successfully realized NSX resource for Pod <unknown> Normal Scheduled pod/webserver-d698d7bd6-9nlbx Successfully assigned playground/webserver-d698d7bd6-9nlbx to esx5.virten.lab 39s Normal Image pod/webserver-d698d7bd6-9nlbx Image nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 bound successfully <unknown> Normal Scheduled pod/webserver-d698d7bd6-csxkb Successfully assigned playground/webserver-d698d7bd6-csxkb to esx5.virten.lab 31s Normal SuccessfulRealizeNSXResource pod/webserver-d698d7bd6-csxkb Successfully realized NSX resource for Pod 38s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 esx3.virten.lab: Image status changed to Resolving 37s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 esx2.virten.lab: Image status changed to Resolving 35s Normal Pulling pod/webserver-d698d7bd6-csxkb Waiting for Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 35s Normal SuccessfulRealizeNSXResource service/webserver Successfully realized NSX resource for Service 35s Normal Pulling pod/webserver-d698d7bd6-9nlbx Waiting for Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 35s Normal Pulling pod/webserver-d698d7bd6-jwjjm Waiting for Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 34s Normal Bind image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 Imagedisk f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d-v1487545 successfully bound 34s Normal Resolve image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 esx3.virten.lab: Image resolved to ChainID sha256:f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d 34s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 Image status changed to Ready 34s Normal Resolve image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 esx3.virten.lab: Image resolved to ChainID sha256:f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d 34s Normal Bind image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 Imagedisk f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d-v1487545 successfully bound 34s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 Image status changed to Ready 34s Normal Pulled pod/webserver-d698d7bd6-9nlbx Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v28985 is ready 34s Normal Pulled pod/webserver-d698d7bd6-jwjjm Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v49255 is ready 33s Normal Bind image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 Imagedisk f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d-v1487545 successfully bound 33s Normal Status image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 Image status changed to Ready 33s Normal Resolve image/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 esx2.virten.lab: Image resolved to ChainID sha256:f693fa68e7be60da5f2631f94268bb7231278a9e0a10e25798173ce6dd2e4d9d 33s Normal Pulled pod/webserver-d698d7bd6-csxkb Image playground/nginx-483640c55b8c28b65fdcc27be449ab8bf2454dd9-v12196 is ready 32s Normal Started pod/webserver-d698d7bd6-9nlbx Started container nginx 32s Normal Started pod/webserver-d698d7bd6-jwjjm Started container nginx 32s Normal Created pod/webserver-d698d7bd6-jwjjm Created container nginx 32s Normal SuccessfulMountVolume pod/webserver-d698d7bd6-9nlbx Successfully mounted volume default-token-b7kc7 32s Normal SuccessfulMountVolume pod/webserver-d698d7bd6-jwjjm Successfully mounted volume default-token-b7kc7 32s Normal Created pod/webserver-d698d7bd6-9nlbx Created container nginx 32s Warning FailedToUpdateEndpoint endpoints/webserver Failed to update endpoint playground/webserver: Operation cannot be fulfilled on endpoints "webserver": the object has been modified; please apply your changes to the latest version and try again 32s Normal Created pod/webserver-d698d7bd6-csxkb Created container nginx 32s Normal Started pod/webserver-d698d7bd6-csxkb Started container nginx 32s Normal SuccessfulMountVolume pod/webserver-d698d7bd6-csxkb Successfully mounted volume default-token-b7kc7
When pods are running you can use the kubectl logs <POD> command to get messages that have been written to stdout.
# kubectl get pods NAME READY STATUS RESTARTS AGE webserver-d698d7bd6-9nlbx 1/1 Running 0 18m webserver-d698d7bd6-csxkb 1/1 Running 0 18m webserver-d698d7bd6-jwjjm 1/1 Running 0 18m # kubectl logs webserver-d698d7bd6-9nlbx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: error: ipv6 not available /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up
You can also stream logs similar to "tail -f" with the kubectl logs <POD> -f.
Interacting with Pods
Use kubectl exec to run commands inside containers.
# kubectl exec webserver-d698d7bd6-9nlbx -- ls -l / total 68 drwxr-xr-x 2 root root 4096 Jul 20 00:00 bin drwxr-xr-x 2 root root 4096 May 2 16:39 boot drwxr-xr-x 1 root root 156 Aug 2 20:29 dev drwxr-xr-x 1 root root 4096 Jul 22 03:23 docker-entrypoint.d -rwxrwxr-x 1 root root 1202 Jul 22 03:22 docker-entrypoint.sh drwxr-xr-x 1 root root 38 Aug 2 20:29 etc drwxr-xr-x 2 root root 4096 May 2 16:39 home drwxr-xr-x 1 root root 4096 Jul 22 03:23 lib drwxr-xr-x 2 root root 4096 Jul 20 00:00 lib64 drwxr-xr-x 2 root root 4096 Jul 20 00:00 media drwxr-xr-x 2 root root 4096 Jul 20 00:00 mnt drwxr-xr-x 2 root root 4096 Jul 20 00:00 opt dr-xr-xr-x 111 root root 0 Aug 2 20:29 proc drwx------ 1 root root 27 Aug 2 21:06 root drwxr-xr-x 1 root root 38 Aug 2 20:29 run drwxr-xr-x 2 root root 4096 Jul 20 00:00 sbin drwxr-xr-x 2 root root 4096 Jul 20 00:00 srv drwxr-xr-x 2 root root 4096 May 2 16:39 sys drwxrwxrwt 1 root root 4096 Jul 22 03:23 tmp drwxr-xr-x 1 root root 4096 Jul 20 00:00 usr drwxr-xr-x 1 root root 19 Jul 20 00:00 var
The exec command can also be used to open an interactive shell inside the pod.
# kubectl exec --stdin --tty webserver-d698d7bd6-9nlbx -- /bin/bash root@webserver-d698d7bd6-9nlbx:/# ls -lh total 68K drwxr-xr-x 2 root root 4.0K Jul 20 00:00 bin drwxr-xr-x 2 root root 4.0K May 2 16:39 boot drwxr-xr-x 1 root root 156 Aug 2 20:29 dev drwxr-xr-x 1 root root 4.0K Jul 22 03:23 docker-entrypoint.d -rwxrwxr-x 1 root root 1.2K Jul 22 03:22 docker-entrypoint.sh drwxr-xr-x 1 root root 38 Aug 2 20:29 etc drwxr-xr-x 2 root root 4.0K May 2 16:39 home drwxr-xr-x 1 root root 4.0K Jul 22 03:23 lib drwxr-xr-x 2 root root 4.0K Jul 20 00:00 lib64 drwxr-xr-x 2 root root 4.0K Jul 20 00:00 media drwxr-xr-x 2 root root 4.0K Jul 20 00:00 mnt drwxr-xr-x 2 root root 4.0K Jul 20 00:00 opt dr-xr-xr-x 113 root root 0 Aug 2 20:29 proc drwx------ 1 root root 27 Aug 2 21:06 root drwxr-xr-x 1 root root 38 Aug 2 20:29 run drwxr-xr-x 2 root root 4.0K Jul 20 00:00 sbin drwxr-xr-x 2 root root 4.0K Jul 20 00:00 srv drwxr-xr-x 2 root root 4.0K May 2 16:39 sys drwxrwxrwt 1 root root 4.0K Jul 22 03:23 tmp drwxr-xr-x 1 root root 4.0K Jul 20 00:00 usr drwxr-xr-x 1 root root 19 Jul 20 00:00 var
vSphere with Kubernetes Guide |