KubeZero 1.24

TODO

Cilium is now the default CNI, calico gets removed
cluster-autoscaler is enabled by default on AWS
worker nodes are now automatically update to latest AMI and config in a rolling fashion
integrated Bitnami Sealed Secrets controller

cilium
metallb
nvidia-device-plugin
aws-node-termination-handler
aws-ebs-csi-driver
aws-efs-csi-driver
istio 1.16
argocd 2.5.5 + tweaks
all things prometheus incl. automated muting of certain alarms, eg. CPUOverCommit when cluster-autoscaler is available

(No, really, you MUST read this before you upgrade)

Ensure your Kube context points to the correct cluster !

Review CFN config for controller and workers, no mandatory changes during this release though
Upgrade CFN stacks for the control plane ONLY ! Updating the workers CFN stacks would trigger rolling updates right away !
Trigger cluster upgrade:
./admin/upgrade_cluster.sh <path to the argocd app kubezero yaml for THIS cluster>
Reboot controller(s) one by one
Wait each time for controller to join and all pods running. Might take a while ...
Upgrade CFN stacks for the workers.
This in turn will trigger automated worker updates by evicting pods and launching new workers in a rolling fashion. Grab a coffee and keep an eye on the cluster to be safe ...
If all looks good, commit the ArgoApp resouce for Kubezero, before re-enabling ArgoCD itself.
git add / commit / push <cluster/env/kubezero/application.yaml>
Head over to ArgoCD and sync all KubeZero modules incl. pruning enabled to remove eg. Calico