# KubeZero 1.24

## TODO

## What's new - Major themes
- Cilium is now the default CNI, calico gets removed
- cluster-autoscaler is enabled by default on AWS
- worker nodes are now automatically update to latest AMI and config in a rolling fashion
- integrated Bitnami Sealed Secrets controller


## Version upgrades
- cilium
- metallb
- nvidia-device-plugin
- aws-node-termination-handler
- aws-ebs-csi-driver
- aws-efs-csi-driver
- istio 1.16
- argocd 2.5.5 + tweaks
- all things prometheus incl. automated muting of certain alarms, eg. CPUOverCommit when cluster-autoscaler is available

### FeatureGates
- PodAndContainerStatsFromCRI
- DelegateFSGroupToCSIDriver

# Upgrade
`(No, really, you MUST read this before you upgrade)`

Ensure your Kube context points to the correct cluster !

1. Review CFN config for controller and workers, no mandatory changes during this release though

2. Upgrade CFN stacks for the control plane *ONLY* !
  Updating the workers CFN stacks would trigger rolling updates right away !

3. Trigger cluster upgrade:  
  `./admin/upgrade_cluster.sh <path to the argocd app kubezero yaml for THIS cluster>`

4. Reboot controller(s) one by one  
Wait each time for controller to join and all pods running.
Might take a while ...

5. Upgrade CFN stacks for the workers.  
  This in turn will trigger automated worker updates by evicting pods and launching new workers in a rolling fashion.
  Grab a coffee and keep an eye on the cluster to be safe ...

6. If all looks good, commit the ArgoApp resouce for Kubezero, before re-enabling ArgoCD itself.  
  git add / commit / push `<cluster/env/kubezero/application.yaml>`

7. Head over to ArgoCD and sync all KubeZero modules incl. `pruning` enabled to remove eg. Calico