KubeZero/docs/v1.25.md

# KubeZero 1.25

## TODO

## What's new - Major themes
- AppArmor is now available and enabled on all controller and worker nodes
- all KubeZero AMIs and containers updated to latest Alpine 3.17
- improved worker initialization, ensure cilium is running before any other pod
- metrics and Grafana dashboards added for Cilium agents, operator and cri-o
- metrics for any EC2 instances added, eg. node_exporter metrics from bastion and NAT instances, incl. support for cross region scraping

## Version upgrades
- cilium 1.13
- nvidia-device-plugin
- aws-neuron-driver
- aws-node-termination-handler
- aws-ebs-csi-driver
- aws-efs-csi-driver
- istio 1.17
- argocd 2.6.7
- fluent-bit 2.0.10
- fluentd 2.6
- all things prometheus

### FeatureGates
- CustomCPUCFSQuotaPeriod
- NodeOutOfServiceVolumeDetach

# Upgrade
`(No, really, you MUST read this before you upgrade)`

Ensure your Kube context points to the correct cluster !

1. Review CFN config for controller and workers, no mandatory changes during this release though

2. Upgrade CFN stacks for the control plane *ONLY* !
  Updating the workers CFN stacks would trigger rolling updates right away !

3. Trigger cluster upgrade:  
  `./admin/upgrade_cluster.sh <path to the argocd app kubezero yaml for THIS cluster>`

4. Review the kubezero-config and if all looks good commit the ArgoApp resouce for Kubezero via regular git  
  git add / commit / push `<cluster/env/kubezero/application.yaml>`  

5. Reboot controller(s) one by one  
Wait each time for controller to join and all pods running.
Might take a while ...

6. Upgrade CFN stacks for the workers.  
  This in turn will trigger automated worker updates by evicting pods and launching new workers in a rolling fashion.
  Grab a coffee and keep an eye on the cluster to be safe ...
  Depending on your cluster size it might take a while to roll over all workers!

7. Re-enable ArgoCD by hitting <return> on the still waiting upgrade script 

8. Quickly head over to ArgoCD and sync the KubeZero main module as soon as possible to reduce potential back and forth in case ArgoCD has legacy state


## Known issues
So far so good.