There is an increasing need to share components among the various
commands, especially with the introduction of the identity broker API.
Rather than trying to assemble an importable python library of code for
the build process I think we can just combine everything into one file
and use argparse sub-commands to integrate them into a set of
individually callable scripts. This change does that integration.
This is paving the way for identity broker improvements for opt-in
regions. Eventually we'll need to hook some region logic into these
scripts so having them written in python will be helpful.
This is paving the way for identity broker improvements for opt-in
regions. The output is functionally identical between the two scripts
modulo the svcs change. Hopefully this makes the transformation process
a little more clear.
* sh doesn't allow nesting of prefix strip
* also update some minor test profile bits
* new AMI revisions (fixed nvme)
* edge AMI release should remain a timestamp
* switch build name from 'current-x86_64' to 'v#_#-x86_64' to avoid any confusion when new versions roll out
* resolvie-alpine.py.in - only warn about disabled regions once, instead of for each profile build
* make-amis - tweak script output
* new set of AMIs for edge, 3.10.0, and 3.9.4
* Makefile - improve/fix check for required make vars
* resolve-profile.py.in
+ build a list of all regions & probe to see which ones are enabled (unknown if special subscription regions like ap-northeast-3 would show up in this list)
+ expand 'ami_regions' 'ALL' meta key to all enabled regions, 'ALL' key's value is preserved (that is, a value of None or False will disable all regions)
+ warn/remove regions in profile config that are found to be disabled.
+ improve checks for [None, False] values
* .gitignore - don't ignore whole dirs and then opt-in specific files
* Makefile
- fail if required vars aren't set
- SCRIPTS --> ALL_SCRIPTS
* profiles/README.md
- variables.yaml --> vars.json
- drop 'not possible to add/modify/remove arbitrary files' comment
* make-amis
- use -eq for integer comparison
- make shellcheck happier with printf's
- remove old bad-idea TODO
* Build Profiles (completion of PR #49)
+ auto-updates version profile when new release detected
+ updates releases/<profile>.yaml after successful builds
* Prune AMIs (in AWS and in releases/<profile>.yaml
+ 'revision' - keep latest revision per release
+ 'release' - keep latest release per version
+ 'version' - remove end-of-life versions
* releases/README.md updater script
* README overhaul
+ Pre-built AMIs --> releases/README.md
+ profiles/README.md for profile configuration details
+ main README.md overhauled to go over how to build and manage custom AMIs
* EBS may prepend '/dev/' in front of the EBS alias, adjust the sanity sed to account for this.
* Attempt to get a sane EBS alias up to 50x, sleep 1/10s in between (max duration ~5 secs).
* Log when we add/fail-to-add/remove EBS alias symlinks.
Release a revised set of Alpine Linux AMIs, including...
* improved nvme-ebs-links mdev script (issue #40)
* start haveged at boot runlevel (issue #39)
* Release Alpine Linux 3.9.0 AMIs
* Update README.md and release.yaml with a fresh batch of 3.9.0 AMIs
* Append GitHub project link to AMI description
* really minor caveat fix
* Match meanings of 'version' and 'release' to how Alpine uses them
* Use optional 'revision' to denote any same-release AMI rebuild
* Include CPU 'arch' in naming/description (may also offer 'aarch64' AMIs someday)
* Upgrade build instance to use Amazon Linux 2 AMIs
* Use env vars to pass details to 'make_ami.sh' instead of via CLI parameters
* make_ami.sh
+ minimum version/release shouldn't be overrideable
+ update APK tools & Alpine keys
+ check build's release vs. installed /etc/alpine-release
* Allow additional services on the AMI's runlevels
I'm using this with my AMIs to add haveged to the boot runlevel to boost the amount of initial entropy on smaller instance types, so sshd can start in under 6s instead of over 2m.
add_svcs:
boot:
- haveged
* fix race condition with nvme-ebs /dev linking
* copy nvme stuff to build target in one operation
* add eu-north-1 region
* Latest Amazon Linux enables 64bid when creating ext4 partitions, which is incompatible with syslinux/extlinux bootloader. Explicitly disable 64bit support, as it's highly unlikely we'll need a boot volume >16 TiB.
* update-extlinux.conf - switch kernel default to 'virt', as 'hardened' no longer exists.
