Commit Graph

27 Commits

Author SHA1 Message Date
tomalok
88f3f1374e
Autodetect Current Revision of Alpine Version (#113)
* continue to use provided 'release' value if specified
* continue to use 'edge' for edge versions
* deduce 'release' value from the version on the alpine-base APK in https://dl-cdn.alpinelinux.org/alpine/v<version/main/<arch>/ 
* update test profile with 3.13
2021-02-02 20:13:33 -08:00
tomalok
2bf6727f67
Release Alpine 3.13.0 (#108)
3.13.0 release
* add version 3.13 profile
* 3.13 end-of-life is 2022-11-01
* note we're an official Alpine project now!
* releases/alpine.yaml has been "--trim release"'d
* updated releases/README.md

scripts/builder.py
* fix ReleaseReadme() to not die when release contains "_rc"
* add --trim to releases-yaml
* tweak some --help for release-yaml and prune

scripts/setup-ami
* explicitly lock AMI root account
2021-01-14 23:53:40 -08:00
tomalok
53fd1d27b1
Builder Overhaul (#106)
Subcommands
* merge 'resolve-profiles' and 'make-amis' into 'amis'
* rename 'update-releases' to 'release-yaml'
* rename 'gen-release-readme' to 'release-readme'
* rename 'prune-amis' to 'prune'
* reorder to match the usual workflow
* use argparse mutually-exclusive group where appropriate
* use argparse 'metavar' and 'nargs' for more salient help

release
* can now specify multiple AMIs on command line
* add explicit '--private' argument
* if no '--private', '--public', or '--allow-account' is specified, default to propagate the source AMI's permissions to its copies
* move 'iter_regions' and 'get*image' methods out of ReleaseAMIs class because they're also used elsewhere
* 'update_image_permissions' resets perms before adding new perms
* pending_copy loop, reports on everything in progress, waits 3m before reporting on everything again, and then waits 30s between reports
* pending_copy also notes when a copy has completed (and only queues for pending_perms if they need adjustment)

Releases class
* used by release-yaml and prune subcommands
* caches region client objects for later use (by prune)
* loads images from region - either from a profile or "unknown" (no profile tag)
* builds the releases object - now structured release -> build (instead of build -> release)

ReleasesReadme
* works with new releases object format
* improve sorting and selection of latest per version per-build AMIs
* empty cell if a region doesn't happen to have a build AMI there

PruneAMIs
* rename 'version' level to 'end-of-life'
* add 'UNKNOWN' pruning level
* works, even if you don't want to --use-broker
* --keep N - keeps an additional N AMIs that would otherwise have been purged per build
* --defer-eol DAYS - give EOL AMIs a grace period past their official EOL date
* no AMI deletion happens unless --no-pretend arg is provided
* improve pruning criteria scan and candidate selection

Co-authored-by: Jake Buchholz <jake@jakesys.net>
2021-01-04 17:36:15 -08:00
Jake Buchholz
e5b574f48f Post-Build Cleanup, etc.
scripts/builder.py...

GenReleaseReadme:
* combine with ReleaseReadmeUpdater
* generates README_<profile>.md
* README_alpine.md is a symlink to README.md
* don't crash when README doesn't preexist
* append image list to README if no list found to replace

MakeAMIs:
* collect all artifact IDs and report after all builds
* don't update releases/readme

PruneAMIs:
* defaults to pretend mode, unless --no-pretend
* improve readability

UpdateReleases:
* replace code with what was RefreshReleases
2020-12-22 15:42:20 -08:00
tomalok
4494aa4463
Release the latest versions (#101)
Release the latest versions
* v3.12.3 (x86_64 & aarch64)
* v3.11.7 (x86_64)
* v3.10.5-r1 (x86_64)
* today's edge (x86_64 & aarch64)
* sort the release AMIs by region

builder.py timings, roughly
* amis - 23m
* release (serial) - 1h38m
* refresh-releases - 4m
* gen-release-readme - instantaneous
2020-12-19 11:16:48 -08:00
tomalok
a530e331f3
Add refresh-releases subcommand, etc. (#97)
* Add refresh-releases subcommand, etc.

* builder.py
  + gen-release-readme
    - convert `build_time` to int
  + release
    - add `source_region` to copied AMI tags
    - check source AMI's permissions, queue for fixing, if necessary
  + refresh-releases
    - update releases/<profile>.yaml based on AMIs that exist in regions
  + explicitly call out `python-dateutil` dependency and `pip install` it into the venv

* Release Alpine 3.12.2 & today's edge
2020-12-14 22:24:29 -08:00
Mike Crute
3b4e395850
New Release Tool (#83)
* Add EC2 data types
* Add release command
2020-12-11 18:02:13 -08:00
Jake Buchholz
c6f5325873 ENI Hotplugging, etc.
ENI Hotplug / udhcpc script
* works with all Alpine versions back to 3.9
* udhcpc handles ENI's primary IPv4
* post-bound/post-renews eth-eni-hook handles secondary IPv4 & IPv6 addresses, route tables, and rules

setup-ami tweaks
* move scripts to be installed into setup-ami.d/
* move config snippets into setup-ami.d/etc/ (previously embedded in setup-ami)
2020-09-15 22:17:16 -07:00
Jake Buchholz
a9ba2532df udhcpc hooks for ENI IPv6 & secondary IPv4
Automatically sets up any IPv6 and secondary IPv4 on instance ENIs when DHCP leases are bound or renewed on that interface.

Resolves #70
2020-09-03 15:04:26 -07:00
Mike Crute
27491bcb20 Add argument checking for commands 2020-08-22 20:56:18 +00:00
Mike Crute
4df71cdc07 Use logging instead of print 2020-08-22 20:56:18 +00:00
Mike Crute
62262b6630 Fix rate-limiting error 2020-08-22 20:56:18 +00:00
tomalok
10058c1113
Fix 'revision' and 'end_of_life' (#80)
Also...
* update alpine.conf with 3.12
* update apk-tools and alpine-keys
* use test profile to test fixes and newer features
2020-08-17 11:11:50 -07:00
Jake Buchholz
2b76c6ebf6 support modification of default AMI user 2020-08-12 18:12:22 -07:00
Jake Buchholz
d593de3833 Optional Additional Setup
Profiles can specify 'setup_script' to do additional things.  If additional files/dirs are required, a 'setup_copy' map will copy them to the build instance so that 'setup_script' can use/install them.

TBD: docs.
2020-08-12 18:12:22 -07:00
Mike Crute
8a09fdda0e Add identity broker docs 2020-05-30 15:07:45 -07:00
Mike Crute
90f7408fc7 Flip broker usage flag 2020-05-30 15:07:45 -07:00
Mike Crute
b804661ffd Remove python3.8 dependency 2020-05-30 15:07:45 -07:00
Mike Crute
83d07e4b9a Initfs features are in profiles 2020-05-30 13:17:56 -07:00
Mike Crute
812eba9597 Remove release vars from packer 2020-05-30 13:17:56 -07:00
Mike Crute
b804d174b3 Allow building without broker 2020-05-30 12:14:48 -07:00
Mike Crute
df53323de9 Add some more python docs 2020-05-29 21:11:25 -07:00
Mike Crute
b53492723d Migrate full ami build to builder script 2020-05-29 20:58:53 -07:00
Mike Crute
5b2f32c9c8 Allow builder to be run by itself 2020-05-29 20:58:11 -07:00
Mike Crute
41f127d77b Authenticate per-region for make-amis 2020-05-28 17:57:13 -07:00
Mike Crute
bfc4bf99bf Convert packer.json builder to python 2020-05-28 17:55:46 -07:00
Mike Crute
7e60c7fb6a Combine all commands into a meta-command
There is an increasing need to share components among the various
commands, especially with the introduction of the identity broker API.
Rather than trying to assemble an importable python library of code for
the build process I think we can just combine everything into one file
and use argparse sub-commands to integrate them into a set of
individually callable scripts. This change does that integration.
2020-05-28 17:55:36 -07:00