Commit Graph

105 Commits

Author SHA1 Message Date
tomalok
2bf6727f67
Release Alpine 3.13.0 (#108)
3.13.0 release
* add version 3.13 profile
* 3.13 end-of-life is 2022-11-01
* note we're an official Alpine project now!
* releases/alpine.yaml has been "--trim release"'d
* updated releases/README.md

scripts/builder.py
* fix ReleaseReadme() to not die when release contains "_rc"
* add --trim to releases-yaml
* tweak some --help for release-yaml and prune

scripts/setup-ami
* explicitly lock AMI root account
2021-01-14 23:53:40 -08:00
Mike Crute
644ac6b2b7 These are now official 2021-01-14 18:35:59 +00:00
tomalok
53fd1d27b1
Builder Overhaul (#106)
Subcommands
* merge 'resolve-profiles' and 'make-amis' into 'amis'
* rename 'update-releases' to 'release-yaml'
* rename 'gen-release-readme' to 'release-readme'
* rename 'prune-amis' to 'prune'
* reorder to match the usual workflow
* use argparse mutually-exclusive group where appropriate
* use argparse 'metavar' and 'nargs' for more salient help

release
* can now specify multiple AMIs on command line
* add explicit '--private' argument
* if no '--private', '--public', or '--allow-account' is specified, default to propagate the source AMI's permissions to its copies
* move 'iter_regions' and 'get*image' methods out of ReleaseAMIs class because they're also used elsewhere
* 'update_image_permissions' resets perms before adding new perms
* pending_copy loop, reports on everything in progress, waits 3m before reporting on everything again, and then waits 30s between reports
* pending_copy also notes when a copy has completed (and only queues for pending_perms if they need adjustment)

Releases class
* used by release-yaml and prune subcommands
* caches region client objects for later use (by prune)
* loads images from region - either from a profile or "unknown" (no profile tag)
* builds the releases object - now structured release -> build (instead of build -> release)

ReleasesReadme
* works with new releases object format
* improve sorting and selection of latest per version per-build AMIs
* empty cell if a region doesn't happen to have a build AMI there

PruneAMIs
* rename 'version' level to 'end-of-life'
* add 'UNKNOWN' pruning level
* works, even if you don't want to --use-broker
* --keep N - keeps an additional N AMIs that would otherwise have been purged per build
* --defer-eol DAYS - give EOL AMIs a grace period past their official EOL date
* no AMI deletion happens unless --no-pretend arg is provided
* improve pruning criteria scan and candidate selection

Co-authored-by: Jake Buchholz <jake@jakesys.net>
2021-01-04 17:36:15 -08:00
Jake Buchholz
e5b574f48f Post-Build Cleanup, etc.
scripts/builder.py...

GenReleaseReadme:
* combine with ReleaseReadmeUpdater
* generates README_<profile>.md
* README_alpine.md is a symlink to README.md
* don't crash when README doesn't preexist
* append image list to README if no list found to replace

MakeAMIs:
* collect all artifact IDs and report after all builds
* don't update releases/readme

PruneAMIs:
* defaults to pretend mode, unless --no-pretend
* improve readability

UpdateReleases:
* replace code with what was RefreshReleases
2020-12-22 15:42:20 -08:00
tomalok
4494aa4463
Release the latest versions (#101)
Release the latest versions
* v3.12.3 (x86_64 & aarch64)
* v3.11.7 (x86_64)
* v3.10.5-r1 (x86_64)
* today's edge (x86_64 & aarch64)
* sort the release AMIs by region

builder.py timings, roughly
* amis - 23m
* release (serial) - 1h38m
* refresh-releases - 4m
* gen-release-readme - instantaneous
2020-12-19 11:16:48 -08:00
tomalok
a530e331f3
Add refresh-releases subcommand, etc. (#97)
* Add refresh-releases subcommand, etc.

* builder.py
  + gen-release-readme
    - convert `build_time` to int
  + release
    - add `source_region` to copied AMI tags
    - check source AMI's permissions, queue for fixing, if necessary
  + refresh-releases
    - update releases/<profile>.yaml based on AMIs that exist in regions
  + explicitly call out `python-dateutil` dependency and `pip install` it into the venv

* Release Alpine 3.12.2 & today's edge
2020-12-14 22:24:29 -08:00
Mike Crute
3b4e395850
New Release Tool (#83)
* Add EC2 data types
* Add release command
2020-12-11 18:02:13 -08:00
tomalok
20ee5f5bc1
Define Bootloader in Profiles, etc. (#94)
* make it easier to switch between bootloaders
* experimental (non-working) EFI_STUB bootloader
* remove apk_tools & alpine_keys from profiles
* determine & install appropriate apk_toosl & alpine_keys in setup-ami based on version and arch.
2020-12-11 17:43:27 -08:00
tomalok
6e252ce9de
Fix 3.12.1 aarch64 Root Resize (#93)
release alpine-ami-3.12.1-aarch64-r1

Also...
* release new edge builds
* Alpine 3.9 is EOL
* build_instance_type set in profiles/arch/
* a couple comment fixes
2020-11-22 16:16:45 -08:00
Jake Buchholz
24bf01621f Fix assembly of /etc/network/interfaces
Include the other interface configs from /etc/network/interfaces.d/ after lo
and existing eth*, as was initially intended.

Also separate out the assembly code into its own script, as this is done both
on boot via eth-eni-setup and on hotplug events by etc-eni-hotplug.

Resolves #91
2020-11-15 20:31:36 -08:00
Jake Buchholz
9ddcac75c3 s/private IPv4/secondary IPv4/ 2020-11-05 08:17:24 -08:00
Jake Buchholz
909e9fd2a8 specify _Alpine_ release 3.12.1 2020-11-05 08:17:24 -08:00
Jake Buchholz
e578d23975 add missing ) after (launch 2020-11-05 08:17:24 -08:00
Jake Buchholz
2ffd039692 fix collapsable sections 2020-11-05 08:17:24 -08:00
Jake Buchholz
79beab0f97 grammar tweak 2020-11-05 08:17:24 -08:00
Jake Buchholz
c49106aadf Update Releases for 3.12.1 2020-11-05 08:17:24 -08:00
tomalok
38e7137662
Update to 3.12.1 (#88)
* Update to Alpine Linux v3.12.1
* Switch to using t3a instances for x86_64 buidls
2020-10-30 11:34:48 -07:00
tomalok
e42c833553
eth-eni-setup init script (#87)
* eth-eni-setup init script

before networking starts up, makes sure eth interfaces match attached ENIs

also fixes a permissions problem with eth-eni-hotplug mdev config

* fix aarch64 build
2020-09-21 19:43:33 -07:00
Jake Buchholz
88e5a6d9cc profiles/README tweaks 2020-09-15 22:17:16 -07:00
Jake Buchholz
e65b2d7720 * updates to READMEs 2020-09-15 22:17:16 -07:00
Jake Buchholz
1cce13e722 remove leftover set -x from setup-ami testing 2020-09-15 22:17:16 -07:00
Jake Buchholz
c6f5325873 ENI Hotplugging, etc.
ENI Hotplug / udhcpc script
* works with all Alpine versions back to 3.9
* udhcpc handles ENI's primary IPv4
* post-bound/post-renews eth-eni-hook handles secondary IPv4 & IPv6 addresses, route tables, and rules

setup-ami tweaks
* move scripts to be installed into setup-ami.d/
* move config snippets into setup-ami.d/etc/ (previously embedded in setup-ami)
2020-09-15 22:17:16 -07:00
Jake Buchholz
a9ba2532df udhcpc hooks for ENI IPv6 & secondary IPv4
Automatically sets up any IPv6 and secondary IPv4 on instance ENIs when DHCP leases are bound or renewed on that interface.

Resolves #70
2020-09-03 15:04:26 -07:00
Mike Crute
27491bcb20 Add argument checking for commands 2020-08-22 20:56:18 +00:00
Mike Crute
4df71cdc07 Use logging instead of print 2020-08-22 20:56:18 +00:00
Mike Crute
62262b6630 Fix rate-limiting error 2020-08-22 20:56:18 +00:00
tomalok
bbd08c72fe
Fix nvme-ebs-links Installation (#81)
ensure that ownership and permissions are set properly
2020-08-17 19:06:08 -07:00
tomalok
10058c1113
Fix 'revision' and 'end_of_life' (#80)
Also...
* update alpine.conf with 3.12
* update apk-tools and alpine-keys
* use test profile to test fixes and newer features
2020-08-17 11:11:50 -07:00
Jake Buchholz
2b76c6ebf6 support modification of default AMI user 2020-08-12 18:12:22 -07:00
Jake Buchholz
d593de3833 Optional Additional Setup
Profiles can specify 'setup_script' to do additional things.  If additional files/dirs are required, a 'setup_copy' map will copy them to the build instance so that 'setup_script' can use/install them.

TBD: docs.
2020-08-12 18:12:22 -07:00
tomalok
d765bd4ab7
Fix edge builds (#78)
alpine-mirrors package removed after 3.12

also add profile for 3.12
2020-08-09 15:11:34 -07:00
Jake Buchholz
001db8ef99 Optional AWS Profile & Cross-Account Access
Allows encoding of the AWS profile to use in the build profile, and enabling the built AMI with a list of AWS accounts that are allowed access.
2020-06-14 16:10:30 -07:00
Mike Crute
925e62ceef Update README 2020-05-30 15:09:49 -07:00
Mike Crute
9d672fbd6a Support ARM instance types 2020-05-30 15:07:45 -07:00
Mike Crute
8a09fdda0e Add identity broker docs 2020-05-30 15:07:45 -07:00
Mike Crute
90f7408fc7 Flip broker usage flag 2020-05-30 15:07:45 -07:00
Mike Crute
b804661ffd Remove python3.8 dependency 2020-05-30 15:07:45 -07:00
Mike Crute
83d07e4b9a Initfs features are in profiles 2020-05-30 13:17:56 -07:00
Mike Crute
b1da6a47d6 Embed nvme config 2020-05-30 13:17:56 -07:00
Mike Crute
a0cc4d8a12 Don't require copying scripts to build 2020-05-30 13:17:56 -07:00
Mike Crute
5ca05bfb46 Remove explicit build subnet 2020-05-30 13:17:56 -07:00
Mike Crute
812eba9597 Remove release vars from packer 2020-05-30 13:17:56 -07:00
Mike Crute
b804d174b3 Allow building without broker 2020-05-30 12:14:48 -07:00
Mike Crute
df53323de9 Add some more python docs 2020-05-29 21:11:25 -07:00
Mike Crute
b53492723d Migrate full ami build to builder script 2020-05-29 20:58:53 -07:00
Mike Crute
5b2f32c9c8 Allow builder to be run by itself 2020-05-29 20:58:11 -07:00
Mike Crute
41f127d77b Authenticate per-region for make-amis 2020-05-28 17:57:13 -07:00
Mike Crute
bfc4bf99bf Convert packer.json builder to python 2020-05-28 17:55:46 -07:00
Mike Crute
7e60c7fb6a Combine all commands into a meta-command
There is an increasing need to share components among the various
commands, especially with the introduction of the identity broker API.
Rather than trying to assemble an importable python library of code for
the build process I think we can just combine everything into one file
and use argparse sub-commands to integrate them into a set of
individually callable scripts. This change does that integration.
2020-05-28 17:55:36 -07:00
Mike Crute
1241cb5aaf Better model make dependencies on build files 2020-05-28 17:55:26 -07:00