Commit Graph

167 Commits

Author SHA1 Message Date
Jake Buchholz Göktürk 4f754f8d59 Merge branch 'initial/part_two' into 'main'
alpine-cloud-images, part two

See merge request tomalok/alpine-cloud-images!127
2021-11-23 06:09:18 +00:00
Jake Buchholz Göktürk 6674286b46 alpine-cloud-images, part two 2021-11-23 06:09:18 +00:00
Jake Buchholz Göktürk 9d882bff7a Merge branch 'initial/part_one' into 'main'
alpine-cloud-images, part one

See merge request tomalok/alpine-cloud-images!126
2021-11-07 21:35:17 +00:00
Jake Buchholz Göktürk e01e56bfa3 alpine-cloud-images, part one
This is the first MR to replace !125, and contains everything except the new python stuff -- which is part two.
2021-11-07 12:37:56 -08:00
tomalok dd5b5ea415
released 3.14.1 (#124) 2021-08-06 22:25:08 -07:00
tomalok b8ac181435
Update for Version 3.14 (#122)
Update for Version 3.14

* remove 3.10 build
* fix aarch64 AMI's /etc/default/grub (resolves #121)
* bump revision for aarch64 3.13 & 3.12 (to rebuild with fix)
* fix comment on 3.13
* fix comment on 3.14
* new 3.14.0 AMIs released
2021-06-15 12:34:38 -07:00
Mike Crute b578a39eb7 Remove backport hack for tiny-ec2-bootstrap 2021-05-03 09:49:16 -07:00
tomalok 3b7fd13083
new Alpine releases (#117) 2021-04-14 18:48:02 -07:00
tomalok 19dc1beaa6
New releases (3.13.4, 3.12.6, 3.11.10, 3.10.8) (#116)
Addresses CVE-2021-28831
2021-03-31 18:53:54 -07:00
tomalok 1fb5e5b663
Alpine 3.13.2 released (#115) 2021-02-17 08:49:13 -08:00
tomalok 88f3f1374e
Autodetect Current Revision of Alpine Version (#113)
* continue to use provided 'release' value if specified
* continue to use 'edge' for edge versions
* deduce 'release' value from the version on the alpine-base APK in https://dl-cdn.alpinelinux.org/alpine/v<version/main/<arch>/ 
* update test profile with 3.13
2021-02-02 20:13:33 -08:00
tomalok 30550530fe
Alpine Linux 3.13.1 released (#111) 2021-01-29 19:25:21 -08:00
tomalok 2bf6727f67
Release Alpine 3.13.0 (#108)
3.13.0 release
* add version 3.13 profile
* 3.13 end-of-life is 2022-11-01
* note we're an official Alpine project now!
* releases/alpine.yaml has been "--trim release"'d
* updated releases/README.md

scripts/builder.py
* fix ReleaseReadme() to not die when release contains "_rc"
* add --trim to releases-yaml
* tweak some --help for release-yaml and prune

scripts/setup-ami
* explicitly lock AMI root account
2021-01-14 23:53:40 -08:00
Mike Crute 644ac6b2b7 These are now official 2021-01-14 18:35:59 +00:00
tomalok 53fd1d27b1
Builder Overhaul (#106)
Subcommands
* merge 'resolve-profiles' and 'make-amis' into 'amis'
* rename 'update-releases' to 'release-yaml'
* rename 'gen-release-readme' to 'release-readme'
* rename 'prune-amis' to 'prune'
* reorder to match the usual workflow
* use argparse mutually-exclusive group where appropriate
* use argparse 'metavar' and 'nargs' for more salient help

release
* can now specify multiple AMIs on command line
* add explicit '--private' argument
* if no '--private', '--public', or '--allow-account' is specified, default to propagate the source AMI's permissions to its copies
* move 'iter_regions' and 'get*image' methods out of ReleaseAMIs class because they're also used elsewhere
* 'update_image_permissions' resets perms before adding new perms
* pending_copy loop, reports on everything in progress, waits 3m before reporting on everything again, and then waits 30s between reports
* pending_copy also notes when a copy has completed (and only queues for pending_perms if they need adjustment)

Releases class
* used by release-yaml and prune subcommands
* caches region client objects for later use (by prune)
* loads images from region - either from a profile or "unknown" (no profile tag)
* builds the releases object - now structured release -> build (instead of build -> release)

ReleasesReadme
* works with new releases object format
* improve sorting and selection of latest per version per-build AMIs
* empty cell if a region doesn't happen to have a build AMI there

PruneAMIs
* rename 'version' level to 'end-of-life'
* add 'UNKNOWN' pruning level
* works, even if you don't want to --use-broker
* --keep N - keeps an additional N AMIs that would otherwise have been purged per build
* --defer-eol DAYS - give EOL AMIs a grace period past their official EOL date
* no AMI deletion happens unless --no-pretend arg is provided
* improve pruning criteria scan and candidate selection

Co-authored-by: Jake Buchholz <jake@jakesys.net>
2021-01-04 17:36:15 -08:00
Jake Buchholz e5b574f48f Post-Build Cleanup, etc.
scripts/builder.py...

GenReleaseReadme:
* combine with ReleaseReadmeUpdater
* generates README_<profile>.md
* README_alpine.md is a symlink to README.md
* don't crash when README doesn't preexist
* append image list to README if no list found to replace

MakeAMIs:
* collect all artifact IDs and report after all builds
* don't update releases/readme

PruneAMIs:
* defaults to pretend mode, unless --no-pretend
* improve readability

UpdateReleases:
* replace code with what was RefreshReleases
2020-12-22 15:42:20 -08:00
tomalok 4494aa4463
Release the latest versions (#101)
Release the latest versions
* v3.12.3 (x86_64 & aarch64)
* v3.11.7 (x86_64)
* v3.10.5-r1 (x86_64)
* today's edge (x86_64 & aarch64)
* sort the release AMIs by region

builder.py timings, roughly
* amis - 23m
* release (serial) - 1h38m
* refresh-releases - 4m
* gen-release-readme - instantaneous
2020-12-19 11:16:48 -08:00
tomalok a530e331f3
Add refresh-releases subcommand, etc. (#97)
* Add refresh-releases subcommand, etc.

* builder.py
  + gen-release-readme
    - convert `build_time` to int
  + release
    - add `source_region` to copied AMI tags
    - check source AMI's permissions, queue for fixing, if necessary
  + refresh-releases
    - update releases/<profile>.yaml based on AMIs that exist in regions
  + explicitly call out `python-dateutil` dependency and `pip install` it into the venv

* Release Alpine 3.12.2 & today's edge
2020-12-14 22:24:29 -08:00
Mike Crute 3b4e395850
New Release Tool (#83)
* Add EC2 data types
* Add release command
2020-12-11 18:02:13 -08:00
tomalok 20ee5f5bc1
Define Bootloader in Profiles, etc. (#94)
* make it easier to switch between bootloaders
* experimental (non-working) EFI_STUB bootloader
* remove apk_tools & alpine_keys from profiles
* determine & install appropriate apk_toosl & alpine_keys in setup-ami based on version and arch.
2020-12-11 17:43:27 -08:00
tomalok 6e252ce9de
Fix 3.12.1 aarch64 Root Resize (#93)
release alpine-ami-3.12.1-aarch64-r1

Also...
* release new edge builds
* Alpine 3.9 is EOL
* build_instance_type set in profiles/arch/
* a couple comment fixes
2020-11-22 16:16:45 -08:00
Jake Buchholz 24bf01621f Fix assembly of /etc/network/interfaces
Include the other interface configs from /etc/network/interfaces.d/ after lo
and existing eth*, as was initially intended.

Also separate out the assembly code into its own script, as this is done both
on boot via eth-eni-setup and on hotplug events by etc-eni-hotplug.

Resolves #91
2020-11-15 20:31:36 -08:00
Jake Buchholz 9ddcac75c3 s/private IPv4/secondary IPv4/ 2020-11-05 08:17:24 -08:00
Jake Buchholz 909e9fd2a8 specify _Alpine_ release 3.12.1 2020-11-05 08:17:24 -08:00
Jake Buchholz e578d23975 add missing ) after (launch 2020-11-05 08:17:24 -08:00
Jake Buchholz 2ffd039692 fix collapsable sections 2020-11-05 08:17:24 -08:00
Jake Buchholz 79beab0f97 grammar tweak 2020-11-05 08:17:24 -08:00
Jake Buchholz c49106aadf Update Releases for 3.12.1 2020-11-05 08:17:24 -08:00
tomalok 38e7137662
Update to 3.12.1 (#88)
* Update to Alpine Linux v3.12.1
* Switch to using t3a instances for x86_64 buidls
2020-10-30 11:34:48 -07:00
tomalok e42c833553
eth-eni-setup init script (#87)
* eth-eni-setup init script

before networking starts up, makes sure eth interfaces match attached ENIs

also fixes a permissions problem with eth-eni-hotplug mdev config

* fix aarch64 build
2020-09-21 19:43:33 -07:00
Jake Buchholz 88e5a6d9cc profiles/README tweaks 2020-09-15 22:17:16 -07:00
Jake Buchholz e65b2d7720 * updates to READMEs 2020-09-15 22:17:16 -07:00
Jake Buchholz 1cce13e722 remove leftover set -x from setup-ami testing 2020-09-15 22:17:16 -07:00
Jake Buchholz c6f5325873 ENI Hotplugging, etc.
ENI Hotplug / udhcpc script
* works with all Alpine versions back to 3.9
* udhcpc handles ENI's primary IPv4
* post-bound/post-renews eth-eni-hook handles secondary IPv4 & IPv6 addresses, route tables, and rules

setup-ami tweaks
* move scripts to be installed into setup-ami.d/
* move config snippets into setup-ami.d/etc/ (previously embedded in setup-ami)
2020-09-15 22:17:16 -07:00
Jake Buchholz a9ba2532df udhcpc hooks for ENI IPv6 & secondary IPv4
Automatically sets up any IPv6 and secondary IPv4 on instance ENIs when DHCP leases are bound or renewed on that interface.

Resolves #70
2020-09-03 15:04:26 -07:00
Mike Crute 27491bcb20 Add argument checking for commands 2020-08-22 20:56:18 +00:00
Mike Crute 4df71cdc07 Use logging instead of print 2020-08-22 20:56:18 +00:00
Mike Crute 62262b6630 Fix rate-limiting error 2020-08-22 20:56:18 +00:00
tomalok bbd08c72fe
Fix nvme-ebs-links Installation (#81)
ensure that ownership and permissions are set properly
2020-08-17 19:06:08 -07:00
tomalok 10058c1113
Fix 'revision' and 'end_of_life' (#80)
Also...
* update alpine.conf with 3.12
* update apk-tools and alpine-keys
* use test profile to test fixes and newer features
2020-08-17 11:11:50 -07:00
Jake Buchholz 2b76c6ebf6 support modification of default AMI user 2020-08-12 18:12:22 -07:00
Jake Buchholz d593de3833 Optional Additional Setup
Profiles can specify 'setup_script' to do additional things.  If additional files/dirs are required, a 'setup_copy' map will copy them to the build instance so that 'setup_script' can use/install them.

TBD: docs.
2020-08-12 18:12:22 -07:00
tomalok d765bd4ab7
Fix edge builds (#78)
alpine-mirrors package removed after 3.12

also add profile for 3.12
2020-08-09 15:11:34 -07:00
Jake Buchholz 001db8ef99 Optional AWS Profile & Cross-Account Access
Allows encoding of the AWS profile to use in the build profile, and enabling the built AMI with a list of AWS accounts that are allowed access.
2020-06-14 16:10:30 -07:00
Mike Crute 925e62ceef Update README 2020-05-30 15:09:49 -07:00
Mike Crute 9d672fbd6a Support ARM instance types 2020-05-30 15:07:45 -07:00
Mike Crute 8a09fdda0e Add identity broker docs 2020-05-30 15:07:45 -07:00
Mike Crute 90f7408fc7 Flip broker usage flag 2020-05-30 15:07:45 -07:00
Mike Crute b804661ffd Remove python3.8 dependency 2020-05-30 15:07:45 -07:00
Mike Crute 83d07e4b9a Initfs features are in profiles 2020-05-30 13:17:56 -07:00