Commit Graph

87 Commits (cache-manifests-tiers)

Author SHA1 Message Date
Ricardo Pardini a70cb6852c add -e ENABLE_MANIFEST_CACHE=true to one the steps in test workflow. 2020-10-30 18:39:45 +01:00
Ricardo Pardini 68325a2945 add -e ENABLE_MANIFEST_CACHE=true to examples, some wording changes 2020-10-30 18:36:01 +01:00
Ricardo Pardini 917fa0f179 add manifest caching/anti-ratelimit usage note to README 2020-10-30 18:06:33 +01:00
Ricardo Pardini 3bfd778757 3-tier implementation of manifest caching; refactor config with includes, and generate from ENVs in entrypoint.sh
- disabled by default; enable with -e ENABLE_MANIFEST_CACHE=true
- default times and regexes are a wild guess, make sure to tune for your use case.
2020-10-30 16:50:54 +01:00
Ricardo Pardini 227a397225 Link image to original repository on GitHub via LABEL 2020-10-30 13:24:08 +01:00
Ricardo Pardini 937b06e593 release 0.5.0; update README to point to ghcr.io as well as DockerHub 2020-10-30 13:14:39 +01:00
Ricardo Pardini 427dd066bd GitHub Actions: tags/releases to GitHub Container Registry too 2020-10-30 13:09:27 +01:00
Ricardo Pardini eafedba51b GitHub Actions: publish :latest and :latest-debug also to GitHub Container Registry
- DockerHub is ok, but will be ratelimited soon, so this could be a nice alternative
2020-10-30 12:56:22 +01:00
Ricardo Pardini 8524e376c1 add request_type logging and headers; values are blob-by-digest, manifest-by-digest, manifest-mutable, blob-mutable 2020-10-30 12:33:41 +01:00
Ricardo Pardini 1856d821fc bump base image to nginx-1.18.0-alpine-3.12.1 2020-10-30 12:31:52 +01:00
Ricardo Pardini ae9cc68300 document DEBUG_HUB 2020-10-30 12:09:10 +01:00
Ricardo Pardini a2fd8bb32b upgrade to mitmproxy 5.x; do not store large bodies in memory; use separate config dirs for each mitmweb; better mitmweb logging 2020-10-30 12:02:56 +01:00
Ricardo Pardini 973f0d020b release 0.4.2; -debug version fixed, 429 ratelimit tolerance, 0s caching 2020-10-30 10:59:59 +01:00
Ricardo Pardini 411edf4c3f manifest caching: drop from 1s to 0s (expire immediately), so we can hit ratelimit faster (#54)
- mostly for sanity-checking purposes, last release in 0.4.x series.
- next release after this will make those configurable.
2020-10-30 10:57:54 +01:00
Ricardo Pardini 19ff1ccbcb GitHub Actions: fix building of -debug release versions by fixing build-args syntax in workflow files
- master/latest was already fixed.
2020-10-30 10:52:29 +01:00
Ricardo Pardini 763515267b use_stale for HTTP 429 Too Many Requests (actually used by DockerHub ratelimiting #54) instead of 403
- tested by trying to pull `ratelimitalways/test:latest`
2020-10-30 01:29:13 +01:00
Ricardo Pardini 36da1c2428 GitHub Actions: different syntax for build-args, lest :latest-debug is not really debug enabled 2020-10-30 00:41:13 +01:00
Ricardo Pardini aaf7fa0308 introduce env DEBUG_HUB; to help understand #54
- injects a mitmproxy between the caching layer and DockerHub's registry-1.docker.io
- it also disables VERIFY_SSL so the mitmproxy can be used;
- DEBUG_HUB's mitmweb is exposed on port 8082
- both DEBUG and DEBUG_HUB can be enabled independently
2020-10-30 00:20:18 +01:00
Ricardo Pardini ffc7540f43 add request method and upstream time to logging; add cache status header; helps investigate #54 2020-10-30 00:14:29 +01:00
Ricardo Pardini 78f96ffe92 badges 2020-10-27 12:48:26 +01:00
Ricardo Pardini 7aeffcfc4f badges 2020-10-20 16:11:41 +02:00
Ricardo Pardini 3c10258e6d 0.4.1 release, since I borked 0.4.0 2020-10-08 17:34:41 +02:00
Ricardo Pardini d57a8ce4f5 0.4.0 release 2020-10-08 16:50:45 +02:00
naftulee 16c3e250b7
access logs log_format to json (#44)
access logs log_format to json
2020-10-08 16:35:25 +02:00
Ricardo Pardini 0419eb4db7 during test, sleep 2s to allow the 1s stale manifest to occur 2020-10-08 16:28:13 +02:00
Brian Goff 6444380ea9
Cache immutable URLs and handle outages (#51)
Cache immutable URLs and handle outages
1. Changes the blob cache to only cache by digest
2. Adds caching for manifests requested by digest
3. Cache but invalidate immediately (after 1s) mutable requests
4. Use invalidated cache (from 3) if the backend is down or 403 unauthorized
2020-10-08 16:18:02 +02:00
Ricardo Pardini 1ecf949482
Merge pull request #52 from rpardini/latest-is-not-multiarch
Mention :latest is amd64 only
2020-10-08 15:50:17 +02:00
Ricardo Pardini b997ccd47c
Mention :latest is amd64 only 2020-10-08 15:43:31 +02:00
Ricardo Pardini 1fc3dfed4a master workflow, builds amd64-only :latest and :latest-debug
- arm64 via qemu is waaay too slow
- tags are still multiarch
2020-10-08 15:37:41 +02:00
Ricardo Pardini 10bac0816b test workflow, naming and cosmetics 2020-10-08 15:27:34 +02:00
Ricardo Pardini 95bbab0f93 test workflow, add logging, fixes 2020-10-08 15:14:37 +02:00
Ricardo Pardini b4646a91ce experimental test workflow 2020-10-08 14:23:54 +02:00
Ricardo Pardini f031bb1919 experimental test workflow 2020-10-08 13:30:17 +02:00
Ricardo Pardini ffc878671c update README for 0.4.0-pre1, debugging scenarios, plain no-auth caching, and some wording 2020-10-08 03:02:23 +02:00
Ricardo Pardini fe388f560b silence mitmweb, nginx logs are hard enough to read 2020-10-08 02:45:49 +02:00
Ricardo Pardini 67edf7efda fix python3 deps for mitmproxy/mitmweb (-e DEBUG=true) 2020-10-08 02:32:00 +02:00
Ricardo Pardini ed592a27a8 Github Actions: quoting troubles 2020-10-08 02:00:31 +02:00
Ricardo Pardini ec4df7b4b7 GitHub Actions: multiarch build, new -debug version, much lighter layers 2020-10-08 01:57:49 +02:00
Ricardo Pardini 6b4a26e23c README: reword, one less header level 2020-10-06 11:16:03 +02:00
Ricardo Pardini f2254685f8 split README into registry-specific sections; add GitLab example (fixes #27 thanks @bephinix); some rewording 2020-10-06 11:06:42 +02:00
ricardop a2c0afcbc4 upgrade to nginx-1.18.0-alpine-3.12; alpine 3.12 changed a lot of python/pip/wheel stuff 2020-09-22 10:15:15 +02:00
ricardop 2c8be197be fix, we need openssl package even in non-debug builds 2020-06-08 15:48:14 +02:00
ricardop 6a1fe384f5 introduce Docker build-args:
- BASE_IMAGE, used in FROM line
- DEBUG_BUILD, build mitmproxy or not
2020-06-08 14:49:21 +02:00
ricardop 67a81cf781 0.3.0-beta2 seems stable enough for a release 2020-04-07 12:22:22 +02:00
ricardop 19cbdfedfc 0.3.0-beta2: don't leak Authorization header from the registry to the redirected destination during @handle_redirects 2020-04-03 15:33:46 +02:00
ricardop bbd4d60bc4 update README; bump to 0.3.0-beta1 2020-04-03 14:16:09 +02:00
ricardop 056c8521f9 use specific mitmproxy version (latest in 4.x branch) 2020-04-03 14:12:43 +02:00
ricardop 72dc9a8819 use alpine's coreutils base64, which supports -w 0 to avoid wrapping -- thanks @miihael 2020-04-03 14:06:19 +02:00
ricardop e72389b690 use explicit include for the proxy cache directive. 2020-04-03 14:05:01 +02:00
ricardop bc0ac351bb Merge branch 'pull/33' 2020-04-03 13:50:54 +02:00