Commit Graph

112 Commits (0d6e3d68902026ce909c5ceefdfc333f1a06eca2)

Author SHA1 Message Date
Jonas Letzbor 0d6e3d6890
Add option to disable IPV6 for resolving 2023-12-27 15:19:41 +01:00
Ales Nosek fcf8fc1ca2 Mitmproxy won't start due to soft_unicode removed from markupsafe
After building the debug image and trying to run it, the mitproxy
would fail to start:

Traceback (most recent call last):
  File "/usr/bin/mitmweb", line 8, in <module>
    sys.exit(mitmweb())
  File "/usr/lib/python3.8/site-packages/mitmproxy/tools/_main.py", line 172, in mitmweb
    from mitmproxy.tools import web
  File "/usr/lib/python3.8/site-packages/mitmproxy/tools/web/__init__.py", line 1, in <module>
    from mitmproxy.tools.web import master
  File "/usr/lib/python3.8/site-packages/mitmproxy/tools/web/master.py", line 5, in <module>
    from mitmproxy import addons
  File "/usr/lib/python3.8/site-packages/mitmproxy/addons/__init__.py", line 12, in <module>
    from mitmproxy.addons import onboarding
  File "/usr/lib/python3.8/site-packages/mitmproxy/addons/onboarding.py", line 2, in <module>
    from mitmproxy.addons.onboardingapp import app
  File "/usr/lib/python3.8/site-packages/mitmproxy/addons/onboardingapp/__init__.py", line 3, in <module>
    from flask import Flask, render_template
  File "/usr/lib/python3.8/site-packages/flask/__init__.py", line 14, in <module>
    from jinja2 import escape
  File "/usr/lib/python3.8/site-packages/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/usr/lib/python3.8/site-packages/jinja2/defaults.py", line 3, in <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/usr/lib/python3.8/site-packages/jinja2/filters.py", line 13, in <module>
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/lib/python3.8/site-packages/markupsafe/__init__.py)

Fixed the issue by explicitly requiring an older version of MarkupSafe.

The issue is related to:
https://github.com/pallets/markupsafe/issues/282
2022-08-12 19:54:59 +02:00
Ales Nosek 00e29f22b8 Allow proxying to any destination port number (not 443 only)
The proxy refused to connect to a registry that was hosted on
a port other than 443. For example, I was not able to connect
to my registry that is hosted on port 5002:

$ https_proxy=proxy.lab.example.com:3128 curl -v https://registry.lab.example.com:5002
* Uses proxy env variable https_proxy == 'proxy.lab.example.com:3128'
*   Trying 192.168.140.1:3128...
* Connected to proxy.lab.example.com (192.168.140.1) port 3128 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to registry.lab.example.com:5002
> CONNECT registry.lab.example.com:5002 HTTP/1.1
> Host: registry.lab.example.com:5002
> User-Agent: curl/7.74.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 403 Forbidden
< Server: nginx/1.20.1
< Date: Thu, 11 Aug 2022 15:12:23 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
<
* Received HTTP code 403 from proxy after CONNECT
* CONNECT phase completed!
* Closing connection 0
curl: (56) Received HTTP code 403 from proxy after CONNECT

The proxy refused to pass through connections to URLs that used
port other than 443. For example, trying to connect to port 8443:

$ https_proxy=proxy.lab.example.com:3128 curl -v https://google.com:8443
* Uses proxy env variable https_proxy == 'proxy.lab.example.com:3128'
*   Trying 192.168.140.1:3128...
* Connected to proxy.lab.example.com (192.168.140.1) port 3128 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to google.com:8443
> CONNECT google.com:8443 HTTP/1.1
> Host: google.com:8443
> User-Agent: curl/7.74.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 403 Forbidden
< Server: nginx/1.20.1
< Date: Thu, 11 Aug 2022 16:05:52 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
<
* Received HTTP code 403 from proxy after CONNECT
* CONNECT phase completed!
* Closing connection 0
curl: (56) Received HTTP code 403 from proxy after CONNECT

This commit fixes the issue by configuring the proxy_connect_allow
paramater to allow connecting to any destination port number. By
default only port 443 and 563 were allowed. See also documentation
here:

https://github.com/chobits/ngx_http_proxy_connect_module#proxy_connect_allow
2022-08-12 19:54:53 +02:00
saces 45ad55d3f1 add option to disable request buffering.
If push is allowed, buffering requests can cause issues on slow upstreams.
The put requests pile up, and since `proxy_ignore_client_abort` is on...
...the push fails.

If you have trouble pushing, set `PROXY_REQUEST_BUFFERING`to false, then fix
remainig issues/timeouts.

Default is true to not change default behavior.
2022-01-10 23:17:49 +01:00
Mahmoud Saada 149da3b6f1
docs: k3d integration (#112)
* docs: k3d integration - Closes #111

* Update README.md
2022-01-05 23:57:42 +01:00
Ricardo Pardini 51ce56407a
use GITHUB_TOKEN instead of PAT for ghcr.io publishing 2021-09-03 16:45:36 +02:00
Ricardo Pardini 2456b35b5c
Merge pull request #99 from leorolland/master
doc: add kind cluster usage
2021-07-13 02:27:40 +02:00
Léo 4bb60c8c05 doc: add kind cluster usage in README 2021-07-12 15:13:48 +02:00
Ricardo Pardini c949381a10
nginx-1.18.0-alpine-3.12.1 -> nginx-1.20.1-alpine-3.12.7 2021-07-11 20:18:39 +02:00
Akos Dudas 26aee2e64a
Instructions for Windows Docker Desktop (#95)
* Instructions for Windows Docker Desktop

* Typo in doc: add missing word
2021-06-23 10:17:32 +02:00
Ricardo Pardini d0101cbac5
try to build & run on podman via one more hack to resolv.conf parsing
- why the hell does nginx need a resolver config, again?
2021-05-27 23:59:46 +02:00
Alex Thomae 2f55921a0d
Remove second check condition for resolvers.conf (#87) 2021-04-20 21:29:28 +02:00
Ricardo Pardini ce3c77b62a
trying to make ipv6 actually work 2021-02-14 14:18:38 +01:00
Mateusz Kowalski b9e5c932ce
Add support for IPv6 (#76) 2021-02-05 07:46:49 +01:00
Sezer Bülent 51585675e6
DRP-70: add timeoutes as ENVs, update README.md, update nginx config (#73)
Authored-by: Bulent <bt.sezer29@gmail.com>
2021-01-28 12:12:18 +01:00
Ricardo Pardini c52c7d3741 release 0.6.1 -- with no breaking changes, hopefully 2020-12-02 15:11:00 +01:00
Federico Gimenez 536f0fc8a0
Allow push (#59)
Introduce ALLOW_PUSH, if set to true, allows non-GET methods through the proxy
2020-12-02 14:58:19 +01:00
hishamanver dfb6a5dea2
add ca srl param input to openssl command (#66)
Co-authored-by: Hisham Anver <Hisham.Anver@team.telstra.com>
2020-12-02 14:57:27 +01:00
Ricardo Pardini 0b6da4be92 add working docker-compose example, fix #69 2020-12-02 14:51:41 +01:00
Jye Lee 8bfb0a9ef6
allow setting certs location so mounted cert folder doesnt need to match expected names (#38) 2020-11-14 09:57:54 +01:00
hishamanver 3bb9f15061
add ca_srl creation when custom cert and key are provided (#65)
Co-authored-by: Hisham Anver <Hisham.Anver@team.telstra.com>
2020-11-14 09:50:58 +01:00
Marcelo Grebois 18c9ca4983
Adding documentation on how to configure Kops to use registry-proxy (#64) 2020-11-14 09:09:03 +01:00
Ricardo Pardini 6dbce2f5cd Docker for Mac's Proxy GUI is a monster; avoid it by setting ENV directly in LinuxKit config 2020-11-09 16:25:04 +01:00
Ricardo Pardini c224375253
warning, Docker-for-Mac is a beast 2020-11-09 14:56:02 +01:00
Ricardo Pardini ae77f739d8 works both edge just as well as stable 2020-11-09 12:39:04 +01:00
Ricardo Pardini 93f3d5749f add some Docker Desktop for Mac instructions (#58, #50 and others) 2020-11-09 12:32:19 +01:00
Ricardo Pardini 6bf3974ade remove Makefile, some PR added it and I didn't notice; we don't use it. 2020-11-09 11:26:29 +01:00
Ricardo Pardini 313f5ea203 release 0.6.0 with ENABLE_MANIFEST_CACHE 2020-10-30 18:58:42 +01:00
Ricardo Pardini a726f88049
3-tier implementation of manifest caching (#57)
* implement manifest caching; refactor config with includes, and generate from ENVs in entrypoint.sh
- disabled by default; enable with -e ENABLE_MANIFEST_CACHE=true
- default times and regexes are a wild guess, make sure to tune for your use case.
-  add manifest caching/anti-ratelimit usage note to README
- add -e ENABLE_MANIFEST_CACHE=true to examples, some wording changes
- add -e ENABLE_MANIFEST_CACHE=true to one the steps in test workflow.
2020-10-30 18:54:10 +01:00
Ricardo Pardini 227a397225 Link image to original repository on GitHub via LABEL 2020-10-30 13:24:08 +01:00
Ricardo Pardini 937b06e593 release 0.5.0; update README to point to ghcr.io as well as DockerHub 2020-10-30 13:14:39 +01:00
Ricardo Pardini 427dd066bd GitHub Actions: tags/releases to GitHub Container Registry too 2020-10-30 13:09:27 +01:00
Ricardo Pardini eafedba51b GitHub Actions: publish :latest and :latest-debug also to GitHub Container Registry
- DockerHub is ok, but will be ratelimited soon, so this could be a nice alternative
2020-10-30 12:56:22 +01:00
Ricardo Pardini 8524e376c1 add request_type logging and headers; values are blob-by-digest, manifest-by-digest, manifest-mutable, blob-mutable 2020-10-30 12:33:41 +01:00
Ricardo Pardini 1856d821fc bump base image to nginx-1.18.0-alpine-3.12.1 2020-10-30 12:31:52 +01:00
Ricardo Pardini ae9cc68300 document DEBUG_HUB 2020-10-30 12:09:10 +01:00
Ricardo Pardini a2fd8bb32b upgrade to mitmproxy 5.x; do not store large bodies in memory; use separate config dirs for each mitmweb; better mitmweb logging 2020-10-30 12:02:56 +01:00
Ricardo Pardini 973f0d020b release 0.4.2; -debug version fixed, 429 ratelimit tolerance, 0s caching 2020-10-30 10:59:59 +01:00
Ricardo Pardini 411edf4c3f manifest caching: drop from 1s to 0s (expire immediately), so we can hit ratelimit faster (#54)
- mostly for sanity-checking purposes, last release in 0.4.x series.
- next release after this will make those configurable.
2020-10-30 10:57:54 +01:00
Ricardo Pardini 19ff1ccbcb GitHub Actions: fix building of -debug release versions by fixing build-args syntax in workflow files
- master/latest was already fixed.
2020-10-30 10:52:29 +01:00
Ricardo Pardini 763515267b use_stale for HTTP 429 Too Many Requests (actually used by DockerHub ratelimiting #54) instead of 403
- tested by trying to pull `ratelimitalways/test:latest`
2020-10-30 01:29:13 +01:00
Ricardo Pardini 36da1c2428 GitHub Actions: different syntax for build-args, lest :latest-debug is not really debug enabled 2020-10-30 00:41:13 +01:00
Ricardo Pardini aaf7fa0308 introduce env DEBUG_HUB; to help understand #54
- injects a mitmproxy between the caching layer and DockerHub's registry-1.docker.io
- it also disables VERIFY_SSL so the mitmproxy can be used;
- DEBUG_HUB's mitmweb is exposed on port 8082
- both DEBUG and DEBUG_HUB can be enabled independently
2020-10-30 00:20:18 +01:00
Ricardo Pardini ffc7540f43 add request method and upstream time to logging; add cache status header; helps investigate #54 2020-10-30 00:14:29 +01:00
Ricardo Pardini 78f96ffe92 badges 2020-10-27 12:48:26 +01:00
Ricardo Pardini 7aeffcfc4f badges 2020-10-20 16:11:41 +02:00
Ricardo Pardini 3c10258e6d 0.4.1 release, since I borked 0.4.0 2020-10-08 17:34:41 +02:00
Ricardo Pardini d57a8ce4f5 0.4.0 release 2020-10-08 16:50:45 +02:00
naftulee 16c3e250b7
access logs log_format to json (#44)
access logs log_format to json
2020-10-08 16:35:25 +02:00
Ricardo Pardini 0419eb4db7 during test, sleep 2s to allow the 1s stale manifest to occur 2020-10-08 16:28:13 +02:00