Development January 2019

development@lists.ipfire.org

16 participants
57 discussions

[PATCH] dracut: update to 048
by Arne Fitzenreiter 03 Jan '19

03 Jan '19

Signed-off-by: Arne Fitzenreiter <arne_f(a)ipfire.org> --- dracut/dracut.nm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dracut/dracut.nm b/dracut/dracut.nm index b7e72c2..8da9822 100644 --- a/dracut/dracut.nm +++ b/dracut/dracut.nm @@ -4,7 +4,7 @@ ############################################################################### name = dracut -version = 046 +version = 048 release = 1 groups = System/Boot -- 2.6.3

2 1

[PATCH] kmod: update to 25
by Arne Fitzenreiter 03 Jan '19

03 Jan '19

Signed-off-by: Arne Fitzenreiter <arne_f(a)ipfire.org> --- kmod/kmod.nm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kmod/kmod.nm b/kmod/kmod.nm index daba803..d14c49e 100644 --- a/kmod/kmod.nm +++ b/kmod/kmod.nm @@ -4,7 +4,7 @@ ############################################################################### name = kmod -version = 22 +version = 25 release = 1 maintainer = Michael Tremer <michael.tremer(a)ipfire.org> @@ -20,7 +20,7 @@ description filesystems are two examples of loaded and unloaded modules. end -source_dl = ftp://ftp.kernel.org/pub/linux/utils/kernel/kmod/ +source_dl = https://www.kernel.org/pub/linux/utils/kernel/kmod/ sources = %{thisapp}.tar.xz build -- 2.6.3

2 1

Core126 udevd problem
by Tapani Tarvainen 02 Jan '19

02 Jan '19

Core126 upgrade killed a machine: boot freezes after udevd start. Experimenting with fresh (flash) installation image (and new CF card) gave same result. Up to and including core125 it works like charm (also tested with a fresh image). This is a bit difficult to debug: the last message displayed is "Starting udev daemon... [ OK ]" After that even keyboard is dead (caps lock doesn't work, nor alt-ctrl-del). And as this happens before disks are mounted, there's no log to read. This is a rather old machine, VIA C3 CPU, 512MB RAM, booting of CF card, but there's nothing obviously wrong with the hardware, and as noted it worked just fine up to and including core125. I could leave it running core125, but a firewall that can't be upgraded is not a workable long-term proposition. Any suggestions as to how I could try to track the problem down? -- Tapani Tarvainen

2 4

[PATCH] mc: Update to 4.8.22
by Matthias Fischer 02 Jan '19

02 Jan '19

For details see: http://midnight-commander.org/wiki/NEWS-4.8.22 Best, Matthias Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org> --- config/rootfiles/packages/mc | 2 ++ lfs/mc | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/config/rootfiles/packages/mc b/config/rootfiles/packages/mc index 4733af6fa..06c536ed4 100644 --- a/config/rootfiles/packages/mc +++ b/config/rootfiles/packages/mc @@ -179,6 +179,7 @@ usr/share/mc/syntax/m4.syntax usr/share/mc/syntax/mail.syntax usr/share/mc/syntax/makefile.syntax usr/share/mc/syntax/markdown.syntax +usr/share/mc/syntax/meson.syntax usr/share/mc/syntax/ml.syntax usr/share/mc/syntax/named.syntax usr/share/mc/syntax/nemerle.syntax @@ -198,6 +199,7 @@ usr/share/mc/syntax/puppet.syntax usr/share/mc/syntax/python.syntax usr/share/mc/syntax/r.syntax usr/share/mc/syntax/ruby.syntax +usr/share/mc/syntax/rust.syntax usr/share/mc/syntax/sh.syntax usr/share/mc/syntax/slang.syntax usr/share/mc/syntax/smalltalk.syntax diff --git a/lfs/mc b/lfs/mc index 1aa07c133..6600702e5 100644 --- a/lfs/mc +++ b/lfs/mc @@ -24,7 +24,7 @@ include Config -VER = 4.8.20 +VER = 4.8.22 THISAPP = mc-$(VER) DL_FILE = $(THISAPP).tar.xz @@ -32,7 +32,7 @@ DL_FROM = $(URL_IPFIRE) DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP) PROG = mc -PAK_VER = 16 +PAK_VER = 17 DEPS = "" @@ -44,7 +44,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_MD5 = 7f808b01f3f7d9aa52152a9efb86dbca +$(DL_FILE)_MD5 = 62bba2bfccadf08eb554321899f475bc install : $(TARGET) -- 2.18.0

2 1

Bugzilla down
by Matthias Fischer 02 Jan '19

02 Jan '19

Hi, it seems I'm the one with the bad news these days... ;-) If someone has the time - and the nerves: https://bugzilla.ipfire.org doesn't talk to us anymore, it only says: "Bad Request Your browser sent a request that this server could not understand." Best regards for the coming New Year, Matthias

2 1

Happy new year
by ummeegge 02 Jan '19

02 Jan '19

Dear all, wanted to wish you and your families a happy great new year. Wanted also to thank you all for your effort and your innovation to give this project the dynamic and life which makes it simply that what it is... Great. All the best and may the code be with you :-) , Erik

3 2

[PATCH] squid 4.4: latest patches (04-08)
by Matthias Fischer 01 Jan '19

01 Jan '19

For details see: http://www.squid-cache.org/Versions/v4/changesets/ Best, Matthias Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org> --- lfs/squid | 5 + ...documentation_and_implementation_277.patch | 164 ++++++++++ ...onfig_when_configure-ing_libxml2_338.patch | 89 ++++++ ...s_Improved_lexgrog_compatibility_340.patch | 24 ++ ...from_accessing_some_web_contents_304.patch | 285 ++++++++++++++++++ ..._a_sslcrtvalidator_used_together_328.patch | 24 ++ .../squid-4.4-fix-max-file-descriptors.patch | 4 +- 7 files changed, 593 insertions(+), 2 deletions(-) create mode 100644 src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch create mode 100644 src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch create mode 100644 src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch create mode 100644 src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch create mode 100644 src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch diff --git a/lfs/squid b/lfs/squid index aaa2d0b96..113b4c3aa 100644 --- a/lfs/squid +++ b/lfs/squid @@ -75,6 +75,11 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/01_Fix_netdb_exchange_with_a_TLS_cache_peer_307.patch cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/02_Maintenance_add_xz_tarball_format_formally_to_make_dist_325.patch cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/03_The_handshake_logformat_code_331.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch cd $(DIR_APP) && autoreconf -vfi diff --git a/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch b/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch new file mode 100644 index 000000000..52c75e097 --- /dev/null +++ b/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch @@ -0,0 +1,164 @@ +commit b3699887c2ffa807c65f153639cd13feb93f4c0f +Author: Eduard Bagdasaryan <eduard.bagdasaryan(a)measurement-factory.com> +Date: 2018-11-28 23:55:29 +0000 + + Fixed forward_max_tries documentation and implementation (#277) + + Before 1c8f25b, FwdState::n_tries counted the total number of forwarding + attempts, including pinned and persistent connection retries. Since that + revision, it started counting just those retries. What should n_tries + count? The counter is used to honor the forward_max_tries directive, but + that directive was documented to limit the number of _different_ paths + to try. Neither 1c8f25b~1 nor 1c8f25b code matched that documentation! + + Continuing to count just pinned and persistent connection retries (as in + 1c8f25b) would violate any reasonable forward_max_tries intent and admin + expectations. There are two ways to fix this problem, synchronizing code + and documentation: + + * Count just the attempts to use a different forwarding path, matching + forward_max_tries documentation but not what Squid has ever done. This + approach makes it difficult for an admin to limit the total number of + forwarding attempts in environments where, say, the second attempt is + unlikely to succeed and will just incur wasteful delays (Squid bug + 4788 report is probably about one of such use cases). Also, + implementing this approach may be more difficult because it requires + adding a new counter for retries and, for some interpretations of + "different", even a container of previously visited paths. + + * Count all forwarding attempts (as before 1c8f25b) and adjust + forward_max_tries documentation to match this historical behavior. + This approach does not have known unique flaws. + + Also fixed FwdState::n_tries off-by-one comparison bug discussed during + Squid bug 4788 triage. + + Also fixed admin concern behind Squid bug 4788 "forward_max_tries 1 does + not prevent some retries": While the old forward_max_tries documentation + actually excluded pconn retries, technically invalidating the bug + report, the admin now has a knob to limit those retries. + +diff --git a/src/FwdState.cc b/src/FwdState.cc +index f201333..b218240 100644 +--- a/src/FwdState.cc ++++ b/src/FwdState.cc +@@ -587,7 +587,7 @@ FwdState::checkRetry() + if (!entry->isEmpty()) + return false; + +- if (n_tries > Config.forward_max_tries) ++ if (exhaustedTries()) + return false; + + if (!EnoughTimeToReForward(start_t)) +@@ -921,6 +921,7 @@ FwdState::connectStart() + Comm::ConnOpener *cs = new Comm::ConnOpener(serverDestinations[0], calls.connector, connTimeout); + if (host) + cs->setHost(host); ++ ++n_tries; + AsyncJob::Start(cs); + } + +@@ -1072,7 +1073,7 @@ FwdState::reforward() + return 0; + } + +- if (n_tries > Config.forward_max_tries) ++ if (exhaustedTries()) + return 0; + + if (request->bodyNibbled()) +@@ -1222,6 +1223,12 @@ FwdState::logReplyStatus(int tries, const Http::StatusCode status) + ++ FwdReplyCodes[tries][status]; + } + ++bool ++FwdState::exhaustedTries() const ++{ ++ return n_tries >= Config.forward_max_tries; ++} ++ + /**** PRIVATE NON-MEMBER FUNCTIONS ********************************************/ + + /* +diff --git a/src/FwdState.h b/src/FwdState.h +index 6619c64..85f0fb9 100644 +--- a/src/FwdState.h ++++ b/src/FwdState.h +@@ -127,6 +127,9 @@ private: + void syncWithServerConn(const char *host); + void syncHierNote(const Comm::ConnectionPointer &server, const char *host); + ++ /// whether we have used up all permitted forwarding attempts ++ bool exhaustedTries() const; ++ + public: + StoreEntry *entry; + HttpRequest *request; +@@ -139,7 +142,7 @@ private: + ErrorState *err; + Comm::ConnectionPointer clientConn; ///< a possibly open connection to the client. + time_t start_t; +- int n_tries; ++ int n_tries; ///< the number of forwarding attempts so far + + // AsyncCalls which we set and may need cancelling. + struct { +diff --git a/src/cf.data.pre b/src/cf.data.pre +index a8ca587..7a7d271 100644 +--- a/src/cf.data.pre ++++ b/src/cf.data.pre +@@ -3791,11 +3791,15 @@ DEFAULT: 25 + TYPE: int + LOC: Config.forward_max_tries + DOC_START +- Controls how many different forward paths Squid will try +- before giving up. See also forward_timeout. ++ Limits the number of attempts to forward the request. ++ ++ For the purpose of this limit, Squid counts all high-level request ++ forwarding attempts, including any same-destination retries after ++ certain persistent connection failures and any attempts to use a ++ different peer. However, low-level connection reopening attempts ++ (enabled using connect_retries) are not counted. + +- NOTE: connect_retries (default: none) can make each of these +- possible forwarding paths be tried multiple times. ++ See also: forward_timeout and connect_retries. + DOC_END + + COMMENT_START +@@ -9854,19 +9858,23 @@ LOC: Config.connect_retries + DEFAULT: 0 + DEFAULT_DOC: Do not retry failed connections. + DOC_START +- This sets the maximum number of connection attempts made for each +- TCP connection. The connect_retries attempts must all still +- complete within the connection timeout period. ++ Limits the number of reopening attempts when establishing a single ++ TCP connection. All these attempts must still complete before the ++ applicable connection opening timeout expires. ++ ++ By default and when connect_retries is set to zero, Squid does not ++ retry failed connection opening attempts. + +- The default is not to re-try if the first connection attempt fails. +- The (not recommended) maximum is 10 tries. ++ The (not recommended) maximum is 10 tries. An attempt to configure a ++ higher value results in the value of 10 being used (with a warning). + +- A warning message will be generated if it is set to a too-high +- value and the configured value will be over-ridden. ++ Squid may open connections to retry various high-level forwarding ++ failures. For an outside observer, that activity may look like a ++ low-level connection reopening attempt, but those high-level retries ++ are governed by forward_max_tries instead. + +- Note: These re-tries are in addition to forward_max_tries +- which limit how many different addresses may be tried to find +- a useful server. ++ See also: connect_timeout, forward_timeout, icap_connect_timeout, ++ ident_timeout, and forward_max_tries. + DOC_END + + NAME: retry_on_error diff --git a/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch b/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch new file mode 100644 index 000000000..7685a9637 --- /dev/null +++ b/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch @@ -0,0 +1,89 @@ +commit c34582b9e8c40529db7eb9c490b081a37972f6d4 +Author: Fabrice Fontaine <fontaine.fabrice(a)gmail.com> +Date: 2018-12-04 17:23:25 +0000 + + Use and prefer pkg-config when ./configure-ing libxml2 (#338) + + The old method of trying to locate libxml2 via AC_CHECK_LIB() is + preserved to accommodate environments without pkg-config. + + pkg-config knows about libxml2 dependencies like lz or -liconv that the + old method misses. + +diff --git a/configure.ac b/configure.ac +index f668567..ef20a53 100644 +--- a/configure.ac ++++ b/configure.ac +@@ -922,41 +922,45 @@ fi + + AC_ARG_WITH(libxml2, AS_HELP_STRING([--without-libxml2],[Do not use libxml2 for ESI. Default: auto-detect])) + if test "x$squid_opt_use_esi" != "xno" -a "x$with_libxml2" != "xno" ; then +- AC_CHECK_LIB([xml2], [main], [XMLLIB="-lxml2"; HAVE_LIBXML2=1]) +- dnl Find the main header and include path... +- AC_CACHE_CHECK([location of libxml2 include files], [ac_cv_libxml2_include], [ +- AC_CHECK_HEADERS([libxml/parser.h], [], [ +- AC_MSG_NOTICE([Testing in /usr/include/libxml2]) +- SAVED_CPPFLAGS="$CPPFLAGS" +- CPPFLAGS="-I/usr/include/libxml2 $CPPFLAGS" +- unset ac_cv_header_libxml_parser_h +- AC_CHECK_HEADERS([libxml/parser.h], [ac_cv_libxml2_include="-I/usr/include/libxml2"], [ +- AC_MSG_NOTICE([Testing in /usr/local/include/libxml2]) +- CPPFLAGS="-I/usr/local/include/libxml2 $SAVED_CPPFLAGS" ++ SQUID_STATE_SAVE([squid_libxml2_save]) ++ PKG_CHECK_MODULES([LIBXML2],[libxml-2.0],[],[ ++ AC_CHECK_LIB([xml2], [main], [LIBXML2_LIBS="$LIBXML2_LIBS -lxml2"]) ++ dnl Find the main header and include path... ++ AC_CACHE_CHECK([location of libxml2 include files], [ac_cv_libxml2_include], [ ++ AC_CHECK_HEADERS([libxml/parser.h], [], [ ++ AC_MSG_NOTICE([Testing in /usr/include/libxml2]) ++ SAVED_CPPFLAGS="$CPPFLAGS" ++ CPPFLAGS="-I/usr/include/libxml2 $CPPFLAGS" + unset ac_cv_header_libxml_parser_h +- AC_CHECK_HEADERS([libxml/parser.h], [ac_cv_libxml2_include="-I/usr/local/include/libxml2"], [ +- AC_MSG_NOTICE([Failed to find libxml2 header file libxml/parser.h]) ++ AC_CHECK_HEADERS([libxml/parser.h], [LIBXML2_CFLAGS="$LIBXML2_CFLAGS -I/usr/include/libxml2"], [ ++ AC_MSG_NOTICE([Testing in /usr/local/include/libxml2]) ++ CPPFLAGS="-I/usr/local/include/libxml2 $SAVED_CPPFLAGS" ++ unset ac_cv_header_libxml_parser_h ++ AC_CHECK_HEADERS([libxml/parser.h], [LIBXML2_CFLAGS="$LIBXML2_CFLAGS -I/usr/local/include/libxml2"], [ ++ AC_MSG_NOTICE([Failed to find libxml2 header file libxml/parser.h]) ++ ]) + ]) ++ CPPFLAGS="$SAVED_CPPFLAGS" + ]) +- CPPFLAGS="$SAVED_CPPFLAGS" + ]) + ]) +- if test "x$ac_cv_libxml2_include" != "x"; then +- SQUID_CXXFLAGS="$ac_cv_libxml2_include $SQUID_CXXFLAGS" +- CPPFLAGS="$ac_cv_libxml2_include $CPPFLAGS" +- fi ++ CPPFLAGS="$CPPFLAGS $LIBXML2_CFLAGS" + dnl Now that we know where to look find the headers... + AC_CHECK_HEADERS(libxml/parser.h libxml/HTMLparser.h libxml/HTMLtree.h) +- AC_DEFINE_UNQUOTED(HAVE_LIBXML2, $HAVE_LIBXML2, [Define to 1 if you have the libxml2 library]) +- AS_IF(test "x$HAVE_LIBXML2" = "x1",[ ++ SQUID_STATE_ROLLBACK([squid_libxml2_save]) ++ ++ if test "x$LIBXML2_LIBS" != "x"; then ++ HAVE_LIBXML2=1 + squid_opt_use_esi=yes +- ],[ +- AS_IF(test "x$with_libxml2" = "xyes",[ +- AC_MSG_ERROR([Required library libxml2 not found.]) +- ],[ +- AC_MSG_NOTICE([Library libxml2 not found.]) +- ]) +- ]) ++ SQUID_CXXFLAGS="$SQUID_CXXFLAGS $LIBXML2_CFLAGS" ++ CPPFLAGS="$CPPFLAGS $LIBXML2_CFLAGS" ++ XMLLIB="$LIBXML2_LIBS" ++ AC_DEFINE_UNQUOTED(HAVE_LIBXML2, $HAVE_LIBXML2, [Define to 1 if you have the libxml2 library]) ++ elif test "x$with_libxml2" = "xyes"; then ++ AC_MSG_ERROR([Required library libxml2 not found]) ++ else ++ AC_MSG_NOTICE([Library libxml2 not found.]) ++ fi + fi + + AS_IF([test "x$squid_opt_use_esi" = "xyes"],[ diff --git a/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch b/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch new file mode 100644 index 000000000..3ffd9724d --- /dev/null +++ b/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch @@ -0,0 +1,24 @@ +commit b5ee1b8a8dd80cabe5de701d6dc59b90841bbb1c +Author: uhliarik <luhliari(a)redhat.com> +Date: 2018-12-11 23:09:46 +0000 + + Docs: Improved lexgrog compatibility (#340) + + The generated url_lfs_reqwrite.8 manpage was not parsable by lexgrog: + + $ lexgrog url_lfs_rewrite.8.gz + url_lfs_rewrite.8.gz: parse failed + +diff --git a/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in b/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in +index a7168e0..2e92769 100755 +--- a/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in ++++ b/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in +@@ -8,7 +8,7 @@ use Pod::Usage; + + =head1 NAME + +-B<url_lfs_rewrite> ++ url_lfs_rewrite - a URL-rewriter based on local file existence + + =head1 SYNOPSIS + diff --git a/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch b/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch new file mode 100644 index 000000000..146561870 --- /dev/null +++ b/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch @@ -0,0 +1,285 @@ +commit 06bae9e152a7f0d8e4f3c301dfdbd13231fec725 +Author: Christos Tsantilas <christos(a)chtsanti.net> +Date: 2018-12-25 13:37:26 +0000 + + Bug 4253: ssl_bump prevents from accessing some web contents (#304) + + Surrogate-Capability should only be sent on accel traffic, not + on bumped traffic. + + This is a Measurement Factory project + +diff --git a/src/client_side.cc b/src/client_side.cc +index d61e278..b3e3977 100644 +--- a/src/client_side.cc ++++ b/src/client_side.cc +@@ -1081,20 +1081,18 @@ findTrailingHTTPVersion(const char *uriAndHTTPVersion, const char *end) + return NULL; + } + +-static void +-prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp) ++static char * ++prepareAcceleratedURL(ConnStateData * conn, const Http1::RequestParserPointer &hp) + { + int vhost = conn->port->vhost; + int vport = conn->port->vport; + static char ipbuf[MAX_IPSTRLEN]; + +- http->flags.accel = true; +- + /* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */ + + static const SBuf cache_object("cache_object://"); + if (hp->requestUri().startsWith(cache_object)) +- return; /* already in good shape */ ++ return nullptr; /* already in good shape */ + + // XXX: re-use proper URL parser for this + SBuf url = hp->requestUri(); // use full provided URI if we abort +@@ -1104,7 +1102,7 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + break; + + if (conn->port->vhost) +- return; /* already in good shape */ ++ return nullptr; /* already in good shape */ + + // skip the URI scheme + static const CharacterSet uriScheme = CharacterSet("URI-scheme","+-.") + CharacterSet::ALPHA + CharacterSet::DIGIT; +@@ -1141,18 +1139,16 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + #endif + + if (vport < 0) +- vport = http->getConn()->clientConnection->local.port(); ++ vport = conn->clientConnection->local.port(); + +- const bool switchedToHttps = conn->switchedToHttps(); +- const bool tryHostHeader = vhost || switchedToHttps; + char *host = NULL; +- if (tryHostHeader && (host = hp->getHeaderField("Host"))) { ++ if (vhost && (host = hp->getHeaderField("Host"))) { + debugs(33, 5, "ACCEL VHOST REWRITE: vhost=" << host << " + vport=" << vport); + char thost[256]; + if (vport > 0) { + thost[0] = '\0'; + char *t = NULL; +- if (host[strlen(host)] != ']' && (t = strrchr(host,':')) != NULL) { ++ if (host[strlen(host) - 1] != ']' && (t = strrchr(host,':')) != nullptr) { + strncpy(thost, host, (t-host)); + snprintf(thost+(t-host), sizeof(thost)-(t-host), ":%d", vport); + host = thost; +@@ -1161,67 +1157,116 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + host = thost; + } + } // else nothing to alter port-wise. +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + strlen(host); +- http->uri = (char *)xcalloc(url_sz, 1); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL VHOST REWRITE: " << http->uri); ++ const int url_sz = scheme.length() + strlen(host) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(url)); ++ debugs(33, 5, "ACCEL VHOST REWRITE: " << uri); ++ return uri; + } else if (conn->port->defaultsite /* && !vhost */) { + debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: defaultsite=" << conn->port->defaultsite << " + vport=" << vport); +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + +- strlen(conn->port->defaultsite); +- http->uri = (char *)xcalloc(url_sz, 1); + char vportStr[32]; + vportStr[0] = '\0'; + if (vport > 0) { + snprintf(vportStr, sizeof(vportStr),":%d",vport); + } + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s%s" SQUIDSBUFPH, ++ const int url_sz = scheme.length() + strlen(conn->port->defaultsite) + sizeof(vportStr) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s%s" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), conn->port->defaultsite, vportStr, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: " << http->uri); ++ debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: " << uri); ++ return uri; + } else if (vport > 0 /* && (!vhost || no Host:) */) { + debugs(33, 5, "ACCEL VPORT REWRITE: *_port IP + vport=" << vport); + /* Put the local socket IP address as the hostname, with whatever vport we found */ +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen; +- http->uri = (char *)xcalloc(url_sz, 1); +- http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); ++ conn->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, ++ const int url_sz = scheme.length() + sizeof(ipbuf) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), ipbuf, vport, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL VPORT REWRITE: " << http->uri); ++ debugs(33, 5, "ACCEL VPORT REWRITE: " << uri); ++ return uri; + } ++ ++ return nullptr; + } + +-static void +-prepareTransparentURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp) ++static char * ++buildUrlFromHost(ConnStateData * conn, const Http1::RequestParserPointer &hp) + { +- // TODO Must() on URI !empty when the parser supports throw. For now avoid assert(). +- if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') +- return; /* already in good shape */ +- ++ char *uri = nullptr; + /* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */ +- + if (const char *host = hp->getHeaderField("Host")) { +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + +- strlen(host); +- http->uri = (char *)xcalloc(url_sz, 1); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, +- SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(hp->requestUri())); +- debugs(33, 5, "TRANSPARENT HOST REWRITE: " << http->uri); +- } else { ++ const int url_sz = scheme.length() + strlen(host) + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, ++ SQUIDSBUFPRINT(scheme), ++ host, ++ SQUIDSBUFPRINT(hp->requestUri())); ++ } ++ return uri; ++} ++ ++char * ++ConnStateData::prepareTlsSwitchingURL(const Http1::RequestParserPointer &hp) ++{ ++ Must(switchedToHttps()); ++ ++ if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') ++ return nullptr; /* already in good shape */ ++ ++ char *uri = buildUrlFromHost(this, hp); ++#if USE_OPENSSL ++ if (!uri) { ++ Must(tlsConnectPort); ++ Must(sslConnectHostOrIp.size()); ++ SBuf useHost; ++ if (!tlsClientSni().isEmpty()) ++ useHost = tlsClientSni(); ++ else ++ useHost.assign(sslConnectHostOrIp.rawBuf(), sslConnectHostOrIp.size()); ++ ++ const SBuf &scheme = AnyP::UriScheme(transferProtocol.protocol).image(); ++ const int url_sz = scheme.length() + useHost.length() + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://" SQUIDSBUFPH ":%d" SQUIDSBUFPH, ++ SQUIDSBUFPRINT(scheme), ++ SQUIDSBUFPRINT(useHost), ++ tlsConnectPort, ++ SQUIDSBUFPRINT(hp->requestUri())); ++ } ++#endif ++ if (uri) ++ debugs(33, 5, "TLS switching host rewrite: " << uri); ++ return uri; ++} ++ ++static char * ++prepareTransparentURL(ConnStateData * conn, const Http1::RequestParserPointer &hp) ++{ ++ // TODO Must() on URI !empty when the parser supports throw. For now avoid assert(). ++ if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') ++ return nullptr; /* already in good shape */ ++ ++ char *uri = buildUrlFromHost(conn, hp); ++ if (!uri) { + /* Put the local socket IP address as the hostname. */ +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen; +- http->uri = (char *)xcalloc(url_sz, 1); + static char ipbuf[MAX_IPSTRLEN]; +- http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); +- const SBuf &scheme = AnyP::UriScheme(http->getConn()->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, ++ conn->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); ++ const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); ++ const int url_sz = sizeof(ipbuf) + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), +- ipbuf, http->getConn()->clientConnection->local.port(), SQUIDSBUFPRINT(hp->requestUri())); +- debugs(33, 5, "TRANSPARENT REWRITE: " << http->uri); ++ ipbuf, conn->clientConnection->local.port(), SQUIDSBUFPRINT(hp->requestUri())); + } ++ ++ if (uri) ++ debugs(33, 5, "TRANSPARENT REWRITE: " << uri); ++ return uri; + } + + /** Parse an HTTP request +@@ -1341,9 +1386,11 @@ parseHttpRequest(ConnStateData *csd, const Http1::RequestParserPointer &hp) + * - remote interception with PROXY protocol + * - remote reverse-proxy with PROXY protocol + */ +- if (csd->transparent()) { ++ if (csd->switchedToHttps()) { ++ http->uri = csd->prepareTlsSwitchingURL(hp); ++ } else if (csd->transparent()) { + /* intercept or transparent mode, properly working with no failures */ +- prepareTransparentURL(csd, http, hp); ++ http->uri = prepareTransparentURL(csd, hp); + + } else if (internalCheck(hp->requestUri())) { // NP: only matches relative-URI + /* internal URL mode */ +@@ -1353,9 +1400,10 @@ parseHttpRequest(ConnStateData *csd, const Http1::RequestParserPointer &hp) + // But have not parsed there yet!! flag for local-only handling. + http->flags.internal = true; + +- } else if (csd->port->flags.accelSurrogate || csd->switchedToHttps()) { ++ } else if (csd->port->flags.accelSurrogate) { + /* accelerator mode */ +- prepareAcceleratedURL(csd, http, hp); ++ http->uri = prepareAcceleratedURL(csd, hp); ++ http->flags.accel = true; + } + + if (!http->uri) { +@@ -2315,6 +2363,7 @@ ConnStateData::ConnStateData(const MasterXaction::Pointer &xact) : + #if USE_OPENSSL + switchedToHttps_(false), + parsingTlsHandshake(false), ++ tlsConnectPort(0), + sslServerBump(NULL), + signAlgorithm(Ssl::algSignTrusted), + #endif +@@ -3050,6 +3099,7 @@ ConnStateData::switchToHttps(HttpRequest *request, Ssl::BumpMode bumpServerMode) + assert(!switchedToHttps_); + + sslConnectHostOrIp = request->url.host(); ++ tlsConnectPort = request->url.port(); + resetSslCommonName(request->url.host()); + + // We are going to read new request +diff --git a/src/client_side.h b/src/client_side.h +index bd4ceb0..70db8ab 100644 +--- a/src/client_side.h ++++ b/src/client_side.h +@@ -274,6 +274,7 @@ public: + #else + bool switchedToHttps() const { return false; } + #endif ++ char *prepareTlsSwitchingURL(const Http1::RequestParserPointer &hp); + + /* clt_conn_tag=tag annotation access */ + const SBuf &connectionTag() const { return connectionTag_; } +@@ -393,6 +394,7 @@ private: + + /// The SSL server host name appears in CONNECT request or the server ip address for the intercepted requests + String sslConnectHostOrIp; ///< The SSL server host name as passed in the CONNECT request ++ unsigned short tlsConnectPort; ///< The TLS server port number as passed in the CONNECT request + SBuf sslCommonName_; ///< CN name for SSL certificate generation + + /// TLS client delivered SNI value. Empty string if none has been received. diff --git a/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch b/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch new file mode 100644 index 000000000..611ff43ea --- /dev/null +++ b/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch @@ -0,0 +1,24 @@ +commit 95251ebca6f285360485369297bea39b85c777ad (refs/remotes/origin/v4) +Author: Christos Tsantilas <christos(a)chtsanti.net> +Date: 2018-12-25 17:01:39 +0000 + + Squid crashes when ICAPS and a sslcrtvalidator used together (#328) + + Squid hits an assertions when tries to remove an comm_close handler from + the already closed connection object. + + This is a Measurement Factory project + +diff --git a/src/adaptation/icap/Xaction.cc b/src/adaptation/icap/Xaction.cc +index 7005b47..6a0f8ad 100644 +--- a/src/adaptation/icap/Xaction.cc ++++ b/src/adaptation/icap/Xaction.cc +@@ -744,7 +744,7 @@ Adaptation::Icap::Xaction::handleSecuredPeer(Security::EncryptorAnswer &answer) + securer = NULL; + + if (closer != NULL) { +- if (answer.conn != NULL) ++ if (Comm::IsConnOpen(answer.conn)) + comm_remove_close_handler(answer.conn->fd, closer); + else + closer->cancel("securing completed"); diff --git a/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch b/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch index 8d1a4e03a..57fd0a6a6 100644 --- a/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch +++ b/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch @@ -1,6 +1,6 @@ --- configure.ac.~ Wed Apr 20 14:26:07 2016 +++ configure.ac Fri Apr 22 17:20:46 2016 -@@ -3156,6 +3156,9 @@ +@@ -3160,6 +3160,9 @@ ;; esac @@ -10,7 +10,7 @@ dnl --with-maxfd present for compatibility with Squid-2. dnl undocumented in ./configure --help to encourage using the Squid-3 directive AC_ARG_WITH(maxfd,, -@@ -3186,8 +3189,6 @@ +@@ -3190,8 +3193,6 @@ esac ]) -- 2.18.0

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Development January 2019