Development December 2018

development@lists.ipfire.org

16 participants
39 discussions

[PATCH] BUG11695: Adding/editing rules with preset broken
by Alexander Marx 15 Jul '21

15 Jul '21

added another check to fill same ports in source and destination when a custom service is selected. Signed-off-by: Alexander Marx <alexander.marx(a)ipfire.org> Reported-by: erik(a)vanlinsteeict.nl --- html/cgi-bin/firewall.cgi | 1 + 1 file changed, 1 insertion(+) diff --git a/html/cgi-bin/firewall.cgi b/html/cgi-bin/firewall.cgi index 499f279..d17afab 100644 --- a/html/cgi-bin/firewall.cgi +++ b/html/cgi-bin/firewall.cgi @@ -757,6 +757,7 @@ sub checkrule } } if($fwdfwsettings{'nat'} eq 'dnat' && $fwdfwsettings{'dnatport'} eq ''){$fwdfwsettings{'dnatport'}=$custsrvport;} + if($fwdfwsettings{'nat'} eq 'dnat' && $fwdfwsettings{'grp3'} eq 'cust_srv'){$fwdfwsettings{'dnatport'}=$custsrvport;} } #check if DNAT port is multiple if($fwdfwsettings{'nat'} eq 'dnat' && $fwdfwsettings{'dnatport'} ne ''){ -- 2.7.4

3 2

Re: Status emails and IP Blocklists
by Michael Tremer 01 Apr '19

01 Apr '19

Hey, > On 1 Dec 2018, at 18:20, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wrote: > > Hi, > > On 30/11/2018 11:04, Michael Tremer wrote: >> Hey Tim, >> >> thanks for your email! >> >> Those addons look great. Quite neat and tidy code and probably they are >> scratching an itch for some people. >> >> On Thu, 2018-11-29 at 21:11 +0000, Tim FitzGeorge wrote: >>> I've written a couple of addons for my installations of IPFire. They're >>> available on github and some other people have tried them; they seem to >>> be fairly well received and it's been suggested that it may be worth >>> making them available through pakfire as official addons. >> Where did you publish them before? > I've not published them before - I didn't even announce them on the > forums, but someone must have looked around after looking at the IDS > rule updater. Well, I guess great software finds its users on its own... >>> The first addon provides the ability to send status emails. You can >>> define multiple schedules and the items to be included in each email. >>> By choosing parameters carefully it's possible to get it to send emails >>> on some error conditions. The emails can be encrypted with GPG. The >>> architecture makes it easy to add further items to be reported on. >> Could you send an example email what it looks like? I do not see any reason why >> this should not be part of the distribution and would like to ask you to submit >> this as a patch that can be merged into mainline. > > I've attached a jpeg of the HTML version of a test email. It's had > certain information redacted. I don't include quite so much information > in my normal reports. It's also capable of some additional information > (for example errors) which only show up when necessary. Wow this is a lot. As in an overwhelming amount of graphs and data. I am not sure if this is useful when its altogether, but I guess that can be decided by each user… About the UI: I guess that could be a lot shorter. I find it quite logical that when someone wants a weekly report, the graphs should show the whole week and not only the last day. So that can be a single switch that makes many of the other options further done redundant. > The text emails can contain everything except the graphs. I've got my > systems set up to send an HTML email at midnight with a summary of the > previous day's information including some graphs, plus a text email of > error conditions every hour - this only gets sent if there are errors. > > > This one just turned up: > > Error check report > > > System > ------ > > SSH > > Logins > > User From Count > root 192.168.999.999 2 > > > I'll start working on a patch. I think my one question at this point is > where should it go in the menus? I put it under 'IPFire' since that > seems to be where miscellaneous addons go, but is there a better place > for it? Good questions. I am not very happy with the IPFire sub-menu because there is no point in it. This is a left-over from about 15 years ago when we used IPCop as a base. I think this could even be part of the email settings CGI; or it should go into logging. >> >> Maybe we can extend this over time and have it send more information if there >> are any requests. > Yes, it's got a plug-in architecture and in most cases adding more > information is quite easy. The main code takes care of formatting, > whether for HTML or Text, so a table can be added with one function call > which is passed an array of arrays. >> >> Would you be up for maintaining this long-term? > Yes. Great! >> >> Did you develop this for yourself or for work or has this been sponsored by >> someone else? > > I did it for myself. As well as my home system, I've got another one > set up at a small charity, and I wanted a way to see its status without > having to go over there. I didn't want to set up a VPN just for logging > in and checking status. Looks like a lot of work as a workaround to not set up a VPN. >>> The second addon handles the setting up and updating of IP Address >>> Blocklists in the firewall. It includes options to select which lists >>> to use, and some control over how frequently to check for updates. >> I guess Peter might be quite excited about this :) >> >> I personally do not have much use for this, but again, why should this not >> become part of IPFire? >> >> I did not install any of these yet, so could you maybe excuse lazy me and send >> screenshots? :) > Attached. The WUI for this is fairly simple. There's also a logwatch > plug in so that a summary of the update status appears in the log summary. See my comments above. I also have some other probably minor questions regarding some things on here, but I guess that can wait… >>> Both include WUI pages for configuration and language files. They're >>> fully functional, but would require some checking and minor updates. >>> The source can be seen at https://github.com/timfprogs . >> I have seen a third one which updates Snort rules. I am sure that you have heard >> about us changing to suricata soon (test images are available). However, the >> rules are roughly the same and the same update tools can be used. So, again, >> would you be interested to have this in the distribution and maintain it? > Definitely. I believe that there's already an automatic updater > provided, but I think mine has more facilities. I'm planning to install > the suricata test image in the next few weeks and have a good look at it. Yes, we should work on one thing after the other. Great that you join testing. Potentially we should think about working on this first now, so that suricata can go out as soon as possible with as many features as possible. Would you be okay with that? >>> I'm aware that there other people have made addons for both these >>> purposes, which maybe suggests that it's functionality that is worth adding. >> Best, >> -Michael >> >> P.S. Did you get any help building these or do you speak four languages? > > Alas, I only really speak English (although I do have some limited > knowledge of French and Latin). I used Google translate, so I expect > some errors - hopefully amusing ones rather than insulting. Good question. I have no idea. We can check with a speaker of any of those languages or ship it English-only. Best, -Michael > > > Tim > > > > <statusmail_email.jpeg><statusmail_wui.jpeg><blocklist-wui.png>

2 2

IPFire meets Suricata - Call for tester
by Stefan Schantl 05 Mar '19

05 Mar '19

Hello list followers, some time ago development for the new implementation of the Intrusion Detection functionality in IPFire has been started. The main goal, in a nutshell, was to give IPFire a modern, feature-rich and user-friendly Intrusion Detection Engine. During this progress, the detection framework has been replaced - now suricata is used instead of snort. Suricata uses a very modern and multi-threaded detection engine with support to perform actions on malicious traffic. So it provides the functionality of detecting any kind of intrusion attempts and the ability of guardian to block them under the same hood. It was a lot of work, but finaly I'm happy to announce the first test version. It is almost feature complete and without any kind of bigger issues. Because Intrusion Detection is a key feature of a firewall system, a lot of testing is required until the new implementation can become part of IPFire - therefore we need your help! Download the test image ( https://people.ipfire.org/~stevee/suricata/Images/), do a lot of hard testing and provide your feedback or suggestions on the develoment mailing list (https://lists.ipfire.org/mailman/listinfo/development). If you find any bugs please file them in the IPFire Bugtracker ( https://bugzilla.ipfire.org/). Many thanks in advance, -Stefan

7 49

New bootstrap/jQuery WUI theme - based on adminlte
by Rachid Groeneveld 06 Feb '19

06 Feb '19

Hi all, I hope I'm doing this right, as it's my first time "posting" to a (development) mailing list. Let me introduce myself first; I'm Rachid (Saiyato on the forums) and I've been working on a new WUI theme, right now it's a bit bloated compared to the other skins, but it also features a lot; especially in terms of icons. I'm not a developer by profession, but I do understand some languages (C#, python, perl, javascript), the only problem is that I lack pattern knowledge, which means my code might not be a neat as I would like it to be. So please do correct my erroneous code when you see it 😉 I do strive to use as little code as possible and combine/re-use portions, as well as google for the best options (if I can find the right keywords). If you have any questions, don't hesitate to contact me, I live in the Netherlands, so Nederlands, English or Deutsch is fine. As for the WUI, I used the bootstrap/adminlte framework, which was fairly easy to implement, I threw in some fonts for icons, so it's quite easy to fancy-up your config pages. As this point I did implement an override script for some parts of the existing UI's, this way I could 'integrate' those pages better, but feel free to provide me with better ways/tip 'n tricks. As you might see I'm struggling with naming... Pascal, camel... all low... What should I do best in the IPFire ecosystem? The attached screenshot give an impression of the UI, I did remove the map though... it caused all sorts of resizing issues and I'm not sure what kind of reporting actually makes sense on the dash. So that's still an open issue. I uploaded the whole lot to github (https://github.com/Saiyato/ipfire-skin-dashboard) and created some issues for known problems, it also features an installation script, which automatically sets the skin (~/main/settings[@THEME]). Since it's bootstrap, it will resize on phones and still remain responsive and modern looking. Most of the values are displaying real info, I used various cgi's to copy code from. The only dummy values are the statistics box (since I'm not sure what makes sense to show there) and guest wifi (maybe someone will make an addon for that, right now it's not supported out-of-the-box, correct me if I'm wrong). Happy holidays to all and a happy new year in advance! Cheers, Rachid PS: something went wrong attaching the images, so one attachment might be corrupt, mea culpa.

2 2

Forward default "DROP" is not applied to ORANGE traffic?!
by Peter Müller 07 Jan '19

07 Jan '19

Hello list, I recently stumbled across a strange behaviour of IPFire 2.x, which seems to be quite critical in my eyes, but I am not sure whether it is intentional or not. Default settings of IPFire allow traffic from internal networks (GREEN, BLUE, ORANGE) to the internet (RED), as documented here: https://wiki.ipfire.org/configuration/firewall/default-policy For several reasons, no direct internet access is desired on most firewall installations I administer, so setting the "default firewall behaviour" to DROP for both FORWARD and OUTGOING usually is one of the first steps after installation. Speaking about GREEN and BLUE, this seems to work: No direct connection is possible except it has been explicitly allowed. It turns out this setting does not apply to traffic from ORANGE: Even default is set to DROP, and no firewall rules allowing anything are in place, a server located in DMZ is able to reach full internet - every port on every IP in every country. This is not my expectation of "default policy" = DROP after all! Could somebody of the core developers urgently have a look at this, please? Thanks, and best regards, Peter Müller -- Microsoft DNS service terminates abnormally when it recieves a response to a DNS query that was never made. Fix Information: Run your DNS service on a different platform. -- bugtraq

3 7

Bugzilla down
by Matthias Fischer 02 Jan '19

02 Jan '19

Hi, it seems I'm the one with the bad news these days... ;-) If someone has the time - and the nerves: https://bugzilla.ipfire.org doesn't talk to us anymore, it only says: "Bad Request Your browser sent a request that this server could not understand." Best regards for the coming New Year, Matthias

2 1

[PATCH] squid 4.4: latest patches (04-08)
by Matthias Fischer 01 Jan '19

01 Jan '19

For details see: http://www.squid-cache.org/Versions/v4/changesets/ Best, Matthias Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org> --- lfs/squid | 5 + ...documentation_and_implementation_277.patch | 164 ++++++++++ ...onfig_when_configure-ing_libxml2_338.patch | 89 ++++++ ...s_Improved_lexgrog_compatibility_340.patch | 24 ++ ...from_accessing_some_web_contents_304.patch | 285 ++++++++++++++++++ ..._a_sslcrtvalidator_used_together_328.patch | 24 ++ .../squid-4.4-fix-max-file-descriptors.patch | 4 +- 7 files changed, 593 insertions(+), 2 deletions(-) create mode 100644 src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch create mode 100644 src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch create mode 100644 src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch create mode 100644 src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch create mode 100644 src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch diff --git a/lfs/squid b/lfs/squid index aaa2d0b96..113b4c3aa 100644 --- a/lfs/squid +++ b/lfs/squid @@ -75,6 +75,11 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/01_Fix_netdb_exchange_with_a_TLS_cache_peer_307.patch cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/02_Maintenance_add_xz_tarball_format_formally_to_make_dist_325.patch cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/03_The_handshake_logformat_code_331.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch cd $(DIR_APP) && autoreconf -vfi diff --git a/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch b/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch new file mode 100644 index 000000000..52c75e097 --- /dev/null +++ b/src/patches/squid/04_Fixed_forward_max_tries_documentation_and_implementation_277.patch @@ -0,0 +1,164 @@ +commit b3699887c2ffa807c65f153639cd13feb93f4c0f +Author: Eduard Bagdasaryan <eduard.bagdasaryan(a)measurement-factory.com> +Date: 2018-11-28 23:55:29 +0000 + + Fixed forward_max_tries documentation and implementation (#277) + + Before 1c8f25b, FwdState::n_tries counted the total number of forwarding + attempts, including pinned and persistent connection retries. Since that + revision, it started counting just those retries. What should n_tries + count? The counter is used to honor the forward_max_tries directive, but + that directive was documented to limit the number of _different_ paths + to try. Neither 1c8f25b~1 nor 1c8f25b code matched that documentation! + + Continuing to count just pinned and persistent connection retries (as in + 1c8f25b) would violate any reasonable forward_max_tries intent and admin + expectations. There are two ways to fix this problem, synchronizing code + and documentation: + + * Count just the attempts to use a different forwarding path, matching + forward_max_tries documentation but not what Squid has ever done. This + approach makes it difficult for an admin to limit the total number of + forwarding attempts in environments where, say, the second attempt is + unlikely to succeed and will just incur wasteful delays (Squid bug + 4788 report is probably about one of such use cases). Also, + implementing this approach may be more difficult because it requires + adding a new counter for retries and, for some interpretations of + "different", even a container of previously visited paths. + + * Count all forwarding attempts (as before 1c8f25b) and adjust + forward_max_tries documentation to match this historical behavior. + This approach does not have known unique flaws. + + Also fixed FwdState::n_tries off-by-one comparison bug discussed during + Squid bug 4788 triage. + + Also fixed admin concern behind Squid bug 4788 "forward_max_tries 1 does + not prevent some retries": While the old forward_max_tries documentation + actually excluded pconn retries, technically invalidating the bug + report, the admin now has a knob to limit those retries. + +diff --git a/src/FwdState.cc b/src/FwdState.cc +index f201333..b218240 100644 +--- a/src/FwdState.cc ++++ b/src/FwdState.cc +@@ -587,7 +587,7 @@ FwdState::checkRetry() + if (!entry->isEmpty()) + return false; + +- if (n_tries > Config.forward_max_tries) ++ if (exhaustedTries()) + return false; + + if (!EnoughTimeToReForward(start_t)) +@@ -921,6 +921,7 @@ FwdState::connectStart() + Comm::ConnOpener *cs = new Comm::ConnOpener(serverDestinations[0], calls.connector, connTimeout); + if (host) + cs->setHost(host); ++ ++n_tries; + AsyncJob::Start(cs); + } + +@@ -1072,7 +1073,7 @@ FwdState::reforward() + return 0; + } + +- if (n_tries > Config.forward_max_tries) ++ if (exhaustedTries()) + return 0; + + if (request->bodyNibbled()) +@@ -1222,6 +1223,12 @@ FwdState::logReplyStatus(int tries, const Http::StatusCode status) + ++ FwdReplyCodes[tries][status]; + } + ++bool ++FwdState::exhaustedTries() const ++{ ++ return n_tries >= Config.forward_max_tries; ++} ++ + /**** PRIVATE NON-MEMBER FUNCTIONS ********************************************/ + + /* +diff --git a/src/FwdState.h b/src/FwdState.h +index 6619c64..85f0fb9 100644 +--- a/src/FwdState.h ++++ b/src/FwdState.h +@@ -127,6 +127,9 @@ private: + void syncWithServerConn(const char *host); + void syncHierNote(const Comm::ConnectionPointer &server, const char *host); + ++ /// whether we have used up all permitted forwarding attempts ++ bool exhaustedTries() const; ++ + public: + StoreEntry *entry; + HttpRequest *request; +@@ -139,7 +142,7 @@ private: + ErrorState *err; + Comm::ConnectionPointer clientConn; ///< a possibly open connection to the client. + time_t start_t; +- int n_tries; ++ int n_tries; ///< the number of forwarding attempts so far + + // AsyncCalls which we set and may need cancelling. + struct { +diff --git a/src/cf.data.pre b/src/cf.data.pre +index a8ca587..7a7d271 100644 +--- a/src/cf.data.pre ++++ b/src/cf.data.pre +@@ -3791,11 +3791,15 @@ DEFAULT: 25 + TYPE: int + LOC: Config.forward_max_tries + DOC_START +- Controls how many different forward paths Squid will try +- before giving up. See also forward_timeout. ++ Limits the number of attempts to forward the request. ++ ++ For the purpose of this limit, Squid counts all high-level request ++ forwarding attempts, including any same-destination retries after ++ certain persistent connection failures and any attempts to use a ++ different peer. However, low-level connection reopening attempts ++ (enabled using connect_retries) are not counted. + +- NOTE: connect_retries (default: none) can make each of these +- possible forwarding paths be tried multiple times. ++ See also: forward_timeout and connect_retries. + DOC_END + + COMMENT_START +@@ -9854,19 +9858,23 @@ LOC: Config.connect_retries + DEFAULT: 0 + DEFAULT_DOC: Do not retry failed connections. + DOC_START +- This sets the maximum number of connection attempts made for each +- TCP connection. The connect_retries attempts must all still +- complete within the connection timeout period. ++ Limits the number of reopening attempts when establishing a single ++ TCP connection. All these attempts must still complete before the ++ applicable connection opening timeout expires. ++ ++ By default and when connect_retries is set to zero, Squid does not ++ retry failed connection opening attempts. + +- The default is not to re-try if the first connection attempt fails. +- The (not recommended) maximum is 10 tries. ++ The (not recommended) maximum is 10 tries. An attempt to configure a ++ higher value results in the value of 10 being used (with a warning). + +- A warning message will be generated if it is set to a too-high +- value and the configured value will be over-ridden. ++ Squid may open connections to retry various high-level forwarding ++ failures. For an outside observer, that activity may look like a ++ low-level connection reopening attempt, but those high-level retries ++ are governed by forward_max_tries instead. + +- Note: These re-tries are in addition to forward_max_tries +- which limit how many different addresses may be tried to find +- a useful server. ++ See also: connect_timeout, forward_timeout, icap_connect_timeout, ++ ident_timeout, and forward_max_tries. + DOC_END + + NAME: retry_on_error diff --git a/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch b/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch new file mode 100644 index 000000000..7685a9637 --- /dev/null +++ b/src/patches/squid/05_Use_and_prefer_pkg-config_when_configure-ing_libxml2_338.patch @@ -0,0 +1,89 @@ +commit c34582b9e8c40529db7eb9c490b081a37972f6d4 +Author: Fabrice Fontaine <fontaine.fabrice(a)gmail.com> +Date: 2018-12-04 17:23:25 +0000 + + Use and prefer pkg-config when ./configure-ing libxml2 (#338) + + The old method of trying to locate libxml2 via AC_CHECK_LIB() is + preserved to accommodate environments without pkg-config. + + pkg-config knows about libxml2 dependencies like lz or -liconv that the + old method misses. + +diff --git a/configure.ac b/configure.ac +index f668567..ef20a53 100644 +--- a/configure.ac ++++ b/configure.ac +@@ -922,41 +922,45 @@ fi + + AC_ARG_WITH(libxml2, AS_HELP_STRING([--without-libxml2],[Do not use libxml2 for ESI. Default: auto-detect])) + if test "x$squid_opt_use_esi" != "xno" -a "x$with_libxml2" != "xno" ; then +- AC_CHECK_LIB([xml2], [main], [XMLLIB="-lxml2"; HAVE_LIBXML2=1]) +- dnl Find the main header and include path... +- AC_CACHE_CHECK([location of libxml2 include files], [ac_cv_libxml2_include], [ +- AC_CHECK_HEADERS([libxml/parser.h], [], [ +- AC_MSG_NOTICE([Testing in /usr/include/libxml2]) +- SAVED_CPPFLAGS="$CPPFLAGS" +- CPPFLAGS="-I/usr/include/libxml2 $CPPFLAGS" +- unset ac_cv_header_libxml_parser_h +- AC_CHECK_HEADERS([libxml/parser.h], [ac_cv_libxml2_include="-I/usr/include/libxml2"], [ +- AC_MSG_NOTICE([Testing in /usr/local/include/libxml2]) +- CPPFLAGS="-I/usr/local/include/libxml2 $SAVED_CPPFLAGS" ++ SQUID_STATE_SAVE([squid_libxml2_save]) ++ PKG_CHECK_MODULES([LIBXML2],[libxml-2.0],[],[ ++ AC_CHECK_LIB([xml2], [main], [LIBXML2_LIBS="$LIBXML2_LIBS -lxml2"]) ++ dnl Find the main header and include path... ++ AC_CACHE_CHECK([location of libxml2 include files], [ac_cv_libxml2_include], [ ++ AC_CHECK_HEADERS([libxml/parser.h], [], [ ++ AC_MSG_NOTICE([Testing in /usr/include/libxml2]) ++ SAVED_CPPFLAGS="$CPPFLAGS" ++ CPPFLAGS="-I/usr/include/libxml2 $CPPFLAGS" + unset ac_cv_header_libxml_parser_h +- AC_CHECK_HEADERS([libxml/parser.h], [ac_cv_libxml2_include="-I/usr/local/include/libxml2"], [ +- AC_MSG_NOTICE([Failed to find libxml2 header file libxml/parser.h]) ++ AC_CHECK_HEADERS([libxml/parser.h], [LIBXML2_CFLAGS="$LIBXML2_CFLAGS -I/usr/include/libxml2"], [ ++ AC_MSG_NOTICE([Testing in /usr/local/include/libxml2]) ++ CPPFLAGS="-I/usr/local/include/libxml2 $SAVED_CPPFLAGS" ++ unset ac_cv_header_libxml_parser_h ++ AC_CHECK_HEADERS([libxml/parser.h], [LIBXML2_CFLAGS="$LIBXML2_CFLAGS -I/usr/local/include/libxml2"], [ ++ AC_MSG_NOTICE([Failed to find libxml2 header file libxml/parser.h]) ++ ]) + ]) ++ CPPFLAGS="$SAVED_CPPFLAGS" + ]) +- CPPFLAGS="$SAVED_CPPFLAGS" + ]) + ]) +- if test "x$ac_cv_libxml2_include" != "x"; then +- SQUID_CXXFLAGS="$ac_cv_libxml2_include $SQUID_CXXFLAGS" +- CPPFLAGS="$ac_cv_libxml2_include $CPPFLAGS" +- fi ++ CPPFLAGS="$CPPFLAGS $LIBXML2_CFLAGS" + dnl Now that we know where to look find the headers... + AC_CHECK_HEADERS(libxml/parser.h libxml/HTMLparser.h libxml/HTMLtree.h) +- AC_DEFINE_UNQUOTED(HAVE_LIBXML2, $HAVE_LIBXML2, [Define to 1 if you have the libxml2 library]) +- AS_IF(test "x$HAVE_LIBXML2" = "x1",[ ++ SQUID_STATE_ROLLBACK([squid_libxml2_save]) ++ ++ if test "x$LIBXML2_LIBS" != "x"; then ++ HAVE_LIBXML2=1 + squid_opt_use_esi=yes +- ],[ +- AS_IF(test "x$with_libxml2" = "xyes",[ +- AC_MSG_ERROR([Required library libxml2 not found.]) +- ],[ +- AC_MSG_NOTICE([Library libxml2 not found.]) +- ]) +- ]) ++ SQUID_CXXFLAGS="$SQUID_CXXFLAGS $LIBXML2_CFLAGS" ++ CPPFLAGS="$CPPFLAGS $LIBXML2_CFLAGS" ++ XMLLIB="$LIBXML2_LIBS" ++ AC_DEFINE_UNQUOTED(HAVE_LIBXML2, $HAVE_LIBXML2, [Define to 1 if you have the libxml2 library]) ++ elif test "x$with_libxml2" = "xyes"; then ++ AC_MSG_ERROR([Required library libxml2 not found]) ++ else ++ AC_MSG_NOTICE([Library libxml2 not found.]) ++ fi + fi + + AS_IF([test "x$squid_opt_use_esi" = "xyes"],[ diff --git a/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch b/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch new file mode 100644 index 000000000..3ffd9724d --- /dev/null +++ b/src/patches/squid/06_Docs_Improved_lexgrog_compatibility_340.patch @@ -0,0 +1,24 @@ +commit b5ee1b8a8dd80cabe5de701d6dc59b90841bbb1c +Author: uhliarik <luhliari(a)redhat.com> +Date: 2018-12-11 23:09:46 +0000 + + Docs: Improved lexgrog compatibility (#340) + + The generated url_lfs_reqwrite.8 manpage was not parsable by lexgrog: + + $ lexgrog url_lfs_rewrite.8.gz + url_lfs_rewrite.8.gz: parse failed + +diff --git a/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in b/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in +index a7168e0..2e92769 100755 +--- a/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in ++++ b/src/http/url_rewriters/LFS/url_lfs_rewrite.pl.in +@@ -8,7 +8,7 @@ use Pod::Usage; + + =head1 NAME + +-B<url_lfs_rewrite> ++ url_lfs_rewrite - a URL-rewriter based on local file existence + + =head1 SYNOPSIS + diff --git a/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch b/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch new file mode 100644 index 000000000..146561870 --- /dev/null +++ b/src/patches/squid/07_Bug_4253_ssl_bump_prevents_from_accessing_some_web_contents_304.patch @@ -0,0 +1,285 @@ +commit 06bae9e152a7f0d8e4f3c301dfdbd13231fec725 +Author: Christos Tsantilas <christos(a)chtsanti.net> +Date: 2018-12-25 13:37:26 +0000 + + Bug 4253: ssl_bump prevents from accessing some web contents (#304) + + Surrogate-Capability should only be sent on accel traffic, not + on bumped traffic. + + This is a Measurement Factory project + +diff --git a/src/client_side.cc b/src/client_side.cc +index d61e278..b3e3977 100644 +--- a/src/client_side.cc ++++ b/src/client_side.cc +@@ -1081,20 +1081,18 @@ findTrailingHTTPVersion(const char *uriAndHTTPVersion, const char *end) + return NULL; + } + +-static void +-prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp) ++static char * ++prepareAcceleratedURL(ConnStateData * conn, const Http1::RequestParserPointer &hp) + { + int vhost = conn->port->vhost; + int vport = conn->port->vport; + static char ipbuf[MAX_IPSTRLEN]; + +- http->flags.accel = true; +- + /* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */ + + static const SBuf cache_object("cache_object://"); + if (hp->requestUri().startsWith(cache_object)) +- return; /* already in good shape */ ++ return nullptr; /* already in good shape */ + + // XXX: re-use proper URL parser for this + SBuf url = hp->requestUri(); // use full provided URI if we abort +@@ -1104,7 +1102,7 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + break; + + if (conn->port->vhost) +- return; /* already in good shape */ ++ return nullptr; /* already in good shape */ + + // skip the URI scheme + static const CharacterSet uriScheme = CharacterSet("URI-scheme","+-.") + CharacterSet::ALPHA + CharacterSet::DIGIT; +@@ -1141,18 +1139,16 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + #endif + + if (vport < 0) +- vport = http->getConn()->clientConnection->local.port(); ++ vport = conn->clientConnection->local.port(); + +- const bool switchedToHttps = conn->switchedToHttps(); +- const bool tryHostHeader = vhost || switchedToHttps; + char *host = NULL; +- if (tryHostHeader && (host = hp->getHeaderField("Host"))) { ++ if (vhost && (host = hp->getHeaderField("Host"))) { + debugs(33, 5, "ACCEL VHOST REWRITE: vhost=" << host << " + vport=" << vport); + char thost[256]; + if (vport > 0) { + thost[0] = '\0'; + char *t = NULL; +- if (host[strlen(host)] != ']' && (t = strrchr(host,':')) != NULL) { ++ if (host[strlen(host) - 1] != ']' && (t = strrchr(host,':')) != nullptr) { + strncpy(thost, host, (t-host)); + snprintf(thost+(t-host), sizeof(thost)-(t-host), ":%d", vport); + host = thost; +@@ -1161,67 +1157,116 @@ prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1 + host = thost; + } + } // else nothing to alter port-wise. +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + strlen(host); +- http->uri = (char *)xcalloc(url_sz, 1); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL VHOST REWRITE: " << http->uri); ++ const int url_sz = scheme.length() + strlen(host) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(url)); ++ debugs(33, 5, "ACCEL VHOST REWRITE: " << uri); ++ return uri; + } else if (conn->port->defaultsite /* && !vhost */) { + debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: defaultsite=" << conn->port->defaultsite << " + vport=" << vport); +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + +- strlen(conn->port->defaultsite); +- http->uri = (char *)xcalloc(url_sz, 1); + char vportStr[32]; + vportStr[0] = '\0'; + if (vport > 0) { + snprintf(vportStr, sizeof(vportStr),":%d",vport); + } + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s%s" SQUIDSBUFPH, ++ const int url_sz = scheme.length() + strlen(conn->port->defaultsite) + sizeof(vportStr) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s%s" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), conn->port->defaultsite, vportStr, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: " << http->uri); ++ debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: " << uri); ++ return uri; + } else if (vport > 0 /* && (!vhost || no Host:) */) { + debugs(33, 5, "ACCEL VPORT REWRITE: *_port IP + vport=" << vport); + /* Put the local socket IP address as the hostname, with whatever vport we found */ +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen; +- http->uri = (char *)xcalloc(url_sz, 1); +- http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); ++ conn->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, ++ const int url_sz = scheme.length() + sizeof(ipbuf) + url.length() + 32; ++ char *uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), ipbuf, vport, SQUIDSBUFPRINT(url)); +- debugs(33, 5, "ACCEL VPORT REWRITE: " << http->uri); ++ debugs(33, 5, "ACCEL VPORT REWRITE: " << uri); ++ return uri; + } ++ ++ return nullptr; + } + +-static void +-prepareTransparentURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp) ++static char * ++buildUrlFromHost(ConnStateData * conn, const Http1::RequestParserPointer &hp) + { +- // TODO Must() on URI !empty when the parser supports throw. For now avoid assert(). +- if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') +- return; /* already in good shape */ +- ++ char *uri = nullptr; + /* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */ +- + if (const char *host = hp->getHeaderField("Host")) { +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + +- strlen(host); +- http->uri = (char *)xcalloc(url_sz, 1); + const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, +- SQUIDSBUFPRINT(scheme), host, SQUIDSBUFPRINT(hp->requestUri())); +- debugs(33, 5, "TRANSPARENT HOST REWRITE: " << http->uri); +- } else { ++ const int url_sz = scheme.length() + strlen(host) + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s" SQUIDSBUFPH, ++ SQUIDSBUFPRINT(scheme), ++ host, ++ SQUIDSBUFPRINT(hp->requestUri())); ++ } ++ return uri; ++} ++ ++char * ++ConnStateData::prepareTlsSwitchingURL(const Http1::RequestParserPointer &hp) ++{ ++ Must(switchedToHttps()); ++ ++ if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') ++ return nullptr; /* already in good shape */ ++ ++ char *uri = buildUrlFromHost(this, hp); ++#if USE_OPENSSL ++ if (!uri) { ++ Must(tlsConnectPort); ++ Must(sslConnectHostOrIp.size()); ++ SBuf useHost; ++ if (!tlsClientSni().isEmpty()) ++ useHost = tlsClientSni(); ++ else ++ useHost.assign(sslConnectHostOrIp.rawBuf(), sslConnectHostOrIp.size()); ++ ++ const SBuf &scheme = AnyP::UriScheme(transferProtocol.protocol).image(); ++ const int url_sz = scheme.length() + useHost.length() + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://" SQUIDSBUFPH ":%d" SQUIDSBUFPH, ++ SQUIDSBUFPRINT(scheme), ++ SQUIDSBUFPRINT(useHost), ++ tlsConnectPort, ++ SQUIDSBUFPRINT(hp->requestUri())); ++ } ++#endif ++ if (uri) ++ debugs(33, 5, "TLS switching host rewrite: " << uri); ++ return uri; ++} ++ ++static char * ++prepareTransparentURL(ConnStateData * conn, const Http1::RequestParserPointer &hp) ++{ ++ // TODO Must() on URI !empty when the parser supports throw. For now avoid assert(). ++ if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/') ++ return nullptr; /* already in good shape */ ++ ++ char *uri = buildUrlFromHost(conn, hp); ++ if (!uri) { + /* Put the local socket IP address as the hostname. */ +- const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen; +- http->uri = (char *)xcalloc(url_sz, 1); + static char ipbuf[MAX_IPSTRLEN]; +- http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); +- const SBuf &scheme = AnyP::UriScheme(http->getConn()->transferProtocol.protocol).image(); +- snprintf(http->uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, ++ conn->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN); ++ const SBuf &scheme = AnyP::UriScheme(conn->transferProtocol.protocol).image(); ++ const int url_sz = sizeof(ipbuf) + hp->requestUri().length() + 32; ++ uri = static_cast<char *>(xcalloc(url_sz, 1)); ++ snprintf(uri, url_sz, SQUIDSBUFPH "://%s:%d" SQUIDSBUFPH, + SQUIDSBUFPRINT(scheme), +- ipbuf, http->getConn()->clientConnection->local.port(), SQUIDSBUFPRINT(hp->requestUri())); +- debugs(33, 5, "TRANSPARENT REWRITE: " << http->uri); ++ ipbuf, conn->clientConnection->local.port(), SQUIDSBUFPRINT(hp->requestUri())); + } ++ ++ if (uri) ++ debugs(33, 5, "TRANSPARENT REWRITE: " << uri); ++ return uri; + } + + /** Parse an HTTP request +@@ -1341,9 +1386,11 @@ parseHttpRequest(ConnStateData *csd, const Http1::RequestParserPointer &hp) + * - remote interception with PROXY protocol + * - remote reverse-proxy with PROXY protocol + */ +- if (csd->transparent()) { ++ if (csd->switchedToHttps()) { ++ http->uri = csd->prepareTlsSwitchingURL(hp); ++ } else if (csd->transparent()) { + /* intercept or transparent mode, properly working with no failures */ +- prepareTransparentURL(csd, http, hp); ++ http->uri = prepareTransparentURL(csd, hp); + + } else if (internalCheck(hp->requestUri())) { // NP: only matches relative-URI + /* internal URL mode */ +@@ -1353,9 +1400,10 @@ parseHttpRequest(ConnStateData *csd, const Http1::RequestParserPointer &hp) + // But have not parsed there yet!! flag for local-only handling. + http->flags.internal = true; + +- } else if (csd->port->flags.accelSurrogate || csd->switchedToHttps()) { ++ } else if (csd->port->flags.accelSurrogate) { + /* accelerator mode */ +- prepareAcceleratedURL(csd, http, hp); ++ http->uri = prepareAcceleratedURL(csd, hp); ++ http->flags.accel = true; + } + + if (!http->uri) { +@@ -2315,6 +2363,7 @@ ConnStateData::ConnStateData(const MasterXaction::Pointer &xact) : + #if USE_OPENSSL + switchedToHttps_(false), + parsingTlsHandshake(false), ++ tlsConnectPort(0), + sslServerBump(NULL), + signAlgorithm(Ssl::algSignTrusted), + #endif +@@ -3050,6 +3099,7 @@ ConnStateData::switchToHttps(HttpRequest *request, Ssl::BumpMode bumpServerMode) + assert(!switchedToHttps_); + + sslConnectHostOrIp = request->url.host(); ++ tlsConnectPort = request->url.port(); + resetSslCommonName(request->url.host()); + + // We are going to read new request +diff --git a/src/client_side.h b/src/client_side.h +index bd4ceb0..70db8ab 100644 +--- a/src/client_side.h ++++ b/src/client_side.h +@@ -274,6 +274,7 @@ public: + #else + bool switchedToHttps() const { return false; } + #endif ++ char *prepareTlsSwitchingURL(const Http1::RequestParserPointer &hp); + + /* clt_conn_tag=tag annotation access */ + const SBuf &connectionTag() const { return connectionTag_; } +@@ -393,6 +394,7 @@ private: + + /// The SSL server host name appears in CONNECT request or the server ip address for the intercepted requests + String sslConnectHostOrIp; ///< The SSL server host name as passed in the CONNECT request ++ unsigned short tlsConnectPort; ///< The TLS server port number as passed in the CONNECT request + SBuf sslCommonName_; ///< CN name for SSL certificate generation + + /// TLS client delivered SNI value. Empty string if none has been received. diff --git a/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch b/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch new file mode 100644 index 000000000..611ff43ea --- /dev/null +++ b/src/patches/squid/08_Squid_crashes_when_ICAPS_and_a_sslcrtvalidator_used_together_328.patch @@ -0,0 +1,24 @@ +commit 95251ebca6f285360485369297bea39b85c777ad (refs/remotes/origin/v4) +Author: Christos Tsantilas <christos(a)chtsanti.net> +Date: 2018-12-25 17:01:39 +0000 + + Squid crashes when ICAPS and a sslcrtvalidator used together (#328) + + Squid hits an assertions when tries to remove an comm_close handler from + the already closed connection object. + + This is a Measurement Factory project + +diff --git a/src/adaptation/icap/Xaction.cc b/src/adaptation/icap/Xaction.cc +index 7005b47..6a0f8ad 100644 +--- a/src/adaptation/icap/Xaction.cc ++++ b/src/adaptation/icap/Xaction.cc +@@ -744,7 +744,7 @@ Adaptation::Icap::Xaction::handleSecuredPeer(Security::EncryptorAnswer &answer) + securer = NULL; + + if (closer != NULL) { +- if (answer.conn != NULL) ++ if (Comm::IsConnOpen(answer.conn)) + comm_remove_close_handler(answer.conn->fd, closer); + else + closer->cancel("securing completed"); diff --git a/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch b/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch index 8d1a4e03a..57fd0a6a6 100644 --- a/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch +++ b/src/patches/squid/squid-4.4-fix-max-file-descriptors.patch @@ -1,6 +1,6 @@ --- configure.ac.~ Wed Apr 20 14:26:07 2016 +++ configure.ac Fri Apr 22 17:20:46 2016 -@@ -3156,6 +3156,9 @@ +@@ -3160,6 +3160,9 @@ ;; esac @@ -10,7 +10,7 @@ dnl --with-maxfd present for compatibility with Squid-2. dnl undocumented in ./configure --help to encourage using the Squid-3 directive AC_ARG_WITH(maxfd,, -@@ -3186,8 +3189,6 @@ +@@ -3190,8 +3193,6 @@ esac ]) -- 2.18.0

1 1

blog.ipfire.org => Error 500
by Matthias Fischer 29 Dec '18

29 Dec '18

Hi, Some headache again: https:/blog.ipfire.org responds with "Error 500 oops, something went wrong" See: https://forum.ipfire.org/viewtopic.php?f=22&t=22094 Best, Matthias

2 1

802.11ac - specific config requirements for hostapd
by Horace Michael 28 Dec '18

28 Dec '18

Hello, Please consider to add to WUI (cgi-bin/wlanap.cgi) a field where specific 802.11ac settings can be added. Ex: vht_oper_chwidth=1 vht_oper_centr_freq_seg0_idx=42 Others specific settings might be needed - as listed in the hostapd configuration file https://w1.fi/cgit/hostap/plain/hostapd/hostapd.conf Currently these settings added to /etc/hostapd.conf don't survire when WUI is used - settings need to be added back afer WUI is used and hostapd restarted. Thank you for considering this improvement, Horace

3 4

[PATCH] wget: Update to 1.20.1
by Matthias Fischer 27 Dec '18

27 Dec '18

Hi, This is a bugfix release: "due to some privacy issues in default settings of Wget, we introduce this bugfix release. The --xattr option (saving original URL and Referer into extended file attributes) was introduced and enabled by default since Wget 1.19. It possibly saved - possibly unrecognized by the user - credentials, access tokes etc that were included in the requested URL. We changed three details as a countermeasure, see below in the NEWS section. With Best Regards, Tim ... NEWS * Changes in Wget 1.20.1 ** --xattr is no longer default since it introduces privacy issues. ** --xattr saves the Referer as scheme/host/port, user/pw/path/query/fragment are no longer saved to prevent privacy issues. ** --xattr saves the Original URL without user/password to prevent privacy issues." Best, Matthias Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org> --- lfs/wget | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lfs/wget b/lfs/wget index 5ccb0029f..b8c83d10d 100644 --- a/lfs/wget +++ b/lfs/wget @@ -24,7 +24,7 @@ include Config -VER = 1.20 +VER = 1.20.1 THISAPP = wget-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -40,7 +40,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_MD5 = 9f1515d083b769e9ff7642ce6016518e +$(DL_FILE)_MD5 = f6ebe9c7b375fc9832fb1b2028271fb7 install : $(TARGET) -- 2.18.0

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Development December 2018