* [git.ipfire.org] IPFire 2.x development tree branch, next, updated. 9fda5e9258de75030316a046bf40f0e2779d69e6
@ 2018-01-07 19:52 git
0 siblings, 0 replies; only message in thread
From: git @ 2018-01-07 19:52 UTC (permalink / raw)
To: ipfire-scm
[-- Attachment #1: Type: text/plain, Size: 19040 bytes --]
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "IPFire 2.x development tree".
The branch, next has been updated
via 9fda5e9258de75030316a046bf40f0e2779d69e6 (commit)
via e04f86b4490c2f22c93fd1fdf9ae5d06971b8581 (commit)
via e331e22bda26e53cd71d04e43d4d8a9d9306b51e (commit)
via 329631dee9f0adfb512f48aac61bbfdfe8f09a7a (commit)
via 645da4f33dd81df39991b510cca22dbead5d5255 (commit)
via 39c0b5b3bcd9d4cf57437d1988df834a1800c66c (commit)
via e7bf3de34b8e382b229338377d9f20342bdc6bf6 (commit)
via 9d79aea24b8554172464600252814b0765434e47 (commit)
via d093117fe4035373f6a921d8cab6ef39f652d7fe (commit)
via 74713741e617acc6d5eb7a74da1188a437a6413c (commit)
via 375c4480d75fc0802cdaa3ac68712348ab8448c1 (commit)
via ff9fdd63889fc39bb64ff5d38e66a5e4fbd119e6 (commit)
via 15f7b86597c816426f5b4a9e728e257fb29969bf (commit)
via d894ef3615b1f3d2a4ed8f78c8fb24083dc9cc57 (commit)
from bc6257f49045c91a36a96e2ead87b52e6c7bfd8a (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 9fda5e9258de75030316a046bf40f0e2779d69e6
Author: Michael Tremer <michael.tremer(a)ipfire.org>
Date: Sun Jan 7 19:51:07 2018 +0000
core118: Ship update accelerator downloader
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit e04f86b4490c2f22c93fd1fdf9ae5d06971b8581
Author: Justin Luth <justin_luth(a)sil.org>
Date: Sat Dec 30 09:25:50 2017 +0300
Fix bug 11567 updxlrator: don't prematurely release lock file
With Microsoft's new style of downloading updates,
where portions of a patch are requested multiple times per second,
it has become extremely common for downloads to reach > 100%.
Due to an early unlinking of the "lock" file, there is a big window of
opportunity (between the unlink and wget actually saving some data)
for multiple download/wget threads to start, adding to the same file.
So not only is bandwidth wasted by duplicate downloads running
simultaneously, but the resulting file is corrupt anyway.
The problem is noticed more often by low bandwidth users
(who need the benefits of updxlrator the most)
because then wget's latency is even longer, creating
a very wide window of opportunity.
Ultimately, this needs something like "flock", where the
file is set and tested in one operation. But for now,
settle with the current test / create lock solution, and
just stop unnecessarily releasing the lock.
Since the file already exists as a lock when wget starts,
wget now must ALWAYS run with --continue, which
works fine on a zero-sized file.
Signed-off-by: Justin Luth <jluth(a)mail.com>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit e331e22bda26e53cd71d04e43d4d8a9d9306b51e
Author: Justin Luth <justin_luth(a)sil.org>
Date: Sat Dec 30 08:48:37 2017 +0300
updxlrator: show hostaddr in debuglog
There is nowhere in the debuglog any indication of
which client is requesting the file that updxlrator
is providing (or caching). Especially for those
huge Windows 10 downloads, it is valuable to
see which client is requesting them, especially
when the same client requests the same download
multiple times a second.
This only impacts users who turn on debugging.
Signed-off-by: Justin Luth <jluth(a)mail.com>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 329631dee9f0adfb512f48aac61bbfdfe8f09a7a
Author: Justin Luth <jluth(a)mail.com>
Date: Sat Dec 30 22:12:01 2017 +0300
Fix bug 11558 updxlrator: use mirror mode for SHA1, filenames
Most Microsoft updates now contain an SHA1 hash in the filename.
Since these files are uniquely identifiable, use mirror mode
(which creates a hash of just the filename instead of the entire URL)
to cache them. (But first check the URL cache to see if it
has been downloaded as a URL already.)
This is a HUGELY needed fix. Windows 10 updates are 5+ GB
per month, and we lose several days of bandwidth downloading
duplicates from different mirrors. Sometimes a single client
will request the same patch from multiple mirrors. That's bad.
This patch will save a ton of bandwidth, and lots of disk space.
The patch limits the SHA1 test to microsoft only, but it
could be easily expanded to other vendors if there is a need.
Signed-off-by: Justin Luth <jluth(a)mail.com>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 645da4f33dd81df39991b510cca22dbead5d5255
Author: Michael Tremer <michael.tremer(a)ipfire.org>
Date: Sun Jan 7 19:28:28 2018 +0000
core118: Ship updated update accelerator
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 39c0b5b3bcd9d4cf57437d1988df834a1800c66c
Author: Justin Luth <jluth(a)mail.com>
Date: Fri Dec 29 17:12:27 2017 +0300
Fix bug 10504: match download's sourceurl mangling in, updxlrator
Updatexlrator stores its files in a hash of the URL.
The download utility mangles the URL for [+/~], but
the updxlrator only does it for [/]. Thus, download
stores the result as one hash, and updxlrator looks for it
with a different hash. The result is that the file is
re-downloaded every time by both the client, and updxlrator.
This is fixed by making updxlrator mangle the url in the
same way as the downloader. apt-get install g++ would
be a good test for this.
Signed-off-by: Justin Luth <jluth(a)mail.com>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit e7bf3de34b8e382b229338377d9f20342bdc6bf6
Author: Michael Tremer <michael.tremer(a)ipfire.org>
Date: Sun Jan 7 19:22:27 2018 +0000
core118: Ship updated ids.cgi
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 9d79aea24b8554172464600252814b0765434e47
Author: Matthias Fischer <matthias.fischer(a)ipfire.org>
Date: Sat Jan 6 09:18:39 2018 +0100
snort 2.9.11.1: 'ids.cgi' - Update for snort rules download url
Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit d093117fe4035373f6a921d8cab6ef39f652d7fe
Author: Michael Tremer <michael.tremer(a)ipfire.org>
Date: Sun Jan 7 19:21:35 2018 +0000
core118: Ship updated snort
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 74713741e617acc6d5eb7a74da1188a437a6413c
Author: Matthias Fischer <matthias.fischer(a)ipfire.org>
Date: Fri Jan 5 18:28:00 2018 +0100
snort: Update to 2.9.11.1
For details see:
Release notes:
https://snort.org/downloads/snort/release_notes_2.9.11.1.txt
Changelog:
https://snort.org/downloads/snort/changelog_2.9.11.1.txt
Best,
Matthias
Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 375c4480d75fc0802cdaa3ac68712348ab8448c1
Author: Michael Tremer <michael.tremer(a)ipfire.org>
Date: Sun Jan 7 19:19:12 2018 +0000
core118: Ship updated fireinfo.cgi
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit ff9fdd63889fc39bb64ff5d38e66a5e4fbd119e6
Author: Peter Müller <peter.mueller(a)link38.eu>
Date: Sun Jan 7 11:01:36 2018 +0100
use HTTPS for links to fireinfo.ipfire.org
Since fireinfo.ipfire.org is now supporting HTTPS, the
links in the WebUI should point to the secure version of the site.
Signed-off-by: Peter Müller <peter.mueller(a)link38.eu>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit 15f7b86597c816426f5b4a9e728e257fb29969bf
Author: Peter Müller <peter.mueller(a)link38.eu>
Date: Sun Jan 7 13:55:05 2018 +0100
ship updated showrequestfromcountry.cgi file
Signed-off-by: Peter Müller <peter.mueller(a)link38.eu>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
commit d894ef3615b1f3d2a4ed8f78c8fb24083dc9cc57
Author: Peter Müller <peter.mueller(a)link38.eu>
Date: Sun Jan 7 13:52:11 2018 +0100
fix GeoIP lookup in showrequestfromcountry.dat
This issue was caused by the rewrite of the perl GeoIP
library.
Fixes #11571.
Signed-off-by: Peter Müller <peter.mueller(a)link38.eu>
Tested-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
-----------------------------------------------------------------------
Summary of changes:
config/rootfiles/core/118/filelists/files | 5 +++++
.../{oldcore/104 => core/118}/filelists/snort | 0
config/rootfiles/core/118/update.sh | 6 ++++--
config/updxlrator/download | 12 ++++++------
config/updxlrator/updxlrator | 19 +++++++++++++++++--
html/cgi-bin/fireinfo.cgi | 4 ++--
html/cgi-bin/ids.cgi | 4 ++--
html/cgi-bin/logs.cgi/showrequestfromcountry.dat | 2 +-
lfs/snort | 4 ++--
9 files changed, 39 insertions(+), 17 deletions(-)
copy config/rootfiles/{oldcore/104 => core/118}/filelists/snort (100%)
Difference in files:
diff --git a/config/rootfiles/core/118/filelists/files b/config/rootfiles/core/118/filelists/files
index cbc59893f..6780a011b 100644
--- a/config/rootfiles/core/118/filelists/files
+++ b/config/rootfiles/core/118/filelists/files
@@ -4,4 +4,9 @@ etc/httpd/conf/loadmodule.conf
etc/httpd/conf/vhosts.d/captive.conf
etc/httpd/conf/vhosts.d/ipfire-interface-ssl.conf
etc/httpd/conf/vhosts.d/ipfire-interface.conf
+srv/web/ipfire/cgi-bin/fireinfo.cgi
+srv/web/ipfire/cgi-bin/ids.cgi
+srv/web/ipfire/cgi-bin/logs.cgi/showrequestfromcountry.dat
+usr/sbin/updxlrator
var/ipfire/langs
+var/ipfire/updatexlrator/bin/download
diff --git a/config/rootfiles/core/118/filelists/snort b/config/rootfiles/core/118/filelists/snort
new file mode 120000
index 000000000..9406ce01c
--- /dev/null
+++ b/config/rootfiles/core/118/filelists/snort
@@ -0,0 +1 @@
+../../../common/snort
\ No newline at end of file
diff --git a/config/rootfiles/core/118/update.sh b/config/rootfiles/core/118/update.sh
index 18a167c3b..60173609f 100644
--- a/config/rootfiles/core/118/update.sh
+++ b/config/rootfiles/core/118/update.sh
@@ -31,6 +31,9 @@ for (( i=1; i<=$core; i++ )); do
rm -f /var/cache/pakfire/core-upgrade-*-$i.ipfire
done
+# Stop services
+/etc/init.d/snort stop
+
# Delete files
rm -rvf \
/etc/httpd/conf.d/php5.conf \
@@ -42,8 +45,6 @@ rm -rvf \
/usr/lib/apache/libphp5.so \
/usr/lib/php
-# Stop services
-
# Extract files
extract_files
@@ -55,6 +56,7 @@ ldconfig
# Start services
/etc/init.d/apache restart
+/etc/init.d/snort start
# This update need a reboot...
#touch /var/run/need_reboot
diff --git a/config/updxlrator/download b/config/updxlrator/download
index dbc722c23..afa6e6cb9 100644
--- a/config/updxlrator/download
+++ b/config/updxlrator/download
@@ -30,7 +30,6 @@ my $unique=0;
my $mirror=1;
my %dlinfo=();
-my $wgetContinueFlag="";
my $vendorid = $ARGV[0]; if (!defined($vendorid) || $vendorid eq '') { exit; }
my $sourceurl = $ARGV[1]; if (!defined($sourceurl) || $sourceurl eq '') { exit; }
@@ -57,16 +56,15 @@ if($restartdl == 0)
# this is a new download
exit if (-e "$repository/download/$vendorid/$updatefile");
- # dotzball: Why is this necessary?
+ # hinder multiple downloads from starting simultaneously. Create empty "lock" file.
+ # TODO: Another thread may sneak in between these two commands - so not fool-proof, but good enough?
system("touch $repository/download/$vendorid/$updatefile");
- $wgetContinueFlag = "-nc";
}
else
{
# this is a restart of a previous (unfinished) download
# -> continue download
- $wgetContinueFlag = "-c";
&writelog("Continue download: $updatefile");
}
@@ -133,7 +131,9 @@ unless($restartdl)
{
# this is a new download
# -> download from scratch
- unlink "$repository/download/$vendorid/$updatefile";
+
+ #already exited earlier if the file existed, and afterwards created this empty "lock", so if not empty now, another thread is already downloading it.
+ exit if ( -s "$repository/download/$vendorid/$updatefile" );
unlink "$repository/download/$vendorid/$updatefile.info";
}
@@ -147,7 +147,7 @@ $dlinfo{'REMOTESIZE'} = $remote_size;
$dlinfo{'STATUS'} = "1";
&UPDXLT::writehash("$repository/download/$vendorid/$updatefile.info", \%dlinfo);
-my $cmd = "$UPDXLT::wget $login $dlrate --user-agent=\"$UPDXLT::useragent\" -q -P $repository/download/$vendorid $wgetContinueFlag $sourceurl";
+my $cmd = "$UPDXLT::wget $login $dlrate --user-agent=\"$UPDXLT::useragent\" -q -P $repository/download/$vendorid --continue $sourceurl";
$_ = system("$cmd");
$ENV{'http_proxy'} = '';
diff --git a/config/updxlrator/updxlrator b/config/updxlrator/updxlrator
index 2ddc6d8e4..ff23b3a95 100644
--- a/config/updxlrator/updxlrator
+++ b/config/updxlrator/updxlrator
@@ -86,6 +86,8 @@ while (<>) {
&& ($source_url !~ m@\&@)
)
{
+ # NOTE: check_cache will change to $mirror instead of $unique if the filename contains an SHA1 hash
+ # and the URL is not found in cache!
$xlrator_url = &check_cache($source_url,$hostaddr,$username,"Microsoft",$unique);
}
@@ -345,7 +347,9 @@ sub check_cache
my $sourceurl=$_[0];
my $cfmirror=$_[4];
+ $sourceurl =~ s@\%2b(a)+@ig;
$sourceurl =~ s@\%2f@/@ig;
+ $sourceurl =~ s@\%7e@~@ig;
$updfile = substr($sourceurl,rindex($sourceurl,"/")+1);
$updfile =~ s@\%20@ @ig;
@@ -395,10 +399,21 @@ sub check_cache
)
{
&debuglog("File exists in cache and is up to date");
- &debuglog("Retrieving file from cache ($updsource)");
+ &debuglog("Retrieving file from cache ($updsource) for $hostaddr");
&setcachestatus("$updcachedir/$vendorid/$uuid/access.log",time);
$cacheurl="http://$netsettings{'GREEN_ADDRESS'}:$http_port/updatecache/$vendorid/$uuid/$updfile";
}
+ elsif (
+ ($cfmirror == $unique) &&
+ ($vendorid == "microsoft") &&
+ ($source_url =~ m(a).*[0-9a-f]{40}\.[^\.]+(a)i)
+ )
+ {
+ # Most Microsoft updates now have an SHA1 hash in the name. These should be treated as unique files.
+ # Since it wasn't found in the URL cache, switch to mirror mode and try again using just the filename.
+ &debuglog("SHA1: $vendorid $uuid not cached. Reprocessing as mirror $sourceurl");
+ $cacheurl = &check_cache($source_url,$hostaddr,$username,$vendorid,$mirror);
+ }
else
{
if (-e "$updcachedir/$vendorid/$uuid/$updfile")
@@ -412,7 +427,7 @@ sub check_cache
&debuglog("Free disk space: " . &diskfree($updcachedir));
&debuglog("Disk usage: " . &diskusage($updcachedir) . "% (max. $maxusage%)");
if (-e "$updcachedir/download/$vendorid/$updfile") { &debuglog("File download/$vendorid/$updfile exists"); }
- &debuglog("Retrieving file from source ($updsource)");
+ &debuglog("Retrieving file from source ($updsource) for $hostaddr");
if ((!$passive_mode) && (&diskusage($updcachedir) <= $maxusage) && ($remote_size <= &diskfree($updcachedir)) && (!-e "$updcachedir/download/$vendorid/$updfile"))
{
&debuglog("Running command $nice$apphome/bin/download $vendorid $sourceurl $cfmirror &");
diff --git a/html/cgi-bin/fireinfo.cgi b/html/cgi-bin/fireinfo.cgi
index 0060a117c..08fee3d8a 100644
--- a/html/cgi-bin/fireinfo.cgi
+++ b/html/cgi-bin/fireinfo.cgi
@@ -67,7 +67,7 @@ if ($fireinfosettings{'ENABLE_FIREINFO'} ne "on") {
<font color="$Header::colourred">
<p>
$Lang::tr{'fireinfo why descr1'}
- $Lang::tr{'fireinfo why descr2'}<a href="http://fireinfo.ipfire.org" target="_blank">$Lang::tr{'fireinfo why read more'}</a>
+ $Lang::tr{'fireinfo why descr2'}<a href="https://fireinfo.ipfire.org" target="_blank">$Lang::tr{'fireinfo why read more'}</a>
</p>
</font>
END
@@ -122,7 +122,7 @@ print <<END;
<tr>
<td>$Lang::tr{'fireinfo your profile id'}:</td>
<td>
- <a href="http://fireinfo.ipfire.org/profile/$profile_id" target="_blank">$profile_id</a>
+ <a href="https://fireinfo.ipfire.org/profile/$profile_id" target="_blank">$profile_id</a>
</td>
</tr>
<tr>
diff --git a/html/cgi-bin/ids.cgi b/html/cgi-bin/ids.cgi
index ff48333b7..17cb23292 100644
--- a/html/cgi-bin/ids.cgi
+++ b/html/cgi-bin/ids.cgi
@@ -259,9 +259,9 @@ if ($snortsettings{'OINKCODE'} ne "") {
if (!$errormessage) {
if ($snortsettings{'RULES'} eq 'subscripted') {
- $url=" https://www.snort.org/rules/snortrules-snapshot-29110.tar.gz?oinkcode=$snortsettings{'OINKCODE'}";
+ $url=" https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$snortsettings{'OINKCODE'}";
} elsif ($snortsettings{'RULES'} eq 'registered') {
- $url=" https://www.snort.org/rules/snortrules-snapshot-29110.tar.gz?oinkcode=$snortsettings{'OINKCODE'}";
+ $url=" https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$snortsettings{'OINKCODE'}";
} elsif ($snortsettings{'RULES'} eq 'community') {
$url=" https://www.snort.org/rules/community";
} else {
diff --git a/html/cgi-bin/logs.cgi/showrequestfromcountry.dat b/html/cgi-bin/logs.cgi/showrequestfromcountry.dat
index 605873ac0..69835370b 100644
--- a/html/cgi-bin/logs.cgi/showrequestfromcountry.dat
+++ b/html/cgi-bin/logs.cgi/showrequestfromcountry.dat
@@ -349,7 +349,7 @@ foreach $_ (@slice)
if($iface eq $country || $srcaddr ne '') {
my $ccode='';
if($iface ne $country) {
- $ccode = $gi->country_code_by_name($srcaddr);
+ $ccode = &GeoIP::lookup($srcaddr);
}
if($iface eq $country || $ccode eq uc($country)) {
my $chain = '';
diff --git a/lfs/snort b/lfs/snort
index f66bf0127..a5d0535b0 100644
--- a/lfs/snort
+++ b/lfs/snort
@@ -24,7 +24,7 @@
include Config
-VER = 2.9.11
+VER = 2.9.11.1
THISAPP = snort-$(VER)
DL_FILE = $(THISAPP).tar.gz
@@ -40,7 +40,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 4f3ee0c2ad330e487273016efd0c9d64
+$(DL_FILE)_MD5 = 378e3938b2b5c8e358f942d0ffce18cc
install : $(TARGET)
hooks/post-receive
--
IPFire 2.x development tree
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2018-01-07 19:52 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-07 19:52 [git.ipfire.org] IPFire 2.x development tree branch, next, updated. 9fda5e9258de75030316a046bf40f0e2779d69e6 git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox