This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 2.x development tree".
The branch, next has been updated via 40ede5ecb007d8fd25f67d2a1a42a616c1cc8e5b (commit) via 55118ae9aaa30c03ff014330086173d8a31670a7 (commit) via 17e1edad4ccf4b5ee0040691297b028ac8a3a84f (commit) via 7cf3a80b476c046e00042db052a5144f11bc9008 (commit) via 4900fdc9630a0f609f98d029a9f4d7a8068fa06a (commit) via 104fd18530cc03afed5d6f04d6f13c4b709a5e85 (commit) via 272ac1b454e2d83a8be1642a02c55c6a0cdde26c (commit) via 15c80cec1b6619993ef21bf92fb425b77c181d71 (commit) via 10c47fc9e43a7731d4f464391e840c149dd97699 (commit) via 534199c289ee1633bc2b6b0e8a41a5c3f7400d28 (commit) via 7e39a893124da94f4e1cdec6e63d25ef57c18f59 (commit) via 7b616db4e2a2cdfb81c2e6cd1be0813747921905 (commit) via 48396bdffedb612a8841bc26bf070c48ac66502f (commit) via 9b39654d9e148f6c7546ff347e8d4b242570be32 (commit) via 9494c6112a5ff2c0438546ef98dcacc439e931ea (commit) via b78a8742ae1938190ee7956fe861d0cee662bb41 (commit) via 7f5cdd2bfd8e9b5f59e41babf72941c14c5e8736 (commit) from 55c645ba4862818beb67ea46242dbd683f7b1d7f (commit)
Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below.
- Log ----------------------------------------------------------------- commit 40ede5ecb007d8fd25f67d2a1a42a616c1cc8e5b Author: Matthias Fischer matthias.fischer@ipfire.org Date: Mon May 23 00:56:27 2016 +0200
log.dat: Fix identation for clamav
Fixes unneeded tabs in 'log.dat'.
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 55118ae9aaa30c03ff014330086173d8a31670a7 Author: Marcel Lorenz marcel.lorenz@ipfire.org Date: Sat May 28 20:57:40 2016 +0200
findutils: update to 4.6.0
Signed-off-by: Marcel Lorenz marcel.lorenz@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 17e1edad4ccf4b5ee0040691297b028ac8a3a84f Author: Marcel Lorenz marcel.lorenz@ipfire.org Date: Sun May 29 08:15:11 2016 +0200
expect: update to 5.45
Signed-off-by: Marcel Lorenz marcel.lorenz@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 7cf3a80b476c046e00042db052a5144f11bc9008 Author: Marcel Lorenz marcel.lorenz@ipfire.org Date: Tue May 3 17:28:25 2016 +0200
coreutils: update to 8.25 v3
Coreutils 8.25 needs the automake 1.15 I have send this patch earlier. Plaese merge automake before coreutils 8.25
Signed-off-by: Marcel Lorenz marcel.lorenz@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 4900fdc9630a0f609f98d029a9f4d7a8068fa06a Author: Marcel Lorenz marcel.lorenz@ipfire.org Date: Wed Mar 9 07:26:47 2016 +0100
automake: update to 1.15
Signed-off-by: Marcel Lorenz marcel.lorenz@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 104fd18530cc03afed5d6f04d6f13c4b709a5e85 Author: Michael Tremer michael.tremer@ipfire.org Date: Wed Jun 1 22:32:33 2016 +0100
libyajl: Fix broken syntax
Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 272ac1b454e2d83a8be1642a02c55c6a0cdde26c Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:13 2016 +0200
New package libvirt
Libvirt is buidl only on i585 and x86_64 because qemu is build only on this arches.
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 15c80cec1b6619993ef21bf92fb425b77c181d71 Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:11 2016 +0200
New package libyajl
libyajl is a dependency of libvirt
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 10c47fc9e43a7731d4f464391e840c149dd97699 Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:12 2016 +0200
Ship gettext, gettext.sh, envsubst
Libvirt needs gettest.sh gettext and envsubst for the libvirt-guests init script.
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 534199c289ee1633bc2b6b0e8a41a5c3f7400d28 Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:10 2016 +0200
New package libpciaccess
libpciacces is a dependdency of libvirt
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 7e39a893124da94f4e1cdec6e63d25ef57c18f59 Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:09 2016 +0200
New package util-macros
This package is a build dependency of libpciaccess, we do not need this as a package. That's why the rootfiles goes into common and all lines are excluded.
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 7b616db4e2a2cdfb81c2e6cd1be0813747921905 Author: Jonatan Schlag jonatan.schlag@ipfire.org Date: Sat May 7 16:01:08 2016 +0200
Network: add macvtap mode
This change make it possible to use a macvtap interface as a standard interface (green0). This is required by libvirt, because libvirt adds macvtap interfaces to the physical interface, but this causes a problem. A VM with this configuration can communicate with the whole network, but not with the Host (IPFire). To solve this problem, the host interface must be also a macvtap interface. This is achieved by: 1. In /var/ipfire/ethernet/settings the mode of a interface could set with GREEN_MODE= ... When the mode is macvtap the physical interface is renamed to green0phys instead of green0. If the mode is not set the normal configuration is applied . 2. The network-hotplug-macvtap script checks if a physical nic ends with "phys". When the interface ends with "phys", the script adds a macvtap interface to the physical nic which is named green0. The MAC address of this interface is set to the MAC address of the physical nic. The MAC address of the physical is set to a random value. We do this because the MAC address of green0 should not change. All services, IP addresses then binds to the macvatap interface, the physical nic is not used. PS.: The script works also with the orange or blue interface, just replace green with orange or blue.
Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 48396bdffedb612a8841bc26bf070c48ac66502f Author: Michael Tremer michael.tremer@ipfire.org Date: Wed Jun 1 22:17:10 2016 +0100
buildsystem: Escape curly brackets
Newer versions of perl complain about using unescaped left curly brackets.
Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 9b39654d9e148f6c7546ff347e8d4b242570be32 Author: Matthias Fischer matthias.fischer@ipfire.org Date: Mon May 23 15:55:13 2016 +0200
log.dat: Added 'squid' and 'snort' plus translations
Added 'squid' and 'snort' to section dropdown in LOGS / SYSTEM LOGS, added translations.
Added translation string for 'web proxy' in '30-network.menu'.
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 9494c6112a5ff2c0438546ef98dcacc439e931ea Author: Matthias Fischer matthias.fischer@ipfire.org Date: Mon May 23 15:55:12 2016 +0200
log.dat: fix missing quotation marks for 'ddns'
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit b78a8742ae1938190ee7956fe861d0cee662bb41 Author: Matthias Fischer matthias.fischer@ipfire.org Date: Mon May 23 14:16:43 2016 +0200
squid 3.5.19: latest patches from upstream
For details, see: http://www.squid-cache.org/Versions/v3/3.5/changesets/
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
commit 7f5cdd2bfd8e9b5f59e41babf72941c14c5e8736 Author: Marcel Lorenz marcel.lorenz@ipfire.org Date: Mon May 30 21:27:35 2016 +0200
flex: update rootfile
Signed-off-by: Marcel Lorenz marcel.lorenz@ipfire.org Signed-off-by: Michael Tremer michael.tremer@ipfire.org
-----------------------------------------------------------------------
Summary of changes: config/menu/30-network.menu | 4 +- config/rootfiles/common/armv5tel/initscripts | 2 + config/rootfiles/common/automake | 245 +- config/rootfiles/common/findutils | 5 +- config/rootfiles/common/flex | 1 + config/rootfiles/common/gettext | 6 +- config/rootfiles/common/i586/initscripts | 2 + config/rootfiles/common/udev | 1 + config/rootfiles/common/util-macros | 4 + config/rootfiles/common/x86_64/initscripts | 2 + config/rootfiles/packages/libpciaccess | 7 + config/rootfiles/packages/libvirt | 285 ++ config/rootfiles/packages/libyajl | 13 + config/udev/60-net.rules | 3 + .../udev/{enable_codel => network-hotplug-macvtap} | 47 +- config/udev/network-hotplug-rename | 11 +- html/cgi-bin/logs.cgi/log.dat | 12 +- lfs/automake | 20 +- lfs/coreutils | 8 +- lfs/expect | 18 +- lfs/findutils | 6 +- lfs/{iperf3 => libpciaccess} | 9 +- lfs/{iperf3 => libvirt} | 23 +- lfs/{opus => libyajl} | 23 +- lfs/squid | 6 + lfs/udev | 2 + lfs/{iperf3 => util-macros} | 9 +- make.sh | 8 +- src/initscripts/init.d/libvirtd | 49 + src/paks/{cups => libvirt}/install.sh | 10 +- src/paks/{fetchmail => libvirt}/uninstall.sh | 5 +- src/paks/{transmission => libvirt}/update.sh | 2 +- src/patches/automake-1.15_perl_warning.patch | 10 + src/patches/coreutils-8.25-i18n-2.patch | 4790 ++++++++++++++++++++ src/patches/expect-5.43.0-spawn-1.patch | 217 - ...ult-behavior-of-libvirt-guests.sh-for-IPF.patch | 30 + src/patches/squid/squid-3.5-14051.patch | 63 + src/patches/squid/squid-3.5-14052.patch | 34 + src/patches/squid/squid-3.5-14053.patch | 46 + src/patches/squid/squid-3.5-14054.patch | 37 + src/patches/squid/squid-3.5-14055.patch | 39 + src/patches/squid/squid-3.5-14056.patch | 36 + tools/check_strings.pl | 2 +- 43 files changed, 5714 insertions(+), 438 deletions(-) create mode 100644 config/rootfiles/common/util-macros create mode 100644 config/rootfiles/packages/libpciaccess create mode 100644 config/rootfiles/packages/libvirt create mode 100644 config/rootfiles/packages/libyajl copy config/udev/{enable_codel => network-hotplug-macvtap} (67%) copy lfs/{iperf3 => libpciaccess} (93%) copy lfs/{iperf3 => libvirt} (73%) copy lfs/{opus => libyajl} (88%) copy lfs/{iperf3 => util-macros} (93%) create mode 100644 src/initscripts/init.d/libvirtd copy src/paks/{cups => libvirt}/install.sh (81%) copy src/paks/{fetchmail => libvirt}/uninstall.sh (95%) copy src/paks/{transmission => libvirt}/update.sh (100%) create mode 100644 src/patches/automake-1.15_perl_warning.patch create mode 100644 src/patches/coreutils-8.25-i18n-2.patch delete mode 100644 src/patches/expect-5.43.0-spawn-1.patch create mode 100644 src/patches/libvirt/0001-Change-default-behavior-of-libvirt-guests.sh-for-IPF.patch create mode 100644 src/patches/squid/squid-3.5-14051.patch create mode 100644 src/patches/squid/squid-3.5-14052.patch create mode 100644 src/patches/squid/squid-3.5-14053.patch create mode 100644 src/patches/squid/squid-3.5-14054.patch create mode 100644 src/patches/squid/squid-3.5-14055.patch create mode 100644 src/patches/squid/squid-3.5-14056.patch
Difference in files: diff --git a/config/menu/30-network.menu b/config/menu/30-network.menu index c50508f..8e1336a 100644 --- a/config/menu/30-network.menu +++ b/config/menu/30-network.menu @@ -3,9 +3,9 @@ 'title' => "$Lang::tr{'net config'}", 'enabled' => 0, }; - $subnetwork->{'20.proxy'} = {'caption' => 'Webproxy', + $subnetwork->{'20.proxy'} = {'caption' => "$Lang::tr{'web proxy'}", 'uri' => '/cgi-bin/proxy.cgi', - 'title' => "Webproxy", + 'title' => "$Lang::tr{'web proxy'}", 'enabled' => 1, }; $subnetwork->{'21.urlfilter'} = {'caption' => $Lang::tr{'url filter'}, diff --git a/config/rootfiles/common/armv5tel/initscripts b/config/rootfiles/common/armv5tel/initscripts index bc253c0..8cbf26f 100644 --- a/config/rootfiles/common/armv5tel/initscripts +++ b/config/rootfiles/common/armv5tel/initscripts @@ -45,6 +45,8 @@ etc/rc.d/init.d/ipsec #etc/rc.d/init.d/lcdproc #etc/rc.d/init.d/lcr etc/rc.d/init.d/leds +etc/rc.d/init.d/libvirt-guests +etc/rc.d/init.d/libvirtd etc/rc.d/init.d/localnet etc/rc.d/init.d/mISDN #etc/rc.d/init.d/mediatomb diff --git a/config/rootfiles/common/automake b/config/rootfiles/common/automake index 0c19dc3..3568fe7 100644 --- a/config/rootfiles/common/automake +++ b/config/rootfiles/common/automake @@ -1,133 +1,130 @@ #usr/bin/aclocal -#usr/bin/aclocal-1.12 +#usr/bin/aclocal-1.15 #usr/bin/automake -#usr/bin/automake-1.12 -#usr/share/aclocal-1.12 -#usr/share/aclocal-1.12/amversion.m4 -#usr/share/aclocal-1.12/ar-lib.m4 -#usr/share/aclocal-1.12/as.m4 -#usr/share/aclocal-1.12/auxdir.m4 -#usr/share/aclocal-1.12/ccstdc.m4 -#usr/share/aclocal-1.12/cond-if.m4 -#usr/share/aclocal-1.12/cond.m4 -#usr/share/aclocal-1.12/depend.m4 -#usr/share/aclocal-1.12/depout.m4 -#usr/share/aclocal-1.12/dmalloc.m4 -#usr/share/aclocal-1.12/gcj.m4 -#usr/share/aclocal-1.12/header.m4 -#usr/share/aclocal-1.12/init.m4 -#usr/share/aclocal-1.12/install-sh.m4 -#usr/share/aclocal-1.12/lead-dot.m4 -#usr/share/aclocal-1.12/lex.m4 -#usr/share/aclocal-1.12/lispdir.m4 -#usr/share/aclocal-1.12/maintainer.m4 -#usr/share/aclocal-1.12/make.m4 -#usr/share/aclocal-1.12/minuso.m4 -#usr/share/aclocal-1.12/missing.m4 -#usr/share/aclocal-1.12/mkdirp.m4 -#usr/share/aclocal-1.12/obsol-gt.m4 -#usr/share/aclocal-1.12/obsol-lt.m4 -#usr/share/aclocal-1.12/obsolete.m4 -#usr/share/aclocal-1.12/options.m4 -#usr/share/aclocal-1.12/protos.m4 -#usr/share/aclocal-1.12/python.m4 -#usr/share/aclocal-1.12/runlog.m4 -#usr/share/aclocal-1.12/sanity.m4 -#usr/share/aclocal-1.12/silent.m4 -#usr/share/aclocal-1.12/strip.m4 -#usr/share/aclocal-1.12/substnot.m4 -#usr/share/aclocal-1.12/tar.m4 -#usr/share/aclocal-1.12/upc.m4 -#usr/share/aclocal-1.12/vala.m4 +#usr/bin/automake-1.15 +#usr/share/aclocal +#usr/share/aclocal-1.15 +#usr/share/aclocal-1.15/amversion.m4 +#usr/share/aclocal-1.15/ar-lib.m4 +#usr/share/aclocal-1.15/as.m4 +#usr/share/aclocal-1.15/auxdir.m4 +#usr/share/aclocal-1.15/cond-if.m4 +#usr/share/aclocal-1.15/cond.m4 +#usr/share/aclocal-1.15/depend.m4 +#usr/share/aclocal-1.15/depout.m4 +#usr/share/aclocal-1.15/dmalloc.m4 +#usr/share/aclocal-1.15/extra-recurs.m4 +#usr/share/aclocal-1.15/gcj.m4 +#usr/share/aclocal-1.15/init.m4 +#usr/share/aclocal-1.15/install-sh.m4 +#usr/share/aclocal-1.15/internal +#usr/share/aclocal-1.15/internal/ac-config-macro-dirs.m4 +#usr/share/aclocal-1.15/lead-dot.m4 +#usr/share/aclocal-1.15/lex.m4 +#usr/share/aclocal-1.15/lispdir.m4 +#usr/share/aclocal-1.15/maintainer.m4 +#usr/share/aclocal-1.15/make.m4 +#usr/share/aclocal-1.15/missing.m4 +#usr/share/aclocal-1.15/mkdirp.m4 +#usr/share/aclocal-1.15/obsolete.m4 +#usr/share/aclocal-1.15/options.m4 +#usr/share/aclocal-1.15/prog-cc-c-o.m4 +#usr/share/aclocal-1.15/python.m4 +#usr/share/aclocal-1.15/runlog.m4 +#usr/share/aclocal-1.15/sanity.m4 +#usr/share/aclocal-1.15/silent.m4 +#usr/share/aclocal-1.15/strip.m4 +#usr/share/aclocal-1.15/substnot.m4 +#usr/share/aclocal-1.15/tar.m4 +#usr/share/aclocal-1.15/upc.m4 +#usr/share/aclocal-1.15/vala.m4 #usr/share/aclocal/README -#usr/share/automake-1.12 -#usr/share/automake-1.12/Automake -#usr/share/automake-1.12/Automake/ChannelDefs.pm -#usr/share/automake-1.12/Automake/Channels.pm -#usr/share/automake-1.12/Automake/Condition.pm -#usr/share/automake-1.12/Automake/Config.pm -#usr/share/automake-1.12/Automake/Configure_ac.pm -#usr/share/automake-1.12/Automake/DisjConditions.pm -#usr/share/automake-1.12/Automake/FileUtils.pm -#usr/share/automake-1.12/Automake/General.pm -#usr/share/automake-1.12/Automake/Getopt.pm -#usr/share/automake-1.12/Automake/Item.pm -#usr/share/automake-1.12/Automake/ItemDef.pm -#usr/share/automake-1.12/Automake/Location.pm -#usr/share/automake-1.12/Automake/Options.pm -#usr/share/automake-1.12/Automake/Rule.pm -#usr/share/automake-1.12/Automake/RuleDef.pm -#usr/share/automake-1.12/Automake/VarDef.pm -#usr/share/automake-1.12/Automake/Variable.pm -#usr/share/automake-1.12/Automake/Version.pm -#usr/share/automake-1.12/Automake/Wrap.pm -#usr/share/automake-1.12/Automake/XFile.pm -#usr/share/automake-1.12/COPYING -#usr/share/automake-1.12/INSTALL -#usr/share/automake-1.12/am -#usr/share/automake-1.12/am/check.am -#usr/share/automake-1.12/am/check2.am -#usr/share/automake-1.12/am/clean-hdr.am -#usr/share/automake-1.12/am/clean.am -#usr/share/automake-1.12/am/compile.am -#usr/share/automake-1.12/am/configure.am -#usr/share/automake-1.12/am/data.am -#usr/share/automake-1.12/am/dejagnu.am -#usr/share/automake-1.12/am/depend.am -#usr/share/automake-1.12/am/depend2.am -#usr/share/automake-1.12/am/distdir.am -#usr/share/automake-1.12/am/footer.am -#usr/share/automake-1.12/am/header-vars.am -#usr/share/automake-1.12/am/header.am -#usr/share/automake-1.12/am/inst-vars.am -#usr/share/automake-1.12/am/install.am -#usr/share/automake-1.12/am/java.am -#usr/share/automake-1.12/am/lang-compile.am -#usr/share/automake-1.12/am/lex.am -#usr/share/automake-1.12/am/library.am -#usr/share/automake-1.12/am/libs.am -#usr/share/automake-1.12/am/libtool.am -#usr/share/automake-1.12/am/lisp.am -#usr/share/automake-1.12/am/ltlib.am -#usr/share/automake-1.12/am/ltlibrary.am -#usr/share/automake-1.12/am/mans-vars.am -#usr/share/automake-1.12/am/mans.am -#usr/share/automake-1.12/am/program.am -#usr/share/automake-1.12/am/progs.am -#usr/share/automake-1.12/am/python.am -#usr/share/automake-1.12/am/remake-hdr.am -#usr/share/automake-1.12/am/scripts.am -#usr/share/automake-1.12/am/subdirs.am -#usr/share/automake-1.12/am/tags.am -#usr/share/automake-1.12/am/texi-vers.am -#usr/share/automake-1.12/am/texibuild.am -#usr/share/automake-1.12/am/texinfos.am -#usr/share/automake-1.12/am/vala.am -#usr/share/automake-1.12/am/yacc.am -#usr/share/automake-1.12/ar-lib -#usr/share/automake-1.12/compile -#usr/share/automake-1.12/config.guess -#usr/share/automake-1.12/config.sub -#usr/share/automake-1.12/depcomp -#usr/share/automake-1.12/elisp-comp -#usr/share/automake-1.12/install-sh -#usr/share/automake-1.12/mdate-sh -#usr/share/automake-1.12/missing -#usr/share/automake-1.12/mkinstalldirs -#usr/share/automake-1.12/py-compile -#usr/share/automake-1.12/tap-driver.pl -#usr/share/automake-1.12/tap-driver.sh -#usr/share/automake-1.12/test-driver -#usr/share/automake-1.12/texinfo.tex -#usr/share/automake-1.12/ylwrap -#usr/share/doc/automake -#usr/share/doc/automake/amhello-1.0.tar.gz +#usr/share/automake-1.15 +#usr/share/automake-1.15/Automake +#usr/share/automake-1.15/Automake/ChannelDefs.pm +#usr/share/automake-1.15/Automake/Channels.pm +#usr/share/automake-1.15/Automake/Condition.pm +#usr/share/automake-1.15/Automake/Config.pm +#usr/share/automake-1.15/Automake/Configure_ac.pm +#usr/share/automake-1.15/Automake/DisjConditions.pm +#usr/share/automake-1.15/Automake/FileUtils.pm +#usr/share/automake-1.15/Automake/General.pm +#usr/share/automake-1.15/Automake/Getopt.pm +#usr/share/automake-1.15/Automake/Item.pm +#usr/share/automake-1.15/Automake/ItemDef.pm +#usr/share/automake-1.15/Automake/Language.pm +#usr/share/automake-1.15/Automake/Location.pm +#usr/share/automake-1.15/Automake/Options.pm +#usr/share/automake-1.15/Automake/Rule.pm +#usr/share/automake-1.15/Automake/RuleDef.pm +#usr/share/automake-1.15/Automake/VarDef.pm +#usr/share/automake-1.15/Automake/Variable.pm +#usr/share/automake-1.15/Automake/Version.pm +#usr/share/automake-1.15/Automake/Wrap.pm +#usr/share/automake-1.15/Automake/XFile.pm +#usr/share/automake-1.15/COPYING +#usr/share/automake-1.15/INSTALL +#usr/share/automake-1.15/am +#usr/share/automake-1.15/am/check.am +#usr/share/automake-1.15/am/check2.am +#usr/share/automake-1.15/am/clean-hdr.am +#usr/share/automake-1.15/am/clean.am +#usr/share/automake-1.15/am/compile.am +#usr/share/automake-1.15/am/configure.am +#usr/share/automake-1.15/am/data.am +#usr/share/automake-1.15/am/dejagnu.am +#usr/share/automake-1.15/am/depend.am +#usr/share/automake-1.15/am/depend2.am +#usr/share/automake-1.15/am/distdir.am +#usr/share/automake-1.15/am/footer.am +#usr/share/automake-1.15/am/header-vars.am +#usr/share/automake-1.15/am/header.am +#usr/share/automake-1.15/am/inst-vars.am +#usr/share/automake-1.15/am/install.am +#usr/share/automake-1.15/am/java.am +#usr/share/automake-1.15/am/lang-compile.am +#usr/share/automake-1.15/am/lex.am +#usr/share/automake-1.15/am/library.am +#usr/share/automake-1.15/am/libs.am +#usr/share/automake-1.15/am/libtool.am +#usr/share/automake-1.15/am/lisp.am +#usr/share/automake-1.15/am/ltlib.am +#usr/share/automake-1.15/am/ltlibrary.am +#usr/share/automake-1.15/am/mans-vars.am +#usr/share/automake-1.15/am/mans.am +#usr/share/automake-1.15/am/program.am +#usr/share/automake-1.15/am/progs.am +#usr/share/automake-1.15/am/python.am +#usr/share/automake-1.15/am/remake-hdr.am +#usr/share/automake-1.15/am/scripts.am +#usr/share/automake-1.15/am/subdirs.am +#usr/share/automake-1.15/am/tags.am +#usr/share/automake-1.15/am/texi-vers.am +#usr/share/automake-1.15/am/texibuild.am +#usr/share/automake-1.15/am/texinfos.am +#usr/share/automake-1.15/am/vala.am +#usr/share/automake-1.15/am/yacc.am +#usr/share/automake-1.15/ar-lib +#usr/share/automake-1.15/compile +#usr/share/automake-1.15/config.guess +#usr/share/automake-1.15/config.sub +#usr/share/automake-1.15/depcomp +#usr/share/automake-1.15/install-sh +#usr/share/automake-1.15/mdate-sh +#usr/share/automake-1.15/missing +#usr/share/automake-1.15/mkinstalldirs +#usr/share/automake-1.15/py-compile +#usr/share/automake-1.15/tap-driver.sh +#usr/share/automake-1.15/test-driver +#usr/share/automake-1.15/texinfo.tex +#usr/share/automake-1.15/ylwrap +#usr/share/doc/automake-1.15 +#usr/share/doc/automake-1.15/amhello-1.0.tar.gz #usr/share/info/automake-history.info #usr/share/info/automake.info #usr/share/info/automake.info-1 #usr/share/info/automake.info-2 -#usr/share/info/automake.info-3 -#usr/share/man/man1/aclocal-1.12.1 +#usr/share/man/man1/aclocal-1.15.1 #usr/share/man/man1/aclocal.1 -#usr/share/man/man1/automake-1.12.1 +#usr/share/man/man1/automake-1.15.1 #usr/share/man/man1/automake.1 diff --git a/config/rootfiles/common/findutils b/config/rootfiles/common/findutils index ac4fffe..edb91a9 100644 --- a/config/rootfiles/common/findutils +++ b/config/rootfiles/common/findutils @@ -1,8 +1,7 @@ +bin/find #etc/fcron.weekly etc/fcron.weekly/updatedb -bin/find usr/bin/locate -#usr/bin/oldfind usr/bin/updatedb usr/bin/xargs #usr/lib/findutils @@ -11,6 +10,8 @@ usr/lib/findutils/code usr/lib/findutils/frcode #usr/share/info/find-maint.info #usr/share/info/find.info +#usr/share/info/find.info-1 +#usr/share/info/find.info-2 #usr/share/man/man1/find.1 #usr/share/man/man1/locate.1 #usr/share/man/man1/updatedb.1 diff --git a/config/rootfiles/common/flex b/config/rootfiles/common/flex index 2b693df..9b97b57 100644 --- a/config/rootfiles/common/flex +++ b/config/rootfiles/common/flex @@ -1,6 +1,7 @@ #usr/bin/flex #usr/bin/lex #usr/include/FlexLexer.h +#usr/info/dir #usr/info/flex.info #usr/info/flex.info-1 #usr/info/flex.info-2 diff --git a/config/rootfiles/common/gettext b/config/rootfiles/common/gettext index e740441..d3b2140 100644 --- a/config/rootfiles/common/gettext +++ b/config/rootfiles/common/gettext @@ -1,7 +1,7 @@ #usr/bin/autopoint -#usr/bin/envsubst -#usr/bin/gettext -#usr/bin/gettext.sh +usr/bin/envsubst +usr/bin/gettext +usr/bin/gettext.sh #usr/bin/gettextize #usr/bin/msgattrib #usr/bin/msgcat diff --git a/config/rootfiles/common/i586/initscripts b/config/rootfiles/common/i586/initscripts index f1024a2..f07bcf6 100644 --- a/config/rootfiles/common/i586/initscripts +++ b/config/rootfiles/common/i586/initscripts @@ -46,6 +46,8 @@ etc/rc.d/init.d/ipsec #etc/rc.d/init.d/lcdproc #etc/rc.d/init.d/lcr etc/rc.d/init.d/leds +etc/rc.d/init.d/libvirt-guests +etc/rc.d/init.d/libvirtd etc/rc.d/init.d/localnet etc/rc.d/init.d/mISDN #etc/rc.d/init.d/mediatomb diff --git a/config/rootfiles/common/udev b/config/rootfiles/common/udev index 4d51954..e1f4bd5 100644 --- a/config/rootfiles/common/udev +++ b/config/rootfiles/common/udev @@ -28,6 +28,7 @@ lib/udev #lib/udev/hwdb.d/60-keyboard.hwdb #lib/udev/init-net-rules.sh #lib/udev/mtd_probe +#lib/udev/network-hotplug-macvtap #lib/udev/network-hotplug-rename #lib/udev/network-hotplug-vlan #lib/udev/rule_generator.functions diff --git a/config/rootfiles/common/util-macros b/config/rootfiles/common/util-macros new file mode 100644 index 0000000..1858667 --- /dev/null +++ b/config/rootfiles/common/util-macros @@ -0,0 +1,4 @@ +#usr/share/aclocal/xorg-macros.m4 +#usr/share/pkgconfig/xorg-macros.pc +#usr/share/util-macros +#usr/share/util-macros/INSTALL diff --git a/config/rootfiles/common/x86_64/initscripts b/config/rootfiles/common/x86_64/initscripts index f1024a2..f07bcf6 100644 --- a/config/rootfiles/common/x86_64/initscripts +++ b/config/rootfiles/common/x86_64/initscripts @@ -46,6 +46,8 @@ etc/rc.d/init.d/ipsec #etc/rc.d/init.d/lcdproc #etc/rc.d/init.d/lcr etc/rc.d/init.d/leds +etc/rc.d/init.d/libvirt-guests +etc/rc.d/init.d/libvirtd etc/rc.d/init.d/localnet etc/rc.d/init.d/mISDN #etc/rc.d/init.d/mediatomb diff --git a/config/rootfiles/packages/libpciaccess b/config/rootfiles/packages/libpciaccess new file mode 100644 index 0000000..d4a3c41 --- /dev/null +++ b/config/rootfiles/packages/libpciaccess @@ -0,0 +1,7 @@ +#usr/include/pciaccess.h +#usr/lib/libpciaccess.a +#usr/lib/libpciaccess.la +#usr/lib/libpciaccess.so +usr/lib/libpciaccess.so.0 +usr/lib/libpciaccess.so.0.11.1 +#usr/lib/pkgconfig/pciaccess.pc diff --git a/config/rootfiles/packages/libvirt b/config/rootfiles/packages/libvirt new file mode 100644 index 0000000..c2d6042 --- /dev/null +++ b/config/rootfiles/packages/libvirt @@ -0,0 +1,285 @@ +#etc/libvirt +etc/libvirt/libvirt.conf +etc/libvirt/libvirtd.conf +#etc/libvirt/nwfilter +#etc/libvirt/nwfilter/allow-arp.xml +#etc/libvirt/nwfilter/allow-dhcp-server.xml +#etc/libvirt/nwfilter/allow-dhcp.xml +#etc/libvirt/nwfilter/allow-incoming-ipv4.xml +#etc/libvirt/nwfilter/allow-ipv4.xml +#etc/libvirt/nwfilter/clean-traffic.xml +#etc/libvirt/nwfilter/no-arp-ip-spoofing.xml +#etc/libvirt/nwfilter/no-arp-mac-spoofing.xml +#etc/libvirt/nwfilter/no-arp-spoofing.xml +#etc/libvirt/nwfilter/no-ip-multicast.xml +#etc/libvirt/nwfilter/no-ip-spoofing.xml +#etc/libvirt/nwfilter/no-mac-broadcast.xml +#etc/libvirt/nwfilter/no-mac-spoofing.xml +#etc/libvirt/nwfilter/no-other-l2-traffic.xml +#etc/libvirt/nwfilter/no-other-rarp-traffic.xml +#etc/libvirt/nwfilter/qemu-announce-self-rarp.xml +#etc/libvirt/nwfilter/qemu-announce-self.xml +etc/libvirt/qemu-lockd.conf +etc/libvirt/qemu.conf +etc/libvirt/virtlockd.conf +etc/logrotate.d/libvirtd +etc/logrotate.d/libvirtd.libxl +#etc/logrotate.d/libvirtd.lxc +etc/logrotate.d/libvirtd.qemu +#etc/logrotate.d/libvirtd.uml +etc/rc.d/init.d/libvirt-guests +etc/rc.d/init.d/libvirtd +usr/bin/virsh +usr/bin/virt-host-validate +usr/bin/virt-pki-validate +usr/bin/virt-xml-validate +#usr/include/libvirt +#usr/include/libvirt/libvirt-domain-snapshot.h +#usr/include/libvirt/libvirt-domain.h +#usr/include/libvirt/libvirt-event.h +#usr/include/libvirt/libvirt-host.h +#usr/include/libvirt/libvirt-interface.h +#usr/include/libvirt/libvirt-lxc.h +#usr/include/libvirt/libvirt-network.h +#usr/include/libvirt/libvirt-nodedev.h +#usr/include/libvirt/libvirt-nwfilter.h +#usr/include/libvirt/libvirt-qemu.h +#usr/include/libvirt/libvirt-secret.h +#usr/include/libvirt/libvirt-storage.h +#usr/include/libvirt/libvirt-stream.h +#usr/include/libvirt/libvirt.h +#usr/include/libvirt/virterror.h +#usr/lib/libvirt +#usr/lib/libvirt-admin.la +#usr/lib/libvirt-admin.so +usr/lib/libvirt-admin.so.0 +usr/lib/libvirt-admin.so.0.1002.18 +#usr/lib/libvirt-lxc.la +#usr/lib/libvirt-lxc.so +usr/lib/libvirt-lxc.so.0 +usr/lib/libvirt-lxc.so.0.1002.18 +#usr/lib/libvirt-qemu.la +#usr/lib/libvirt-qemu.so +usr/lib/libvirt-qemu.so.0 +usr/lib/libvirt-qemu.so.0.1002.18 +#usr/lib/libvirt.la +#usr/lib/libvirt.so +usr/lib/libvirt.so.0 +usr/lib/libvirt.so.0.1002.18 +#usr/lib/libvirt/connection-driver +#usr/lib/libvirt/connection-driver/libvirt_driver_interface.la +usr/lib/libvirt/connection-driver/libvirt_driver_interface.so +#usr/lib/libvirt/connection-driver/libvirt_driver_nodedev.la +usr/lib/libvirt/connection-driver/libvirt_driver_nodedev.so +#usr/lib/libvirt/connection-driver/libvirt_driver_nwfilter.la +usr/lib/libvirt/connection-driver/libvirt_driver_nwfilter.so +#usr/lib/libvirt/connection-driver/libvirt_driver_qemu.la +usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so +#usr/lib/libvirt/connection-driver/libvirt_driver_secret.la +usr/lib/libvirt/connection-driver/libvirt_driver_secret.so +#usr/lib/libvirt/connection-driver/libvirt_driver_storage.la +usr/lib/libvirt/connection-driver/libvirt_driver_storage.so +#usr/lib/libvirt/lock-driver +#usr/lib/libvirt/lock-driver/lockd.la +usr/lib/libvirt/lock-driver/lockd.so +#usr/lib/pkgconfig/libvirt-lxc.pc +#usr/lib/pkgconfig/libvirt-qemu.pc +#usr/lib/pkgconfig/libvirt.pc +#usr/lib/sysctl.d +usr/lib/sysctl.d/60-libvirtd.conf +usr/libexec/libvirt_iohelper +usr/sbin/libvirtd +usr/sbin/virtlockd +#usr/share/augeas +#usr/share/augeas/lenses +#usr/share/augeas/lenses/libvirt_lockd.aug +#usr/share/augeas/lenses/libvirtd.aug +#usr/share/augeas/lenses/libvirtd_qemu.aug +#usr/share/augeas/lenses/tests +#usr/share/augeas/lenses/tests/test_libvirt_lockd.aug +#usr/share/augeas/lenses/tests/test_libvirtd.aug +#usr/share/augeas/lenses/tests/test_libvirtd_qemu.aug +#usr/share/augeas/lenses/tests/test_virtlockd.aug +#usr/share/augeas/lenses/virtlockd.aug +#usr/share/doc/libvirt-1.2.18.3 +#usr/share/doc/libvirt-1.2.18.3/html +#usr/share/doc/libvirt-1.2.18.3/html/32favicon.png +#usr/share/doc/libvirt-1.2.18.3/html/404.html +#usr/share/doc/libvirt-1.2.18.3/html/acl.html +#usr/share/doc/libvirt-1.2.18.3/html/aclpolkit.html +#usr/share/doc/libvirt-1.2.18.3/html/api.html +#usr/share/doc/libvirt-1.2.18.3/html/api_extension.html +#usr/share/doc/libvirt-1.2.18.3/html/apps.html +#usr/share/doc/libvirt-1.2.18.3/html/archdomain.html +#usr/share/doc/libvirt-1.2.18.3/html/architecture.gif +#usr/share/doc/libvirt-1.2.18.3/html/architecture.html +#usr/share/doc/libvirt-1.2.18.3/html/archnetwork.html +#usr/share/doc/libvirt-1.2.18.3/html/archnode.html +#usr/share/doc/libvirt-1.2.18.3/html/archstorage.html +#usr/share/doc/libvirt-1.2.18.3/html/auditlog.html +#usr/share/doc/libvirt-1.2.18.3/html/auth.html +#usr/share/doc/libvirt-1.2.18.3/html/bindings.html +#usr/share/doc/libvirt-1.2.18.3/html/bugs.html +#usr/share/doc/libvirt-1.2.18.3/html/cgroups.html +#usr/share/doc/libvirt-1.2.18.3/html/compiling.html +#usr/share/doc/libvirt-1.2.18.3/html/contact.html +#usr/share/doc/libvirt-1.2.18.3/html/csharp.html +#usr/share/doc/libvirt-1.2.18.3/html/deployment.html +#usr/share/doc/libvirt-1.2.18.3/html/devguide.html +#usr/share/doc/libvirt-1.2.18.3/html/docs.html +#usr/share/doc/libvirt-1.2.18.3/html/downloads.html +#usr/share/doc/libvirt-1.2.18.3/html/drivers.html +#usr/share/doc/libvirt-1.2.18.3/html/drvbhyve.html +#usr/share/doc/libvirt-1.2.18.3/html/drvesx.html +#usr/share/doc/libvirt-1.2.18.3/html/drvhyperv.html +#usr/share/doc/libvirt-1.2.18.3/html/drvlxc.html +#usr/share/doc/libvirt-1.2.18.3/html/drvopenvz.html +#usr/share/doc/libvirt-1.2.18.3/html/drvparallels.html +#usr/share/doc/libvirt-1.2.18.3/html/drvphyp.html +#usr/share/doc/libvirt-1.2.18.3/html/drvqemu.html +#usr/share/doc/libvirt-1.2.18.3/html/drvremote.html +#usr/share/doc/libvirt-1.2.18.3/html/drvtest.html +#usr/share/doc/libvirt-1.2.18.3/html/drvuml.html +#usr/share/doc/libvirt-1.2.18.3/html/drvvbox.html +#usr/share/doc/libvirt-1.2.18.3/html/drvvmware.html +#usr/share/doc/libvirt-1.2.18.3/html/drvxen.html +#usr/share/doc/libvirt-1.2.18.3/html/errors.html +#usr/share/doc/libvirt-1.2.18.3/html/et.png +#usr/share/doc/libvirt-1.2.18.3/html/firewall.html +#usr/share/doc/libvirt-1.2.18.3/html/footer_corner.png +#usr/share/doc/libvirt-1.2.18.3/html/footer_pattern.png +#usr/share/doc/libvirt-1.2.18.3/html/format.html +#usr/share/doc/libvirt-1.2.18.3/html/formatcaps.html +#usr/share/doc/libvirt-1.2.18.3/html/formatdomain.html +#usr/share/doc/libvirt-1.2.18.3/html/formatdomaincaps.html +#usr/share/doc/libvirt-1.2.18.3/html/formatnetwork.html +#usr/share/doc/libvirt-1.2.18.3/html/formatnode.html +#usr/share/doc/libvirt-1.2.18.3/html/formatnwfilter.html +#usr/share/doc/libvirt-1.2.18.3/html/formatsecret.html +#usr/share/doc/libvirt-1.2.18.3/html/formatsnapshot.html +#usr/share/doc/libvirt-1.2.18.3/html/formatstorage.html +#usr/share/doc/libvirt-1.2.18.3/html/formatstorageencryption.html +#usr/share/doc/libvirt-1.2.18.3/html/generic.css +#usr/share/doc/libvirt-1.2.18.3/html/goals.html +#usr/share/doc/libvirt-1.2.18.3/html/governance.html +#usr/share/doc/libvirt-1.2.18.3/html/hacking.html +#usr/share/doc/libvirt-1.2.18.3/html/hooks.html +#usr/share/doc/libvirt-1.2.18.3/html/html +#usr/share/doc/libvirt-1.2.18.3/html/html/home.png +#usr/share/doc/libvirt-1.2.18.3/html/html/index.html +#usr/share/doc/libvirt-1.2.18.3/html/html/left.png +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-domain-snapshot.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-domain.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-event.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-host.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-interface.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-network.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-nodedev.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-nwfilter.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-secret.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-storage.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-libvirt-stream.html +#usr/share/doc/libvirt-1.2.18.3/html/html/libvirt-virterror.html +#usr/share/doc/libvirt-1.2.18.3/html/html/right.png +#usr/share/doc/libvirt-1.2.18.3/html/html/up.png +#usr/share/doc/libvirt-1.2.18.3/html/hvsupport.html +#usr/share/doc/libvirt-1.2.18.3/html/index.html +#usr/share/doc/libvirt-1.2.18.3/html/internals +#usr/share/doc/libvirt-1.2.18.3/html/internals.html +#usr/share/doc/libvirt-1.2.18.3/html/internals/command.html +#usr/share/doc/libvirt-1.2.18.3/html/internals/locking.html +#usr/share/doc/libvirt-1.2.18.3/html/internals/oomtesting.html +#usr/share/doc/libvirt-1.2.18.3/html/internals/rpc.html +#usr/share/doc/libvirt-1.2.18.3/html/intro.html +#usr/share/doc/libvirt-1.2.18.3/html/java.html +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-daemon-arch.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-driver-arch.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-header-bg.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-header-logo.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-net-logical.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-net-physical.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt-object-model.png +#usr/share/doc/libvirt-1.2.18.3/html/libvirt.css +#usr/share/doc/libvirt-1.2.18.3/html/libvirtLogo.png +#usr/share/doc/libvirt-1.2.18.3/html/locking-lockd.html +#usr/share/doc/libvirt-1.2.18.3/html/locking-sanlock.html +#usr/share/doc/libvirt-1.2.18.3/html/locking.html +#usr/share/doc/libvirt-1.2.18.3/html/logging.html +#usr/share/doc/libvirt-1.2.18.3/html/madeWith.png +#usr/share/doc/libvirt-1.2.18.3/html/main.css +#usr/share/doc/libvirt-1.2.18.3/html/migration-managed-direct.png +#usr/share/doc/libvirt-1.2.18.3/html/migration-managed-p2p.png +#usr/share/doc/libvirt-1.2.18.3/html/migration-native.png +#usr/share/doc/libvirt-1.2.18.3/html/migration-tunnel.png +#usr/share/doc/libvirt-1.2.18.3/html/migration-unmanaged-direct.png +#usr/share/doc/libvirt-1.2.18.3/html/migration.html +#usr/share/doc/libvirt-1.2.18.3/html/news.html +#usr/share/doc/libvirt-1.2.18.3/html/node.gif +#usr/share/doc/libvirt-1.2.18.3/html/pending.html +#usr/share/doc/libvirt-1.2.18.3/html/php.html +#usr/share/doc/libvirt-1.2.18.3/html/python.html +#usr/share/doc/libvirt-1.2.18.3/html/relatedlinks.html +#usr/share/doc/libvirt-1.2.18.3/html/remote.html +#usr/share/doc/libvirt-1.2.18.3/html/secureusage.html +#usr/share/doc/libvirt-1.2.18.3/html/securityprocess.html +#usr/share/doc/libvirt-1.2.18.3/html/sitemap.html +#usr/share/doc/libvirt-1.2.18.3/html/storage.html +#usr/share/doc/libvirt-1.2.18.3/html/testapi.html +#usr/share/doc/libvirt-1.2.18.3/html/testsuites.html +#usr/share/doc/libvirt-1.2.18.3/html/testtck.html +#usr/share/doc/libvirt-1.2.18.3/html/todo.html +#usr/share/doc/libvirt-1.2.18.3/html/uri.html +#usr/share/doc/libvirt-1.2.18.3/html/virshcmdref.html +#usr/share/doc/libvirt-1.2.18.3/html/windows.html +#usr/share/gtk-doc/html/libvirt +#usr/share/gtk-doc/html/libvirt/general.html +#usr/share/gtk-doc/html/libvirt/home.png +#usr/share/gtk-doc/html/libvirt/index.html +#usr/share/gtk-doc/html/libvirt/left.png +#usr/share/gtk-doc/html/libvirt/libvirt-virterror.html +#usr/share/gtk-doc/html/libvirt/libvirt.devhelp +#usr/share/gtk-doc/html/libvirt/right.png +#usr/share/gtk-doc/html/libvirt/style.css +#usr/share/gtk-doc/html/libvirt/up.png +#usr/share/libvirt +#usr/share/libvirt/api +usr/share/libvirt/api/libvirt-api.xml +usr/share/libvirt/api/libvirt-lxc-api.xml +usr/share/libvirt/api/libvirt-qemu-api.xml +usr/share/libvirt/cpu_map.xml +#usr/share/libvirt/libvirtLogo.png +#usr/share/libvirt/schemas +usr/share/libvirt/schemas/basictypes.rng +usr/share/libvirt/schemas/capability.rng +usr/share/libvirt/schemas/domain.rng +usr/share/libvirt/schemas/domaincaps.rng +usr/share/libvirt/schemas/domaincommon.rng +usr/share/libvirt/schemas/domainsnapshot.rng +usr/share/libvirt/schemas/interface.rng +usr/share/libvirt/schemas/network.rng +usr/share/libvirt/schemas/networkcommon.rng +usr/share/libvirt/schemas/nodedev.rng +usr/share/libvirt/schemas/nwfilter.rng +usr/share/libvirt/schemas/secret.rng +usr/share/libvirt/schemas/storagecommon.rng +usr/share/libvirt/schemas/storagepool.rng +usr/share/libvirt/schemas/storagevol.rng +#usr/share/man/man1/virsh.1 +#usr/share/man/man1/virt-host-validate.1 +#usr/share/man/man1/virt-pki-validate.1 +#usr/share/man/man1/virt-xml-validate.1 +#usr/share/man/man8/libvirtd.8 +#usr/share/man/man8/virtlockd.8 +#var/cache/libvirt +var/cache/libvirt/qemu +#var/lib/libvirt +var/lib/libvirt/boot +var/lib/libvirt/filesystems +var/lib/libvirt/images +#var/lib/libvirt/lockd +var/lib/libvirt/lockd/files +var/lib/libvirt/qemu +#var/log/libvirt +#var/log/libvirt/lxc +var/log/libvirt/qemu +#var/log/libvirt/uml diff --git a/config/rootfiles/packages/libyajl b/config/rootfiles/packages/libyajl new file mode 100644 index 0000000..f575f6a --- /dev/null +++ b/config/rootfiles/packages/libyajl @@ -0,0 +1,13 @@ +usr/bin/json_reformat +usr/bin/json_verify +#usr/include/yajl +#usr/include/yajl/yajl_common.h +#usr/include/yajl/yajl_gen.h +#usr/include/yajl/yajl_parse.h +#usr/include/yajl/yajl_tree.h +#usr/include/yajl/yajl_version.h +#usr/lib/libyajl.so +usr/lib/libyajl.so.2 +usr/lib/libyajl.so.2.1.0 +#usr/lib/libyajl_s.a +#usr/share/pkgconfig/yajl.pc diff --git a/config/udev/60-net.rules b/config/udev/60-net.rules index e82320c..e031e7a 100644 --- a/config/udev/60-net.rules +++ b/config/udev/60-net.rules @@ -5,3 +5,6 @@ ACTION=="add", SUBSYSTEM=="net", PROGRAM="/lib/udev/network-hotplug-rename", RES # Call a script that will create all virtual devices for a parent device # that has just come up. ACTION=="add", SUBSYSTEM=="net", RUN+="/lib/udev/network-hotplug-vlan" + +# Call a script that will set up macvtap interfaces +ACTION=="add", SUBSYSTEM=="net", RUN+="/lib/udev/network-hotplug-macvtap" diff --git a/config/udev/network-hotplug-macvtap b/config/udev/network-hotplug-macvtap new file mode 100644 index 0000000..7f5da12 --- /dev/null +++ b/config/udev/network-hotplug-macvtap @@ -0,0 +1,46 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2016 IPFire Team info@ipfire.org # +# # +############################################################################ + +[ -n "${INTERFACE}" ] || exit 2 + +PHYSICAL_INTERFACE="${INTERFACE}" +VIRTUAL_INTERFACE="${INTERFACE%phys}" +#VIRTUAL_INTERFACE="${VIRTUAL_INTERFACE}0" + +# Do nothing if the physical interface does not end with "phys" +case "${PHYSICAL_INTERFACE}" in + *phys) + ;; + *) + exit 0 + ;; +esac + +ADDRESS="$(</sys/class/net/${PHYSICAL_INTERFACE}/address)" +rand="$(</proc/sys/kernel/random/uuid)" +rand="${rand//-/}" +GENERATED_ADDRESS=$(echo "02:${rand:0:2}:${rand:2:2}:${rand:4:2}:${rand:6:2}:${rand:8:2}") + +ip link add link "${PHYSICAL_INTERFACE}" "${VIRTUAL_INTERFACE}" address "${ADDRESS}" type macvlan mode bridge +ip link set "${PHYSICAL_INTERFACE}" address "${GENERATED_ADDRESS}" +ip link set "${PHYSICAL_INTERFACE}" up diff --git a/config/udev/network-hotplug-rename b/config/udev/network-hotplug-rename index 331b788..aaae641 100644 --- a/config/udev/network-hotplug-rename +++ b/config/udev/network-hotplug-rename @@ -57,16 +57,23 @@ ADDRESS="$(</sys/class/net/${INTERFACE}/address)" for zone in ${ZONES}; do address="${zone}_MACADDR" device="${zone}_DEV" + mode="${zone}_MODE"
# Skip if address or device is unset [ -n "${!address}" -a -n "${!device}" ] || continue
+ # Compare MAC addresses + [ "${ADDRESS}" = "${!address}" ] || continue + # If a matching interface has been found we will # print the name to which udev will rename it. - if [ "${ADDRESS}" = "${!address}" ]; then + if [ "${!mode}" = "macvtap" ]; then + echo "${!device}phys" + else echo "${!device}" - exit 0 fi + + exit 0 done
# If we get here we have not found a matching device, diff --git a/html/cgi-bin/logs.cgi/log.dat b/html/cgi-bin/logs.cgi/log.dat index e8635d7..a8a7ba4 100644 --- a/html/cgi-bin/logs.cgi/log.dat +++ b/html/cgi-bin/logs.cgi/log.dat @@ -65,20 +65,22 @@ my %sections = ( 'ipsec' => '(ipsec_[\w_]+: |pluto[.*]: |charon: |vpnwatch: )', 'openvpn' => '(openvpnserver[.*]: |.*n2n[.*]: )', 'pakfire' => '(pakfire:) ', - 'wireless' => '(hostapd:|kernel: ath.*:|kernel: wifi[0-9]:) ' + 'wireless' => '(hostapd:|kernel: ath.*:|kernel: wifi[0-9]:) ', + 'squid' => '(squid[.*]: |squid: )', + 'snort' => '(snort[.*]: )' );
# Translations for the %sections array. my %trsections = ( 'ipfire' => 'IPFire', 'red' => 'RED', - 'ddns' => $Lang::tr{'dynamic dns'}, + 'ddns' => "$Lang::tr{'dynamic dns'}", 'dns' => 'DNS', 'dma' => 'Mail', 'dhcp' => "$Lang::tr{'dhcp server'}", 'cron' => 'Cron', 'collectd' => 'Collectd', - 'clamav' => 'ClamAV', + 'clamav' => 'ClamAV', 'ntp' => 'NTP', 'ssh' => 'SSH', 'auth' => "$Lang::tr{'loginlogout'}", @@ -86,7 +88,9 @@ my %trsections = ( 'ipsec' => 'IPSec', 'openvpn' => 'OpenVPN', 'pakfire' => 'Pakfire', - 'wireless' => 'Wireless' + 'wireless' => 'Wireless', + 'squid' => "$Lang::tr{'web proxy'}", + 'snort' => "$Lang::tr{'intrusion detection'}" );
diff --git a/lfs/automake b/lfs/automake index 9b19a1b..a372443 100644 --- a/lfs/automake +++ b/lfs/automake @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2012 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@
include Config
-VER = 1.12.4 +VER = 1.15
THISAPP = automake-$(VER) DL_FILE = $(THISAPP).tar.xz @@ -32,6 +32,16 @@ DL_FROM = $(URL_IPFIRE) DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP)
+# Normal build or /tools build. +# +ifeq "$(ROOT)" "" + PREFIX = /usr + TARGET = $(DIR_INFO)/$(THISAPP) +else + PREFIX = /tools + TARGET = $(DIR_INFO)/$(THISAPP)-tools +endif + ############################################################################### # Top-level Rules ############################################################################### @@ -40,7 +50,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 7395a0420ecb5c9bc43e5fcf4824df36 +$(DL_FILE)_MD5 = 9a1ddb0e053474d9d1105cfe39b0c48d
install : $(TARGET)
@@ -70,7 +80,9 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar Jxf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && ./configure --prefix=/usr --disable-nls + cd $(DIR_APP) && patch -Np1 < $(DIR_SRC)/src/patches/automake-1.15_perl_warning.patch + cd $(DIR_APP) && ./configure --prefix=$(PREFIX) --disable-nls \ + --docdir=/usr/share/doc/automake-1.15 cd $(DIR_APP) && make $(MAKETUNING) cd $(DIR_APP) && make install @rm -rf $(DIR_APP) diff --git a/lfs/coreutils b/lfs/coreutils index ede0493..644871f 100644 --- a/lfs/coreutils +++ b/lfs/coreutils @@ -24,7 +24,7 @@
include Config
-VER = 8.24 +VER = 8.25
THISAPP = coreutils-$(VER) DL_FILE = $(THISAPP).tar.xz @@ -42,7 +42,6 @@ else endif
CONFIGURE_OPTIONS += \ - --disable-selinux \ --enable-largefile \ --disable-rpath \ --enable-install-program=arch \ @@ -60,7 +59,7 @@ objects =$(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 40efdbce865d2458d8da0a9dcee7c16c +$(DL_FILE)_MD5 = 070e43ba7f618d747414ef56ab248a48
install : $(TARGET)
@@ -90,7 +89,8 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && ./configure $(CONFIGURE_OPTIONS) + cd $(DIR_APP) && patch -Np1 < $(DIR_SRC)/src/patches/coreutils-8.25-i18n-2.patch + cd $(DIR_APP) && FORCE_UNSAFE_CONFIGURE=1 && ./configure $(CONFIGURE_OPTIONS) cd $(DIR_APP) && make $(MAKETUNING) cd $(DIR_APP) && make install
diff --git a/lfs/expect b/lfs/expect index d885f2f..5e5c7a6 100644 --- a/lfs/expect +++ b/lfs/expect @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,12 +24,12 @@
include Config
-VER = 5.43.0 +VER = 5.45
-THISAPP = expect-$(VER) +THISAPP = expect$(VER) DL_FILE = $(THISAPP).tar.gz DL_FROM = $(URL_IPFIRE) -DIR_APP = $(DIR_SRC)/expect-5.43 +DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP)-tools
############################################################################### @@ -40,7 +40,7 @@ objects=$(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 43e1dc0e0bc9492cf2e1a6f59f276bc3 +$(DL_FILE)_MD5 = 44e1a4f4c877e9ddc5a542dfa7ecc92b
install : $(TARGET)
@@ -70,8 +70,12 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) $(DIR_SRC)/tcl* && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && patch -Np1 < $(DIR_SRC)/src/patches/$(THISAPP)-spawn-1.patch - cd $(DIR_APP) && ./configure --prefix=/tools --host=$(BUILDTARGET) --with-tcl=/tools/lib --with-tclinclude=/tools/include --with-x=no --disable-nls + cd $(DIR_APP) && ./configure \ + --prefix=/tools \ + --with-tcl=/tools/lib \ + --with-tclinclude=/tools/include \ + --with-x=no \ + --disable-nls cd $(DIR_APP) && make $(MAKETUNING) cd $(DIR_APP) && make SCRIPTS="" install @rm -rf $(DIR_APP) $(DIR_SRC)/tcl* diff --git a/lfs/findutils b/lfs/findutils index ac1ffdb..d1a520b 100644 --- a/lfs/findutils +++ b/lfs/findutils @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@
include Config
-VER = 4.4.2 +VER = 4.6.0
THISAPP = findutils-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -54,7 +54,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 351cc4adb07d54877fa15f75fb77d39f +$(DL_FILE)_MD5 = 9936aa8009438ce185bea2694a997fc1
install : $(TARGET)
diff --git a/lfs/libpciaccess b/lfs/libpciaccess new file mode 100644 index 0000000..0df4f45 --- /dev/null +++ b/lfs/libpciaccess @@ -0,0 +1,83 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = 0.13.4 + +THISAPP = libpciaccess-$(VER) +DL_FILE = $(THISAPP).tar.gz +DL_FROM = $(URL_IPFIRE) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) +PROG = libpciaccess +PAK_VER = 1 + +DEPS = "" + +############################################################################### +# Top-level Rules +############################################################################### + +objects = $(DL_FILE) + +$(DL_FILE) = $(DL_FROM)/$(DL_FILE) + +$(DL_FILE)_MD5 = cc1fad87da60682af1d5fa43a5da45a4 + +install : $(TARGET) +check : $(patsubst %,$(DIR_CHK)/%,$(objects)) + +download :$(patsubst %,$(DIR_DL)/%,$(objects)) + +md5 : $(subst %,%_MD5,$(objects)) + +dist:. + $(PAK) + +############################################################################### +# Downloading, checking, md5sum +############################################################################### + +$(patsubst %,$(DIR_CHK)/%,$(objects)) : + @$(CHECK) + +$(patsubst %,$(DIR_DL)/%,$(objects)) : + @$(LOAD) + +$(subst %,%_MD5,$(objects)) : + @$(MD5) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) + @$(PREBUILD) + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && ./configure --prefix=/usr + cd $(DIR_APP) && make $(MAKETUNING) $(EXTRA_MAKE) + cd $(DIR_APP) && make install + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/lfs/libvirt b/lfs/libvirt new file mode 100644 index 0000000..b18364b --- /dev/null +++ b/lfs/libvirt @@ -0,0 +1,93 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = 1.2.18.3 + +THISAPP = libvirt-$(VER) +DL_FILE = $(THISAPP).tar.gz +DL_FROM = $(URL_IPFIRE) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) +SUP_ARCH = i586 x86_64 +PROG = libvirt +PAK_VER = 1 + +DEPS = "libpciaccess libyajl ncat qemu" + +############################################################################### +# Top-level Rules +############################################################################### + +objects = $(DL_FILE) + +$(DL_FILE) = $(DL_FROM)/$(DL_FILE) + +$(DL_FILE)_MD5 = bcb0738ff66972ddb25cfe0d086c5c37 + +install : $(TARGET) +check : $(patsubst %,$(DIR_CHK)/%,$(objects)) + +download :$(patsubst %,$(DIR_DL)/%,$(objects)) + +md5 : $(subst %,%_MD5,$(objects)) + +dist:. + $(PAK) + +############################################################################### +# Downloading, checking, md5sum +############################################################################### + +$(patsubst %,$(DIR_CHK)/%,$(objects)) : + @$(CHECK) + +$(patsubst %,$(DIR_DL)/%,$(objects)) : + @$(LOAD) + +$(subst %,%_MD5,$(objects)) : + @$(MD5) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) + @$(PREBUILD) + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && patch -Np1 -i $(DIR_SRC)/src/patches/libvirt/0001-Change-default-behavior-of-libvirt-guests.sh-for-IPF.patch + cd $(DIR_APP) && ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc \ + --with-openssl --without-sasl \ + --without-uml --without-vbox --without-lxc --without-esx --without-vmware --without-openvz \ + --without-firewalld --without-network -with-interface --with-virtualport --with-macvtap \ + --disable-nls --without-avahi --without-test-suite -without-dbus \ + --with-storage-dir --without-storage-fs --without-storage-lvm --without-storage-iscsi \ + --without-storage-scsi --without-storage-mpath --without-storage-disk --without-storage-rbd --without-storage-sheepdog --without-storage-gluster --without-storage-zfs + cd $(DIR_APP) && make $(MAKETUNING) $(EXTRA_MAKE) + cd $(DIR_APP) && make install + install -v -m 754 $(DIR_SRC)/src/initscripts/init.d/libvirtd /etc/rc.d/init.d/libvirtd + mv /usr/libexec/libvirt-guests.sh /etc/rc.d/init.d/libvirt-guests + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/lfs/libyajl b/lfs/libyajl new file mode 100644 index 0000000..8748386 --- /dev/null +++ b/lfs/libyajl @@ -0,0 +1,84 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = 2.1.0 + +THISAPP = libyajl-$(VER) +DL_FILE = $(THISAPP).tar.gz +DL_FROM = $(URL_IPFIRE) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) +PROG = libyajl +PAK_VER = 1 + +DEPS = "" + +############################################################################### +# Top-level Rules +############################################################################### + +objects = $(DL_FILE) + +$(DL_FILE) = $(DL_FROM)/$(DL_FILE) + +$(DL_FILE)_MD5 = 58c61232e1be991fd9a7ceecfc78836c + +install: $(TARGET) + +check: $(patsubst %,$(DIR_CHK)/%,$(objects)) + +download: $(patsubst %,$(DIR_DL)/%,$(objects)) + +md5: $(subst %,%_MD5,$(objects)) + +dist: + $(PAK) + +############################################################################### +# Downloading, checking, md5sum +############################################################################### + +$(patsubst %,$(DIR_CHK)/%,$(objects)) : + @$(CHECK) + +$(patsubst %,$(DIR_DL)/%,$(objects)) : + @$(LOAD) + +$(subst %,%_MD5,$(objects)) : + @$(MD5) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) + @$(PREBUILD) + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && cmake -DCMAKE_INSTALL_PREFIX=/usr . + cd $(DIR_APP) && make + cd $(DIR_APP) && make install + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/lfs/squid b/lfs/squid index 74a4f19..edaf943 100644 --- a/lfs/squid +++ b/lfs/squid @@ -70,6 +70,12 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar xaf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14051.patch + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14052.patch + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14053.patch + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14054.patch + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14055.patch + cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid/squid-3.5-14056.patch cd $(DIR_APP) && patch -Np0 -i $(DIR_SRC)/src/patches/squid-3.5.17-fix-max-file-descriptors.patch
cd $(DIR_APP) && autoreconf -vfi diff --git a/lfs/udev b/lfs/udev index 7d5bdbc..61bd337 100644 --- a/lfs/udev +++ b/lfs/udev @@ -109,6 +109,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) /lib/udev/network-hotplug-rename install -v -m 755 $(DIR_SRC)/config/udev/network-hotplug-vlan \ /lib/udev/network-hotplug-vlan + install -v -m 755 $(DIR_SRC)/config/udev/network-hotplug-macvtap \ + /lib/udev/network-hotplug-macvtap install -v -m 644 $(DIR_SRC)/config/udev/60-net.rules \ /lib/udev/rules.d
diff --git a/lfs/util-macros b/lfs/util-macros new file mode 100644 index 0000000..f2c347f --- /dev/null +++ b/lfs/util-macros @@ -0,0 +1,83 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2016 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = 1.19.0 + +THISAPP = util-macros-$(VER) +DL_FILE = $(THISAPP).tar.gz +DL_FROM = $(URL_IPFIRE) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) +PROG = util-macros +PAK_VER = 1 + +DEPS = "" + +############################################################################### +# Top-level Rules +############################################################################### + +objects = $(DL_FILE) + +$(DL_FILE) = $(DL_FROM)/$(DL_FILE) + +$(DL_FILE)_MD5 = 40e1caa49a71a26e0aa68ddd00203717 + +install : $(TARGET) +check : $(patsubst %,$(DIR_CHK)/%,$(objects)) + +download :$(patsubst %,$(DIR_DL)/%,$(objects)) + +md5 : $(subst %,%_MD5,$(objects)) + +dist:. + $(PAK) + +############################################################################### +# Downloading, checking, md5sum +############################################################################### + +$(patsubst %,$(DIR_CHK)/%,$(objects)) : + @$(CHECK) + +$(patsubst %,$(DIR_DL)/%,$(objects)) : + @$(LOAD) + +$(subst %,%_MD5,$(objects)) : + @$(MD5) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) + @$(PREBUILD) + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && ./configure --prefix=/usr + cd $(DIR_APP) && make $(MAKETUNING) $(EXTRA_MAKE) + cd $(DIR_APP) && make install + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/make.sh b/make.sh index 3293fd8..dd0cf91 100755 --- a/make.sh +++ b/make.sh @@ -336,6 +336,8 @@ buildbase() { lfsmake2 gcc lfsmake2 sed lfsmake2 berkeley + lfsmake2 autoconf + lfsmake2 automake lfsmake2 coreutils lfsmake2 iana-etc lfsmake2 m4 @@ -348,8 +350,6 @@ buildbase() { lfsmake2 readline-compat lfsmake2 pcre lfsmake2 pcre-compat - lfsmake2 autoconf - lfsmake2 automake lfsmake2 bash lfsmake2 bzip2 lfsmake2 diffutils @@ -863,6 +863,10 @@ buildipfire() { ipfiremake dmidecode ipfiremake mcelog ipfiremake rtpproxy + ipfiremake util-macros + ipfiremake libpciaccess + ipfiremake libyajl + ipfiremake libvirt }
buildinstaller() { diff --git a/src/initscripts/init.d/libvirtd b/src/initscripts/init.d/libvirtd new file mode 100644 index 0000000..f97d208 --- /dev/null +++ b/src/initscripts/init.d/libvirtd @@ -0,0 +1,49 @@ +#!/bin/sh +######################################################################## +# Begin $rc_base/init.d/libvirtd +# +# Description : libvirtd init script +# +# Authors : Jonatan Schlag - jonatan.schlag@ipfire.org +# +# Version : 01.00 +# +# Notes : +# +######################################################################## + +. /etc/sysconfig/rc +. $rc_functions + +case $1 in + start) + boot_mesg "Load required kernel modules for Libvirt" + modprobe tun + evaluate_retval + boot_mesg "Starting Libvirt Daemon..." + loadproc /usr/sbin/libvirtd -d + + ;; + + stop) + boot_mesg "Stopping Libvirt Daemon..." + killproc /usr/sbin/libvirtd + ;; + + restart) + $0 stop + sleep 1 + $0 start + ;; + + status) + statusproc /usr/sbin/libvirtd + ;; + + *) + echo "Usage: $0 {start|stop|restart|status}" + exit 1 + ;; +esac + +# End $rc_base/init.d/libvirtd diff --git a/src/paks/libvirt/install.sh b/src/paks/libvirt/install.sh new file mode 100644 index 0000000..2832197 --- /dev/null +++ b/src/paks/libvirt/install.sh @@ -0,0 +1,33 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007 IPFire-Team info@ipfire.org. # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +extract_files +start_service --delay 300 --background ${NAME} +ln -svf /etc/init.d/libvirtd /etc/rc.d/rc0.d/K20libvirtd +ln -svf /etc/init.d/libvirtd /etc/rc.d/rc3.d/S70libvirtd +ln -svf /etc/init.d/libvirtd /etc/rc.d/rc6.d/K20libvirtd + +ln -svf /etc/init.d/libvirt-guests /etc/rc.d/rc0.d/K19libvirt-guests +ln -svf /etc/init.d/libvirt-guests /etc/rc.d/rc3.d/S71libvirt-guests +ln -svf /etc/init.d/libvirt-guests /etc/rc.d/rc6.d/K19libvirt-guests diff --git a/src/paks/libvirt/uninstall.sh b/src/paks/libvirt/uninstall.sh new file mode 100644 index 0000000..a558460 --- /dev/null +++ b/src/paks/libvirt/uninstall.sh @@ -0,0 +1,30 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007 IPFire-Team info@ipfire.org. # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +stop_service ${NAME} +remove_files + +rm -f /etc/rc.d/rc*.d/*libvirt-guests +rm -f /etc/rc.d/rc*.d/*libvirtd + diff --git a/src/paks/libvirt/update.sh b/src/paks/libvirt/update.sh new file mode 100644 index 0000000..d0b3ba1 --- /dev/null +++ b/src/paks/libvirt/update.sh @@ -0,0 +1,27 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007 IPFire-Team info@ipfire.org. # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +./uninstall.sh +./install.sh + diff --git a/src/patches/automake-1.15_perl_warning.patch b/src/patches/automake-1.15_perl_warning.patch new file mode 100644 index 0000000..9a031e8 --- /dev/null +++ b/src/patches/automake-1.15_perl_warning.patch @@ -0,0 +1,10 @@ +--- automake-1.15/bin/automake.in ++++ automake-1.15/bin/automake.in +@@ -3878,7 +3878,7 @@ + sub substitute_ac_subst_variables + { + my ($text) = @_; +- $text =~ s/${([^ \t=:+{}]+)}/substitute_ac_subst_variables_worker ($1)/ge; ++ $text =~ s/${([^ \t=:+{}]+)}/substitute_ac_subst_variables_worker ($1)/ge; + return $text; + } diff --git a/src/patches/coreutils-8.25-i18n-2.patch b/src/patches/coreutils-8.25-i18n-2.patch new file mode 100644 index 0000000..ad984ec --- /dev/null +++ b/src/patches/coreutils-8.25-i18n-2.patch @@ -0,0 +1,4790 @@ +Submitted by: DJ Lucas (dj_AT_linuxfromscratch_DOT_org) +Date: 2016-02-09 +Initial Package Version: 8.25 +Upstream Status: Rejected +Origin: Based on Suse's i18n patches at https://build.opensuse.org/package/view_file/Base:System/coreutils/coreutils... +Description: Fixes several i18n issues with various Coreutils programs + +diff -Naurp coreutils-8.25-orig/lib/linebuffer.h coreutils-8.25/lib/linebuffer.h +--- coreutils-8.25-orig/lib/linebuffer.h 2016-01-01 07:45:55.000000000 -0600 ++++ coreutils-8.25/lib/linebuffer.h 2016-02-08 19:07:10.298944609 -0600 +@@ -21,6 +21,11 @@ + + # include <stdio.h> + ++/* Get mbstate_t. */ ++# if HAVE_WCHAR_H ++# include <wchar.h> ++# endif ++ + /* A 'struct linebuffer' holds a line of text. */ + + struct linebuffer +@@ -28,6 +33,9 @@ struct linebuffer + size_t size; /* Allocated. */ + size_t length; /* Used. */ + char *buffer; ++# if HAVE_WCHAR_H ++ mbstate_t state; ++# endif + }; + + /* Initialize linebuffer LINEBUFFER for use. */ +diff -Naurp coreutils-8.25-orig/src/cut.c coreutils-8.25/src/cut.c +--- coreutils-8.25-orig/src/cut.c 2016-01-13 05:08:59.000000000 -0600 ++++ coreutils-8.25/src/cut.c 2016-02-08 19:07:10.300944616 -0600 +@@ -28,6 +28,11 @@ + #include <assert.h> + #include <getopt.h> + #include <sys/types.h> ++ ++/* Get mbstate_t, mbrtowc(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif + #include "system.h" + + #include "error.h" +@@ -38,6 +43,18 @@ + + #include "set-fields.h" + ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX < 2 ++# undef MB_LEN_MAX ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "cut" + +@@ -54,6 +71,52 @@ + } \ + while (0) + ++/* Refill the buffer BUF to get a multibyte character. */ ++#define REFILL_BUFFER(BUF, BUFPOS, BUFLEN, STREAM) \ ++ do \ ++ { \ ++ if (BUFLEN < MB_LEN_MAX && !feof (STREAM) && !ferror (STREAM)) \ ++ { \ ++ memmove (BUF, BUFPOS, BUFLEN); \ ++ BUFLEN += fread (BUF + BUFLEN, sizeof(char), BUFSIZ, STREAM); \ ++ BUFPOS = BUF; \ ++ } \ ++ } \ ++ while (0) ++ ++/* Get wide character on BUFPOS. BUFPOS is not included after that. ++ If byte sequence is not valid as a character, CONVFAIL is true. Otherwise false. */ ++#define GET_NEXT_WC_FROM_BUFFER(WC, BUFPOS, BUFLEN, MBLENGTH, STATE, CONVFAIL) \ ++ do \ ++ { \ ++ mbstate_t state_bak; \ ++ \ ++ if (BUFLEN < 1) \ ++ { \ ++ WC = WEOF; \ ++ break; \ ++ } \ ++ \ ++ /* Get a wide character. */ \ ++ CONVFAIL = false; \ ++ state_bak = STATE; \ ++ MBLENGTH = mbrtowc ((wchar_t *)&WC, BUFPOS, BUFLEN, &STATE); \ ++ \ ++ switch (MBLENGTH) \ ++ { \ ++ case (size_t)-1: \ ++ case (size_t)-2: \ ++ CONVFAIL = true; \ ++ STATE = state_bak; \ ++ /* Fall througn. */ \ ++ \ ++ case 0: \ ++ MBLENGTH = 1; \ ++ break; \ ++ } \ ++ } \ ++ while (0) ++ + + /* Pointer inside RP. When checking if a byte or field is selected + by a finite range, we check if it is between CURRENT_RP.LO +@@ -61,6 +124,9 @@ + CURRENT_RP.HI then we make CURRENT_RP to point to the next range pair. */ + static struct field_range_pair *current_rp; + ++/* Length of the delimiter given as argument to -d. */ ++size_t delimlen; ++ + /* This buffer is used to support the semantics of the -s option + (or lack of same) when the specified field list includes (does + not include) the first field. In both of those cases, the entire +@@ -77,15 +143,25 @@ enum operating_mode + { + undefined_mode, + +- /* Output characters that are in the given bytes. */ ++ /* Output bytes that are at the given positions. */ + byte_mode, + ++ /* Output characters that are at the given positions. */ ++ character_mode, ++ + /* Output the given delimiter-separated fields. */ + field_mode + }; + + static enum operating_mode operating_mode; + ++/* If nonzero, when in byte mode, don't split multibyte characters. */ ++static int byte_mode_character_aware; ++ ++/* If nonzero, the function for single byte locale is work ++ if this program runs on multibyte locale. */ ++static int force_singlebyte_mode; ++ + /* If true do not output lines containing no delimiter characters. + Otherwise, all such lines are printed. This option is valid only + with field mode. */ +@@ -97,6 +173,9 @@ static bool complement; + + /* The delimiter character for field mode. */ + static unsigned char delim; ++#if HAVE_WCHAR_H ++static wchar_t wcdelim; ++#endif + + /* The delimiter for each line/record. */ + static unsigned char line_delim = '\n'; +@@ -164,7 +243,7 @@ Print selected parts of lines from each + -f, --fields=LIST select only these fields; also print any line\n\ + that contains no delimiter character, unless\n\ + the -s option is specified\n\ +- -n (ignored)\n\ ++ -n with -b: don't split multibyte characters\n\ + "), stdout); + fputs (_("\ + --complement complement the set of selected bytes, characters\n\ +@@ -280,6 +359,82 @@ cut_bytes (FILE *stream) + } + } + ++#if HAVE_MBRTOWC ++/* This function is in use for the following case. ++ ++ 1. Read from the stream STREAM, printing to standard output any selected ++ characters. ++ ++ 2. Read from stream STREAM, printing to standard output any selected bytes, ++ without splitting multibyte characters. */ ++ ++static void ++cut_characters_or_cut_bytes_no_split (FILE *stream) ++{ ++ size_t idx; /* number of bytes or characters in the line so far. */ ++ char buf[MB_LEN_MAX + BUFSIZ]; /* For spooling a read byte sequence. */ ++ char *bufpos; /* Next read position of BUF. */ ++ size_t buflen; /* The length of the byte sequence in buf. */ ++ wint_t wc; /* A gotten wide character. */ ++ size_t mblength; /* The byte size of a multibyte character which shows ++ as same character as WC. */ ++ mbstate_t state; /* State of the stream. */ ++ bool convfail = false; /* true, when conversion failed. Otherwise false. */ ++ /* Whether to begin printing delimiters between ranges for the current line. ++ Set after we've begun printing data corresponding to the first range. */ ++ bool print_delimiter = false; ++ ++ idx = 0; ++ buflen = 0; ++ bufpos = buf; ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ current_rp = frp; ++ ++ while (1) ++ { ++ REFILL_BUFFER (buf, bufpos, buflen, stream); ++ ++ GET_NEXT_WC_FROM_BUFFER (wc, bufpos, buflen, mblength, state, convfail); ++ (void) convfail; /* ignore unused */ ++ ++ if (wc == WEOF) ++ { ++ if (idx > 0) ++ putchar (line_delim); ++ break; ++ } ++ else if (wc == line_delim) ++ { ++ putchar (line_delim); ++ idx = 0; ++ print_delimiter = false; ++ current_rp = frp; ++ } ++ else ++ { ++ next_item (&idx); ++ if (print_kth (idx)) ++ { ++ if (output_delimiter_specified) ++ { ++ if (print_delimiter && is_range_start_index (idx)) ++ { ++ fwrite (output_delimiter_string, sizeof (char), ++ output_delimiter_length, stdout); ++ } ++ print_delimiter = true; ++ } ++ fwrite (bufpos, mblength, sizeof(char), stdout); ++ } ++ } ++ ++ buflen -= mblength; ++ bufpos += mblength; ++ } ++} ++#endif ++ + /* Read from stream STREAM, printing to standard output any selected fields. */ + + static void +@@ -425,13 +580,211 @@ cut_fields (FILE *stream) + } + } + ++#if HAVE_MBRTOWC ++static void ++cut_fields_mb (FILE *stream) ++{ ++ int c; ++ size_t field_idx; ++ int found_any_selected_field; ++ int buffer_first_field; ++ int empty_input; ++ char buf[MB_LEN_MAX + BUFSIZ]; /* For spooling a read byte sequence. */ ++ char *bufpos; /* Next read position of BUF. */ ++ size_t buflen; /* The length of the byte sequence in buf. */ ++ wint_t wc = 0; /* A gotten wide character. */ ++ size_t mblength; /* The byte size of a multibyte character which shows ++ as same character as WC. */ ++ mbstate_t state; /* State of the stream. */ ++ bool convfail = false; /* true, when conversion failed. Otherwise false. */ ++ ++ current_rp = frp; ++ ++ found_any_selected_field = 0; ++ field_idx = 1; ++ bufpos = buf; ++ buflen = 0; ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ c = getc (stream); ++ empty_input = (c == EOF); ++ if (c != EOF) ++ { ++ ungetc (c, stream); ++ wc = 0; ++ } ++ else ++ wc = WEOF; ++ ++ /* To support the semantics of the -s flag, we may have to buffer ++ all of the first field to determine whether it is `delimited.' ++ But that is unnecessary if all non-delimited lines must be printed ++ and the first field has been selected, or if non-delimited lines ++ must be suppressed and the first field has *not* been selected. ++ That is because a non-delimited line has exactly one field. */ ++ buffer_first_field = (suppress_non_delimited ^ !print_kth (1)); ++ ++ while (1) ++ { ++ if (field_idx == 1 && buffer_first_field) ++ { ++ int len = 0; ++ ++ while (1) ++ { ++ REFILL_BUFFER (buf, bufpos, buflen, stream); ++ ++ GET_NEXT_WC_FROM_BUFFER ++ (wc, bufpos, buflen, mblength, state, convfail); ++ ++ if (wc == WEOF) ++ break; ++ ++ field_1_buffer = xrealloc (field_1_buffer, len + mblength); ++ memcpy (field_1_buffer + len, bufpos, mblength); ++ len += mblength; ++ buflen -= mblength; ++ bufpos += mblength; ++ ++ if (!convfail && (wc == line_delim || wc == wcdelim)) ++ break; ++ } ++ ++ if (len <= 0 && wc == WEOF) ++ break; ++ ++ /* If the first field extends to the end of line (it is not ++ delimited) and we are printing all non-delimited lines, ++ print this one. */ ++ if (convfail || (!convfail && wc != wcdelim)) ++ { ++ if (suppress_non_delimited) ++ { ++ /* Empty. */ ++ } ++ else ++ { ++ fwrite (field_1_buffer, sizeof (char), len, stdout); ++ /* Make sure the output line is newline terminated. */ ++ if (convfail || (!convfail && wc != line_delim)) ++ putchar (line_delim); ++ } ++ continue; ++ } ++ ++ if (print_kth (1)) ++ { ++ /* Print the field, but not the trailing delimiter. */ ++ fwrite (field_1_buffer, sizeof (char), len - 1, stdout); ++ found_any_selected_field = 1; ++ } ++ next_item (&field_idx); ++ } ++ ++ if (wc != WEOF) ++ { ++ if (print_kth (field_idx)) ++ { ++ if (found_any_selected_field) ++ { ++ fwrite (output_delimiter_string, sizeof (char), ++ output_delimiter_length, stdout); ++ } ++ found_any_selected_field = 1; ++ } ++ ++ while (1) ++ { ++ REFILL_BUFFER (buf, bufpos, buflen, stream); ++ ++ GET_NEXT_WC_FROM_BUFFER ++ (wc, bufpos, buflen, mblength, state, convfail); ++ ++ if (wc == WEOF) ++ break; ++ else if (!convfail && (wc == wcdelim || wc == line_delim)) ++ { ++ buflen -= mblength; ++ bufpos += mblength; ++ break; ++ } ++ ++ if (print_kth (field_idx)) ++ fwrite (bufpos, mblength, sizeof(char), stdout); ++ ++ buflen -= mblength; ++ bufpos += mblength; ++ } ++ } ++ ++ if ((!convfail || wc == line_delim) && buflen < 1) ++ wc = WEOF; ++ ++ if (!convfail && wc == wcdelim) ++ next_item (&field_idx); ++ else if (wc == WEOF || (!convfail && wc == line_delim)) ++ { ++ if (found_any_selected_field ++ || (!empty_input && !(suppress_non_delimited && field_idx == 1))) ++ putchar (line_delim); ++ if (wc == WEOF) ++ break; ++ field_idx = 1; ++ current_rp = frp; ++ found_any_selected_field = 0; ++ } ++ } ++} ++#endif ++ + static void + cut_stream (FILE *stream) + { +- if (operating_mode == byte_mode) +- cut_bytes (stream); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1 && !force_singlebyte_mode) ++ { ++ switch (operating_mode) ++ { ++ case byte_mode: ++ if (byte_mode_character_aware) ++ cut_characters_or_cut_bytes_no_split (stream); ++ else ++ cut_bytes (stream); ++ break; ++ ++ case character_mode: ++ cut_characters_or_cut_bytes_no_split (stream); ++ break; ++ ++ case field_mode: ++ if (delimlen == 1) ++ { ++ /* Check if we have utf8 multibyte locale, so we can use this ++ optimization because of uniqueness of characters, which is ++ not true for e.g. SJIS */ ++ char * loc = setlocale(LC_CTYPE, NULL); ++ if (loc && (strstr (loc, "UTF-8") || strstr (loc, "utf-8") || ++ strstr (loc, "UTF8") || strstr (loc, "utf8"))) ++ { ++ cut_fields (stream); ++ break; ++ } ++ } ++ cut_fields_mb (stream); ++ break; ++ ++ default: ++ abort (); ++ } ++ } + else +- cut_fields (stream); ++#endif ++ { ++ if (operating_mode == field_mode) ++ cut_fields (stream); ++ else ++ cut_bytes (stream); ++ } + } + + /* Process file FILE to standard output. +@@ -483,6 +836,7 @@ main (int argc, char **argv) + bool ok; + bool delim_specified = false; + char *spec_list_string IF_LINT ( = NULL); ++ char mbdelim[MB_LEN_MAX + 1]; + + initialize_main (&argc, &argv); + set_program_name (argv[0]); +@@ -505,7 +859,6 @@ main (int argc, char **argv) + switch (optc) + { + case 'b': +- case 'c': + /* Build the byte list. */ + if (operating_mode != undefined_mode) + FATAL_ERROR (_("only one type of list may be specified")); +@@ -513,6 +866,14 @@ main (int argc, char **argv) + spec_list_string = optarg; + break; + ++ case 'c': ++ /* Build the character list. */ ++ if (operating_mode != undefined_mode) ++ FATAL_ERROR (_("only one type of list may be specified")); ++ operating_mode = character_mode; ++ spec_list_string = optarg; ++ break; ++ + case 'f': + /* Build the field list. */ + if (operating_mode != undefined_mode) +@@ -524,10 +885,38 @@ main (int argc, char **argv) + case 'd': + /* New delimiter. */ + /* Interpret -d '' to mean 'use the NUL byte as the delimiter.' */ +- if (optarg[0] != '\0' && optarg[1] != '\0') +- FATAL_ERROR (_("the delimiter must be a single character")); +- delim = optarg[0]; +- delim_specified = true; ++ { ++#if HAVE_MBRTOWC ++ if(MB_CUR_MAX > 1) ++ { ++ mbstate_t state; ++ ++ memset (&state, '\0', sizeof(mbstate_t)); ++ delimlen = mbrtowc (&wcdelim, optarg, strnlen(optarg, MB_LEN_MAX), &state); ++ ++ if (delimlen == (size_t)-1 || delimlen == (size_t)-2) ++ ++force_singlebyte_mode; ++ else ++ { ++ delimlen = (delimlen < 1) ? 1 : delimlen; ++ if (wcdelim != L'\0' && *(optarg + delimlen) != '\0') ++ FATAL_ERROR (_("the delimiter must be a single character")); ++ memcpy (mbdelim, optarg, delimlen); ++ mbdelim[delimlen] = '\0'; ++ if (delimlen == 1) ++ delim = *optarg; ++ } ++ } ++ ++ if (MB_CUR_MAX <= 1 || force_singlebyte_mode) ++#endif ++ { ++ if (optarg[0] != '\0' && optarg[1] != '\0') ++ FATAL_ERROR (_("the delimiter must be a single character")); ++ delim = (unsigned char) optarg[0]; ++ } ++ delim_specified = true; ++ } + break; + + case OUTPUT_DELIMITER_OPTION: +@@ -540,6 +929,7 @@ main (int argc, char **argv) + break; + + case 'n': ++ byte_mode_character_aware = 1; + break; + + case 's': +@@ -579,15 +969,34 @@ main (int argc, char **argv) + | (complement ? SETFLD_COMPLEMENT : 0) ); + + if (!delim_specified) +- delim = '\t'; ++ { ++ delim = '\t'; ++#ifdef HAVE_MBRTOWC ++ wcdelim = L'\t'; ++ mbdelim[0] = '\t'; ++ mbdelim[1] = '\0'; ++ delimlen = 1; ++#endif ++ } + + if (output_delimiter_string == NULL) + { +- static char dummy[2]; +- dummy[0] = delim; +- dummy[1] = '\0'; +- output_delimiter_string = dummy; +- output_delimiter_length = 1; ++#ifdef HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1 && !force_singlebyte_mode) ++ { ++ output_delimiter_string = xstrdup(mbdelim); ++ output_delimiter_length = delimlen; ++ } ++ ++ if (MB_CUR_MAX <= 1 || force_singlebyte_mode) ++#endif ++ { ++ static char dummy[2]; ++ dummy[0] = delim; ++ dummy[1] = '\0'; ++ output_delimiter_string = dummy; ++ output_delimiter_length = 1; ++ } + } + + if (optind == argc) +diff -Naurp coreutils-8.25-orig/src/expand.c coreutils-8.25/src/expand.c +--- coreutils-8.25-orig/src/expand.c 2016-01-01 07:48:50.000000000 -0600 ++++ coreutils-8.25/src/expand.c 2016-02-08 19:07:10.301944619 -0600 +@@ -37,12 +37,34 @@ + #include <stdio.h> + #include <getopt.h> + #include <sys/types.h> ++ ++/* Get mbstate_t, mbrtowc(), wcwidth(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ ++/* Get iswblank(). */ ++#if HAVE_WCTYPE_H ++# include <wctype.h> ++#endif ++ + #include "system.h" + #include "error.h" + #include "fadvise.h" + #include "quote.h" + #include "xstrndup.h" + ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX < 2 ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "expand" + +@@ -357,6 +379,142 @@ expand (void) + } + } + ++#if HAVE_MBRTOWC ++static void ++expand_multibyte (void) ++{ ++ FILE *fp; /* Input strem. */ ++ mbstate_t i_state; /* Current shift state of the input stream. */ ++ mbstate_t i_state_bak; /* Back up the I_STATE. */ ++ mbstate_t o_state; /* Current shift state of the output stream. */ ++ char buf[MB_LEN_MAX + BUFSIZ]; /* For spooling a read byte sequence. */ ++ char *bufpos = buf; /* Next read position of BUF. */ ++ size_t buflen = 0; /* The length of the byte sequence in buf. */ ++ wchar_t wc; /* A gotten wide character. */ ++ size_t mblength; /* The byte size of a multibyte character ++ which shows as same character as WC. */ ++ int tab_index = 0; /* Index in `tab_list' of next tabstop. */ ++ int column = 0; /* Column on screen of the next char. */ ++ int next_tab_column; /* Column the next tab stop is on. */ ++ int convert = 1; /* If nonzero, perform translations. */ ++ ++ fp = next_file ((FILE *) NULL); ++ if (fp == NULL) ++ return; ++ ++ memset (&o_state, '\0', sizeof(mbstate_t)); ++ memset (&i_state, '\0', sizeof(mbstate_t)); ++ ++ for (;;) ++ { ++ /* Refill the buffer BUF. */ ++ if (buflen < MB_LEN_MAX && !feof(fp) && !ferror(fp)) ++ { ++ memmove (buf, bufpos, buflen); ++ buflen += fread (buf + buflen, sizeof(char), BUFSIZ, fp); ++ bufpos = buf; ++ } ++ ++ /* No character is left in BUF. */ ++ if (buflen < 1) ++ { ++ fp = next_file (fp); ++ ++ if (fp == NULL) ++ break; /* No more files. */ ++ else ++ { ++ memset (&i_state, '\0', sizeof(mbstate_t)); ++ continue; ++ } ++ } ++ ++ /* Get a wide character. */ ++ i_state_bak = i_state; ++ mblength = mbrtowc (&wc, bufpos, buflen, &i_state); ++ ++ switch (mblength) ++ { ++ case (size_t)-1: /* illegal byte sequence. */ ++ case (size_t)-2: ++ mblength = 1; ++ i_state = i_state_bak; ++ if (convert) ++ { ++ ++column; ++ if (convert_entire_line == 0 && !isblank(*bufpos)) ++ convert = 0; ++ } ++ putchar (*bufpos); ++ break; ++ ++ case 0: /* null. */ ++ mblength = 1; ++ if (convert && convert_entire_line == 0) ++ convert = 0; ++ putchar ('\0'); ++ break; ++ ++ default: ++ if (wc == L'\n') /* LF. */ ++ { ++ tab_index = 0; ++ column = 0; ++ convert = 1; ++ putchar ('\n'); ++ } ++ else if (wc == L'\t' && convert) /* Tab. */ ++ { ++ if (tab_size == 0) ++ { ++ /* Do not let tab_index == first_free_tab; ++ stop when it is 1 less. */ ++ while (tab_index < first_free_tab - 1 ++ && column >= tab_list[tab_index]) ++ tab_index++; ++ next_tab_column = tab_list[tab_index]; ++ if (tab_index < first_free_tab - 1) ++ tab_index++; ++ if (column >= next_tab_column) ++ next_tab_column = column + 1; ++ } ++ else ++ next_tab_column = column + tab_size - column % tab_size; ++ ++ while (column < next_tab_column) ++ { ++ putchar (' '); ++ ++column; ++ } ++ } ++ else /* Others. */ ++ { ++ if (convert) ++ { ++ if (wc == L'\b') ++ { ++ if (column > 0) ++ --column; ++ } ++ else ++ { ++ int width; /* The width of WC. */ ++ ++ width = wcwidth (wc); ++ column += (width > 0) ? width : 0; ++ if (convert_entire_line == 0 && !iswblank(wc)) ++ convert = 0; ++ } ++ } ++ fwrite (bufpos, sizeof(char), mblength, stdout); ++ } ++ } ++ buflen -= mblength; ++ bufpos += mblength; ++ } ++} ++#endif ++ + int + main (int argc, char **argv) + { +@@ -421,7 +579,12 @@ main (int argc, char **argv) + + file_list = (optind < argc ? &argv[optind] : stdin_argv); + +- expand (); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ expand_multibyte (); ++ else ++#endif ++ expand (); + + if (have_read_stdin && fclose (stdin) != 0) + error (EXIT_FAILURE, errno, "-"); +diff -Naurp coreutils-8.25-orig/src/fold.c coreutils-8.25/src/fold.c +--- coreutils-8.25-orig/src/fold.c 2016-01-01 07:48:50.000000000 -0600 ++++ coreutils-8.25/src/fold.c 2016-02-08 19:07:10.302944622 -0600 +@@ -22,11 +22,33 @@ + #include <getopt.h> + #include <sys/types.h> + ++/* Get mbstate_t, mbrtowc(), wcwidth(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ ++/* Get iswprint(), iswblank(), wcwidth(). */ ++#if HAVE_WCTYPE_H ++# include <wctype.h> ++#endif ++ + #include "system.h" + #include "error.h" + #include "fadvise.h" + #include "xdectoint.h" + ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX < 2 ++# undef MB_LEN_MAX ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ + #define TAB_WIDTH 8 + + /* The official name of this program (e.g., no 'g' prefix). */ +@@ -34,20 +56,41 @@ + + #define AUTHORS proper_name ("David MacKenzie") + ++#define FATAL_ERROR(Message) \ ++ do \ ++ { \ ++ error (0, 0, (Message)); \ ++ usage (2); \ ++ } \ ++ while (0) ++ ++enum operating_mode ++{ ++ /* Fold texts by columns that are at the given positions. */ ++ column_mode, ++ ++ /* Fold texts by bytes that are at the given positions. */ ++ byte_mode, ++ ++ /* Fold texts by characters that are at the given positions. */ ++ character_mode, ++}; ++ ++/* The argument shows current mode. (Default: column_mode) */ ++static enum operating_mode operating_mode; ++ + /* If nonzero, try to break on whitespace. */ + static bool break_spaces; + +-/* If nonzero, count bytes, not column positions. */ +-static bool count_bytes; +- + /* If nonzero, at least one of the files we read was standard input. */ + static bool have_read_stdin; + +-static char const shortopts[] = "bsw:0::1::2::3::4::5::6::7::8::9::"; ++static char const shortopts[] = "bcsw:0::1::2::3::4::5::6::7::8::9::"; + + static struct option const longopts[] = + { + {"bytes", no_argument, NULL, 'b'}, ++ {"characters", no_argument, NULL, 'c'}, + {"spaces", no_argument, NULL, 's'}, + {"width", required_argument, NULL, 'w'}, + {GETOPT_HELP_OPTION_DECL}, +@@ -75,6 +118,7 @@ Wrap input lines in each FILE, writing t + + fputs (_("\ + -b, --bytes count bytes rather than columns\n\ ++ -c, --characters count characters rather than columns\n\ + -s, --spaces break at spaces\n\ + -w, --width=WIDTH use WIDTH columns instead of 80\n\ + "), stdout); +@@ -92,7 +136,7 @@ Wrap input lines in each FILE, writing t + static size_t + adjust_column (size_t column, char c) + { +- if (!count_bytes) ++ if (operating_mode != byte_mode) + { + if (c == '\b') + { +@@ -115,30 +159,14 @@ adjust_column (size_t column, char c) + to stdout, with maximum line length WIDTH. + Return true if successful. */ + +-static bool +-fold_file (char const *filename, size_t width) ++static void ++fold_text (FILE *istream, size_t width, int *saved_errno) + { +- FILE *istream; + int c; + size_t column = 0; /* Screen column where next char will go. */ + size_t offset_out = 0; /* Index in 'line_out' for next char. */ + static char *line_out = NULL; + static size_t allocated_out = 0; +- int saved_errno; +- +- if (STREQ (filename, "-")) +- { +- istream = stdin; +- have_read_stdin = true; +- } +- else +- istream = fopen (filename, "r"); +- +- if (istream == NULL) +- { +- error (0, errno, "%s", quotef (filename)); +- return false; +- } + + fadvise (istream, FADVISE_SEQUENTIAL); + +@@ -168,6 +196,15 @@ fold_file (char const *filename, size_t + bool found_blank = false; + size_t logical_end = offset_out; + ++ /* If LINE_OUT has no wide character, ++ put a new wide character in LINE_OUT ++ if column is bigger than width. */ ++ if (offset_out == 0) ++ { ++ line_out[offset_out++] = c; ++ continue; ++ } ++ + /* Look for the last blank. */ + while (logical_end) + { +@@ -214,11 +251,221 @@ fold_file (char const *filename, size_t + line_out[offset_out++] = c; + } + +- saved_errno = errno; ++ *saved_errno = errno; ++ ++ if (offset_out) ++ fwrite (line_out, sizeof (char), (size_t) offset_out, stdout); ++ ++} ++ ++#if HAVE_MBRTOWC ++static void ++fold_multibyte_text (FILE *istream, size_t width, int *saved_errno) ++{ ++ char buf[MB_LEN_MAX + BUFSIZ]; /* For spooling a read byte sequence. */ ++ size_t buflen = 0; /* The length of the byte sequence in buf. */ ++ char *bufpos = buf; /* Next read position of BUF. */ ++ wint_t wc; /* A gotten wide character. */ ++ size_t mblength; /* The byte size of a multibyte character which shows ++ as same character as WC. */ ++ mbstate_t state, state_bak; /* State of the stream. */ ++ int convfail = 0; /* 1, when conversion is failed. Otherwise 0. */ ++ ++ static char *line_out = NULL; ++ size_t offset_out = 0; /* Index in `line_out' for next char. */ ++ static size_t allocated_out = 0; ++ ++ int increment; ++ size_t column = 0; ++ ++ size_t last_blank_pos; ++ size_t last_blank_column; ++ int is_blank_seen; ++ int last_blank_increment = 0; ++ int is_bs_following_last_blank; ++ size_t bs_following_last_blank_num; ++ int is_cr_after_last_blank; ++ ++#define CLEAR_FLAGS \ ++ do \ ++ { \ ++ last_blank_pos = 0; \ ++ last_blank_column = 0; \ ++ is_blank_seen = 0; \ ++ is_bs_following_last_blank = 0; \ ++ bs_following_last_blank_num = 0; \ ++ is_cr_after_last_blank = 0; \ ++ } \ ++ while (0) ++ ++#define START_NEW_LINE \ ++ do \ ++ { \ ++ putchar ('\n'); \ ++ column = 0; \ ++ offset_out = 0; \ ++ CLEAR_FLAGS; \ ++ } \ ++ while (0) ++ ++ CLEAR_FLAGS; ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ for (;; bufpos += mblength, buflen -= mblength) ++ { ++ if (buflen < MB_LEN_MAX && !feof (istream) && !ferror (istream)) ++ { ++ memmove (buf, bufpos, buflen); ++ buflen += fread (buf + buflen, sizeof(char), BUFSIZ, istream); ++ bufpos = buf; ++ } ++ ++ if (buflen < 1) ++ break; ++ ++ /* Get a wide character. */ ++ state_bak = state; ++ mblength = mbrtowc ((wchar_t *)&wc, bufpos, buflen, &state); ++ ++ switch (mblength) ++ { ++ case (size_t)-1: ++ case (size_t)-2: ++ convfail++; ++ state = state_bak; ++ /* Fall through. */ ++ ++ case 0: ++ mblength = 1; ++ break; ++ } ++ ++rescan: ++ if (operating_mode == byte_mode) /* byte mode */ ++ increment = mblength; ++ else if (operating_mode == character_mode) /* character mode */ ++ increment = 1; ++ else /* column mode */ ++ { ++ if (convfail) ++ increment = 1; ++ else ++ { ++ switch (wc) ++ { ++ case L'\n': ++ fwrite (line_out, sizeof(char), offset_out, stdout); ++ START_NEW_LINE; ++ continue; ++ ++ case L'\b': ++ increment = (column > 0) ? -1 : 0; ++ break; ++ ++ case L'\r': ++ increment = -1 * column; ++ break; ++ ++ case L'\t': ++ increment = 8 - column % 8; ++ break; ++ ++ default: ++ increment = wcwidth (wc); ++ increment = (increment < 0) ? 0 : increment; ++ } ++ } ++ } ++ ++ if (column + increment > width && break_spaces && last_blank_pos) ++ { ++ fwrite (line_out, sizeof(char), last_blank_pos, stdout); ++ putchar ('\n'); ++ ++ offset_out = offset_out - last_blank_pos; ++ column = column - last_blank_column + ((is_cr_after_last_blank) ++ ? last_blank_increment : bs_following_last_blank_num); ++ memmove (line_out, line_out + last_blank_pos, offset_out); ++ CLEAR_FLAGS; ++ goto rescan; ++ } ++ ++ if (column + increment > width && column != 0) ++ { ++ fwrite (line_out, sizeof(char), offset_out, stdout); ++ START_NEW_LINE; ++ goto rescan; ++ } ++ ++ if (allocated_out < offset_out + mblength) ++ { ++ line_out = X2REALLOC (line_out, &allocated_out); ++ } ++ ++ memcpy (line_out + offset_out, bufpos, mblength); ++ offset_out += mblength; ++ column += increment; ++ ++ if (is_blank_seen && !convfail && wc == L'\r') ++ is_cr_after_last_blank = 1; ++ ++ if (is_bs_following_last_blank && !convfail && wc == L'\b') ++ ++bs_following_last_blank_num; ++ else ++ is_bs_following_last_blank = 0; ++ ++ if (break_spaces && !convfail && iswblank (wc)) ++ { ++ last_blank_pos = offset_out; ++ last_blank_column = column; ++ is_blank_seen = 1; ++ last_blank_increment = increment; ++ is_bs_following_last_blank = 1; ++ bs_following_last_blank_num = 0; ++ is_cr_after_last_blank = 0; ++ } ++ } ++ ++ *saved_errno = errno; + + if (offset_out) + fwrite (line_out, sizeof (char), (size_t) offset_out, stdout); + ++} ++#endif ++ ++/* Fold file FILENAME, or standard input if FILENAME is "-", ++ to stdout, with maximum line length WIDTH. ++ Return 0 if successful, 1 if an error occurs. */ ++ ++static bool ++fold_file (char const *filename, size_t width) ++{ ++ FILE *istream; ++ int saved_errno; ++ ++ if (STREQ (filename, "-")) ++ { ++ istream = stdin; ++ have_read_stdin = 1; ++ } ++ else ++ istream = fopen (filename, "r"); ++ ++ if (istream == NULL) ++ { ++ error (0, errno, "%s", quotef (filename)); ++ return 1; ++ } ++ ++ /* Define how ISTREAM is being folded. */ ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ fold_multibyte_text (istream, width, &saved_errno); ++ else ++#endif ++ fold_text (istream, width, &saved_errno); ++ + if (ferror (istream)) + { + error (0, saved_errno, "%s", quotef (filename)); +@@ -251,7 +498,8 @@ main (int argc, char **argv) + + atexit (close_stdout); + +- break_spaces = count_bytes = have_read_stdin = false; ++ operating_mode = column_mode; ++ break_spaces = have_read_stdin = false; + + while ((optc = getopt_long (argc, argv, shortopts, longopts, NULL)) != -1) + { +@@ -260,7 +508,15 @@ main (int argc, char **argv) + switch (optc) + { + case 'b': /* Count bytes rather than columns. */ +- count_bytes = true; ++ if (operating_mode != column_mode) ++ FATAL_ERROR (_("only one way of folding may be specified")); ++ operating_mode = byte_mode; ++ break; ++ ++ case 'c': ++ if (operating_mode != column_mode) ++ FATAL_ERROR (_("only one way of folding may be specified")); ++ operating_mode = character_mode; + break; + + case 's': /* Break at word boundaries. */ +diff -Naurp coreutils-8.25-orig/src/join.c coreutils-8.25/src/join.c +--- coreutils-8.25-orig/src/join.c 2016-01-13 05:08:59.000000000 -0600 ++++ coreutils-8.25/src/join.c 2016-02-08 19:07:10.303944625 -0600 +@@ -22,18 +22,32 @@ + #include <sys/types.h> + #include <getopt.h> + ++/* Get mbstate_t, mbrtowc(), mbrtowc(), wcwidth(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ ++/* Get iswblank(), towupper. */ ++#if HAVE_WCTYPE_H ++# include <wctype.h> ++#endif ++ + #include "system.h" + #include "error.h" + #include "fadvise.h" + #include "hard-locale.h" + #include "linebuffer.h" +-#include "memcasecmp.h" + #include "quote.h" + #include "stdio--.h" + #include "xmemcoll.h" + #include "xstrtol.h" + #include "argmatch.h" + ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "join" + +@@ -135,10 +149,12 @@ static struct outlist outlist_head; + /* Last element in 'outlist', where a new element can be added. */ + static struct outlist *outlist_end = &outlist_head; + +-/* Tab character separating fields. If negative, fields are separated +- by any nonempty string of blanks, otherwise by exactly one +- tab character whose value (when cast to unsigned char) equals TAB. */ +-static int tab = -1; ++/* Tab character separating fields. If NULL, fields are separated ++ by any nonempty string of blanks. */ ++static char *tab = NULL; ++ ++/* The number of bytes used for tab. */ ++static size_t tablen = 0; + + /* If nonzero, check that the input is correctly ordered. */ + static enum +@@ -275,13 +291,14 @@ xfields (struct line *line) + if (ptr == lim) + return; + +- if (0 <= tab && tab != '\n') ++ if (tab != NULL) + { ++ unsigned char t = tab[0]; + char *sep; +- for (; (sep = memchr (ptr, tab, lim - ptr)) != NULL; ptr = sep + 1) ++ for (; (sep = memchr (ptr, t, lim - ptr)) != NULL; ptr = sep + 1) + extract_field (line, ptr, sep - ptr); + } +- else if (tab < 0) ++ else + { + /* Skip leading blanks before the first field. */ + while (field_sep (*ptr)) +@@ -305,6 +322,147 @@ xfields (struct line *line) + extract_field (line, ptr, lim - ptr); + } + ++#if HAVE_MBRTOWC ++static void ++xfields_multibyte (struct line *line) ++{ ++ char *ptr = line->buf.buffer; ++ char const *lim = ptr + line->buf.length - 1; ++ wchar_t wc = 0; ++ size_t mblength = 1; ++ mbstate_t state, state_bak; ++ ++ memset (&state, 0, sizeof (mbstate_t)); ++ ++ if (ptr >= lim) ++ return; ++ ++ if (tab != NULL) ++ { ++ char *sep = ptr; ++ for (; ptr < lim; ptr = sep + mblength) ++ { ++ sep = ptr; ++ while (sep < lim) ++ { ++ state_bak = state; ++ mblength = mbrtowc (&wc, sep, lim - sep + 1, &state); ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ if (mblength == tablen && !memcmp (sep, tab, mblength)) ++ break; ++ else ++ { ++ sep += mblength; ++ continue; ++ } ++ } ++ ++ if (sep >= lim) ++ break; ++ ++ extract_field (line, ptr, sep - ptr); ++ } ++ } ++ else ++ { ++ /* Skip leading blanks before the first field. */ ++ while(ptr < lim) ++ { ++ state_bak = state; ++ mblength = mbrtowc (&wc, ptr, lim - ptr + 1, &state); ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ break; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ if (!iswblank(wc) && wc != '\n') ++ break; ++ ptr += mblength; ++ } ++ ++ do ++ { ++ char *sep; ++ state_bak = state; ++ mblength = mbrtowc (&wc, ptr, lim - ptr + 1, &state); ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ break; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ sep = ptr + mblength; ++ while (sep < lim) ++ { ++ state_bak = state; ++ mblength = mbrtowc (&wc, sep, lim - sep + 1, &state); ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ break; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ if (iswblank (wc) || wc == '\n') ++ break; ++ ++ sep += mblength; ++ } ++ ++ extract_field (line, ptr, sep - ptr); ++ if (sep >= lim) ++ return; ++ ++ state_bak = state; ++ mblength = mbrtowc (&wc, sep, lim - sep + 1, &state); ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ break; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ ptr = sep + mblength; ++ while (ptr < lim) ++ { ++ state_bak = state; ++ mblength = mbrtowc (&wc, ptr, lim - ptr + 1, &state); ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ mblength = 1; ++ state = state_bak; ++ break; ++ } ++ mblength = (mblength < 1) ? 1 : mblength; ++ ++ if (!iswblank (wc) && wc != '\n') ++ break; ++ ++ ptr += mblength; ++ } ++ } ++ while (ptr < lim); ++ } ++ ++ extract_field (line, ptr, lim - ptr); ++} ++#endif ++ + static void + freeline (struct line *line) + { +@@ -326,56 +484,133 @@ keycmp (struct line const *line1, struct + size_t jf_1, size_t jf_2) + { + /* Start of field to compare in each file. */ +- char *beg1; +- char *beg2; +- +- size_t len1; +- size_t len2; /* Length of fields to compare. */ ++ char *beg[2]; ++ char *copy[2]; ++ size_t len[2]; /* Length of fields to compare. */ + int diff; ++ int i, j; ++ int mallocd = 0; + + if (jf_1 < line1->nfields) + { +- beg1 = line1->fields[jf_1].beg; +- len1 = line1->fields[jf_1].len; ++ beg[0] = line1->fields[jf_1].beg; ++ len[0] = line1->fields[jf_1].len; + } + else + { +- beg1 = NULL; +- len1 = 0; ++ beg[0] = NULL; ++ len[0] = 0; + } + + if (jf_2 < line2->nfields) + { +- beg2 = line2->fields[jf_2].beg; +- len2 = line2->fields[jf_2].len; ++ beg[1] = line2->fields[jf_2].beg; ++ len[1] = line2->fields[jf_2].len; + } + else + { +- beg2 = NULL; +- len2 = 0; ++ beg[1] = NULL; ++ len[1] = 0; + } + +- if (len1 == 0) +- return len2 == 0 ? 0 : -1; +- if (len2 == 0) ++ if (len[0] == 0) ++ return len[1] == 0 ? 0 : -1; ++ if (len[1] == 0) + return 1; + + if (ignore_case) + { +- /* FIXME: ignore_case does not work with NLS (in particular, +- with multibyte chars). */ +- diff = memcasecmp (beg1, beg2, MIN (len1, len2)); ++#ifdef HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ size_t mblength; ++ wchar_t wc, uwc; ++ mbstate_t state, state_bak; ++ ++ memset (&state, '\0', sizeof (mbstate_t)); ++ ++ for (i = 0; i < 2; i++) ++ { ++ mallocd = 1; ++ copy[i] = xmalloc (len[i] + 1); ++ memset (copy[i], '\0',len[i] + 1); ++ ++ for (j = 0; j < MIN (len[0], len[1]);) ++ { ++ state_bak = state; ++ mblength = mbrtowc (&wc, beg[i] + j, len[i] - j, &state); ++ ++ switch (mblength) ++ { ++ case (size_t) -1: ++ case (size_t) -2: ++ state = state_bak; ++ /* Fall through */ ++ case 0: ++ mblength = 1; ++ break; ++ ++ default: ++ uwc = towupper (wc); ++ ++ if (uwc != wc) ++ { ++ mbstate_t state_wc; ++ size_t mblen; ++ ++ memset (&state_wc, '\0', sizeof (mbstate_t)); ++ mblen = wcrtomb (copy[i] + j, uwc, &state_wc); ++ assert (mblen != (size_t)-1); ++ } ++ else ++ memcpy (copy[i] + j, beg[i] + j, mblength); ++ } ++ j += mblength; ++ } ++ copy[i][j] = '\0'; ++ } ++ } ++ else ++#endif ++ { ++ for (i = 0; i < 2; i++) ++ { ++ mallocd = 1; ++ copy[i] = xmalloc (len[i] + 1); ++ ++ for (j = 0; j < MIN (len[0], len[1]); j++) ++ copy[i][j] = toupper (beg[i][j]); ++ ++ copy[i][j] = '\0'; ++ } ++ } + } + else + { +- if (hard_LC_COLLATE) +- return xmemcoll (beg1, len1, beg2, len2); +- diff = memcmp (beg1, beg2, MIN (len1, len2)); ++ copy[0] = beg[0]; ++ copy[1] = beg[1]; ++ } ++ ++ if (hard_LC_COLLATE) ++ { ++ diff = xmemcoll ((char *) copy[0], len[0], (char *) copy[1], len[1]); ++ ++ if (mallocd) ++ for (i = 0; i < 2; i++) ++ free (copy[i]); ++ ++ return diff; + } ++ diff = memcmp (copy[0], copy[1], MIN (len[0], len[1])); ++ ++ if (mallocd) ++ for (i = 0; i < 2; i++) ++ free (copy[i]); ++ + + if (diff) + return diff; +- return len1 < len2 ? -1 : len1 != len2; ++ return len[0] - len[1]; + } + + /* Check that successive input lines PREV and CURRENT from input file +@@ -467,6 +702,11 @@ get_line (FILE *fp, struct line **linep, + } + ++line_no[which - 1]; + ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ xfields_multibyte (line); ++ else ++#endif + xfields (line); + + if (prevline[which - 1]) +@@ -566,21 +806,28 @@ prfield (size_t n, struct line const *li + + /* Output all the fields in line, other than the join field. */ + ++#define PUT_TAB_CHAR \ ++ do \ ++ { \ ++ (tab != NULL) ? \ ++ fwrite(tab, sizeof(char), tablen, stdout) : putchar (' '); \ ++ } \ ++ while (0) ++ + static void + prfields (struct line const *line, size_t join_field, size_t autocount) + { + size_t i; + size_t nfields = autoformat ? autocount : line->nfields; +- char output_separator = tab < 0 ? ' ' : tab; + + for (i = 0; i < join_field && i < nfields; ++i) + { +- putchar (output_separator); ++ PUT_TAB_CHAR; + prfield (i, line); + } + for (i = join_field + 1; i < nfields; ++i) + { +- putchar (output_separator); ++ PUT_TAB_CHAR; + prfield (i, line); + } + } +@@ -591,7 +838,6 @@ static void + prjoin (struct line const *line1, struct line const *line2) + { + const struct outlist *outlist; +- char output_separator = tab < 0 ? ' ' : tab; + size_t field; + struct line const *line; + +@@ -625,7 +871,7 @@ prjoin (struct line const *line1, struct + o = o->next; + if (o == NULL) + break; +- putchar (output_separator); ++ PUT_TAB_CHAR; + } + putchar (eolchar); + } +@@ -1103,21 +1349,46 @@ main (int argc, char **argv) + + case 't': + { +- unsigned char newtab = optarg[0]; ++ char *newtab = NULL; ++ size_t newtablen; ++ newtab = xstrdup (optarg); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ mbstate_t state; ++ ++ memset (&state, 0, sizeof (mbstate_t)); ++ newtablen = mbrtowc (NULL, newtab, ++ strnlen (newtab, MB_LEN_MAX), ++ &state); ++ if (newtablen == (size_t) 0 ++ || newtablen == (size_t) -1 ++ || newtablen == (size_t) -2) ++ newtablen = 1; ++ } ++ else ++#endif ++ newtablen = 1; + if (! newtab) +- newtab = '\n'; /* '' => process the whole line. */ ++ { ++ newtab = (char*)"\n"; /* '' => process the whole line. */ ++ } + else if (optarg[1]) + { +- if (STREQ (optarg, "\0")) +- newtab = '\0'; +- else +- error (EXIT_FAILURE, 0, _("multi-character tab %s"), +- quote (optarg)); ++ if (newtablen == 1 && newtab[1]) ++ { ++ if (STREQ (newtab, "\0")) ++ newtab[0] = '\0'; ++ } ++ } ++ if (tab != NULL && strcmp (tab, newtab)) ++ { ++ free (newtab); ++ error (EXIT_FAILURE, 0, _("incompatible tabs")); + } +- if (0 <= tab && tab != newtab) +- error (EXIT_FAILURE, 0, _("incompatible tabs")); + tab = newtab; +- } ++ tablen = newtablen; ++ } + break; + + case 'z': +diff -Naurp coreutils-8.25-orig/src/pr.c coreutils-8.25/src/pr.c +--- coreutils-8.25-orig/src/pr.c 2016-01-01 07:48:50.000000000 -0600 ++++ coreutils-8.25/src/pr.c 2016-02-08 19:07:10.306944635 -0600 +@@ -311,6 +311,24 @@ + + #include <getopt.h> + #include <sys/types.h> ++ ++/* Get MB_LEN_MAX. */ ++#include <limits.h> ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX == 1 ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Get MB_CUR_MAX. */ ++#include <stdlib.h> ++ ++/* Solaris 2.5 has a bug: <wchar.h> must be included before <wctype.h>. */ ++/* Get mbstate_t, mbrtowc(), wcwidth(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ + #include "system.h" + #include "error.h" + #include "fadvise.h" +@@ -323,6 +341,18 @@ + #include "xstrtol.h" + #include "xdectoint.h" + ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ ++#ifndef HAVE_DECL_WCWIDTH ++"this configure-time declaration test was not run" ++#endif ++#if !HAVE_DECL_WCWIDTH ++extern int wcwidth (); ++#endif ++ + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "pr" + +@@ -415,7 +445,20 @@ struct COLUMN + + typedef struct COLUMN COLUMN; + +-static int char_to_clump (char c); ++/* Funtion pointers to switch functions for single byte locale or for ++ multibyte locale. If multibyte functions do not exist in your sysytem, ++ these pointers always point the function for single byte locale. */ ++static void (*print_char) (char c); ++static int (*char_to_clump) (char c); ++ ++/* Functions for single byte locale. */ ++static void print_char_single (char c); ++static int char_to_clump_single (char c); ++ ++/* Functions for multibyte locale. */ ++static void print_char_multi (char c); ++static int char_to_clump_multi (char c); ++ + static bool read_line (COLUMN *p); + static bool print_page (void); + static bool print_stored (COLUMN *p); +@@ -427,6 +470,7 @@ static void add_line_number (COLUMN *p); + static void getoptnum (const char *n_str, int min, int *num, + const char *errfmt); + static void getoptarg (char *arg, char switch_char, char *character, ++ int *character_length, int *character_width, + int *number); + static void print_files (int number_of_files, char **av); + static void init_parameters (int number_of_files); +@@ -440,7 +484,6 @@ static void store_char (char c); + static void pad_down (unsigned int lines); + static void read_rest_of_line (COLUMN *p); + static void skip_read (COLUMN *p, int column_number); +-static void print_char (char c); + static void cleanup (void); + static void print_sep_string (void); + static void separator_string (const char *optarg_S); +@@ -452,7 +495,7 @@ static COLUMN *column_vector; + we store the leftmost columns contiguously in buff. + To print a line from buff, get the index of the first character + from line_vector[i], and print up to line_vector[i + 1]. */ +-static char *buff; ++static unsigned char *buff; + + /* Index of the position in buff where the next character + will be stored. */ +@@ -556,7 +599,7 @@ static int chars_per_column; + static bool untabify_input = false; + + /* (-e) The input tab character. */ +-static char input_tab_char = '\t'; ++static char input_tab_char[MB_LEN_MAX] = "\t"; + + /* (-e) Tabstops are at chars_per_tab, 2*chars_per_tab, 3*chars_per_tab, ... + where the leftmost column is 1. */ +@@ -566,7 +609,10 @@ static int chars_per_input_tab = 8; + static bool tabify_output = false; + + /* (-i) The output tab character. */ +-static char output_tab_char = '\t'; ++static char output_tab_char[MB_LEN_MAX] = "\t"; ++ ++/* (-i) The byte length of output tab character. */ ++static int output_tab_char_length = 1; + + /* (-i) The width of the output tab. */ + static int chars_per_output_tab = 8; +@@ -636,7 +682,13 @@ static int line_number; + static bool numbered_lines = false; + + /* (-n) Character which follows each line number. */ +-static char number_separator = '\t'; ++static char number_separator[MB_LEN_MAX] = "\t"; ++ ++/* (-n) The byte length of the character which follows each line number. */ ++static int number_separator_length = 1; ++ ++/* (-n) The character width of the character which follows each line number. */ ++static int number_separator_width = 0; + + /* (-n) line counting starts with 1st line of input file (not with 1st + line of 1st page printed). */ +@@ -689,6 +741,7 @@ static bool use_col_separator = false; + -a|COLUMN|-m is a 'space' and with the -J option a 'tab'. */ + static char *col_sep_string = (char *) ""; + static int col_sep_length = 0; ++static int col_sep_width = 0; + static char *column_separator = (char *) " "; + static char *line_separator = (char *) "\t"; + +@@ -839,6 +892,13 @@ separator_string (const char *optarg_S) + col_sep_length = (int) strlen (optarg_S); + col_sep_string = xmalloc (col_sep_length + 1); + strcpy (col_sep_string, optarg_S); ++ ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ col_sep_width = mbswidth (col_sep_string, 0); ++ else ++#endif ++ col_sep_width = col_sep_length; + } + + int +@@ -863,6 +923,21 @@ main (int argc, char **argv) + + atexit (close_stdout); + ++/* Define which functions are used, the ones for single byte locale or the ones ++ for multibyte locale. */ ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ print_char = print_char_multi; ++ char_to_clump = char_to_clump_multi; ++ } ++ else ++#endif ++ { ++ print_char = print_char_single; ++ char_to_clump = char_to_clump_single; ++ } ++ + n_files = 0; + file_names = (argc > 1 + ? xmalloc ((argc - 1) * sizeof (char *)) +@@ -939,8 +1014,12 @@ main (int argc, char **argv) + break; + case 'e': + if (optarg) +- getoptarg (optarg, 'e', &input_tab_char, +- &chars_per_input_tab); ++ { ++ int dummy_length, dummy_width; ++ ++ getoptarg (optarg, 'e', input_tab_char, &dummy_length, ++ &dummy_width, &chars_per_input_tab); ++ } + /* Could check tab width > 0. */ + untabify_input = true; + break; +@@ -953,8 +1032,12 @@ main (int argc, char **argv) + break; + case 'i': + if (optarg) +- getoptarg (optarg, 'i', &output_tab_char, +- &chars_per_output_tab); ++ { ++ int dummy_width; ++ ++ getoptarg (optarg, 'i', output_tab_char, &output_tab_char_length, ++ &dummy_width, &chars_per_output_tab); ++ } + /* Could check tab width > 0. */ + tabify_output = true; + break; +@@ -972,8 +1055,8 @@ main (int argc, char **argv) + case 'n': + numbered_lines = true; + if (optarg) +- getoptarg (optarg, 'n', &number_separator, +- &chars_per_number); ++ getoptarg (optarg, 'n', number_separator, &number_separator_length, ++ &number_separator_width, &chars_per_number); + break; + case 'N': + skip_count = false; +@@ -997,7 +1080,7 @@ main (int argc, char **argv) + old_s = false; + /* Reset an additional input of -s, -S dominates -s */ + col_sep_string = bad_cast (""); +- col_sep_length = 0; ++ col_sep_length = col_sep_width = 0; + use_col_separator = true; + if (optarg) + separator_string (optarg); +@@ -1152,10 +1235,45 @@ getoptnum (const char *n_str, int min, i + a number. */ + + static void +-getoptarg (char *arg, char switch_char, char *character, int *number) ++getoptarg (char *arg, char switch_char, char *character, int *character_length, ++ int *character_width, int *number) + { + if (!ISDIGIT (*arg)) +- *character = *arg++; ++ { ++#ifdef HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) /* for multibyte locale. */ ++ { ++ wchar_t wc; ++ size_t mblength; ++ int width; ++ mbstate_t state = {'\0'}; ++ ++ mblength = mbrtowc (&wc, arg, strnlen(arg, MB_LEN_MAX), &state); ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ *character_length = 1; ++ *character_width = 1; ++ } ++ else ++ { ++ *character_length = (mblength < 1) ? 1 : mblength; ++ width = wcwidth (wc); ++ *character_width = (width < 0) ? 0 : width; ++ } ++ ++ strncpy (character, arg, *character_length); ++ arg += *character_length; ++ } ++ else /* for single byte locale. */ ++#endif ++ { ++ *character = *arg++; ++ *character_length = 1; ++ *character_width = 1; ++ } ++ } ++ + if (*arg) + { + long int tmp_long; +@@ -1177,6 +1295,11 @@ static void + init_parameters (int number_of_files) + { + int chars_used_by_number = 0; ++ int mb_len = 1; ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ mb_len = MB_LEN_MAX; ++#endif + + lines_per_body = lines_per_page - lines_per_header - lines_per_footer; + if (lines_per_body <= 0) +@@ -1214,7 +1337,7 @@ init_parameters (int number_of_files) + else + col_sep_string = column_separator; + +- col_sep_length = 1; ++ col_sep_length = col_sep_width = 1; + use_col_separator = true; + } + /* It's rather pointless to define a TAB separator with column +@@ -1244,11 +1367,11 @@ init_parameters (int number_of_files) + + TAB_WIDTH (chars_per_input_tab, chars_per_number); */ + + /* Estimate chars_per_text without any margin and keep it constant. */ +- if (number_separator == '\t') ++ if (number_separator[0] == '\t') + number_width = (chars_per_number + + TAB_WIDTH (chars_per_default_tab, chars_per_number)); + else +- number_width = chars_per_number + 1; ++ number_width = chars_per_number + number_separator_width; + + /* The number is part of the column width unless we are + printing files in parallel. */ +@@ -1257,7 +1380,7 @@ init_parameters (int number_of_files) + } + + chars_per_column = (chars_per_line - chars_used_by_number +- - (columns - 1) * col_sep_length) / columns; ++ - (columns - 1) * col_sep_width) / columns; + + if (chars_per_column < 1) + error (EXIT_FAILURE, 0, _("page width too narrow")); +@@ -1275,7 +1398,7 @@ init_parameters (int number_of_files) + We've to use 8 as the lower limit, if we use chars_per_default_tab = 8 + to expand a tab which is not an input_tab-char. */ + free (clump_buff); +- clump_buff = xmalloc (MAX (8, chars_per_input_tab)); ++ clump_buff = xmalloc (mb_len * MAX (8, chars_per_input_tab)); + } + + /* Open the necessary files, +@@ -1383,7 +1506,7 @@ init_funcs (void) + + /* Enlarge p->start_position of first column to use the same form of + padding_not_printed with all columns. */ +- h = h + col_sep_length; ++ h = h + col_sep_width; + + /* This loop takes care of all but the rightmost column. */ + +@@ -1417,7 +1540,7 @@ init_funcs (void) + } + else + { +- h = h_next + col_sep_length; ++ h = h_next + col_sep_width; + h_next = h + chars_per_column; + } + } +@@ -1708,9 +1831,9 @@ static void + align_column (COLUMN *p) + { + padding_not_printed = p->start_position; +- if (padding_not_printed - col_sep_length > 0) ++ if (padding_not_printed - col_sep_width > 0) + { +- pad_across_to (padding_not_printed - col_sep_length); ++ pad_across_to (padding_not_printed - col_sep_width); + padding_not_printed = ANYWHERE; + } + +@@ -1981,13 +2104,13 @@ store_char (char c) + /* May be too generous. */ + buff = X2REALLOC (buff, &buff_allocated); + } +- buff[buff_current++] = c; ++ buff[buff_current++] = (unsigned char) c; + } + + static void + add_line_number (COLUMN *p) + { +- int i; ++ int i, j; + char *s; + int num_width; + +@@ -2004,22 +2127,24 @@ add_line_number (COLUMN *p) + /* Tabification is assumed for multiple columns, also for n-separators, + but 'default n-separator = TAB' hasn't been given priority over + equal column_width also specified by POSIX. */ +- if (number_separator == '\t') ++ if (number_separator[0] == '\t') + { + i = number_width - chars_per_number; + while (i-- > 0) + (p->char_func) (' '); + } + else +- (p->char_func) (number_separator); ++ for (j = 0; j < number_separator_length; j++) ++ (p->char_func) (number_separator[j]); + } + else + /* To comply with POSIX, we avoid any expansion of default TAB + separator with a single column output. No column_width requirement + has to be considered. */ + { +- (p->char_func) (number_separator); +- if (number_separator == '\t') ++ for (j = 0; j < number_separator_length; j++) ++ (p->char_func) (number_separator[j]); ++ if (number_separator[0] == '\t') + output_position = POS_AFTER_TAB (chars_per_output_tab, + output_position); + } +@@ -2180,7 +2305,7 @@ print_white_space (void) + while (goal - h_old > 1 + && (h_new = POS_AFTER_TAB (chars_per_output_tab, h_old)) <= goal) + { +- putchar (output_tab_char); ++ fwrite (output_tab_char, sizeof(char), output_tab_char_length, stdout); + h_old = h_new; + } + while (++h_old <= goal) +@@ -2200,6 +2325,7 @@ print_sep_string (void) + { + char *s; + int l = col_sep_length; ++ int not_space_flag; + + s = col_sep_string; + +@@ -2213,6 +2339,7 @@ print_sep_string (void) + { + for (; separators_not_printed > 0; --separators_not_printed) + { ++ not_space_flag = 0; + while (l-- > 0) + { + /* 3 types of sep_strings: spaces only, spaces and chars, +@@ -2226,12 +2353,15 @@ print_sep_string (void) + } + else + { ++ not_space_flag = 1; + if (spaces_not_printed > 0) + print_white_space (); + putchar (*s++); +- ++output_position; + } + } ++ if (not_space_flag) ++ output_position += col_sep_width; ++ + /* sep_string ends with some spaces */ + if (spaces_not_printed > 0) + print_white_space (); +@@ -2259,7 +2389,7 @@ print_clump (COLUMN *p, int n, char *clu + required number of tabs and spaces. */ + + static void +-print_char (char c) ++print_char_single (char c) + { + if (tabify_output) + { +@@ -2283,6 +2413,74 @@ print_char (char c) + putchar (c); + } + ++#ifdef HAVE_MBRTOWC ++static void ++print_char_multi (char c) ++{ ++ static size_t mbc_pos = 0; ++ static char mbc[MB_LEN_MAX] = {'\0'}; ++ static mbstate_t state = {'\0'}; ++ mbstate_t state_bak; ++ wchar_t wc; ++ size_t mblength; ++ int width; ++ ++ if (tabify_output) ++ { ++ state_bak = state; ++ mbc[mbc_pos++] = c; ++ mblength = mbrtowc (&wc, mbc, mbc_pos, &state); ++ ++ while (mbc_pos > 0) ++ { ++ switch (mblength) ++ { ++ case (size_t)-2: ++ state = state_bak; ++ return; ++ ++ case (size_t)-1: ++ state = state_bak; ++ ++output_position; ++ putchar (mbc[0]); ++ memmove (mbc, mbc + 1, MB_CUR_MAX - 1); ++ --mbc_pos; ++ break; ++ ++ case 0: ++ mblength = 1; ++ ++ default: ++ if (wc == L' ') ++ { ++ memmove (mbc, mbc + mblength, MB_CUR_MAX - mblength); ++ --mbc_pos; ++ ++spaces_not_printed; ++ return; ++ } ++ else if (spaces_not_printed > 0) ++ print_white_space (); ++ ++ /* Nonprintables are assumed to have width 0, except L'\b'. */ ++ if ((width = wcwidth (wc)) < 1) ++ { ++ if (wc == L'\b') ++ --output_position; ++ } ++ else ++ output_position += width; ++ ++ fwrite (mbc, sizeof(char), mblength, stdout); ++ memmove (mbc, mbc + mblength, MB_CUR_MAX - mblength); ++ mbc_pos -= mblength; ++ } ++ } ++ return; ++ } ++ putchar (c); ++} ++#endif ++ + /* Skip to page PAGE before printing. + PAGE may be larger than total number of pages. */ + +@@ -2462,9 +2660,9 @@ read_line (COLUMN *p) + align_empty_cols = false; + } + +- if (padding_not_printed - col_sep_length > 0) ++ if (padding_not_printed - col_sep_width > 0) + { +- pad_across_to (padding_not_printed - col_sep_length); ++ pad_across_to (padding_not_printed - col_sep_width); + padding_not_printed = ANYWHERE; + } + +@@ -2534,7 +2732,7 @@ print_stored (COLUMN *p) + int i; + + int line = p->current_line++; +- char *first = &buff[line_vector[line]]; ++ unsigned char *first = &buff[line_vector[line]]; + /* FIXME + UMR: Uninitialized memory read: + * This is occurring while in: +@@ -2546,7 +2744,7 @@ print_stored (COLUMN *p) + xmalloc [xmalloc.c:94] + init_store_cols [pr.c:1648] + */ +- char *last = &buff[line_vector[line + 1]]; ++ unsigned char *last = &buff[line_vector[line + 1]]; + + pad_vertically = true; + +@@ -2565,9 +2763,9 @@ print_stored (COLUMN *p) + } + } + +- if (padding_not_printed - col_sep_length > 0) ++ if (padding_not_printed - col_sep_width > 0) + { +- pad_across_to (padding_not_printed - col_sep_length); ++ pad_across_to (padding_not_printed - col_sep_width); + padding_not_printed = ANYWHERE; + } + +@@ -2580,8 +2778,8 @@ print_stored (COLUMN *p) + if (spaces_not_printed == 0) + { + output_position = p->start_position + end_vector[line]; +- if (p->start_position - col_sep_length == chars_per_margin) +- output_position -= col_sep_length; ++ if (p->start_position - col_sep_width == chars_per_margin) ++ output_position -= col_sep_width; + } + + return true; +@@ -2600,7 +2798,7 @@ print_stored (COLUMN *p) + number of characters is 1.) */ + + static int +-char_to_clump (char c) ++char_to_clump_single (char c) + { + unsigned char uc = c; + char *s = clump_buff; +@@ -2610,10 +2808,10 @@ char_to_clump (char c) + int chars; + int chars_per_c = 8; + +- if (c == input_tab_char) ++ if (c == input_tab_char[0]) + chars_per_c = chars_per_input_tab; + +- if (c == input_tab_char || c == '\t') ++ if (c == input_tab_char[0] || c == '\t') + { + width = TAB_WIDTH (chars_per_c, input_position); + +@@ -2694,6 +2892,164 @@ char_to_clump (char c) + return chars; + } + ++#ifdef HAVE_MBRTOWC ++static int ++char_to_clump_multi (char c) ++{ ++ static size_t mbc_pos = 0; ++ static char mbc[MB_LEN_MAX] = {'\0'}; ++ static mbstate_t state = {'\0'}; ++ mbstate_t state_bak; ++ wchar_t wc; ++ size_t mblength; ++ int wc_width; ++ register char *s = clump_buff; ++ register int i, j; ++ char esc_buff[4]; ++ int width; ++ int chars; ++ int chars_per_c = 8; ++ ++ state_bak = state; ++ mbc[mbc_pos++] = c; ++ mblength = mbrtowc (&wc, mbc, mbc_pos, &state); ++ ++ width = 0; ++ chars = 0; ++ while (mbc_pos > 0) ++ { ++ switch (mblength) ++ { ++ case (size_t)-2: ++ state = state_bak; ++ return 0; ++ ++ case (size_t)-1: ++ state = state_bak; ++ mblength = 1; ++ ++ if (use_esc_sequence || use_cntrl_prefix) ++ { ++ width = +4; ++ chars = +4; ++ *s++ = '\'; ++ sprintf (esc_buff, "%03o", (unsigned char) mbc[0]); ++ for (i = 0; i <= 2; ++i) ++ *s++ = (int) esc_buff[i]; ++ } ++ else ++ { ++ width += 1; ++ chars += 1; ++ *s++ = mbc[0]; ++ } ++ break; ++ ++ case 0: ++ mblength = 1; ++ /* Fall through */ ++ ++ default: ++ if (memcmp (mbc, input_tab_char, mblength) == 0) ++ chars_per_c = chars_per_input_tab; ++ ++ if (memcmp (mbc, input_tab_char, mblength) == 0 || c == '\t') ++ { ++ int width_inc; ++ ++ width_inc = TAB_WIDTH (chars_per_c, input_position); ++ width += width_inc; ++ ++ if (untabify_input) ++ { ++ for (i = width_inc; i; --i) ++ *s++ = ' '; ++ chars += width_inc; ++ } ++ else ++ { ++ for (i = 0; i < mblength; i++) ++ *s++ = mbc[i]; ++ chars += mblength; ++ } ++ } ++ else if ((wc_width = wcwidth (wc)) < 1) ++ { ++ if (use_esc_sequence) ++ { ++ for (i = 0; i < mblength; i++) ++ { ++ width += 4; ++ chars += 4; ++ *s++ = '\'; ++ sprintf (esc_buff, "%03o", (unsigned char) mbc[i]); ++ for (j = 0; j <= 2; ++j) ++ *s++ = (int) esc_buff[j]; ++ } ++ } ++ else if (use_cntrl_prefix) ++ { ++ if (wc < 0200) ++ { ++ width += 2; ++ chars += 2; ++ *s++ = '^'; ++ *s++ = wc ^ 0100; ++ } ++ else ++ { ++ for (i = 0; i < mblength; i++) ++ { ++ width += 4; ++ chars += 4; ++ *s++ = '\'; ++ sprintf (esc_buff, "%03o", (unsigned char) mbc[i]); ++ for (j = 0; j <= 2; ++j) ++ *s++ = (int) esc_buff[j]; ++ } ++ } ++ } ++ else if (wc == L'\b') ++ { ++ width += -1; ++ chars += 1; ++ *s++ = c; ++ } ++ else ++ { ++ width += 0; ++ chars += mblength; ++ for (i = 0; i < mblength; i++) ++ *s++ = mbc[i]; ++ } ++ } ++ else ++ { ++ width += wc_width; ++ chars += mblength; ++ for (i = 0; i < mblength; i++) ++ *s++ = mbc[i]; ++ } ++ } ++ memmove (mbc, mbc + mblength, MB_CUR_MAX - mblength); ++ mbc_pos -= mblength; ++ } ++ ++ /* Too many backspaces must put us in position 0 -- never negative. */ ++ if (width < 0 && input_position == 0) ++ { ++ chars = 0; ++ input_position = 0; ++ } ++ else if (width < 0 && input_position <= -width) ++ input_position = 0; ++ else ++ input_position += width; ++ ++ return chars; ++} ++#endif ++ + /* We've just printed some files and need to clean up things before + looking for more options and printing the next batch of files. + +diff -Naurp coreutils-8.25-orig/src/sort.c coreutils-8.25/src/sort.c +--- coreutils-8.25-orig/src/sort.c 2016-01-16 13:09:33.000000000 -0600 ++++ coreutils-8.25/src/sort.c 2016-02-08 19:07:10.310944648 -0600 +@@ -29,6 +29,14 @@ + #include <sys/wait.h> + #include <signal.h> + #include <assert.h> ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++/* Get isw* functions. */ ++#if HAVE_WCTYPE_H ++# include <wctype.h> ++#endif ++ + #include "system.h" + #include "argmatch.h" + #include "error.h" +@@ -163,14 +171,39 @@ static int decimal_point; + /* Thousands separator; if -1, then there isn't one. */ + static int thousands_sep; + ++/* True if -f is specified. */ ++static bool folding; ++ + /* Nonzero if the corresponding locales are hard. */ + static bool hard_LC_COLLATE; +-#if HAVE_NL_LANGINFO ++#if HAVE_LANGINFO_CODESET + static bool hard_LC_TIME; + #endif + + #define NONZERO(x) ((x) != 0) + ++/* get a multibyte character's byte length. */ ++#define GET_BYTELEN_OF_CHAR(LIM, PTR, MBLENGTH, STATE) \ ++ do \ ++ { \ ++ wchar_t wc; \ ++ mbstate_t state_bak; \ ++ \ ++ state_bak = STATE; \ ++ mblength = mbrtowc (&wc, PTR, LIM - PTR, &STATE); \ ++ \ ++ switch (MBLENGTH) \ ++ { \ ++ case (size_t)-1: \ ++ case (size_t)-2: \ ++ STATE = state_bak; \ ++ /* Fall through. */ \ ++ case 0: \ ++ MBLENGTH = 1; \ ++ } \ ++ } \ ++ while (0) ++ + /* The kind of blanks for '-b' to skip in various options. */ + enum blanktype { bl_start, bl_end, bl_both }; + +@@ -344,13 +377,11 @@ static bool reverse; + they were read if all keys compare equal. */ + static bool stable; + +-/* If TAB has this value, blanks separate fields. */ +-enum { TAB_DEFAULT = CHAR_MAX + 1 }; +- +-/* Tab character separating fields. If TAB_DEFAULT, then fields are ++/* Tab character separating fields. If tab_length is 0, then fields are + separated by the empty string between a non-blank character and a blank + character. */ +-static int tab = TAB_DEFAULT; ++static char tab[MB_LEN_MAX + 1]; ++static size_t tab_length = 0; + + /* Flag to remove consecutive duplicate lines from the output. + Only the last of a sequence of equal lines will be output. */ +@@ -810,6 +841,46 @@ reap_all (void) + reap (-1); + } + ++/* Function pointers. */ ++static void ++(*inittables) (void); ++static char * ++(*begfield) (const struct line*, const struct keyfield *); ++static char * ++(*limfield) (const struct line*, const struct keyfield *); ++static void ++(*skipblanks) (char **ptr, char *lim); ++static int ++(*getmonth) (char const *, size_t, char **); ++static int ++(*keycompare) (const struct line *, const struct line *); ++static int ++(*numcompare) (const char *, const char *); ++ ++/* Test for white space multibyte character. ++ Set LENGTH the byte length of investigated multibyte character. */ ++#if HAVE_MBRTOWC ++static int ++ismbblank (const char *str, size_t len, size_t *length) ++{ ++ size_t mblength; ++ wchar_t wc; ++ mbstate_t state; ++ ++ memset (&state, '\0', sizeof(mbstate_t)); ++ mblength = mbrtowc (&wc, str, len, &state); ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ *length = 1; ++ return 0; ++ } ++ ++ *length = (mblength < 1) ? 1 : mblength; ++ return iswblank (wc) || wc == '\n'; ++} ++#endif ++ + /* Clean up any remaining temporary files. */ + + static void +@@ -1254,7 +1325,7 @@ zaptemp (char const *name) + free (node); + } + +-#if HAVE_NL_LANGINFO ++#if HAVE_LANGINFO_CODESET + + static int + struct_month_cmp (void const *m1, void const *m2) +@@ -1269,7 +1340,7 @@ struct_month_cmp (void const *m1, void c + /* Initialize the character class tables. */ + + static void +-inittables (void) ++inittables_uni (void) + { + size_t i; + +@@ -1281,7 +1352,7 @@ inittables (void) + fold_toupper[i] = toupper (i); + } + +-#if HAVE_NL_LANGINFO ++#if HAVE_LANGINFO_CODESET + /* If we're not in the "C" locale, read different names for months. */ + if (hard_LC_TIME) + { +@@ -1363,6 +1434,84 @@ specify_nmerge (int oi, char c, char con + xstrtol_fatal (e, oi, c, long_options, s); + } + ++#if HAVE_MBRTOWC ++static void ++inittables_mb (void) ++{ ++ int i, j, k, l; ++ char *name, *s, *lc_time, *lc_ctype; ++ size_t s_len, mblength; ++ char mbc[MB_LEN_MAX]; ++ wchar_t wc, pwc; ++ mbstate_t state_mb, state_wc; ++ ++ lc_time = setlocale (LC_TIME, ""); ++ if (lc_time) ++ lc_time = xstrdup (lc_time); ++ ++ lc_ctype = setlocale (LC_CTYPE, ""); ++ if (lc_ctype) ++ lc_ctype = xstrdup (lc_ctype); ++ ++ if (lc_time && lc_ctype) ++ /* temporarily set LC_CTYPE to match LC_TIME, so that we can convert ++ * the names of months to upper case */ ++ setlocale (LC_CTYPE, lc_time); ++ ++ for (i = 0; i < MONTHS_PER_YEAR; i++) ++ { ++ s = (char *) nl_langinfo (ABMON_1 + i); ++ s_len = strlen (s); ++ monthtab[i].name = name = (char *) xmalloc (s_len + 1); ++ monthtab[i].val = i + 1; ++ ++ memset (&state_mb, '\0', sizeof (mbstate_t)); ++ memset (&state_wc, '\0', sizeof (mbstate_t)); ++ ++ for (j = 0; j < s_len;) ++ { ++ if (!ismbblank (s + j, s_len - j, &mblength)) ++ break; ++ j += mblength; ++ } ++ ++ for (k = 0; j < s_len;) ++ { ++ mblength = mbrtowc (&wc, (s + j), (s_len - j), &state_mb); ++ assert (mblength != (size_t)-1 && mblength != (size_t)-2); ++ if (mblength == 0) ++ break; ++ ++ pwc = towupper (wc); ++ if (pwc == wc) ++ { ++ memcpy (mbc, s + j, mblength); ++ j += mblength; ++ } ++ else ++ { ++ j += mblength; ++ mblength = wcrtomb (mbc, pwc, &state_wc); ++ assert (mblength != (size_t)0 && mblength != (size_t)-1); ++ } ++ ++ for (l = 0; l < mblength; l++) ++ name[k++] = mbc[l]; ++ } ++ name[k] = '\0'; ++ } ++ qsort ((void *) monthtab, MONTHS_PER_YEAR, ++ sizeof (struct month), struct_month_cmp); ++ ++ if (lc_time && lc_ctype) ++ /* restore the original locales */ ++ setlocale (LC_CTYPE, lc_ctype); ++ ++ free (lc_ctype); ++ free (lc_time); ++} ++#endif ++ + /* Specify the amount of main memory to use when sorting. */ + static void + specify_sort_size (int oi, char c, char const *s) +@@ -1596,7 +1745,7 @@ buffer_linelim (struct buffer const *buf + by KEY in LINE. */ + + static char * +-begfield (struct line const *line, struct keyfield const *key) ++begfield_uni (const struct line *line, const struct keyfield *key) + { + char *ptr = line->text, *lim = ptr + line->length - 1; + size_t sword = key->sword; +@@ -1605,10 +1754,10 @@ begfield (struct line const *line, struc + /* The leading field separator itself is included in a field when -t + is absent. */ + +- if (tab != TAB_DEFAULT) ++ if (tab_length) + while (ptr < lim && sword--) + { +- while (ptr < lim && *ptr != tab) ++ while (ptr < lim && *ptr != tab[0]) + ++ptr; + if (ptr < lim) + ++ptr; +@@ -1634,11 +1783,70 @@ begfield (struct line const *line, struc + return ptr; + } + ++#if HAVE_MBRTOWC ++static char * ++begfield_mb (const struct line *line, const struct keyfield *key) ++{ ++ int i; ++ char *ptr = line->text, *lim = ptr + line->length - 1; ++ size_t sword = key->sword; ++ size_t schar = key->schar; ++ size_t mblength; ++ mbstate_t state; ++ ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ if (tab_length) ++ while (ptr < lim && sword--) ++ { ++ while (ptr < lim && memcmp (ptr, tab, tab_length) != 0) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ if (ptr < lim) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ } ++ else ++ while (ptr < lim && sword--) ++ { ++ while (ptr < lim && ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ if (ptr < lim) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ while (ptr < lim && !ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ } ++ ++ if (key->skipsblanks) ++ while (ptr < lim && ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ ++ for (i = 0; i < schar; i++) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ++ if (ptr + mblength > lim) ++ break; ++ else ++ ptr += mblength; ++ } ++ ++ return ptr; ++} ++#endif ++ + /* Return the limit of (a pointer to the first character after) the field + in LINE specified by KEY. */ + + static char * +-limfield (struct line const *line, struct keyfield const *key) ++limfield_uni (const struct line *line, const struct keyfield *key) + { + char *ptr = line->text, *lim = ptr + line->length - 1; + size_t eword = key->eword, echar = key->echar; +@@ -1653,10 +1861,10 @@ limfield (struct line const *line, struc + 'beginning' is the first character following the delimiting TAB. + Otherwise, leave PTR pointing at the first 'blank' character after + the preceding field. */ +- if (tab != TAB_DEFAULT) ++ if (tab_length) + while (ptr < lim && eword--) + { +- while (ptr < lim && *ptr != tab) ++ while (ptr < lim && *ptr != tab[0]) + ++ptr; + if (ptr < lim && (eword || echar)) + ++ptr; +@@ -1702,10 +1910,10 @@ limfield (struct line const *line, struc + */ + + /* Make LIM point to the end of (one byte past) the current field. */ +- if (tab != TAB_DEFAULT) ++ if (tab_length) + { + char *newlim; +- newlim = memchr (ptr, tab, lim - ptr); ++ newlim = memchr (ptr, tab[0], lim - ptr); + if (newlim) + lim = newlim; + } +@@ -1736,6 +1944,130 @@ limfield (struct line const *line, struc + return ptr; + } + ++#if HAVE_MBRTOWC ++static char * ++limfield_mb (const struct line *line, const struct keyfield *key) ++{ ++ char *ptr = line->text, *lim = ptr + line->length - 1; ++ size_t eword = key->eword, echar = key->echar; ++ int i; ++ size_t mblength; ++ mbstate_t state; ++ ++ if (echar == 0) ++ eword++; /* skip all of end field. */ ++ ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ if (tab_length) ++ while (ptr < lim && eword--) ++ { ++ while (ptr < lim && memcmp (ptr, tab, tab_length) != 0) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ if (ptr < lim && (eword | echar)) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ } ++ else ++ while (ptr < lim && eword--) ++ { ++ while (ptr < lim && ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ if (ptr < lim) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ while (ptr < lim && !ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ } ++ ++ ++# ifdef POSIX_UNSPECIFIED ++ /* Make LIM point to the end of (one byte past) the current field. */ ++ if (tab_length) ++ { ++ char *newlim, *p; ++ ++ newlim = NULL; ++ for (p = ptr; p < lim;) ++ { ++ if (memcmp (p, tab, tab_length) == 0) ++ { ++ newlim = p; ++ break; ++ } ++ ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ p += mblength; ++ } ++ } ++ else ++ { ++ char *newlim; ++ newlim = ptr; ++ ++ while (newlim < lim && ismbblank (newlim, lim - newlim, &mblength)) ++ newlim += mblength; ++ if (ptr < lim) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ptr += mblength; ++ } ++ while (newlim < lim && !ismbblank (newlim, lim - newlim, &mblength)) ++ newlim += mblength; ++ lim = newlim; ++ } ++# endif ++ ++ if (echar != 0) ++ { ++ /* If we're skipping leading blanks, don't start counting characters ++ * until after skipping past any leading blanks. */ ++ if (key->skipeblanks) ++ while (ptr < lim && ismbblank (ptr, lim - ptr, &mblength)) ++ ptr += mblength; ++ ++ memset (&state, '\0', sizeof(mbstate_t)); ++ ++ /* Advance PTR by ECHAR (if possible), but no further than LIM. */ ++ for (i = 0; i < echar; i++) ++ { ++ GET_BYTELEN_OF_CHAR (lim, ptr, mblength, state); ++ ++ if (ptr + mblength > lim) ++ break; ++ else ++ ptr += mblength; ++ } ++ } ++ ++ return ptr; ++} ++#endif ++ ++static void ++skipblanks_uni (char **ptr, char *lim) ++{ ++ while (*ptr < lim && blanks[to_uchar (**ptr)]) ++ ++(*ptr); ++} ++ ++#if HAVE_MBRTOWC ++static void ++skipblanks_mb (char **ptr, char *lim) ++{ ++ size_t mblength; ++ while (*ptr < lim && ismbblank (*ptr, lim - *ptr, &mblength)) ++ (*ptr) += mblength; ++} ++#endif ++ + /* Fill BUF reading from FP, moving buf->left bytes from the end + of buf->buf to the beginning first. If EOF is reached and the + file wasn't terminated by a newline, supply one. Set up BUF's line +@@ -1822,8 +2154,22 @@ fillbuf (struct buffer *buf, FILE *fp, c + else + { + if (key->skipsblanks) +- while (blanks[to_uchar (*line_start)]) +- line_start++; ++ { ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ size_t mblength; ++ while (line_start < line->keylim && ++ ismbblank (line_start, ++ line->keylim - line_start, ++ &mblength)) ++ line_start += mblength; ++ } ++ else ++#endif ++ while (blanks[to_uchar (*line_start)]) ++ line_start++; ++ } + line->keybeg = line_start; + } + } +@@ -1944,7 +2290,7 @@ human_numcompare (char const *a, char co + hideously fast. */ + + static int +-numcompare (char const *a, char const *b) ++numcompare_uni (const char *a, const char *b) + { + while (blanks[to_uchar (*a)]) + a++; +@@ -1954,6 +2300,25 @@ numcompare (char const *a, char const *b + return strnumcmp (a, b, decimal_point, thousands_sep); + } + ++#if HAVE_MBRTOWC ++static int ++numcompare_mb (const char *a, const char *b) ++{ ++ size_t mblength, len; ++ len = strlen (a); /* okay for UTF-8 */ ++ while (*a && ismbblank (a, len > MB_CUR_MAX ? MB_CUR_MAX : len, &mblength)) ++ { ++ a += mblength; ++ len -= mblength; ++ } ++ len = strlen (b); /* okay for UTF-8 */ ++ while (*b && ismbblank (b, len > MB_CUR_MAX ? MB_CUR_MAX : len, &mblength)) ++ b += mblength; ++ ++ return strnumcmp (a, b, decimal_point, thousands_sep); ++} ++#endif /* HAV_EMBRTOWC */ ++ + /* Work around a problem whereby the long double value returned by glibc's + strtold ("NaN", ...) contains uninitialized bits: clear all bytes of + A and B before calling strtold. FIXME: remove this function once +@@ -2004,7 +2369,7 @@ general_numcompare (char const *sa, char + Return 0 if the name in S is not recognized. */ + + static int +-getmonth (char const *month, char **ea) ++getmonth_uni (char const *month, size_t len, char **ea) + { + size_t lo = 0; + size_t hi = MONTHS_PER_YEAR; +@@ -2280,15 +2645,14 @@ debug_key (struct line const *line, stru + char saved = *lim; + *lim = '\0'; + +- while (blanks[to_uchar (*beg)]) +- beg++; ++ skipblanks (&beg, lim); + + char *tighter_lim = beg; + + if (lim < beg) + tighter_lim = lim; + else if (key->month) +- getmonth (beg, &tighter_lim); ++ getmonth (beg, lim-beg, &tighter_lim); + else if (key->general_numeric) + ignore_value (strtold (beg, &tighter_lim)); + else if (key->numeric || key->human_numeric) +@@ -2432,7 +2796,7 @@ key_warnings (struct keyfield const *gke + bool maybe_space_aligned = !hard_LC_COLLATE && default_key_compare (key) + && !(key->schar || key->echar); + bool line_offset = key->eword == 0 && key->echar != 0; /* -k1.x,1.y */ +- if (!gkey_only && tab == TAB_DEFAULT && !line_offset ++ if (!gkey_only && !tab_length && !line_offset + && ((!key->skipsblanks && !(implicit_skip || maybe_space_aligned)) + || (!key->skipsblanks && key->schar) + || (!key->skipeblanks && key->echar))) +@@ -2490,11 +2854,87 @@ key_warnings (struct keyfield const *gke + error (0, 0, _("option '-r' only applies to last-resort comparison")); + } + ++#if HAVE_MBRTOWC ++static int ++getmonth_mb (const char *s, size_t len, char **ea) ++{ ++ char *month; ++ register size_t i; ++ register int lo = 0, hi = MONTHS_PER_YEAR, result; ++ char *tmp; ++ size_t wclength, mblength; ++ const char *pp; ++ const wchar_t *wpp; ++ wchar_t *month_wcs; ++ mbstate_t state; ++ ++ while (len > 0 && ismbblank (s, len, &mblength)) ++ { ++ s += mblength; ++ len -= mblength; ++ } ++ ++ if (len == 0) ++ return 0; ++ ++ if (SIZE_MAX - len < 1) ++ xalloc_die (); ++ ++ month = (char *) xnmalloc (len + 1, MB_CUR_MAX); ++ ++ pp = tmp = (char *) xnmalloc (len + 1, MB_CUR_MAX); ++ memcpy (tmp, s, len); ++ tmp[len] = '\0'; ++ wpp = month_wcs = (wchar_t *) xnmalloc (len + 1, sizeof (wchar_t)); ++ memset (&state, '\0', sizeof (mbstate_t)); ++ ++ wclength = mbsrtowcs (month_wcs, &pp, len + 1, &state); ++ if (wclength == (size_t)-1 || pp != NULL) ++ error (SORT_FAILURE, 0, _("Invalid multibyte input %s."), quote(s)); ++ ++ for (i = 0; i < wclength; i++) ++ { ++ month_wcs[i] = towupper(month_wcs[i]); ++ if (iswblank (month_wcs[i])) ++ { ++ month_wcs[i] = L'\0'; ++ break; ++ } ++ } ++ ++ mblength = wcsrtombs (month, &wpp, (len + 1) * MB_CUR_MAX, &state); ++ assert (mblength != (-1) && wpp == NULL); ++ ++ do ++ { ++ int ix = (lo + hi) / 2; ++ ++ if (strncmp (month, monthtab[ix].name, strlen (monthtab[ix].name)) < 0) ++ hi = ix; ++ else ++ lo = ix; ++ } ++ while (hi - lo > 1); ++ ++ result = (!strncmp (month, monthtab[lo].name, strlen (monthtab[lo].name)) ++ ? monthtab[lo].val : 0); ++ ++ if (ea && result) ++ *ea = (char*) s + strlen (monthtab[lo].name); ++ ++ free (month); ++ free (tmp); ++ free (month_wcs); ++ ++ return result; ++} ++#endif ++ + /* Compare two lines A and B trying every key in sequence until there + are no more keys or a difference is found. */ + + static int +-keycompare (struct line const *a, struct line const *b) ++keycompare_uni (const struct line *a, const struct line *b) + { + struct keyfield *key = keylist; + +@@ -2579,7 +3019,7 @@ keycompare (struct line const *a, struct + else if (key->human_numeric) + diff = human_numcompare (ta, tb); + else if (key->month) +- diff = getmonth (ta, NULL) - getmonth (tb, NULL); ++ diff = getmonth (ta, tlena, NULL) - getmonth (tb, tlenb, NULL); + else if (key->random) + diff = compare_random (ta, tlena, tb, tlenb); + else if (key->version) +@@ -2695,6 +3135,211 @@ keycompare (struct line const *a, struct + return key->reverse ? -diff : diff; + } + ++#if HAVE_MBRTOWC ++static int ++keycompare_mb (const struct line *a, const struct line *b) ++{ ++ struct keyfield *key = keylist; ++ ++ /* For the first iteration only, the key positions have been ++ precomputed for us. */ ++ char *texta = a->keybeg; ++ char *textb = b->keybeg; ++ char *lima = a->keylim; ++ char *limb = b->keylim; ++ ++ size_t mblength_a, mblength_b; ++ wchar_t wc_a, wc_b; ++ mbstate_t state_a, state_b; ++ ++ int diff = 0; ++ ++ memset (&state_a, '\0', sizeof(mbstate_t)); ++ memset (&state_b, '\0', sizeof(mbstate_t)); ++ /* Ignore keys with start after end. */ ++ if (a->keybeg - a->keylim > 0) ++ return 0; ++ ++ ++ /* Ignore and/or translate chars before comparing. */ ++# define IGNORE_CHARS(NEW_LEN, LEN, TEXT, COPY, WC, MBLENGTH, STATE) \ ++ do \ ++ { \ ++ wchar_t uwc; \ ++ char mbc[MB_LEN_MAX]; \ ++ mbstate_t state_wc; \ ++ \ ++ for (NEW_LEN = i = 0; i < LEN;) \ ++ { \ ++ mbstate_t state_bak; \ ++ \ ++ state_bak = STATE; \ ++ MBLENGTH = mbrtowc (&WC, TEXT + i, LEN - i, &STATE); \ ++ \ ++ if (MBLENGTH == (size_t)-2 || MBLENGTH == (size_t)-1 \ ++ || MBLENGTH == 0) \ ++ { \ ++ if (MBLENGTH == (size_t)-2 || MBLENGTH == (size_t)-1) \ ++ STATE = state_bak; \ ++ if (!ignore) \ ++ COPY[NEW_LEN++] = TEXT[i]; \ ++ i++; \ ++ continue; \ ++ } \ ++ \ ++ if (ignore) \ ++ { \ ++ if ((ignore == nonprinting && !iswprint (WC)) \ ++ || (ignore == nondictionary \ ++ && !iswalnum (WC) && !iswblank (WC))) \ ++ { \ ++ i += MBLENGTH; \ ++ continue; \ ++ } \ ++ } \ ++ \ ++ if (translate) \ ++ { \ ++ \ ++ uwc = towupper(WC); \ ++ if (WC == uwc) \ ++ { \ ++ memcpy (mbc, TEXT + i, MBLENGTH); \ ++ i += MBLENGTH; \ ++ } \ ++ else \ ++ { \ ++ i += MBLENGTH; \ ++ WC = uwc; \ ++ memset (&state_wc, '\0', sizeof (mbstate_t)); \ ++ \ ++ MBLENGTH = wcrtomb (mbc, WC, &state_wc); \ ++ assert (MBLENGTH != (size_t)-1 && MBLENGTH != 0); \ ++ } \ ++ \ ++ for (j = 0; j < MBLENGTH; j++) \ ++ COPY[NEW_LEN++] = mbc[j]; \ ++ } \ ++ else \ ++ for (j = 0; j < MBLENGTH; j++) \ ++ COPY[NEW_LEN++] = TEXT[i++]; \ ++ } \ ++ COPY[NEW_LEN] = '\0'; \ ++ } \ ++ while (0) ++ ++ /* Actually compare the fields. */ ++ ++ for (;;) ++ { ++ /* Find the lengths. */ ++ size_t lena = lima <= texta ? 0 : lima - texta; ++ size_t lenb = limb <= textb ? 0 : limb - textb; ++ ++ char enda IF_LINT (= 0); ++ char endb IF_LINT (= 0); ++ ++ char const *translate = key->translate; ++ bool const *ignore = key->ignore; ++ ++ if (ignore || translate) ++ { ++ if (SIZE_MAX - lenb - 2 < lena) ++ xalloc_die (); ++ char *copy_a = (char *) xnmalloc (lena + lenb + 2, MB_CUR_MAX); ++ char *copy_b = copy_a + lena * MB_CUR_MAX + 1; ++ size_t new_len_a, new_len_b; ++ size_t i, j; ++ ++ IGNORE_CHARS (new_len_a, lena, texta, copy_a, ++ wc_a, mblength_a, state_a); ++ IGNORE_CHARS (new_len_b, lenb, textb, copy_b, ++ wc_b, mblength_b, state_b); ++ texta = copy_a; textb = copy_b; ++ lena = new_len_a; lenb = new_len_b; ++ } ++ else ++ { ++ /* Use the keys in-place, temporarily null-terminated. */ ++ enda = texta[lena]; texta[lena] = '\0'; ++ endb = textb[lenb]; textb[lenb] = '\0'; ++ } ++ ++ if (key->random) ++ diff = compare_random (texta, lena, textb, lenb); ++ else if (key->numeric | key->general_numeric | key->human_numeric) ++ { ++ char savea = *lima, saveb = *limb; ++ ++ *lima = *limb = '\0'; ++ diff = (key->numeric ? numcompare (texta, textb) ++ : key->general_numeric ? general_numcompare (texta, textb) ++ : human_numcompare (texta, textb)); ++ *lima = savea, *limb = saveb; ++ } ++ else if (key->version) ++ diff = filevercmp (texta, textb); ++ else if (key->month) ++ diff = getmonth (texta, lena, NULL) - getmonth (textb, lenb, NULL); ++ else if (lena == 0) ++ diff = - NONZERO (lenb); ++ else if (lenb == 0) ++ diff = 1; ++ else if (hard_LC_COLLATE && !folding) ++ { ++ diff = xmemcoll0 (texta, lena + 1, textb, lenb + 1); ++ } ++ else ++ { ++ diff = memcmp (texta, textb, MIN (lena, lenb)); ++ if (diff == 0) ++ diff = lena < lenb ? -1 : lena != lenb; ++ } ++ ++ if (ignore || translate) ++ free (texta); ++ else ++ { ++ texta[lena] = enda; ++ textb[lenb] = endb; ++ } ++ ++ if (diff) ++ goto not_equal; ++ ++ key = key->next; ++ if (! key) ++ break; ++ ++ /* Find the beginning and limit of the next field. */ ++ if (key->eword != -1) ++ lima = limfield (a, key), limb = limfield (b, key); ++ else ++ lima = a->text + a->length - 1, limb = b->text + b->length - 1; ++ ++ if (key->sword != -1) ++ texta = begfield (a, key), textb = begfield (b, key); ++ else ++ { ++ texta = a->text, textb = b->text; ++ if (key->skipsblanks) ++ { ++ while (texta < lima && ismbblank (texta, lima - texta, &mblength_a)) ++ texta += mblength_a; ++ while (textb < limb && ismbblank (textb, limb - textb, &mblength_b)) ++ textb += mblength_b; ++ } ++ } ++ } ++ ++not_equal: ++ if (key && key->reverse) ++ return -diff; ++ else ++ return diff; ++} ++#endif ++ + /* Compare two lines A and B, returning negative, zero, or positive + depending on whether A compares less than, equal to, or greater than B. */ + +@@ -2722,7 +3367,7 @@ compare (struct line const *a, struct li + diff = - NONZERO (blen); + else if (blen == 0) + diff = 1; +- else if (hard_LC_COLLATE) ++ else if (hard_LC_COLLATE && !folding) + { + /* Note xmemcoll0 is a performance enhancement as + it will not unconditionally write '\0' after the +@@ -4121,6 +4766,7 @@ set_ordering (char const *s, struct keyf + break; + case 'f': + key->translate = fold_toupper; ++ folding = true; + break; + case 'g': + key->general_numeric = true; +@@ -4199,7 +4845,7 @@ main (int argc, char **argv) + initialize_exit_failure (SORT_FAILURE); + + hard_LC_COLLATE = hard_locale (LC_COLLATE); +-#if HAVE_NL_LANGINFO ++#if HAVE_LANGINFO_CODESET + hard_LC_TIME = hard_locale (LC_TIME); + #endif + +@@ -4220,6 +4866,29 @@ main (int argc, char **argv) + thousands_sep = -1; + } + ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ inittables = inittables_mb; ++ begfield = begfield_mb; ++ limfield = limfield_mb; ++ skipblanks = skipblanks_mb; ++ getmonth = getmonth_mb; ++ keycompare = keycompare_mb; ++ numcompare = numcompare_mb; ++ } ++ else ++#endif ++ { ++ inittables = inittables_uni; ++ begfield = begfield_uni; ++ limfield = limfield_uni; ++ skipblanks = skipblanks_uni; ++ getmonth = getmonth_uni; ++ keycompare = keycompare_uni; ++ numcompare = numcompare_uni; ++ } ++ + have_read_stdin = false; + inittables (); + +@@ -4494,13 +5163,34 @@ main (int argc, char **argv) + + case 't': + { +- char newtab = optarg[0]; +- if (! newtab) ++ char newtab[MB_LEN_MAX + 1]; ++ size_t newtab_length = 1; ++ strncpy (newtab, optarg, MB_LEN_MAX); ++ if (! newtab[0]) + error (SORT_FAILURE, 0, _("empty tab")); +- if (optarg[1]) ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ wchar_t wc; ++ mbstate_t state; ++ ++ memset (&state, '\0', sizeof (mbstate_t)); ++ newtab_length = mbrtowc (&wc, newtab, strnlen (newtab, ++ MB_LEN_MAX), ++ &state); ++ switch (newtab_length) ++ { ++ case (size_t) -1: ++ case (size_t) -2: ++ case 0: ++ newtab_length = 1; ++ } ++ } ++#endif ++ if (newtab_length == 1 && optarg[1]) + { + if (STREQ (optarg, "\0")) +- newtab = '\0'; ++ newtab[0] = '\0'; + else + { + /* Provoke with 'sort -txx'. Complain about +@@ -4511,9 +5201,12 @@ main (int argc, char **argv) + quote (optarg)); + } + } +- if (tab != TAB_DEFAULT && tab != newtab) ++ if (tab_length ++ && (tab_length != newtab_length ++ || memcmp (tab, newtab, tab_length) != 0)) + error (SORT_FAILURE, 0, _("incompatible tabs")); +- tab = newtab; ++ memcpy (tab, newtab, newtab_length); ++ tab_length = newtab_length; + } + break; + +@@ -4751,12 +5444,10 @@ main (int argc, char **argv) + sort (files, nfiles, outfile, nthreads); + } + +-#ifdef lint + if (files_from) + readtokens0_free (&tok); + else + free (files); +-#endif + + if (have_read_stdin && fclose (stdin) == EOF) + die (_("close failed"), "-"); +diff -Naurp coreutils-8.25-orig/src/unexpand.c coreutils-8.25/src/unexpand.c +--- coreutils-8.25-orig/src/unexpand.c 2016-01-01 07:48:50.000000000 -0600 ++++ coreutils-8.25/src/unexpand.c 2016-02-08 19:07:10.311944651 -0600 +@@ -38,12 +38,29 @@ + #include <stdio.h> + #include <getopt.h> + #include <sys/types.h> ++ ++/* Get mbstate_t, mbrtowc(), wcwidth(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ + #include "system.h" + #include "error.h" + #include "fadvise.h" + #include "quote.h" + #include "xstrndup.h" + ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX < 2 ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif ++ + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "unexpand" + +@@ -103,6 +120,210 @@ static struct option const longopts[] = + {NULL, 0, NULL, 0} + }; + ++static FILE *next_file (FILE *fp); ++ ++#if HAVE_MBRTOWC ++static void ++unexpand_multibyte (void) ++{ ++ FILE *fp; /* Input stream. */ ++ mbstate_t i_state; /* Current shift state of the input stream. */ ++ mbstate_t i_state_bak; /* Back up the I_STATE. */ ++ mbstate_t o_state; /* Current shift state of the output stream. */ ++ char buf[MB_LEN_MAX + BUFSIZ]; /* For spooling a read byte sequence. */ ++ char *bufpos = buf; /* Next read position of BUF. */ ++ size_t buflen = 0; /* The length of the byte sequence in buf. */ ++ wint_t wc; /* A gotten wide character. */ ++ size_t mblength; /* The byte size of a multibyte character ++ which shows as same character as WC. */ ++ bool prev_tab = false; ++ ++ /* Index in `tab_list' of next tabstop: */ ++ int tab_index = 0; /* For calculating width of pending tabs. */ ++ int print_tab_index = 0; /* For printing as many tabs as possible. */ ++ unsigned int column = 0; /* Column on screen of next char. */ ++ int next_tab_column; /* Column the next tab stop is on. */ ++ int convert = 1; /* If nonzero, perform translations. */ ++ unsigned int pending = 0; /* Pending columns of blanks. */ ++ ++ fp = next_file ((FILE *) NULL); ++ if (fp == NULL) ++ return; ++ ++ memset (&o_state, '\0', sizeof(mbstate_t)); ++ memset (&i_state, '\0', sizeof(mbstate_t)); ++ ++ for (;;) ++ { ++ if (buflen < MB_LEN_MAX && !feof(fp) && !ferror(fp)) ++ { ++ memmove (buf, bufpos, buflen); ++ buflen += fread (buf + buflen, sizeof(char), BUFSIZ, fp); ++ bufpos = buf; ++ } ++ ++ /* Get a wide character. */ ++ if (buflen < 1) ++ { ++ mblength = 1; ++ wc = WEOF; ++ } ++ else ++ { ++ i_state_bak = i_state; ++ mblength = mbrtowc ((wchar_t *)&wc, bufpos, buflen, &i_state); ++ } ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ i_state = i_state_bak; ++ wc = L'\0'; ++ } ++ ++ if (wc == L' ' && convert && column < INT_MAX) ++ { ++ ++pending; ++ ++column; ++ } ++ else if (wc == L'\t' && convert) ++ { ++ if (tab_size == 0) ++ { ++ /* Do not let tab_index == first_free_tab; ++ stop when it is 1 less. */ ++ while (tab_index < first_free_tab - 1 ++ && column >= tab_list[tab_index]) ++ tab_index++; ++ next_tab_column = tab_list[tab_index]; ++ if (tab_index < first_free_tab - 1) ++ tab_index++; ++ if (column >= next_tab_column) ++ { ++ convert = 0; /* Ran out of tab stops. */ ++ goto flush_pend_mb; ++ } ++ } ++ else ++ { ++ next_tab_column = column + tab_size - column % tab_size; ++ } ++ pending += next_tab_column - column; ++ column = next_tab_column; ++ } ++ else ++ { ++flush_pend_mb: ++ /* Flush pending spaces. Print as many tabs as possible, ++ then print the rest as spaces. */ ++ if (pending == 1 && column != 1 && !prev_tab) ++ { ++ putchar (' '); ++ pending = 0; ++ } ++ column -= pending; ++ while (pending > 0) ++ { ++ if (tab_size == 0) ++ { ++ /* Do not let print_tab_index == first_free_tab; ++ stop when it is 1 less. */ ++ while (print_tab_index < first_free_tab - 1 ++ && column >= tab_list[print_tab_index]) ++ print_tab_index++; ++ next_tab_column = tab_list[print_tab_index]; ++ if (print_tab_index < first_free_tab - 1) ++ print_tab_index++; ++ } ++ else ++ { ++ next_tab_column = ++ column + tab_size - column % tab_size; ++ } ++ if (next_tab_column - column <= pending) ++ { ++ putchar ('\t'); ++ pending -= next_tab_column - column; ++ column = next_tab_column; ++ } ++ else ++ { ++ --print_tab_index; ++ column += pending; ++ while (pending != 0) ++ { ++ putchar (' '); ++ pending--; ++ } ++ } ++ } ++ ++ if (wc == WEOF) ++ { ++ fp = next_file (fp); ++ if (fp == NULL) ++ break; /* No more files. */ ++ else ++ { ++ memset (&i_state, '\0', sizeof(mbstate_t)); ++ continue; ++ } ++ } ++ ++ if (mblength == (size_t)-1 || mblength == (size_t)-2) ++ { ++ if (convert) ++ { ++ ++column; ++ if (convert_entire_line == 0) ++ convert = 0; ++ } ++ mblength = 1; ++ putchar (buf[0]); ++ } ++ else if (mblength == 0) ++ { ++ if (convert && convert_entire_line == 0) ++ convert = 0; ++ mblength = 1; ++ putchar ('\0'); ++ } ++ else ++ { ++ if (convert) ++ { ++ if (wc == L'\b') ++ { ++ if (column > 0) ++ --column; ++ } ++ else ++ { ++ int width; /* The width of WC. */ ++ ++ width = wcwidth (wc); ++ column += (width > 0) ? width : 0; ++ if (convert_entire_line == 0) ++ convert = 0; ++ } ++ } ++ ++ if (wc == L'\n') ++ { ++ tab_index = print_tab_index = 0; ++ column = pending = 0; ++ convert = 1; ++ } ++ fwrite (bufpos, sizeof(char), mblength, stdout); ++ } ++ } ++ prev_tab = wc == L'\t'; ++ buflen -= mblength; ++ bufpos += mblength; ++ } ++} ++#endif ++ ++ + void + usage (int status) + { +@@ -523,7 +744,12 @@ main (int argc, char **argv) + + file_list = (optind < argc ? &argv[optind] : stdin_argv); + +- unexpand (); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ unexpand_multibyte (); ++ else ++#endif ++ unexpand (); + + if (have_read_stdin && fclose (stdin) != 0) + error (EXIT_FAILURE, errno, "-"); +diff -Naurp coreutils-8.25-orig/src/uniq.c coreutils-8.25/src/uniq.c +--- coreutils-8.25-orig/src/uniq.c 2016-01-13 05:08:59.000000000 -0600 ++++ coreutils-8.25/src/uniq.c 2016-02-08 19:07:10.312944654 -0600 +@@ -21,6 +21,17 @@ + #include <getopt.h> + #include <sys/types.h> + ++/* Get mbstate_t, mbrtowc(). */ ++#if HAVE_WCHAR_H ++# include <wchar.h> ++#endif ++ ++/* Get isw* functions. */ ++#if HAVE_WCTYPE_H ++# include <wctype.h> ++#endif ++#include <assert.h> ++ + #include "system.h" + #include "argmatch.h" + #include "linebuffer.h" +@@ -33,6 +44,18 @@ + #include "xstrtol.h" + #include "memcasecmp.h" + #include "quote.h" ++#include "xmemcoll.h" ++ ++/* MB_LEN_MAX is incorrectly defined to be 1 in at least one GCC ++ installation; work around this configuration error. */ ++#if !defined MB_LEN_MAX || MB_LEN_MAX < 2 ++# define MB_LEN_MAX 16 ++#endif ++ ++/* Some systems, like BeOS, have multibyte encodings but lack mbstate_t. */ ++#if HAVE_MBRTOWC && defined mbstate_t ++# define mbrtowc(pwc, s, n, ps) (mbrtowc) (pwc, s, n, 0) ++#endif + + /* The official name of this program (e.g., no 'g' prefix). */ + #define PROGRAM_NAME "uniq" +@@ -143,6 +166,10 @@ enum + GROUP_OPTION = CHAR_MAX + 1 + }; + ++/* Function pointers. */ ++static char * ++(*find_field) (struct linebuffer *line); ++ + static struct option const longopts[] = + { + {"count", no_argument, NULL, 'c'}, +@@ -252,7 +279,7 @@ size_opt (char const *opt, char const *m + return a pointer to the beginning of the line's field to be compared. */ + + static char * _GL_ATTRIBUTE_PURE +-find_field (struct linebuffer const *line) ++find_field_uni (struct linebuffer *line) + { + size_t count; + char const *lp = line->buffer; +@@ -272,6 +299,83 @@ find_field (struct linebuffer const *lin + return line->buffer + i; + } + ++#if HAVE_MBRTOWC ++ ++# define MBCHAR_TO_WCHAR(WC, MBLENGTH, LP, POS, SIZE, STATEP, CONVFAIL) \ ++ do \ ++ { \ ++ mbstate_t state_bak; \ ++ \ ++ CONVFAIL = 0; \ ++ state_bak = *STATEP; \ ++ \ ++ MBLENGTH = mbrtowc (&WC, LP + POS, SIZE - POS, STATEP); \ ++ \ ++ switch (MBLENGTH) \ ++ { \ ++ case (size_t)-2: \ ++ case (size_t)-1: \ ++ *STATEP = state_bak; \ ++ CONVFAIL++; \ ++ /* Fall through */ \ ++ case 0: \ ++ MBLENGTH = 1; \ ++ } \ ++ } \ ++ while (0) ++ ++static char * ++find_field_multi (struct linebuffer *line) ++{ ++ size_t count; ++ char *lp = line->buffer; ++ size_t size = line->length - 1; ++ size_t pos; ++ size_t mblength; ++ wchar_t wc; ++ mbstate_t *statep; ++ int convfail = 0; ++ ++ pos = 0; ++ statep = &(line->state); ++ ++ /* skip fields. */ ++ for (count = 0; count < skip_fields && pos < size; count++) ++ { ++ while (pos < size) ++ { ++ MBCHAR_TO_WCHAR (wc, mblength, lp, pos, size, statep, convfail); ++ ++ if (convfail || !(iswblank (wc) || wc == '\n')) ++ { ++ pos += mblength; ++ break; ++ } ++ pos += mblength; ++ } ++ ++ while (pos < size) ++ { ++ MBCHAR_TO_WCHAR (wc, mblength, lp, pos, size, statep, convfail); ++ ++ if (!convfail && (iswblank (wc) || wc == '\n')) ++ break; ++ ++ pos += mblength; ++ } ++ } ++ ++ /* skip fields. */ ++ for (count = 0; count < skip_chars && pos < size; count++) ++ { ++ MBCHAR_TO_WCHAR (wc, mblength, lp, pos, size, statep, convfail); ++ pos += mblength; ++ } ++ ++ return lp + pos; ++} ++#endif ++ + /* Return false if two strings OLD and NEW match, true if not. + OLD and NEW point not to the beginnings of the lines + but rather to the beginnings of the fields to compare. +@@ -280,6 +384,8 @@ find_field (struct linebuffer const *lin + static bool + different (char *old, char *new, size_t oldlen, size_t newlen) + { ++ char *copy_old, *copy_new; ++ + if (check_chars < oldlen) + oldlen = check_chars; + if (check_chars < newlen) +@@ -287,15 +393,104 @@ different (char *old, char *new, size_t + + if (ignore_case) + { +- /* FIXME: This should invoke strcoll somehow. */ +- return oldlen != newlen || memcasecmp (old, new, oldlen); ++ size_t i; ++ ++ copy_old = xmalloc (oldlen + 1); ++ copy_new = xmalloc (oldlen + 1); ++ ++ for (i = 0; i < oldlen; i++) ++ { ++ copy_old[i] = toupper (old[i]); ++ copy_new[i] = toupper (new[i]); ++ } ++ bool rc = xmemcoll (copy_old, oldlen, copy_new, newlen); ++ free (copy_old); ++ free (copy_new); ++ return rc; + } +- else if (hard_LC_COLLATE) +- return xmemcoll (old, oldlen, new, newlen) != 0; + else +- return oldlen != newlen || memcmp (old, new, oldlen); ++ { ++ copy_old = (char *)old; ++ copy_new = (char *)new; ++ } ++ ++ return xmemcoll (copy_old, oldlen, copy_new, newlen); ++ + } + ++#if HAVE_MBRTOWC ++static int ++different_multi (const char *old, const char *new, size_t oldlen, size_t newlen, mbstate_t oldstate, mbstate_t newstate) ++{ ++ size_t i, j, chars; ++ const char *str[2]; ++ char *copy[2]; ++ size_t len[2]; ++ mbstate_t state[2]; ++ size_t mblength; ++ wchar_t wc, uwc; ++ mbstate_t state_bak; ++ ++ str[0] = old; ++ str[1] = new; ++ len[0] = oldlen; ++ len[1] = newlen; ++ state[0] = oldstate; ++ state[1] = newstate; ++ ++ for (i = 0; i < 2; i++) ++ { ++ copy[i] = xmalloc (len[i] + 1); ++ memset (copy[i], '\0', len[i] + 1); ++ ++ for (j = 0, chars = 0; j < len[i] && chars < check_chars; chars++) ++ { ++ state_bak = state[i]; ++ mblength = mbrtowc (&wc, str[i] + j, len[i] - j, &(state[i])); ++ ++ switch (mblength) ++ { ++ case (size_t)-1: ++ case (size_t)-2: ++ state[i] = state_bak; ++ /* Fall through */ ++ case 0: ++ mblength = 1; ++ break; ++ ++ default: ++ if (ignore_case) ++ { ++ uwc = towupper (wc); ++ ++ if (uwc != wc) ++ { ++ mbstate_t state_wc; ++ size_t mblen; ++ ++ memset (&state_wc, '\0', sizeof(mbstate_t)); ++ mblen = wcrtomb (copy[i] + j, uwc, &state_wc); ++ assert (mblen != (size_t)-1); ++ } ++ else ++ memcpy (copy[i] + j, str[i] + j, mblength); ++ } ++ else ++ memcpy (copy[i] + j, str[i] + j, mblength); ++ } ++ j += mblength; ++ } ++ copy[i][j] = '\0'; ++ len[i] = j; ++ } ++ int rc = xmemcoll (copy[0], len[0], copy[1], len[1]); ++ free (copy[0]); ++ free (copy[1]); ++ return rc; ++ ++} ++#endif ++ + /* Output the line in linebuffer LINE to standard output + provided that the switches say it should be output. + MATCH is true if the line matches the previous line. +@@ -359,19 +554,38 @@ check_file (const char *infile, const ch + char *prevfield IF_LINT ( = NULL); + size_t prevlen IF_LINT ( = 0); + bool first_group_printed = false; ++#if HAVE_MBRTOWC ++ mbstate_t prevstate; ++ ++ memset (&prevstate, '\0', sizeof (mbstate_t)); ++#endif + + while (!feof (stdin)) + { + char *thisfield; + size_t thislen; + bool new_group; ++#if HAVE_MBRTOWC ++ mbstate_t thisstate; ++#endif + + if (readlinebuffer_delim (thisline, stdin, delimiter) == 0) + break; + + thisfield = find_field (thisline); + thislen = thisline->length - 1 - (thisfield - thisline->buffer); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ thisstate = thisline->state; + ++ new_group = (prevline->length == 0 ++ || different_multi (thisfield, prevfield, ++ thislen, prevlen, ++ thisstate, prevstate)); ++ } ++ else ++#endif + new_group = (prevline->length == 0 + || different (thisfield, prevfield, thislen, prevlen)); + +@@ -389,6 +603,10 @@ check_file (const char *infile, const ch + SWAP_LINES (prevline, thisline); + prevfield = thisfield; + prevlen = thislen; ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ prevstate = thisstate; ++#endif + first_group_printed = true; + } + } +@@ -401,17 +619,26 @@ check_file (const char *infile, const ch + size_t prevlen; + uintmax_t match_count = 0; + bool first_delimiter = true; ++#if HAVE_MBRTOWC ++ mbstate_t prevstate; ++#endif + + if (readlinebuffer_delim (prevline, stdin, delimiter) == 0) + goto closefiles; + prevfield = find_field (prevline); + prevlen = prevline->length - 1 - (prevfield - prevline->buffer); ++#if HAVE_MBRTOWC ++ prevstate = prevline->state; ++#endif + + while (!feof (stdin)) + { + bool match; + char *thisfield; + size_t thislen; ++#if HAVE_MBRTOWC ++ mbstate_t thisstate = thisline->state; ++#endif + if (readlinebuffer_delim (thisline, stdin, delimiter) == 0) + { + if (ferror (stdin)) +@@ -420,6 +647,14 @@ check_file (const char *infile, const ch + } + thisfield = find_field (thisline); + thislen = thisline->length - 1 - (thisfield - thisline->buffer); ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ match = !different_multi (thisfield, prevfield, ++ thislen, prevlen, thisstate, prevstate); ++ } ++ else ++#endif + match = !different (thisfield, prevfield, thislen, prevlen); + match_count += match; + +@@ -452,6 +687,9 @@ check_file (const char *infile, const ch + SWAP_LINES (prevline, thisline); + prevfield = thisfield; + prevlen = thislen; ++#if HAVE_MBRTOWC ++ prevstate = thisstate; ++#endif + if (!match) + match_count = 0; + } +@@ -498,6 +736,19 @@ main (int argc, char **argv) + + atexit (close_stdout); + ++#if HAVE_MBRTOWC ++ if (MB_CUR_MAX > 1) ++ { ++ find_field = find_field_multi; ++ } ++ else ++#endif ++ { ++ find_field = find_field_uni; ++ } ++ ++ ++ + skip_chars = 0; + skip_fields = 0; + check_chars = SIZE_MAX; +diff -Naurp coreutils-8.25-orig/tests/i18n/sort-month.sh coreutils-8.25/tests/i18n/sort-month.sh +--- coreutils-8.25-orig/tests/i18n/sort-month.sh 1969-12-31 18:00:00.000000000 -0600 ++++ coreutils-8.25/tests/i18n/sort-month.sh 2016-02-08 19:07:10.312944654 -0600 +@@ -0,0 +1,34 @@ ++#!/bin/sh ++# Verify sort -M multi-byte support. ++ ++. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src ++print_ver_ sort ++require_valgrind_ ++ ++# Skip this test if some deallocations are ++# avoided at process end. ++grep '^#define lint 1' $CONFIG_HEADER > /dev/null || ++ skip_ 'Allocation checks only work reliably in "lint" mode' ++ ++export LC_ALL=en_US.UTF-8 ++locale -k LC_CTYPE | grep -q "charmap.*UTF-8" \ ++ || skip_ "No UTF-8 locale available" ++ ++# Note the use of ɑ here which expands to ++# a wider representation upon case conversion ++# which triggered an assertion in sort -M ++cat <<EOF > exp ++. ++ɑ ++EOF ++ ++ ++# check large mem leak with --month-sort ++# https://bugzilla.redhat.com/show_bug.cgi?id=1259942 ++valgrind --leak-check=full \ ++ --error-exitcode=1 --errors-for-leak-kinds=definite \ ++ sort -M < exp > out || fail=1 ++compare exp out || { fail=1; cat out; } ++ ++ ++Exit $fail +diff -Naurp coreutils-8.25-orig/tests/i18n/sort.sh coreutils-8.25/tests/i18n/sort.sh +--- coreutils-8.25-orig/tests/i18n/sort.sh 1969-12-31 18:00:00.000000000 -0600 ++++ coreutils-8.25/tests/i18n/sort.sh 2016-02-08 19:07:10.312944654 -0600 +@@ -0,0 +1,29 @@ ++#!/bin/sh ++# Verify sort's multi-byte support. ++ ++. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src ++print_ver_ sort ++ ++export LC_ALL=en_US.UTF-8 ++locale -k LC_CTYPE | grep -q "charmap.*UTF-8" \ ++ || skip_ "No UTF-8 locale available" ++ ++# Enable heap consistency checkng on older systems ++export MALLOC_CHECK_=2 ++ ++ ++# check buffer overflow issue due to ++# expanding multi-byte representation due to case conversion ++# https://bugzilla.suse.com/show_bug.cgi?id=928749 ++cat <<EOF > exp ++. ++ɑ ++EOF ++cat <<EOF | sort -f > out || fail=1 ++. ++ɑ ++EOF ++compare exp out || { fail=1; cat out; } ++ ++ ++Exit $fail +diff -Naurp coreutils-8.25-orig/tests/local.mk coreutils-8.25/tests/local.mk +--- coreutils-8.25-orig/tests/local.mk 2016-01-16 12:18:13.000000000 -0600 ++++ coreutils-8.25/tests/local.mk 2016-02-08 19:07:10.313944658 -0600 +@@ -344,6 +344,9 @@ all_tests = \ + tests/misc/sort-discrim.sh \ + tests/misc/sort-files0-from.pl \ + tests/misc/sort-float.sh \ ++ tests/misc/sort-mb-tests.sh \ ++ tests/i18n/sort.sh \ ++ tests/i18n/sort-month.sh \ + tests/misc/sort-merge.pl \ + tests/misc/sort-merge-fdlimit.sh \ + tests/misc/sort-month.sh \ +diff -Naurp coreutils-8.25-orig/tests/misc/cut.pl coreutils-8.25/tests/misc/cut.pl +--- coreutils-8.25-orig/tests/misc/cut.pl 2016-01-16 12:18:13.000000000 -0600 ++++ coreutils-8.25/tests/misc/cut.pl 2016-02-08 19:07:10.314944661 -0600 +@@ -23,9 +23,11 @@ use strict; + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + +-my $mb_locale = $ENV{LOCALE_FR_UTF8}; ++my $mb_locale; ++# uncommented enable multibyte paths ++$mb_locale = $ENV{LOCALE_FR_UTF8}; + ! defined $mb_locale || $mb_locale eq 'none' +- and $mb_locale = 'C'; ++ and $mb_locale = 'C'; + + my $prog = 'cut'; + my $try = "Try '$prog --help' for more information.\n"; +@@ -240,6 +242,7 @@ if ($mb_locale ne 'C') + my @new_t = @$t; + my $test_name = shift @new_t; + ++ next if ($test_name =~ "newline-[12][0-9]"); + push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; + } + push @Tests, @new; +diff -Naurp coreutils-8.25-orig/tests/misc/expand.pl coreutils-8.25/tests/misc/expand.pl +--- coreutils-8.25-orig/tests/misc/expand.pl 2016-01-16 12:18:13.000000000 -0600 ++++ coreutils-8.25/tests/misc/expand.pl 2016-02-08 19:07:10.314944661 -0600 +@@ -23,6 +23,15 @@ use strict; + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + ++#comment out next line to disable multibyte tests ++my $mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ ++my $prog = 'expand'; ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + my @Tests = + ( + ['t1', '--tabs=3', {IN=>"a\tb"}, {OUT=>"a b"}], +@@ -31,6 +40,37 @@ my @Tests = + ['i2', '--tabs=3 -i', {IN=>" \ta\tb"}, {OUT=>" a\tb"}], + ); + ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether expand is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ ++ ++@Tests = triple_test @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + +diff -Naurp coreutils-8.25-orig/tests/misc/fold.pl coreutils-8.25/tests/misc/fold.pl +--- coreutils-8.25-orig/tests/misc/fold.pl 2016-01-16 12:18:13.000000000 -0600 ++++ coreutils-8.25/tests/misc/fold.pl 2016-02-08 19:07:10.314944661 -0600 +@@ -20,9 +20,18 @@ use strict; + + (my $program_name = $0) =~ s|.*/||; + ++my $prog = 'fold'; ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + ++# uncommented to enable multibyte paths ++my $mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ + my @Tests = + ( + ['s1', '-w2 -s', {IN=>"a\t"}, {OUT=>"a\n\t"}], +@@ -31,9 +40,48 @@ my @Tests = + ['s4', '-w4 -s', {IN=>"abc ef\n"}, {OUT=>"abc \nef\n"}], + ); + ++# Add _POSIX2_VERSION=199209 to the environment of each test ++# that uses an old-style option like +1. ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether fold is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ ++@Tests = triple_test @Tests; ++ ++# Remember that triple_test creates from each test with exactly one "IN" ++# file two more tests (.p and .r suffix on name) corresponding to reading ++# input from a file and from a pipe. The pipe-reading test would fail ++# due to a race condition about 1 in 20 times. ++# Remove the IN_PIPE version of the "output-is-input" test above. ++# The others aren't susceptible because they have three inputs each. ++@Tests = grep {$_->[0] ne 'output-is-input.p'} @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + +-my $prog = 'fold'; + my $fail = run_tests ($program_name, $prog, @Tests, $save_temps, $verbose); + exit $fail; +diff -Naurp coreutils-8.25-orig/tests/misc/join.pl coreutils-8.25/tests/misc/join.pl +--- coreutils-8.25-orig/tests/misc/join.pl 2016-01-16 12:18:13.000000000 -0600 ++++ coreutils-8.25/tests/misc/join.pl 2016-02-08 19:07:10.315944664 -0600 +@@ -25,6 +25,15 @@ my $limits = getlimits (); + + my $prog = 'join'; + ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ ++my $mb_locale; ++#Comment out next line to disable multibyte tests ++$mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ + my $delim = chr 0247; + sub t_subst ($) + { +@@ -329,8 +338,49 @@ foreach my $t (@tv) + push @Tests, $new_ent; + } + ++# Add _POSIX2_VERSION=199209 to the environment of each test ++# that uses an old-style option like +1. ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether join is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ #Adjust the output some error messages including test_name for mb ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR}} ++ (@new_t)) ++ { ++ my $sub2 = {ERR_SUBST => "s/$test_name-mb/$test_name/"}; ++ push @new_t, $sub2; ++ push @$t, $sub2; ++ } ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ + @Tests = triple_test @Tests; + ++#skip invalid-j-mb test, it is failing because of the format ++@Tests = grep {$_->[0] ne 'invalid-j-mb'} @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + +diff -Naurp coreutils-8.25-orig/tests/misc/sort-mb-tests.sh coreutils-8.25/tests/misc/sort-mb-tests.sh +--- coreutils-8.25-orig/tests/misc/sort-mb-tests.sh 1969-12-31 18:00:00.000000000 -0600 ++++ coreutils-8.25/tests/misc/sort-mb-tests.sh 2016-02-08 19:07:10.315944664 -0600 +@@ -0,0 +1,45 @@ ++#!/bin/sh ++# Verify sort's multi-byte support. ++ ++. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src ++print_ver_ sort ++ ++export LC_ALL=en_US.UTF-8 ++locale -k LC_CTYPE | grep -q "charmap.*UTF-8" \ ++ || skip_ "No UTF-8 locale available" ++ ++ ++cat <<EOF > exp ++Banana＠5 ++Apple＠10 ++Citrus＠20 ++Cherry＠30 ++EOF ++ ++cat <<EOF | sort -t ＠ -k2 -n > out || fail=1 ++Apple＠10 ++Banana＠5 ++Citrus＠20 ++Cherry＠30 ++EOF ++ ++compare exp out || { fail=1; cat out; } ++ ++ ++cat <<EOF > exp ++Citrus＠ＡＡ20＠＠5 ++Cherry＠ＡＡ30＠＠10 ++Apple＠ＡＡ10＠＠20 ++Banana＠ＡＡ5＠＠30 ++EOF ++ ++cat <<EOF | sort -t ＠ -k4 -n > out || fail=1 ++Apple＠ＡＡ10＠＠20 ++Banana＠ＡＡ5＠＠30 ++Citrus＠ＡＡ20＠＠5 ++Cherry＠ＡＡ30＠＠10 ++EOF ++ ++compare exp out || { fail=1; cat out; } ++ ++Exit $fail +diff -Naurp coreutils-8.25-orig/tests/misc/sort-merge.pl coreutils-8.25/tests/misc/sort-merge.pl +--- coreutils-8.25-orig/tests/misc/sort-merge.pl 2016-01-16 12:18:14.000000000 -0600 ++++ coreutils-8.25/tests/misc/sort-merge.pl 2016-02-08 19:07:10.316944667 -0600 +@@ -26,6 +26,15 @@ my $prog = 'sort'; + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + ++my $mb_locale; ++# uncommented according to upstream commit enabling multibyte paths ++$mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + # three empty files and one that says 'foo' + my @inputs = (+(map{{IN=> {"empty$_"=> ''}}}1..3), {IN=> {foo=> "foo\n"}}); + +@@ -77,6 +86,39 @@ my @Tests = + {OUT=>$big_input}], + ); + ++# Add _POSIX2_VERSION=199209 to the environment of each test ++# that uses an old-style option like +1. ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether sort is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ next if ($test_name =~ "nmerge-."); ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ ++@Tests = triple_test @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + +diff -Naurp coreutils-8.25-orig/tests/misc/sort.pl coreutils-8.25/tests/misc/sort.pl +--- coreutils-8.25-orig/tests/misc/sort.pl 2016-01-16 12:18:14.000000000 -0600 ++++ coreutils-8.25/tests/misc/sort.pl 2016-02-08 19:07:10.316944667 -0600 +@@ -24,10 +24,15 @@ my $prog = 'sort'; + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + +-my $mb_locale = $ENV{LOCALE_FR_UTF8}; ++my $mb_locale; ++#Comment out next line to disable multibyte tests ++$mb_locale = $ENV{LOCALE_FR_UTF8}; + ! defined $mb_locale || $mb_locale eq 'none' + and $mb_locale = 'C'; + ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + # Since each test is run with a file name and with redirected stdin, + # the name in the diagnostic is either the file name or "-". + # Normalize each diagnostic to use '-'. +@@ -424,6 +429,38 @@ foreach my $t (@Tests) + } + } + ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether sort is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ #disable several failing tests until investigation, disable all tests with envvars set ++ next if (grep {ref $_ eq 'HASH' && exists $_->{ENV}} (@new_t)); ++ next if ($test_name =~ "18g" or $test_name =~ "sort-numeric" or $test_name =~ "08[ab]" or $test_name =~ "03[def]" or $test_name =~ "h4" or $test_name =~ "n1" or $test_name =~ "2[01]a"); ++ next if ($test_name =~ "11[ab]"); # avoid FP: expected result differs to MB result due to collation rules. ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ + @Tests = triple_test @Tests; + + # Remember that triple_test creates from each test with exactly one "IN" +@@ -433,6 +470,7 @@ foreach my $t (@Tests) + # Remove the IN_PIPE version of the "output-is-input" test above. + # The others aren't susceptible because they have three inputs each. + @Tests = grep {$_->[0] ne 'output-is-input.p'} @Tests; ++@Tests = grep {$_->[0] ne 'output-is-input-mb.p'} @Tests; + + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; +diff -Naurp coreutils-8.25-orig/tests/misc/unexpand.pl coreutils-8.25/tests/misc/unexpand.pl +--- coreutils-8.25-orig/tests/misc/unexpand.pl 2016-01-16 12:18:14.000000000 -0600 ++++ coreutils-8.25/tests/misc/unexpand.pl 2016-02-08 19:07:10.317944671 -0600 +@@ -27,6 +27,14 @@ my $limits = getlimits (); + + my $prog = 'unexpand'; + ++# comment out next line to disable multibyte tests ++my $mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + my @Tests = + ( + ['a1', {IN=> ' 'x 1 ."y\n"}, {OUT=> ' 'x 1 ."y\n"}], +@@ -92,6 +100,37 @@ my @Tests = + {EXIT => 1}, {ERR => "$prog: tab stop value is too large\n"}], + ); + ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether unexpand is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ next if ($test_name =~ 'b-1'); ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ ++@Tests = triple_test @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + +diff -Naurp coreutils-8.25-orig/tests/misc/uniq.pl coreutils-8.25/tests/misc/uniq.pl +--- coreutils-8.25-orig/tests/misc/uniq.pl 2016-01-16 12:18:14.000000000 -0600 ++++ coreutils-8.25/tests/misc/uniq.pl 2016-02-08 19:07:10.317944671 -0600 +@@ -23,9 +23,17 @@ my $limits = getlimits (); + my $prog = 'uniq'; + my $try = "Try '$prog --help' for more information.\n"; + ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + # Turn off localization of executable's output. + @ENV{qw(LANGUAGE LANG LC_ALL)} = ('C') x 3; + ++my $mb_locale; ++#Comment out next line to disable multibyte tests ++$mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ + # When possible, create a "-z"-testing variant of each test. + sub add_z_variants($) + { +@@ -262,6 +270,53 @@ foreach my $t (@Tests) + and push @$t, {ENV=>'_POSIX2_VERSION=199209'}; + } + ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether uniq is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ # In test #145, replace the each ‘...’ by '...'. ++ if ($test_name =~ "145") ++ { ++ my $sub = { ERR_SUBST => "s/‘([^’]+)’/'$1'/g"}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ next if ( $test_name =~ "schar" ++ or $test_name =~ "^obs-plus" ++ or $test_name =~ "119"); ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ ++# Remember that triple_test creates from each test with exactly one "IN" ++# file two more tests (.p and .r suffix on name) corresponding to reading ++# input from a file and from a pipe. The pipe-reading test would fail ++# due to a race condition about 1 in 20 times. ++# Remove the IN_PIPE version of the "output-is-input" test above. ++# The others aren't susceptible because they have three inputs each. ++ ++@Tests = grep {$_->[0] ne 'output-is-input.p'} @Tests; ++ + @Tests = add_z_variants @Tests; + @Tests = triple_test @Tests; + +diff -Naurp coreutils-8.25-orig/tests/pr/pr-tests.pl coreutils-8.25/tests/pr/pr-tests.pl +--- coreutils-8.25-orig/tests/pr/pr-tests.pl 2016-01-16 12:18:14.000000000 -0600 ++++ coreutils-8.25/tests/pr/pr-tests.pl 2016-02-08 19:07:10.318944674 -0600 +@@ -24,6 +24,15 @@ use strict; + my $prog = 'pr'; + my $normalize_strerror = "s/': .*/'/"; + ++my $mb_locale; ++#Uncomment the following line to enable multibyte tests ++$mb_locale = $ENV{LOCALE_FR_UTF8}; ++! defined $mb_locale || $mb_locale eq 'none' ++ and $mb_locale = 'C'; ++ ++my $try = "Try `$prog --help' for more information.\n"; ++my $inval = "$prog: invalid byte, character or field list\n$try"; ++ + my @tv = ( + + # -b option is no longer an official option. But it's still working to +@@ -467,8 +476,48 @@ push @Tests, + {IN=>{3=>"x\ty\tz\n"}}, + {OUT=>join("\t", qw(a b c m n o x y z)) . "\n"} ]; + ++# Add _POSIX2_VERSION=199209 to the environment of each test ++# that uses an old-style option like +1. ++if ($mb_locale ne 'C') ++ { ++ # Duplicate each test vector, appending "-mb" to the test name and ++ # inserting {ENV => "LC_ALL=$mb_locale"} in the copy, so that we ++ # provide coverage for the distro-added multi-byte code paths. ++ my @new; ++ foreach my $t (@Tests) ++ { ++ my @new_t = @$t; ++ my $test_name = shift @new_t; ++ ++ # Depending on whether pr is multi-byte-patched, ++ # it emits different diagnostics: ++ # non-MB: invalid byte or field list ++ # MB: invalid byte, character or field list ++ # Adjust the expected error output accordingly. ++ if (grep {ref $_ eq 'HASH' && exists $_->{ERR} && $_->{ERR} eq $inval} ++ (@new_t)) ++ { ++ my $sub = {ERR_SUBST => 's/, character//'}; ++ push @new_t, $sub; ++ push @$t, $sub; ++ } ++ #temporarily skip some failing tests ++ next if ($test_name =~ "col-0" or $test_name =~ "col-inval"); ++ push @new, ["$test_name-mb", @new_t, {ENV => "LC_ALL=$mb_locale"}]; ++ } ++ push @Tests, @new; ++ } ++ + @Tests = triple_test @Tests; + ++# Remember that triple_test creates from each test with exactly one "IN" ++# file two more tests (.p and .r suffix on name) corresponding to reading ++# input from a file and from a pipe. The pipe-reading test would fail ++# due to a race condition about 1 in 20 times. ++# Remove the IN_PIPE version of the "output-is-input" test above. ++# The others aren't susceptible because they have three inputs each. ++@Tests = grep {$_->[0] ne 'output-is-input.p'} @Tests; ++ + my $save_temps = $ENV{DEBUG}; + my $verbose = $ENV{VERBOSE}; + diff --git a/src/patches/expect-5.43.0-spawn-1.patch b/src/patches/expect-5.43.0-spawn-1.patch deleted file mode 100644 index c2033c0..0000000 --- a/src/patches/expect-5.43.0-spawn-1.patch +++ /dev/null @@ -1,217 +0,0 @@ -Submitted By: LFS Book lfs-book@linuxfromscratch.org -Date: 2003-10-05 -Initial Package Version: 5.38 -Origin: Redhat RPM (Patch by HJ Lu) -Description: NA -diff -uNr expect-5.38.orig/exp_chan.c expect-5.38/exp_chan.c ---- expect-5.38.orig/exp_chan.c 2002-02-12 13:00:55.000000000 +1100 -+++ expect-5.38/exp_chan.c 2003-03-01 10:36:18.000000000 +1100 -@@ -519,6 +519,7 @@ - esPtr->buffer = Tcl_NewStringObj("",0); - Tcl_IncrRefCount(esPtr->buffer); - esPtr->umsize = exp_default_match_max; -+ esPtr->umsize_changed = exp_default_match_max_changed; - /* this will reallocate object with an appropriate sized buffer */ - expAdjust(esPtr); - -diff -uNr expect-5.38.orig/exp_command.h expect-5.38/exp_command.h ---- expect-5.38.orig/exp_command.h 2002-04-08 08:57:20.000000000 +1000 -+++ expect-5.38/exp_command.h 2003-03-01 10:36:18.000000000 +1100 -@@ -25,6 +25,7 @@ - EXTERN char * exp_get_var _ANSI_ARGS_((Tcl_Interp *,char *)); - - EXTERN int exp_default_match_max; -+EXTERN int exp_default_match_max_changed; - EXTERN int exp_default_parity; - EXTERN int exp_default_rm_nulls; - -@@ -97,6 +98,7 @@ - int msize; /* # of bytes that buffer can hold (max) */ - int umsize; /* # of bytes (min) that is guaranteed to match */ - /* this comes from match_max command */ -+ int umsize_changed; /* is umsize changed by user? */ - int printed; /* # of bytes written to stdout (if logging on) */ - /* but not actually returned via a match yet */ - int echoed; /* additional # of bytes (beyond "printed" above) */ -diff -uNr expect-5.38.orig/expect.c expect-5.38/expect.c ---- expect-5.38.orig/expect.c 2002-04-08 09:00:33.000000000 +1000 -+++ expect-5.38/expect.c 2003-03-01 10:36:18.000000000 +1100 -@@ -41,8 +41,17 @@ - #include "tcldbg.h" - #endif - -+/* The initial length is 2000. We increment it by 2000. The maximum -+ is 8MB (0x800000). */ -+#define EXP_MATCH_MAX 2000 -+#define EXP_MATCH_INC 2000 -+#define EXP_MATCH_STEP_LIMIT 0x700000 -+#define EXP_MATCH_LIMIT 0x800000 -+#define EXP_MATCH_LIMIT_QUOTE "0x800000" -+ - /* initial length of strings that we can guarantee patterns can match */ --int exp_default_match_max = 2000; -+int exp_default_match_max = EXP_MATCH_MAX; -+int exp_default_match_max_changed = 0; - #define INIT_EXPECT_TIMEOUT_LIT "10" /* seconds */ - #define INIT_EXPECT_TIMEOUT 10 /* seconds */ - int exp_default_parity = TRUE; -@@ -1618,6 +1627,76 @@ - return newsize; - } - -+/* returns # of bytes until we see a newline at the end or EOF. */ -+/*ARGSUSED*/ -+static int -+expReadNewLine(interp,esPtr,save_flags) /* INTL */ -+Tcl_Interp *interp; -+ExpState *esPtr; -+int save_flags; -+{ -+ int size; -+ int exp_size; -+ int full_size; -+ int count; -+ char *str; -+ -+ count = 0; -+ for (;;) { -+ exp_size = expSizeGet(esPtr); -+ -+ /* When we reach the limit, we will only read one char at a -+ time. */ -+ if (esPtr->umsize >= EXP_MATCH_STEP_LIMIT) -+ size = TCL_UTF_MAX; -+ else -+ size = exp_size; -+ -+ if (exp_size + TCL_UTF_MAX >= esPtr->msize) { -+ if (esPtr->umsize >= EXP_MATCH_LIMIT) { -+ expDiagLogU("WARNING: interact buffer is full. probably your program\r\n"); -+ expDiagLogU("is not interactive or has a very long output line. The\r\n"); -+ expDiagLogU("current limit is " EXP_MATCH_LIMIT_QUOTE ".\r\n"); -+ expDiagLogU("Dumping first half of buffer in order to continue\r\n"); -+ expDiagLogU("Recommend you enlarge the buffer.\r\n"); -+ exp_buffer_shuffle(interp,esPtr,save_flags,EXPECT_OUT,"expect"); -+ return count; -+ } -+ else { -+ esPtr->umsize += EXP_MATCH_INC; -+ expAdjust(esPtr); -+ } -+ } -+ -+ full_size = esPtr->msize - (size / TCL_UTF_MAX); -+ size = Tcl_ReadChars(esPtr->channel, -+ esPtr->buffer, -+ full_size, -+ 1 /* append */); -+ if (size > 0) { -+ count += size; -+ /* We try again if there are more to read and we haven't -+ seen a newline at the end. */ -+ if (size == full_size) { -+ str = Tcl_GetStringFromObj(esPtr->buffer, &size); -+ if (str[size - 1] != '\n') -+ continue; -+ } -+ } -+ else { -+ /* It is even trickier. We got an error from read. We have -+ to recover from it. Let's make sure the size of -+ buffer is correct. It can be corrupted. */ -+ str = Tcl_GetString(esPtr->buffer); -+ Tcl_SetObjLength(esPtr->buffer, strlen(str)); -+ } -+ -+ break; -+ } -+ -+ return count; -+} -+ - /* returns # of bytes read or (non-positive) error of form EXP_XXX */ - /* returns 0 for end of file */ - /* If timeout is non-zero, set an alarm before doing the read, else assume */ -@@ -1632,6 +1711,8 @@ - { - int cc = EXP_TIMEOUT; - int size = expSizeGet(esPtr); -+ int full_size; -+ int count; - - if (size + TCL_UTF_MAX >= esPtr->msize) - exp_buffer_shuffle(interp,esPtr,save_flags,EXPECT_OUT,"expect"); -@@ -1648,11 +1729,43 @@ - } - #endif - -- -+ /* FIXME: If we ask less than what is available in the tcl buffer -+ when tcl has seen EOF, we will throw away the remaining data -+ since the next read will get EOF. Since expect is line-oriented, -+ we exand our buffer to get EOF or the next newline at the end of -+ the input buffer. I don't know if it is the right fix. H.J. */ -+ count = 0; -+ full_size = esPtr->msize - (size / TCL_UTF_MAX); - cc = Tcl_ReadChars(esPtr->channel, -- esPtr->buffer, -- esPtr->msize - (size / TCL_UTF_MAX), -- 1 /* append */); -+ esPtr->buffer, -+ full_size, -+ 1 /* append */); -+ if (cc > 0) { -+ count += cc; -+ /* It gets very tricky. There are more to read. We will expand -+ our buffer and get EOF or a newline at the end unless the -+ buffer length has been changed. */ -+ if (cc == full_size) { -+ char *str; -+ str = Tcl_GetStringFromObj(esPtr->buffer, &size); -+ if (str[size - 1] != '\n') { -+ if (esPtr->umsize_changed) { -+ char buf[20]; /* big enough for 64bit int in hex. */ -+ snprintf(buf,sizeof(buf),"0x%x", esPtr->umsize); -+ expDiagLogU("WARNING: interact buffer is not large enough to hold\r\n"); -+ expDiagLogU("all output. probably your program is not interactive or\r\n"); -+ expDiagLogU("has a very long output line. The current limit is "); -+ expDiagLogU(buf); -+ expDiagLogU(".\r\n"); -+ } -+ else { -+ cc = expReadNewLine(interp,esPtr,save_flags); -+ if (cc > 0) -+ count += cc; -+ } -+ } -+ } -+ } - i_read_errno = errno; - - #ifdef SIMPLE_EVENT -@@ -1673,7 +1786,7 @@ - } - } - #endif -- return cc; -+ return count > 0 ? count : cc; - } - - /* -@@ -2746,8 +2859,14 @@ - return(TCL_ERROR); - } - -- if (Default) exp_default_match_max = size; -- else esPtr->umsize = size; -+ if (Default) { -+ exp_default_match_max = size; -+ exp_default_match_max_changed = 1; -+ } -+ else { -+ esPtr->umsize = size; -+ esPtr->umsize_changed = 1; -+ } - - return(TCL_OK); - } diff --git a/src/patches/libvirt/0001-Change-default-behavior-of-libvirt-guests.sh-for-IPF.patch b/src/patches/libvirt/0001-Change-default-behavior-of-libvirt-guests.sh-for-IPF.patch new file mode 100644 index 0000000..12af9a5 --- /dev/null +++ b/src/patches/libvirt/0001-Change-default-behavior-of-libvirt-guests.sh-for-IPF.patch @@ -0,0 +1,30 @@ +From a50fa0195e36773d57593006152828ce2c0523fd Mon Sep 17 00:00:00 2001 +From: Jonatan Schlag jonatan.schlag@ipfire.org +Date: Fri, 6 May 2016 11:38:08 +0200 +Subject: [PATCH] Change default behavior of libvirt-guests.sh for IPFire + +Signed-off-by: Jonatan Schlag jonatan.schlag@ipfire.org +--- + tools/libvirt-guests.sh.in | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/tools/libvirt-guests.sh.in b/tools/libvirt-guests.sh.in +index 7f74b85..87aceb7 100644 +--- a/tools/libvirt-guests.sh.in ++++ b/tools/libvirt-guests.sh.in +@@ -30,9 +30,9 @@ test ! -r "$sysconfdir"/rc.d/init.d/functions || + + export TEXTDOMAIN="@PACKAGE@" TEXTDOMAINDIR="@localedir@" + +-URIS=default +-ON_BOOT=start +-ON_SHUTDOWN=suspend ++URIS=qemu:///system ++ON_BOOT=ignore ++ON_SHUTDOWN=shutdown + SHUTDOWN_TIMEOUT=300 + PARALLEL_SHUTDOWN=0 + START_DELAY=0 +-- +2.1.4 + diff --git a/src/patches/squid/squid-3.5-14051.patch b/src/patches/squid/squid-3.5-14051.patch new file mode 100644 index 0000000..58892dc --- /dev/null +++ b/src/patches/squid/squid-3.5-14051.patch @@ -0,0 +1,63 @@ +------------------------------------------------------------ +revno: 14051 +revision-id: squid3@treenet.co.nz-20160517145850-uos9z00nrt7xd9ik +parent: squid3@treenet.co.nz-20160508124125-fytgvn68zppfr8ix +author: Steve Hill steve@opendium.com +committer: Amos Jeffries squid3@treenet.co.nz +branch nick: 3.5 +timestamp: Wed 2016-05-18 02:58:50 +1200 +message: + Support unified EUI format code in external_acl_type + + Squid supports %>eui as a logformat specifier, which produces an EUI-48 + for IPv4 clients and an EUI-64 for IPv6 clients. However, This is not + allowed as a format specifier for the external ACLs, and you have to use + %SRCEUI48 and %SRCEUI64 instead. %SRCEUI48 is only useful for IPv4 + clients and %SRCEUI64 is only useful for IPv6 clients, so supporting + both v4 and v6 is a bit messy. + + Adds the %>eui specifier for external ACLs and behaves in the same way + as the logformat specifier. +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squid3@treenet.co.nz-20160517145850-uos9z00nrt7xd9ik +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: ad0743717948a65cfd4f306acc2bbaa9343e9a76 +# timestamp: 2016-05-17 15:50:54 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squid3@treenet.co.nz-20160508124125-\ +# fytgvn68zppfr8ix +# +# Begin patch +=== modified file 'src/external_acl.cc' +--- src/external_acl.cc 2016-01-01 00:14:27 +0000 ++++ src/external_acl.cc 2016-05-17 14:58:50 +0000 +@@ -356,6 +356,8 @@ + else if (strcmp(token, "%SRCPORT") == 0 || strcmp(token, "%>p") == 0) + format->type = Format::LFT_CLIENT_PORT; + #if USE_SQUID_EUI ++ else if (strcmp(token, "%>eui") == 0) ++ format->type = Format::LFT_CLIENT_EUI; + else if (strcmp(token, "%SRCEUI48") == 0) + format->type = Format::LFT_EXT_ACL_CLIENT_EUI48; + else if (strcmp(token, "%SRCEUI64") == 0) +@@ -944,6 +946,18 @@ + break; + + #if USE_SQUID_EUI ++ case Format::LFT_CLIENT_EUI: ++ // TODO make the ACL checklist have a direct link to any TCP details. ++ if (request->clientConnectionManager.valid() && request->clientConnectionManager->clientConnection != NULL) ++ { ++ if (request->clientConnectionManager->clientConnection->remote.isIPv4()) ++ request->clientConnectionManager->clientConnection->remoteEui48.encode(buf, sizeof(buf)); ++ else ++ request->clientConnectionManager->clientConnection->remoteEui64.encode(buf, sizeof(buf)); ++ str = buf; ++ } ++ break; ++ + case Format::LFT_EXT_ACL_CLIENT_EUI48: + if (request->clientConnectionManager.valid() && request->clientConnectionManager->clientConnection != NULL && + request->clientConnectionManager->clientConnection->remoteEui48.encode(buf, sizeof(buf))) + diff --git a/src/patches/squid/squid-3.5-14052.patch b/src/patches/squid/squid-3.5-14052.patch new file mode 100644 index 0000000..4fba159 --- /dev/null +++ b/src/patches/squid/squid-3.5-14052.patch @@ -0,0 +1,34 @@ +------------------------------------------------------------ +revno: 14052 +revision-id: squidadm@squid-cache.org-20160517181416-sfrjdosd9dhx7u8o +parent: squid3@treenet.co.nz-20160517145850-uos9z00nrt7xd9ik +committer: Source Maintenance squidadm@squid-cache.org +branch nick: 3.5 +timestamp: Tue 2016-05-17 18:14:16 +0000 +message: + SourceFormat Enforcement +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squidadm@squid-cache.org-20160517181416-\ +# sfrjdosd9dhx7u8o +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: e30c12805cacdb559925da08cc6a25fe4a39c19b +# timestamp: 2016-05-17 18:51:06 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squid3@treenet.co.nz-20160517145850-\ +# uos9z00nrt7xd9ik +# +# Begin patch +=== modified file 'src/external_acl.cc' +--- src/external_acl.cc 2016-05-17 14:58:50 +0000 ++++ src/external_acl.cc 2016-05-17 18:14:16 +0000 +@@ -956,7 +956,7 @@ + request->clientConnectionManager->clientConnection->remoteEui64.encode(buf, sizeof(buf)); + str = buf; + } +- break; ++ break; + + case Format::LFT_EXT_ACL_CLIENT_EUI48: + if (request->clientConnectionManager.valid() && request->clientConnectionManager->clientConnection != NULL && + diff --git a/src/patches/squid/squid-3.5-14053.patch b/src/patches/squid/squid-3.5-14053.patch new file mode 100644 index 0000000..f669449 --- /dev/null +++ b/src/patches/squid/squid-3.5-14053.patch @@ -0,0 +1,46 @@ +------------------------------------------------------------ +revno: 14053 +revision-id: squid3@treenet.co.nz-20160521130058-zq8zugw0fohwfu3z +parent: squidadm@squid-cache.org-20160517181416-sfrjdosd9dhx7u8o +committer: Amos Jeffries squid3@treenet.co.nz +branch nick: 3.5 +timestamp: Sun 2016-05-22 01:00:58 +1200 +message: + Do not override user defined -std option +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squid3@treenet.co.nz-20160521130058-zq8zugw0fohwfu3z +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: a75245a622ccfa385ef5e4722f9a9fb438a16135 +# timestamp: 2016-05-21 13:08:06 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squidadm@squid-cache.org-20160517181416-\ +# sfrjdosd9dhx7u8o +# +# Begin patch +=== modified file 'configure.ac' +--- configure.ac 2016-05-08 12:41:25 +0000 ++++ configure.ac 2016-05-21 13:00:58 +0000 +@@ -95,6 +95,9 @@ + # Guess the compiler type (sets squid_cv_compiler) + SQUID_CC_GUESS_VARIANT + ++# If the user did not specify a C++ version. ++user_cxx=`echo "$PRESET_CXXFLAGS" | grep -o -E "-std="` ++if test "x$user_cxx" = "x"; then + # Check for C++11 compiler support + # + # BUG 3613: when clang -std=c++0x is used, it activates a "strict mode" +@@ -103,8 +106,9 @@ + # + # Similar POSIX issues on MinGW 32-bit and Cygwin + # +-if ! test "x$squid_host_os" = "xmingw" -o "x$squid_host_os" = "xcygwin" -o "x$squid_cv_compiler" = "xclang"; then +- AX_CXX_COMPILE_STDCXX_11([noext],[optional]) ++ if ! test "x$squid_host_os" = "xmingw" -o "x$squid_host_os" = "xcygwin" -o "x$squid_cv_compiler" = "xclang"; then ++ AX_CXX_COMPILE_STDCXX_11([noext],[optional]) ++ fi + fi + + # test for programs + diff --git a/src/patches/squid/squid-3.5-14054.patch b/src/patches/squid/squid-3.5-14054.patch new file mode 100644 index 0000000..90b34c1 --- /dev/null +++ b/src/patches/squid/squid-3.5-14054.patch @@ -0,0 +1,37 @@ +------------------------------------------------------------ +revno: 14054 +revision-id: squid3@treenet.co.nz-20160521130144-6xtcayieij00fm5v +parent: squid3@treenet.co.nz-20160521130058-zq8zugw0fohwfu3z +committer: Amos Jeffries squid3@treenet.co.nz +branch nick: 3.5 +timestamp: Sun 2016-05-22 01:01:44 +1200 +message: + Fix OpenSSL detection on FreeBSD +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squid3@treenet.co.nz-20160521130144-6xtcayieij00fm5v +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: 3d8c0d7a9f1886523ac55d79e4d3e8f0340e2ec9 +# timestamp: 2016-05-21 13:08:08 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squid3@treenet.co.nz-20160521130058-\ +# zq8zugw0fohwfu3z +# +# Begin patch +=== modified file 'configure.ac' +--- configure.ac 2016-05-21 13:00:58 +0000 ++++ configure.ac 2016-05-21 13:01:44 +0000 +@@ -1348,10 +1348,10 @@ + + AC_CHECK_LIB(crypto,[CRYPTO_new_ex_data],[LIBOPENSSL_LIBS="-lcrypto $LIBOPENSSL_LIBS"],[ + AC_MSG_ERROR([library 'crypto' is required for OpenSSL]) +- ]) ++ ],$LIBOPENSSL_LIBS) + AC_CHECK_LIB(ssl,[SSL_library_init],[LIBOPENSSL_LIBS="-lssl $LIBOPENSSL_LIBS"],[ + AC_MSG_ERROR([library 'ssl' is required for OpenSSL]) +- ]) ++ ],$LIBOPENSSL_LIBS) + ]) + + # This is a workaround for RedHat 9 brain damage.. + diff --git a/src/patches/squid/squid-3.5-14055.patch b/src/patches/squid/squid-3.5-14055.patch new file mode 100644 index 0000000..ac04bb6 --- /dev/null +++ b/src/patches/squid/squid-3.5-14055.patch @@ -0,0 +1,39 @@ +------------------------------------------------------------ +revno: 14055 +revision-id: squid3@treenet.co.nz-20160521155202-pp53utwamdhkugvg +parent: squid3@treenet.co.nz-20160521130144-6xtcayieij00fm5v +author: Alex Rousskov rousskov@measurement-factory.com +committer: Amos Jeffries squid3@treenet.co.nz +branch nick: 3.5 +timestamp: Sun 2016-05-22 03:52:02 +1200 +message: + Fix icons loading speed. + + Since trunk r14100 (Bug 3875: bad mimeLoadIconFile error handling), each + icon was read from disk and written to Store one character at a time. I + did not measure startup delays in production, but in debugging runs, + fixing this bug sped up icons loading from 1 minute to 4 seconds. +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squid3@treenet.co.nz-20160521155202-pp53utwamdhkugvg +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: 79b78480d81666c15406d23837608ba9a578da4b +# timestamp: 2016-05-21 16:51:00 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squid3@treenet.co.nz-20160521130144-\ +# 6xtcayieij00fm5v +# +# Begin patch +=== modified file 'src/mime.cc' +--- src/mime.cc 2016-01-01 00:14:27 +0000 ++++ src/mime.cc 2016-05-21 15:52:02 +0000 +@@ -430,7 +430,7 @@ + /* read the file into the buffer and append it to store */ + int n; + char *buf = (char *)memAllocate(MEM_4K_BUF); +- while ((n = FD_READ_METHOD(fd, buf, sizeof(*buf))) > 0) ++ while ((n = FD_READ_METHOD(fd, buf, 4096)) > 0) + e->append(buf, n); + + file_close(fd); + diff --git a/src/patches/squid/squid-3.5-14056.patch b/src/patches/squid/squid-3.5-14056.patch new file mode 100644 index 0000000..4ea3808 --- /dev/null +++ b/src/patches/squid/squid-3.5-14056.patch @@ -0,0 +1,36 @@ +------------------------------------------------------------ +revno: 14056 +revision-id: squid3@treenet.co.nz-20160521172919-du6cbdirqcxdjbtr +parent: squid3@treenet.co.nz-20160521155202-pp53utwamdhkugvg +author: Christos Tsantilas chtsanti@users.sourceforge.net +committer: Amos Jeffries squid3@treenet.co.nz +branch nick: 3.5 +timestamp: Sun 2016-05-22 05:29:19 +1200 +message: + Increase debug level in a peek-and-splice related debug message + + It may produced one debugging line for each SSL transaction in some cases +------------------------------------------------------------ +# Bazaar merge directive format 2 (Bazaar 0.90) +# revision_id: squid3@treenet.co.nz-20160521172919-du6cbdirqcxdjbtr +# target_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# testament_sha1: 76c2e864289dabb1065c470c954f9fc5ec4c7b4f +# timestamp: 2016-05-21 17:50:54 +0000 +# source_branch: http://bzr.squid-cache.org/bzr/squid3/3.5 +# base_revision_id: squid3@treenet.co.nz-20160521155202-\ +# pp53utwamdhkugvg +# +# Begin patch +=== modified file 'src/ssl/PeerConnector.cc' +--- src/ssl/PeerConnector.cc 2016-02-15 11:29:50 +0000 ++++ src/ssl/PeerConnector.cc 2016-05-21 17:29:19 +0000 +@@ -598,7 +598,7 @@ + + case SSL_ERROR_WANT_WRITE: + if ((srvBio->bumpMode() == Ssl::bumpPeek || srvBio->bumpMode() == Ssl::bumpStare) && srvBio->holdWrite()) { +- debugs(81, DBG_IMPORTANT, "hold write on SSL connection on FD " << fd); ++ debugs(81, 3, "hold write on SSL connection on FD " << fd); + checkForPeekAndSplice(); + return; + } + diff --git a/tools/check_strings.pl b/tools/check_strings.pl index d521068..b623778 100755 --- a/tools/check_strings.pl +++ b/tools/check_strings.pl @@ -36,7 +36,7 @@ require "${basedir}/langs/$lang/cgi-bin/$lang.pl"; sub wanted { if ( -f $File::Find::name && open(FILE, $File::Find::name)) { while (<FILE>) { - while ($_ =~ /$Lang::tr{'([A-Za-z0-9,:_\s/.-]+)'}/g) { + while ($_ =~ /$Lang::tr{'([A-Za-z0-9,:_\s/.-]+)'}/g) { $tr2{$1} = 'empty string'; } }
hooks/post-receive -- IPFire 2.x development tree