- Update from version 5.2.5 to 5.2.8
- Update of rootfile
- Remove xzgrep-ZDI-CAN-16587 patch as the contents are now integrated into the source
tarball and with an improved quicker method - see changelog below.
- Changelog
5.2.8 (2022-11-13)
* xz:
- If xz cannot remove an input file when it should, this
is now treated as a warning (exit status 2) instead of
an error (exit status 1). This matches GNU gzip and it
is more logical as at that point the output file has
already been successfully closed.
- Fix handling of .xz files with an unsupported check type.
Previously such printed a warning message but then xz
behaved as if an error had occurred (didn't decompress,
exit status 1). Now a warning is printed, decompression
is done anyway, and exit status is 2. This used to work
slightly before 5.0.0. In practice this bug matters only
if xz has been built with some check types disabled. As
instructed in PACKAGERS, such builds should be done in
special situations only.
- Fix "xz -dc --single-stream tests/files/good-0-empty.xz"
which failed with "Internal error (bug)". That is,
--single-stream was broken if the first .xz stream in
the input file didn't contain any uncompressed data.
- Fix displaying file sizes in the progress indicator when
working in passthru mode and there are multiple input files.
Just like "gzip -cdf", "xz -cdf" works like "cat" when the
input file isn't a supported compressed file format. In
this case the file size counters weren't reset between
files so with multiple input files the progress indicator
displayed an incorrect (too large) value.
* liblzma:
- API docs in lzma/container.h:
* Update the list of decoder flags in the decoder
function docs.
* Explain LZMA_CONCATENATED behavior with .lzma files
in lzma_auto_decoder() docs.
- OpenBSD: Use HW_NCPUONLINE to detect the number of
available hardware threads in lzma_physmem().
- Fix use of wrong macro to detect x86 SSE2 support.
__SSE2_MATH__ was used with GCC/Clang but the correct
one is __SSE2__. The first one means that SSE2 is used
for floating point math which is irrelevant here.
The affected SSE2 code isn't used on x86-64 so this affects
only 32-bit x86 builds that use -msse2 without -mfpmath=sse
(there is no runtime detection for SSE2). It improves LZMA
compression speed (not decompression).
- Fix the build with Intel C compiler 2021 (ICC, not ICX)
on Linux. It defines __GNUC__ to 10 but doesn't support
the __symver__ attribute introduced in GCC 10.
* Scripts: Ignore warnings from xz by using --quiet --no-warn.
This is needed if the input .xz files use an unsupported
check type.
* Translations:
- Updated Croatian and Turkish translations.
- One new translations wasn't included because it needed
technical fixes. It will be in upcoming 5.4.0. No new
translations will be added to the 5.2.x branch anymore.
- Renamed the French man page translation file from
fr_FR.po to fr.po and thus also its install directory
(like /usr/share/man/fr_FR -> .../fr).
- Man page translations for upcoming 5.4.0 are now handled
in the Translation Project.
* Update doc/faq.txt a little so it's less out-of-date.
5.2.7 (2022-09-30)
* liblzma:
- Made lzma_filters_copy() to never modify the destination
array if an error occurs. lzma_stream_encoder() and
lzma_stream_encoder_mt() already assumed this. Before this
change, if a tiny memory allocation in lzma_filters_copy()
failed it would lead to a crash (invalid free() or invalid
memory reads) in the cleanup paths of these two encoder
initialization functions.
- Added missing integer overflow check to lzma_index_append().
This affects xz --list and other applications that decode
the Index field from .xz files using lzma_index_decoder().
Normal decompression of .xz files doesn't call this code
and thus most applications using liblzma aren't affected
by this bug.
- Single-threaded .xz decoder (lzma_stream_decoder()): If
lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible
to use lzma_memlimit_set() to increase the limit and continue
decoding. This was supposed to work from the beginning
but there was a bug. With other decoders (.lzma or
threaded .xz decoder) this already worked correctly.
- Fixed accumulation of integrity check type statistics in
lzma_index_cat(). This bug made lzma_index_checks() return
only the type of the integrity check of the last Stream
when multiple lzma_indexes were concatenated. Most
applications don't use these APIs but in xz it made
xz --list not list all check types from concatenated .xz
files. In xz --list --verbose only the per-file "Check:"
lines were affected and in xz --robot --list only the "file"
line was affected.
- Added ABI compatibility with executables that were linked
against liblzma in RHEL/CentOS 7 or other liblzma builds
that had copied the problematic patch from RHEL/CentOS 7
(xz-5.2.2-compat-libs.patch). For the details, see the
comment at the top of src/liblzma/validate_map.sh.
WARNING: This uses __symver__ attribute with GCC >= 10.
In other cases the traditional __asm__(".symver ...")
is used. Using link-time optimization (LTO, -flto) with
GCC versions older than 10 can silently result in
broken liblzma.so.5 (incorrect symbol versions)! If you
want to use -flto with GCC, you must use GCC >= 10.
LTO with Clang seems to work even with the traditional
__asm__(".symver ...") method.
* xzgrep: Fixed compatibility with old shells that break if
comments inside command substitutions have apostrophes (').
This problem was introduced in 5.2.6.
* Build systems:
- New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
- Windows: Fixed liblzma.dll build with Visual Studio project
files. It broke in 5.2.6 due to a change that was made to
improve CMake support.
- Windows: Building liblzma with UNICODE defined should now
work.
- CMake files are now actually included in the release tarball.
They should have been in 5.2.5 already.
- Minor CMake fixes and improvements.
* Added a new translation: Turkish
5.2.6 (2022-08-12)
* xz:
- The --keep option now accepts symlinks, hardlinks, and
setuid, setgid, and sticky files. Previously this required
using --force.
- When copying metadata from the source file to the destination
file, don't try to set the group (GID) if it is already set
correctly. This avoids a failure on OpenBSD (and possibly on
a few other OSes) where files may get created so that their
group doesn't belong to the user, and fchown(2) can fail even
if it needs to do nothing.
- Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on
MIPS32 because on MIPS32 userspace processes are limited
to 2 GiB of address space.
* liblzma:
- Fixed a missing error-check in the threaded encoder. If a
small memory allocation fails, a .xz file with an invalid
Index field would be created. Decompressing such a file would
produce the correct output but result in an error at the end.
Thus this is a "mild" data corruption bug. Note that while
a failed memory allocation can trigger the bug, it cannot
cause invalid memory access.
- The decoder for .lzma files now supports files that have
uncompressed size stored in the header and still use the
end of payload marker (end of stream marker) at the end
of the LZMA stream. Such files are rare but, according to
the documentation in LZMA SDK, they are valid.
doc/lzma-file-format.txt was updated too.
- Improved 32-bit x86 assembly files:
* Support Intel Control-flow Enforcement Technology (CET)
* Use non-executable stack on FreeBSD.
- Visual Studio: Use non-standard _MSVC_LANG to detect C++
standard version in the lzma.h API header. It's used to
detect when "noexcept" can be used.
* xzgrep:
- Fixed arbitrary command injection via a malicious filename
(CVE-2022-1271, ZDI-CAN-16587). A standalone patch for
this was released to the public on 2022-04-07. A slight
robustness improvement has been made since then and, if
using GNU or *BSD grep, a new faster method is now used
that doesn't use the old sed-based construct at all. This
also fixes bad output with GNU grep >= 3.5 (2020-09-27)
when xzgrepping binary files.
This vulnerability was discovered by:
cleemy desu wayo working with Trend Micro Zero Day Initiative
- Fixed detection of corrupt .bz2 files.
- Improved error handling to fix exit status in some situations
and to fix handling of signals: in some situations a signal
didn't make xzgrep exit when it clearly should have. It's
possible that the signal handling still isn't quite perfect
but hopefully it's good enough.
- Documented exit statuses on the man page.
- xzegrep and xzfgrep now use "grep -E" and "grep -F" instead
of the deprecated egrep and fgrep commands.
- Fixed parsing of the options -E, -F, -G, -P, and -X. The
problem occurred when multiple options were specied in
a single argument, for example,
echo foo | xzgrep -Fe foo
treated foo as a filename because -Fe wasn't correctly
split into -F -e.
- Added zstd support.
* xzdiff/xzcmp:
- Fixed wrong exit status. Exit status could be 2 when the
correct value is 1.
- Documented on the man page that exit status of 2 is used
for decompression errors.
- Added zstd support.
* xzless:
- Fix less(1) version detection. It failed if the version number
from "less -V" contained a dot.
* Translations:
- Added new translations: Catalan, Croatian, Esperanto,
Korean, Portuguese, Romanian, Serbian, Spanish, Swedish,
and Ukrainian
- Updated the Brazilian Portuguese translation.
- Added French man page translation. This and the existing
German translation aren't complete anymore because the
English man pages got a few updates and the translators
weren't reached so that they could update their work.
* Build systems:
- Windows: Fix building of resource files when config.h isn't
used. CMake + Visual Studio can now build liblzma.dll.
- Various fixes to the CMake support. Building static or shared
liblzma should work fine in most cases. In contrast, building
the command line tools with CMake is still clearly incomplete
and experimental and should be used for testing only.
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
config/rootfiles/common/xz | 37 ++++++++--
lfs/xz | 5 +-
src/patches/xzgrep-ZDI-CAN-16587.patch | 94 --------------------------
3 files changed, 32 insertions(+), 104 deletions(-)
delete mode 100644 src/patches/xzgrep-ZDI-CAN-16587.patch
diff --git a/config/rootfiles/common/xz b/config/rootfiles/common/xz
index c7bd0b302..d2f1d44cc 100644
--- a/config/rootfiles/common/xz
+++ b/config/rootfiles/common/xz
@@ -41,7 +41,7 @@ usr/bin/xzmore
#usr/lib/liblzma.la
usr/lib/liblzma.so
usr/lib/liblzma.so.5
-usr/lib/liblzma.so.5.2.5
+usr/lib/liblzma.so.5.2.8
#usr/lib/pkgconfig/liblzma.pc
#usr/share/doc/xz
#usr/share/doc/xz/AUTHORS
@@ -65,15 +65,26 @@ usr/lib/liblzma.so.5.2.5
#usr/share/doc/xz/history.txt
#usr/share/doc/xz/lzma-file-format.txt
#usr/share/doc/xz/xz-file-format.txt
+#usr/share/locale/ca/LC_MESSAGES/xz.mo
#usr/share/locale/cs/LC_MESSAGES/xz.mo
#usr/share/locale/da/LC_MESSAGES/xz.mo
#usr/share/locale/de/LC_MESSAGES/xz.mo
+#usr/share/locale/eo/LC_MESSAGES/xz.mo
+#usr/share/locale/es/LC_MESSAGES/xz.mo
#usr/share/locale/fi/LC_MESSAGES/xz.mo
#usr/share/locale/fr/LC_MESSAGES/xz.mo
+#usr/share/locale/hr/LC_MESSAGES/xz.mo
#usr/share/locale/hu/LC_MESSAGES/xz.mo
#usr/share/locale/it/LC_MESSAGES/xz.mo
+#usr/share/locale/ko/LC_MESSAGES/xz.mo
#usr/share/locale/pl/LC_MESSAGES/xz.mo
+#usr/share/locale/pt/LC_MESSAGES/xz.mo
#usr/share/locale/pt_BR/LC_MESSAGES/xz.mo
+#usr/share/locale/ro/LC_MESSAGES/xz.mo
+#usr/share/locale/sr/LC_MESSAGES/xz.mo
+#usr/share/locale/sv/LC_MESSAGES/xz.mo
+#usr/share/locale/tr/LC_MESSAGES/xz.mo
+#usr/share/locale/uk/LC_MESSAGES/xz.mo
#usr/share/locale/vi/LC_MESSAGES/xz.mo
#usr/share/locale/zh_CN/LC_MESSAGES/xz.mo
#usr/share/locale/zh_TW/LC_MESSAGES/xz.mo
@@ -82,9 +93,6 @@ usr/lib/liblzma.so.5.2.5
#usr/share/man/de/man1/lzcat.1
#usr/share/man/de/man1/lzcmp.1
#usr/share/man/de/man1/lzdiff.1
-#usr/share/man/de/man1/lzegrep.1
-#usr/share/man/de/man1/lzfgrep.1
-#usr/share/man/de/man1/lzgrep.1
#usr/share/man/de/man1/lzless.1
#usr/share/man/de/man1/lzma.1
#usr/share/man/de/man1/lzmadec.1
@@ -96,11 +104,26 @@ usr/lib/liblzma.so.5.2.5
#usr/share/man/de/man1/xzcmp.1
#usr/share/man/de/man1/xzdec.1
#usr/share/man/de/man1/xzdiff.1
-#usr/share/man/de/man1/xzegrep.1
-#usr/share/man/de/man1/xzfgrep.1
-#usr/share/man/de/man1/xzgrep.1
#usr/share/man/de/man1/xzless.1
#usr/share/man/de/man1/xzmore.1
+#usr/share/man/fr
+#usr/share/man/fr/man1
+#usr/share/man/fr/man1/lzcat.1
+#usr/share/man/fr/man1/lzcmp.1
+#usr/share/man/fr/man1/lzdiff.1
+#usr/share/man/fr/man1/lzless.1
+#usr/share/man/fr/man1/lzma.1
+#usr/share/man/fr/man1/lzmadec.1
+#usr/share/man/fr/man1/lzmore.1
+#usr/share/man/fr/man1/unlzma.1
+#usr/share/man/fr/man1/unxz.1
+#usr/share/man/fr/man1/xz.1
+#usr/share/man/fr/man1/xzcat.1
+#usr/share/man/fr/man1/xzcmp.1
+#usr/share/man/fr/man1/xzdec.1
+#usr/share/man/fr/man1/xzdiff.1
+#usr/share/man/fr/man1/xzless.1
+#usr/share/man/fr/man1/xzmore.1
#usr/share/man/man1/lzcat.1
#usr/share/man/man1/lzcmp.1
#usr/share/man/man1/lzdiff.1
diff --git a/lfs/xz b/lfs/xz
index 9345df954..83a724e1a 100644
--- a/lfs/xz
+++ b/lfs/xz
@@ -24,7 +24,7 @@
include Config
-VER = 5.2.5
+VER = 5.2.8
THISAPP = xz-$(VER)
DL_FILE = $(THISAPP).tar.xz
@@ -45,7 +45,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_BLAKE2 = 9b9b58e33722ecfe799bb50f3ffe4e86386f734ab4468eb54ff92771ddf899302d1ffa4d88bdb0de351fc3eab8a6ea103f00d7e79f33db879fe22b2e1a7e62db
+$(DL_FILE)_BLAKE2 = 44d1ddd783b2527f3b17481fc277b671808eb5639c10d31bfaca9fd29ac4413628654ecb9e207955a9477c83eb30f61cf5607cd9a49dd71732707731e4444ace
install : $(TARGET)
@@ -75,7 +75,6 @@ $(subst %,%_BLAKE2,$(objects)) :
$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
@$(PREBUILD)
@rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
- cd $(DIR_APP) && patch -p1 -i $(DIR_SRC)/src/patches/xzgrep-ZDI-CAN-16587.patch
cd $(DIR_APP) && ./configure --prefix=$(PREFIX)
cd $(DIR_APP) && make $(MAKETUNING)
cd $(DIR_APP) && make install
diff --git a/src/patches/xzgrep-ZDI-CAN-16587.patch b/src/patches/xzgrep-ZDI-CAN-16587.patch
deleted file mode 100644
index 406ded590..000000000
--- a/src/patches/xzgrep-ZDI-CAN-16587.patch
+++ /dev/null
@@ -1,94 +0,0 @@
-From 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 Mon Sep 17 00:00:00 2001
-From: Lasse Collin <lasse.collin(a)tukaani.org>
-Date: Tue, 29 Mar 2022 19:19:12 +0300
-Subject: [PATCH] xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587).
-
-Malicious filenames can make xzgrep to write to arbitrary files
-or (with a GNU sed extension) lead to arbitrary code execution.
-
-xzgrep from XZ Utils versions up to and including 5.2.5 are
-affected. 5.3.1alpha and 5.3.2alpha are affected as well.
-This patch works for all of them.
-
-This bug was inherited from gzip's zgrep. gzip 1.12 includes
-a fix for zgrep.
-
-The issue with the old sed script is that with multiple newlines,
-the N-command will read the second line of input, then the
-s-commands will be skipped because it's not the end of the
-file yet, then a new sed cycle starts and the pattern space
-is printed and emptied. So only the last line or two get escaped.
-
-One way to fix this would be to read all lines into the pattern
-space first. However, the included fix is even simpler: All lines
-except the last line get a backslash appended at the end. To ensure
-that shell command substitution doesn't eat a possible trailing
-newline, a colon is appended to the filename before escaping.
-The colon is later used to separate the filename from the grep
-output so it is fine to add it here instead of a few lines later.
-
-The old code also wasn't POSIX compliant as it used \n in the
-replacement section of the s-command. Using \<newline> is the
-POSIX compatible method.
-
-LC_ALL=C was added to the two critical sed commands. POSIX sed
-manual recommends it when using sed to manipulate pathnames
-because in other locales invalid multibyte sequences might
-cause issues with some sed implementations. In case of GNU sed,
-these particular sed scripts wouldn't have such problems but some
-other scripts could have, see:
-
- info '(sed)Locale Considerations'
-
-This vulnerability was discovered by:
-cleemy desu wayo working with Trend Micro Zero Day Initiative
-
-Thanks to Jim Meyering and Paul Eggert discussing the different
-ways to fix this and for coordinating the patch release schedule
-with gzip.
----
- src/scripts/xzgrep.in | 20 ++++++++++++--------
- 1 file changed, 12 insertions(+), 8 deletions(-)
-
-diff --git a/src/scripts/xzgrep.in b/src/scripts/xzgrep.in
-index b180936..e5186ba 100644
---- a/src/scripts/xzgrep.in
-+++ b/src/scripts/xzgrep.in
-@@ -180,22 +180,26 @@ for i; do
- { test $# -eq 1 || test $no_filename -eq 1; }; then
- eval "$grep"
- else
-+ # Append a colon so that the last character will never be a newline
-+ # which would otherwise get lost in shell command substitution.
-+ i="$i:"
-+
-+ # Escape & \ | and newlines only if such characters are present
-+ # (speed optimization).
- case $i in
- (*'
- '* | *'&'* | *'\'* | *'|'*)
-- i=$(printf '%s\n' "$i" |
-- sed '
-- $!N
-- $s/[&\|]/\\&/g
-- $s/\n/\\n/g
-- ');;
-+ i=$(printf '%s\n' "$i" | LC_ALL=C sed 's/[&\|]/\\&/g; $!s/$/\\/');;
- esac
-- sed_script="s|^|$i:|"
-+
-+ # $i already ends with a colon so don't add it here.
-+ sed_script="s|^|$i|"
-
- # Fail if grep or sed fails.
- r=$(
- exec 4>&1
-- (eval "$grep" 4>&-; echo $? >&4) 3>&- | sed "$sed_script" >&3 4>&-
-+ (eval "$grep" 4>&-; echo $? >&4) 3>&- |
-+ LC_ALL=C sed "$sed_script" >&3 4>&-
- ) || r=2
- exit $r
- fi >&3 5>&-
---
-2.35.1
-
--
2.38.1
