Reviewed-by: Michael Tremer michael.tremer@ipfire.org
On 21 Nov 2022, at 08:28, Peter Müller peter.mueller@ipfire.org wrote:
Reviewed-by: Peter Müller peter.mueller@ipfire.org
- Update from version 5.2.5 to 5.2.8
- Update of rootfile
- Remove xzgrep-ZDI-CAN-16587 patch as the contents are now integrated into the source tarball and with an improved quicker method - see changelog below.
- Changelog
5.2.8 (2022-11-13)
- xz:
- If xz cannot remove an input file when it should, this is now treated as a warning (exit status 2) instead of an error (exit status 1). This matches GNU gzip and it is more logical as at that point the output file has already been successfully closed.
- Fix handling of .xz files with an unsupported check type. Previously such printed a warning message but then xz behaved as if an error had occurred (didn't decompress, exit status 1). Now a warning is printed, decompression is done anyway, and exit status is 2. This used to work slightly before 5.0.0. In practice this bug matters only if xz has been built with some check types disabled. As instructed in PACKAGERS, such builds should be done in special situations only.
- Fix "xz -dc --single-stream tests/files/good-0-empty.xz" which failed with "Internal error (bug)". That is, --single-stream was broken if the first .xz stream in the input file didn't contain any uncompressed data.
- Fix displaying file sizes in the progress indicator when working in passthru mode and there are multiple input files. Just like "gzip -cdf", "xz -cdf" works like "cat" when the input file isn't a supported compressed file format. In this case the file size counters weren't reset between files so with multiple input files the progress indicator displayed an incorrect (too large) value.
- liblzma:
- API docs in lzma/container.h:
- Update the list of decoder flags in the decoder function docs.
- Explain LZMA_CONCATENATED behavior with .lzma files in lzma_auto_decoder() docs.
- OpenBSD: Use HW_NCPUONLINE to detect the number of available hardware threads in lzma_physmem().
- Fix use of wrong macro to detect x86 SSE2 support. __SSE2_MATH__ was used with GCC/Clang but the correct one is __SSE2__. The first one means that SSE2 is used for floating point math which is irrelevant here. The affected SSE2 code isn't used on x86-64 so this affects only 32-bit x86 builds that use -msse2 without -mfpmath=sse (there is no runtime detection for SSE2). It improves LZMA compression speed (not decompression).
- Fix the build with Intel C compiler 2021 (ICC, not ICX) on Linux. It defines __GNUC__ to 10 but doesn't support the __symver__ attribute introduced in GCC 10.
- Scripts: Ignore warnings from xz by using --quiet --no-warn. This is needed if the input .xz files use an unsupported check type.
- Translations:
- Updated Croatian and Turkish translations.
- One new translations wasn't included because it needed technical fixes. It will be in upcoming 5.4.0. No new translations will be added to the 5.2.x branch anymore.
- Renamed the French man page translation file from fr_FR.po to fr.po and thus also its install directory (like /usr/share/man/fr_FR -> .../fr).
- Man page translations for upcoming 5.4.0 are now handled in the Translation Project.
- Update doc/faq.txt a little so it's less out-of-date.
5.2.7 (2022-09-30)
- liblzma:
- Made lzma_filters_copy() to never modify the destination array if an error occurs. lzma_stream_encoder() and lzma_stream_encoder_mt() already assumed this. Before this change, if a tiny memory allocation in lzma_filters_copy() failed it would lead to a crash (invalid free() or invalid memory reads) in the cleanup paths of these two encoder initialization functions.
- Added missing integer overflow check to lzma_index_append(). This affects xz --list and other applications that decode the Index field from .xz files using lzma_index_decoder(). Normal decompression of .xz files doesn't call this code and thus most applications using liblzma aren't affected by this bug.
- Single-threaded .xz decoder (lzma_stream_decoder()): If lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible to use lzma_memlimit_set() to increase the limit and continue decoding. This was supposed to work from the beginning but there was a bug. With other decoders (.lzma or threaded .xz decoder) this already worked correctly.
- Fixed accumulation of integrity check type statistics in lzma_index_cat(). This bug made lzma_index_checks() return only the type of the integrity check of the last Stream when multiple lzma_indexes were concatenated. Most applications don't use these APIs but in xz it made xz --list not list all check types from concatenated .xz files. In xz --list --verbose only the per-file "Check:" lines were affected and in xz --robot --list only the "file" line was affected.
- Added ABI compatibility with executables that were linked against liblzma in RHEL/CentOS 7 or other liblzma builds that had copied the problematic patch from RHEL/CentOS 7 (xz-5.2.2-compat-libs.patch). For the details, see the comment at the top of src/liblzma/validate_map.sh. WARNING: This uses __symver__ attribute with GCC >= 10. In other cases the traditional __asm__(".symver ...") is used. Using link-time optimization (LTO, -flto) with GCC versions older than 10 can silently result in broken liblzma.so.5 (incorrect symbol versions)! If you want to use -flto with GCC, you must use GCC >= 10. LTO with Clang seems to work even with the traditional __asm__(".symver ...") method.
- xzgrep: Fixed compatibility with old shells that break if comments inside command substitutions have apostrophes ('). This problem was introduced in 5.2.6.
- Build systems:
- New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
- Windows: Fixed liblzma.dll build with Visual Studio project files. It broke in 5.2.6 due to a change that was made to improve CMake support.
- Windows: Building liblzma with UNICODE defined should now work.
- CMake files are now actually included in the release tarball. They should have been in 5.2.5 already.
- Minor CMake fixes and improvements.
- Added a new translation: Turkish
5.2.6 (2022-08-12)
- xz:
- The --keep option now accepts symlinks, hardlinks, and setuid, setgid, and sticky files. Previously this required using --force.
- When copying metadata from the source file to the destination file, don't try to set the group (GID) if it is already set correctly. This avoids a failure on OpenBSD (and possibly on a few other OSes) where files may get created so that their group doesn't belong to the user, and fchown(2) can fail even if it needs to do nothing.
- Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on MIPS32 because on MIPS32 userspace processes are limited to 2 GiB of address space.
- liblzma:
- Fixed a missing error-check in the threaded encoder. If a small memory allocation fails, a .xz file with an invalid Index field would be created. Decompressing such a file would produce the correct output but result in an error at the end. Thus this is a "mild" data corruption bug. Note that while a failed memory allocation can trigger the bug, it cannot cause invalid memory access.
- The decoder for .lzma files now supports files that have uncompressed size stored in the header and still use the end of payload marker (end of stream marker) at the end of the LZMA stream. Such files are rare but, according to the documentation in LZMA SDK, they are valid. doc/lzma-file-format.txt was updated too.
- Improved 32-bit x86 assembly files:
- Support Intel Control-flow Enforcement Technology (CET)
- Use non-executable stack on FreeBSD.
- Visual Studio: Use non-standard _MSVC_LANG to detect C++ standard version in the lzma.h API header. It's used to detect when "noexcept" can be used.
- xzgrep:
- Fixed arbitrary command injection via a malicious filename (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for this was released to the public on 2022-04-07. A slight robustness improvement has been made since then and, if using GNU or *BSD grep, a new faster method is now used that doesn't use the old sed-based construct at all. This also fixes bad output with GNU grep >= 3.5 (2020-09-27) when xzgrepping binary files. This vulnerability was discovered by: cleemy desu wayo working with Trend Micro Zero Day Initiative
- Fixed detection of corrupt .bz2 files.
- Improved error handling to fix exit status in some situations and to fix handling of signals: in some situations a signal didn't make xzgrep exit when it clearly should have. It's possible that the signal handling still isn't quite perfect but hopefully it's good enough.
- Documented exit statuses on the man page.
- xzegrep and xzfgrep now use "grep -E" and "grep -F" instead of the deprecated egrep and fgrep commands.
- Fixed parsing of the options -E, -F, -G, -P, and -X. The problem occurred when multiple options were specied in a single argument, for example, echo foo | xzgrep -Fe foo treated foo as a filename because -Fe wasn't correctly split into -F -e.
- Added zstd support.
- xzdiff/xzcmp:
- Fixed wrong exit status. Exit status could be 2 when the correct value is 1.
- Documented on the man page that exit status of 2 is used for decompression errors.
- Added zstd support.
- xzless:
- Fix less(1) version detection. It failed if the version number from "less -V" contained a dot.
- Translations:
- Added new translations: Catalan, Croatian, Esperanto, Korean, Portuguese, Romanian, Serbian, Spanish, Swedish, and Ukrainian
- Updated the Brazilian Portuguese translation.
- Added French man page translation. This and the existing German translation aren't complete anymore because the English man pages got a few updates and the translators weren't reached so that they could update their work.
- Build systems:
- Windows: Fix building of resource files when config.h isn't used. CMake + Visual Studio can now build liblzma.dll.
- Various fixes to the CMake support. Building static or shared liblzma should work fine in most cases. In contrast, building the command line tools with CMake is still clearly incomplete and experimental and should be used for testing only.
Signed-off-by: Adolf Belka adolf.belka@ipfire.org
config/rootfiles/common/xz | 37 ++++++++-- lfs/xz | 5 +- src/patches/xzgrep-ZDI-CAN-16587.patch | 94 -------------------------- 3 files changed, 32 insertions(+), 104 deletions(-) delete mode 100644 src/patches/xzgrep-ZDI-CAN-16587.patch
diff --git a/config/rootfiles/common/xz b/config/rootfiles/common/xz index c7bd0b302..d2f1d44cc 100644 --- a/config/rootfiles/common/xz +++ b/config/rootfiles/common/xz @@ -41,7 +41,7 @@ usr/bin/xzmore #usr/lib/liblzma.la usr/lib/liblzma.so usr/lib/liblzma.so.5 -usr/lib/liblzma.so.5.2.5 +usr/lib/liblzma.so.5.2.8 #usr/lib/pkgconfig/liblzma.pc #usr/share/doc/xz #usr/share/doc/xz/AUTHORS @@ -65,15 +65,26 @@ usr/lib/liblzma.so.5.2.5 #usr/share/doc/xz/history.txt #usr/share/doc/xz/lzma-file-format.txt #usr/share/doc/xz/xz-file-format.txt +#usr/share/locale/ca/LC_MESSAGES/xz.mo #usr/share/locale/cs/LC_MESSAGES/xz.mo #usr/share/locale/da/LC_MESSAGES/xz.mo #usr/share/locale/de/LC_MESSAGES/xz.mo +#usr/share/locale/eo/LC_MESSAGES/xz.mo +#usr/share/locale/es/LC_MESSAGES/xz.mo #usr/share/locale/fi/LC_MESSAGES/xz.mo #usr/share/locale/fr/LC_MESSAGES/xz.mo +#usr/share/locale/hr/LC_MESSAGES/xz.mo #usr/share/locale/hu/LC_MESSAGES/xz.mo #usr/share/locale/it/LC_MESSAGES/xz.mo +#usr/share/locale/ko/LC_MESSAGES/xz.mo #usr/share/locale/pl/LC_MESSAGES/xz.mo +#usr/share/locale/pt/LC_MESSAGES/xz.mo #usr/share/locale/pt_BR/LC_MESSAGES/xz.mo +#usr/share/locale/ro/LC_MESSAGES/xz.mo +#usr/share/locale/sr/LC_MESSAGES/xz.mo +#usr/share/locale/sv/LC_MESSAGES/xz.mo +#usr/share/locale/tr/LC_MESSAGES/xz.mo +#usr/share/locale/uk/LC_MESSAGES/xz.mo #usr/share/locale/vi/LC_MESSAGES/xz.mo #usr/share/locale/zh_CN/LC_MESSAGES/xz.mo #usr/share/locale/zh_TW/LC_MESSAGES/xz.mo @@ -82,9 +93,6 @@ usr/lib/liblzma.so.5.2.5 #usr/share/man/de/man1/lzcat.1 #usr/share/man/de/man1/lzcmp.1 #usr/share/man/de/man1/lzdiff.1 -#usr/share/man/de/man1/lzegrep.1 -#usr/share/man/de/man1/lzfgrep.1 -#usr/share/man/de/man1/lzgrep.1 #usr/share/man/de/man1/lzless.1 #usr/share/man/de/man1/lzma.1 #usr/share/man/de/man1/lzmadec.1 @@ -96,11 +104,26 @@ usr/lib/liblzma.so.5.2.5 #usr/share/man/de/man1/xzcmp.1 #usr/share/man/de/man1/xzdec.1 #usr/share/man/de/man1/xzdiff.1 -#usr/share/man/de/man1/xzegrep.1 -#usr/share/man/de/man1/xzfgrep.1 -#usr/share/man/de/man1/xzgrep.1 #usr/share/man/de/man1/xzless.1 #usr/share/man/de/man1/xzmore.1 +#usr/share/man/fr +#usr/share/man/fr/man1 +#usr/share/man/fr/man1/lzcat.1 +#usr/share/man/fr/man1/lzcmp.1 +#usr/share/man/fr/man1/lzdiff.1 +#usr/share/man/fr/man1/lzless.1 +#usr/share/man/fr/man1/lzma.1 +#usr/share/man/fr/man1/lzmadec.1 +#usr/share/man/fr/man1/lzmore.1 +#usr/share/man/fr/man1/unlzma.1 +#usr/share/man/fr/man1/unxz.1 +#usr/share/man/fr/man1/xz.1 +#usr/share/man/fr/man1/xzcat.1 +#usr/share/man/fr/man1/xzcmp.1 +#usr/share/man/fr/man1/xzdec.1 +#usr/share/man/fr/man1/xzdiff.1 +#usr/share/man/fr/man1/xzless.1 +#usr/share/man/fr/man1/xzmore.1 #usr/share/man/man1/lzcat.1 #usr/share/man/man1/lzcmp.1 #usr/share/man/man1/lzdiff.1 diff --git a/lfs/xz b/lfs/xz index 9345df954..83a724e1a 100644 --- a/lfs/xz +++ b/lfs/xz @@ -24,7 +24,7 @@
include Config
-VER = 5.2.5 +VER = 5.2.8
THISAPP = xz-$(VER) DL_FILE = $(THISAPP).tar.xz @@ -45,7 +45,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_BLAKE2 = 9b9b58e33722ecfe799bb50f3ffe4e86386f734ab4468eb54ff92771ddf899302d1ffa4d88bdb0de351fc3eab8a6ea103f00d7e79f33db879fe22b2e1a7e62db +$(DL_FILE)_BLAKE2 = 44d1ddd783b2527f3b17481fc277b671808eb5639c10d31bfaca9fd29ac4413628654ecb9e207955a9477c83eb30f61cf5607cd9a49dd71732707731e4444ace
install : $(TARGET)
@@ -75,7 +75,6 @@ $(subst %,%_BLAKE2,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
- cd $(DIR_APP) && patch -p1 -i $(DIR_SRC)/src/patches/xzgrep-ZDI-CAN-16587.patch
cd $(DIR_APP) && ./configure --prefix=$(PREFIX) cd $(DIR_APP) && make $(MAKETUNING) cd $(DIR_APP) && make install diff --git a/src/patches/xzgrep-ZDI-CAN-16587.patch b/src/patches/xzgrep-ZDI-CAN-16587.patch deleted file mode 100644 index 406ded590..000000000 --- a/src/patches/xzgrep-ZDI-CAN-16587.patch +++ /dev/null @@ -1,94 +0,0 @@ -From 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 Mon Sep 17 00:00:00 2001 -From: Lasse Collin lasse.collin@tukaani.org -Date: Tue, 29 Mar 2022 19:19:12 +0300 -Subject: [PATCH] xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587).
-Malicious filenames can make xzgrep to write to arbitrary files -or (with a GNU sed extension) lead to arbitrary code execution.
-xzgrep from XZ Utils versions up to and including 5.2.5 are -affected. 5.3.1alpha and 5.3.2alpha are affected as well. -This patch works for all of them.
-This bug was inherited from gzip's zgrep. gzip 1.12 includes -a fix for zgrep.
-The issue with the old sed script is that with multiple newlines, -the N-command will read the second line of input, then the -s-commands will be skipped because it's not the end of the -file yet, then a new sed cycle starts and the pattern space -is printed and emptied. So only the last line or two get escaped.
-One way to fix this would be to read all lines into the pattern -space first. However, the included fix is even simpler: All lines -except the last line get a backslash appended at the end. To ensure -that shell command substitution doesn't eat a possible trailing -newline, a colon is appended to the filename before escaping. -The colon is later used to separate the filename from the grep -output so it is fine to add it here instead of a few lines later.
-The old code also wasn't POSIX compliant as it used \n in the -replacement section of the s-command. Using <newline> is the -POSIX compatible method.
-LC_ALL=C was added to the two critical sed commands. POSIX sed -manual recommends it when using sed to manipulate pathnames -because in other locales invalid multibyte sequences might -cause issues with some sed implementations. In case of GNU sed, -these particular sed scripts wouldn't have such problems but some -other scripts could have, see:
- info '(sed)Locale Considerations'
-This vulnerability was discovered by: -cleemy desu wayo working with Trend Micro Zero Day Initiative
-Thanks to Jim Meyering and Paul Eggert discussing the different -ways to fix this and for coordinating the patch release schedule
-with gzip.
- src/scripts/xzgrep.in | 20 ++++++++++++--------
- 1 file changed, 12 insertions(+), 8 deletions(-)
-diff --git a/src/scripts/xzgrep.in b/src/scripts/xzgrep.in -index b180936..e5186ba 100644 ---- a/src/scripts/xzgrep.in -+++ b/src/scripts/xzgrep.in -@@ -180,22 +180,26 @@ for i; do
{ test $# -eq 1 || test $no_filename -eq 1; }; theneval "$grep"else-+ # Append a colon so that the last character will never be a newline -+ # which would otherwise get lost in shell command substitution. -+ i="$i:" -+ -+ # Escape & \ | and newlines only if such characters are present -+ # (speed optimization).
case $i in(*'- '* | *'&'* | *''* | *'|'*)
-- i=$(printf '%s\n' "$i" | -- sed ' -- $!N -- $s/[&|]/\&/g -- $s/\n/\n/g -- ');; -+ i=$(printf '%s\n' "$i" | LC_ALL=C sed 's/[&|]/\&/g; $!s/$/\/');;
esac-- sed_script="s|^|$i:|" -+ -+ # $i already ends with a colon so don't add it here. -+ sed_script="s|^|$i|"
# Fail if grep or sed fails.r=$(exec 4>&1-- (eval "$grep" 4>&-; echo $? >&4) 3>&- | sed "$sed_script" >&3 4>&- -+ (eval "$grep" 4>&-; echo $? >&4) 3>&- | -+ LC_ALL=C sed "$sed_script" >&3 4>&-
) || r=2exit $rfi >&3 5>&-
-2.35.1