From: Adolf Belka <adolf.belka@ipfire.org>
To: development@lists.ipfire.org
Subject: [PATCH] xz: Update to version 5.4.0
Date: Sat, 17 Dec 2022 13:14:27 +0100 [thread overview]
Message-ID: <20221217121427.3532488-8-adolf.belka@ipfire.org> (raw)
In-Reply-To: <20221217121427.3532488-1-adolf.belka@ipfire.org>
[-- Attachment #1: Type: text/plain, Size: 15216 bytes --]
- Update from version 5.2.8 to 5.4.0
- Update of rootfile
- Changelog
5.4.0 (2022-12-13)
This bumps the minor version of liblzma because new features were
added. The API and ABI are still backward compatible with liblzma
5.2.x and 5.0.x.
Since 5.3.5beta:
* All fixes from 5.2.10.
* The ARM64 filter is now stable. The xz option is now --arm64.
Decompression requires XZ Utils 5.4.0. In the future the ARM64
filter will be supported by XZ for Java, XZ Embedded (including
the version in Linux), LZMA SDK, and 7-Zip.
* Translations:
- Updated Catalan, Croatian, German, Romanian, and Turkish
translations.
- Updated German man page translations.
- Added Romanian man page translations.
Summary of new features added in the 5.3.x development releases:
* liblzma:
- Added threaded .xz decompressor lzma_stream_decoder_mt().
It can use multiple threads with .xz files that have multiple
Blocks with size information in Block Headers. The threaded
encoder in xz has always created such files.
Single-threaded encoder cannot store the size information in
Block Headers even if one used LZMA_FULL_FLUSH to create
multiple Blocks, so this threaded decoder cannot use multiple
threads with such files.
If there are multiple Streams (concatenated .xz files), one
Stream will be decompressed completely before starting the
next Stream.
- A new decoder flag LZMA_FAIL_FAST was added. It makes the
threaded decompressor report errors soon instead of first
flushing all pending data before the error location.
- New Filter IDs:
* LZMA_FILTER_ARM64 is for ARM64 binaries.
* LZMA_FILTER_LZMA1EXT is for raw LZMA1 streams that don't
necessarily use the end marker.
- Added lzma_str_to_filters(), lzma_str_from_filters(), and
lzma_str_list_filters() to convert a preset or a filter chain
string to a lzma_filter[] and vice versa. These should make
it easier to write applications that allow users to specify
custom compression options.
- Added lzma_filters_free() which can be convenient for freeing
the filter options in a filter chain (an array of lzma_filter
structures).
- lzma_file_info_decoder() to makes it a little easier to get
the Index field from .xz files. This helps in getting the
uncompressed file size but an easy-to-use random access
API is still missing which has existed in XZ for Java for
a long time.
- Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
It is used by erofs-utils and may be used by others too.
The MicroLZMA format is a raw LZMA stream (without end marker)
whose first byte (always 0x00) has been replaced with
bitwise-negation of the LZMA properties (lc/lp/pb). It was
created for use in EROFS but may be used in other contexts
as well where it is important to avoid wasting bytes for
stream headers or footers. The format is also supported by
XZ Embedded (the XZ Embedded version in Linux got MicroLZMA
support in Linux 5.16).
The MicroLZMA encoder API in liblzma can compress into a
fixed-sized output buffer so that as much data is compressed
as can be fit into the buffer while still creating a valid
MicroLZMA stream. This is needed for EROFS.
- Added lzma_lzip_decoder() to decompress the .lz (lzip) file
format version 0 and the original unextended version 1 files.
Also lzma_auto_decoder() supports .lz files.
- lzma_filters_update() can now be used with the multi-threaded
encoder (lzma_stream_encoder_mt()) to change the filter chain
after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
- In lzma_options_lzma, allow nice_len = 2 and 3 with the match
finders that require at least 3 or 4. Now it is internally
rounded up if needed.
- CLMUL-based CRC64 on x86-64 and E2K with runtime processor
detection. On 32-bit x86 it currently isn't available unless
--disable-assembler is used which can make the non-CLMUL
CRC64 slower; this might be fixed in the future.
- Building with --disable-threads --enable-small
is now thread-safe if the compiler supports
__attribute__((__constructor__)).
* xz:
- Using -T0 (--threads=0) will now use multi-threaded encoder
even on a single-core system. This is to ensure that output
from the same xz binary is identical on both single-core and
multi-core systems.
- --threads=+1 or -T+1 is now a way to put xz into
multi-threaded mode while using only one worker thread.
The + is ignored if the number is not 1.
- A default soft memory usage limit is now used for compression
when -T0 is used and no explicit limit has been specified.
This soft limit is used to restrict the number of threads
but if the limit is exceeded with even one thread then xz
will continue with one thread using the multi-threaded
encoder and this limit is ignored. If the number of threads
is specified manually then no default limit will be used;
this affects only -T0.
This change helps on systems that have very many cores and
using all of them for xz makes no sense. Previously xz -T0
could run out of memory on such systems because it attempted
to reserve memory for too many threads.
This also helps with 32-bit builds which don't have a large
amount of address space that would be required for many
threads. The default soft limit for -T0 is at most 1400 MiB
on all 32-bit platforms.
- Previously a low value in --memlimit-compress wouldn't cause
xz to switch from multi-threaded mode to single-threaded mode
if the limit cannot otherwise be met; xz failed instead. Now
xz can switch to single-threaded mode and then, if needed,
scale down the LZMA2 dictionary size too just like it already
did when it was started in single-threaded mode.
- The option --no-adjust no longer prevents xz from scaling down
the number of threads as that doesn't affect the compressed
output (only performance). Now --no-adjust only prevents
adjustments that affect compressed output, that is, with
--no-adjust xz won't switch from multi-threaded mode to
single-threaded mode and won't scale down the LZMA2
dictionary size.
- Added a new option --memlimit-mt-decompress=LIMIT. This is
used to limit the number of decompressor threads (possibly
falling back to single-threaded mode) but it will never make
xz refuse to decompress a file. This has a system-specific
default value because without any limit xz could end up
allocating memory for the whole compressed input file, the
whole uncompressed output file, multiple thread-specific
decompressor instances and so on. Basically xz could
attempt to use an insane amount of memory even with fairly
common files. The system-specific default value is currently
the same as the one used for compression with -T0.
The new option works together with the existing option
--memlimit-decompress=LIMIT. The old option sets a hard limit
that must not be exceeded (xz will refuse to decompress)
while the new option only restricts the number of threads.
If the limit set with --memlimit-mt-decompress is greater
than the limit set with --memlimit-compress, then the latter
value is used also for --memlimit-mt-decompress.
- Added new information to the output of xz --info-memory and
new fields to the output of xz --robot --info-memory.
- In --lzma2=nice=NUMBER allow 2 and 3 with all match finders
now that liblzma handles it.
- Don't mention endianness for ARM and ARM-Thumb filters in
--long-help. The filters only work for little endian
instruction encoding but modern ARM processors using
big endian data access still use little endian
instruction encoding. So the help text was misleading.
In contrast, the PowerPC filter is only for big endian
32/64-bit PowerPC code. Little endian PowerPC would need
a separate filter.
- Added decompression support for the .lz (lzip) file format
version 0 and the original unextended version 1. It is
autodetected by default. See also the option --format on
the xz man page.
- Sandboxing enabled by default:
* Capsicum (FreeBSD)
* pledge(2) (OpenBSD)
* Scripts now support the .lz format using xz.
* A few new tests were added.
* The liblzma-specific tests are now supported in CMake-based
builds too ("make test").
5.2.10 (2022-12-13)
* xz: Don't modify argv[] when parsing the --memlimit* and
--block-list command line options. This fixes confusing
arguments in process listing (like "ps auxf").
* GNU/Linux only: Use __has_attribute(__symver__) to detect if
that attribute is supported. This fixes build on Mandriva where
Clang is patched to define __GNUC__ to 11 by default (instead
of 4 as used by Clang upstream).
5.2.9 (2022-11-30)
* liblzma:
- Fixed an infinite loop in LZMA encoder initialization
if dict_size >= 2 GiB. (The encoder only supports up
to 1536 MiB.)
- Fixed two cases of invalid free() that can happen if
a tiny allocation fails in encoder re-initialization
or in lzma_filters_update(). These bugs had some
similarities with the bug fixed in 5.2.7.
- Fixed lzma_block_encoder() not allowing the use of
LZMA_SYNC_FLUSH with lzma_code() even though it was
documented to be supported. The sync-flush code in
the Block encoder was already used internally via
lzma_stream_encoder(), so this was just a missing flag
in the lzma_block_encoder() API function.
- GNU/Linux only: Don't put symbol versions into static
liblzma as it breaks things in some cases (and even if
it didn't break anything, symbol versions in static
libraries are useless anyway). The downside of the fix
is that if the configure options --with-pic or --without-pic
are used then it's not possible to build both shared and
static liblzma at the same time on GNU/Linux anymore;
with those options --disable-static or --disable-shared
must be used too.
* New email address for bug reports is <xz(a)tukaani.org> which
forwards messages to Lasse Collin and Jia Tan.
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
config/rootfiles/common/xz | 32 +++++++++++++++++++++++++++++++-
lfs/xz | 6 +++---
2 files changed, 34 insertions(+), 4 deletions(-)
diff --git a/config/rootfiles/common/xz b/config/rootfiles/common/xz
index d2f1d44cc..1e4a43729 100644
--- a/config/rootfiles/common/xz
+++ b/config/rootfiles/common/xz
@@ -41,7 +41,7 @@ usr/bin/xzmore
#usr/lib/liblzma.la
usr/lib/liblzma.so
usr/lib/liblzma.so.5
-usr/lib/liblzma.so.5.2.8
+usr/lib/liblzma.so.5.4.0
#usr/lib/pkgconfig/liblzma.pc
#usr/share/doc/xz
#usr/share/doc/xz/AUTHORS
@@ -93,6 +93,9 @@ usr/lib/liblzma.so.5.2.8
#usr/share/man/de/man1/lzcat.1
#usr/share/man/de/man1/lzcmp.1
#usr/share/man/de/man1/lzdiff.1
+#usr/share/man/de/man1/lzegrep.1
+#usr/share/man/de/man1/lzfgrep.1
+#usr/share/man/de/man1/lzgrep.1
#usr/share/man/de/man1/lzless.1
#usr/share/man/de/man1/lzma.1
#usr/share/man/de/man1/lzmadec.1
@@ -104,6 +107,9 @@ usr/lib/liblzma.so.5.2.8
#usr/share/man/de/man1/xzcmp.1
#usr/share/man/de/man1/xzdec.1
#usr/share/man/de/man1/xzdiff.1
+#usr/share/man/de/man1/xzegrep.1
+#usr/share/man/de/man1/xzfgrep.1
+#usr/share/man/de/man1/xzgrep.1
#usr/share/man/de/man1/xzless.1
#usr/share/man/de/man1/xzmore.1
#usr/share/man/fr
@@ -147,3 +153,27 @@ usr/lib/liblzma.so.5.2.8
#usr/share/man/man1/xzgrep.1
#usr/share/man/man1/xzless.1
#usr/share/man/man1/xzmore.1
+#usr/share/man/ro
+#usr/share/man/ro/man1
+#usr/share/man/ro/man1/lzcat.1
+#usr/share/man/ro/man1/lzcmp.1
+#usr/share/man/ro/man1/lzdiff.1
+#usr/share/man/ro/man1/lzegrep.1
+#usr/share/man/ro/man1/lzfgrep.1
+#usr/share/man/ro/man1/lzgrep.1
+#usr/share/man/ro/man1/lzless.1
+#usr/share/man/ro/man1/lzma.1
+#usr/share/man/ro/man1/lzmadec.1
+#usr/share/man/ro/man1/lzmore.1
+#usr/share/man/ro/man1/unlzma.1
+#usr/share/man/ro/man1/unxz.1
+#usr/share/man/ro/man1/xz.1
+#usr/share/man/ro/man1/xzcat.1
+#usr/share/man/ro/man1/xzcmp.1
+#usr/share/man/ro/man1/xzdec.1
+#usr/share/man/ro/man1/xzdiff.1
+#usr/share/man/ro/man1/xzegrep.1
+#usr/share/man/ro/man1/xzfgrep.1
+#usr/share/man/ro/man1/xzgrep.1
+#usr/share/man/ro/man1/xzless.1
+#usr/share/man/ro/man1/xzmore.1
diff --git a/lfs/xz b/lfs/xz
index 83a724e1a..1e4020dde 100644
--- a/lfs/xz
+++ b/lfs/xz
@@ -1,7 +1,7 @@
###############################################################################
# #
# IPFire.org - A linux based firewall #
-# Copyright (C) 2007-2020 IPFire Team <info(a)ipfire.org> #
+# Copyright (C) 2007-2022 IPFire Team <info(a)ipfire.org> #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@@ -24,7 +24,7 @@
include Config
-VER = 5.2.8
+VER = 5.4.0
THISAPP = xz-$(VER)
DL_FILE = $(THISAPP).tar.xz
@@ -45,7 +45,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_BLAKE2 = 44d1ddd783b2527f3b17481fc277b671808eb5639c10d31bfaca9fd29ac4413628654ecb9e207955a9477c83eb30f61cf5607cd9a49dd71732707731e4444ace
+$(DL_FILE)_BLAKE2 = 8896044ae2761561161f4dcfc1e8e0bc6a58a2a7784567156b23e589cdb80028b37655af65802a20d2631ef05c209e406424f061c0293458a41b16dbf75a4f29
install : $(TARGET)
--
2.39.0
next prev parent reply other threads:[~2022-12-17 12:14 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-17 12:14 [PATCH] iproute2: Update to version 6.1.0 Adolf Belka
2022-12-17 12:14 ` [PATCH] krb5: Update to version 1.20.1 Adolf Belka
2022-12-26 8:49 ` Peter Müller
2022-12-17 12:14 ` [PATCH] libarchive: Update to version 3.6.2 Adolf Belka
2022-12-26 8:52 ` Peter Müller
2022-12-17 12:14 ` [PATCH] libmpc: Update to version 1.3.1 Adolf Belka
2022-12-27 16:32 ` Peter Müller
2022-12-17 12:14 ` [PATCH] nano: Update to version 7.1 Adolf Belka
2022-12-27 16:35 ` Peter Müller
2022-12-17 12:14 ` [PATCH] psmisc: Update to version 23.6 Adolf Belka
2022-12-26 8:55 ` Peter Müller
2022-12-17 12:14 ` [PATCH] samba: Update to version 4.17.4 Adolf Belka
2022-12-17 12:14 ` Adolf Belka [this message]
2022-12-27 16:36 ` [PATCH] xz: Update to version 5.4.0 Peter Müller
2022-12-26 8:53 ` [PATCH] iproute2: Update to version 6.1.0 Peter Müller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221217121427.3532488-8-adolf.belka@ipfire.org \
--to=adolf.belka@ipfire.org \
--cc=development@lists.ipfire.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox