[PATCH 1/3] borgbackup: Update to version 1.2.3 and fix bug 13032

* [PATCH 1/3] borgbackup: Update to version 1.2.3 and fix bug 13032
@ 2023-01-21 18:03 Adolf Belka
  2023-01-21 18:03 ` [PATCH 2/3] python3-msgpack: Add comment about version link with borgbackup Adolf Belka
  2023-01-21 18:03 ` [PATCH 3/3] python3-packaging: Update to version 23.0 Adolf Belka
  0 siblings, 2 replies; 3+ messages in thread
From: Adolf Belka @ 2023-01-21 18:03 UTC (permalink / raw)
  To: development

[-- Attachment #1: Type: text/plain, Size: 23064 bytes --]

- Update from version 1.2.0 to 1.2.3
- Update of rootfile
- This update works with python3-msgpack-1.0.4 and fixes bug 13032
- To make it work then the borgbackup-1.2.3-py3.10.egg-info directory must be the only
   egg-info directory for borgbackup otherwise version 1.2.3 will end up with an error.
   Versions 1.2.2 and earlier workled without any problem if there was an earlier egg-info
   directory for a different version number. The borgbackup rootfile had the egg-info
   directory commented out so an uninstall cleared the directory but did not remove it.
   This patch has the egg-info directory in the rootfile uncommented and so an uninstall
   removes the directory.
- borgbackup paks files created so that the uninstall.sh file will remove any egg-info
   directory that starts with "borgbackup-1." as the first ever borgbackup was 1.0.12
   When the old 1.2.0 or earlier borgbackup is uninstalled it will use the old default
   paks uninstall.sh file and rootfile which will leave the old egg-info directory in
   place. When version 1.2.3 is installed it will use the new install.sh script which
   will remove any existing egg-info directories present still.
- Changelog
Version 1.2.3 (2022-12-24)
Upgrade notes:
Some things can be recommended for the upgrade process from borg 1.1.x (please also read the important compatibility notes below):
    do you already want to upgrade? 1.1.x also will get fixes for a while.
    be careful, first upgrade your less critical / smaller repos.
    first upgrade to a recent 1.1.x release - especially if you run some older 1.1.* or even 1.0.* borg release.
    using that, run at least one borg create (your normal backup), prune and especially a check to see everything is in a good state.
    check the output of borg check - if there is anything special, consider a borg check --repair followed by another borg check.
    if everything is fine so far (borg check reports no issues), you can consider upgrading to 1.2.x. if not, please first fix any already existing issue.
    if you want to play safer, first create a backup of your borg repository.
    upgrade to latest borg 1.2.x release (you could use the fat binary from github releases page)
    run borg compact --cleanup-commits to clean up a ton of 17 bytes long files in your repo caused by a borg 1.1 bug
    run borg check again (now with borg 1.2.x) and check if there is anything special.
    run borg info (with borg 1.2.x) to build the local pre12-meta cache (can take significant time, but after that it will be fast) - for more details see below.
    check the compatibility notes (see below) and adapt your scripts, if needed.
    if you run into any issues, please check the github issue tracker before posting new issues there or elsewhere.
If you follow this procedure, you can help avoiding that we get a lot of “borg 1.2” issue reports that are not really 1.2 issues, but existed before and maybe just were not noticed.
Compatibility notes:
    matching of path patterns has been aligned with borg storing relative paths. Borg archives file paths without leading slashes. Previously, include/exclude patterns could contain leading slashes. You should check your patterns and remove leading slashes.
    dropped support / testing for older Pythons, minimum requirement is 3.8. In case your OS does not provide Python >= 3.8, consider using our binary, which does not need an external Python interpreter. Or continue using borg 1.1.x, which is still supported.
    freeing repository space only happens when “borg compact” is invoked.
    mount: the default for --numeric-ids is False now (same as borg extract)
    borg create --noatime is deprecated. Not storing atime is the default behaviour now (use --atime if you want to store the atime).
    --prefix is deprecated, use -a / --glob-archives, see #6806
    list: corrected mix-up of “isomtime” and “mtime” formats. Previously, “isomtime” was the default but produced a verbose human format, while “mtime” produced a ISO-8601-like format. The behaviours have been swapped (so “mtime” is human, “isomtime” is ISO-like), and the default is now “mtime”. “isomtime” is now a real ISO-8601 format (“T” between date and time, not a space).
    create/recreate --list: file status for all files used to get announced AFTER the file (with borg < 1.2). Now, file status is announced BEFORE the file contents are processed. If the file status changes later (e.g. due to an error or a content change), the updated/final file status will be printed again.
    removed deprecated-since-long stuff (deprecated since):
        command “borg change-passphrase” (2017-02), use “borg key …”
        option “--keep-tag-files” (2017-01), use “--keep-exclude-tags”
        option “--list-format” (2017-10), use “--format”
        option “--ignore-inode” (2017-09), use “--files-cache” w/o “inode”
        option “--no-files-cache” (2017-09), use “--files-cache=disabled”
    removed BORG_HOSTNAME_IS_UNIQUE env var. to use borg you must implement one of these 2 scenarios:
            the combination of FQDN and result of uuid.getnode() must be unique and stable (this should be the case for almost everybody, except when having duplicate FQDN and MAC address or all-zero MAC address)
            if you are aware that 1) is not the case for you, you must set BORG_HOST_ID env var to something unique.
    exit with 128 + signal number, #5161. if you have scripts expecting rc == 2 for a signal exit, you need to update them to check for >= 128.
Fixes:
    create: fix --list --dry-run output for directories, #7209
    diff/recreate: normalize chunker params before comparing them, #7079
    check: fix uninitialised variable if repo is completely empty, #7034
    xattrs: improve error handling, #6988
    fix args.paths related argparsing, #6994
    archive.save(): always use metadata from stats (e.g. nfiles, size, …), #7072
    tar_filter: recognize .tar.zst as zstd, #7093
    get_chunker: fix missing sparse=False argument, #7056
    file_integrity.py: make sure file_fd is always closed on exit
    repository: cleanup(): close segment before unlinking
    repository: use os.replace instead of os.rename
Other changes:
    remove python < 3.7 compatibility code
    do not use version_tuple placeholder in setuptools_scm template
    CI: fix tox4 passenv issue, #7199
    vagrant: update to python 3.9.16, use the openbsd 7.1 box
    misc. test suite and docs fixes / improvements
    remove deprecated --prefix from docs, #7109
    Windows: use MSYS2 for Github CI, remove Appveyor CI
Version 1.2.2 (2022-08-20)
New features:
    prune/delete --checkpoint-interval=1800 and ctrl-c/SIGINT support, #6284
Fixes:
    SaveFile: use a custom mkstemp with mode support, #6933, #6400, #6786. This fixes umask/mode/ACL issues (and also “chmod not supported” exceptions seen in 1.2.1) of files updated using SaveFile, e.g. the repo config.
    hashindex_compact: fix eval order (check idx before use), #5899
    create --paths-from-(stdin|command): normalize paths, #6778
    secure_erase: avoid collateral damage, #6768. If a hardlink copy of a repo was made and a new repo config shall be saved, do NOT fill in random garbage before deleting the previous repo config, because that would damage the hardlink copy.
    list: fix {flags:<WIDTH>} formatting, #6081
    check: try harder to create the key, #5719
    misc commands: ctrl-c must not kill other subprocesses, #6912
        borg create with a remote repo via ssh
        borg create --content-from-command
        borg create --paths-from-command
        (de)compression filter process of import-tar / export-tar
Other changes:
    deprecate --prefix, use -a / --glob-archives, see #6806
    make setuptools happy (“package would be ignored”), #6874
    fix pyproject.toml to create a fixed _version.py file, compatible with both old and new setuptools_scm version, #6875
    automate asciinema screencasts
    CI: test on macOS 12 without fuse / fuse tests (too troublesome on github CI due to kernel extensions needed by macFUSE)
    tests: fix test_obfuscate byte accounting
    repository: add debug logging for issue #6687
    _chunker.c: fix warnings on macOS
    requirements.lock.txt: use the latest cython 0.29.32
    docs:
        add info on man page installation, #6894
        update archive_progress json description about “finished”, #6570
        json progress_percent: some values are optional, #4074
        FAQ: full quota / full disk, #5960
        correct shell syntax for installation using git
Version 1.2.1 (2022-06-06)
Fixes:
    create: skip with warning if opening the parent dir of recursion root fails, #6374
    create: fix crash. metadata stream can produce all-zero chunks, #6587
    fix crash when computing stats, escape % chars in archive name, #6500
    fix transaction rollback: use files cache filename as found in txn.active/, #6353
    import-tar: kill filter process in case of borg exceptions, #6401 #6681
    import-tar: fix mtime type bug
    ensure_dir: respect umask for created directory modes, #6400
    SaveFile: respect umask for final file mode, #6400
    check archive: improve error handling for corrupt archive metadata block, make robust_iterator more robust, #4777
    pre12-meta cache: do not use the cache if want_unique is True, #6612
    fix scp-style repo url parsing for ip v6 address, #6526
    mount -o versions: give clear error msg instead of crashing. it does not make sense to request versions view if you only look at 1 archive, but the code shall not crash in that case as it did, but give a clear error msg.
    show_progress: add finished=true/false to archive_progress json, #6570
    delete/prune: fix --iec mode output (decimal vs. binary units), #6606
    info: fix authenticated mode repo to show “Encrypted: No”, #6462
    diff: support presence change for blkdev, chrdev and fifo items, #6615
New features:
    delete: add repository id and location to prompt, #6453
    borg debug dump-repo-objs --ghost: new --segment=S --offset=O options
Other changes:
    support python 3.11
    allow msgpack 1.0.4, #6716
    load_key: no key is same as empty key, #6441
    give a more helpful error msg for unsupported key formats, #6561
    better error msg for defect or unsupported repo configs, #6566
    docs:
        document borg 1.2 pattern matching behavior change, #6407 Make clear that absolute paths always go into the matcher as if they are relative (without leading slash). Adapt all examples accordingly.
        authentication primitives: improved security and performance infos
        mention BORG_FILES_CACHE_SUFFIX as alternative to BORG_FILES_CACHE_TTL, #5602
        FAQ: add a hint about --debug-topic=files_cache
        improve borg check --max-duration description
        fix values of TAG bytes, #6515
        borg compact --cleanup-commits also runs a normal compaction, #6324
        virtualization speed tips
        recommend umask for passphrase file perms
        borg 1.2 is security supported
        update link to ubuntu packages, #6485
        use --numeric-ids in pull mode docs
        remove blake2 docs, blake2 code not bundled any more, #6371
        clarify on-disk order and size of segment file log entry fields, #6357
        docs building: do not transform --/--- to unicode dashes
    tests:
        check that borg does not require pytest for normal usage, fixes #6563
        fix OpenBSD symlink mode test failure, #2055
    vagrant:
        darwin64: remove fakeroot, #6314
        update development.lock.txt
        use pyinstaller 4.10 and python 3.9.13 for binary build
        upgrade VMCPUS and xdistn from 4 to 16, maybe this speeds up the tests
    crypto:
        use hmac.compare_digest instead of ==, #6470
        hmac_sha256: replace own cython wrapper code by hmac.digest python stdlib (since py38)
        hmac and blake2b minor optimizations and cleanups
        removed some unused crypto related code, #6472
        avoid losing the key (potential use-after-free). this never could happen in 1.2 due to the way we use the code. The issue was discovered in master after other changes, so we also “fixed” it here before it bites us.
    setup / build:
        add pyproject.toml, fix sys.path, #6466
        setuptools_scm: also require it via pyproject.toml
        allow extra compiler flags for every extension build
        fix misc. C / Cython compiler warnings, deprecation warnings
        fix zstd.h include for bundled zstd, #6369
    source using python 3.8 features: pyupgrade --py38-plus ./**/*.py

Fixes: Bug #13032
Tested-by: Adolf Belka <adolf.belka(a)ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
 config/rootfiles/packages/borgbackup | 17 +++++++++--------
 lfs/borgbackup                       |  9 +++++----
 src/paks/borgbackup/install.sh       | 28 ++++++++++++++++++++++++++++
 src/paks/borgbackup/uninstall.sh     | 27 +++++++++++++++++++++++++++
 src/paks/borgbackup/update.sh        | 27 +++++++++++++++++++++++++++
 5 files changed, 96 insertions(+), 12 deletions(-)
 create mode 100644 src/paks/borgbackup/install.sh
 create mode 100644 src/paks/borgbackup/uninstall.sh
 create mode 100644 src/paks/borgbackup/update.sh

diff --git a/config/rootfiles/packages/borgbackup b/config/rootfiles/packages/borgbackup
index b744b0b99..feaca4156 100644
--- a/config/rootfiles/packages/borgbackup
+++ b/config/rootfiles/packages/borgbackup
@@ -91,11 +91,12 @@ usr/lib/python3.10/site-packages/borg/testsuite/xattr.py
 usr/lib/python3.10/site-packages/borg/upgrader.py
 usr/lib/python3.10/site-packages/borg/version.py
 usr/lib/python3.10/site-packages/borg/xattr.py
-#usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/PKG-INFO
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/SOURCES.txt
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/dependency_links.txt
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/entry_points.txt
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/not-zip-safe
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/requires.txt
-usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/top_level.txt
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/PKG-INFO
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/SOURCES.txt
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/dependency_links.txt
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/entry_points.txt
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/not-zip-safe
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/requires.txt
+usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/top_level.txt
+
diff --git a/lfs/borgbackup b/lfs/borgbackup
index 9c761eac4..0cbb809b7 100644
--- a/lfs/borgbackup
+++ b/lfs/borgbackup
@@ -1,7 +1,7 @@
 ###############################################################################
 #                                                                             #
 # IPFire.org - A linux based firewall                                         #
-# Copyright (C) 2007-2020  IPFire Team  <info(a)ipfire.org>                     #
+# Copyright (C) 2007-2023  IPFire Team  <info(a)ipfire.org>                     #
 #                                                                             #
 # This program is free software: you can redistribute it and/or modify        #
 # it under the terms of the GNU General Public License as published by        #
@@ -24,7 +24,7 @@
 
 include Config
 
-VER        = 1.2.0
+VER        = 1.2.3
 SUMMARY    = Deduplicating backup program with compression and authenticated encryption
 
 THISAPP    = borgbackup-$(VER)
@@ -33,9 +33,10 @@ DL_FROM    = $(URL_IPFIRE)
 DIR_APP    = $(DIR_SRC)/$(THISAPP)
 TARGET     = $(DIR_INFO)/$(THISAPP)
 PROG       = borgbackup
-PAK_VER    = 13
+PAK_VER    = 14
 
 DEPS       = python3-pkgconfig python3-msgpack python3-packaging python3-pyfuse3
+# borgbackup only works with specific versions of python3-msgpack
 
 SERVICES   =
 
@@ -47,7 +48,7 @@ objects = $(DL_FILE)
 
 $(DL_FILE) = $(DL_FROM)/$(DL_FILE)
 
-$(DL_FILE)_BLAKE2 = 9e6cb8d85ca196cbdd6baba694bc7987d63f85d2ba0e25f3ac1e59400882fff71e29b04ca218ee78f23daeb52d13547062b0c80bd0d3f5b460b28b4f274d11ec
+$(DL_FILE)_BLAKE2 = 970fad62428b44eb22df761020a4c25f4aa8d3e8f5cc2c81aa96b1afb051dc24b3d124b4fc84e81cf8eaa6a9b19a200f6f9bda1f7f4c2794a7e8d33e8d5020f1
 
 install : $(TARGET)
 
diff --git a/src/paks/borgbackup/install.sh b/src/paks/borgbackup/install.sh
new file mode 100644
index 000000000..f8e61e684
--- /dev/null
+++ b/src/paks/borgbackup/install.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+############################################################################
+#                                                                          #
+# This file is part of the IPFire Firewall.                                #
+#                                                                          #
+# IPFire is free software; you can redistribute it and/or modify           #
+# it under the terms of the GNU General Public License as published by     #
+# the Free Software Foundation; either version 2 of the License, or        #
+# (at your option) any later version.                                      #
+#                                                                          #
+# IPFire is distributed in the hope that it will be useful,                #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of           #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the            #
+# GNU General Public License for more details.                             #
+#                                                                          #
+# You should have received a copy of the GNU General Public License        #
+# along with IPFire; if not, write to the Free Software                    #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA #
+#                                                                          #
+# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>.                        #
+#                                                                          #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+rm -rfv /usr/lib/python3.10/site-packages/borgbackup-1.*
+extract_files
+restore_backup ${NAME}
+start_service --background ${NAME}
diff --git a/src/paks/borgbackup/uninstall.sh b/src/paks/borgbackup/uninstall.sh
new file mode 100644
index 000000000..a7b8a5370
--- /dev/null
+++ b/src/paks/borgbackup/uninstall.sh
@@ -0,0 +1,27 @@
+#!/bin/bash
+############################################################################
+#                                                                          #
+# This file is part of the IPFire Firewall.                                #
+#                                                                          #
+# IPFire is free software; you can redistribute it and/or modify           #
+# it under the terms of the GNU General Public License as published by     #
+# the Free Software Foundation; either version 2 of the License, or        #
+# (at your option) any later version.                                      #
+#                                                                          #
+# IPFire is distributed in the hope that it will be useful,                #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of           #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the            #
+# GNU General Public License for more details.                             #
+#                                                                          #
+# You should have received a copy of the GNU General Public License        #
+# along with IPFire; if not, write to the Free Software                    #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA #
+#                                                                          #
+# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>.                        #
+#                                                                          #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+stop_service ${NAME}
+make_backup ${NAME}
+remove_files
diff --git a/src/paks/borgbackup/update.sh b/src/paks/borgbackup/update.sh
new file mode 100644
index 000000000..99776659c
--- /dev/null
+++ b/src/paks/borgbackup/update.sh
@@ -0,0 +1,27 @@
+#!/bin/bash
+############################################################################
+#                                                                          #
+# This file is part of the IPFire Firewall.                                #
+#                                                                          #
+# IPFire is free software; you can redistribute it and/or modify           #
+# it under the terms of the GNU General Public License as published by     #
+# the Free Software Foundation; either version 2 of the License, or        #
+# (at your option) any later version.                                      #
+#                                                                          #
+# IPFire is distributed in the hope that it will be useful,                #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of           #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the            #
+# GNU General Public License for more details.                             #
+#                                                                          #
+# You should have received a copy of the GNU General Public License        #
+# along with IPFire; if not, write to the Free Software                    #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA #
+#                                                                          #
+# Copyright (C) 2007-2020 IPFire-Team <info(a)ipfire.org>.                   #
+#                                                                          #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+extract_backup_includes
+./uninstall.sh
+./install.sh
-- 
2.39.0


^ permalink raw reply	[flat|nested] 3+ messages in thread