From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH 1/3] borgbackup: Update to version 1.2.3 and fix bug 13032 Date: Sat, 21 Jan 2023 19:03:30 +0100 Message-ID: <20230121180332.3142785-1-adolf.belka@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8994007420951276670==" List-Id: --===============8994007420951276670== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable - Update from version 1.2.0 to 1.2.3 - Update of rootfile - This update works with python3-msgpack-1.0.4 and fixes bug 13032 - To make it work then the borgbackup-1.2.3-py3.10.egg-info directory must be= the only egg-info directory for borgbackup otherwise version 1.2.3 will end up with= an error. Versions 1.2.2 and earlier workled without any problem if there was an ear= lier egg-info directory for a different version number. The borgbackup rootfile had the = egg-info directory commented out so an uninstall cleared the directory but did not = remove it. This patch has the egg-info directory in the rootfile uncommented and so a= n uninstall removes the directory. - borgbackup paks files created so that the uninstall.sh file will remove any= egg-info directory that starts with "borgbackup-1." as the first ever borgbackup wa= s 1.0.12 When the old 1.2.0 or earlier borgbackup is uninstalled it will use the ol= d default paks uninstall.sh file and rootfile which will leave the old egg-info dire= ctory in place. When version 1.2.3 is installed it will use the new install.sh scri= pt which will remove any existing egg-info directories present still. - Changelog Version 1.2.3 (2022-12-24) Upgrade notes: Some things can be recommended for the upgrade process from borg 1.1.x (pleas= e also read the important compatibility notes below): do you already want to upgrade? 1.1.x also will get fixes for a while. be careful, first upgrade your less critical / smaller repos. first upgrade to a recent 1.1.x release - especially if you run some olde= r 1.1.* or even 1.0.* borg release. using that, run at least one borg create (your normal backup), prune and = especially a check to see everything is in a good state. check the output of borg check - if there is anything special, consider a= borg check --repair followed by another borg check. if everything is fine so far (borg check reports no issues), you can cons= ider upgrading to 1.2.x. if not, please first fix any already existing issue. if you want to play safer, first create a backup of your borg repository. upgrade to latest borg 1.2.x release (you could use the fat binary from g= ithub releases page) run borg compact --cleanup-commits to clean up a ton of 17 bytes long fil= es in your repo caused by a borg 1.1 bug run borg check again (now with borg 1.2.x) and check if there is anything= special. run borg info (with borg 1.2.x) to build the local pre12-meta cache (can = take significant time, but after that it will be fast) - for more details see= below. check the compatibility notes (see below) and adapt your scripts, if need= ed. if you run into any issues, please check the github issue tracker before = posting new issues there or elsewhere. If you follow this procedure, you can help avoiding that we get a lot of =E2= =80=9Cborg 1.2=E2=80=9D issue reports that are not really 1.2 issues, but exi= sted before and maybe just were not noticed. Compatibility notes: matching of path patterns has been aligned with borg storing relative pat= hs. Borg archives file paths without leading slashes. Previously, include/exc= lude patterns could contain leading slashes. You should check your patterns a= nd remove leading slashes. dropped support / testing for older Pythons, minimum requirement is 3.8. = In case your OS does not provide Python >=3D 3.8, consider using our binary, = which does not need an external Python interpreter. Or continue using borg 1.= 1.x, which is still supported. freeing repository space only happens when =E2=80=9Cborg compact=E2=80=9D= is invoked. mount: the default for --numeric-ids is False now (same as borg extract) borg create --noatime is deprecated. Not storing atime is the default beh= aviour now (use --atime if you want to store the atime). --prefix is deprecated, use -a / --glob-archives, see #6806 list: corrected mix-up of =E2=80=9Cisomtime=E2=80=9D and =E2=80=9Cmtime= =E2=80=9D formats. Previously, =E2=80=9Cisomtime=E2=80=9D was the default but= produced a verbose human format, while =E2=80=9Cmtime=E2=80=9D produced a IS= O-8601-like format. The behaviours have been swapped (so =E2=80=9Cmtime=E2=80= =9D is human, =E2=80=9Cisomtime=E2=80=9D is ISO-like), and the default is now= =E2=80=9Cmtime=E2=80=9D. =E2=80=9Cisomtime=E2=80=9D is now a real ISO-8601 f= ormat (=E2=80=9CT=E2=80=9D between date and time, not a space). create/recreate --list: file status for all files used to get announced A= FTER the file (with borg < 1.2). Now, file status is announced BEFORE the fil= e contents are processed. If the file status changes later (e.g. due to an er= ror or a content change), the updated/final file status will be printed again. removed deprecated-since-long stuff (deprecated since): command =E2=80=9Cborg change-passphrase=E2=80=9D (2017-02), use =E2= =80=9Cborg key =E2=80=A6=E2=80=9D option =E2=80=9C--keep-tag-files=E2=80=9D (2017-01), use =E2=80=9C--k= eep-exclude-tags=E2=80=9D option =E2=80=9C--list-format=E2=80=9D (2017-10), use =E2=80=9C--form= at=E2=80=9D option =E2=80=9C--ignore-inode=E2=80=9D (2017-09), use =E2=80=9C--fil= es-cache=E2=80=9D w/o =E2=80=9Cinode=E2=80=9D option =E2=80=9C--no-files-cache=E2=80=9D (2017-09), use =E2=80=9C--f= iles-cache=3Ddisabled=E2=80=9D removed BORG_HOSTNAME_IS_UNIQUE env var. to use borg you must implement o= ne of these 2 scenarios: the combination of FQDN and result of uuid.getnode() must be uniq= ue and stable (this should be the case for almost everybody, except when havi= ng duplicate FQDN and MAC address or all-zero MAC address) if you are aware that 1) is not the case for you, you must set BO= RG_HOST_ID env var to something unique. exit with 128 + signal number, #5161. if you have scripts expecting rc = =3D=3D 2 for a signal exit, you need to update them to check for >=3D 128. Fixes: create: fix --list --dry-run output for directories, #7209 diff/recreate: normalize chunker params before comparing them, #7079 check: fix uninitialised variable if repo is completely empty, #7034 xattrs: improve error handling, #6988 fix args.paths related argparsing, #6994 archive.save(): always use metadata from stats (e.g. nfiles, size, =E2=80= =A6), #7072 tar_filter: recognize .tar.zst as zstd, #7093 get_chunker: fix missing sparse=3DFalse argument, #7056 file_integrity.py: make sure file_fd is always closed on exit repository: cleanup(): close segment before unlinking repository: use os.replace instead of os.rename Other changes: remove python < 3.7 compatibility code do not use version_tuple placeholder in setuptools_scm template CI: fix tox4 passenv issue, #7199 vagrant: update to python 3.9.16, use the openbsd 7.1 box misc. test suite and docs fixes / improvements remove deprecated --prefix from docs, #7109 Windows: use MSYS2 for Github CI, remove Appveyor CI Version 1.2.2 (2022-08-20) New features: prune/delete --checkpoint-interval=3D1800 and ctrl-c/SIGINT support, #6284 Fixes: SaveFile: use a custom mkstemp with mode support, #6933, #6400, #6786. Th= is fixes umask/mode/ACL issues (and also =E2=80=9Cchmod not supported=E2=80= =9D exceptions seen in 1.2.1) of files updated using SaveFile, e.g. the repo = config. hashindex_compact: fix eval order (check idx before use), #5899 create --paths-from-(stdin|command): normalize paths, #6778 secure_erase: avoid collateral damage, #6768. If a hardlink copy of a rep= o was made and a new repo config shall be saved, do NOT fill in random garbag= e before deleting the previous repo config, because that would damage the har= dlink copy. list: fix {flags:} formatting, #6081 check: try harder to create the key, #5719 misc commands: ctrl-c must not kill other subprocesses, #6912 borg create with a remote repo via ssh borg create --content-from-command borg create --paths-from-command (de)compression filter process of import-tar / export-tar Other changes: deprecate --prefix, use -a / --glob-archives, see #6806 make setuptools happy (=E2=80=9Cpackage would be ignored=E2=80=9D), #6874 fix pyproject.toml to create a fixed _version.py file, compatible with bo= th old and new setuptools_scm version, #6875 automate asciinema screencasts CI: test on macOS 12 without fuse / fuse tests (too troublesome on github= CI due to kernel extensions needed by macFUSE) tests: fix test_obfuscate byte accounting repository: add debug logging for issue #6687 _chunker.c: fix warnings on macOS requirements.lock.txt: use the latest cython 0.29.32 docs: add info on man page installation, #6894 update archive_progress json description about =E2=80=9Cfinished=E2= =80=9D, #6570 json progress_percent: some values are optional, #4074 FAQ: full quota / full disk, #5960 correct shell syntax for installation using git Version 1.2.1 (2022-06-06) Fixes: create: skip with warning if opening the parent dir of recursion root fai= ls, #6374 create: fix crash. metadata stream can produce all-zero chunks, #6587 fix crash when computing stats, escape % chars in archive name, #6500 fix transaction rollback: use files cache filename as found in txn.active= /, #6353 import-tar: kill filter process in case of borg exceptions, #6401 #6681 import-tar: fix mtime type bug ensure_dir: respect umask for created directory modes, #6400 SaveFile: respect umask for final file mode, #6400 check archive: improve error handling for corrupt archive metadata block,= make robust_iterator more robust, #4777 pre12-meta cache: do not use the cache if want_unique is True, #6612 fix scp-style repo url parsing for ip v6 address, #6526 mount -o versions: give clear error msg instead of crashing. it does not = make sense to request versions view if you only look at 1 archive, but the co= de shall not crash in that case as it did, but give a clear error msg. show_progress: add finished=3Dtrue/false to archive_progress json, #6570 delete/prune: fix --iec mode output (decimal vs. binary units), #6606 info: fix authenticated mode repo to show =E2=80=9CEncrypted: No=E2=80=9D= , #6462 diff: support presence change for blkdev, chrdev and fifo items, #6615 New features: delete: add repository id and location to prompt, #6453 borg debug dump-repo-objs --ghost: new --segment=3DS --offset=3DO options Other changes: support python 3.11 allow msgpack 1.0.4, #6716 load_key: no key is same as empty key, #6441 give a more helpful error msg for unsupported key formats, #6561 better error msg for defect or unsupported repo configs, #6566 docs: document borg 1.2 pattern matching behavior change, #6407 Make clear = that absolute paths always go into the matcher as if they are relative (witho= ut leading slash). Adapt all examples accordingly. authentication primitives: improved security and performance infos mention BORG_FILES_CACHE_SUFFIX as alternative to BORG_FILES_CACHE_TT= L, #5602 FAQ: add a hint about --debug-topic=3Dfiles_cache improve borg check --max-duration description fix values of TAG bytes, #6515 borg compact --cleanup-commits also runs a normal compaction, #6324 virtualization speed tips recommend umask for passphrase file perms borg 1.2 is security supported update link to ubuntu packages, #6485 use --numeric-ids in pull mode docs remove blake2 docs, blake2 code not bundled any more, #6371 clarify on-disk order and size of segment file log entry fields, #6357 docs building: do not transform --/--- to unicode dashes tests: check that borg does not require pytest for normal usage, fixes #6563 fix OpenBSD symlink mode test failure, #2055 vagrant: darwin64: remove fakeroot, #6314 update development.lock.txt use pyinstaller 4.10 and python 3.9.13 for binary build upgrade VMCPUS and xdistn from 4 to 16, maybe this speeds up the tests crypto: use hmac.compare_digest instead of =3D=3D, #6470 hmac_sha256: replace own cython wrapper code by hmac.digest python st= dlib (since py38) hmac and blake2b minor optimizations and cleanups removed some unused crypto related code, #6472 avoid losing the key (potential use-after-free). this never could hap= pen in 1.2 due to the way we use the code. The issue was discovered in master= after other changes, so we also =E2=80=9Cfixed=E2=80=9D it here before it bi= tes us. setup / build: add pyproject.toml, fix sys.path, #6466 setuptools_scm: also require it via pyproject.toml allow extra compiler flags for every extension build fix misc. C / Cython compiler warnings, deprecation warnings fix zstd.h include for bundled zstd, #6369 source using python 3.8 features: pyupgrade --py38-plus ./**/*.py Fixes: Bug #13032 Tested-by: Adolf Belka Signed-off-by: Adolf Belka --- config/rootfiles/packages/borgbackup | 17 +++++++++-------- lfs/borgbackup | 9 +++++---- src/paks/borgbackup/install.sh | 28 ++++++++++++++++++++++++++++ src/paks/borgbackup/uninstall.sh | 27 +++++++++++++++++++++++++++ src/paks/borgbackup/update.sh | 27 +++++++++++++++++++++++++++ 5 files changed, 96 insertions(+), 12 deletions(-) create mode 100644 src/paks/borgbackup/install.sh create mode 100644 src/paks/borgbackup/uninstall.sh create mode 100644 src/paks/borgbackup/update.sh diff --git a/config/rootfiles/packages/borgbackup b/config/rootfiles/packages= /borgbackup index b744b0b99..feaca4156 100644 --- a/config/rootfiles/packages/borgbackup +++ b/config/rootfiles/packages/borgbackup @@ -91,11 +91,12 @@ usr/lib/python3.10/site-packages/borg/testsuite/xattr.py usr/lib/python3.10/site-packages/borg/upgrader.py usr/lib/python3.10/site-packages/borg/version.py usr/lib/python3.10/site-packages/borg/xattr.py -#usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/PKG-INFO -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/SOURCES.txt -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/dependency= _links.txt -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/entry_poin= ts.txt -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/not-zip-sa= fe -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/requires.t= xt -usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info/top_level.= txt +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/PKG-INFO +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/SOURCES.txt +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/dependency= _links.txt +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/entry_poin= ts.txt +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/not-zip-sa= fe +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/requires.t= xt +usr/lib/python3.10/site-packages/borgbackup-1.2.3-py3.10.egg-info/top_level.= txt + diff --git a/lfs/borgbackup b/lfs/borgbackup index 9c761eac4..0cbb809b7 100644 --- a/lfs/borgbackup +++ b/lfs/borgbackup @@ -1,7 +1,7 @@ ############################################################################= ### # = # # IPFire.org - A linux based firewall = # -# Copyright (C) 2007-2020 IPFire Team = # +# Copyright (C) 2007-2023 IPFire Team = # # = # # This program is free software: you can redistribute it and/or modify = # # it under the terms of the GNU General Public License as published by = # @@ -24,7 +24,7 @@ =20 include Config =20 -VER =3D 1.2.0 +VER =3D 1.2.3 SUMMARY =3D Deduplicating backup program with compression and authenticat= ed encryption =20 THISAPP =3D borgbackup-$(VER) @@ -33,9 +33,10 @@ DL_FROM =3D $(URL_IPFIRE) DIR_APP =3D $(DIR_SRC)/$(THISAPP) TARGET =3D $(DIR_INFO)/$(THISAPP) PROG =3D borgbackup -PAK_VER =3D 13 +PAK_VER =3D 14 =20 DEPS =3D python3-pkgconfig python3-msgpack python3-packaging python3-p= yfuse3 +# borgbackup only works with specific versions of python3-msgpack =20 SERVICES =3D =20 @@ -47,7 +48,7 @@ objects =3D $(DL_FILE) =20 $(DL_FILE) =3D $(DL_FROM)/$(DL_FILE) =20 -$(DL_FILE)_BLAKE2 =3D 9e6cb8d85ca196cbdd6baba694bc7987d63f85d2ba0e25f3ac1e59= 400882fff71e29b04ca218ee78f23daeb52d13547062b0c80bd0d3f5b460b28b4f274d11ec +$(DL_FILE)_BLAKE2 =3D 970fad62428b44eb22df761020a4c25f4aa8d3e8f5cc2c81aa96b1= afb051dc24b3d124b4fc84e81cf8eaa6a9b19a200f6f9bda1f7f4c2794a7e8d33e8d5020f1 =20 install : $(TARGET) =20 diff --git a/src/paks/borgbackup/install.sh b/src/paks/borgbackup/install.sh new file mode 100644 index 000000000..f8e61e684 --- /dev/null +++ b/src/paks/borgbackup/install.sh @@ -0,0 +1,28 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007 IPFire-Team . = # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +rm -rfv /usr/lib/python3.10/site-packages/borgbackup-1.* +extract_files +restore_backup ${NAME} +start_service --background ${NAME} diff --git a/src/paks/borgbackup/uninstall.sh b/src/paks/borgbackup/uninstall= .sh new file mode 100644 index 000000000..a7b8a5370 --- /dev/null +++ b/src/paks/borgbackup/uninstall.sh @@ -0,0 +1,27 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007 IPFire-Team . = # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +stop_service ${NAME} +make_backup ${NAME} +remove_files diff --git a/src/paks/borgbackup/update.sh b/src/paks/borgbackup/update.sh new file mode 100644 index 000000000..99776659c --- /dev/null +++ b/src/paks/borgbackup/update.sh @@ -0,0 +1,27 @@ +#!/bin/bash +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007-2020 IPFire-Team . = # +# # +############################################################################ +# +. /opt/pakfire/lib/functions.sh +extract_backup_includes +./uninstall.sh +./install.sh --=20 2.39.0 --===============8994007420951276670==--