Re: [PATCH] backup: Fix broken globbing expansion

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 13906 bytes --]


> On 29 Mar 2022, at 15:22, Bernhard Bitsch <bbitsch(a)ipfire.org> wrote:
> 
> Looks like your include.user file contains absolute paths.

Or an empty line maybe?

> 
> Am 29.03.2022 um 16:14 schrieb Adolf Belka:
>> Hi All,
>> This is what I get with the pushd/popd version in place on the same system
>> -bash-5.1$ sudo /var/ipfire/backup/bin/backup.pl list
>> etc/group
>> etc/hosts
>> etc/hosts.allow
>> etc/hosts.deny
>> etc/httpd/server.crt
>> etc/httpd/server.csr
>> etc/httpd/server-ecdsa.crt
>> etc/httpd/server-ecdsa.csr
>> etc/httpd/server-ecdsa.key
>> etc/httpd/server.key
>> etc/ipsec.user.conf
>> etc/ipsec.user.secrets
>> etc/logrotate.d
>> etc/passwd
>> etc/shadow
>> etc/ssh/sshd_config
>> etc/ssh/ssh_host_ecdsa_key
>> etc/ssh/ssh_host_ecdsa_key.pub
>> etc/ssh/ssh_host_ed25519_key
>> etc/ssh/ssh_host_ed25519_key.pub
>> etc/ssh/ssh_host_rsa_key
>> etc/ssh/ssh_host_rsa_key.pub
>> /etc/sudoers
>> etc/sysconfig/createfiles
>> etc/sysconfig/firewall.local
>> etc/sysconfig/lm_sensors
>> etc/sysconfig/modules
>> etc/sysconfig/ramdisk
>> etc/sysconfig/rc
>> etc/sysconfig/rc.local
>> etc/unbound
>> /home/ahb
>> root/.bash_history
>> var/ipfire/accounting/settings.conf
>> var/ipfire/auth/users
>> var/ipfire/backup/addons/backup
>> var/ipfire/backup/exclude.user
>> var/ipfire/backup/include.user
>> var/ipfire/ca/cacert.pem
>> var/ipfire/captive/agb.txt
>> var/ipfire/captive/clients
>> var/ipfire/captive/coupons
>> var/ipfire/captive/logo.dat
>> var/ipfire/captive/settings
>> var/ipfire/captive/terms.txt
>> var/ipfire/captive/voucher_out
>> var/ipfire/certs/hostcert.pem
>> var/ipfire/certs/hostkey.pem
>> var/ipfire/certs/phoebevmipseccert.pem
>> var/ipfire/connscheduler/connscheduler.conf
>> var/ipfire/crls/cacrl.pem
>> var/ipfire/cups/cups-browsed.conf
>> var/ipfire/cups/subscriptions.conf
>> var/ipfire/ddns/config
>> var/ipfire/ddns/ddns.conf
>> var/ipfire/ddns/settings
>> var/ipfire/dhcp/advoptions
>> var/ipfire/dhcp/advoptions-list
>> var/ipfire/dhcpc/dhcpcd.conf
>> var/ipfire/dhcp/dhcpd.conf
>> var/ipfire/dhcp/dhcpd.conf.local
>> var/ipfire/dhcp/enable_blue
>> var/ipfire/dhcp/enable_green
>> var/ipfire/dhcp/fixleases
>> var/ipfire/dhcp/settings
>> var/ipfire/dma/auth.conf
>> var/ipfire/dma/dma.conf
>> var/ipfire/dma/mail.conf
>> var/ipfire/dns
>> var/ipfire/dnsforward/config
>> var/ipfire/dns/settings
>> var/ipfire/ethernet/aliases
>> var/ipfire/ethernet/settings
>> var/ipfire/ethernet/wireless
>> var/ipfire/extrahd/settings
>> var/ipfire/firewall
>> var/ipfire/firewall/config
>> var/ipfire/firewall/settings
>> var/ipfire/fwhosts
>> var/ipfire/isdn/settings
>> var/ipfire/logging/settings
>> var/ipfire/mac/settings
>> var/ipfire/main/firstsetup_ok
>> var/ipfire/main/gpl_accepted
>> var/ipfire/main/hostname.conf
>> var/ipfire/main/hosts
>> var/ipfire/main/manualpages
>> var/ipfire/main/routing
>> var/ipfire/main/security
>> var/ipfire/main/send_profile
>> var/ipfire/main/settings
>> var/ipfire/modem/settings
>> var/ipfire/optionsfw/settings
>> var/ipfire/ovpn
>> var/ipfire/ovpn/ccd.conf
>> var/ipfire/ovpn/collectd.vpn
>> var/ipfire/ovpn/enable
>> var/ipfire/ovpn/server.conf
>> var/ipfire/ovpn/settings
>> var/ipfire/pakfire/settings
>> var/ipfire/ppp
>> var/ipfire/ppp/fake-resolv.conf
>> var/ipfire/ppp/settings
>> var/ipfire/private/cakey.pem
>> var/ipfire/proxy
>> var/ipfire/proxy/asnbl-helper.conf
>> var/ipfire/proxy/cachemgr.conf
>> var/ipfire/proxy/enable
>> var/ipfire/proxy/settings
>> var/ipfire/proxy/squid.conf
>> var/ipfire/qos/bin
>> var/ipfire/qos/classes
>> var/ipfire/qos/level7config
>> var/ipfire/qos/portconfig
>> var/ipfire/qos/settings
>> var/ipfire/qos/subclasses
>> var/ipfire/qos/tosconfig
>> var/ipfire/remote/enablessh
>> var/ipfire/remote/settings
>> var/ipfire/sensors/settings
>> var/ipfire/suricata/oinkmaster.conf
>> var/ipfire/suricata/oinkmaster-modify-sids.conf
>> var/ipfire/suricata/oinkmaster-provider-includes.conf
>> var/ipfire/suricata/providers-settings
>> var/ipfire/suricata/settings
>> var/ipfire/suricata/suricata-default-rules.yaml
>> var/ipfire/suricata/suricata-dns-servers.yaml
>> var/ipfire/suricata/suricata-emerging-used-rulefiles.yaml
>> var/ipfire/suricata/suricata-homenet.yaml
>> var/ipfire/suricata/suricata-http-ports.yaml
>> var/ipfire/suricata/suricata-sslbl_blacklist-used-rulefiles.yaml
>> var/ipfire/suricata/suricata-used-providers.yaml
>> var/ipfire/time/
>> var/ipfire/time/counter.conf
>> var/ipfire/time/enable
>> var/ipfire/time/settime.conf
>> var/ipfire/time/settings
>> var/ipfire/upnp/settings
>> var/ipfire/urlfilter
>> var/ipfire/urlfilter/settings
>> var/ipfire/urlfilter/squidGuard.conf
>> var/ipfire/vpn
>> var/ipfire/vpn/config
>> var/ipfire/vpn/ipsec.conf
>> var/ipfire/vpn/settings
>> var/ipfire/wakeonlan/clients.conf
>> var/ipfire/wio/wio.conf
>> var/ipfire/wireless/config
>> var/ipfire/wireless/settings
>> var/lib/suricata
>> var/log/rrd/collectd
>> var/log/rrd/hddshutdown-md127.rrd
>> var/log/rrd/hddshutdown-sda.rrd
>> var/log/rrd/hddshutdown-sdb.rrd
>> var/log/rrd/hddtemp-md127.rrd
>> var/log/rrd/hddtemp-sda.rrd
>> var/log/rrd/hddtemp-sdb.rrd
>> var/log/rrd/wio
>> var/log/vnstat
>> var/tmp/idsrules-emerging.tar.gz
>> var/tmp/idsrules-sslbl_blacklist.rules
>> The following are in the previous list but not in this one:-
>> /
>> /etc/conntrackd/conntrackd.conf
>> /etc/ipsec.user-post.conf
>> /root/.gitconfig
>> /root/.ssh
>> Regards,
>> Adolf.
>> On 29/03/2022 16:01, Adolf Belka wrote:
>>> Hi All,
>>> 
>>> On 29/03/2022 15:36, Bernhard Bitsch wrote:
>>>> Hi,
>>>> 
>>>> Am 29.03.2022 um 15:11 schrieb Michael Tremer:
>>>>> Hello,
>>>>> 
>>>>> Could you please send the file listing to find out what is being included what shouldn’t?
>>> Unfortunately, as I stopped the backup continuing once it had reached 1.2GB, the file created was not able to be opened. Probably stopping the backup corrupted it in some way.
>>>>> 
>>>> 
>>>> Just for easy quick handling, you can use
>>>> '/var/ipfire/backup/bin/backup.pl list' to show the files included.
>>> I gave this a go and here is the output from the command.
>>> 
>>> -bash-5.1$ sudo /var/ipfire/backup/bin/backup.pl list
>>> /
>>> /etc/conntrackd/conntrackd.conf
>>> /etc/group
>>> /etc/hosts
>>> /etc/hosts.allow
>>> /etc/hosts.deny
>>> /etc/httpd/server.crt
>>> /etc/httpd/server.csr
>>> /etc/httpd/server-ecdsa.crt
>>> /etc/httpd/server-ecdsa.csr
>>> /etc/httpd/server-ecdsa.key
>>> /etc/httpd/server.key
>>> /etc/ipsec.user.conf
>>> /etc/ipsec.user-post.conf
>>> /etc/ipsec.user.secrets
>>> /etc/logrotate.d
>>> /etc/passwd
>>> /etc/shadow
>>> /etc/squid/squid.conf.local
>>> /etc/squid/squid.conf.pre.local
>>> /etc/ssh/sshd_config
>>> /etc/ssh/ssh_host_ecdsa_key
>>> /etc/ssh/ssh_host_ecdsa_key.pub
>>> /etc/ssh/ssh_host_ed25519_key
>>> /etc/ssh/ssh_host_ed25519_key.pub
>>> /etc/ssh/ssh_host_rsa_key
>>> /etc/ssh/ssh_host_rsa_key.pub
>>> //etc/sudoers
>>> /etc/sysconfig/createfiles
>>> /etc/sysconfig/firewall.local
>>> /etc/sysconfig/lm_sensors
>>> /etc/sysconfig/modules
>>> /etc/sysconfig/ramdisk
>>> /etc/sysconfig/rc
>>> /etc/sysconfig/rc.local
>>> /etc/unbound
>>> //home/ahb
>>> /root/.bash_history
>>> /root/.gitconfig
>>> /root/.ssh
>>> /var/ipfire/accounting/settings.conf
>>> /var/ipfire/auth/users
>>> /var/ipfire/backup/addons/backup
>>> /var/ipfire/backup/exclude.user
>>> /var/ipfire/backup/include.user
>>> /var/ipfire/ca/cacert.pem
>>> /var/ipfire/captive/agb.txt
>>> /var/ipfire/captive/clients
>>> /var/ipfire/captive/coupons
>>> /var/ipfire/captive/logo.dat
>>> /var/ipfire/captive/settings
>>> /var/ipfire/captive/terms.txt
>>> /var/ipfire/captive/voucher_out
>>> /var/ipfire/certs/hostcert.pem
>>> /var/ipfire/certs/hostkey.pem
>>> /var/ipfire/certs/phoebevmipseccert.pem
>>> /var/ipfire/connscheduler/connscheduler.conf
>>> /var/ipfire/crls/cacrl.pem
>>> /var/ipfire/cups/cups-browsed.conf
>>> /var/ipfire/cups/subscriptions.conf
>>> /var/ipfire/ddns/config
>>> /var/ipfire/ddns/ddns.conf
>>> /var/ipfire/ddns/settings
>>> /var/ipfire/dhcp/advoptions
>>> /var/ipfire/dhcp/advoptions-list
>>> /var/ipfire/dhcpc/dhcpcd.conf
>>> /var/ipfire/dhcp/dhcpd.conf
>>> /var/ipfire/dhcp/dhcpd.conf.local
>>> /var/ipfire/dhcp/enable_blue
>>> /var/ipfire/dhcp/enable_green
>>> /var/ipfire/dhcp/fixleases
>>> /var/ipfire/dhcp/settings
>>> /var/ipfire/dma/auth.conf
>>> /var/ipfire/dma/dma.conf
>>> /var/ipfire/dma/mail.conf
>>> /var/ipfire/dns
>>> /var/ipfire/dnsforward/config
>>> /var/ipfire/dns/settings
>>> /var/ipfire/ethernet/aliases
>>> /var/ipfire/ethernet/settings
>>> /var/ipfire/ethernet/wireless
>>> /var/ipfire/extrahd/settings
>>> /var/ipfire/firewall
>>> /var/ipfire/firewall/config
>>> /var/ipfire/firewall/settings
>>> /var/ipfire/fwhosts
>>> /var/ipfire/isdn/settings
>>> /var/ipfire/logging/settings
>>> /var/ipfire/mac/settings
>>> /var/ipfire/main/firstsetup_ok
>>> /var/ipfire/main/gpl_accepted
>>> /var/ipfire/main/hostname.conf
>>> /var/ipfire/main/hosts
>>> /var/ipfire/main/manualpages
>>> /var/ipfire/main/routing
>>> /var/ipfire/main/security
>>> /var/ipfire/main/send_profile
>>> /var/ipfire/main/settings
>>> /var/ipfire/modem/settings
>>> /var/ipfire/optionsfw/settings
>>> /var/ipfire/ovpn
>>> /var/ipfire/ovpn/ccd.conf
>>> /var/ipfire/ovpn/collectd.vpn
>>> /var/ipfire/ovpn/enable
>>> /var/ipfire/ovpn/server.conf
>>> /var/ipfire/ovpn/settings
>>> /var/ipfire/pakfire/settings
>>> /var/ipfire/ppp
>>> /var/ipfire/ppp/fake-resolv.conf
>>> /var/ipfire/ppp/settings
>>> /var/ipfire/private/cakey.pem
>>> /var/ipfire/proxy
>>> /var/ipfire/proxy/asnbl-helper.conf
>>> /var/ipfire/proxy/cachemgr.conf
>>> /var/ipfire/proxy/enable
>>> /var/ipfire/proxy/settings
>>> /var/ipfire/proxy/squid.conf
>>> /var/ipfire/qos/bin
>>> /var/ipfire/qos/bin/qos.sh
>>> /var/ipfire/qos/classes
>>> /var/ipfire/qos/level7config
>>> /var/ipfire/qos/portconfig
>>> /var/ipfire/qos/settings
>>> /var/ipfire/qos/subclasses
>>> /var/ipfire/qos/tosconfig
>>> /var/ipfire/remote/enablessh
>>> /var/ipfire/remote/settings
>>> /var/ipfire/sensors/settings
>>> /var/ipfire/suricata/oinkmaster.conf
>>> /var/ipfire/suricata/oinkmaster-modify-sids.conf
>>> /var/ipfire/suricata/oinkmaster-provider-includes.conf
>>> /var/ipfire/suricata/providers-settings
>>> /var/ipfire/suricata/settings
>>> /var/ipfire/suricata/suricata-default-rules.yaml
>>> /var/ipfire/suricata/suricata-dns-servers.yaml
>>> /var/ipfire/suricata/suricata-emerging-used-rulefiles.yaml
>>> /var/ipfire/suricata/suricata-homenet.yaml
>>> /var/ipfire/suricata/suricata-http-ports.yaml
>>> /var/ipfire/suricata/suricata-sslbl_blacklist-used-rulefiles.yaml
>>> /var/ipfire/suricata/suricata-used-providers.yaml
>>> /var/ipfire/time/
>>> /var/ipfire/time/counter.conf
>>> /var/ipfire/time/enable
>>> /var/ipfire/time/settime.conf
>>> /var/ipfire/time/settings
>>> /var/ipfire/upnp/settings
>>> /var/ipfire/urlfilter
>>> /var/ipfire/urlfilter/settings
>>> /var/ipfire/urlfilter/squidGuard.conf
>>> /var/ipfire/vpn
>>> /var/ipfire/vpn/config
>>> /var/ipfire/vpn/ipsec.conf
>>> /var/ipfire/vpn/settings
>>> /var/ipfire/wakeonlan/clients.conf
>>> /var/ipfire/wio/wio.conf
>>> /var/ipfire/wireless/config
>>> /var/ipfire/wireless/settings
>>> /var/lib/suricata
>>> /var/log/rrd/collectd
>>> /var/log/rrd/hddshutdown-md127.rrd
>>> /var/log/rrd/hddshutdown-sda.rrd
>>> /var/log/rrd/hddshutdown-sdb.rrd
>>> /var/log/rrd/hddtemp-md127.rrd
>>> /var/log/rrd/hddtemp-sda.rrd
>>> /var/log/rrd/hddtemp-sdb.rrd
>>> /var/log/rrd/wio
>>> /var/log/vnstat
>>> /var/tmp/idsrules-emerging.tar.gz
>>> /var/tmp/idsrules-sslbl_blacklist.rules
>>> 
>>> Regards,
>>> Adolf.
>>>> 
>>>> Bernhard
>>>>> -Michael
>>>>> 
>>>>>> On 29 Mar 2022, at 14:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>> 
>>>>>> Hi Michael,
>>>>>> 
>>>>>> Just tried this patch out on my vm testbed system and it still doesn't work for me. The backup file had got to 1.3GB when I deleted the backup file as it was still growing. The normal correct backup file on that vm machine is around 7MB
>>>>>> 
>>>>>> The above was the case for both running it from the WUI or from the command line from my unprivileged user using sudo backupctrl exclude
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf.
>>>>>> 
>>>>>> 
>>>>>> On 29/03/2022 14:27, Michael Tremer wrote:
>>>>>>> This patch fixes globbing expansion in the backup include file list
>>>>>>> which got broken in c7e0d73e7cfd7be95db9d0a5f3392b8241813d5b.
>>>>>>> 
>>>>>>> Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
>>>>>>> ---
>>>>>>>   config/backup/backup.pl | 8 ++++----
>>>>>>>   1 file changed, 4 insertions(+), 4 deletions(-)
>>>>>>> 
>>>>>>> diff --git a/config/backup/backup.pl b/config/backup/backup.pl
>>>>>>> index a2337cf23..6f9295e94 100644
>>>>>>> --- a/config/backup/backup.pl
>>>>>>> +++ b/config/backup/backup.pl
>>>>>>> @@ -19,6 +19,8 @@
>>>>>>> # #
>>>>>>> ############################################################################### 
>>>>>>>   +shopt -s nullglob
>>>>>>> +
>>>>>>>   NOW="$(date "+%Y-%m-%d-%H:%M")"
>>>>>>>     list_addons() {
>>>>>>> @@ -38,10 +40,8 @@ process_includes() {
>>>>>>>       for include in $@; do
>>>>>>>           local file
>>>>>>>           while read -r file; do
>>>>>>> -            for file in ${file}; do
>>>>>>> -                if [ -e "/${file}" ]; then
>>>>>>> -                    echo "${file}"
>>>>>>> -                fi
>>>>>>> +            for file in /${file}; do
>>>>>>> +                echo "${file}"
>>>>>>>               done
>>>>>>>           done < "${include}"
>>>>>>>       done | sort -u
>>>>>