Re: [PATCH] backup: Fix broken globbing expansion

[Bernhard Bitsch <bbitsch@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 13584 bytes --]

Looks like your include.user file contains absolute paths.

Am 29.03.2022 um 16:14 schrieb Adolf Belka:
> Hi All,
> 
> This is what I get with the pushd/popd version in place on the same system
> 
> -bash-5.1$ sudo /var/ipfire/backup/bin/backup.pl list
> etc/group
> etc/hosts
> etc/hosts.allow
> etc/hosts.deny
> etc/httpd/server.crt
> etc/httpd/server.csr
> etc/httpd/server-ecdsa.crt
> etc/httpd/server-ecdsa.csr
> etc/httpd/server-ecdsa.key
> etc/httpd/server.key
> etc/ipsec.user.conf
> etc/ipsec.user.secrets
> etc/logrotate.d
> etc/passwd
> etc/shadow
> etc/ssh/sshd_config
> etc/ssh/ssh_host_ecdsa_key
> etc/ssh/ssh_host_ecdsa_key.pub
> etc/ssh/ssh_host_ed25519_key
> etc/ssh/ssh_host_ed25519_key.pub
> etc/ssh/ssh_host_rsa_key
> etc/ssh/ssh_host_rsa_key.pub
> /etc/sudoers
> etc/sysconfig/createfiles
> etc/sysconfig/firewall.local
> etc/sysconfig/lm_sensors
> etc/sysconfig/modules
> etc/sysconfig/ramdisk
> etc/sysconfig/rc
> etc/sysconfig/rc.local
> etc/unbound
> /home/ahb
> root/.bash_history
> var/ipfire/accounting/settings.conf
> var/ipfire/auth/users
> var/ipfire/backup/addons/backup
> var/ipfire/backup/exclude.user
> var/ipfire/backup/include.user
> var/ipfire/ca/cacert.pem
> var/ipfire/captive/agb.txt
> var/ipfire/captive/clients
> var/ipfire/captive/coupons
> var/ipfire/captive/logo.dat
> var/ipfire/captive/settings
> var/ipfire/captive/terms.txt
> var/ipfire/captive/voucher_out
> var/ipfire/certs/hostcert.pem
> var/ipfire/certs/hostkey.pem
> var/ipfire/certs/phoebevmipseccert.pem
> var/ipfire/connscheduler/connscheduler.conf
> var/ipfire/crls/cacrl.pem
> var/ipfire/cups/cups-browsed.conf
> var/ipfire/cups/subscriptions.conf
> var/ipfire/ddns/config
> var/ipfire/ddns/ddns.conf
> var/ipfire/ddns/settings
> var/ipfire/dhcp/advoptions
> var/ipfire/dhcp/advoptions-list
> var/ipfire/dhcpc/dhcpcd.conf
> var/ipfire/dhcp/dhcpd.conf
> var/ipfire/dhcp/dhcpd.conf.local
> var/ipfire/dhcp/enable_blue
> var/ipfire/dhcp/enable_green
> var/ipfire/dhcp/fixleases
> var/ipfire/dhcp/settings
> var/ipfire/dma/auth.conf
> var/ipfire/dma/dma.conf
> var/ipfire/dma/mail.conf
> var/ipfire/dns
> var/ipfire/dnsforward/config
> var/ipfire/dns/settings
> var/ipfire/ethernet/aliases
> var/ipfire/ethernet/settings
> var/ipfire/ethernet/wireless
> var/ipfire/extrahd/settings
> var/ipfire/firewall
> var/ipfire/firewall/config
> var/ipfire/firewall/settings
> var/ipfire/fwhosts
> var/ipfire/isdn/settings
> var/ipfire/logging/settings
> var/ipfire/mac/settings
> var/ipfire/main/firstsetup_ok
> var/ipfire/main/gpl_accepted
> var/ipfire/main/hostname.conf
> var/ipfire/main/hosts
> var/ipfire/main/manualpages
> var/ipfire/main/routing
> var/ipfire/main/security
> var/ipfire/main/send_profile
> var/ipfire/main/settings
> var/ipfire/modem/settings
> var/ipfire/optionsfw/settings
> var/ipfire/ovpn
> var/ipfire/ovpn/ccd.conf
> var/ipfire/ovpn/collectd.vpn
> var/ipfire/ovpn/enable
> var/ipfire/ovpn/server.conf
> var/ipfire/ovpn/settings
> var/ipfire/pakfire/settings
> var/ipfire/ppp
> var/ipfire/ppp/fake-resolv.conf
> var/ipfire/ppp/settings
> var/ipfire/private/cakey.pem
> var/ipfire/proxy
> var/ipfire/proxy/asnbl-helper.conf
> var/ipfire/proxy/cachemgr.conf
> var/ipfire/proxy/enable
> var/ipfire/proxy/settings
> var/ipfire/proxy/squid.conf
> var/ipfire/qos/bin
> var/ipfire/qos/classes
> var/ipfire/qos/level7config
> var/ipfire/qos/portconfig
> var/ipfire/qos/settings
> var/ipfire/qos/subclasses
> var/ipfire/qos/tosconfig
> var/ipfire/remote/enablessh
> var/ipfire/remote/settings
> var/ipfire/sensors/settings
> var/ipfire/suricata/oinkmaster.conf
> var/ipfire/suricata/oinkmaster-modify-sids.conf
> var/ipfire/suricata/oinkmaster-provider-includes.conf
> var/ipfire/suricata/providers-settings
> var/ipfire/suricata/settings
> var/ipfire/suricata/suricata-default-rules.yaml
> var/ipfire/suricata/suricata-dns-servers.yaml
> var/ipfire/suricata/suricata-emerging-used-rulefiles.yaml
> var/ipfire/suricata/suricata-homenet.yaml
> var/ipfire/suricata/suricata-http-ports.yaml
> var/ipfire/suricata/suricata-sslbl_blacklist-used-rulefiles.yaml
> var/ipfire/suricata/suricata-used-providers.yaml
> var/ipfire/time/
> var/ipfire/time/counter.conf
> var/ipfire/time/enable
> var/ipfire/time/settime.conf
> var/ipfire/time/settings
> var/ipfire/upnp/settings
> var/ipfire/urlfilter
> var/ipfire/urlfilter/settings
> var/ipfire/urlfilter/squidGuard.conf
> var/ipfire/vpn
> var/ipfire/vpn/config
> var/ipfire/vpn/ipsec.conf
> var/ipfire/vpn/settings
> var/ipfire/wakeonlan/clients.conf
> var/ipfire/wio/wio.conf
> var/ipfire/wireless/config
> var/ipfire/wireless/settings
> var/lib/suricata
> var/log/rrd/collectd
> var/log/rrd/hddshutdown-md127.rrd
> var/log/rrd/hddshutdown-sda.rrd
> var/log/rrd/hddshutdown-sdb.rrd
> var/log/rrd/hddtemp-md127.rrd
> var/log/rrd/hddtemp-sda.rrd
> var/log/rrd/hddtemp-sdb.rrd
> var/log/rrd/wio
> var/log/vnstat
> var/tmp/idsrules-emerging.tar.gz
> var/tmp/idsrules-sslbl_blacklist.rules
> 
> The following are in the previous list but not in this one:-
> 
> /
> /etc/conntrackd/conntrackd.conf
> /etc/ipsec.user-post.conf
> /root/.gitconfig
> /root/.ssh
> 
> Regards,
> 
> Adolf.
> 
> On 29/03/2022 16:01, Adolf Belka wrote:
>> Hi All,
>>
>> On 29/03/2022 15:36, Bernhard Bitsch wrote:
>>> Hi,
>>>
>>> Am 29.03.2022 um 15:11 schrieb Michael Tremer:
>>>> Hello,
>>>>
>>>> Could you please send the file listing to find out what is being 
>>>> included what shouldn’t?
>> Unfortunately, as I stopped the backup continuing once it had reached 
>> 1.2GB, the file created was not able to be opened. Probably stopping 
>> the backup corrupted it in some way.
>>>>
>>>
>>> Just for easy quick handling, you can use
>>> '/var/ipfire/backup/bin/backup.pl list' to show the files included.
>> I gave this a go and here is the output from the command.
>>
>> -bash-5.1$ sudo /var/ipfire/backup/bin/backup.pl list
>> /
>> /etc/conntrackd/conntrackd.conf
>> /etc/group
>> /etc/hosts
>> /etc/hosts.allow
>> /etc/hosts.deny
>> /etc/httpd/server.crt
>> /etc/httpd/server.csr
>> /etc/httpd/server-ecdsa.crt
>> /etc/httpd/server-ecdsa.csr
>> /etc/httpd/server-ecdsa.key
>> /etc/httpd/server.key
>> /etc/ipsec.user.conf
>> /etc/ipsec.user-post.conf
>> /etc/ipsec.user.secrets
>> /etc/logrotate.d
>> /etc/passwd
>> /etc/shadow
>> /etc/squid/squid.conf.local
>> /etc/squid/squid.conf.pre.local
>> /etc/ssh/sshd_config
>> /etc/ssh/ssh_host_ecdsa_key
>> /etc/ssh/ssh_host_ecdsa_key.pub
>> /etc/ssh/ssh_host_ed25519_key
>> /etc/ssh/ssh_host_ed25519_key.pub
>> /etc/ssh/ssh_host_rsa_key
>> /etc/ssh/ssh_host_rsa_key.pub
>> //etc/sudoers
>> /etc/sysconfig/createfiles
>> /etc/sysconfig/firewall.local
>> /etc/sysconfig/lm_sensors
>> /etc/sysconfig/modules
>> /etc/sysconfig/ramdisk
>> /etc/sysconfig/rc
>> /etc/sysconfig/rc.local
>> /etc/unbound
>> //home/ahb
>> /root/.bash_history
>> /root/.gitconfig
>> /root/.ssh
>> /var/ipfire/accounting/settings.conf
>> /var/ipfire/auth/users
>> /var/ipfire/backup/addons/backup
>> /var/ipfire/backup/exclude.user
>> /var/ipfire/backup/include.user
>> /var/ipfire/ca/cacert.pem
>> /var/ipfire/captive/agb.txt
>> /var/ipfire/captive/clients
>> /var/ipfire/captive/coupons
>> /var/ipfire/captive/logo.dat
>> /var/ipfire/captive/settings
>> /var/ipfire/captive/terms.txt
>> /var/ipfire/captive/voucher_out
>> /var/ipfire/certs/hostcert.pem
>> /var/ipfire/certs/hostkey.pem
>> /var/ipfire/certs/phoebevmipseccert.pem
>> /var/ipfire/connscheduler/connscheduler.conf
>> /var/ipfire/crls/cacrl.pem
>> /var/ipfire/cups/cups-browsed.conf
>> /var/ipfire/cups/subscriptions.conf
>> /var/ipfire/ddns/config
>> /var/ipfire/ddns/ddns.conf
>> /var/ipfire/ddns/settings
>> /var/ipfire/dhcp/advoptions
>> /var/ipfire/dhcp/advoptions-list
>> /var/ipfire/dhcpc/dhcpcd.conf
>> /var/ipfire/dhcp/dhcpd.conf
>> /var/ipfire/dhcp/dhcpd.conf.local
>> /var/ipfire/dhcp/enable_blue
>> /var/ipfire/dhcp/enable_green
>> /var/ipfire/dhcp/fixleases
>> /var/ipfire/dhcp/settings
>> /var/ipfire/dma/auth.conf
>> /var/ipfire/dma/dma.conf
>> /var/ipfire/dma/mail.conf
>> /var/ipfire/dns
>> /var/ipfire/dnsforward/config
>> /var/ipfire/dns/settings
>> /var/ipfire/ethernet/aliases
>> /var/ipfire/ethernet/settings
>> /var/ipfire/ethernet/wireless
>> /var/ipfire/extrahd/settings
>> /var/ipfire/firewall
>> /var/ipfire/firewall/config
>> /var/ipfire/firewall/settings
>> /var/ipfire/fwhosts
>> /var/ipfire/isdn/settings
>> /var/ipfire/logging/settings
>> /var/ipfire/mac/settings
>> /var/ipfire/main/firstsetup_ok
>> /var/ipfire/main/gpl_accepted
>> /var/ipfire/main/hostname.conf
>> /var/ipfire/main/hosts
>> /var/ipfire/main/manualpages
>> /var/ipfire/main/routing
>> /var/ipfire/main/security
>> /var/ipfire/main/send_profile
>> /var/ipfire/main/settings
>> /var/ipfire/modem/settings
>> /var/ipfire/optionsfw/settings
>> /var/ipfire/ovpn
>> /var/ipfire/ovpn/ccd.conf
>> /var/ipfire/ovpn/collectd.vpn
>> /var/ipfire/ovpn/enable
>> /var/ipfire/ovpn/server.conf
>> /var/ipfire/ovpn/settings
>> /var/ipfire/pakfire/settings
>> /var/ipfire/ppp
>> /var/ipfire/ppp/fake-resolv.conf
>> /var/ipfire/ppp/settings
>> /var/ipfire/private/cakey.pem
>> /var/ipfire/proxy
>> /var/ipfire/proxy/asnbl-helper.conf
>> /var/ipfire/proxy/cachemgr.conf
>> /var/ipfire/proxy/enable
>> /var/ipfire/proxy/settings
>> /var/ipfire/proxy/squid.conf
>> /var/ipfire/qos/bin
>> /var/ipfire/qos/bin/qos.sh
>> /var/ipfire/qos/classes
>> /var/ipfire/qos/level7config
>> /var/ipfire/qos/portconfig
>> /var/ipfire/qos/settings
>> /var/ipfire/qos/subclasses
>> /var/ipfire/qos/tosconfig
>> /var/ipfire/remote/enablessh
>> /var/ipfire/remote/settings
>> /var/ipfire/sensors/settings
>> /var/ipfire/suricata/oinkmaster.conf
>> /var/ipfire/suricata/oinkmaster-modify-sids.conf
>> /var/ipfire/suricata/oinkmaster-provider-includes.conf
>> /var/ipfire/suricata/providers-settings
>> /var/ipfire/suricata/settings
>> /var/ipfire/suricata/suricata-default-rules.yaml
>> /var/ipfire/suricata/suricata-dns-servers.yaml
>> /var/ipfire/suricata/suricata-emerging-used-rulefiles.yaml
>> /var/ipfire/suricata/suricata-homenet.yaml
>> /var/ipfire/suricata/suricata-http-ports.yaml
>> /var/ipfire/suricata/suricata-sslbl_blacklist-used-rulefiles.yaml
>> /var/ipfire/suricata/suricata-used-providers.yaml
>> /var/ipfire/time/
>> /var/ipfire/time/counter.conf
>> /var/ipfire/time/enable
>> /var/ipfire/time/settime.conf
>> /var/ipfire/time/settings
>> /var/ipfire/upnp/settings
>> /var/ipfire/urlfilter
>> /var/ipfire/urlfilter/settings
>> /var/ipfire/urlfilter/squidGuard.conf
>> /var/ipfire/vpn
>> /var/ipfire/vpn/config
>> /var/ipfire/vpn/ipsec.conf
>> /var/ipfire/vpn/settings
>> /var/ipfire/wakeonlan/clients.conf
>> /var/ipfire/wio/wio.conf
>> /var/ipfire/wireless/config
>> /var/ipfire/wireless/settings
>> /var/lib/suricata
>> /var/log/rrd/collectd
>> /var/log/rrd/hddshutdown-md127.rrd
>> /var/log/rrd/hddshutdown-sda.rrd
>> /var/log/rrd/hddshutdown-sdb.rrd
>> /var/log/rrd/hddtemp-md127.rrd
>> /var/log/rrd/hddtemp-sda.rrd
>> /var/log/rrd/hddtemp-sdb.rrd
>> /var/log/rrd/wio
>> /var/log/vnstat
>> /var/tmp/idsrules-emerging.tar.gz
>> /var/tmp/idsrules-sslbl_blacklist.rules
>>
>> Regards,
>> Adolf.
>>>
>>> Bernhard
>>>> -Michael
>>>>
>>>>> On 29 Mar 2022, at 14:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> Just tried this patch out on my vm testbed system and it still 
>>>>> doesn't work for me. The backup file had got to 1.3GB when I 
>>>>> deleted the backup file as it was still growing. The normal correct 
>>>>> backup file on that vm machine is around 7MB
>>>>>
>>>>> The above was the case for both running it from the WUI or from the 
>>>>> command line from my unprivileged user using sudo backupctrl exclude
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>
>>>>>
>>>>> On 29/03/2022 14:27, Michael Tremer wrote:
>>>>>> This patch fixes globbing expansion in the backup include file list
>>>>>> which got broken in c7e0d73e7cfd7be95db9d0a5f3392b8241813d5b.
>>>>>>
>>>>>> Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
>>>>>> ---
>>>>>>   config/backup/backup.pl | 8 ++++----
>>>>>>   1 file changed, 4 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> diff --git a/config/backup/backup.pl b/config/backup/backup.pl
>>>>>> index a2337cf23..6f9295e94 100644
>>>>>> --- a/config/backup/backup.pl
>>>>>> +++ b/config/backup/backup.pl
>>>>>> @@ -19,6 +19,8 @@
>>>>>> # #
>>>>>> ############################################################################### 
>>>>>>
>>>>>>   +shopt -s nullglob
>>>>>> +
>>>>>>   NOW="$(date "+%Y-%m-%d-%H:%M")"
>>>>>>     list_addons() {
>>>>>> @@ -38,10 +40,8 @@ process_includes() {
>>>>>>       for include in $@; do
>>>>>>           local file
>>>>>>           while read -r file; do
>>>>>> -            for file in ${file}; do
>>>>>> -                if [ -e "/${file}" ]; then
>>>>>> -                    echo "${file}"
>>>>>> -                fi
>>>>>> +            for file in /${file}; do
>>>>>> +                echo "${file}"
>>>>>>               done
>>>>>>           done < "${include}"
>>>>>>       done | sort -u
>>>>