Splitting bundled database file out from auto-update flow

Splitting bundled database file out from auto-update flow

[Valters Jansons]

Hello,

Currently, I have a bit of a concern for package managers and the
database file updates.

The location tool wants to update /var/lib/location/database.db ---
however, there is also logic to bundle this file in package managers.
So, when there is a new package version, the package manager believes
it should be in charge of the file. When it looks at the update, it
sees that the "user" has made changes to the file. This results in a
conflict, and potentially a confusing message stating that the user
has made modifications to the file, while it has been automatic
updates.

It would be nice to split it into two distinct locations - one file
being modified only by the package manager (to be used as a fallback),
and the other file being the live dynamic auto-updated choice. Has
there been any consideration for supporting such a set-up?

Best regards,
Valters Jansons

Re: Splitting bundled database file out from auto-update flow

[Michael Tremer]

Hello Valters,

No, this has not been considered before, but it is always brought up for discussion whenever someone packages this for a new distribution because they have all different rules.

We followed the process of Debian and how they package clamav for example which also comes with a packaged database that will be updated by a third tool and the package manager will just have to deal with it.

I am not against having multiple paths where the database is being opened from, although I don’t think it is a good investment of disk space. The database is currently slightly over 40 MiB extracted, so I wouldn’t want to spend the space twice. Especially since the old database is not of any use after a newer version has been downloaded.

I am happy to accept patches if this suites your use-case for as long as the current one for Debian, Fedora and IPFire is not being modified.

Which distribution are you packaging this for that has these requirements, or is this a private project?

Best,
-Michael

> On 5 Jun 2026, at 12:05, Valters Jansons <valter.jansons@gmail.com> wrote:
> 
> Hello,
> 
> Currently, I have a bit of a concern for package managers and the
> database file updates.
> 
> The location tool wants to update /var/lib/location/database.db ---
> however, there is also logic to bundle this file in package managers.
> So, when there is a new package version, the package manager believes
> it should be in charge of the file. When it looks at the update, it
> sees that the "user" has made changes to the file. This results in a
> conflict, and potentially a confusing message stating that the user
> has made modifications to the file, while it has been automatic
> updates.
> 
> It would be nice to split it into two distinct locations - one file
> being modified only by the package manager (to be used as a fallback),
> and the other file being the live dynamic auto-updated choice. Has
> there been any consideration for supporting such a set-up?
> 
> Best regards,
> Valters Jansons
> 

Re: Splitting bundled database file out from auto-update flow

[Valters Jansons]

I can understand the concern for redundant disk space usage.

It's actually Debian packaging that I am concerned about.

For your ClamAV reference, I cannot find where the current release
ships its database file inside the package manager. There are
references to a clamav-data package, which would do this, but I cannot
find the package itself right away --
https://salsa.debian.org/clamav-team/clamav/-/blob/unstable/debian/control#L120

Instead, they seem to expect clamav-freshclam binary to be installed
which would dynamically download the file --
https://packages.debian.org/sid/all/clamav-freshclam/filelist

In any case, I did a double-take on this, and it seems that Apt will
clobber the database file. It would only cause confusion for
configuration file directories, and /var/lib should be outside of that
scope. I would still suggest a /usr/share/location/database.db for
fallback, which could be packaged. But the doubled disk space usage
concern still stands, and it adds increased complexity on top of
current simple database file selection.

So I digress, it sounds reasonable to leave it as-is.

Thank you for the quick response in any case, and wish you a nice weekend!

Best regards,
Valters Jansons

On Fri, Jun 5, 2026 at 8:11 PM Michael Tremer <michael.tremer@ipfire.org> wrote:
>
> Hello Valters,
>
> No, this has not been considered before, but it is always brought up for discussion whenever someone packages this for a new distribution because they have all different rules.
>
> We followed the process of Debian and how they package clamav for example which also comes with a packaged database that will be updated by a third tool and the package manager will just have to deal with it.
>
> I am not against having multiple paths where the database is being opened from, although I don’t think it is a good investment of disk space. The database is currently slightly over 40 MiB extracted, so I wouldn’t want to spend the space twice. Especially since the old database is not of any use after a newer version has been downloaded.
>
> I am happy to accept patches if this suites your use-case for as long as the current one for Debian, Fedora and IPFire is not being modified.
>
> Which distribution are you packaging this for that has these requirements, or is this a private project?
>
> Best,
> -Michael
>
> > On 5 Jun 2026, at 12:05, Valters Jansons <valter.jansons@gmail.com> wrote:
> >
> > Hello,
> >
> > Currently, I have a bit of a concern for package managers and the
> > database file updates.
> >
> > The location tool wants to update /var/lib/location/database.db ---
> > however, there is also logic to bundle this file in package managers.
> > So, when there is a new package version, the package manager believes
> > it should be in charge of the file. When it looks at the update, it
> > sees that the "user" has made changes to the file. This results in a
> > conflict, and potentially a confusing message stating that the user
> > has made modifications to the file, while it has been automatic
> > updates.
> >
> > It would be nice to split it into two distinct locations - one file
> > being modified only by the package manager (to be used as a fallback),
> > and the other file being the live dynamic auto-updated choice. Has
> > there been any consideration for supporting such a set-up?
> >
> > Best regards,
> > Valters Jansons
> >
>

Re: Splitting bundled database file out from auto-update flow

[Michael Tremer]

Hello,

> On 5 Jun 2026, at 12:35, Valters Jansons <valter.jansons@gmail.com> wrote:
> 
> I can understand the concern for redundant disk space usage.
> 
> It's actually Debian packaging that I am concerned about.
> 
> For your ClamAV reference, I cannot find where the current release
> ships its database file inside the package manager. There are
> references to a clamav-data package, which would do this, but I cannot
> find the package itself right away --
> https://salsa.debian.org/clamav-team/clamav/-/blob/unstable/debian/control#L120

My information might be outdated. We packaged libloc for bookworm and the approach for clamav might have changed since then.

I don’t maintain that package, so I would be happy to learn if there are new rules.

> Instead, they seem to expect clamav-freshclam binary to be installed
> which would dynamically download the file --
> https://packages.debian.org/sid/all/clamav-freshclam/filelist

Yes, this would pretty much happen very quickly after the package has been installed, but we decided that we want at least *some* database to be available immediately. It will be outdated, but it is better than nothing.

> In any case, I did a double-take on this, and it seems that Apt will
> clobber the database file. It would only cause confusion for
> configuration file directories, and /var/lib should be outside of that
> scope. I would still suggest a /usr/share/location/database.db for
> fallback, which could be packaged. But the doubled disk space usage
> concern still stands, and it adds increased complexity on top of
> current simple database file selection.

I don’t think it would be too complex, but more complex than what we have now indeed.

> So I digress, it sounds reasonable to leave it as-is.
> 
> Thank you for the quick response in any case, and wish you a nice weekend!

No problem. I appreciate your thoughts and patches very much. So please keep them coming.

Best,
-Michael

> Best regards,
> Valters Jansons
> 
> On Fri, Jun 5, 2026 at 8:11 PM Michael Tremer <michael.tremer@ipfire.org> wrote:
>> 
>> Hello Valters,
>> 
>> No, this has not been considered before, but it is always brought up for discussion whenever someone packages this for a new distribution because they have all different rules.
>> 
>> We followed the process of Debian and how they package clamav for example which also comes with a packaged database that will be updated by a third tool and the package manager will just have to deal with it.
>> 
>> I am not against having multiple paths where the database is being opened from, although I don’t think it is a good investment of disk space. The database is currently slightly over 40 MiB extracted, so I wouldn’t want to spend the space twice. Especially since the old database is not of any use after a newer version has been downloaded.
>> 
>> I am happy to accept patches if this suites your use-case for as long as the current one for Debian, Fedora and IPFire is not being modified.
>> 
>> Which distribution are you packaging this for that has these requirements, or is this a private project?
>> 
>> Best,
>> -Michael
>> 
>>> On 5 Jun 2026, at 12:05, Valters Jansons <valter.jansons@gmail.com> wrote:
>>> 
>>> Hello,
>>> 
>>> Currently, I have a bit of a concern for package managers and the
>>> database file updates.
>>> 
>>> The location tool wants to update /var/lib/location/database.db ---
>>> however, there is also logic to bundle this file in package managers.
>>> So, when there is a new package version, the package manager believes
>>> it should be in charge of the file. When it looks at the update, it
>>> sees that the "user" has made changes to the file. This results in a
>>> conflict, and potentially a confusing message stating that the user
>>> has made modifications to the file, while it has been automatic
>>> updates.
>>> 
>>> It would be nice to split it into two distinct locations - one file
>>> being modified only by the package manager (to be used as a fallback),
>>> and the other file being the live dynamic auto-updated choice. Has
>>> there been any consideration for supporting such a set-up?
>>> 
>>> Best regards,
>>> Valters Jansons
>>> 
>> 

Re: Splitting bundled database file out from auto-update flow

[Jochen Sprickerhof]

[-- Attachment #1: Type: text/plain, Size: 1544 bytes --]

Hi Valters,

* Valters Jansons <valter.jansons@gmail.com> [2026-06-05 20:05]:
>Hello,
>
>Currently, I have a bit of a concern for package managers and the
>database file updates.
>
>The location tool wants to update /var/lib/location/database.db ---
>however, there is also logic to bundle this file in package managers.
>So, when there is a new package version, the package manager believes
>it should be in charge of the file. When it looks at the update, it
>sees that the "user" has made changes to the file. This results in a
>conflict, and potentially a confusing message stating that the user
>has made modifications to the file, while it has been automatic
>updates.
>
>It would be nice to split it into two distinct locations - one file
>being modified only by the package manager (to be used as a fallback),
>and the other file being the live dynamic auto-updated choice. Has
>there been any consideration for supporting such a set-up?

This should already be the case in Debian. There is an initial Database 
shipped in libloc-database and installed into 
/usr/share/libloc-location/location.db:

https://packages.debian.org/sid/all/libloc-database/filelist

And libloc1t64 package installs a symlink by default that is outside of 
apt/dpkg control and can be replaces by an updated file:

https://sources.debian.org/src/libloc/0.9.18-3/debian/libloc1t64.postinst#L8

It is also cleaned up again on removal:

https://sources.debian.org/src/libloc/0.9.18-3/debian/libloc1t64.prerm

Cheers Jochen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]