Re: What generates database.db from database.txt?

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 3907 bytes --]

Hello,

> On 21 Oct 2020, at 12:07, Gisle Vanem <gisle.vanem(a)gmail.com> wrote:
> 
> Michael Tremer wrote:
> 
>> This probably requires some more debugging and log output.
>> You can set LOC_LOG=7 as an environment variable and then run test/test-country.
> 
> With this, test-country only reports an extra:
>  libloc: loc_new: ctx 05021298 created
> 
> But with '-DENABLE_DEBUG' there's way too much trace
> to be useful.

That is the kind of debug information I would need.

> 
>> I pulled the database last week because we had some issue with some data in it that triggered some bugs.
>> You can download it from the archive still until the updater works again:
>>   https://location.ipfire.org/databases/1/archive/location-2020-10-16.db.xz
> 
> 
>> Yes, I assumed so. There are some tools that use but they might potentially just bundle it with their own software which I want to avoid.
>> It probably is possible to #ifdef the Windows crypto library here and check the signature that way. But since I have never worked with it, I do not know how to do that. Mac OS X has the same issue.
> 
> An issue with OpenSSL and 'loc_database_verify()'?

No, that OpenSSL isn’t easily available on that platform without shipping it manually.

>> Have you ever worked with it?
> 
> Yes, but 'test-signature' works fine. It's just when calling
> 'EVP_VerifyXX()' functions from a .dll (like a Python module), it
> fails with 'no OPENSSL_AppLink".
> Ref:
> https://stackoverflow.com/questions/38933943/openssl-code-works-natively-but-as-dll-causes-openssl-uplink-no-openssl-applink
> 
> With your URL above,
> set loc_log=7 & py -3 location.py --database location-2020-10-16.db verify:
> 
> libloc: loc_new: ctx 0FDACEE8 created
> libloc: loc_database_read_as_section_v1: Read 48148 ASes from the database
> libloc: loc_database_read_network_nodes_section_v1: Read 2628939 network nodes from the database
> libloc: loc_database_read_networks_section_v1: Read 1318972 networks from the database
> libloc: loc_database_read_countries_section_v1: Read 253 countries from the database
> libloc: loc_database_read: Opened database in 16.0000ms
> libloc: loc_database_read_as_section_v1: Read 48148 ASes from the database
> libloc: loc_database_read_network_nodes_section_v1: Read 2628939 network nodes from the database
> libloc: loc_database_read_networks_section_v1: Read 1318972 networks from the database
> libloc: loc_database_read_countries_section_v1: Read 253 countries from the database
> libloc: loc_database_read: Opened database in 27.0000ms
> OPENSSL_Uplink(640BC6B0,08): no OPENSSL_Applink

I have absolutely no idea how to fix this unfortunately.

I do not even have access to a Windows system to reproduce this.

> BUT, with 'location-2020-05-15.db', there is this error:
> 
> libloc: loc_new: ctx 0FA35190 created
> libloc: loc_database_read_header: Incompatible database version: 0
> Traceback (most recent call last):
>  File "F:\gv\VC_project\ws_trace\src\Geo-IP\IPFire\libloc\src\py3_install\location.py", line 618, in <module>
>    main()
>  File "F:\gv\VC_project\ws_trace\src\Geo-IP\IPFire\libloc\src\py3_install\location.py", line 616, in main
>    c.run()
>  File "F:\gv\VC_project\ws_trace\src\Geo-IP\IPFire\libloc\src\py3_install\location.py", line 200, in run
>    db = location.Database(args.database)
> SystemError: <class 'location.Database'> returned NULL without setting an error
> libloc: loc_unref: context 0FA35190 released

This is a small bug where we do not raise the error message with Python.

However, the debug output should tell you why this didn’t work.

The database version that should be shown there is 1. You have a zero.

> ----
> 
> Check yourself if the 'location-2020-05-15.db.xz' is still in the archive.

Did you extract the file?

Best,
-Michael

> 
> 
> -- 
> --gv