Re: What generates database.db from database.txt?

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 4995 bytes --]

Hi,

> On 21 Oct 2020, at 14:58, Gisle Vanem <gisle.vanem(a)gmail.com> wrote:
> 
>>> OPENSSL_Uplink(640BC6B0,08): no OPENSSL_Applink
>> I have absolutely no idea how to fix this unfortunately.
> 
> It cannot easily be solve unless libloc.dll, _location.pyd
> and the Python .dll use the same CRT.

And a CRT is what?

>> This is a small bug where we do not raise the error message with Python.
> 
> Why not? Not very user friendly to have such obscure abort.

This is not intended. I need to know what error is happening at what point.

We catch the usual ones like “file does not exist”, or the file simply has some content that we did not expect. Potentially, windows is retuning a different error code somewhere which we didn’t catch.

>>> Check yourself if the 'location-2020-05-15.db.xz' is still in the archive.
>> Did you extract the file?
> 
> Duh! Yes.

Just double-checking.

> BTW, what's up with test-network.c and IPv4 addresses?
> I did this patch:
> 
> --- a/test-network.c 2020-10-19 17:35:01
> +++ b/test-network.c 2020-10-21 14:02:41
> @@ -26,6 +26,8 @@
> #include <loc/network.h>
> #include <loc/writer.h>
> 
> +#define IPV4_TEST
> +
> int main(int argc, char** argv) {
>        int err;
> 
> @@ -116,7 +118,7 @@
>        }
> 
>        size_t nodes = loc_network_tree_count_nodes(tree);
> -       printf("The tree has %zu nodes\n", nodes);
> +       printf("The tree has %zu IPv6-only nodes\n", nodes);
> 
>        // Check subnet function
>        err = loc_network_is_subnet_of(network1, network2);
> @@ -160,6 +162,23 @@
>        // Set ASN
>        loc_network_set_asn(network4, 1024);
> 
> +#ifdef IPV4_TEST
> +       struct loc_network* network5;
> +       err = loc_writer_add_network(writer, &network5, "1.2.3.4/16");
> +       if (err) {
> +               fprintf(stderr, "Could not add IPv4 network\n");
> +               exit(EXIT_FAILURE);
> +       }
> +
> +       // Set country code
> +       loc_network_set_country_code(network4, "ZZ");

I assume you meant to set the country code for network5?

> +
> +       // Set ASN
> +       loc_network_set_asn(network4, 1234);

Same as above.

> +       loc_network_tree_dump(tree);
> +       printf("The tree has %zu IPv4/6-mixed nodes\n", loc_network_tree_count_nodes(tree));
> +#endif
> +
>        FILE* f = tmpfile();
>        if (!f) {
>                fprintf(stderr, "Could not open file for writing: %s\n", strerror(errno));
> @@ -203,6 +222,15 @@
>        }
>        loc_network_unref(network1);
> 
> +#ifdef IPV4_TEST
> +       err = loc_database_lookup_from_string(db, "1.2.3.4", &network5);
> +       if (err) {
> +               fprintf(stderr, "Could not look up 1.2.3.4\n");
> +               exit(EXIT_FAILURE);
> +       }
> +       loc_network_unref(network5);
> +#endif
> +
>        loc_unref(ctx);
>        fclose(f);
> 
> ---------
> 
> But I get some errors from 'loc_parse_address()'.
> So maybe it's better to check for an IPv4 first?
> With this:
>  https://github.com/gvanem/wsock-trace/commit/4dc6dc6e09aa024fdcedf1a87dd3f578fdbcfcc9

I do not really understand, again, what you are trying to achieve here.

IPv6 and IPv4 addresses can easily be mixed in the same database.

You are adding "1.2.3.4/16” which actually should have been written as "1.2.0.0/16”. The database will automatically convert this into “1.2.0.0/16” for you.

The result of loc_parse_address() should be the same no matter which one you are trying first (IPv6 or IPv4). In my code, I always put IPv6 first.

Clearly “struct in6_addr” is defined differently in Windows and therefore the latter part of those changes might be required. There will be other places in the code that read from that structure bit by bit to put the IP address into the tree. It is quite likely, that this will go wrong too and therefore loc_database_lookup won’t find the correct entry.

> it works much better (no errors from inet_pton()).
> But the node-count is stuck at 49:
> 
> libloc: loc_new: ctx 049D4CF8 created
> libloc: __loc_network_tree_dump: 2001:db8::/32
> libloc: __loc_network_tree_dump: 2001:db8:ffff::/48
> The tree has 49 IPv6-only nodes
> libloc: __loc_network_tree_dump: 2001:db8::/32
> libloc: __loc_network_tree_dump: 2001:db8:ffff::/48
> The tree has 49 IPv/6-mixed nodes
> libloc: loc_database_read_as_section_v1: Read 0 ASes from the database
> libloc: loc_database_read_network_nodes_section_v1: Read 207 network nodes from the database
> libloc: loc_database_read_networks_section_v1: Read 3 networks from the database
> libloc: loc_database_read_countries_section_v1: Read 0 countries from the database
> libloc: loc_database_read: Opened database in 14.0000ms
> 
> Are we not supposed to add both AF_INET+AF_INET6 addresses to
> the same tree?

Yes.

Best,
-Michael

P.S. Please do not forget to keep the list copied in your replies.

> -- 
> --gv