Hi,
I'm currently evaluating whether I should use ipfire or RIPEstat for AS information. (low number of lookups, lookup latency is not an issue)
For tor relay IPs I found 54 ASNs where ipfire's DB has no AS name information. These are likely related to LACNIC and JPNIC.
Are there plans to add AS names for these regions as well or will this not improve in the near future (before 2022)?
kind regards, nusenu
+-----------+---------+ | as_number | country | +-----------+---------+ | AS61493 | ar | | AS7303 | ar | | AS18881 | br | | AS19182 | br | | AS262287 | br | | AS264381 | br | | AS267436 | br | | AS268207 | br | | AS269070 | br | | AS27699 | br | | AS27715 | br | | AS28126 | br | | AS28271 | br | | AS28573 | br | | AS28668 | br | | AS53024 | br | | AS53111 | br | | AS53187 | br | | AS7162 | br | | AS7738 | br | | AS8167 | br | | AS22047 | cl | | AS265831 | cl | | AS270013 | cl | | AS27651 | cl | | AS52368 | cl | | AS64111 | cl | | AS7418 | cl | | AS13489 | co | | AS11830 | cr | | AS3790 | cr | | AS52423 | cr | | AS28118 | do | | AS10010 | jp | | AS17676 | jp | | AS18126 | jp | | AS2510 | jp | | AS2514 | jp | | AS2516 | jp | | AS2527 | jp | | AS4685 | jp | | AS4694 | jp | | AS4713 | jp | | AS4725 | jp | | AS59103 | jp | | AS7506 | jp | | AS7684 | jp | | AS9370 | jp | | AS9371 | jp | | AS13999 | mx | | AS278 | mx | | AS6503 | mx | | AS8151 | mx | | AS27956 | pa | +-----------+---------+
Hi nusenu,
thanks for getting in touch again.
Are there plans to add AS names for these regions as well or will this not improve in the near future (before 2022)?
I have not yet looked at JPNIC in detail. If they provide a feed similar to RIPE, AFRINIC, or APNIC, integrating it should be done pretty quickly. (Same goes for TWNIC and KRNIC, since I am not sure whether their information is mirrored back to APNIC...)
LACNIC is - well - tedious. Perhaps we are going to scrape RIPEstat for missing AS names, where we would have this RIR covered. Personally, I would consider this the last option, but I don't think they will allow us to fetch the complete information from their servers.
(This remains a task for ARIN as well, since we are still missing some country information for suballocations there.)
Having a lot of other ${dayjob}-related things on my plate, I cannot give you an ETA for all of this. Let's hope it won't become 2022. :-)
Sorry for this rather vague answer.
Thanks, and best regards, Peter Müller
Hello nusenu,
since the JPNIC feed is pretty straightforward, I just submitted a patch for processing it as well (see: https://patchwork.ipfire.org/project/location/patch/8a35bdb5-401b-6a7f-2ae4-...).
A quick and dirty check with the AS list provided by you indicates we now have coverage for all of them:
$ for asn in $( cat nusenu-jpnic-missing-asns ); do location --database tmp-database get-as $asn; done AS2510 belongs to INFOWEB_FUJITSU AS2514 belongs to NTT PC Communications, Inc. AS2516 belongs to KDDI CORPORATION AS2527 belongs to Sony Network Communications Inc. AS4685 belongs to Asahi Net, Inc. AS4694 belongs to IDC Frontier Inc. AS4713 belongs to NTT Communications Corporation AS4725 belongs to SOFTBANK Corp. Open Data Network. AS7506 belongs to GMO Internet INC. AS7684 belongs to SAKURA Internet // HOKKAIDO BACKBONE AS9370 belongs to SAKURA Internet // EAST JAPAN BACKBONE AS9371 belongs to SAKURA Internet // WEST JAPAN BACKBONE AS10010 belongs to TOKAI AS17676 belongs to SoftBank Corp. AS18126 belongs to Chubu Telecommunications Company, Inc. AS59103 belongs to SoftEther Corporation
Since libloc 0.9.8 was released just a the other day (@Michael: Shouldn't we announce this properly?), this patch will go into the next version of it. If you are building libloc yourselves from scratch, you can also apply it to the current "master" branch.
Unless I missed something, TWNIC and KRNIC are mirroring important information back to APNIC, so we aren't missing anything here. This leaves us with LACNIC still in need of an improvement...
Thanks, and best regards, Peter Müller
Hello Peter,
Peter Müller:
Hello nusenu,
since the JPNIC feed is pretty straightforward, I just submitted a patch for processing it as well (see: https://patchwork.ipfire.org/project/location/patch/8a35bdb5-401b-6a7f-2ae4-...).
A quick and dirty check with the AS list provided by you indicates we now have coverage for all of them
thank you very much, I'm looking forward to using the new data in the near future.
kind regards, nusenu
Hi,
I came across 2 ASNs from JP which have no AS name information.
used version: Sun, 05 Dec 2021 06:00:44 GMT
Network : 133.175.0.0/16 Country : Japan Autonomous System : AS2519
Network : 49.242.192.0/18 Country : Japan Autonomous System : AS10013
https://stat.ripe.net/widget/prefix-overview#w.resource=%20133.175.0.0/16 https://stat.ripe.net/widget/prefix-overview#w.resource=49.242.192.0%2F18
Is there a specific reason why they have no AS name information despite the improvements for JPNIC?
kind regards, nusenu
Hello nusenu,
thanks for your mail. Please excuse my tardy reply.
Network : 133.175.0.0/16 Country : Japan Autonomous System : AS2519
Network : 49.242.192.0/18 Country : Japan Autonomous System : AS10013
Both AS lack an "aut-num" object in JPNIC, which is why there is no parseable information for them at hand in there. Not sure where RIPEstat gets their description from...
Anyways, ARIN, LACNIC and a better approach to missing data is on my list for next week. I'll see what we can to then.
Thanks, and best regards, Peter Müller
Peter Müller:
Both AS lack an "aut-num" object in JPNIC, which is why there is no parseable information for them at hand in there. Not sure where RIPEstat gets their description from...
Anyways, ARIN, LACNIC and a better approach to missing data is on my list for next week. I'll see what we can to then.
AS names (the holder's name) shown on RIPEstat come from the CIDR report https://www.cidr-report.org/as2.0/ according to RIPE NCC's reply.
Maybe this is a one for all solution.
kind regards, nusenu
Both AS lack an "aut-num" object in JPNIC, which is why there is no parseable information for them at hand in there. Not sure where RIPEstat gets their description from...
Anyways, ARIN, LACNIC and a better approach to missing data is on my list for next week. I'll see what we can to then.
AS names (the holder's name) shown on RIPEstat come from the CIDR report https://www.cidr-report.org/as2.0/ according to RIPE NCC's reply.
Maybe this is a one for all solution.
Since you apparently don't like this data source and I always thought RIPEstat has pretty good data quality: Would you mind sharing your opinion on this?
kind regards, nusenu
Hello,
On 16 Jan 2022, at 22:16, nusenu nusenu-lists@riseup.net wrote:
Both AS lack an "aut-num" object in JPNIC, which is why there is no parseable information for them at hand in there. Not sure where RIPEstat gets their description from...
Anyways, ARIN, LACNIC and a better approach to missing data is on my list for next week. I'll see what we can to then.
AS names (the holder's name) shown on RIPEstat come from the CIDR report https://www.cidr-report.org/as2.0/ according to RIPE NCC's reply. Maybe this is a one for all solution.
Since you apparently don't like this data source
Apparently?
and I always thought RIPEstat has pretty good data quality: Would you mind sharing your opinion on this?
I do not see the good data quality here. Some have proper names, others are called something like "XO-AS15”, or “CMCS”, or “AS17054”. This isn’t great.
Arguably, this is better than nothing and I wouldn’t mind using the description wherever we don’t have anything better, but it isn’t first choice.
@Peter: Do you want to look into extracting information from this?
-Michael
kind regards, nusenu -- https://nusenu.github.io
Hello nusenu, hello Michael,
Since you apparently don't like this data source and I always thought RIPEstat has pretty good data quality: Would you mind sharing your opinion on this?
sorry for not replying on this sooner. Actually, I do not like or dislike RIPEstat; I just did not have sufficient time to made myself an educated opinion on this.
At the moment, things are quite packed on my end, but that will hopefully over at the beginning of February. So, this is not forgotten or silently discarded, but just a very tardy reply due to my "load average"...
@Peter: Do you want to look into extracting information from this?
Yes.
Without looking at the amount of queries we'd probably need to do: Do you think this makes sense while running the location-importer, or should this become a dedicated script, which we can run in the background all the time, so it won't slow down the daily generation of the actual database.
In case of the latter, we could actually do some scraping on the ARIN AS names, too. Since we keep track of their source, this should not be too hard, and if we get some more human-readable names for some of them, it might be worth the effort.
Thanks, and best regards, Peter Müller
Hello,
On 18 Jan 2022, at 21:10, Peter Müller peter.mueller@ipfire.org wrote:
Hello nusenu, hello Michael,
Since you apparently don't like this data source and I always thought RIPEstat has pretty good data quality: Would you mind sharing your opinion on this?
sorry for not replying on this sooner. Actually, I do not like or dislike RIPEstat; I just did not have sufficient time to made myself an educated opinion on this.
At the moment, things are quite packed on my end, but that will hopefully over at the beginning of February. So, this is not forgotten or silently discarded, but just a very tardy reply due to my "load average"...
@Peter: Do you want to look into extracting information from this?
Yes.
Without looking at the amount of queries we'd probably need to do: Do you think this makes sense while running the location-importer, or should this become a dedicated script, which we can run in the background all the time, so it won't slow down the daily generation of the actual database.
In case of the latter, we could actually do some scraping on the ARIN AS names, too. Since we keep track of their source, this should not be too hard, and if we get some more human-readable names for some of them, it might be worth the effort.
I thought we were talking about parsing an HTML table. That should not be a process that is either complicated nor something I would call scraping.
Scraping is what I would consider sending one request per piece of information you would want to obtain and this is always a bad idea. It is slow, it has a lot of overhead on our side and of course on the server side - this is not what a good citizen of the internet would do. So I would be against this. Most places have this excluded in their t&cs for exactly this reason.
-Michael
Thanks, and best regards, Peter Müller
Michael Tremer:
Since you apparently don't like this data source
Apparently?
I'm glad if I misunderstood that.
I thought we were talking about parsing an HTML table.
I just reached out to RIPE NCC again since I was hoping there is some CSV file on that site somewhere. I'll let you know what their answer was once I have it.
kind regards, nusenu
-
Michael Tremer -
nusenu -
Peter Müller