Hello, > On 5 Jun 2021, at 13:40, Peter Müller wrote: > > Hello Michael, > > thanks for your reply. > >>>>> (b) Introduce something like "location-importer update-3rd-party-feeds", which is a blanket function for >>>>> updating all the 3rd party feeds we will have at some day, as Amazon for sure won't be the only one. >>>> >>>> Does this need a third command? Why can this not be part of “update-whois”? >>> >>> Because we do not necessarily have the BGP data available at this step. If we want to build in AS-based >>> safeguards, we will have to parse 3rd party feeds after running "location-importer update-announcements". >> >> I would say that this is a difficult dependency. We should have the “owner” of that subnet somewhere in the WHOIS data which is deterministic while the BGP isn’t. >> >> There are “route” objects which should allow you to do what you want to do. > > Unfortunately, we do not have these data for ARIN and LACNIC space - at least not in a manner we can properly > use in an automated way. For the rest of the RIRs, I agree. I hate that we have to have a lot of duplicate code to deal with two RIRs. >> >>>> >>>>> (c) In case of Amazon, download their feed, parse it and put the results in a temporary table. >>>>> (d) Process a list of Autonomous Systems owned or controlled by Amazon. >>>> >>>> Where is this list coming from? >>> >>> Something similar to "countries.txt", I guess. It would definitely be something we will have to maintain >>> on our own. A simple .txt file per 3rd party source, containing one ASN per line, would do it in my point >>> of view. >> >> Hmm, okay. >> >>>> >>>>> (d) Delete every IP network from this temporary table which is not announced by one of the Autonomous >>>>> Systems. That way, we limit potential damage by a broken or manipulated Amazon IP feed to their ASNs. >>>> >>>> This is your second step (d). >>> >>> ? >> >> You mis-numbered them. Never mind. >> >>> >>>> >>>> When you say you are comparing this, what is the authority for this? The BGP feed? Whois? >>> >>> The BGP feed. We cannot rely on RIR data for this job, as they do not reflect reality and we don't have them >>> for ARIN- and LACNIC-maintained space. >> >> Well, interesting. I would have said the opposite. > > Normative power of the factual (a bit phony, I guess): If something is announced via BGP, traffic to it will > be routed towards the announced ASN - things like RPKI not taken into account here -, no matter what a RIR database > contains for this network. > > I don't like saying that, but I guess this is what we have. RPKI uses the RIR database. So we would just enforce RPKI for the entire database. I guess it is pretty much the same. > >> >>>>> (e) Anything left in the temporary table is safe to go, and will be merged into the overrides table. >>>>> >>>>> Sounds a bit complicated than my first patch looked like, but is more versatile and robust. :-) >>>> >>>> I kind of liked the first patch. It was simple and it worked. >>> >>> Indeed. But it allowed Amazon to inject arbitrary data. This is bad enough for RIRs already, I do not want >>> to extend the list of entities being able to do this to some profit-oriented companies... >> >> I consider that a different problem. Not necessarily less important, but probably we should not try to fix all problems in only one patch. > > All right, I agree. Given the state of discussion, I propose to work and submit a two-part patchset: The first > one adds a source column to the network_overrides, so we can debug things better on our systems at least, while > the second one imports Amazon AWS feed in the same way as I did initially. > > We would then care about additional safeguards later... *fingers crossed* :-] > > Would you agree on that? Okay. Merged. > > Thanks, and best regards, > Peter Müller