Skip to main content
Solved

Collecting device info based on CSV of IP addresses

  • September 12, 2023
  • 2 replies
  • 94 views
  • Translate

Forum|alt.badge.img

I want to feed a list of ~5K IP addresses into forward and track the IP down to whatever info is available (device interface, standby address, device interface neighbor, host).


I have two functions that build the lists of records that I need to join by ip address.
FromCSV  (~5K records) and allKnownIps  (~200-300K records)


What is the most efficient / elegant way to join essentially join these tables?

I tried to build the smaller list first and then build the known IP list only if there is a match and then ran a nested loop

csvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPs
allknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlist
foreach knownIp in allknownIpList
foreach csvIP in FromCSV
where knownIp.ipAddr == csvIP.ipaddr
select{
};

So I have two questions:

  1. Is there a more elegant way to do this?
  2. 5K IP's trying to match against the 250K Potential IP's (I have normalized everything so that it's ipv4 addresses in string format without a prefix) only returns about 60 matches.   Upon spot checking I am not finding any IP's that forward actually knows that is being missed here, so I suspect that the input (I don't know yet where the IP's come from) is just IPs that forward doesn't know about, but is there anything obvious I am missing here?

Best answer by jack.shen

 

Your first two lines 

csvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPs

allknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlist

 

are doing the same as the next two lines 

foreach knownIp in allknownIpList

foreach csvIP in FromCSV

where knownIp.ipAddr == csvIP.ipadd

 

So I don’t see the benefit of having the first two lines. You can simply run a nest loop for both allnowIp and csvListIp and find IPs that exist in both list. 

 

For your second question, it is difficult to tshoot that relies on snapshot data. Make sure allKnownIpList aggregates all the interface ip and arp entries. 

 

 

View original
Did this topic help you find an answer to your question?

2 replies

Tyson Henrie
Employee
Forum|alt.badge.img+2
  • Employee
  • 70 replies
  • September 22, 2023

I believe you could get a bit better performance by removing a foreach in this statement.  

csvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPs
allknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlist

foreach knownIp in allknownIpList
where knownIp.ipAddr in csvListIPs
select{
known_IP: knownIp.ipAddr
}

The key part being

where knownIp.ipAddr in csvListIPs

“is knownIp.ipAddr contained within the list of csvListIPs”

That they provides a “yes” or “no” answer. If Yes, finish the loop and Select this knownIp info.  If No, do not Select this address, move on to the next address in allknownIpList.

Translate

  • Employee
  • 8 replies
  • Answer
  • September 22, 2023

 

Your first two lines 

csvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPs

allknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlist

 

are doing the same as the next two lines 

foreach knownIp in allknownIpList

foreach csvIP in FromCSV

where knownIp.ipAddr == csvIP.ipadd

 

So I don’t see the benefit of having the first two lines. You can simply run a nest loop for both allnowIp and csvListIp and find IPs that exist in both list. 

 

For your second question, it is difficult to tshoot that relies on snapshot data. Make sure allKnownIpList aggregates all the interface ip and arp entries. 

 

 

Translate

Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings