Skip to main content

NQE

Network Query Engine (NQE) enables your network and security teams to search your network just like a database. NQE provides an open platform for accessing information about your network in easy to read reports and diagrams.

  • 166 Topics
  • 0 Replies
166 Topics
G
Employee
GaryBEmployee
posted in NQE Discussions
Complex state parsing example using Juniper inventory Part 2

In Part 1 we introduced several techniques for dealing with semi-structured data to parse a Juniper Inventory. In Part 2 we will leverage a few more techniques to parse a more complex output. 1. Let’s setup our test by grabbing the output from the “show chassis hardware” command. remember to use the multi-string block delimiter “””We have 5 columns Item, Version, Part number, Serial number and Description, but we are only interested in three: Item, Serial number and Description. output = """Hardware inventory:Item Version Part number Serial number DescriptionChassis JN2221837ABC MX960Midplane REV 03 710-013698 TR0123 MX960 BackplaneFPM Board REV 03 710-014974 JZ1111 Front Panel DisplayPDM Rev 03 740-013110 QCS1012345U Power Distribution ModulePEM 0 Rev 07 740-029344 QCS1012346U DC 4.1kW Power Entry ModulePEM 1 Rev 07 740-029344 Q

1770
G
Employee
G
Employee
GaryBEmployee
posted in NQE Discussions
Complex state parsing example using Juniper inventory Part 1

Sometimes you need to parse out information that is not in the NQE data model for various reasons. It maybe device specific information that is not generalizable across vendors, or not generally useful in dataplane analysis but maybe important for operations. Importing this data is beyond the scope of this post but you can read more about it here: Getting Custom CommandsFirst, I am going to demonstrate how you build up your parsing chops by leveraging a testing approach to NQE, then will demonstrate how this can be applied to your collection data. Setting up your testing. We need to represent the data that our custom command will import into the snapshot. We can represent this by setting a variable to a multiline string using the delimiter ”””output = """Item Version Part number Serial number FRU model numberMidplane REV 03 710-013698 TR0001 CHAS-BP-MX960-SFPM Board REV 03 710-014974 JZ2323 CRAFT-MX960-SPEM 0 Re

1700
G
Employee
cariddir
Spotter
cariddirSpotter
posted in NQE Discussions
BFD Checks - Arista & Cisco-IOSXE

This is a check I wrote to check BFD Status on Arista and IOS-XE as a single combined query. Unfortunately, I can’t change the name of the title to include Arista && IOS-XEscreenShot /*** @intent BFD State Arista & IOS-XE* @description Add Command 'show bfd neighbors || peers' and report status - apply to EOS and IOX-XE Devices**/EOS_BFDPattern = ```{DstAddr:ipv4Address} {MyDisc:number} {YourDisc:string} {Interface:string} {Type:string} {LastUp:string} {LastDown:string} {LastDiag:string} {State1:string} {State2:string} {State3:string} {State4:string} ```;iosXE_BFDPattern = ```{ip:ipv4Address} {LD:string} {RD:string} {State:string} {Int1:string}```;// Define EOS FunctionArista_BFD = foreach Device in network.deviceslet Platform = Device.platformwhere Platform.os == OS.ARISTA_EOSlet Outputs = Device.outputsforeach Command in Outputs.commandswhere Command.commandText == "show bfd peers"let parsed = parseConfigBlocks(OS.ARISTA_EOS, Command.response)let matchData = blockMatche

1201
M
Employee
GTurner
Employee
GTurnerEmployee
posted in NQE Discussions
Fun with NQE parsing and data extraction.

 Many customers often ask me about digging into data, especially when it involves sifting through device status or custom command outputs. Sometimes, it's a piece of cake to handle because the data is nicely structured, but other times, it's like solving a puzzle. Just last week, I was helping out a customer who needed to extract security policy data from their enforcement points to keep tabs on various compliance matters. The data they had to deal with was all over the place – lines of varying lengths, multiple strings scattered within the lines, and positional changes for the data we needed.This got me thinking, and I wanted to share some techniques for handling these tricky data chunks. I can't share the exact customer query with you, but I've thrown together a sample query to show one way of tackling this.Imagine you've got data like this:"Don't teach me how to extract stuff with IP address 10.1.1.1 and float 10 from string." "Please teach me how not to extract stuff with IP addres

2810
GTurner
Employee
cariddir
Spotter
cariddirSpotter
posted in NQE Discussions
NQE Validate DNS Configuration on Arista & Cisco

Audit our DNS configuration on Arista Devices and Cisco Devices - Would like to be able to combine the script as a single check, and started it (Second Code Script), but it’s not working yet, I need a little tutoring. /** * @intent Audit DNS Configuration on Arista * @description Searches through the running configuration for a pattern match. */DNS_Standard = ["name-server 1.7.7.7 & 1.7.7.8", "dns domain net.xyz.com ", "ip domain lookup source-interface Loopback0"];AristaPatternDNS =```ip domain lookup source-interface Loopback0dns domain net.xyz.comip name-server vrf default {ipv4Address} ip name-server vrf default {ipv4Address} ```;/* Select a list of devices by vendor */foreach device in network.deviceswhere device.platform.vendor == Vendor.ARISTA || device.platform.os == OS.ARISTA_EOSlet outputs = device.outputsforeach command in outputs.commandswhere command.commandType == CommandType.CONFIGlet response = command.response/* parse out the pattern defined above from the respons

1352
cariddir
Spotter
Novak
NovakRamping Up
asked in NQE Discussions
Building new NQE dashboards

Is there a reference to building new dashboards using NQE derived values as inputs?

531
Tyson Henrie
Employee
Kachow
KachowRamping Up
asked in NQE Discussions
Adding additional values as columns

Hello Team, we have an NQE thatsearches all L2 switches obtains each interface status obtains speed obtains sfp informationfor all ports in up state. So far, we have the following:foreach device in network.devicesforeach item in device.platform.componentswhere item.partType == DevicePartType.TRANSCEIVERlet interface = (foreach interface in device.interfaces where isPresent(findInterface(device, item.name)) where interface.name == findInterface(device, item.name).interface.name select interface)where length(interface) == 1let int = min(interface)select { deviceName: device.name, iface: int.name, speed: int.ethernet.negotiatedPortSpeed, partType: item.partType, partName: item.name,} Is there a way we could extract and add the below underlined in BLUE as additional columns as well?  Thanks in advance for your time and help

401
Tyson Henrie
Employee
G
gschmittRamping Up
asked in NQE Discussions
Collecting device info based on CSV of IP addresses

I want to feed a list of ~5K IP addresses into forward and track the IP down to whatever info is available (device interface, standby address, device interface neighbor, host).I have two functions that build the lists of records that I need to join by ip address.FromCSV  (~5K records) and allKnownIps  (~200-300K records)What is the most efficient / elegant way to join essentially join these tables?I tried to build the smaller list first and then build the known IP list only if there is a match and then ran a nested loopcsvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPsallknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlistforeach knownIp in allknownIpListforeach csvIP in FromCSVwhere knownIp.ipAddr == csvIP.ipaddrselect{};So I have two questions:Is there a more elegant way to do this? 5K IP's trying to match against the 250K Potential IP's (I have normalized everything so that it's ipv4

862
J
Employee
Novak
NovakRamping Up
asked in NQE Discussions
I wrote a useful NQE query with no help! Could it be more efficient?

I finally managed to write a useful NQE query with no help. Yay! This query lists all of the loopback interfaces and their IP addresses. I tested it, and it works. But I am curious if there is a more efficient or elegant way to enumerate the IP addresses. Seems to me a little clumsy. getSubIfaceIpInfos(iface) = foreach subIface in iface.subinterfaces foreach subnet in subIface.ipv4.addresses select subnet;foreach device in network.devicesforeach interface in device.interfaceswhere interface.loopbackModelet ifaceSubnets = getSubIfaceIpInfos(interface)select { deviceName: device.name, interfaceName: interface.name, ipAddress: foreach subnet in ifaceSubnets select ipSubnet(subnet.ip, subnet.prefixLength)} 

881
J
Employee
G
gschmittRamping Up
asked in NQE Discussions
Prettify is removing parenthesis

I am trying to use a where filter like so: where (a && b) || cPrettify keeps removing the parenthesis. Doesn’t this mess with the order of operations?

252
G
Lane
LaneRamping Up
asked in NQE Discussions
Inventory device state from Description field

I am trying to use the harvest the INVENTORY device state file but target the Description field, but I am unsure on on how to do that so that I can select it. I have the following:import "External/Juniper-EoS";foreach device in network.deviceswhere device.platform.vendor == Vendor.JUNIPERlet outputs=device.outputsforeach command in outputs.commandswhere command.commandType == CommandType.INVENTORYlet platform = device.platformwhere isPresent(platform.model)foreach part in platform.componentswhere isPresent(part.serialNumber)select { deviceName: device.name, model: platform.model, part: part.serialNumber, violation: if part.partId in juniper_eos then true else false} 

621
A
Employee
J
JenRamping Up
asked in NQE Discussions
"Or" statement in NQE?

Is there a way to add an "or" statement in a NQE? We are checking IOS versions against a model and Cisco says it can be this or that but the NQE we have looks like it's looking for both and throwing a false positive.

503
J
Lane
LaneRamping Up
asked in NQE Discussions
Filter out duplicate MAC entries

Is there a way to filter out duplicate MAC entries to get a unique MAC address count? Using the standard NQE library query under "L2 -> Mac Entries" returns every VLAN and device a MAC entry is seen on, skewing the results as shown in the example image.  

661
Rich
Employee
davetee
Community Manager
daveteeCommunity Manager
published in NQE Discussions
How to share an NQE with the communityBeginner

We created the Forward Networks Community as a place to share NQE solutions and collaborate on innovative ways to better understand our networks. Network Query Engine (NQE) enables your network and security teams to search your network just like a database. NQE provides an open platform for accessing information about your network in easy to read reports and diagrams. This offers an interface for the uniform presentation of dozens of vendors, thousands of devices, billions of lines of configuration code, and multiple public cloud providers across your enterprise network.  ❗Before you share any NQE script or screenshots from Forward Networks, ensure you remove sensitive information. This may include host names, IP addresses, company names, or other information you’d prefer to keep private. You can also choose an anonymous username if you wish to have an additional layer of privacy.  What to include when you share an NQE: What it doesInclude a short description of what your NQE script do

850
davetee
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE - Using String Data TypeBeginner

 Curious on how to work with strings in NQE?  This video covers the basics of working with the String datatype in NQE.  Let us know what have you done with strings that made your co-workers say “How did you do that!”  Post your queries or questions below 👇🏻 Code examples used in videoReturn all devices where the first three characters of the device name is atlforeach device in network.deviceswhere prefix(device.name, 3) == "atl" select { deviceName: device.name}Return all devices where the beginning of the device name is atlforeach device in network.deviceswhere matches(device.name, "atl*")select { deviceName: device.name}Return all devices where the beginning of the device name is atl and contains the word spineforeach device in network.deviceswhere matches(device.name, "atl*spine*")select { deviceName: device.name} 

1520
Mike
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
How to Navigate the NQE LibraryBeginner

 In todays NQE video, join Mike on learning how to navigate the NQE library!    Have any questions?  Post them in the comments below 👇🏻

1250
Mike
Community Manager
  • NQE Contributor
    AricaFNhas earned the badge NQE Contributor
  • Community Champ Award
    CarlBhas earned the badge Community Champ Award
  • Community Champ Award
    Christopherhas earned the badge Community Champ Award
  • NQE Collaborator
    Rohit_809 Kumarhas earned the badge NQE Collaborator
  • Community Champ Award
    Vladi Bhas earned the badge Community Champ Award
Show all badges
Terms of UseCookie settings