Skip to main content

NQE

Network Query Engine (NQE) enables your network and security teams to search your network just like a database. NQE provides an open platform for accessing information about your network in easy to read reports and diagrams.

  • 150 Topics
  • 0 Replies
150 Topics
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE Foreach and Where StatementsTutorial

In today's video I will introduce the NQE foreach and where statements. We explore the data model and how to start off your NQE queries as well as go into when and why you would want to use a where statement.Code examples used in videoReturn all IPsec tunnels and create a report of the peer IP address, interface status and operational statusforeach device in network.devicesforeach interface in device.interfaceswhere interface.interfaceType == IfaceType.IF_TUNNEL_IPSECselect { deviceName: device.name, tunnelName: interface.name, peerIP: interface.tunnel.dst, adminStatus: interface.adminStatus, operStatus: interface.operStatus}You could modify the above to violate if the administrative status is up but the operational status not up with the following NQE query:foreach device in network.devicesforeach interface in device.interfaceswhere interface.interfaceType == IfaceType.IF_TUNNEL_IPSECselect { deviceName: device.name, tunnelName: interface.name, peerIP: interface.tunnel.dst, v

1240
Mike
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
What is NQE?Tutorial

 Join Mike on a journey to introduce a feature of the Forward Enterprise platform called the Network Query Engine (A.K.A NQE) and why it will make your job as a network or security operator much easier.   Let us know what you think about NQE; what do you see yourself using it for?  If you have any questions ask them below 👇🏻 Code example used in videoReturn a violation if an interface on a Cisco IOS XE device is administratively up but operationally downforeach device in network.devicesforeach interface in device.interfaceswhere device.platform.os == OS.IOS_XEselect { deviceName: device.name, interfaceName: interface.name, adminStatus: interface.adminStatus, operStatus: interface.operStatus, violation: interface.adminStatus == AdminStatus.UP && interface.operStatus != OperStatus.UP} 

3402
Mike
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE Comparison OperatorsTutorial

Join me for a look at the comparisons you can use in NQE to filter out on what data you need to see in your query. Code examples used in videoUsing equals (==) to return any default routes in any VRF foreach device in network.devicesforeach networkInstance in device.networkInstanceslet afts = networkInstance.aftslet ipv4Unicast = afts.ipv4Unicastforeach ipEntry in ipv4Unicast.ipEntrieswhere ipEntry.prefix == ipSubnet("0.0.0.0/0")foreach nh in ipEntry.nextHopswhere isPresent(nh.ipAddress)select { deviceName: device.name, vrfName: networkInstance.name, RouteEntry: ipEntry.prefix, nextHop: nh.ipAddress}Using not equal to (!=), to exclude ASA’s a platform inventory reportforeach device in network.deviceswhere device.platform.model != "ASAv"select { deviceName: device.name, deviceModel: device.platform.model }Choose whether or not to report on interface MTU that is:Greater than 1500 ( > ) Greater than or equal to 1500 ( >= ) Less than 1500 ( < ) Less than or equal to 1500

1110
Mike
Community Manager
captainpacket
Employee
captainpacketEmployee
posted in NQE Discussions
Streamlining Cloud Networks: An NQE Query for Finding Duplicate Subnets

In cloud environments, the complexity of network architecture, especially with the prevalent use of NAT (Network Address Translation), often obscures visibility, making the detection of duplicate subnets a challenging task. This lack of transparency can lead to network inefficiencies, conflicts, and potential security risks. The NQE query I'm sharing specifically addresses this challenge, offering a streamlined approach to uncover hidden duplicate subnets, a crucial step in maintaining a robust and efficient cloud network. To tackle the issue of detecting overlapping IPv4 CIDR blocks across different VPCs in cloud environments, I've developed a precise NQE (Network Query Engine) query. This query methodically scans through all cloud accounts and their respective VPCs, identifying any instances where IPv4 CIDR blocks overlap. The query is designed to compare each VPC within a cloud account against others, ensuring comprehensive coverage and accurate detection of duplications. Below is t

830
captainpacket
Employee
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE - List Data TypeTutorial

Join me on an introduction to working with Lists in NQE. We cover some of the basic list comparisons and the list function length().Code examples used in videoReturn all Arista devices, and if the device is not running the approved OS, return a violation for that deviceapproved_os = ["4.15.0F"];foreach device in network.deviceswhere device.platform.os == OS.ARISTA_EOSselect { violation: device.platform.osVersion not in approved_os, name: device.name, version: device.platform.osVersion} There is another way to write this query. We can write it so that it will only return all devices that are violating versus returning every device and violating the devices that are not compliant. This query would look like:approved_os = ["4.15.0F"]foreach device in network.deviceswhere device.platform.os == OS.ARISTA_EOSwhere device.platform.osVersion not in approved_osselect {violation: truename: device.name, version: device.platform.osVersion} We are using where only to filter devices that are run

1090
Mike
Community Manager
R
rrogers33Ramping Up
asked in NQE Discussions
NQE for confirming configuration within Cisco switch interfaces

Hi!  Looking for help with an NQE to scan/identify configuration for all Cisco switches within a workspace that contain one of two QOS policy maps within at least one of the device interfaces. For example interface gigabitethernet1/1 service-policy output 1P7Q3T orservice-policy output 1P7Q1T Thanks        

1824
G
Employee
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE - Numbers and Booleans Data TypesTutorial

In todays video, we are going to look at the basics of working with numbers and booleans in NQE.  Code examples used in videoCheck to make sure that the distribution switches that run spanning-tree in a newly acquired DC are running the correct root bridge priority, if not show a violation rbPri = 5297;foreach device in network.devicesforeach stpRoot in device.stp.rapidPvst.vlanswhere matches(device.name, "sjc*dist*")select { violation: rbPri != stpRoot.bridgePriority, device: device.name, vlanID: stpRoot.vlanId }Any questions or comments, post them in the comments below 👇🏻

680
Mike
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
The NQE Files TabTutorial

In todays video, we are going to go into a deeper dive on the NQE files tab. We discuss the different states the NQE queries can be in and go over the permission structure in NQE. Any questions or comments, feel free to post them below

500
Mike
Community Manager
E
ebalaRamping Up
asked in NQE Discussions
BGP advertised and received routes for Arista EOS and NXOS devices

I am trying to get the BGP advertised and received prefixes for Arista EOS and Cisco NXOA devices. I tried the below two queries and they only work for IOS and IOS-XE. devicesBGP Received routes ----------------------------------------------------------foreach device in network.devices// where device.name == "jcdwans01"where isPresent(device.bgpRib)let bgpRib = device.bgpRibforeach afiSafi in bgpRib.afiSafisforeach neighbor in afiSafi.neighborswhere isPresent(neighbor.adjRibInPost)let adjRibInPost = neighbor.adjRibInPostselect {  deviceName: device.name,  platform: device.platform.os,  afiSafiAfiSafiName: afiSafi.afiSafiName,  neighborNeighborAddress: neighbor.neighborAddress,  routesCount: length(adjRibInPost.routes)}BGP advertised routes -foreach device in network.deviceswhere isPresent(device.bgpRib)let bgpRib = device.bgpRibforeach afiSafi in bgpRib.afiSafisforeach neighbor in afiSafi.neighborswhere isPresent(neighbor.adjRibOutPost)let adjRibOutPost = neighbor.adjRibOutPostforeach

2211
Tyson Henrie
Employee
G
Employee
GaryBEmployee
posted in NQE Discussions
Complex state parsing example using Juniper inventory Part 2

In Part 1 we introduced several techniques for dealing with semi-structured data to parse a Juniper Inventory. In Part 2 we will leverage a few more techniques to parse a more complex output. 1. Let’s setup our test by grabbing the output from the “show chassis hardware” command. remember to use the multi-string block delimiter “””We have 5 columns Item, Version, Part number, Serial number and Description, but we are only interested in three: Item, Serial number and Description. output = """Hardware inventory:Item Version Part number Serial number DescriptionChassis JN2221837ABC MX960Midplane REV 03 710-013698 TR0123 MX960 BackplaneFPM Board REV 03 710-014974 JZ1111 Front Panel DisplayPDM Rev 03 740-013110 QCS1012345U Power Distribution ModulePEM 0 Rev 07 740-029344 QCS1012346U DC 4.1kW Power Entry ModulePEM 1 Rev 07 740-029344 Q

1560
G
Employee
G
Employee
GaryBEmployee
posted in NQE Discussions
Complex state parsing example using Juniper inventory Part 1

Sometimes you need to parse out information that is not in the NQE data model for various reasons. It maybe device specific information that is not generalizable across vendors, or not generally useful in dataplane analysis but maybe important for operations. Importing this data is beyond the scope of this post but you can read more about it here: Getting Custom CommandsFirst, I am going to demonstrate how you build up your parsing chops by leveraging a testing approach to NQE, then will demonstrate how this can be applied to your collection data. Setting up your testing. We need to represent the data that our custom command will import into the snapshot. We can represent this by setting a variable to a multiline string using the delimiter ”””output = """Item Version Part number Serial number FRU model numberMidplane REV 03 710-013698 TR0001 CHAS-BP-MX960-SFPM Board REV 03 710-014974 JZ2323 CRAFT-MX960-SPEM 0 Re

1520
G
Employee
cariddir
Spotter
cariddirSpotter
posted in NQE Discussions
BFD Checks - Arista & Cisco-IOSXE

This is a check I wrote to check BFD Status on Arista and IOS-XE as a single combined query. Unfortunately, I can’t change the name of the title to include Arista && IOS-XEscreenShot /*** @intent BFD State Arista & IOS-XE* @description Add Command 'show bfd neighbors || peers' and report status - apply to EOS and IOX-XE Devices**/EOS_BFDPattern = ```{DstAddr:ipv4Address} {MyDisc:number} {YourDisc:string} {Interface:string} {Type:string} {LastUp:string} {LastDown:string} {LastDiag:string} {State1:string} {State2:string} {State3:string} {State4:string} ```;iosXE_BFDPattern = ```{ip:ipv4Address} {LD:string} {RD:string} {State:string} {Int1:string}```;// Define EOS FunctionArista_BFD = foreach Device in network.deviceslet Platform = Device.platformwhere Platform.os == OS.ARISTA_EOSlet Outputs = Device.outputsforeach Command in Outputs.commandswhere Command.commandText == "show bfd peers"let parsed = parseConfigBlocks(OS.ARISTA_EOS, Command.response)let matchData = blockMatche

1151
M
Employee
GTurner
Employee
GTurnerEmployee
posted in NQE Discussions
Fun with NQE parsing and data extraction.

 Many customers often ask me about digging into data, especially when it involves sifting through device status or custom command outputs. Sometimes, it's a piece of cake to handle because the data is nicely structured, but other times, it's like solving a puzzle. Just last week, I was helping out a customer who needed to extract security policy data from their enforcement points to keep tabs on various compliance matters. The data they had to deal with was all over the place – lines of varying lengths, multiple strings scattered within the lines, and positional changes for the data we needed.This got me thinking, and I wanted to share some techniques for handling these tricky data chunks. I can't share the exact customer query with you, but I've thrown together a sample query to show one way of tackling this.Imagine you've got data like this:"Don't teach me how to extract stuff with IP address 10.1.1.1 and float 10 from string." "Please teach me how not to extract stuff with IP addres

2460
GTurner
Employee
cariddir
Spotter
cariddirSpotter
posted in NQE Discussions
NQE Validate DNS Configuration on Arista & Cisco

Audit our DNS configuration on Arista Devices and Cisco Devices - Would like to be able to combine the script as a single check, and started it (Second Code Script), but it’s not working yet, I need a little tutoring. /** * @intent Audit DNS Configuration on Arista * @description Searches through the running configuration for a pattern match. */DNS_Standard = ["name-server 1.7.7.7 & 1.7.7.8", "dns domain net.xyz.com ", "ip domain lookup source-interface Loopback0"];AristaPatternDNS =```ip domain lookup source-interface Loopback0dns domain net.xyz.comip name-server vrf default {ipv4Address} ip name-server vrf default {ipv4Address} ```;/* Select a list of devices by vendor */foreach device in network.deviceswhere device.platform.vendor == Vendor.ARISTA || device.platform.os == OS.ARISTA_EOSlet outputs = device.outputsforeach command in outputs.commandswhere command.commandType == CommandType.CONFIGlet response = command.response/* parse out the pattern defined above from the respons

1232
cariddir
Spotter
Novak
NovakRamping Up
asked in NQE Discussions
Building new NQE dashboards

Is there a reference to building new dashboards using NQE derived values as inputs?

531
Tyson Henrie
Employee
Kachow
KachowRamping Up
asked in NQE Discussions
Adding additional values as columns

Hello Team, we have an NQE thatsearches all L2 switches obtains each interface status obtains speed obtains sfp informationfor all ports in up state. So far, we have the following:foreach device in network.devicesforeach item in device.platform.componentswhere item.partType == DevicePartType.TRANSCEIVERlet interface = (foreach interface in device.interfaces where isPresent(findInterface(device, item.name)) where interface.name == findInterface(device, item.name).interface.name select interface)where length(interface) == 1let int = min(interface)select { deviceName: device.name, iface: int.name, speed: int.ethernet.negotiatedPortSpeed, partType: item.partType, partName: item.name,} Is there a way we could extract and add the below underlined in BLUE as additional columns as well?  Thanks in advance for your time and help

401
Tyson Henrie
Employee
G
gschmittRamping Up
asked in NQE Discussions
Collecting device info based on CSV of IP addresses

I want to feed a list of ~5K IP addresses into forward and track the IP down to whatever info is available (device interface, standby address, device interface neighbor, host).I have two functions that build the lists of records that I need to join by ip address.FromCSV  (~5K records) and allKnownIps  (~200-300K records)What is the most efficient / elegant way to join essentially join these tables?I tried to build the smaller list first and then build the known IP list only if there is a match and then ran a nested loopcsvListIPs = (foreach ip in FromCSV select ip.ipAddr);// List of ~5K IPsallknownIpList = (foreach knownIp in allKnownIps where knownIp.ipAddr in csvListIPs select ip);//Filter for only known IPs in CSVlistforeach knownIp in allknownIpListforeach csvIP in FromCSVwhere knownIp.ipAddr == csvIP.ipaddrselect{};So I have two questions:Is there a more elegant way to do this? 5K IP's trying to match against the 250K Potential IP's (I have normalized everything so that it's ipv4

812
J
Employee
Novak
NovakRamping Up
asked in NQE Discussions
I wrote a useful NQE query with no help! Could it be more efficient?

I finally managed to write a useful NQE query with no help. Yay! This query lists all of the loopback interfaces and their IP addresses. I tested it, and it works. But I am curious if there is a more efficient or elegant way to enumerate the IP addresses. Seems to me a little clumsy. getSubIfaceIpInfos(iface) = foreach subIface in iface.subinterfaces foreach subnet in subIface.ipv4.addresses select subnet;foreach device in network.devicesforeach interface in device.interfaceswhere interface.loopbackModelet ifaceSubnets = getSubIfaceIpInfos(interface)select { deviceName: device.name, interfaceName: interface.name, ipAddress: foreach subnet in ifaceSubnets select ipSubnet(subnet.ip, subnet.prefixLength)} 

861
J
Employee
G
gschmittRamping Up
asked in NQE Discussions
Prettify is removing parenthesis

I am trying to use a where filter like so: where (a && b) || cPrettify keeps removing the parenthesis. Doesn’t this mess with the order of operations?

252
G
Lane
LaneRamping Up
asked in NQE Discussions
Inventory device state from Description field

I am trying to use the harvest the INVENTORY device state file but target the Description field, but I am unsure on on how to do that so that I can select it. I have the following:import "External/Juniper-EoS";foreach device in network.deviceswhere device.platform.vendor == Vendor.JUNIPERlet outputs=device.outputsforeach command in outputs.commandswhere command.commandType == CommandType.INVENTORYlet platform = device.platformwhere isPresent(platform.model)foreach part in platform.componentswhere isPresent(part.serialNumber)select { deviceName: device.name, model: platform.model, part: part.serialNumber, violation: if part.partId in juniper_eos then true else false} 

621
A
Employee
J
JenRamping Up
asked in NQE Discussions
"Or" statement in NQE?

Is there a way to add an "or" statement in a NQE? We are checking IOS versions against a model and Cisco says it can be this or that but the NQE we have looks like it's looking for both and throwing a false positive.

503
J
Lane
LaneRamping Up
asked in NQE Discussions
Filter out duplicate MAC entries

Is there a way to filter out duplicate MAC entries to get a unique MAC address count? Using the standard NQE library query under "L2 -> Mac Entries" returns every VLAN and device a MAC entry is seen on, skewing the results as shown in the example image.  

651
Rich
Employee
davetee
Community Manager
daveteeCommunity Manager
published in NQE Discussions
How to share an NQE with the communityTutorial

We created the Forward Networks Community as a place to share NQE solutions and collaborate on innovative ways to better understand our networks. Network Query Engine (NQE) enables your network and security teams to search your network just like a database. NQE provides an open platform for accessing information about your network in easy to read reports and diagrams. This offers an interface for the uniform presentation of dozens of vendors, thousands of devices, billions of lines of configuration code, and multiple public cloud providers across your enterprise network.  ❗Before you share any NQE script or screenshots from Forward Networks, ensure you remove sensitive information. This may include host names, IP addresses, company names, or other information you’d prefer to keep private. You can also choose an anonymous username if you wish to have an additional layer of privacy.  What to include when you share an NQE: What it doesInclude a short description of what your NQE script do

840
davetee
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
NQE - Using String Data TypeTutorial

 Curious on how to work with strings in NQE?  This video covers the basics of working with the String datatype in NQE.  Let us know what have you done with strings that made your co-workers say “How did you do that!”  Post your queries or questions below 👇🏻 Code examples used in videoReturn all devices where the first three characters of the device name is atlforeach device in network.deviceswhere prefix(device.name, 3) == "atl" select { deviceName: device.name}Return all devices where the beginning of the device name is atlforeach device in network.deviceswhere matches(device.name, "atl*")select { deviceName: device.name}Return all devices where the beginning of the device name is atl and contains the word spineforeach device in network.deviceswhere matches(device.name, "atl*spine*")select { deviceName: device.name} 

1350
Mike
Community Manager
Mike
Community Manager
MikeCommunity Manager
published in NQE Discussions
How to Navigate the NQE LibraryTutorial

 In todays NQE video, join Mike on learning how to navigate the NQE library!    Have any questions?  Post them in the comments below 👇🏻

1190
Mike
Community Manager
  • Community Champ Award
    CarlBhas earned the badge Community Champ Award
  • Community Champ Award
    Christopherhas earned the badge Community Champ Award
  • NQE Collaborator
    Rohit_809 Kumarhas earned the badge NQE Collaborator
  • Community Champ Award
    Vladi Bhas earned the badge Community Champ Award
  • Community Champ Award
    dakotagloryhas earned the badge Community Champ Award
Show all badges
Terms of UseCookie settings