Skip to main content

When an NQE runs in parallel, it executes the same logic across all devices in the snapshot inventory simultaneously, rather than one device at a time in sequence. In practice, this can be the difference between a query that risks hitting the 20-minute timeout and one that completes in a fraction of the time.

 

When Parallelization Happens

A query will run in parallel if:

  1. It starts with an iteration over network.devices
  2. It does not access network.devices in any other way (only one iteration)
  3. It does not access network.endpoints
  4. It does not use the group … by qualifier

How to Check

From the NQE IDE:

  • Windows: Alt + left-click Execute
  • Mac: Option + left-click Execute

In the debug panel, look for:

parallel_foreach device in network.devices

AD_4nXcsoSLZJ_I3bi0wJY6f2skvLrRdbQpJA3wi2MLGj6ohSb0wPRMKmpX1lQglsnFEa6nX_gwVEJdGXNAg_RgwIJXjj7WmhwjFICQilXaMPTemayD0Gd6itV2boAg812wT-eEqVwGeOg?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXez4nfKqd9b4JLc08QXfG382NbTy7KQ_EnrFKPsIAJUqiiUcqL9KNmxo-TDrspD0YzQIumCgTLHzsg7AF7cG6tqI0caQxMV4mEju-OAWgnHa_gp4Uepds-M97nw71soLBFkeL5IUQ?key=U2w6P9NgFMJ_s__uwahN_Q

If you only see foreach device in network.devices, it’s not parallelized.

Example 1 — Pass the Device Object, Not Its Name

 

Broken (serial)

Passing device.name into a function and then re-looking up the device causes a second iteration over network.devices.

getDeviceVersion(deviceName) =

  foreach device in network.devices

  where device.name == deviceName

  select device.platform.osVersion;



foreach device in network.devices

foreach interface in device.interfaces

foreach subinterface in interface.subinterfaces

foreach arp in subinterface.ipv4.neighbors

select {

  device: device.name,

  deviceVersion: getDeviceVersion(device.name),

  interface: interface.name,

  arp: arp.ip,

  mac: arp.linkLayerAddress

}

 

AD_4nXcLDJNNiXlqLWfrkuUNd6zACZhe_SoF__ptlc7LmpzcZozIZvhxMSlYKejJpXLOGSly5Dve8LOmmSRnEQVnm8clYYLBHq__bFWbMwp59gik_15jjxPLYafLXAB-fgtLJXfp0_h3?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXfJgVOj6UxKkttOyWpBjxDI8ace7O5h_rdbZPv4hoeE9fSnSUn_myTDKJUVyYByt3sd7ZhRwotR5YkQ0RiGYOMFIeSmJxMXjvRqKQ7snQFU1b9E5FhLpNiToR6c-KgdnsI6pHcE?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)

foreach device in network.devices

AD_4nXfGyK7s2oj5Il-hyDvfg3KMRlrmKXmW7obsKdwGuS-NqIqzcCIeBaQRe3EPlMtBZ6AHdYVaC_OlAHE_F0HoJkvPyUln5kGq4AB94POdo4wISX09tUaG0rOExIFop6x9JJNrG9Za?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXdBDztJ5UVtu_uryNrXLkFDnnWAdSrEb2SwQZTbl7tdJuKFxpLpBdWShvAmqTkuZgFRJ6ru1ul8mXy4hNAAusVqLEO1Aefw2VDop0Tf2Kud2F8VVY7JGmXwECWBNS9G4we4jMF5gg?key=U2w6P9NgFMJ_s__uwahN_Q

Result: No parallel_foreach — not parallelized.

Fixed (parallel)

Pass the entire device object to the function to keep a single pass over network.devices.

getDeviceVersion(device) =

  foreach device in >device]

  select device.platform.osVersion;



foreach device in network.devices

foreach interface in device.interfaces

foreach subinterface in interface.subinterfaces

foreach arp in subinterface.ipv4.neighbors

select {

  device: device.name,

  deviceVersion: getDeviceVersion(device),

  interface: interface.name,

  arp: arp.ip,

  mac: arp.linkLayerAddress

}

 

AD_4nXeBX88vn2XuGEM5Y6P-nbihJ0-COyYIXQji4Q4dF39F9pDDwQO3hVHzBHnNGcRQ6flE6ogRxBRpAOl7QB0YN60wm931jhdtlvf-fwcoAi3RBlOWKzh55dZIwkJ5hxjpoLag7LBaDw?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXfFT8Xo8nxaUsgoXtA_kT_flmj5dxoSDIkyDskJZZLaFuNY7wCCWWvS2KWOUhL1vkqNFJAjmF7jITuNkqDRxNb_8OVoESwxcRANrC_ZqTqE4WBvRLHoIh-wIDRqji0odklcaqMa?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)

parallel_foreach device in network.devices

AD_4nXd0I3Mj5mLMhOIKSdxYNKAOShAaP5MbRPSeJTPVZsNayNCPuYEXB9FY2PInMGtzS8KdZ01HbK8La8e9xrv553uyj2SFbvJpT2ofrOiV2X25mMDzz2ub1w9_Uc7b3SWUmdwg5voWTw?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXcMgdv8PKAtd07ST8qL6Qlh4TXtS7u-tuU3pYRE8nakvl9C14bG_I_uoR9E3UEO1uB3GfH35c4iYBf9EA68CBQsnA0iNp6eFYnZqFoNK2Lp813QsnyyHNbWqnsuzH5ZLjJ6tsdm?key=U2w6P9NgFMJ_s__uwahN_Q

Result: Parallelized

Example 2 — Avoid Building Multiple Device Lists

Broken (serial)

Constructing multiple device lists invokes network.devices more than once, which results in the query being executed in serial.

ciscoDevices =

  foreach device in network.devices

  where device.platform.vendor == Vendor.CISCO

  select device;



paloDevices =

  foreach device in network.devices

  where device.platform.vendor == Vendor.PALO_ALTO_NETWORKS

  select device;



foreach device in ciscoDevices + paloDevices

foreach interface in device.interfaces

foreach subinterface in interface.subinterfaces

foreach arp in subinterface.ipv4.neighbors

select {

  device: device.name,

  deviceVersion: device.platform.osVersion,

  interface: interface.name,

  arp: arp.ip,

  mac: arp.linkLayerAddress

}

AD_4nXdfVLhRmupo9CffiHxhR8o67omdPQ4NtjYPLwCi6Sygd5X8OaMI5Uf-DvxPakyD4vqJ58CA5ScWQym2ovCriQdi1eYd0EkXKyKoHIeG9DEn7PKL5feSHmT8uIL4invrvisOGqQ2rQ?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXdfBsYcwV2JS81xqQsw1G_J1oht2QtnEN2i4p208rJbt1_YCeDTtSGHawY6wdFwZb4xLjo1hyHArNeygXPoHgLiVCCLHwJHqzkBqDno-AJdNwLOLRHRzoRRcYOKbB0AgFunq5X1?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)

foreach device in (ciscoDevices + paloDevices)

AD_4nXcDVWJK1yDjomuwiRGAVcXGfqXVrGXCrl0Tn4-Sw5l8WNFkuOTPEBc5NIjYRCpw7ynahpVa9goRACfaReRYizNwJC0AAO0er73eMtKsJhwVDM1mWZfiAcp-DGM-4uyJnhFZlw0Elw?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXf6DU4Zq06ayDtgWXm2TUYkAlHQu5QcPy3FHAgzgt4RxpoTJ7dllB_EzuzZSMNONCyISyEmr-UthlBFTT90GVQs2IsXBnLTibK7qrKUTPu8-1YAf_wdaVgeu30dX_iXZOfv5kMI3A?key=U2w6P9NgFMJ_s__uwahN_Q

(No parallel_foreach — not parallelized.)

Fixed (parallel)

Filter in a single pass over network.devices.

foreach device in network.devices

where device.platform.vendor in eVendor.CISCO, Vendor.PALO_ALTO_NETWORKS]

foreach interface in device.interfaces

foreach subinterface in interface.subinterfaces

foreach arp in subinterface.ipv4.neighbors

select {

  device: device.name,

  deviceVersion: device.platform.osVersion,

  interface: interface.name,

  arp: arp.ip,

  mac: arp.linkLayerAddress

}

AD_4nXc_Dz4Ed6PN0Ht6gR_LC7P2HLa6ScJXoNd9dfPTx5-mrbB4UPJ3s4v7vWU5glmH3xK1KrCzNTotKmfCG5IkQHTcTrlWkeeBLoFV54ZwlWTtGqYbTGXTcPsv-qv0_9bm0DL-PzYP9A?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXe-hg922WCkHaAVMR50ozghLWW8bWwwdU0RUrm_a_ENkUhGSkLz103BPESsITqA9VrelAPdgoDSge20MLkwCdK5NW2zQWeG4HYCekXiyx8fR8V4BqbLzVFP0fRReup_HQ8nwyEaMw?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)

parallel_foreach device in network.devices

AD_4nXfOlmFjQWa9T5dGQgkRabBNhUVpywqCXTB1RUuhR8G63eOS6K8PByAQOma073XFtz-144lhRxa5Z5yWFCpDDnzLCAonlB1VN5wEo-uZS6Rf_q4A4vP-Sxb3LIR0dj_FE5jo__3w5w?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXc59JsrJOuxovQTVvOlRnAcfbDpMYZMyDkAZM9VG1zrmecMHczHQwH2ahfvHrMGAgHC74Ti3Db39mLsyoNiJr0SvEPdrTLyDM_bx4GSF9ONKGiHuCtJQrkuExnytjIDk6aRr2sCjg?key=U2w6P9NgFMJ_s__uwahN_Q

Result: Parallelized

Example 3 — Multiple Checks, One Device Loop

Broken (serial)

Running separate checks that each iterate over network.devices invokes multiple passes and prevents parallelization.

lineVtyPattern = ```

line vty 0 4

  session-timeout

  access-class

```;



vtpModePattern = ```

vtp mode transparent

```;



check1Results =

  foreach device in network.devices

  where device.platform.os == OS.IOS_XE

  let result = blockDiff(device.files.config, vtpModePattern)

  select {

    device: device.name,

    version: device.platform.osVersion,

    check: "VTP Mode Is Transparent",

    violation: result.diffCount > 0,

    diff: result.blocks

  };



check2Results =

  foreach device in network.devices

  where device.platform.os == OS.IOS_XE

  let result = blockDiff(device.files.config, lineVtyPattern)

  select {

    device: device.name,

    version: device.platform.osVersion,

    check: "Line VTY Configuration contains access-class and timeout",

    violation: result.diffCount > 0,

    diff: result.blocks

  };



foreach result in check1Results + check2Results

select result

AD_4nXcCHk9IEQ3GwakozavWFltbOtR5r2YGJ3ftWBAC6vYN8ydBcbjEnCairZrfweun2FaRJ0Gip3V3sexOXe3CRwlJBue65dafri0nyeEQsk2gnOpMNtfCBf7fK4PTCJII7cTEjRTHVg?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXcsI2N4SEdhE3BAY0-UqrqaOr1ok_GH_uRvX-Q1neLiZ_BR7kuK8eQ6rahE1YR9D_J6TQecmd6YZpeHAr0VVqkFWECMTDOr98WfLJSVDvInt8tF4DKpzN1xbLPfwyA5FzJlcHpW?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)  

foreach device in network.devices  // in check1Results  

foreach device in network.devices  // in check2Results  

AD_4nXeGt22SpyykFyW770OVW7HykEsIEDEZxb4nsoXxlCUbrC2Yb0jw4ho3zc24KqCfBIaA0T2SRKf0AxRzViMHWKTt_DM-YaSpGh6RPP3yARJAINIPyPDgb6FhcKUlqiRTySOKM-PAGg?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXe2lfY7jN4Dd04aM-XoxitxSeEoe-Rri5FHdAHB3Y8HIsi-P0TK957LKT_y7x2P1I_7BbBH6Sa4Skhy_MA_LXjSq1L51QwWrOFxeTzGUN4IfFilmDhPfNKIbVSBLVuhDHt2jBceqQ?key=U2w6P9NgFMJ_s__uwahN_Q

Result: Not parallelized

Fixed (parallel)  


Keep “one function per check,” but pass the device into each check so there’s only one iteration over network.devices.

lineVtyPattern = ```

line vty 0 4

  session-timeout

  access-class

```;



vtpModePattern = ```

vtp mode transparent

```;



check1Results(device) =

  foreach device in bdevice]

  where device.platform.os == OS.IOS_XE

  let result = blockDiff(device.files.config, vtpModePattern)

  select {

    device: device.name,

    version: device.platform.osVersion,

    check: "VTP Mode Is Transparent",

    violation: result.diffCount > 0,

    diff: result.blocks

  };



check2Results(device) =

  foreach device in tdevice]

  where device.platform.os == OS.IOS_XE

  let result = blockDiff(device.files.config, lineVtyPattern)

  select {

    device: device.name,

    version: device.platform.osVersion,

    check: "Line VTY Configuration contains access-class and timeout",

    violation: result.diffCount > 0,

    diff: result.blocks

  };



foreach device in network.devices

foreach result in check1Results(device) + check2Results(device)

select result

AD_4nXd2SqbUSPx05jQp_kNArCgjAnpNPVY4HeuNB5dhgOcTvGIctz5eXsX3O_5x0bl1doN5PBufM1b61t0qB839ft67Cl117RNn_9cf7W0_OHov-d6qBujJIMyg9kQFpFvbNKWxQicW?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXfeUhi2_XCMQHLkMMXYHuLGihNsS2g6TofEsS1AK9Z6BhyJi8jtQ1S2ECNoftOxF37N6jsl3z6M3iaDF5PlTgpJ_IjhHtVHZDbh2FgU9-ABP_wwCYkPSxy9To3Orzj4dPwjcrxtkQ?key=U2w6P9NgFMJ_s__uwahN_Q

Debug (abridged)  

parallel_foreach device in network.devices  

AD_4nXc-5ESMOrpNd3NqMmfvx-xnXAIURYe_i3xCc2l9LRcBr5ZuVBusqc8T0Ys8qDOoS4_lMZKvv2hq9tPbjogMs5aycfsBflqwlefcnlggvQ5B4ygT_L5cHpsOqaKmJ8tMpfttDwKzqg?key=U2w6P9NgFMJ_s__uwahN_QAD_4nXe28aVE7lHXsbwKSAchClm0iiuwjgULOpRjgC_vzlXASjk901HiOdKTzS-SurUwqTvTfoTBrBXJQWl3zEBrcrchHW7YWCqDOl0rVlDEFZvykdeZ6T-YPsT-rTRSj4tmX3zHxgeakQ?key=U2w6P9NgFMJ_s__uwahN_Q

Result: Parallelized

Takeaways  

  • Golden rule: iterate over network.devices once
  • Prefer using the data model when possible over manually parsing information
  • Pass the device object into helper functions; don’t re-look it up by name.  
  • Combine filters with where instead of building multiple device lists.  
  • Use the debug panel and confirm you see parallel_foreach device in network.devices.  

Final Notes

There are cases where writing an NQE to execute in serial is unavoidable, such as in the Forward Library examples Uncollected CDP-LLDP Neighbors or IP Address Uniqueness. NQE fully supports serial queries and can often process very complex logic across large datasets in serial without issue. The takeaway should not be that a serial query is inherently bad practice. Rather, advanced NQE authors should understand how parallelization works, know how to leverage it when possible, and reserve serial execution for situations where it is the only viable way to solve the problem.

 

Be the first to reply!

Reply