Sometimes you need to parse out information that is not in the NQE data model for various reasons. It maybe device specific information that is not generalizable across vendors, or not generally useful in dataplane analysis but maybe important for operations. Importing this data is beyond the scope of this post but you can read more about it here: Getting Custom Commands
First, I am going to demonstrate how you build up your parsing chops by leveraging a testing approach to NQE, then will demonstrate how this can be applied to your collection data.
- Setting up your testing.
We need to represent the data that our custom command will import into the snapshot. We can represent this by setting a variable to a multiline string using the delimiter ”””
output =
"""
Item Version Part number Serial number FRU model number
Midplane REV 03 710-013698 TR0001 CHAS-BP-MX960-S
FPM Board REV 03 710-014974 JZ2323 CRAFT-MX960-S
PEM 0 Rev 07 740-029344 QZY2122ABCD PWR-MX960-4100-DC-S
PEM 1 Rev 07 740-029344 QZY2123ABCE PWR-MX960-4100-DC-S
PEM 2 Rev 07 740-029344 QZY2124ABCF PWR-MX960-4100-DC-S
PEM 3 Rev 07 740-029344 QZY2125ABCG PWR-MX960-4100-DC-S
Routing Engine 0 REV 20 750-054758 ZAUY1234 RE-S-X6-64G-S
Routing Engine 1 REV 20 750-054758 ZAUY1235 RE-S-X6-64G-S
CB 0 REV 12 750-062572 XYZE1234 SCBE2-MX-S
CB 1 REV 12 750-062572 XYZE1235 SCBE2-MX-S
CB 2 REV 12 750-062572 XYZE1236 SCBE2-MX-S
FPC 0 REV 57 750-053323 ZYGB2222 MPC7E-10G
FPC 1 REV 46 750-056519 ZYGB2223 MPC7E-MRATE
FPC 3 REV 01 750-136058 ZYGB2223 MPC7E-10G
FPC 7 REV 26 750-028467 BVCC1234 MPC-3D-16XGE-SFPP
FPC 8 REV 46 750-056519 CABC2912 MPC7E-MRATE
Fan Tray 0 REV 08 740-031521 ACCA1234 FFANTRAY-MX960-HC-S
Fan Tray 1 REV 08 740-031521 ACCA1235 FFANTRAY-MX960-HC-S
""";
Next we define some helper functions:
fixRev(s) = replaceMatches(s, "REV 20", " REV 20");
fixBlocks(blocks) =
foreach line in blocks
select if matches(line.text, "Routing Engine*")
then replaceMatches(fixRev(line.text), "\\b\\s\\b", "-")
else replaceMatches(line.text, "\\b\\s\\b", "-");
So you may ask yourself why do we need these helper functions. Well the reason is parsing data can be very complicated and NQE tries to make this as simple as possible but it can’t account for every possible scenario. For instance in our output you may notice that we have a table of 5 columns labeled Item, Version, Part number, Serial number and FRU model number
The output for these columns vary from simple strings e.g. Midplane to a list of strings PEM 0 or Routing Engine 0 . In order for NQE to group these appropriately we need to manipulate the strings or marshal them into a more structured representation.
Let’s break down what fixBlocks is doing. It takes in a variable called blocks, which is of type List<ConfigLine> and iterates over the collection with a comprehension. When we encounter a line matching the glob “Routing Engine*”, we use the replaceMatches function to first give us a little room between the 0 and 1 and the next word block so we can use our word boundary regex then we convert the spaces between Routing Engine <number> to Routing-Engine-<number>. You may notice that we don’t have to do this for Fan Tray 1 because there is sufficient space between the <number> and the next column.
The rest of the query should be self explanatory if you are familiar with NQE pattern matching and data extraction.
pattern =
`{!"Item"} {item:string} {version:string} {partNo:string} {serialNum:string} {FRUModel:string}`;
foreach x in b0]
let blocks = parseConfigBlocks(OS.UNKNOWN, output)
let newBlocks = fixBlocks(blocks)
foreach block in newBlocks
let matches = patternMatch(block, pattern)
where matches?.item != null : String
select {
item: matches?.item,
version: matches?.version,
partNo: matches?.partNo,
serialNo: matches?.serialNum,
FRUMpdel: matches?.FRUModel
}
Here is the full query
/**
* @intent Provide structure report on inventory
* @description This query is a test to parse out the outputs of JunOS "show chassis hardware models"
*/
output =
"""
Item Version Part number Serial number FRU model number
Midplane REV 03 710-013698 TR0001 CHAS-BP-MX960-S
FPM Board REV 03 710-014974 JZ2323 CRAFT-MX960-S
PEM 0 Rev 07 740-029344 QZY2122ABCD PWR-MX960-4100-DC-S
PEM 1 Rev 07 740-029344 QZY2123ABCE PWR-MX960-4100-DC-S
PEM 2 Rev 07 740-029344 QZY2124ABCF PWR-MX960-4100-DC-S
PEM 3 Rev 07 740-029344 QZY2125ABCG PWR-MX960-4100-DC-S
Routing Engine 0 REV 20 750-054758 ZAUY1234 RE-S-X6-64G-S
Routing Engine 1 REV 20 750-054758 ZAUY1235 RE-S-X6-64G-S
CB 0 REV 12 750-062572 XYZE1234 SCBE2-MX-S
CB 1 REV 12 750-062572 XYZE1235 SCBE2-MX-S
CB 2 REV 12 750-062572 XYZE1236 SCBE2-MX-S
FPC 0 REV 57 750-053323 ZYGB2222 MPC7E-10G
FPC 1 REV 46 750-056519 ZYGB2223 MPC7E-MRATE
FPC 3 REV 01 750-136058 ZYGB2223 MPC7E-10G
FPC 7 REV 26 750-028467 BVCC1234 MPC-3D-16XGE-SFPP
FPC 8 REV 46 750-056519 CABC2912 MPC7E-MRATE
Fan Tray 0 REV 08 740-031521 ACCA1234 FFANTRAY-MX960-HC-S
Fan Tray 1 REV 08 740-031521 ACCA1235 FFANTRAY-MX960-HC-S
""";
fixRev(s) = replaceMatches(s, "REV 20", " REV 20");
fixBlocks(blocks) =
foreach line in blocks
select if matches(line.text, "Routing Engine*")
then replaceMatches(fixRev(line.text), "\\b\\s\\b", "-")
else replaceMatches(line.text, "\\b\\s\\b", "-");
pattern =
`{!"Item"} {item:string} {version:string} {partNo:string} {serialNum:string} {FRUModel:string}`;
foreach x in ;0]
let blocks = parseConfigBlocks(OS.UNKNOWN, output)
let newBlocks = fixBlocks(blocks)
foreach block in newBlocks
let matches = patternMatch(block, pattern)
where matches?.item != null : String
select {
item: matches?.item,
version: matches?.version,
partNo: matches?.partNo,
serialNo: matches?.serialNum,
FRUModel: matches?.FRUModel
}
Now that we have tested our parsing we can apply this to the snapshot with the following modifications.
- Running against your snapshots
/**
* @intent Provide structure report on JunOS "show chassis hardware models"
* @description Report for JunOS "show chassis hardware models"
*/
command_eval = "show chassis hardware models";
fixRev(s) = replaceMatches(s, "REV 20", " REV 20");
fixBlocks(blocks) =
foreach line in blocks
select if matches(line.text, "Routing Engine*")
then replaceMatches(fixRev(line.text), "\\b\\s\\b", "-")
else replaceMatches(line.text, "\\b\\s\\b", "-");
pattern =
`{!"Item"} {item:string} {version:string} {partNo:string} {serialNum:string} {FRUModel:string}`;
foreach device in network.devices
where device.platform.os == OS.JUNOS
foreach output in device.outputs.commands
where output.commandText == command_eval
let blocks = parseConfigBlocks(OS.UNKNOWN, output.response)
let newBlocks = fixBlocks(blocks)
foreach block in newBlocks
let matches = patternMatch(block, pattern)
where matches?.item != null : String
select {
item: matches?.item,
version: matches?.version,
partNo: matches?.partNo,
serialNo: matches?.serialNum,
FRUModel: matches?.FRUModel
}
In Part 2 we will look at an even more complex example. Thanks for hanging in..
