Skip to main content

I had to change a lot, but this keeps the intent.  This is from a PANOS firewall.  The problem is that I am trying to find the objects in this list.  However, some objects have double quotes to show there are spaces in the name of the object and some do not.  I’m not sure how to clean this since NQE natively does not use double quotes to designate a single object that happens to contain spaces.

 

sampleInput = """
policy {
shared;
panorama {
address-group {
THE-GRAPE-Subnets {
static s "Green Grapes Are Good" RedGrapes BlackGrapes "Wine Grapes" PurpleGrapes];
tag GRAPES;
description "This is a list of Grapes";
}
""";

pattern01 = ```
policy
panorama
address-group
THE-GRAPE-Subnets
static "t {object: (string*)}
```;

foreach x in h1]
let config = parseConfigBlocks(OS.PAN_OS, sampleInput)
foreach match in blockMatches(config, pattern01)
select {
objects: match.data,
block: match.blocks
}

Then this is the result from the “objects” column.

{object:e"Green, Grapes, Are, Good", RedGrapes, BlackGrapes, "Wine, Grapes", PurpleGrapes]"]}

 

How can we get this to be a list of only 5 objects like it was in the sampleInput?

I have an idea that involves replacing each double quote with a new line and the parsing the lines from the resulting string.  I haven’t fully worked through the idea though.


@Tyson Williams Have you had an opportunity to look at this.


This quarter, we plan to add regex support to NQE.  Then it should be trivial to solve this problem with a good (i.e. non-hacky) solution.

Do you want to wait for regex support or is this urgent enough that you want a hacky solution?


I guess waiting for the Regex support is fine.

 

Thank you.


I think i solved it…. 

sampleInput =
"""
policy {
shared;
panorama {
address-group {
THE-GRAPE-Subnets {
static i "Green Grapes Are Good" RedGrapes BlackGrapes "Wine Grapes" PurpleGrapes];
tag GRAPES;
description "This is a list of Grapes";
}
""";

pattern01 =
```
policy
panorama
address-group
THE-GRAPE-Subnets
static "c {object:(string*)}
```;

stringdelimeter = `{string*}`;

getlistitems(text) =
max(foreach x in i1]
let listitem = patternMatch(text, stringdelimeter)
select listitem);

getlists =
foreach x in i1]
let config = parseConfigBlocks(OS.PAN_OS, sampleInput)
foreach match in blockMatches(config, pattern01)
// clean up the match...
let joined = replace(replace(join(" ", match.data.object), "]", ""), " ", "_")
// replace quotes with a carriage return, and set first character of line to a minus
let items = replaceMatches(joined, "\"", "\n-")
// replace all spaces with underscore
let items = replace(items, "_", " ")
// remove any leading spaces and minus signs for those lines that are text, lines that are not text, will have leading minus
let items = replace(items, "- ", "")
// now a list of lines, so process as if a config
let config = parseConfigBlocks(OS.UNKNOWN, items)
// process each line
foreach line in config
// ignore those with only the minus sign
where line.text != "-"
// set the linetype for the given line
let linetype = if prefix(line.text, 1) == "-" then "text" else "list"
// for lines that start with a minus, then just remove the minus and place in a list
// for lines that don't start with a minus, then we need to grab the list items
let listitems = if linetype == "text"
then esuffix(line.text, length(line.text) - 1)]
else getlistitems(line.text)
select listitems;

// main function
foreach x in i1]
// get all the list of lists
let listoflists = getlists()
// process each list
foreach list in listoflists
// we need to pick each listitem
foreach listitem in list
select { listitem }

It may be a bit contrived in how i did this.

The technique was to clean up the text first, split into multiple lines and encode what the line was, either a single item or multiple.

Results below….

 

 


To close the loop on this.  Andi had a method that worked using existing NQE methods.  However, in 24.10, regular expressions have been added.  Here is a version of that query using the regexMatches.

You can see the “Guide” to using regular Expressions here:

https://fwd.app/docs/nqe/guides/regexes/

sampleInput = """
policy {
shared;
panorama {
address-group {
THE-GRAPE-Subnets {
static s "Green Grapes Are Good" RedGrapes BlackGrapes "Wine Grapes" PurpleGrapes];
tag GRAPES;
description "This is a list of Grapes";
}
""";

foreach payload in regexMatches(sampleInput,
re`\s*static\s+\W+\s+(?<objects>c^\]]*)\W+`)
foreach thingy in regexMatches(payload.data.objects,
re`"(?<object>.+?)"|(?<object>\w+)`)
let stuff = thingy.data.object
select {stuff}

 

The first regexMatches is to pull out everything between the square brackets. Plus remove those square brackets.

static > ];

That means payload becomes 

"Green Grapes Are Good" RedGrapes BlackGrapes "Wine Grapes" PurpleGrapes

 

Now there are two types of objects: one with “” and the other without.

re`"(?<object>.+?)"|(?<object>\w+)`)

This is 

"(?<object>.+?)" = capture everything between double quotes. I’ll explain the .+? some more below.

| = this is a logical OR

(?<object>\w+) = capture all word characters, but don’t include spaces.

 

The .+? is interesting.  The ? says to “match the shortest possible string”

This means that if you just use the regex .+ you would get

Green Grapes Are Good" RedGrapes BlackGrapes "Wine Grapes

Because that is the longest string between a pair of double quotes.

There are two short strings between double quotes

Green Grapes Are Good
Wine Grapes

That is why the ? is needed in the regex .+?.


Excellent explanation - thanks Tyson


Reply