Introduction
Hello Community! I wanted to share something I’ve been working on that may be helpful to you. It's a query designed to manipulate list arrays by transforming list elements into fixed-length strings, which then lets us easily join, split, or pop elements. NQE built-in functions provide list iteration and for the most part, this satisfies data model iteration for our queries. The approach presented here extends NQE list operation abilities to manipulate lists or extract specific values when required.
To accomplish this we can apply a well known process called fixed-length string encoding which involves standardizing the length of each item in a list. By transforming variable-length strings into uniform sizes, we can “index” each item making if possible to perform operations like joining, splitting, and extracting elements. It opens up new possibilities for customizing functions beyond the basics of list iteration and hopefully, it will serve as a helpful tool when you need to work with more advanced queries or custom data operations.
I have included 2 examples of list operations in the query that pop all list items or extract a data element at a specific index. This provides a foundation for additional list operations and there are many more I could have included but I didn’t want to spoil the fun for any of you that are motivated.
Here are some additional ideas:
- Given an item in the list, find its index.
- Slice a list at a given index
- Create a function to multiply NxN matrices
- Make query modifications to allow import of a list operations into other queries.
- Sort and Reverse the list
In addition, there is at least one more method available to us in NQE to enumerate list items. Leave your guesses in the comment section if you like.
Looking forward to any feedback or ideas you might have! Thanks so much!
What it does
The query takes a list datatype (listArray) and creates a fixed-length array where all elements have the same length. This array is concatenated into a single string. In addition, it provides ways to split this string back into individual elements and perform pop operations on the string elements.
Query walk-through.
1. Variables
- listArray: Sample data elements
="r", "pi", "3.14", "golden", "ratio", "1.618", "Euler", "2.71828", "i^2", "-1"]
- delChar: Character used for string manipulation, which is used for padding and delimiting data elements "#"
2. Functions
getWordLength(word)
This function calculates the length of a string (word) and is used as a maxBy filter
getWordLength(word) = foreach x in <0] select length(word);
padLen(listObj)
This function determines the padding length for the elements in the input list (listObj):
- It finds the element with the maximum length using the maxBy function.
- Converts the element to a string and calculates its length (padLength).
- It loops from 1 to padLength and generates a padded string by replacing each element with the delimiter delChar.
createFixedLenStringArray(listArray, maxPad)
This function creates a list of strings from listArray, where each element is adjusted to a fixed length (maxPad):
- It converts each element of the listArray to a string.
- It calculates the necessary padding to make each element a fixed length by using substring.
- Returns the newly generated fixed-length string array.
Sample List Operations
StringToListArray(stringArray, delimiter)
Pops all values: This function splits a string into a list of substrings based on a given delimiter:
- It iterates through the stringArray, dividing it by the delimiter.
- Extracts each substring using substring(stringArray, index0, index1).
- Removes any characters defined by delChar.
pop(stringArray, delimiter, index)
This function simulates popping an element from a list (string) at a specified index:
- It calculates the position of the element based on the delimiter and an extraction window
- Extracts the substring corresponding to the element's index window and returns it without any padding characters.
3. Main Block
- It creates a fixed-length string array (stringArray) by applying createFixedLenStringArray.
- Defines the delimiter and pad based on the length of the padding.
4. Select block
- OriginalList: The original listArray.
- ListArrayDim: The length of listArray.
- FixedLengthElement: The length of the fixed-length elements
- StringArrayLength: The length of the joined stringArray.
- PadMax: The maximum padding calculated from padLen.
- StringArray: The fully joined stringArray.
- PopAllElements: Converts the arrayString back to a list using StringToListArray.
- Pop: Pops the last element from the arrayString.
- PopIndex0: Pops the first element from the arrayString.
//Attributes//
listArray = p"r" , "pi", "3.14" , "golden" , "ratio" , "1.618" , "Euler" , "2.71828" , "i^2" , "-1"];
delChar = "#";
//Functions//
//MaxBy Filter1
getWordLength(word) =
foreach x in >0]
select length(word);
//Calculate pad length-return pad string
padLen(listObj)=
foreach e in >0]
let maxElement = maxBy(listObj, getWordLength)
let maxElemString = toString(maxElement)
let padLength = length(maxElemString)
foreach p in fromTo(1,padLength-1)
let padCharList = replaceMatches(toString(p) , "*" , delChar)
let padCharList = prefix(padCharList,1)
select padCharList;
//Create fixed length string from array
createFixedLenStringArray(listArray , maxPad) =
foreach e in listArray
let eString = toString(e)
let fixedLenElement = substring(maxPad,0,(length(maxPad)-length(eString)+1)) + eString
select (fixedLenElement);
//String Array Operations//
//Transform Fixed Delimited String to ListArray
StringToListArray(stringArray, delimiter)=
foreach i in fromTo(0,(length(stringArray)/delimiter)-1)
let index0 = i * delimiter
let index1 = index0 + delimiter
let stringElement = substring(stringArray,index0,index1)
select replace(stringElement , delChar , "");
//Pop Element from Index
pop(stringArray, delimiter , index) =
foreach i in l0]
let delimiter = delimiter+1
let index0 = index * (delimiter)
let index1 = index0 + delimiter
let stringElement = substring(stringArray,index0,index1)
select replace(stringElement , delChar , "");
//Get Index of Element
//Sort List
//Main//
foreach x in /0]
let stringArray = createFixedLenStringArray(listArray , join("" , padLen(listArray)))
let delimiter = length(padLen(listArray))
let arrayString = join("" , stringArray)
select {
OriginalList:listArray,
ListArrayDim:length(listArray),
FixedLengthElement:length(padLen(listArray)) + 1,
StringArrayLength:length(join("" , stringArray)),
PadMax:join("" , padLen(listArray)),
StringArray:arrayString ,
//List Operations
PopAllElements: StringToListArray(arrayString, delimiter+1),
Pop: pop(join("" , stringArray) , delimiter , length(listArray)-1),
PopIndex0: pop(arrayString , delimiter , 0)
}
