Skip to main content
Solved

Parse externalSource HttpEndPoint API ResponseBody

  • 30 August 2024
  • 4 replies
  • 73 views

Custom source profile with Endpoint that is using “url/path?_return_type=json-pretty” but responseBody is one long line.

I attempt to use splitJsonObjects; however, it isn’t returning in a structured layout for me to pattern match.  

The “splitJsonObjects” is just going to split the output, probably with spaces.  It won’t structure the data.  There is an another function called “extractJson”. But normally actual json output is parsed on its own.

Is it possible you can share the output. Or a sanitized version of the output to test with?


Unfortunately there isn’t a lot I can provide but it is just this long string continues on to the end of the file. I’ll try extractJson.


Hi @dakotaglory ,

splitJsonObjects will split the text and return a collection of chunks of the text, each of which contains a JSON object. You can then use extractJson on each chunk to get the data into NQE in a structured form. For example, it might look like  something like this:

...
foreach block in splitJsonObjects(ep.responseBody)
let item = extractJsons{kind: String}](block)
select item

Note that to use extractJson you have to give it the expected structure of the data you are trying to extract. For example, in the snippet above, I set the expected type as {kind: String}, which denotes a record (aka object) that has a field (aka property) called “kind” and the data in that field is a String. You can add more fields and more specific types (Number, IpAddress, List<String>, etc.) as you need based on your data. You can find more info on the types NQE supports here: https://fwd.app/docs/nqe/guides/nqe-types/.

For reference, docs on splitJsonObjects and extractJson are https://fwd.app/docs/nqe/std-lib/splitJsonObjects/ and https://fwd.app/docs/nqe/std-lib/extractJson/.


Thank you! Greatly appreciated.


Reply