Custom source profile with Endpoint that is using “url/path?_return_type=json-pretty” but responseBody is one long line.
I attempt to use splitJsonObjects; however, it isn’t returning in a structured layout for me to pattern match.
Solved
Parse externalSource HttpEndPoint API ResponseBody
- Spotter
- 12 replies
Best answer by Andreas
Hi
splitJsonObjects will split the text and return a collection of chunks of the text, each of which contains a JSON object. You can then use extractJson on each chunk to get the data into NQE in a structured form. For example, it might look like something like this:
...
foreach block in splitJsonObjects(ep.responseBody)
let item = extractJson[{kind: String}](block)
select item
Note that to use extractJson you have to give it the expected structure of the data you are trying to extract. For example, in the snippet above, I set the expected type as {kind: String}, which denotes a record (aka object) that has a field (aka property) called “kind” and the data in that field is a String. You can add more fields and more specific types (Number, IpAddress, List<String>, etc.) as you need based on your data. You can find more info on the types NQE supports here: https://docs.fwd.app/latest/application/reference/nqe/guides/nqe-types/.
For reference, docs on splitJsonObjects and extractJson are https://docs.fwd.app/latest/application/reference/nqe/std-lib/splitJsonObjects/ and https://docs.fwd.app/latest/application/reference/nqe/std-lib/extractJson/.
+2The “splitJsonObjects” is just going to split the output, probably with spaces. It won’t structure the data. There is an another function called “extractJson”. But normally actual json output is parsed on its own.
Is it possible you can share the output. Or a sanitized version of the output to test with?
- Spotter
- 12 replies
Unfortunately there isn’t a lot I can provide but it is just this long string continues on to the end of the file. I’ll try extractJson.
Hi
splitJsonObjects will split the text and return a collection of chunks of the text, each of which contains a JSON object. You can then use extractJson on each chunk to get the data into NQE in a structured form. For example, it might look like something like this:
...
foreach block in splitJsonObjects(ep.responseBody)
let item = extractJson[{kind: String}](block)
select item
Note that to use extractJson you have to give it the expected structure of the data you are trying to extract. For example, in the snippet above, I set the expected type as {kind: String}, which denotes a record (aka object) that has a field (aka property) called “kind” and the data in that field is a String. You can add more fields and more specific types (Number, IpAddress, List<String>, etc.) as you need based on your data. You can find more info on the types NQE supports here: https://docs.fwd.app/latest/application/reference/nqe/guides/nqe-types/.
For reference, docs on splitJsonObjects and extractJson are https://docs.fwd.app/latest/application/reference/nqe/std-lib/splitJsonObjects/ and https://docs.fwd.app/latest/application/reference/nqe/std-lib/extractJson/.
Sign up
Already have an account? Login
Welcome to the Forward Networks Community
Select a login option:
Register / Login Forward Employee LoginEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.