Recently I needed to query something from a json document, but I just couldn't f...

naniwaduni · on Oct 2, 2023

    % jq -c '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    ["ams01","zrh02"]

2m28s from time I downloaded hosts.json to producing this solution, though I must admit I had to take a moment to look up contains to realized it had to take an array—I could've used index/1 and had a better time.

    % jq -c '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json
    ["ams01","zrh02"]

This one was much quicker, just 1m5s, but of course after I'd already done the previous one.

It's a language. You can learn it, and then it can be easier to write programs in it. Or you can blame the tool, sure.

edit: and for the curious, this is the path I took to produce these programs:

    wget https://gist.github.com/ruuda/31cffffaa595492a335a4637503f4b7a/raw/7ae5bef0ac331eb2fb7d67cac6cc867e13a675f3/hosts.json
    jq . hosts.json
    jq .tags hosts.json
    jq '.tags|path(.[]|select(contains("tag:primary")))' hosts.json
    jq '.tags|path(.[]|select(contains(["tag:primary"])))' hosts.json
    jq '.tags|path(.[]|select(contains(["tag:primary"])))[-1]' hosts.json
    jq 'path(.tags[]|select(contains(["tag:primary"])))[-1]' hosts.json
    jq '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(contains(["tag:primary"])))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(index("tag:primary")|debug))[-1]]' hosts.json
    jq -c '[path(.tags[]|select(index("tag:primary")))[-1]]' hosts.json
    wget https://gist.github.com/ruuda/31cffffaa595492a335a4637503f4b7a/raw/7ae5bef0ac331eb2fb7d67cac6cc867e13a675f3/hosts_advanced.json
    jq . hosts_advanced.json
    jq '.hosts' hosts_advanced.json
    jq '.hosts[]' hosts_advanced.json
    jq '.hosts[]|select(.tags|index("tag:primary"))' hosts_advanced.json
    jq '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json
    jq -c '[.hosts[]|select(.tags|index("tag:primary")).name]' hosts_advanced.json

mcint · on Oct 2, 2023

I didn't time myself, but took 25-50% more queries.

I'll echo the value of ijq, https://sr.ht/~gpanders/ijq/ The live updating makes for less to remember and visualize. Though there is some value in having intermediate history recorded, for the second and later questions you ask of the data. (And supports commenting out for error diagnosis without deleting half the expression, `# ...`, hash and space required.)

I hadn't known of `debug`, quite convenient. I've often used multiple outputs at the tail, for example `... | keys, .` to remind myself of the structure as I explore down. Also `gron` and `fzf --multi`.

llwj · on Oct 2, 2023

Even GPT-3.5 can solve that on the first try. I didn't measure time but it should be within a minute. https://chat.openai.com/share/7f6a97de-9aef-49db-946b-15bfd9...

ruuda · on Oct 2, 2023

I tried GPT-3.5, it got it wrong but it did get the overall setup right, just forgot about a detail. But I don't want to reach for ChatGPT every time I need to query something, I want something that I can write without having to solve a puzzle first. (True, maybe I should spend some time learning jq better.)

dan-robertson · on Oct 2, 2023

Isn’t it:

  .tags | to_entries | map(select(.value[] == "tag:primary") | .key))

This took me a little while as I wasn’t at a computer so I couldn’t build the query interactively.

I think the big advantage jq has is that ChatGPT can generate filters from English descriptions.