Difference between revisions of "Jq"
From Christoph's Personal Wiki
(→Example usage) |
(→External links) |
||
Line 180: | Line 180: | ||
==External links== | ==External links== | ||
*[https://stedolan.github.io/jq/ Official website] | *[https://stedolan.github.io/jq/ Official website] | ||
+ | *[https://starkandwayne.com/blog/bash-for-loop-over-json-array-using-jq/ Bash For Look Over JSON Array Using Jq] | ||
[[Category:Linux Command Line Tools]] | [[Category:Linux Command Line Tools]] |
Revision as of 00:50, 9 July 2020
jq is a lightweight and flexible command-line JSON processor. jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep, and friends let you play with text.
Example usage
$ cat azones.json
{ "availabilityZoneInfo": [ { "hosts": { "node-1.example.com": { "nova-compute": { "active": true, "available": true } }, "node-2.example.com": { "nova-compute": { "active": true, "available": true } } }, "zoneName": "az1", "zoneState": { "available": true } }, { "hosts": { "node-3.example.com": { "nova-compute": { "active": true, "available": true } }, "node-4.example.com": { "nova-compute": { "active": true, "available": true } } }, "zoneName": "az2", "zoneState": { "available": true } } ] }
- Capture just the availability zone names:
$ cat azones.json | jq '[.availabilityZoneInfo[] | .zoneName]'
[ "az1", "az2" ]
Or, for compact instead of pretty-printed output:
$ cat azones3.json | jq -c '[.availabilityZoneInfo[] | .zoneName]' ["az1","az2"]
- Capture just the hostname (e.g., "
node-1.example.com
") key for availability zone "az1":
$ cat azones.json | jq '[.availabilityZoneInfo[] | select(.zoneName == "az1") | {hosts: .hosts|keys}]'
[ { "hosts": [ "node-1.example.com", "node-2.example.com" ] } ]
Or, for a more script-friendly output:
$ cat azones.json | jq -cM '[.availabilityZoneInfo[] | select(.zoneName == "az1") | {hosts: .hosts|keys}]' | sed -e 's/["}\[]//g;s/\]//g;s/{hosts://g;s/,/ /g' #~OR~ $ foo=($(cat azones3.json | jq -cM '[.availabilityZoneInfo[] | select(.zoneName == "az1") | {hosts: .hosts|keys}]' | sed -e 's/["}\[]//g;s/\]//g;s/{hosts://g;s/,/ /g')) $ echo ${foo[0]} #=> node-1.example.com
- Get just the raw values:
$ echo '{ "packet_loss": [ {"ips": "10.0.0.10 10.0.0.11 10.0.0.12", "node-17": "3/3" }] }' | jq -r '[.packet_loss[] | .ips] | .[]' 10.0.0.10 10.0.0.11 10.0.0.12
Practical example
Here is how to print out all the OpenStack compute nodes in my example environment:
#!/bin/bash # AUTHOR: Christoph Champ <christoph.champ@gmail.com> # Requires jq 1.5+ JQ=$(which jq) OS_AUTH_URL=http://1.2.3.4:5000/v2.0/ OS_TENANT_NAME=admin OS_USERNAME=admin OS_PASSWORD=admin INFO=$(curl -sXPOST "${OS_AUTH_URL}/tokens" \ -H "Content-Type: application/json" \ -d "{\"auth\":{\"tenantName\":\"$OS_TENANT_NAME\",\"passwordCredentials\":\ {\"username\":\"$OS_USERNAME\",\"password\":\"$OS_PASSWORD\"}}}" | \ ${JQ} -crM '[.access.token.id + "," + (.access.serviceCatalog[] | select(.name == "nova") | .endpoints[].publicURL)] | .[]') TOKEN=${INFO%%,*} NOVA_ENDPOINT=${INFO#*,} IGNORE_ZONES="internal|nova" raw=$(curl -s -H "X-Auth-Token: ${TOKEN}" "${NOVA_ENDPOINT}/os-availability-zone/detail" | \ ${JQ} -crM '[.availabilityZoneInfo[].zoneName] | .[]' | \ grep -vE "(${IGNORE_ZONES})" | tr '\n' ',') IFS=',' read -r -a zones <<< "${raw%,}" for zone in "${zones[@]}"; do raw=($(curl -s -H "X-Auth-Token: ${TOKEN}" "${NOVA_ENDPOINT}/os-availability-zone/detail" | \ ${JQ} --arg zone "$zone" '[.availabilityZoneInfo[] | select(.zoneName==$zone) | .hosts|keys] | .[]' | \ tr -d '[]",' | sed '/^$/d' | tr '\n' ',' | tr -d ' ')) IFS=',' read -r -a nodes <<< "${raw%,}" for node in "${nodes[@]}"; do echo "node: $zone $node" done done
Running the above script produces the following output:
node: az1 node-1.example.com node: az1 node-2.example.com node: az2 node-3.example.com node: az2 node-4.example.com
Append to JSON
- Example of how to append key/values to an already existing JSON structure:
$ cat foo.json { "name": "bob", "age": 30 } $ cat foo.json | jq 'to_entries' [ { "key": "name", "value": "bob" }, { "key": "age", "value": 30 } ] $ cat foo.json | BEARERTOKEN="Bearer abc123" jq 'to_entries | . + [{"key":"routes","value":[{"path":"api/v1","url":"http://example.com","headers":[{"name":"Authorization","content":env.BEARERTOKEN}]}]}] | from_entries' { "name": "bob", "age": 30, "routes": [ { "path": "api/v1", "url": "http://example.com", "headers": [ { "name": "Authorization", "content": "Bearer abc123" } ] } ] }