Working with JSON using `jq`

Summary: Parse and format JSON data in shell.

JSON (JavaScript Object Notation) has become the de facto standard for data interchange in web applications and APIs. While tools like Python and JavaScript provide robust libraries for handling JSON, sometimes you just need to parse or format JSON quickly in the terminal. This is where jq shines—a lightweight, command-line JSON processor.

This article guides you through the essential jq operations for parsing and formatting JSON data in your shell workflow.

What is `jq`?

jq is a command-line tool that allows you to slice, filter, map, and transform structured JSON data. Think of it as sed or awk for JSON. It's available on Linux, macOS, and Windows.

Installing `jq`

Installation is straightforward.

On Ubuntu/Debian:
```
sudo apt-get install jq
```
On macOS (with Homebrew):
```
brew install jq
```
On Windows: Use Chocolatey or download a binary from the official site.

Check your installation with:

jq --version

Basic Usage

Assume you have a file called data.json containing:

[
  {
    "id": 1,
    "name": "Alice",
    "email": "alice@example.com"
  },
  {
    "id": 2,
    "name": "Bob",
    "email": "bob@example.com"
  }
]

Pretty-printing JSON

The most basic use for jq is pretty-printing:

jq . data.json

This prettifies and colorizes the output, making it easier to read.

Extracting Values

Suppose you want the names of all users:

jq '.[].name' data.json
# Output:
# "Alice"
# "Bob"

Filtering Objects

To filter and retrieve the object where the name is "Bob":

jq '.[] | select(.name == "Bob")' data.json

Retrieving Specific Fields

To create a list of emails:

jq '.[].email' data.json

Or, as an array:

jq '[.[].email]' data.json
# Output:
# [
#   "alice@example.com",
#   "bob@example.com"
# ]

Advanced jq Operations

Combining Fields

Extract name and email as a simplified object:

jq '.[] | {name, email}' data.json
# Output:
# {
#   "name": "Alice",
#   "email": "alice@example.com"
# }
# {
#   "name": "Bob",
#   "email": "bob@example.com"
# }

Counting Elements

Count users:

jq 'length' data.json
# Output: 2

Modifying JSON

Add a new field (without changing the original file):

jq '.[] | .active = true' data.json

If you want to modify and save back:

jq 'map(.active = true)' data.json > new_data.json

Sorting

Sort entries by name:

jq 'sort_by(.name)' data.json

Getting Help

jq is a powerful tool with a rich set of features. For help, use:

man jq

or visit the jq manual.

Piping JSON Data

jq works well in data pipelines. For instance, you can use curl to fetch JSON from an API and process it with jq:

curl -s https://api.github.com/users/octocat | jq '.name, .public_repos'

Conclusion

jq makes it easy to parse, filter, and format JSON right from your shell. Whether you're analyzing API output, batch-processing configuration files, or debugging, jq brings the power of JSON manipulation to your fingertips.

Next Steps:

Explore the jq manual
Try jq on real API responses
Integrate jq into your shell scripts for automated JSON processing

With practice, you'll find jq an indispensable addition to your command-line toolbox!

Working with JSON using jq

What is jq?

Installing jq