Using curl in Shell Scripts for APIs

Interact with REST APIs using curl.

When it comes to shell scripting, one command stands out for web interactions: curl. Its versatility allows shell scripts to seamlessly communicate with REST APIs—enabling automation, data retrieval, and integration with cloud services. In this article, you'll learn the essentials of using curl within shell scripts to interact with APIs.

Why Use curl for APIs?

  • Lightweight: No dependencies other than itself.
  • Universality: Included on most Unix-like systems by default.
  • Flexibility: Supports HTTP methods (GET, POST, PUT, DELETE), custom headers, authentication, SSL, and more.
  • Automation: Easily integrated into cron jobs, pipelines, and larger scripts.

Basic Usage: Making API Requests

1. Simple GET Request

To retrieve data from an API:

curl https://api.example.com/v1/users

This command fetches the users' data. However, APIs often expect more—like authentication or headers.

2. Passing Headers

Most APIs require custom headers, such as Authorization tokens or specific formats.

curl -H "Authorization: Bearer YOUR_API_TOKEN" \
     -H "Accept: application/json" \
     https://api.example.com/v1/users

3. Saving and Parsing Output

You typically want to work with the API response in your script.

response=$(curl -s -H "Authorization: Bearer $TOKEN" https://api.example.com/data)
echo "Received: $response"

The -s flag silences progress messages.

Sending Data: POST Requests

To create or update resources, send data (often as JSON).

curl -X POST -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"name": "Alice", "email": "alice@example.com"}' \
     https://api.example.com/v1/users

With Data from a File

curl -X POST -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d @data.json \
     https://api.example.com/v1/users

Full Script Example: GET and Parse JSON

Let's get a list of users and extract user IDs using jq (a command-line JSON processor):

#!/bin/bash

API="https://api.example.com/v1/users"
TOKEN="your-secret-token"

response=$(curl -s -H "Authorization: Bearer $TOKEN" "$API")

# Pretty print response
echo "$response" | jq

# Extract user IDs
echo "$response" | jq -r '.[].id'

Tip: Use jq to parse, filter, and extract specific fields from JSON.

Error Handling and Status Codes

Always check the HTTP status code to ensure your request succeeded.

response=$(curl -s -w "%{http_code}" -o response.json \
  -H "Authorization: Bearer $TOKEN" "$API")

status="${response: -3}"

if [ "$status" -ne 200 ]; then
    echo "API request failed with status $status"
    exit 1
else
    cat response.json | jq
fi
  • -w "%{http_code}": Appends the status code.
  • -o response.json: Outputs body to file.

Authentication Patterns

  • Bearer Token: -H "Authorization: Bearer $TOKEN"
  • Basic Auth: -u username:password
  • API Keys: -H "x-api-key: YOUR_KEY"

Example with basic auth:

curl -u user:password https://api.example.com/secure

Common curl Flags for APIs

Option Description
-X Set HTTP method (GET, POST, etc.)
-H HTTP headers
-d / --data Data payload (for POST/PUT)
-s Silent (no progress meter)
-u Basic auth
-o Write output to file
-w Custom output, e.g. HTTP code
--fail Exit with error on HTTP error

Final Thoughts

Integrating curl with shell scripts allows you to automate everything from report generation to system updates by leveraging APIs. Combine curl with parsing tools (jq, grep, etc.) for robust workflows.

Key takeaways:

  • Use appropriate headers and authentication methods.
  • Always check the response and handle errors gracefully.
  • Employ tools to parse and process API responses.

Start scripting with curl today and unlock the power of APIs in your command line!

Back to All Articles