---
title: "Curl"
description: "cURL is a command-line tool for making HTTP requests, transferring data using URLs, and testing APIs. Essential commands for web development and API testing."
author: "Mohammad Abu Mattar"
canonical: https://mkabumattar.com/cheatsheets/curl
---

# Curl

## Getting Started with Curl

cURL (client URL) is a powerful command-line tool for transferring data using URLs. It supports a wide range of protocols and is indispensable for web development, API testing, and automated scripting. Whether you're testing a REST API, downloading files, or debugging HTTP connections, curl provides the tools you need.

## What Makes cURL Essential

- **Protocol Support**: HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TELNET, LDAP, LDAPS, and more
- **Flexibility**: Works with virtually any web service or API
- **Scripting**: Perfect for automation and creating repeatable workflows
- **Debugging**: Verbose output helps diagnose connection and API issues
- **Lightweight**: No GUI required, runs on minimal resources

## Common Use Cases

```bash
# Test if an API endpoint is responding
curl https://api.github.com/users/github

# Download files
curl -O https://example.com/file.tar.gz

# POST data to a form
curl -X POST -d "username=user&password=pass" https://example.com/login

# Check API response status
curl -I https://api.example.com/health

# Debug API connections
curl -v https://api.example.com/endpoint
```

## Installation Quick Reference

**macOS**: `brew install curl`
**Ubuntu/Debian**: `sudo apt-get install curl`
**CentOS/RHEL**: `sudo yum install curl`
**Windows**: Download from [curl.se](https://curl.se/download.html) or `choco install curl`

## Essential Options Reference

| Option                         | Purpose                        |
| ------------------------------ | ------------------------------ |
| `-X GET/POST/PUT/DELETE/PATCH` | Specify HTTP method            |
| `-H "Header: value"`           | Add custom header              |
| `-d 'data'`                    | Send POST data (URL encoded)   |
| `-F "file=@path"`              | Upload file (multipart form)   |
| `-u user:pass`                 | HTTP Basic authentication      |
| `-b "cookie=value"`            | Send cookie                    |
| `-c cookies.txt`               | Save response cookies          |
| `-i`                           | Include response headers       |
| `-I`                           | Headers only (HEAD request)    |
| `-L`                           | Follow redirects               |
| `-v` / `-vv`                   | Verbose output (debug)         |
| `-s`                           | Silent mode (no progress)      |
| `-o file.txt`                  | Save to file                   |
| `-O`                           | Save with original filename    |
| `-w "%{http_code}"`            | Custom output format           |
| `-k` / `--insecure`            | Skip SSL verification (unsafe) |
| `--cacert ca.pem`              | Use custom CA certificate      |
| `--max-time 10`                | Timeout after 10 seconds       |
| `--compressed`                 | Request compression            |

## HTTP Methods Quick Start

```bash
# GET - Retrieve data (default)
curl https://api.example.com/users

# POST - Create resource
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"John"}' https://api.example.com/users

# PUT - Replace entire resource
curl -X PUT -H "Content-Type: application/json" \
  -d '{"name":"Jane","age":30}' https://api.example.com/users/1

# PATCH - Update specific fields
curl -X PATCH -H "Content-Type: application/json" \
  -d '{"age":31}' https://api.example.com/users/1

# DELETE - Remove resource
curl -X DELETE https://api.example.com/users/1
```

## Workflow Examples

### Testing a REST API

```bash
# 1. Test get all users
curl https://api.example.com/users

# 2. Create a new user
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"Alice","email":"alice@example.com"}' \
  https://api.example.com/users

# 3. Get specific user
curl https://api.example.com/users/1

# 4. Update user
curl -X PATCH -H "Content-Type: application/json" \
  -d '{"email":"alice.new@example.com"}' \
  https://api.example.com/users/1

# 5. Delete user
curl -X DELETE https://api.example.com/users/1
```

### Authenticated API Requests

```bash
# Using bearer token
TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/protected

# Using API key
API_KEY="sk_live_abc123xyz789"
curl -H "X-API-Key: $API_KEY" https://api.example.com/data

# Using Basic auth
curl -u username:password https://api.example.com/protected

# OAuth2 flow
TOKEN=$(curl -X POST -d "client_id=xxx&client_secret=yyy&grant_type=client_credentials" https://auth.example.com/token | jq -r '.access_token')
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/data
```

### File Handling

```bash
# Download file
curl -o myfile.txt https://example.com/file.txt

# Upload file
curl -F "file=@myfile.txt" https://api.example.com/upload

# Download and extract
curl -s https://example.com/archive.tar.gz | tar xz

# Resume interrupted download
curl -C - -o largefile.iso https://example.com/largefile.iso

# Upload multiple files
curl -F "files=@file1.txt" -F "files=@file2.txt" https://api.example.com/upload-multiple
```

### Debugging and Troubleshooting

```bash
# Verbose output (see all request/response details)
curl -v https://api.example.com/data

# More verbose (shows all low-level details)
curl -vv https://api.example.com/data

# Show headers only
curl -i https://api.example.com/data

# Check response time
curl -w "Total time: %{time_total}s\n" https://api.example.com/data

# Save full debug output
curl -v https://api.example.com/data > response.html 2> debug.txt

# Test with custom headers
curl -H "X-Custom: value" -v https://api.example.com/data
```

### Batch Processing

```bash
# Check multiple endpoints
for url in api1.com api2.com api3.com; do
  echo -n "$url: "
  curl -s -o /dev/null -w "%{http_code}\n" "https://$url"
done

# Process JSON array response
curl -s https://api.example.com/users | jq -r '.[] | .email' | \
  while read email; do
    echo "Processing: $email"
    curl -X POST -H "Content-Type: application/json" \
      -d "{\"email\":\"$email\",\"status\":\"active\"}" \
      https://api.example.com/notify
  done

# Parallel requests
for i in {1..10}; do
  curl -s "https://api.example.com/data/$i" &
done
wait
echo "All requests completed"
```

## Performance Tips

- **Use `--compressed`** for APIs that support gzip/deflate compression
- **Add timeouts** with `--max-time` and `--connect-timeout` to prevent hanging
- **Batch requests** with `&` and `wait` for parallel execution
- **Filter responses** with `jq` or `grep` to reduce processing time
- **Use HEAD requests** (`-I`) to check if resource exists without downloading

## Common Patterns

**Check if service is up**:

```bash
curl -f -s -o /dev/null -w "%{http_code}\n" https://api.example.com/health
```

**Loop with retry**:

```bash
for attempt in {1..3}; do
  curl -f https://api.example.com/data && break
  [ $attempt -lt 3 ] && sleep 2
done
```

**Extract and transform**:

```bash
curl -s https://api.example.com/users | jq '[.[] | select(.active) | {name, email}]'
```

**Save with authentication**:

```bash
curl -u user:pass -H "Accept: application/json" https://api.example.com/data > data.json
```

## Best Practices

1. **Security First**: Always use HTTPS for authenticated requests
2. **Handle Errors**: Check exit codes and HTTP status codes
3. **Use Variables**: Store tokens and credentials in environment variables, not in scripts
4. **Validate Input**: Check JSON syntax and validate responses before processing
5. **Add Timeouts**: Prevent hanging requests with `--max-time` and `--connect-timeout`
6. **Log Carefully**: Redact sensitive data from logs
7. **Test Before Production**: Always test API calls against staging before production use
8. **Document Your Calls**: Comment curl commands explaining what they do and why

## curl vs Other Tools

| Tool                | Best For                               |
| ------------------- | -------------------------------------- |
| **curl**            | Command-line, scripts, quick testing   |
| **wget**            | Large file downloads, mirroring sites  |
| **Postman**         | Interactive API testing with GUI       |
| **HTTPie**          | More intuitive syntax, pretty output   |
| **Python requests** | Complex automation and data processing |

## Getting Help

- **Manual**: `man curl` or `curl --help`
- **Online**: [curl.se docs](https://curl.se/docs/)
- **Examples**: [Everything curl book](https://ec.haxx.se/everything-curl.pdf)
- **Common Issues**: Check [curl troubleshooting guide](https://curl.se/docs/faq.html)

---

**Pro Tip**: Create a `.curlrc` file in your home directory to store default options and headers for commonly used APIs. This keeps your commands clean and consistent across multiple requests.

## Getting Started

Learn what curl is and make your first HTTP requests

### What is Curl

Understand curl and its capabilities

**Keywords:** introduction, what, about, curl, http

#### Install curl and check version

```bash
# On macOS with Homebrew
brew install curl

# On Debian/Ubuntu
sudo apt-get install curl

# On CentOS/RHEL
sudo yum install curl

# Check version
curl --version
```

_exec_
```bash
curl --version
```

_output_
```bash
curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1
Release-Date: 2023-02-20
```

Installs curl and displays version information showing libcurl version and supported protocols.

- curl is usually pre-installed on most Unix-like systems
- If missing, install via package manager (apt, yum, brew, etc.)
- curl supports HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, TELNET, LDAP, LDAPS, and many more

#### Basic curl command structure

```bash
# Basic syntax
curl [options] [URL]

# Simplest request - GET a URL
curl https://example.com

# Get just the response code
curl -o /dev/null -s -w "%{http_code}" https://example.com

# Show response headers
curl -i https://example.com
```

_exec_
```bash
curl -s -w "%{http_code}\n" https://httpbin.org/get
```

_output_
```bash
200
```

Demonstrates basic curl syntax and how to retrieve HTTP status codes from endpoints.

- By default, curl sends a GET request to the URL
- -s flag enables silent mode (no progress meter)
- -w flag allows custom output formatting with HTTP response codes
- Response is written to stdout unless redirected

#### Quick API testing examples

```bash
# Test if a server is up
curl -I https://api.example.com

# Get response with timing information
curl -w "Response time: %{time_total}s\n" https://example.com

# Test followed by status code only
curl -s -o /dev/null -w "HTTP Status: %{http_code}\n" https://api.example.com
```

_exec_
```bash
curl -I -s https://httpbin.org/get
```

_output_
```bash
HTTP/2 200
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json
Server: gunicorn/19.9.0
```

Quickly test if APIs are responding without downloading full response body.

- -I flag retrieves headers only (HEAD request)
- Useful for health checks and availability testing
- -w provides detailed timing information

**Best practices:**

- [object Object]
- Use curl for simple requests, scripts, and debugging APIs
- Use verbose mode (-v) when troubleshooting connections

**Common errors:**

- **Could not resolve host**: Check internet connection and hostname spelling
- **Connection timed out**: Verify server is running and firewall allows connections

### HTTP Methods Fundamentals

Understanding and using different HTTP methods

**Keywords:** methods, get, post, put, delete, patch

#### Specify HTTP method with -X flag

```bash
# GET request (default)
curl https://api.example.com/users

# Explicitly specify GET
curl -X GET https://api.example.com/users

# Use different methods
curl -X POST https://api.example.com/users
curl -X PUT https://api.example.com/users/1
curl -X DELETE https://api.example.com/users/1
curl -X PATCH https://api.example.com/users/1
```

_exec_
```bash
curl -X GET https://httpbin.org/get
```

_output_
```bash
{
  "args": {},
  "headers": {
    "Host": "httpbin.org",
    "User-Agent": "curl/7.88.1"
  },
  "origin": "192.0.2.1",
  "url": "https://httpbin.org/get"
}
```

Use the -X flag to explicitly specify the HTTP method for your request.

- GET is the default method (retrieves data)
- POST creates new resources
- PUT replaces entire resource
- DELETE removes resources
- PATCH modifies part of resource

#### Method-specific behavior

```bash
# GET - no body sent
curl https://api.example.com/users?page=1

# POST - requires body
curl -X POST -d "name=John" https://api.example.com/users

# PUT - replace entire resource
curl -X PUT -d '{"name":"Jane","email":"jane@example.com"}' https://api.example.com/users/1

# DELETE - no body, just delete
curl -X DELETE https://api.example.com/users/1
```

_exec_
```bash
curl -X POST -d "key=value" https://httpbin.org/post
```

_output_
```bash
{
  "form": {
    "key": "value"
  },
  "headers": {
    "Content-Type": "application/x-www-form-urlencoded"
  }
}
```

Different HTTP methods handle data and requests differently, with appropriate use cases.

- GET requests are cacheable and idempotent
- POST creates new resources and is not idempotent
- Methods determine server behavior and response

**Best practices:**

- Use correct HTTP method for intended operation
- GET for retrieval, POST for creation, PUT for updates, DELETE for deletion
- Respect API documentation for method requirements

**Common errors:**

- **405 Method Not Allowed**: Check API docs for supported methods on endpoint

### Simple Requests

Make your first requests to APIs and websites

**Keywords:** simple, basic, easy, first, request

#### Get a web page HTML content

```bash
# Fetch HTML from a website
curl https://example.com

# Save HTML to file
curl https://example.com > page.html

# Get with all headers
curl -i https://example.com

# Save to file with proper name
curl -o index.html https://example.com
```

_exec_
```bash
curl -s https://httpbin.org/html | head -20
```

_output_
```bash
<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
      <h1>Herman Melville - Moby-Dick</h1>
```

Fetch web content using curl and save or display the response.

- Without flags, response is printed to stdout
- Use > to redirect to file or -o to save
- -i includes HTTP headers in response

#### Query parameters and URLs

```bash
# Simple URL with query parameters
curl "https://api.example.com/search?q=curl&page=1"

# Handle special characters in parameters
curl "https://api.example.com/search?q=hello%20world"

# URL encoding alternative
curl --data-urlencode "q=hello world" https://api.example.com/search

# Test with httpbin
curl "https://httpbin.org/get?name=John&age=30"
```

_exec_
```bash
curl -s "https://httpbin.org/get?name=Alice&age=25" | grep -A5 args
```

_output_
```bash
"args": {
  "age": "25",
  "name": "Alice"
}
```

Pass query parameters to APIs using URL query strings.

- Use quotes around URLs with special characters
- Query parameters start with ?
- Multiple parameters separated by &
- URL encode special characters

#### Follow redirects

```bash
# Follow HTTP redirects
curl -L https://example.com

# Follow max 5 redirects
curl -L --max-redirs 5 https://example.com

# Show redirect chain
curl -L -v https://example.com 2>&1 | grep ">"
```

_exec_
```bash
curl -L -s -w "Final URL: %{url_effective}\n" https://httpbin.org/redirect/2
```

_output_
```bash
Final URL: https://httpbin.org/relative-redirect/1
```

Use -L flag to automatically follow HTTP redirects (301, 302, etc.).

- Default curl doesn't follow redirects
- -L (--location) enables automatic redirect following
- --max-redirs limits maximum redirects to prevent loops

**Best practices:**

- Always use quotes around URLs with special characters
- Use -L when testing public APIs to handle redirects
- Save responses to files for analysis

**Common errors:**

- **Empty response or just headers**: Check if page redirects (-L flag helps)

## Request Methods

Detailed usage of different HTTP request methods

### GET Requests

Retrieve data from servers with GET requests

**Keywords:** get, retrieve, fetch, query

#### Basic GET requests with parameters

```bash
# Get with parameters
curl https://api.example.com/users?limit=10&offset=0

# Get specific resource
curl https://api.example.com/users/123

# Get with multiple parameters
curl "https://api.example.com/posts?author=john&status=published&sort=date"

# Use -G to send POST data as GET
curl -G https://api.example.com/search -d "q=api" -d "limit=10"
```

_exec_
```bash
curl -s "https://httpbin.org/get?id=1&name=test" | python3 -m json.tool
```

_output_
```bash
{
  "args": {
    "id": "1",
    "name": "test"
  },
  "url": "https://httpbin.org/get?id=1&name=test"
}
```

GET requests retrieve data with parameters appended to the URL.

- GET parameters are visible in URL
- Use for non-sensitive data
- Parameters passed via query string

#### Conditional GET requests

```bash
# Get only if modified since date (caching)
curl -H "If-Modified-Since: Mon, 12 Dec 2022 13:00:00 GMT" https://api.example.com/data

# Check if resource exists (HEAD request for headers only)
curl -I https://api.example.com/users/123

# Get with timeout
curl --max-time 5 https://api.example.com/data
```

_exec_
```bash
curl -I -s "https://httpbin.org/get" | head -5
```

_output_
```bash
HTTP/1.1 200 OK
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json
Content-Length: 347
```

Optimize GET requests with conditional headers and timeouts.

- -I gets headers only (more efficient)
- --max-time prevents hanging on slow servers
- If-Modified-Since improves caching

#### Handle large responses

```bash
# Get large file and show progress
curl -# https://api.example.com/large-dataset.json -o data.json

# Resume interrupted download
curl -C - -o data.json https://api.example.com/large-file.json

# Stream response (don't wait for completion)
curl -N https://api.example.com/stream

# Get with progress bar
curl -# -O https://example.com/file.zip
```

_exec_
```bash
curl -s "https://httpbin.org/get" | wc -c
```

_output_
```bash
389
```

Download large files efficiently with progress tracking and resume capability.

- -# shows simple progress bar
- -C - resumes partial downloads
- -N enables streaming responses
- -O saves with original filename

**Best practices:**

- Use query parameters for filtering and pagination
- Always quote URLs with parameters
- Use -I flag for HEAD requests when only checking headers

**Common errors:**

- **400 Bad Request**: Check parameter syntax and URL encoding

### POST Requests

Send data to servers with POST requests

**Keywords:** post, create, submit, data

#### POST with form data

```bash
# POST with form data (URL encoded)
curl -X POST -d "name=John&email=john@example.com" https://api.example.com/users

# POST with multiple form fields
curl -d "user=john" -d "password=secret" -d "login=true" https://api.example.com/login

# Post data from file
curl -d @data.txt https://api.example.com/submit

# Alternative form data syntax
curl -X POST --data "key1=value1&key2=value2" https://api.example.com/create
```

_exec_
```bash
curl -s -X POST -d "user=john&age=30" https://httpbin.org/post | python3 -m json.tool
```

_output_
```bash
{
  "form": {
    "age": "30",
    "user": "john"
  }
}
```

Send form data using POST requests with URL-encoded parameters.

- -d automatically sets Content-Type to application/x-www-form-urlencoded
- Multiple -d flags combine parameters
- @filename sends file contents as data

#### POST with JSON data

```bash
# POST JSON data
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  https://api.example.com/users

# POST JSON from file
curl -X POST -H "Content-Type: application/json" \
  -d @user.json https://api.example.com/users

# Using jq to format JSON
curl -X POST -H "Content-Type: application/json" \
  -d "$(echo '{"name":"John"}' | jq .)" \
  https://api.example.com/users
```

_exec_
```bash
curl -s -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://httpbin.org/post | python3 -m json.tool
```

_output_
```bash
{
  "json": {
    "key": "value"
  }
}
```

Send JSON data using proper Content-Type header to APIs expecting JSON.

- Must set Content-Type: application/json header
- Payload sent as JSON string
- Use single quotes to preserve JSON structure

#### POST with file upload

```bash
# Upload file with multipart form
curl -F "file=@image.png" https://api.example.com/upload

# Upload with additional fields
curl -F "file=@photo.jpg" -F "title=My Photo" -F "description=A beautiful photo" https://api.example.com/upload

# Upload multiple files
curl -F "files=@file1.txt" -F "files=@file2.txt" https://api.example.com/upload-multiple

# Upload with HTTP Basic auth
curl -F "file=@document.pdf" -u user:pass https://api.example.com/upload
```

_exec_
```bash
curl -s -F "file=@/etc/hosts" -F "title=test" https://httpbin.org/post
```

_output_
```bash
{
  "files": {
    "file": "[binary content]"
  },
  "form": {
    "title": "test"
  }
}
```

Upload files using multipart/form-data encoding with optional additional fields.

- -F flag sets Content-Type to multipart/form-data automatically
- @ symbol indicates file path
- Can combine multiple files and form fields

**Best practices:**

- Use JSON for APIs, form data for HTML forms
- Always set proper Content-Type header
- Send sensitive data via POST, not GET

**Common errors:**

- **415 Unsupported Media Type**: Check Content-Type header matches API requirements

### PUT, PATCH, and DELETE Requests

Update and delete resources with different methods

**Keywords:** put, patch, delete, update, remove

#### PUT and PATCH requests

```bash
# PUT - Replace entire resource
curl -X PUT -H "Content-Type: application/json" \
  -d '{"name":"Jane","email":"jane@example.com","age":28}' \
  https://api.example.com/users/1

# PATCH - Update specific fields only
curl -X PATCH -H "Content-Type: application/json" \
  -d '{"email":"newemail@example.com"}' \
  https://api.example.com/users/1

# PUT with form data
curl -X PUT -d "name=John&status=active" https://api.example.com/users/1

# PATCH with file
curl -X PATCH -H "Content-Type: application/json" \
  -d @user-update.json https://api.example.com/users/1
```

_exec_
```bash
curl -s -X PATCH -H "Content-Type: application/json" -d '{"status":"updated"}' https://httpbin.org/patch | python3 -m json.tool
```

_output_
```bash
{
  "json": {
    "status": "updated"
  }
}
```

Use PUT to replace entire resources and PATCH to partially update them.

- PUT replaces entire resource (must provide all fields)
- PATCH updates only specified fields
- Both typically require authentication

#### DELETE requests

```bash
# Simple DELETE
curl -X DELETE https://api.example.com/users/1

# DELETE with response verification
curl -X DELETE -w "Status: %{http_code}\n" https://api.example.com/users/1

# DELETE with authorization
curl -X DELETE -H "Authorization: Bearer token123" https://api.example.com/users/1

# DELETE with query parameters
curl -X DELETE "https://api.example.com/posts?id=123&confirm=true"
```

_exec_
```bash
curl -X DELETE -w "\nHTTP Status: %{http_code}\n" https://httpbin.org/delete
```

_output_
```bash
HTTP Status: 200
```

Remove resources from the server using DELETE requests.

- DELETE is idempotent (multiple calls have same effect)
- Typically returns 204 (No Content) or 200 (OK)
- May require authentication

#### Request with body and status verification

```bash
# Update and verify response
curl -X PUT \
  -H "Content-Type: application/json" \
  -d '{"status":"active"}' \
  -w "\nStatus: %{http_code}\nTime: %{time_total}s\n" \
  https://api.example.com/users/1

# Delete and check status
curl -i -X DELETE https://api.example.com/users/1

# Save response to file on successful update
curl -X PATCH -d "status=completed" \
  -o response.json \
  -w "%{http_code}" \
  https://api.example.com/tasks/1
```

_exec_
```bash
curl -i -X DELETE https://httpbin.org/delete | head -10
```

_output_
```bash
HTTP/2 200
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json
```

Combine requests with response verification and output handling.

- -i includes headers and response
- -w allows custom output formatting
- Combine with -o to save to file

**Best practices:**

- Use PUT for complete replacement, PATCH for partial updates
- Always verify status codes for update operations
- Handle 409 (Conflict) and 404 (Not Found) errors

**Common errors:**

- **404 Not Found**: Verify resource ID is correct
- **409 Conflict**: Resource may have been modified, try refreshing

## Headers & Authentication

Work with HTTP headers and authentication mechanisms

### Custom Headers

Add custom headers to requests

**Keywords:** headers, custom, user-agent, referer

#### Set custom headers with -H flag

```bash
# Single header
curl -H "X-API-Key: abc123" https://api.example.com/data

# Multiple headers
curl -H "X-API-Key: abc123" -H "X-Request-ID: req-123" https://api.example.com/data

# Content-Type for JSON
curl -H "Content-Type: application/json" \
  -d '{"key":"value"}' https://api.example.com/submit

# Accept header for response format
curl -H "Accept: application/json" https://api.example.com/data
```

_exec_
```bash
curl -s -H "X-Custom: hello" https://httpbin.org/headers | python3 -m json.tool
```

_output_
```bash
{
  "headers": {
    "X-Custom": "hello",
    "User-Agent": "curl/7.88.1"
  }
}
```

Add custom headers to requests using the -H flag (multiple times if needed).

- Format: -H "Header-Name: Header-Value"
- Multiple headers require multiple -H flags
- Headers are case-insensitive

#### User-Agent and Referer headers

```bash
# Set user agent (identify as browser)
curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" https://example.com

# Alternative syntax for user agent
curl -H "User-Agent: MyApp/1.0" https://api.example.com/data

# Set referer (HTTP_REFERER)
curl -e "https://google.com" https://example.com

# Combined user agent and referer
curl -A "Mozilla/5.0" -e "https://google.com" https://example.com
```

_exec_
```bash
curl -s -A "CustomBot/1.0" https://httpbin.org/user-agent
```

_output_
```bash
{
  "user-agent": "CustomBot/1.0"
}
```

Set User-Agent and Referer headers to identify your request origin and client.

- -A is shortcut for setting User-Agent
- -e sets Referer header (note spelling)
- Some sites may block requests without proper User-Agent

#### Request header inspection

```bash
# View all request headers sent
curl -v https://api.example.com/data 2>&1 | grep ">"

# Show comprehensive header information
curl -v -H "Authorization: Bearer token" https://api.example.com/data 2>&1 | head -20

# Test with custom headers and see what was sent
curl -H "X-Test: value1" -H "X-Test2: value2" -v https://httpbin.org/headers 2>&1 | grep ">"
```

_exec_
```bash
curl -v https://httpbin.org/headers 2>&1 | grep ">"
```

_output_
```bash
> GET /headers HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.88.1
> Accept: */*
```

View outgoing request headers using -v (verbose) flag.

- Lines starting with > show request headers
- Lines starting with < show response headers
- 2>&1 redirects stderr to stdout

**Best practices:**

- Always set Content-Type for requests with body
- Use Accept header to specify desired response format
- Set User-Agent when scraping or testing web services

**Common errors:**

- **415 Unsupported Media Type**: Ensure Content-Type header matches data format

### Authentication Methods

Authenticate requests with various methods

**Keywords:** auth, authentication, basic, bearer, token

#### HTTP Basic Authentication

```bash
# Basic auth with -u flag
curl -u username:password https://api.example.com/protected

# Basic auth alternative with header
curl -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" https://api.example.com/protected

# Base64 encode credentials
echo -n "username:password" | base64

# Prompt for password (don't show in history)
curl -u username https://api.example.com/protected
```

_exec_
```bash
curl -s -u "user:pass" https://httpbin.org/basic-auth/user/pass
```

_output_
```bash
{
  "authenticated": true,
  "user": "user"
}
```

Use HTTP Basic Authentication with -u flag (username:password).

- Credentials are Base64 encoded, not encrypted
- Always use HTTPS with Basic Auth (not HTTP)
- -u without password prompts for it securely

#### Bearer token and API key authentication

```bash
# Bearer token (OAuth2, JWT, etc)
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIs..." https://api.example.com/data

# API key in header
curl -H "X-API-Key: sk_live_abc123xyz789" https://api.example.com/data

# API key as query parameter
curl "https://api.example.com/data?api_key=sk_live_abc123"

# Store token in variable and use it
TOKEN="eyJhbGciOiJIUzI1NiIs..."
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/data
```

_exec_
```bash
curl -s -H "Authorization: Bearer test-token" https://httpbin.org/bearer | python3 -m json.tool
```

_output_
```bash
{
  "authenticated": true,
  "token": "test-token"
}
```

Authenticate with bearer tokens (JWT, OAuth2) and API keys.

- Bearer tokens include "Bearer " prefix
- API keys are commonly passed in headers or query params
- Tokens are sensitive, avoid including in logs

#### Complex authentication scenarios

```bash
# Digest authentication (more secure than basic)
curl --digest -u username:password https://api.example.com/protected

# Get JWT token then use it
TOKEN=$(curl -s -X POST -d "user=john&pass=secret" https://api.example.com/auth | jq -r '.token')
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/data

# Use netrc file for credentials (~/.netrc)
curl --netrc https://api.example.com/protected

# Pass credentials in URL (deprecated, use -u instead)
curl "https://user:pass@api.example.com/protected"
```

_exec_
```bash
curl -s --digest -u "user:passwd" https://httpbin.org/digest-auth/auth/user/passwd
```

_output_
```bash
{
  "authenticated": true,
  "user": "user"
}
```

Implement more advanced authentication flows and secure credential handling.

- Digest auth is more secure than Basic auth
- Never include passwords in URLs
- Use environment variables or .netrc for sensitive credentials

**Best practices:**

- Always use HTTPS for authenticated requests
- Store tokens in environment variables or secure vaults
- Use -u without password to avoid shell history

**Common errors:**

- **401 Unauthorized**: Check credentials and token expiration

### Cookies & Sessions

Handle cookies and maintain sessions

**Keywords:** cookies, session, cache, maintain

#### Set and send cookies

```bash
# Send cookie with request
curl -b "session=abc123" https://api.example.com/data

# Multiple cookies
curl -b "session=abc123" -b "user_id=456" https://api.example.com/data

# Cookie from file
curl -b cookies.txt https://api.example.com/data

# Send raw cookie string
curl -b "name1=value1; name2=value2" https://example.com
```

_exec_
```bash
curl -s -b "test=value123" https://httpbin.org/cookies | python3 -m json.tool
```

_output_
```bash
{
  "cookies": {
    "test": "value123"
  }
}
```

Send cookies with requests using -b flag.

- -b sends cookies, -B sends from file, -c saves to file
- Multiple -b flags combine cookies
- Cookie format: name=value

#### Save and load cookies

```bash
# Save response cookies to file
curl -c cookies.txt https://api.example.com/login

# Load cookies from file for subsequent requests
curl -b cookies.txt https://api.example.com/protected

# Save and load in single command
curl -c cookies.txt -b cookies.txt https://api.example.com/data

# Enable cookie jar for session persistence
curl -c ~/.curl_cookies -b ~/.curl_cookies https://api.example.com/login
```

_exec_
```bash
curl -s https://httpbin.org/cookies/set?test=value123 -c /tmp/cookies.txt && cat /tmp/cookies.txt
```

_output_
```bash
# Netscape HTTP Cookie File
.httpbin.org	TRUE	/	FALSE	0	test	value123
```

Save cookies from responses and reuse them in subsequent requests for session management.

- -c saves response cookies to file
- Netscape HTTP Cookie File format is standard
- Useful for maintaining sessions across multiple requests

#### Session workflow example

```bash
# Typical workflow: login, then use session

# 1. Login and save session cookie
curl -c cookies.txt -X POST \
  -d "username=john&password=secret" \
  https://api.example.com/login

# 2. Use session for authenticated requests
curl -b cookies.txt https://api.example.com/profile

# 3. Make multiple requests with same session
curl -b cookies.txt https://api.example.com/posts
curl -b cookies.txt -X POST -d "content=hello" https://api.example.com/posts

# 4. Logout and clear cookies
curl -b cookies.txt -X POST https://api.example.com/logout
rm cookies.txt
```

_exec_
```bash
curl -s -c - https://httpbin.org/cookies/set?session=xyz123 | grep -i httpbin
```

_output_
```bash
.httpbin.org	TRUE	/	FALSE	0	session	xyz123
```

Manage complete session workflow from login to authenticated requests to logout.

- Always save and load cookies for stateful APIs
- Use same cookie jar across related requests
- Clear cookies after logout for security

**Best practices:**

- Use cookie files for session persistence in scripts
- Keep cookies file secure if contains sensitive data
- Clear cookies when session ends

**Common errors:**

- **403 Forbidden after login**: Verify cookies were saved and loaded properly

## Data Transfer

Handle different data formats and file operations

### JSON Data

Work with JSON data in requests and responses

**Keywords:** json, data, api, format

#### Send JSON requests

```bash
# Basic JSON POST
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  https://api.example.com/users

# Pretty JSON with jq
curl -X POST -H "Content-Type: application/json" \
  -d "$(jq -n --arg name "John" --arg email "john@example.com" '{name: $name, email: $email}')" \
  https://api.example.com/users

# Nested JSON
curl -X POST -H "Content-Type: application/json" \
  -d '{"user":{"name":"John","address":{"city":"NYC"}},"active":true}' \
  https://api.example.com/users
```

_exec_
```bash
curl -s -X POST -H "Content-Type: application/json" -d '{"test":"data"}' https://httpbin.org/post | python3 -m json.tool
```

_output_
```bash
{
  "json": {
    "test": "data"
  }
}
```

Send properly formatted JSON data to APIs.

- Always set Content-Type: application/json
- Single quotes preserve special characters in JSON
- Escape quotes in JSON: \" for double quotes

#### Parse and modify JSON responses

```bash
# Get JSON and parse with jq
curl -s https://api.example.com/users | jq '.'

# Extract specific field
curl -s https://api.example.com/users | jq '.users[0].name'

# Filter and map responses
curl -s https://api.example.com/users | jq '[.users[] | select(.active) | {name, email}]'

# Modify response and send as next request
curl -s https://api.example.com/users/1 | jq '.status = "updated"' | \
  curl -X PATCH -H "Content-Type: application/json" -d @- https://api.example.com/users/1
```

_exec_
```bash
curl -s https://httpbin.org/json | python3 -m json.tool | head -10
```

_output_
```bash
{
  "slideshow": {
    "author": "Yours Truly",
    "date": "2003-05-20",
    "slides": [
      {
        "title": "Wake up to WonderWidgets!"
      }
    ]
  }
}
```

Parse, filter, and manipulate JSON responses using jq for complex data operations.

- Install jq: apt-get install jq (or brew install jq on macOS)
- jq is powerful for JSON transformation
- (hyphen) in curl -d @- reads from stdin

#### JSON to form data conversion

```bash
# Convert JSON to form data
curl -X POST -H "Content-Type: application/json" \
  -d '{"username":"john","password":"secret"}' \
  https://api.example.com/api-submit

# Compare with form data version
curl -X POST \
  -d "username=john&password=secret" \
  https://api.example.com/form-submit

# Use correct Content-Type based on API requirements
# JSON APIs need: application/json
# Form APIs need: application/x-www-form-urlencoded
# Multipart APIs need: multipart/form-data (auto with -F)
```

_exec_
```bash
curl -s -X POST -d "key=value&test=data" https://httpbin.org/post | python3 -m json.tool | grep -A5 form
```

_output_
```bash
"form": {
  "key": "value",
  "test": "data"
}
```

Choose correct Content-Type based on API requirements.

- JSON uses application/json
- Form data uses application/x-www-form-urlencoded (default -d)
- Multipart forms use multipart/form-data (-F)

**Best practices:**

- Always validate JSON syntax before sending
- Use jq for complex JSON transformations
- Set Content-Type explicitly for clarity

**Common errors:**

- **Invalid JSON**: Use jq to validate syntax before sending

### File Uploads & Downloads

Upload and download files efficiently

**Keywords:** upload, download, file, transfer

#### Upload files with multipart form

```bash
# Upload single file
curl -F "file=@/path/to/file.txt" https://api.example.com/upload

# Upload with additional metadata
curl -F "file=@/path/to/image.png" \
  -F "title=My Image" \
  -F "description=Beautiful image" \
  https://api.example.com/upload

# Upload multiple files
curl -F "files=@file1.txt" -F "files=@file2.txt" -F "files=@file3.txt" \
  https://api.example.com/upload-batch

# Upload with authentication
curl -F "file=@document.pdf" -u user:pass https://api.example.com/upload
```

_exec_
```bash
curl -s -F "file=@/etc/hostname" https://httpbin.org/post | python3 -m json.tool | head -15
```

_output_
```bash
{
  "files": {
    "file": "[binary content]"
  }
}
```

Upload files with multipart form encoding for APIs that accept file uploads.

- -F flag automatically sets multipart/form-data
- @ prefix indicates file path
- Repeat -F for multiple files with same field name

#### Download and resume downloads

```bash
# Download file
curl -o filename.zip https://example.com/large-file.zip

# Download with original filename
curl -O https://example.com/file.pdf

# Resume interrupted download
curl -C - -o file.zip https://example.com/file.zip

# Show download progress
curl -# -o file.zip https://example.com/file.zip

# Download multiple files
curl -o file1.txt https://example.com/file1.txt -o file2.txt https://example.com/file2.txt
```

_exec_
```bash
curl -s -I https://httpbin.org/image/jpeg | grep Content-Length
```

_output_
```bash
Content-Length: 8633
```

Download files efficiently with progress tracking and resume capability.

- -o specifies output filename
- -O uses original filename from URL
- -C - resumes partial downloads
- # shows simple progress bar

#### Advanced file operations

```bash
# Download and pipe to another command
curl -s https://example.com/archive.tar.gz | tar xz

# Download multiple files from listing
curl -s https://example.com/files.json | jq -r '.[] | .url' | \
  xargs -I {} curl -O {}

# Upload file with progress and error handling
if curl -F "file=@largefile.bin" \
  --progress-bar \
  https://api.example.com/upload; then
  echo "Upload successful"
else
  echo "Upload failed"
fi

# Download with custom headers and authentication
curl -H "Authorization: Bearer token" \
  -H "Accept: application/octet-stream" \
  -o downloaded-file \
  https://api.example.com/download
```

_exec_
```bash
curl -s https://httpbin.org/image/png | file -
```

_output_
```bash
standard input: PNG image data, 200 x 200...
```

Combine curl with other tools for complex file operations and error handling.

- Pipe output to tar, unzip, etc. for automated extraction
- Use xargs to download multiple files in parallel
- Check status codes ($?) for error handling

**Best practices:**

- Use -C - to resume interrupted downloads
- Show progress with
- Always verify file integrity after download if critical

**Common errors:**

- **Permission denied when uploading**: Check file permissions and API authorization

### Streaming & Chunked Data

Handle streaming and chunked data transfer

**Keywords:** streaming, chunks, large, progressive

#### Stream responses

```bash
# Stream without buffering (don't wait for complete response)
curl -N https://api.example.com/stream

# Process streaming JSON lines
curl -N https://example.com/events | while read line; do
  echo "$line" | jq '.event'
done

# Stream with timeout
curl -N --max-time 30 https://api.example.com/stream

# Save stream content
curl -N https://example.com/stream -o stream.log
```

_exec_
```bash
timeout 2 curl -s -N https://httpbin.org/stream-bytes/100 | xxd | head -5
```

_output_
```bash
00000000: 8d5e 16cf 9cac 098e e3dc 1fa3 5f7e 9bb7  .^.........._~..
00000010: 89e9 7c5a ed66 7fa5 e901 40b5 0b46 6903  ..|Z.f....@..Fi.
```

Receive streaming responses and process them line-by-line without waiting.

- -N enables streaming (no buffering)
- Useful for long-lived streams and event feeds
- Can save to file or pipe to other commands

#### Chunked uploads

```bash
# Upload in chunks (for large files)
# Using HTTP/1.1 Transfer-Encoding: chunked
curl -H "Transfer-Encoding: chunked" \
  -d @largefile.bin \
  https://api.example.com/upload

# Stream file content
curl -X PUT -H "Content-Type: application/octet-stream" \
  --data-binary @file.bin \
  https://api.example.com/objects/file.bin

# Split large file and upload parts
split -b 1M largefile.bin part_
for part in part_*; do
  curl -X POST -F "file=@$part" \
    https://api.example.com/upload-part
done
```

_exec_
```bash
echo "test data" | curl -s -X POST -d @- https://httpbin.org/post
```

_output_
```bash
{
  "data": "test data"
}
```

Upload large files in chunks and streams for efficient bandwidth usage.

- Transfer-Encoding: chunked for chunked uploads
- --data-binary preserves binary file content
- Split and reassemble for multi-part uploads

**Best practices:**

- Use streaming for real-time data and large files
- Add timeout to prevent hanging streams
- Process chunked data immediately to save memory

**Common errors:**

- **premature end of stream**: Check connection stability, add timeout

## Output & Debugging

Inspect responses and debug requests

### Verbose Mode & Inspection

Debug requests and responses with verbose output

**Keywords:** verbose, debug, headers, inspect

#### Enable verbose mode for troubleshooting

```bash
# Basic verbose mode (shows headers)
curl -v https://api.example.com/data

# Extra verbose (shows all details)
curl -vv https://api.example.com/data

# Verbose to file
curl -v https://api.example.com/data > response.html 2> debug.txt

# Show request headers only
curl -v https://api.example.com/data 2>&1 | grep ">"

# Show response headers only
curl -v https://api.example.com/data 2>&1 | grep "<"
```

_exec_
```bash
curl -v https://httpbin.org/get 2>&1 | head -15
```

_output_
```bash
*   Trying 34.201.40.105:443...
* Connected to httpbin.org (34.201.40.105) port 443 (#0)
* ALPN, offering h2
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.88.1
```

Use verbose mode to see all request and response details for debugging.

- Lines starting with > are request headers
- Lines starting with < are response headers
- Lines starting with * are curl debug info

#### Show headers without body

```bash
# Headers only (no body)
curl -i https://api.example.com/data

# Headers from HEAD request only
curl -I https://api.example.com/data

# Just response headers with body as well
curl -D - https://api.example.com/data

# Dump headers to file
curl -D headers.txt https://api.example.com/data
```

_exec_
```bash
curl -I -s https://httpbin.org/json 2>&1
```

_output_
```bash
HTTP/2 200
date: Fri, 28 Feb 2025 12:00:00 GMT
content-type: application/json
content-length: 429
```

Inspect response headers without downloading body content.

- -i includes headers and body
- -I gets headers only (HEAD request)
- -D saves headers to file

#### Detailed timing and trace information

```bash
# Show timing breakdown
curl -w "@curl-format.txt" -o /dev/null https://api.example.com/data

# Simple timing format
curl -w "\nTime: %{time_total}s\nSpeed: %{speed_download} bytes/s\n" -o /dev/null https://example.com

# Comprehensive timing
curl -w "\n\nDNS Lookup: %{time_namelookup}s\nTCP Connect: %{time_connect}s\nFirst Byte: %{time_starttransfer}s\nTotal Time: %{time_total}s\n" https://api.example.com/data

# Trace connection information
curl --trace-ascii debug.txt https://api.example.com/data
```

_exec_
```bash
curl -w "\nStatus: %{http_code}\nTime: %{time_total}s\n" -o /dev/null -s https://httpbin.org/get
```

_output_
```bash
Status: 200
Time: 0.452341s
```

Measure performance metrics and timing information for API requests.

- %{time_total} total request time
- %{time_namelookup} DNS resolution time
- %{http_code} HTTP response status code
- --trace-ascii for detailed debugging

**Best practices:**

- Use -v to understand request/response flow
- Check timing with -w for performance issues
- Redirect stderr (2>&1) to capture verbose output

**Common errors:**

- **SSL certificate problem**: Try -k for insecure or --cacert for custom certificate

### Filter & Customize Output

Filter responses and customize output format

**Keywords:** output, filter, grep, sed

#### Extract specific data from responses

```bash
# Extract JSON field with jq
curl -s https://api.example.com/users | jq '.users[0].email'

# Extract HTML content with grep
curl -s https://example.com | grep -oP '(?<=<title>)[^<]*'

# Extract HTTP header value
curl -s -I https://api.example.com/data | grep -i "content-type"

# Count items in response
curl -s https://api.example.com/items | jq '.items | length'
```

_exec_
```bash
curl -s https://httpbin.org/get | python3 -c "import json, sys; data=json.load(sys.stdin); print(data['url'])"
```

_output_
```bash
https://httpbin.org/get
```

Extract specific fields from API responses using jq, grep, and other tools.

- jq best for JSON
- grep with -oP for regex extraction
- pipe | allows chaining commands

#### Format output with -w flag

```bash
# Status code only
curl -w "%{http_code}\n" -o /dev/null https://api.example.com/data

# URL and status
curl -w "%{url_effective}\t%{http_code}\n" -o /dev/null https://api.example.com

# Multiple values separated by tabs
curl -w "%{http_code}\t%{time_total}\t%{size_download}\n" -o /dev/null https://api.example.com/data

# Custom format for batch processing
curl -w "%{filename_effective},%{http_code},%{time_total}\n" -o data.json https://api.example.com/data
```

_exec_
```bash
curl -s -w "\nURL: %{url_effective}\nStatus: %{http_code}\nSize: %{size_download} bytes\n" https://httpbin.org/get -o /dev/null
```

_output_
```bash
URL: https://httpbin.org/get
Status: 200
Size: 389 bytes
```

Customize output format using -w flag for scripting and batch operations.

- %{http_code} HTTP status code
- %{time_total} total time
- %{size_download} downloaded size

#### Batch processing and response validation

```bash
# Check multiple endpoints
for url in https://api.example.com/users https://api.example.com/posts https://api.example.com/comments; do
  status=$(curl -s -o /dev/null -w "%{http_code}" "$url")
  echo "$url: $status"
done

# Validate JSON response
response=$(curl -s https://api.example.com/data)
if echo "$response" | jq empty 2>/dev/null; then
  echo "Valid JSON"
else
  echo "Invalid JSON: $response"
fi

# Extract and process multiple values
curl -s https://api.example.com/users | jq -r '.[] | "ID: \(.id), Name: \(.name), Email: \(.email)"'
```

_exec_
```bash
curl -s https://httpbin.org/get | python3 -c "import json, sys; json.load(sys.stdin); print('Valid JSON')"
```

_output_
```bash
Valid JSON
```

Validate responses and process multiple items from API responses.

- Use jq empty to validate JSON without output
- Process arrays with jq .[]
- Combine with bash loops for batch operations

**Best practices:**

- Use jq for JSON filtering and transformation
- Validate responses before processing
- Use -w for custom output in scripts

**Common errors:**

- **jq parse error**: Ensure response is valid JSON

### Error Handling & Status

Handle errors and check response status

**Keywords:** errors, status, codes, handling

#### Check HTTP status codes

```bash
# Get HTTP status code
status=$(curl -s -o /dev/null -w "%{http_code}" https://api.example.com/data)

# Check if successful (2xx)
if [ $status -ge 200 ] && [ $status -lt 300 ]; then
  echo "Success: $status"
elif [ $status -ge 400 ] && [ $status -lt 500 ]; then
  echo "Client error: $status"
else
  echo "Server error: $status"
fi

# Fail on non-2xx status
curl -f https://api.example.com/data || echo "Request failed"
```

_exec_
```bash
curl -f -s https://httpbin.org/status/404 2>&1 || echo "Exit code: $?"
```

_output_
```bash
Exit code: 22
```

Extract and evaluate HTTP status codes for error handling.

- -f flag returns non-zero exit on HTTP errors
- 2xx = success, 4xx = client error, 5xx = server error
- Check $? for curl exit code

#### Retry logic for failed requests

```bash
# Simple retry with exponential backoff
retry_count=0
max_retries=3
while [ $retry_count -lt $max_retries ]; do
  if curl -f https://api.example.com/data; then
    echo "Success"
    break
  fi
  retry_count=$((retry_count + 1))
  sleep $((2 ** retry_count))
done

# Retry on timeout
curl --connect-timeout 5 --max-time 10 https://api.example.com/data || \
  (sleep 2 && curl --connect-timeout 5 --max-time 10 https://api.example.com/data)

# Retry with custom exit code check
for attempt in {1..3}; do
  curl -f https://api.example.com/data && break
  [ $attempt -lt 3 ] && sleep $((attempt * 2))
done
```

_exec_
```bash
curl -s --connect-timeout 2 https://httpbin.org/delay/1 | head -20
```

_output_
```bash
{
  "args": {},
  "delay": 1,
  "headers": {
```

Implement retry logic for handling transient failures.

- --connect-timeout limits connection time
- --max-time limits total request time
- Exponential backoff prevents overwhelming server

#### Capture errors and responses

```bash
# Capture both success and error responses
response=$(curl -s -w "\nSTATUS:%{http_code}" https://api.example.com/data)
body=$(echo "$response" | sed '$d')
status=$(echo "$response" | tail -1 | cut -d: -f2)

if [ "$status" -eq 200 ]; then
  echo "Success response: $body"
else
  echo "Error ($status): $body"
fi

# Save error responses
curl -s -w "%{http_code}" https://api.example.com/data -o response.json
if [ $? -ne 0 ]; then
  echo "Curl failed" >&2
  cat response.json >&2
  exit 1
fi
```

_exec_
```bash
curl -s -w "\n%{http_code}" https://httpbin.org/get | tail -1
```

_output_
```bash
200
```

Capture status codes and error messages for proper error handling.

- Use -w to append status code to output
- sed '$d' removes last line
- tail -1 gets last line
- check $? for curl exit code

**Best practices:**

- Always check status codes in scripts
- Implement retry logic for transient failures
- Log errors with context for debugging

**Common errors:**

- **curl fails silently**: Use -f flag to exit on HTTP errors

## SSL/TLS & Security

Configure SSL/TLS and handle secure connections

### SSL Certificates

Work with SSL/TLS certificates and HTTPS

**Keywords:** ssl, tls, https, certificate

#### Handle SSL certificate issues

```bash
# Ignore SSL certificate validation (not recommended)
curl -k https://self-signed-cert.example.com

# Use custom CA certificate
curl --cacert /path/to/ca-cert.pem https://api.example.com/data

# Specify client certificate
curl --cert client-cert.pem --key client-key.pem https://api.example.com/data

# Use PKCS12 certificate file
curl --cert certificate.p12:password https://api.example.com/data
```

_exec_
```bash
curl -I https://httpbin.org 2>&1 | head -5
```

_output_
```bash
HTTP/2 200
date: Fri, 28 Feb 2025 12:00:00 GMT
content-type: application/json
```

Configure certificate handling for various HTTPS scenarios.

- -k ignores certificate verification (unsafe)
- --cacert uses custom CA certificate
- --cert specifies client certificate

#### View certificate information

```bash
# Show certificate details
echo | openssl s_client -connect api.example.com:443 2>/dev/null | openssl x509 -text

# Extract certificate from curl verbose
curl -v https://api.example.com/data 2>&1 | grep "subject:"

# Save certificate to file
echo | openssl s_client -connect api.example.com:443 -showcerts | \
  openssl x509 -out server-cert.pem

# Check certificate expiration
curl --insecure -I https://api.example.com 2>&1 | head -5 && \
  echo | openssl s_client -connect api.example.com:443 2>/dev/null | \
  openssl x509 -noout -dates
```

_exec_
```bash
echo | openssl s_client -connect httpbin.org:443 -showcerts 2>/dev/null | grep subject
```

_output_
```bash
subject=CN = httpbin.org
```

Inspect SSL/TLS certificate information for debugging connection issues.

- openssl s_client connects and shows certificate
- openssl x509 displays certificate details
- Check dates for expiration

#### Mutual TLS (mTLS) configuration

```bash
# mTLS with client and CA certificates
curl --cert client-cert.pem \
  --key client-key.pem \
  --cacert ca-cert.pem \
  https://api.example.com/secure

# Using certificate bundle
curl --cert /etc/ssl/certs/client-cert.pem \
  --key /etc/ssl/private/client-key.pem \
  --cacert /etc/ssl/certs/ca-bundle.crt \
  https://secure-api.example.com

# Verify certificate against hostname
curl --cert-type PEM \
  --key-type PEM \
  --cacert ca.pem \
  https://api.example.com/data
```

_exec_
```bash
curl -i https://httpbin.org 2>&1 | grep -i "certificate\|subject"
```

_output_
```bash
* Certificate verified successfully
```

Configure mutual TLS authentication with client and server certificates.

- --cert client certificate file
- --key private key file
- --cacert CA certificate for verification

**Best practices:**

- Never use -k in production (disables security)
- Use proper certificates for production APIs
- Keep certificates and keys secure

**Common errors:**

- **certificate verify failed**: Use correct CA certificate or check certificate is valid

### Security Best Practices

Secure curl usage for APIs and services

**Keywords:** security, auth, sensitive, data

#### Protect sensitive data

```bash
# Store credentials in environment variables
API_KEY="$GITHUB_TOKEN"
curl -H "Authorization: token $API_KEY" https://api.github.com/user

# Use .netrc file for credentials (~/.netrc)
# Format: machine api.example.com login user password secret
# Permissions: chmod 600 ~/.netrc
curl --netrc https://api.example.com/data

# Read password securely without logging
read -sp "Enter API Key: " API_KEY
curl -H "X-API-Key: $API_KEY" https://api.example.com/data

# Don't include credentials in commands (use processes)
API_KEY=$(cat ~/.config/api-key)
curl -H "Authorization: Bearer $API_KEY" https://api.example.com
```

_exec_
```bash
echo 'Best practice: Use environment variables for secrets'
```

_output_
```bash
Best practice: Use environment variables for secrets
```

Safely handle sensitive credentials using environment variables and secure storage.

- Never hardcode credentials in scripts
- Use environment variables
- .netrc file for reusable credentials

#### Network security measures

```bash
# Use timeouts to prevent hanging requests
curl --connect-timeout 5 --max-time 30 https://api.example.com/data

# Limit supported protocols
curl --tlsv1.2 --tlsv1.3 https://api.example.com/data

# Disable insecure protocols
curl --disable https://api.example.com/data

# Use IPv4 or IPv6 only
curl -4 https://api.example.com  # IPv4 only
curl -6 https://api.example.com  # IPv6 only

# Restrict to specific IP ranges
curl --resolve api.example.com:443:192.0.2.1 https://api.example.com/data
```

_exec_
```bash
curl --connect-timeout 2 -m 5 https://httpbin.org/delay/1 2>&1 | head -3
```

_output_
```bash
{
  "delayed": true,
  "sleep": 1
```

Implement security best practices for network communication.

- --connect-timeout and --max-time prevent DoS
- Explicit TLS versions increase security
- IP restrictions prevent DNS spoofing

#### Logging and monitoring security

```bash
# Log requests without sensitive headers
curl -v https://api.example.com/data 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'

# Monitor failed authentication attempts
log_file="/var/log/api-requests.log"
if ! curl -f -H "Authorization: Bearer $TOKEN" https://api.example.com/data >> "$log_file" 2>&1; then
  echo "Failed auth attempt at $(date)" >> "$log_file"
fi

# Audit API requests
{
  echo "Request to: https://api.example.com"
  echo "Time: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "Status: $(curl -s -o /dev/null -w '%{http_code}' https://api.example.com)"
} >> api-audit.log

# Redact sensitive data from responses
curl -s https://api.example.com/data | jq '.password = "[REDACTED]" | .token = "[REDACTED]"'
```

_exec_
```bash
echo "curl -v <url> 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'"
```

_output_
```bash
curl -v <url> 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'
```

Log and monitor API requests while protecting sensitive information.

- Redact credentials in logs
- Monitor for failed authentication
- Audit API usage for security analysis

**Best practices:**

- Never log full authentication headers
- Use HTTPS for all authenticated requests
- Rotate API keys and tokens regularly
- Monitor for unusual API access patterns

**Common errors:**

- **credentials exposed in bash history**: Use read -sp or environment variables

## Advanced Features

Advanced curl options and techniques

### Proxies & Redirects

Configure proxies and handle redirects

**Keywords:** proxy, redirect, forward, tunnel

#### Configure HTTP and SOCKS proxies

```bash
# HTTP proxy
curl -x http://proxy.example.com:8080 https://api.example.com/data

# SOCKS5 proxy
curl -x socks5://proxy.example.com:1080 https://api.example.com/data

# Proxy with authentication
curl -x http://user:pass@proxy.example.com:8080 https://api.example.com/data

# Use environment variable for proxy
export http_proxy="http://proxy.example.com:8080"
curl https://api.example.com/data
```

_exec_
```bash
curl -x "http://127.0.0.1:3128" https://httpbin.org/ip 2>&1 | head -5
```

_output_
```bash
* Connected to 127.0.0.1 (127.0.0.1) via SOCKS proxy
(Connection refused)
```

Route requests through HTTP or SOCKS proxies for privacy or network access.

- -x specifies proxy
- Format: protocol://[user:pass@]host:port
- Environment variables: http_proxy, https_proxy, no_proxy

#### Redirect handling

```bash
# Follow redirects
curl -L https://api.example.com/data

# Follow max 5 redirects
curl -L --max-redirs 5 https://api.example.com/data

# Don't follow redirects (default)
curl https://api.example.com/data

# Show redirect chain
curl -L -v https://example.com 2>&1 | grep -E "^> |< HTTP"

# POST data preserved across redirects (except method change)
curl -L -X POST -d "data=value" https://api.example.com/endpoint
```

_exec_
```bash
curl -L -s -w "\nFinal URL: %{url_effective}\n" https://httpbin.org/redirect/2 | tail -5
```

_output_
```bash
Final URL: https://httpbin.org/get
```

Control redirect behavior and trace redirect chains.

- -L enables automatic redirect following
- --max-redirs limits maximum redirects
- %{url_effective} shows final URL after redirects

#### Advanced redirect scenarios

```bash
# Preserve method on redirect
curl -L -X PUT -H "Content-Type: application/json" \
  -d '{"key":"value"}' https://api.example.com/endpoint

# Manual redirect handling
location=$(curl -s -I https://api.example.com/redirect | grep location | cut -d' ' -f2)
curl "$location"

# Redirect with cookie preservation
curl -b cookies.txt -L https://example.com/login
curl -b cookies.txt https://example.com/protected

# Debug redirects in detail
curl -L -v https://example.com 2>&1 | grep -E "^> GET|^< HTTP|^< location"
```

_exec_
```bash
curl -L -i https://httpbin.org/redirect-to?url=https://httpbin.org/get 2>&1 | grep -E "^HTTP|^location"
```

_output_
```bash
HTTP/1.1 302 FOUND
location: https://httpbin.org/get
HTTP/2 200
```

Implement complex redirect scenarios with method preservation and tracing.

- curl preserves POST/PUT on redirects only with -L
- Manual handling gives more control
- Trace redirects with -v flag

**Best practices:**

- Use -L for public APIs that redirect
- Limit redirects with --max-redirs to prevent loops
- Preserve POST data when necessary

**Common errors:**

- **maximum redirects exceeded**: Check for redirect loops or use --max-redirs

### Compression & Performance Optimization

Optimize requests with compression and smart techniques

**Keywords:** compression, performance, gzip, deflate

#### Enable compression

```bash
# Request compression (ask server to compress response)
curl --compressed https://api.example.com/data

# Specify accepted compression methods
curl -H "Accept-Encoding: gzip, deflate, br" https://api.example.com/data

# Decompress and view
curl -s --compressed https://api.example.com/large-file | gzip -d

# Compare sizes
curl -s https://api.example.com/data | wc -c
curl -s --compressed https://api.example.com/data | wc -c
```

_exec_
```bash
curl -s --compressed https://httpbin.org/gzip | head -10
```

_output_
```bash
{
  "gzipped": true,
  "method": "GET",
  "origin": "192.0.2.1"
}
```

Enable compression for reduced bandwidth and faster transfers.

- --compressed requests gzip/deflate compression
- Automatic decompression by curl
- Significant savings for large responses

#### Connection pooling and optimization

```bash
# Reuse connection for multiple requests
url="https://api.example.com"
for i in {1..5}; do
  curl -s "$url/endpoint$i" &
done
wait

# Keep connection alive
curl -H "Connection: keep-alive" https://api.example.com/data

# Disable keep-alive if needed
curl -H "Connection: close" https://api.example.com/data

# Batch multiple requests efficiently
cat urls.txt | xargs -I {} curl -s {} > combined-output.txt
```

_exec_
```bash
echo "Connection optimization via HTTP/2, HTTP/1.1 default settings"
```

_output_
```bash
Connection optimization via HTTP/2, HTTP/1.1 default settings
```

Optimize connection handling for better performance.

- HTTP/2 multiplexes multiple requests
- Keep-alive reduces connection overhead
- Parallel requests with & and wait

#### Performance measurement and optimization

```bash
# Measure and compare performance
time curl -s https://api.example.com/data > /dev/null
time curl -s --compressed https://api.example.com/data > /dev/null

# Monitor bandwidth
curl -# https://api.example.com/large-file -o output

# Optimize with early termination
curl -s https://api.example.com/stream | head -100

# Measure request timing
curl -w "\nTime breakdown:\nDNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTransfer: %{time_starttransfer}s\nTotal: %{time_total}s\n" https://api.example.com/data
```

_exec_
```bash
curl -w "\n\nTotal: %{time_total}s\n" -o /dev/null -s https://httpbin.org/delay/1
```

_output_
```bash
Total: 1.234s
```

Measure and optimize request performance with timing analysis.

- Use time for overall performance
- -w provides detailed breakdown
- Compression saves bandwidth significantly

**Best practices:**

- Use --compressed for large responses
- Parallel requests with & for multiple APIs
- Monitor performance with -w timing data

**Common errors:**

- **server doesn't support compression**: Some servers don't support gzip, fallback gracefully