Working with Pipes and Tee Command

Summary: Use | and tee to process and capture output.

Working in the command-line environment opens up powerful ways to process data efficiently. If you've written or reviewed shell scripts, or ever needed to combine several command-line utilities, you've probably used the pipe (|) operator. The tee command is another invaluable tool that lets you capture output while continuing to process it. In this article, we'll explore how the pipe works, how tee fits in, and plenty of practical examples to make your everyday shell work more productive.

Understanding Pipes (`|`)

The pipe (|) is a fundamental feature in Unix-like shells (such as Bash, Zsh, or Sh) that allows you to redirect the output of one command into the input of another. Rather than writing output to the console, you can send it directly to a second command for further processing.

Syntax

command1 | command2

command1 runs, and its standard output (stdout) goes straight into
command2 as standard input (stdin).

Simple Example

Suppose you wish to see only the five most recently-modified files in a directory:

ls -lt | head -n 5

ls -lt lists files sorted by modification time.
head -n 5 takes the first five lines of output.

The `tee` Command

The tee command is named for a T-splitter in plumbing. It splits standard input into both standard output and one or more files. This means you can see output on your terminal (stdout) and capture it into a file at the same time.

Basic Syntax

command | tee [OPTION]... [FILE]...

By default, tee overwrites the files you specify.
Use -a to append instead of overwrite (e.g., tee -a log.txt).

Why Use `tee`?

Log what you see: Capture output for records, review, or debugging without hiding it from your console.
Intermediate storage: Save data for inspection between processing steps.

Practical Examples

Let's tie it all together with some real-world scenarios.

1. Save and View Output Simultaneously

df -h | tee disk_usage.txt

Shows disk space usage and saves it to disk_usage.txt.

2. Chain Multiple Pipes with `tee`

Suppose you want to filter logs, save the filtered portion, then perform more processing:

cat /var/log/syslog | grep "error" | tee errors.txt | sort | uniq -c

Extract "error" lines.
Save them to errors.txt.
Sort them and count unique occurrences.

3. Appending With `tee -a`

If you want to build a log over time:

echo "$(date): System update check" | tee -a update.log
sudo apt update | tee -a update.log

Each execution appends output, keeping a complete log.

4. Capture Both stdout and stderr

By default, | only sends stdout. To include stderr:

myscript.sh 2>&1 | tee output.log

2>&1 redirects stderr (file descriptor 2) to stdout (1).

5. Use `tee` with Multiple Files

You can provide multiple output files:

process_data | tee output1.txt output2.txt

Output will be written to both output1.txt and output2.txt as well as the terminal.

Tips and Gotchas

Appending: Use tee -a to prevent accidental overwrites.
Piping to sudo: Sometimes, you’ll need to run tee with sudo to write to protected files. Example:
```
echo "new config line" | sudo tee -a /etc/someconfig.conf
```
Silencing Output: Add > /dev/null after tee if you don't want to see the output on your screen.
Use in scripts: tee is script-friendly for creating logs of each run.

Conclusion

The simple mechanics of the pipe (|) and the versatility of tee can make a big difference to your command-line efficiency. Whether you’re logging output, saving intermediate data, or debugging, mastering these tools will make your workflows smoother and more robust. Try incorporating tee the next time you're chaining commands together — capturing what you need and seeing it right on your screen!

Further reading: