Section 7: Pipelines and Filters
CONCEPT: UNIX allows you to connect processes, by letting the
standard output of one process feed into the standard input of another
process. That mechanism is called a pipe.
Connecting simple processes with a pipe allows you to perform
complex tasks without writing complex programs.
EXAMPLE: Using the more command, and a pipe, send the contents
of your .rhosts and .cshrc files to the screen by typing
- cat .rhosts .cshrc | more
to the shell.
EXERCISE: How could you use head and tail with a pipe to display lines 25 through 75 of a file?
ANSWER: The command
- cat file | head -75 | tail
-50
would work. The cat command feeds the file into the
pipe. The head command gets the first 75 lines of the file, and
passes them down the pipe to tail. The tail command then filters out all but the last 50 lines of the input it received from head. It is important to note that in the above example, tail never sees the original file, but only sees the part of the file that was passed to
it by the head command.
It is easy for beginners to confuse the usage of the input/output
redirection symbols < and >, with the usage of the pipe.
Remember that input/output redirection connects processes with
files, while the pipe connects processes with other
processes.
Grep
The grep (global regular expression and print) utility is one of the most useful filters in UNIX. The grep utility searches line-by-line for a specified pattern, and outputs any line
that matches the pattern. The basic syntax for the grep command is
grep [-options] pattern [file]. If the file argument is
omitted, grep will read from standard input. It is always best to
enclose the pattern within single quotes, to prevent the shell from
misinterpreting the command.
The grep utility recognizes a variety of patterns. Here are some of the characters you can use to build grep expressions:
- The carat (^) matches the beginning of a line.
- The dollar sign ($) matches the end of a line.
- The period (.) matches any single character.
- The asterisk (*) matches zero or more occurrences of the previous
character.
- The expression [a-b] matches any characters that are
lexically between a and b.
EXAMPLE: Type the command
- grep "usr"
/.cshrc
to search the /.cshrc file for any lines
containing the string "usr".
EXAMPLE: Type the command
- grep "^alias"
/.cshrc
to see the lines in /.cshrc that begin with
the character string "alias".
EXERCISE:List all the files in the /tmp directory owned by the
user root.
EXPLANATION: The command
- ls -l /tmp | grep
'root'
would show all file listings with the word "root"
somewhere in the line. That doesn't necessarily mean that all the
files would be owned by root, but using the grep filter can cut
the down the number of files you will have to look at.
EXERCISE:List all the processes owned by the user root.
EXPLANATION: The command
- ps -eaf | grep 'root'
would show all processes with the word 'root' somewhere in the line. Once
again this doesn't necessarily mean that the process would be owned by root,
but it does narrow down the possibilities.