Redirection, pipeline and filter
● Standard input, output and error
● Redirection
● Pipeline
● Filter utilities
Version 1.0 [Link]
Standard input, output, error
standard input standard output
process
(stdin) (stdout)
stdin is an input for process is a running stdout is an output
a process which is linux command line that shows up on
typed from keyboard the screen
standard error
(stderr)
stderr is an error
message that shows
up on the screen
Version 1.0 [Link]
Standar output (stdout)
Let's say we executed a command:
$ ls process
Maildir public_html [Link] stdout
$ whoami proses
joni stdout
$ cat /etc/passwd process
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh stdout
sys:x:3:3:sys:/dev:/bin/sh
...
Version 1.0 [Link]
Standar error (stderr)
Let's say we executed another command:
$ hello process
bash: hello: command not found stderr
$ pure-ftpd process
The program 'pure-ftpd' is currently not
installed. You can install it by typing:
sudo apt-get install pure-ftpd stderr
bash: pure-ftpd: command not found
Version 1.0 [Link]
Standar input (stdin)
Let's say we executed another command:
$ cat process
Hallo world... stdin (input from keyboard)
Hallo world... stdout (output shows up on the screen)
Ctrl+D EOF (end of file)
Version 1.0 [Link]
Redirection
normal redirection symbol
stdin from keyboard from file <
stdout to screen to file > or 1>
stderr to screen to file 2>
Version 1.0 [Link]
Redirection stdin
Without redirection:
$ cat process
Hallo world... stdin (input from keyboard)
Hallo world... stdout (output shows up on the screen)
With redirection stdin (input from file, not from keyboard):
process redirection stdin
$ cat < /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh stdout
sys:x:3:3:sys:/dev:/bin/sh
...
The command above is the same with: $ cat /etc/passwd
Version 1.0 [Link]
Redirection stdout
Redirection stdin (input from file, not from keyboard):
process redirection stdin
$ cat < /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh stdout
bin:x:2:2:bin:/bin:/bin/sh
Redirection stdout (output to file, not to screen):
process redirection stdout
QUIZ
$ cat > ~/[Link] Try following command:
Hallo world... $ cat >> ~/[Link]
stdin (type some lines and press Ctrl+D)
How're you? Bye.. :)
Display the content. What happen?
Ctrl+D EOF (end of file)
To display the content of file, type: $ cat [Link]
Version 1.0 [Link]
Redirection stdout usage
Redirection stdout generally used for capturing and
documenting command's results:
$ ls -l > [Link]
$ cat [Link]
$ ifconfig > [Link]
$ cat [Link]
QUIZ
Is the command below valid? What it uses for?
$ cat < [Link] > [Link]
Version 1.0 [Link]
Homework
Using redirection technique, assemble commands that will
create a file which is recursively increasing its own size
infinitely. It won't stop until terminated by user, or the hard disk
is full.
Version 1.0 [Link]
Redirection stderr
Without redirection stderr:
$ hallo process
bash: hallo: command not found stderr
With redirection stderr:
redirection stderr
$ hallo 2> [Link]
Notice that the error message didn't show up on the screen,
but redirected and saved in file named [Link]. To see:
$ cat [Link]
bash: hallo: command not found
Version 1.0 [Link]
Redirection stdout and stderr
$ find / -name xyz > [Link] 2>&1
The meaning of command above is:
● Search results will be redirected in file named [Link]
● If any error messages show up, it will also redirected to the same
file [Link]
Version 1.0 [Link]
Pipeline
Pipeline is a technique for “flowing” a command's output as
an input of another command.
process1 output1 = input2 process2 output2 ...
In command line:
$ process1 | process2 | process3 | ...
Command that able to accept an input and
produce an output at the same time called filter
Pipeline example:
$ cat /var/log/messages | more
$ ls /etc | grep passwd
$ dpkg -l | grep apache
Version 1.0 [Link]
Filter utilities
● more ● paste
● less ● split
● grep ● sort
● head ● tr
● tail ● wc
● cut ● pr
Version 1.0 [Link]
more
more is a filter for paging through text one screenful at a time.
For navigations use enter to scroll per line, or space to scroll
per screen.
$ cat /var/log/messages
$ more /var/log/messages
$ cat /var/log/messages | more
Version 1.0 [Link]
less
less is like more, but besides “standard” enter and space
navigations, less is also have additional navigations like up
arrow, down arrow, Page Up, and Page Down.
$ cat /var/log/messages
$ less /var/log/messages
$ cat /var/log/messages | less
Version 1.0 [Link]
grep
grep searches lines containing a match to the given pattern. For
example if we want to search all lines containing “root”:
$ grep root /etc/passwd
$ cat /etc/passwd | grep root
To search all lines, NOT containing “root”:
$ grep -v root /etc/passwd
To count how many lines matched:
$ grep -c root /etc/passwd
Search with case sensitive:
$ grep -i ROOT /etc/passwd
Version 1.0 [Link]
head and tail
head: Print the first 10 lines of each FILE to standard output:
$ head -5 /etc/passwd
$ cat /etc/passwd | head -2
$ cat -n /etc/passwd | head -3
tail: Print the last 10 lines of each FILE to standard output:
$ tail -5 /etc/passwd
$ cat /etc/passwd | head -2
$ cat -n /etc/passwd | head -3
QUIZ
Using head, tail and pipeline,
assemble commands for
printing only the third row.
Version 1.0 [Link]
cut
cut is an utility to print selected column with a specific delimiter.
For example /etc/passwd filled with columns separated by “ : ”.
How many columns are in /etc/passwd?
$ cat /etc/passwd
To print the first column:
$ cut -d: -f1 /etc/passwd
To print the first and the fifth column:
$ cut -d: -f1,5 /etc/passwd
To print the first to the fifth column:
$ cut -d: -f1-5 /etc/passwd
Version 1.0 [Link]
paste
paste is an utility for merging columns and print it to standard
output.
Take the first column and save it:
$ cut -d: -f1 /etc/passwd > [Link]
Take the second column and save it:
$ cut -d: -f2 /etc/passwd > [Link]
Merge the two files above with paste:
$ paste [Link] [Link]
Merge with delimiter “ : “
$ paste -d: [Link] [Link]
Version 1.0 [Link]
split
split is an utility for splitting a file into pieces with certain size.
Split /etc/passwd into files each contain 10 rows:
$ cat -n /etc/passwd | split -10
$ split -10 /etc/passwd
The results are files with prefix “xa”: xaa, xab, xac:
$ cat xaa
If we want different prefix like “xxx”:
$ split -10 /etc/passwd xxx
To merge again back to original:
$ cat xxx* > [Link]
Version 1.0 [Link]
sort
sort is a utility for sorting text.
Take field user name from /etc/passwd:
$ cat /etc/passwd | cut -d: -f1
Then sort ascending:
$ cat /etc/passwd | cut -d: -f1 | sort
Then sort descending:
$ cat /etc/passwd | cut -d: -f1 | sort -r
To sort numerically (third field: uid):
$ cat /etc/passwd | cut -d: -f3 | sort -n
Version 1.0 [Link]
tr
tr (translate) is an utility for translate or delete characters.
Translate character “a” into “o”:
$ tr “a” “o”
Hallo
Hollo
Ctrl+D
Translate file's contents into capital:
$ cat /etc/passwd | tr “a-z” “A-Z”
Version 1.0 [Link]
wc
wc (word count) is an utility for printing newline, word, and byte
counts for each file.
To count lines from /etc/passwd:
$ cat /etc/passwd | wc -l
To count words from /etc/passwd:
$ cat /etc/passwd | wc -w
To count characters from /etc/passwd:
$ cat /etc/passwd | wc -c
To count files in your home directory:
$ ls ~ | wc -l
Version 1.0 [Link]
pr
pr is a utility for converting text files for printing.
To format /etc/passwd:
$ cat /etc/passwd | pr
To format /etc/passwd with header:
$ cat /etc/passwd | pr -h “File /etc/passwd”
QUIZ: Try to format /var/log/messages for printing and
redirect it to ~/[Link]
Version 1.0 [Link]
