Introduction to Unix commands

cal
This command will print a calendar for a specified month and/or year.

To show this month's calendar, enter:

cal

To show a twelve-month calendar for 2008, enter:

cal 2008

To show a calendar for just the month of June 1970, enter:

cal 6 1970

cat
This command outputs the contents of a text file. You can use it to read brief files or to concatenate files together.

To append file1 onto the end of file2, enter:

cat file1 >> file2

To view the contents of a file named myfile, enter:

cat myfile

Because cat displays text without pausing, its output may quickly scroll off your screen. Use the less command
(described below) or an editor for reading longer text files.

For more, see In Unix, how do I combine several text files into a single file?

cd
This command changes your current directory location. By default, your Unix login session begins in your home
directory.

To switch to a subdirectory (of the current directory) named myfiles, enter:

cd myfiles

To switch to a directory named /home/dvader/empire_docs, enter:

cd /home/dvader/empire_docs

To move to the parent directory of the current directory, enter:

cd ..
To move to the root directory, enter:

cd /

To return to your home directory, enter:

cd

chmod
This command changes the permission information associated with a file. Every file (including directories, which Unix
treats as files) on a Unix system is stored with records indicating who has permission to read, write, or execute the file,
abbreviated as r, w, and x. These permissions are broken down for three categories of user: first, the owner of the file;
second, a group with which both the user and the file may be associated; and third, all other users. These categories are
abbreviated as u for owner (or user), g for group, and o for other.

To allow yourself to execute a file that you own named myfile, enter:

chmod u+x myfile

To allow anyone who has access to the directory in which myfile is stored to read or execute myfile, enter:

chmod o+rx myfile

You can view the permission settings of a file using the ls command, described below.

Note: Be careful with the chmod command. If you tamper with the directory permissions of your home directory, for
example, you could lock yourself out or allow others unrestricted access to your account and its contents.

cp
This command copies a file, preserving the original and creating an identical copy. If you already have a file with the
new name, cp will overwrite and destroy the duplicate. For this reason, it's safest to always add -i after the cp command,
to force the system to ask for your approval before it destroys any files. The general syntax for cp is:

cp -i oldfile newfile

To copy a file named meeting1 in the directory /home/dvader/notes to your current directory, enter:

cp -i /home/dvader/notes/meeting1 .

The . (period) indicates the current directory as destination, and the -i ensures that if there is another file named
meeting1 in the current directory, you will not overwrite it by accident.

To copy a file named oldfile in the current directory to the new name newfile in the mystuff subdirectory of your
home directory, enter:

cp -i oldfile ~/mystuff/newfile

The ~ character (tilde) is interpreted as the path of your home directory.

Note: You must have permission to read a file in order to copy it.

date
The date command displays the current day, date, time, and year.

To see this information, enter:

date

df
This command reports file system disk usage (i.e., the amount of space taken up on mounted file systems). For each
mounted file system, df reports the file system device, the number of blocks used, the number of blocks available, and
the directory where the file system is mounted.

To find out how much disk space is used on each file system, enter the following command:

df

If the df command is not configured to show blocks in kilobytes by default, you can issue the following command:

df -k

du
This command reports disk usage (i.e., the amount of space taken up by a group of files). The du command descends all
subdirectories from the directory in which you enter the command, reporting the size of their contents, and finally
reporting a total size for all the files it finds.

To find out how much disk space your files take up, switch to your home directory with the cd command, and enter:

du

The numbers reported are the sizes of the files; on different systems, these sizes will be in units of either 512 byte blocks
or kilobytes. To learn which is the case, use the man command, described below. On most systems, du -k will give sizes
in kilobytes.

find
The find command lists all of the files within a directory and its subdirectories that match a set of conditions. This
command is most commonly used to find all of the files that have a certain name.

To find all of the files named myfile.txt in your current directory and all of its subdirectories, enter:

find . -name myfile.txt -print

To look in your current directory and its subdirectories for all of the files that end in the extension .txt, enter:

find . -name "*.txt" -print

In these examples, the . (period) represents your current directory. It can be replaced by the full pathname of another
directory to search. For instance, to search for files named myfile.txt in the directory /home/user/myusername and its
subdirectories, enter:

find /home/user/myusername/ -name myfile.txt -print

On some systems, omitting the final / (slash) after the directory name can cause find to fail to return any results.
As a shortcut for searching in your home directory, enter:

find "$HOME/" -name myfile.txt -print

jobs
This command reports any programs that you suspended and still have running or waiting in the background (if you had
pressed Ctrl-z to suspend an editing session, for example). For a list of suspended jobs, enter:

jobs

Each job will be listed with a number; to resume a job, enter % (percent sign) followed by the number of the job. To
restart job number two, for example, enter:

%2

This command is only available in the csh, bash, tcsh, and ksh shells.

kill
Use this command as a last resort to destroy any jobs or programs that you suspended and are unable to restart. Use the
jobs command to see a list of suspended jobs. To kill suspended job number three, for example, enter:

kill %3

Now check the jobs command again. If the job has not been cancelled, harsher measures may be necessary. Enter:

kill -9 %3

less and more

Both less and more display the contents of a file one screen at a time, waiting for you to press the Spacebar between
screens. This lets you read text without it scrolling quickly off your screen. The less utility is generally more flexible
and powerful than more, but more is available on all Unix systems while less may not be.

To read the contents of a file named textfile in the current directory, enter:

less textfile

The less utility is often used for reading the output of other commands. For example, to read the output of the ls
command one screen at a time, enter:

ls -la | less

In both examples, you could substitute more for less with similar results. To exit either less or more, press q. To exit
less after viewing the file, press q.

Note: Do not use less or more with executables (binary files), such as output files produced by compilers. Doing so will
display garbage and may lock up your terminal.

lpr and lp
These commands print a file on a printer connected to the computer network. The lpr command is used on BSD
systems, and the lp command is used in System V. Both commands may be used on the UITS systems.

To print a file named myfile on a printer named lp1 with lpr, enter:

lpr -Plp1 myfile

To print the same file to the same printer with lp, enter:

lp -dlp1 myfile

Note: Do not print to a printer whose name or location is unfamiliar to you.

ls
This command will list the files stored in a directory. To see a brief, multi-column list of the files in the current directory,
enter:

ls

To also see "dot" files (configuration files that begin with a period, such as .login), enter:

ls -a

To see the file permissions, owners, and sizes of all files, enter:

ls -la

If the listing is long and scrolls off your screen before you can read it, combine ls with the less utility, for example:

ls -la | less

man
This command displays the manual page for a particular command. If you are unsure how to use a command or want to
find out all its options, you might want to try using man to view the manual page.

For example, to learn more about the ls command, enter:

man ls

To learn more about man, enter:

man man

If you are not sure of the exact command name, you can use man with the -k option to help you find the command you
need. To see one line summaries of each reference page that contains the keyword you specify, enter:

man -k keyword

Replace keyword in the above example with the keyword which you want to reference. Also see Use the Unix man
command to read manual pages.

mkdir
This command will make a new subdirectory.

To create a subdirectory named mystuff in the current directory, enter:

mkdir mystuff

To create a subdirectory named morestuff in the existing directory named /tmp, enter:

mkdir /tmp/morestuff

Note: To make a subdirectory in a particular directory, you must have permission to write to that directory.

mv
This command will move a file. You can use mv not only to change the directory location of a file, but also to rename
files. Unlike the cp command, mv will not preserve the original file.

Note: As with the cp command, you should always use -i to make sure you do not overwrite an existing file.

To rename a file named oldname in the current directory to the new name newname, enter:

mv -i oldname newname

To move a file named hw1 from a subdirectory named newhw to another subdirectory named oldhw (both subdirectories
of the current directory), enter:

mv -i newhw/hw1 oldhw

If, in this last operation, you also wanted to give the file a new name, such as firsthw, you would enter:

mv -i newhw/hw1 oldhw/firsthw

ps
The ps command displays information about programs (i.e., processes) that are currently running. Entered without
arguments, it lists basic information about interactive processes you own. However, it also has many options for
determining what processes to display, as well as the amount of information about each. Like lp and lpr, the options
available differ between BSD and System V implementations. For example, to view detailed information about all
running processes, in a BSD system, you would use ps with the following arguments:

ps -alxww

To display similar information in System V, use the arguments:

ps -elf

For more information about ps refer to the ps man page on your system.

pwd
This command reports the current directory path. Enter the command by itself:

pwd
rm
This command will remove (destroy) a file. You should enter this command with the -i option, so that you'll be asked to
confirm each file deletion. To remove a file named junk, enter:

rm -i junk

Note: Using rm will remove a file permanently, so be sure you really want to delete a file before you use rm.

To remove a non-empty subdirectory, rm accepts the -r option. On most systems this will prompt you to confirm the
removal of each file. This behavior can be prevented by adding the -f option. To remove an entire subdirectory named
oldstuff and all of its contents, enter:

rm -rf oldstuff

Note: Using this command will cause rm to descend into each subdirectory within the specified subdirectory and remove
all files without prompting you. Use this command with caution, as it is very easy to accidently delete important files. As
a precaution, use the ls command to list the files within the subdirectory you wish to remove. To browse through a
subdirectory named oldstuff, enter:

ls -R oldstuff | less

rmdir
This command will remove a subdirectory. To remove a subdirectory named oldstuff, enter:

rmdir oldstuff

Note: The directory you specify for removal must be empty. To clean it out, switch to the directory and use the ls and
rm commands to inspect and delete files.

set
This command displays or changes various settings and options associated with your Unix session.

To see the status of all settings, enter the command without options:

set

If the output scrolls off your screen, combine set with less:

set | less

The syntax used for changing settings is different for the various kinds of Unix shells; see the man entries for set and the
references listed at the end of this document for more information.

vi
This command starts the vi text editor. To edit a file named myfile in the current directory, enter:

vi myfile
The vi editor works fairly differently from other text editors. If you have not used it before, you should probably look at
a tutorial, such as Use the vi text editor.

The very least you need to know to start using vi is that in order to enter text, you need to switch the program from
command mode to insert mode by pressing i. To navigate around the document with the cursor keys, you must switch
back to command mode by pressing Esc. To execute any of the following commands, you must switch from command
mode to ex mode by pressing : (the colon key): Enter w to save; wq to save and quit; q! to quit without saving.

w and who
The w and who commands are similar programs that list all users logged into the computer. If you use w, you also get a
list of what they are doing. If you use who, you also get the IP numbers or computer names of the terminals they are
using.

Unix Commands With Examples:

1. Listing files

The first thing after logging into the unix system, everyone does is listing the files in a directory. The ls command is used
to list the files in a directory.
>ls

add.sh
logfile.txt
prime.pl

If you simply execute ls on the command prompt, then it will display the files and directories in the current directory.
>ls /usr/local/bin

You can pass a directory as an argument to ls command. In this case, the ls command prints all the files and directories in
the specific directory you have passed.

2. Displaying the contents of a file.

The next thing is to display the contents of a file. The cat command is used to display the contents in a file.
>cat file.txt
This is a sample unix file
Learning about unix server is awesome

3. Displaying first few lines from a file.

The head command can be used to print the specified number of lines from the starting of a file. The below head
command displays the first five lines of file.
>head -5 logfile.dat

4. Displaying last few lines from a file.

The tail command can be used to print the specified number of lines from the ending of a file. The below tail command
displays the last three lines of file.
>tail -3 logfile.dat

5. Changing the directories

The cd command can be used to change from one directory to another directory. You need to specify the target directory
where you want to go.
>cd /var/tmp

After typing this cd command you will be in /var/tmp directory.

6. Creating a file.

The touch command simply creates an empty file. The below touch command creates a new file in the current directory.
touch new_file.txt

7. copying the contents of one file into another.

The cp command is used to copy the content of source file into the target file. If the target file already have data, then it
will be overwritten.
>cp source_file target_file

8. Creating a directory.

Directories are a way of organizing your files. The mkdir command is used to create the specified directory.
>mkdir backup

This will create the backup directory in the current directory.

9. Renaming and moving the files.

The mv command is used to rename the files and it also used for moving the files from one directory into another
directory.
Renaming the file.

>mv file.txt new_file.txt

Moving the file to another directory.

>mv new_file.txt tmp/

10. Finding the number of lines in a file

The wc command can be used to find the number of line, words and characters in a file.
>wc logfile.txt
21 26 198 logfile.txt

To know about the unix command, it is always good to see the man pages. To see the man pages simply pass the
command as an argument to the man.
man ls

Top Unix Interview Questions - Part 1

1. How to display the 10th line of a file?
head -10 filename | tail -1

2. How to remove the header from a file?

sed -i '1 d' filename

3. How to remove the footer from a file?

sed -i '$ d' filename

4. Write a command to find the length of a line in a file?

The below command can be used to get a line from a file.

sed –n '<n> p' filename

We will see how to find the length of 10th line in a file

sed -n '10 p' filename|wc -c

5. How to get the nth word of a line in Unix?

cut –f<n> -d' '

6. How to reverse a string in unix?

echo "java" | rev

7. How to get the last word from a line in Unix file?

echo "unix is good" | rev | cut -f1 -d' ' | rev

8. How to replace the n-th line in a file with a new line in Unix?
sed -i'' '10 d' filename # d stands for delete
sed -i'' '10 i new inserted line' filename # i stands for insert

9. How to check if the last command was successful in Unix?

echo $?

10. Write command to list all the links from a directory?

ls -lrt | grep "^l"

11. How will you find which operating system your system is running on in UNIX?
uname -a

12. Create a read-only file in your home directory?

touch file; chmod 400 file

13. How do you see command line history in UNIX?

The 'history' command can be used to get the list of commands that we are executed.

14. How to display the first 20 lines of a file?

By default, the head command displays the first 10 lines from a file. If we change the option of head, then we can display
as many lines as we want.
head -20 filename

An alternative solution is using the sed command

sed '21,$ d' filename

The d option here deletes the lines from 21 to the end of the file

15. Write a command to print the last line of a file?

The tail command can be used to display the last lines from a file.
tail -1 filename

Alternative solutions are:

sed -n '$ p' filename
awk 'END{print $0}' filename

Top Unix Interview Questions - Part 2

1. How do you rename the files in a directory with _new as suffix?

ls -lrt|grep '^-'| awk '{print "mv "$9" "$9".new"}' | sh

2. Write a command to convert a string from lower case to upper case?

echo "apple" | tr [a-z] [A-Z]

3. Write a command to convert a string to Initcap.

echo apple | awk '{print toupper(substr($1,1,1)) tolower(substr($1,2))}'

4. Write a command to redirect the output of date command to multiple files?

The tee command writes the output to multiple files and also displays the output on the terminal.
date | tee -a file1 file2 file3

5. How do you list the hidden files in current directory?

ls -a | grep '^\.'

6. List out some of the Hot Keys available in bash shell?

• Ctrl+l - Clears the Screen.

• Ctrl+r - Does a search in previously given commands in shell.
• Ctrl+u - Clears the typing before the hotkey.
• Ctrl+a - Places cursor at the beginning of the command at shell.
• Ctrl+e - Places cursor at the end of the command at shell.
• Ctrl+d - Kills the shell.
• Ctrl+z - Places the currently running process into background.

7. How do you make an existing file empty?

cat /dev/null > filename

8. How do you remove the first number on 10th line in file?

sed '10 s/[0-9][0-9]*//' < filename

9. What is the difference between join -v and join -a?

join -v : outputs only matched lines between two files.
join -a : In addition to the matched lines, this will output unmatched lines also.

10. How do you display from the 5th character to the end of the line from a file?
cut -c 5- filename

Top Unix Interview Questions - Part 3

1. Display all the files in current directory sorted by size?

ls -l | grep '^-' | awk '{print $5,$9}' |sort -n|awk '{print $2}'

2. Write a command to search for the file 'map' in the current directory?
find -name map -type f

3. How to display the first 10 characters from each line of a file?

cut -c -10 filename

4. Write a command to remove the first number on all lines that start with "@"?
sed '\,^@, s/[0-9][0-9]*//' < filename

5. How to print the file names in a directory that has the word "term"?
grep -l term *

The '-l' option make the grep command to print only the filename without printing the content of the file. As soon as the
grep command finds the pattern in a file, it prints the pattern and stops searching other lines in the file.

6. How to run awk command specified in a file?

awk -f filename

7. How do you display the calendar for the month march in the year 1985?

The cal command can be used to display the current month calendar. You can pass the month and year as arguments to
display the required year, month combination calendar.
cal 03 1985

This will display the calendar for the March month and year 1985.

8. Write a command to find the total number of lines in a file?

wc -l filename

Other ways to print the total number of lines are

awk 'BEGIN {sum=0} {sum=sum+1} END {print sum}' filename
awk 'END{print NR}' filename

9. How to duplicate empty lines in a file?

sed '/^$/ p' < filename

10. Explain iostat, vmstat and netstat?

• Iostat: reports on terminal, disk and tape I/O activity.

• Vmstat: reports on virtual memory statistics for processes, disk, tape and CPU activity.
• Netstat: reports on the contents of network data structures.

Top Unix Interview Questions - Part 4

1. How do you write the contents of 3 files into a single file?

cat file1 file2 file3 > file

2. How to display the fields in a text file in reverse order?

awk 'BEGIN {ORS=""} { for(i=NF;i>0;i--) print $i," "; print "\n"}' filename

3. Write a command to find the sum of bytes (size of file) of all files in a directory.
ls -l | grep '^-'| awk 'BEGIN {sum=0} {sum = sum + $5} END {print sum}'

4. Write a command to print the lines which end with the word "end"?
grep 'end$' filename
The '$' symbol specifies the grep command to search for the pattern at the end of the line.

5. Write a command to select only those lines containing "july" as a whole word?
grep -w july filename

The '-w' option makes the grep command to search for exact whole words. If the specified pattern is found in a string,
then it is not considered as a whole word. For example: In the string "mikejulymak", the pattern "july" is found. However
"july" is not a whole word in that string.

6. How to remove the first 10 lines from a file?

sed '1,10 d' < filename

7. Write a command to duplicate each line in a file?

sed 'p' < filename

8. How to extract the username from 'who am i' comamnd?

who am i | cut -f1 -d' '

9. Write a command to list the files in '/usr' directory that start with 'ch' and then display the number of lines in each file?
wc -l /usr/ch*

Another way is
find /usr -name 'ch*' -type f -exec wc -l {} \;

10. How to remove blank lines in a file ?

grep -v ‘^$’ filename > new_filename

Top Unix Interview Questions - Part 5

1. How to display the processes that were run by your user name ?

ps -aef | grep <user_name>

2. Write a command to display all the files recursively with path under current directory?
find . -depth -print

3. Display zero byte size files in the current directory?

find -size 0 -type f

4. Write a command to display the third and fifth character from each line of a file?
cut -c 3,5 filename

5. Write a command to print the fields from 10th to the end of the line. The fields in the line are delimited by a comma?
cut -d',' -f10- filename

6. How to replace the word "Gun" with "Pen" in the first 100 lines of a file?
sed '1,00 s/Gun/Pen/' < filename

7. Write a Unix command to display the lines in a file that do not contain the word "RAM"?
grep -v RAM filename

The '-v' option tells the grep to print the lines that do not contain the specified pattern.

8. How to print the squares of numbers from 1 to 10 using awk command

awk 'BEGIN { for(i=1;i<=10;i++) {print "square of",i,"is",i*i;}}'

9. Write a command to display the files in the directory by file size?

ls -l | grep '^-' |sort -nr -k 5

10. How to find out the usage of the CPU by the processes?

The top utility can be used to display the CPU usage by the processes.

Top Unix Interview Questions - Part 4

1. How do you write the contents of 3 files into a single file?

The basename utility deletes any prefix ending in /. The usage is mentioned below:
basename /usr/local/bin/file

This will display only file

2. How to display zero byte size files?

ls -l | grep '^-' | awk '/^-/ {if ($5 !=0 ) print $9 }'

3. How to replace the second occurrence of the word "bat" with "ball" in a file?
sed 's/bat/ball/2' < filename

4. How to remove all the occurrences of the word "jhon" except the first one in a line with in the entire file?
sed 's/jhon//2g' < filename

5. How to replace the word "lite" with "light" from 100th line to last line in a file?
sed '100,$ s/lite/light/' < filename

6. How to list the files that are accessed 5 days ago in the current directory?
find -atime 5 -type f

7. How to list the files that were modified 5 days ago in the current directory?
find -mtime 5 -type f

8. How to list the files whose status is changed 5 days ago in the current directory?
find -ctime 5 -type f

9. How to replace the character '/' with ',' in a file?

sed 's/\//,/' < filename
sed 's|/|,|' < filename

10. Write a command to find the number of files in a directory.

ls -l|grep '^-'|wc -l

Top Unix Interview Questions - Part 7

1. Write a command to display your name 100 times.

The Yes utility can be used to repeatedly output a line with the specified string or 'y'.
yes <your_name> | head -100

2. Write a command to display the first 10 characters from each line of a file?
cut -c -10 filename

3. The fields in each line are delimited by comma. Write a command to display third field from each line of a file?
cut -d',' -f2 filename

4. Write a command to print the fields from 10 to 20 from each line of a file?
cut -d',' -f10-20 filename

5. Write a command to print the first 5 fields from each line?

cut -d',' -f-5 filename

6. By default the cut command displays the entire line if there is no delimiter in it. Which cut option is used to suppress
these kind of lines?

The -s option is used to suppress the lines that do not contain the delimiter.

7. Write a command to replace the word "bad" with "good" in file?

sed s/bad/good/ < filename

8. Write a command to replace the word "bad" with "good" globally in a file?
sed s/bad/good/g < filename

9. Write a command to replace the word "apple" with "(apple)" in a file?

sed s/apple/(&)/ < filename

10. Write a command to switch the two consecutive words "apple" and "mango" in a file?
sed 's/\(apple\) \(mango\)/\2 \1/' < filename

11. Write a command to display the characters from 10 to 20 from each line of a file?
cut -c 10-20 filename

Top Unix Interview Questions - Part 8

1. Write a command to print the lines that has the the pattern "july" in all the files in a particular directory?

grep july *

This will print all the lines in all files that contain the word “july” along with the file name. If any of the files contain
words like "JULY" or "July", the above command would not print those lines.

2. Write a command to print the lines that has the word "july" in all the files in a directory and also suppress the file
name in the output.
grep -h july *

3. Write a command to print the lines that has the word "july" while ignoring the case.
grep -i july *

The option i make the grep command to treat the pattern as case insensitive.

4. When you use a single file as input to the grep command to search for a pattern, it won't print the filename in the
output. Now write a grep command to print the file name in the output without using the '-H' option.
grep pattern file name /dev/null

The /dev/null or null device is special file that discards the data written to it. So, the /dev/null is always an empty file.
Another way to print the file name is using the '-H' option. The grep command for this is
grep -H pattern filename

5. Write a command to print the file names in a directory that does not contain the word "july"?
grep -L july *

The '-L' option makes the grep command to print the file names that do not contain the specified pattern.

6. Write a command to print the line numbers along with the line that has the word "july"?
grep -n july filename

The '-n' option is used to print the line numbers in a file. The line numbers start from 1

7. Write a command to print the lines that starts with the word "start"?
grep '^start' filename

The '^' symbol specifies the grep command to search for the pattern at the start of the line.

8. In the text file, some lines are delimited by colon and some are delimited by space. Write a command to print the third
field of each line.
awk '{ if( $0 ~ /:/ ) { FS=":"; } else { FS =" "; } print $3 }' filename

9. Write a command to print the line number before each line?

awk '{print NR, $0}' filename

10. Write a command to print the second and third line of a file without using NR.
awk 'BEGIN {RS="";FS="\n"} {print $2,$3}' filename

11. How to create an alias for the complex command and remove the alias?

The alias utility is used to create the alias for a command. The below command creates alias for ps -aef command.
alias pg='ps -aef'

If you use pg, it will work the same way as ps -aef.

To remove the alias simply use the unalias command as

unalias pg

12. Write a command to display today's date in the format of 'yyyy-mm-dd'?

The date command can be used to display today's date with time
date '+%Y-%m-%d'

Top Examples of Awk Command in Unix

Awk is one of the most powerful tools in Unix used for processing the rows and columns in a file. Awk has built in string functions
and associative arrays. Awk supports most of the operators, conditional blocks, and loops available in C language.

One of the good things is that you can convert Awk scripts into Perl scripts using a2p utility.

The basic syntax of AWK:

awk 'BEGIN {start_action} {action} END {stop_action}' filename

Here the actions in the begin block are performed before processing the file and the actions in the end block are
performed after processing the file. The rest of the actions are performed while processing the file.

Examples:

Create a file input_file with the following data. This file can be easily created using the output of ls -l.
-rw-r--r-- 1 center center 0 Dec 8 21:39 p1
-rw-r--r-- 1 center center 17 Dec 8 21:15 t1
-rw-r--r-- 1 center center 26 Dec 8 21:38 t2
-rw-r--r-- 1 center center 25 Dec 8 21:38 t3
-rw-r--r-- 1 center center 43 Dec 8 21:39 t4
-rw-r--r-- 1 center center 48 Dec 8 21:39 t5
From the data, you can observe that this file has rows and columns. The rows are separated by a new line character and
the columns are separated by a space characters. We will use this file as the input for the examples discussed here.

1. awk '{print $1}' input_file

Here $1 has a meaning. $1, $2, $3... represents the first, second, third columns... in a row respectively. This awk
command will print the first column in each row as shown below.
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--

To print the 4th and 6th columns in a file use awk '{print $4,$5}' input_file

Here the Begin and End blocks are not used in awk. So, the print command will be executed for each row it reads from
the file. In the next example we will see how to use the Begin and End blocks.

2. awk 'BEGIN {sum=0} {sum=sum+$5} END {print sum}' input_file

This will prints the sum of the value in the 5th column. In the Begin block the variable sum is assigned with value 0. In
the next block the value of 5th column is added to the sum variable. This addition of the 5th column to the sum variable
repeats for every row it processed. When all the rows are processed the sum variable will hold the sum of the values in
the 5th column. This value is printed in the End block.

3. In this example we will see how to execute the awk script written in a file. Create a file sum_column and paste the
below script in that file
#!/usr/bin/awk -f
BEGIN {sum=0}
{sum=sum+$5}
END {print sum}

Now execute the the script using awk command as

awk -f sum_column input_file.

This will run the script in sum_column file and displays the sum of the 5th column in the input_file.

4. awk '{ if($9 == "t4") print $0;}' input_file

This awk command checks for the string "t4" in the 9th column and if it finds a match then it will print the entire line.
The output of this awk command is
-rw-r--r-- 1 pcenter pcenter 43 Dec 8 21:39 t4

5. awk 'BEGIN { for(i=1;i<=5;i++) print "square of", i, "is",i*i; }'

This will print the squares of first numbers from 1 to 5. The output of the command is
square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25
Notice that the syntax of “if” and “for” are similar to the C language.

Awk Built in Variables:

You have already seen $0, $1, $2... which prints the entire line, first column, second column... respectively. Now we will
see other built in variables with examples.

FS - Input field separator variable:

So far, we have seen the fields separted by a space character. By default Awk assumes that fields in a file are separted by
space characters. If the fields in the file are separted by any other character, we can use the FS variable to tell about the
delimiter.

6. awk 'BEGIN {FS=":"} {print $2}' input_file

OR
awk -F: '{print $2}' input_file

This will print the result as

39 p1
15 t1
38 t2
38 t3
39 t4
39 t5

OFS - Output field separator variable:

By default whenever we printed the fields using the print statement the fields are displayed with space character as
delimiter. For example

7. awk '{print $4,$5}' input_file

The output of this command will be

center 0
center 17
center 26
center 25
center 43
center 48

We can change this default behavior using the OFS variable as

awk 'BEGIN {OFS=":"} {print $4,$5}' input_file

center:0
center:17
center:26
center:25
center:43
center:48

Note: print $4,$5 and print $4$5 will not work the same way. The first one displays the output with space as delimiter.
The second one displays the output without any delimiter.

NF - Number of fileds variable:

The NF can be used to know the number of fields in line

8. awk '{print NF}' input_file

This will display the number of columns in each row.

NR - number of records variable:

The NR can be used to know the line number or count of lines in a file.

9. awk '{print NR}' input_file

This will display the line numbers from 1.

10. awk 'END {print NR}' input_file

This will display the total number of lines in the file.

String functions in Awk:

Some of the string functions in awk are:

index(string,search)
length(string)
split(string,array,separator)
substr(string,position)
substr(string,position,max)
tolower(string)
toupper(string)

Advanced Examples:

1. Filtering lines using Awk split function

The awk split function splits a string into an array using the delimiter.

The syntax of split function is

split(string, array, delimiter)

Now we will see how to filter the lines using the split function with an example.

The input "file.txt" contains the data in the following format

1 U,N,UNIX,000
2 N,P,SHELL,111
3 I,M,UNIX,222
4 X,Y,BASH,333
5 P,R,SCRIPT,444

Required output: Now we have to print only the lines in which whose 2nd field has the string "UNIX" as the 3rd field(
The 2nd filed in the line is separated by comma delimiter ).
The ouptut is:
1 U,N,UNIX,000
3 I,M,UNIX,222

The awk command for getting the output is:

awk '{
split($2,arr,",");
if(arr[3] == "UNIX")
print $0
} ' file.txt

Examples of Awk Command in Unix - Part 2

1. Inserting a new line after every 2 lines

We will see how to implement this using the awk command with an example.
The input "file.txt" contains the below data:
1 A
2 B
3 C
4 D
5 E
6 F

Let say, we want to insert the new line "9 Z" after every two lines in the input file. The required output data after
inserting a new line looks as
1 A
2 B
9 Z
3 C
4 D
9 Z
5 E
6 F
9 Z

The awk command for getting this output is

awk '{
if(NR%2 == 0)
{
print $0"\n9 Z";
}
else
{
print $0
}
}' file.txt

2. Replace the Nth occurrence of a pattern

The input file contains the data.

AAA 1
BBB 2
CCC 3
AAA 4
AAA 5
BBB 6
CCC 7
AAA 8
BBB 9
AAA 0

Now we want to replace the fourth occurrence of the first filed "AAA" with "ZZZ" in the file.
The required output is:
AAA 1
BBB 2
CCC 3
AAA 4
AAA 5
BBB 6
CCC 7
ZZZ 8
BBB 9
AAA 0

The awk command for getting this output is

awk 'BEGIN {count=0}
{
if($1 == "AAA")
{
count++
}
if(count == 4)
{
sub("AAA","ZZZ",$1)
}
}
{
print $0
}' file.txt

3. Find the sum of even and odd lines separately

The input file data:
A 10
B 39
C 22
D 44
E 75
F 89
G 67

You have to get the second field and then find the sum the even and odd lines.
The required output is
174, 172

The awk command for producing this output is

awk '{
if(NR%2 == 1)
{
sum_e = sum_e + $2
}
else
{
sum_o = sum_o + $2
}
}
END { print sum_e,sum_o }' file.txt

4. Fibonacci series using awk command

Now we will produce the Fibonacci series using the awk command.
awk ' BEGIN{
for(i=0;i<=10;i++)
{
 if (i <=1 )
{
x=0;
y=1;
print i;
}
else
{
z=x+y;
print z;
x=y;
y=z;
}
}
}'

The output is
0
1
1
2
3
5
8
13
21
34
55

5. Remove leading zeros from a file using the awk command. The input file contains the below data.
0012345
05678
01010
00001

After removing the leading zeros, the output should contain the below data.
12345
5678
1010
1

The awk command for this is.

awk '{print $1 + 0}' file.txt
awk '{printf "%d\n",$0}' file.txt

Sed Command in Unix and Linux Examples

Sed is a Stream Editor used for modifying the files in unix (or linux). Whenever you want to make changes to the file automatically,
sed comes in handy to do this. Most people never learn its power; they just simply use sed to replace text. You can do many things
apart from replacing text with sed. Here I will describe the features of sed with examples.

Consider the below text file as an input.

>cat file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
unixlinux which one you choose.
Sed Command Examples

1. Replacing or substituting string

Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word "unix" with "linux" in
the file.

>sed 's/unix/linux/' file.txt

linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here the "s" specifies the substitution operation. The "/" are delimiters. The "unix" is the search pattern and the "linux" is the
replacement string.

By default, the sed command replaces the first occurrence of the pattern in each line and it won't replace the second,
third...occurrence in the line.

2. Replacing the nth occurrence of a pattern in a line.

Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second
occurrence of the word "unix" with "linux" in a line.

>sed 's/unix/linux/2' file.txt

unix is great os. linux is opensource. unix is free os.
learn operating system.
unixlinux which one you choose.

3. Replacing all the occurrence of the pattern in a line.

The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.

>sed 's/unix/linux/g' file.txt

linux is great os. linux is opensource. linux is free os.
learn operating system.
linuxlinux which one you choose.

4. Replacing from nth occurrence to all occurrences in a line.

Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following
sed command replaces the third, fourth, fifth... "unix" word with "linux" word in a line.

>sed 's/unix/linux/3g' file.txt

unix is great os. unix is opensource. linux is free os.
learn operating system.
unixlinux which one you choose.

5. Changing the slash (/) delimiter

You can use any delimiter other than the slash. As an example if you want to change the web url to another url as

>sed 's/http:\/\//www/' file.txt

In this case the url consists the delimiter character which we used. In that case you have to escape the slash with backslash
character, otherwise the substitution won't work.

Using too many backslashes makes the sed command look awkward. In this case we can change the delimiter to another character
as shown in the below example.

>sed 's_http://_www_' file.txt

>sed 's|http://|www|' file.txt

6. Using & as the matched string

There might be cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In
such cases & comes in handy. The & represents the matched string.

>sed 's/unix/{&}/' file.txt

{unix} is great os. unix is opensource. unix is free os.
learn operating system.
{unix}linux which one you choose.

>sed 's/unix/{&&}/' file.txt

{unixunix} is great os. unix is opensource. unix is free os.
learn operating system.
{unixunix}linux which one you choose.

7. Using \1,\2 and so on to \9

The first pair of parenthesis specified in the pattern represents the \1, the second represents the \2 and so on. The \1,\2 can be
used in the replacement string to make changes to the source string. As an example, if you want to replace the word "unix" in a
line with twice as the word like "unixunix" use the sed command as below.

>sed 's/\(unix\)/\1\1/' file.txt

unixunix is great os. unix is opensource. unix is free os.
learn operating system.
unixunixlinux which one you choose.

The parenthesis needs to be escaped with the backslash character. Another example is if you want to switch the words "unixlinux"
as "linuxunix", the sed command is

>sed 's/\(unix\)\(linux\)/\2\1/' file.txt

unix is great os. unix is opensource. unix is free os.
learn operating system.
linuxunix which one you choose.

Another example is switching the first three characters in a line

>sed 's/^\(.\)\(.\)\(.\)/\3\2\1/' file.txt

inux is great os. unix is opensource. unix is free os.
aelrn operating system.
inuxlinux which one you choose.

8. Duplicating the replaced line with /p flag

The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then
the /p prints that line only once.

>sed 's/unix/linux/p' file.txt

linux is great os. unix is opensource. unix is free os.
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.
linuxlinux which one you choose.

9. Printing only the replaced lines

Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows
generated by the /p flag and prints the replaced lines only one time.

>sed -n 's/unix/linux/p' file.txt

linux is great os. unix is opensource. unix is free os.
linuxlinux which one you choose.

If you use -n alone without /p, then the sed does not print anything.

10. Running multiple sed commands.

You can run multiple sed commands by piping the output of one sed command as input to another sed command.

>sed 's/unix/linux/' file.txt| sed 's/os/system/'

linux is great system. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed
command as shown below.

>sed -e 's/unix/linux/' -e 's/os/system/' file.txt

linux is great system. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

11. Replacing string on a specific line number.

You can restrict the sed command to replace the string on a specific line number. An example is

>sed '3 s/unix/linux/' file.txt

unix is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

The above sed command replaces the string only on the third line.

12. Replacing string on a range of lines.

You can specify a range of line numbers to the sed command for replacing a string.

>sed '1,3 s/unix/linux/' file.txt

linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here the sed command replaces the lines with range from 1 to 3. Another example is

>sed '2,$ s/unix/linux/' file.txt

linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file.

13. Replace on a lines which matches a pattern.

You can specify a pattern to the sed command to match in a line. If the pattern match occurs, then only the sed command looks
for the string to be replaced and if it finds, then the sed command replaces the string.

>sed '/linux/ s/unix/centos/' file.txt

unix is great os. unix is opensource. unix is free os.
learn operating system.
centoslinux which one you choose.

Here the sed command first looks for the lines which has the pattern "linux" and then replaces the word "unix" with "centos".

14. Deleting lines.

You can delete the lines a file by specifying the line number or a range or numbers.

>sed '2 d' file.txt

>sed '5,$ d' file.txt

15. Duplicating lines

You can make the sed command to print each line of a file two times.

>sed 'p' file.txt

16. Sed as grep command

You can make sed command to work as similar to grep command.

>grep 'unix' file.txt

>sed -n '/unix/ p' file.txt

Here the sed command looks for the pattern "unix" in each line of a file and prints those lines that has the pattern.

You can also make the sed command to work as grep -v, just by using the reversing the sed with NOT (!).

>grep -v 'unix' file.txt

>sed -n '/unix/ !p' file.txt

The ! here inverts the pattern match.

17. Add a line after a match.

The sed command can add a new line after a pattern match is found. The "a" command to sed tells it to add a new line after a
match is found.

>sed '/unix/ a "Add a new line"' file.txt

unix is great os. unix is opensource. unix is free os.
"Add a new line"
learn operating system.
unixlinux which one you choose.
"Add a new line"

18. Add a line before a match

The sed command can add a new line before a pattern match is found. The "i" command to sed tells it to add a new line before a
match is found.

>sed '/unix/ i "Add a new line"' file.txt

"Add a new line"
unix is great os. unix is opensource. unix is free os.
learn operating system.
"Add a new line"
unixlinux which one you choose.

19. Change a line

The sed command can be used to replace an entire line with a new line. The "c" command to sed tells it to change the line.

>sed '/unix/ c "Change line"' file.txt

"Change line"
learn operating system.
"Change line"

20. Transform like tr command

The sed command can be used to convert the lower case letters to upper case letters by using the transform "y" option.

>sed 'y/ul/UL/' file.txt

Unix is great os. Unix is opensoUrce. Unix is free os.
Learn operating system.
UnixLinUx which one yoU choose.

Here the sed command transforms the alphabets "ul" into their uppercase format "UL"

Grep Command in Unix and Linux Examples

Grep is the frequently used command in Unix (or Linux). Most of us use grep just for finding the words in a file. The power of grep
comes with using its options and regular expressions. You can analyze large sets of log files with the help of grep command.

Grep stands for Global search for Regular Expressions and Print.

The basic syntax of grep command is

grep [options] pattern [list of files]

Let see some practical examples on grep command.

1. Running the last executed grep command

This saves a lot of time if you are executing the same command again and again.
!grep
This displays the last executed grep command and also prints the result set of the command on the terminal.

2. Search for a string in a file

This is the basic usage of grep command. It searches for the given string in the specified file.
grep "Error" logfile.txt
This searches for the string "Error" in the log file and prints all the lines that has the word "Error".

3. Searching for a string in multiple files.

grep "string" file1 file2
grep "string" file_pattern
This is also the basic usage of the grep command. You can manually specify the list of files you want to search or you
can specify a file pattern (use regular expressions) to search for.

4. Case insensitive search

The -i option enables to search for a string case insensitively in the give file. It matches the words like "UNIX", "Unix",
"unix".
grep -i "UNix" file.txt

5. Specifying the search string as a regular expression pattern.

grep "^[0-9].*" file.txt
This will search for the lines which starts with a number. Regular expressions is huge topic and I am not covering it here.
This example is just for providing the usage of regular expressions.

6. Checking for the whole words in a file.

By default, grep matches the given string/pattern even if it found as a substring in a file. The -w option to grep makes it
match only the whole words.
grep -w "world" file.txt

7. Displaying the lines before the match.

Some times, if you are searching for an error in a log file; it is always good to know the lines around the error lines to
know the cause of the error.
grep -B 2 "Error" file.txt
This will prints the matched lines along with the two lines before the matched lines.

8. Displaying the lines after the match.

grep -A 3 "Error" file.txt
This will display the matched lines along with the three lines after the matched lines.

9. Displaying the lines around the match

grep -C 5 "Error" file.txt
This will display the matched lines and also five lines before and after the matched lines.

10. Searching for a sting in all files recursively

You can search for a string in all the files under the current directory and sub-directories with the help -r option.
grep -r "string" *

11. Inverting the pattern match

You can display the lines that are not matched with the specified search sting pattern using the -v option.
grep -v "string" file.txt

12. Displaying the non-empty lines

You can remove the blank lines using the grep command.
grep -v "^$" file.txt

13. Displaying the count of number of matches.

We can find the number of lines that matches the given string/pattern
grep -c "sting" file.txt

14. Display the file names that matches the pattern.

We can just display the files that contains the given string/pattern.
grep -l "string" *

15. Display the file names that do not contain the pattern.

We can display the files which do not contain the matched string/pattern.
grep -L "string" *

16. Displaying only the matched pattern.

By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched
string by using the -o option.
grep -o "string" file.txt

17. Displaying the line numbers.

We can make the grep command to display the position of the line which contains the matched string in a file using the -
n option
grep -n "string" file.txt

18. Displaying the position of the matched string in the line

The -b option allows the grep command to display the character position of the matched string in a file.
grep -o -b "string" file.txt

19. Matching the lines that start with a string

The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with
the given string or pattern.
grep "^start" file.txt

20. Matching the lines that end with a string

The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with
the given string or pattern.
grep "end$" file.txt

Find Command in Unix and Linux Examples

Find is one of the powerful utility of Unix (or Linux) used for searching the files in a directory hierarchy. The syntax of find
command is

find [pathnames] [conditions]

Let see some practical exercises on using find command.

1. How to run the last executed find command?
!find

This will execute the last find command. It also displays the last find command executed along with the result on the
terminal.

2. How to find for a file using name?

find -name "sum.java"
./bkp/sum.java
./sum.java

This will find all the files with name "sum.java" in the current directory and sub-directories.

3. How to find for files using name and ignoring case?

find -iname "sum.java"
./SUM.java
./bkp/sum.java
./sum.java

This will find all the files with name "sum.java" while ignoring the case in the current directory and sub-directories.

4. How to find for a file in the current directory only?

find -maxdepth 1 -name "sum.java"
./sum.java

This will find for the file "sum.java" in the current directory only

5. How to find for files containing a specific word in its name?

find -name "*java*"
./SUM.java
./bkp/sum.java
./sum.java
./multiply.java

It displayed all the files which have the word "java" in the filename

6. How to find for files in a specific directory?

find /etc -name "*java*"

This will look for the files in the /etc directory with "java" in the filename

7. How to find the files whose name are not "sum.java"?

find -not -name "sum.java"
.
./SUM.java
./bkp
./multiply.java

This is like inverting the match. It prints all the files except the given file "sum.java".

8. How to limit the file searches to specific directories?

find -name "sum.java"
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java

You can see here the find command displayed all the files with name "sum.java" in the current directory and sub-
directories.

a. How to print the files in the current directory and one level down to the current directory?
find -maxdepth 2 -name "sum.java"
./tmp/sum.java
./bkp/sum.java
./sum.java

b. How to print the files in the current directory and two levels down to the current directory?
find -maxdepth 3 -name "sum.java"
./tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java

c. How to print the files in the subdirectories between level 1 and 4?

find -mindepth 2 -maxdepth 5 -name "sum.java"
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java

9. How to find the empty files in a directory?

find . -maxdepth 1 -empty
./empty_file

10. How to find the largest file in the current directory and sub directories
find . -type f -exec ls -s {} \; | sort -n -r | head -1

The find command "find . -type f -exec ls -s {} \;" will list all the files along with the size of the file. Then the sort
command will sort the files based on the size. The head command will pick only the first line from the output of sort.

11. How to find the smallest file in the current directory and sub directories
find . -type f -exec ls -s {} \; | sort -n -r | tail -1

Another method using find is

find . -type f -exec ls -s {} \; | sort -n | head -1

12. How to find files based on the file type?

a. Finding socket files

find . -type s
b. Finding directories
find . -type d

c. Finding hidden directories

find -type d -name ".*"

d. Finding regular files

find . -type f

e. Finding hidden files

find . -type f -name ".*"

13. How to find files based on the size?

a. Finding files whose size is exactly 10M

find . -size 10M

b. Finding files larger than 10M size

find . -size +10M

c. Finding files smaller than 10M size

find . -size -10M

14. How to find the files which are modified after the modification of a give file.
find -newer "sum.java"

This will display all the files which are modified after the file "sum.java"

15. Display the files which are accessed after the modification of a give file.
find -anewer "sum.java"

16. Display the files which are changed after the modification of a give file.
find -cnewer "sum.java"

17. How to find the files based on the file permissions?

find . -perm 777

This will display the files which have read, write, and execute permissions. To know the permissions of files and
directories use the command "ls -l".

18. Find the files which are modified within 30 minutes.

find . -mmin -30

19. Find the files which are modified within 1 day.

find . -mtime -1
20. How to find the files which are modified 30 minutes back
find . -not -mmin -30

21. How to find the files which are modified 1 day back.
find . -not -mtime -1

22. Print the files which are accessed within 1 hour.

find . -amin -60

23. Print the files which are accessed within 1 day.

find . -atime -1

24. Display the files which are changed within 2 hours.

find . -cmin -120

25. Display the files which are changed within 2 days.

find . -ctime -2

26. How to find the files which are created between two files.
find . -cnewer f1 -and ! -cnewer f2

So far we have just find the files and displayed on the terminal. Now we will see how to perform some operations on the
files.

1. How to find the permissions of the files which contain the name "java"?
find -name "*java*"|xargs ls -l

Alternate method is
find -name "*java*" -exec ls -l {} \;

2. Find the files which have the name "java" in it and then display only the files which have "class" word in them?
find -name "*java*" -exec grep -H class {} \;

3. How to remove files which contain the name "java".

find -name "*java*" -exec rm -r {} \;

This will delete all the files which have the word “java" in the file name in the current directory and sub-directories.