Linux Programming
Linux Programming
ON
LINUX PROGRAMMING
(R18A0525)
IV B.TECH YEAR – I
SEM(2022-23)
DEPARTMENT
OF
COMPUTER SCIENCE AND ENGINEERING
MALLAREDDYCOLLEGEOFENGINEERING&TECHNOLOGY
(AutonomousInstitution–UGC,Govt.ofIndia)
Recognized under2(f)and 12(B)ofUGCACT1956
(AffiliatedtoJNTUH,Hyderabad,ApprovedbyAICTE-AccreditedbyNBA& NAAC–‘A’Grade- ISO9001:2015Certified)
Maisammaguda,Dhulapally(PostVia. Hakimpet),Secunderabad–500100,TelanganaState,India
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
VISION
To acknowledge quality education and instill high patterns of discipline making the
students technologically superior and ethically strong which involves the improvement in the
quality of lime in human race.
MISSION
To achieve and impart holistic technical education using the best of infrastructure,
outstanding technical and teaching expertise to establish the students into competent and
confident engineers.
Evolving the center of excellence through creative and innovative teaching learning
practices for promoting academic achievement to produce internationally accepted competitive
and world class professionals.
PROGRAMME EDUCATIONAL OBJECTIVES (PEOs)
To facilitate the graduates with the ability to visualize, gather information, articulate,
analyze, solve complex problems, and make decisions. These are essential to address the
To facilitate the graduates with the technical skills that prepare them for immediate employment
and pursue certification providing a deeper understanding of the technology in advanced areas of
computer science and related fields, thus encouraging to pursue higher education and research
To facilitate the graduates with the soft skills that include fulfilling the mission, setting goals,
in team- work, being a leader, managing their career and their life.
To facilitate the graduates with the knowledge of professional and ethical responsibilities by
paying attention to grooming, being conservative with style, following dress codes, safety codes,
After the completion of the course, B. Tech Computer Science and Engineering, the
Able to Understand the working principles of the computer System and its components,
Apply the knowledge to build, asses, and analyze the software and hardware aspects of it
methodologies, and able to plan, develop, test, analyze, and manage the software and
Able to use the professional, managerial, interdisciplinary skill set, and domain specific
tools in development processes, identify the research gaps, and provide innovative
solutions to them.
PROGRAM OUTCOMES (POs)
Engineering Graduates should possess the following:
1. Engineering knowledge: Apply the knowledge of mathematics, science, engineering fundamentals,
and an engineering specialization to the solution of complex engineering problems.
2. Problem analysis: Identify, formulate, review research literature, and analyze complex engineering
problems reaching substantiated conclusions using first principles of mathematics, natural sciences, and
engineering sciences.
3. Design / development of solutions: Design solutions for complex engineering problems and design
system components or processes that meet the specified needs with appropriate consideration for the
public health and safety, and the cultural, societal, and environmental considerations.
4. Conduct investigations of complex problems: Use research-based knowledge and research methods
including design of experiments, analysis and interpretation of data, and synthesis of the information to
provide valid conclusions.
5. Modern tool usage: Create, select, and apply appropriate techniques, resources, and modern
engineering and IT tools including prediction and modeling to complex engineering activities with an
understanding of the limitations.
6. The engineer and society: Apply reasoning informed by the contextual knowledge to assess societal,
health, safety, legal and cultural issues and the consequent responsibilities relevant to the professional
engineering practice.
7. Environment and sustainability: Understand the impact of the professional engineering solutions in
societal and environmental contexts, and demonstrate the knowledge of, and need for sustainable
development.
8. Ethics: Apply ethical principles and commit to professional ethics and responsibilities and norms of
the engineering practice.
9. Individual and team work: Function effectively as an individual, and as a member or leader in
diverse teams, and in multidisciplinary settings.
10. Communication: Communicate effectively on complex engineering activities with the engineering
community and with society at large, such as, being able to comprehend and write effective reports and
design documentation, make effective presentations, and give and receive clear instructions.
11. Project management and finance: Demonstrate knowledge and understanding of the engineering
and management principles and apply these to one’s own work, as a member and leader in a team, to
manage projects and in multi disciplinary environments.
12. Life long learning: Recognize the need for, and have the preparation and ability to engage in
independent and life-long learning in the broadest context of technological change.
Syllabus
UNIT I
Linux Utilities - File handling utilities, Security by file permissions, Process utilities, Disk
utilities, Networking commands, Filters, Text processing utilities and Backup utilities. Sed-
Scripts, Operation, Addresses, Commands, Applications, awk- Execution, Fields and Records,
Scripts, Operation, Patterns, Actions, Associative Arrays, String and Mathematical functions,
System commands in awk, Applications. Shell programming with Bourne again shell(bash) -
Introduction, shell responsibilities, pipes and Redirection, here documents, running a shell script,
the shell as a programming language, shell meta characters, file name substitution, shell
variables, command substitution, shell commands, the environment, quoting, test command,
control structures, arithmetic in shell, shell script examples, interrupt processing, functions,
debugging shell scripts.
UNIT II
Files and Directories- File Concept, File types, File System Structure, file metadata-Inodes,
kernel support for files, system calls for file I/O operations- open, create, read, write, close,
lseek, dup2,file status information-stat family, file and record locking-lockf and fcntl functions,
file permissions - chmod, fchmod, file ownership-chown, lchown, fchown, links soft links and
hard links – symlink, link, unlink. Directories - Creating, removing and changing Directories-
mkdir, rmdir, chdir, obtaining current working directory-getcwd, Directory contents, Scanning
Directories-opendir, readdir, closedir, rewinddir, seekdir, telldir functions.
UNIT III
Process – Process concept, Kernel support for process, process identification, process hierarchy,
process states, process control - process creation, waiting for a process, process termination,
zombie process, orphan process, system call interface for process management-fork, vfork, exit,
wait, waitpid, exec family, system, I/O redirection Signals – Introduction to signals, Signal
generation and handling, Kernel support for signals, Signal function, unreliable signals, reliable
signals, kill, raise , alarm, pause, abort, sleep functions.
UNIT IV
Inter process Communication - Introduction to IPC, IPC between processes on a single
computer system, IPC between processes on different systems, pipes-creation, IPC between
related processes using unnamed pipes, FIFOs-creation, IPC between unrelated processes using
FIFOs (Named pipes),differences between unnamed and named pipes, popen and pclose library
functions. Message Queues- Kernel support for messages, APIs for message queues, client/server
example. Semaphores - Kernel support for semaphores, APIs for semaphores, file locking with
semaphores.
UNIT V
Shared Memory- Kernel support for shared memory, APIs for shared memory, shared memory
example. Sockets- Introduction to Berkeley Sockets, IPC over a network, Client-Server model,
Socket address structures (Unix domain and Internet domain), Socket system calls for connection
oriented protocol and connectionless protocol, example-client/server programs-Single Server-
Client connection, Multiple simultaneous clients, Comparison of IPC mechanisms.
TEXT BOOKS:
1. Unix System Programming using C++, T.Chan, PHI.
2. Unix Concepts and Applications, 4th Edition, Sumitabha Das, TMH,2006.
3. Unix Network Programming, W.R.Stevens, PHI
REFERENCE BOOKS:
1. Linux System Programming, Robert Love, O’Reilly, SPD, rp-2007.
2. Unix for programmers and users, 3rd Edition, Graham Glass, King Ables, Pearson2003,
3. Advanced Programming in the Unix environment, 2nd Edition, W.R.Stevens, Pearson.
4. System Programming with C and Unix, A.Hoover, Pearson
Course Objectives & Outcomes
Course Objectives
To develop the skills necessary for Unix systems programming including file system
To make effective use of Unix utilities and Shell scripting language such as bash.
To develop the basic skills required to write network programs using Sockets.
Course Outcomes
01-84
Introduction and Linux Utilities
I
Shell programming with Bourne 85–109
again shell(bash)
II 110-129
Files and Directories
130-149
III Process
Signals 150-161
Syllabus
UNIT I
Linux Utilities - File handling utilities, Security by file permissions, Process utilities,
Disk utilities, Networking commands, Filters, Text processing utilities and Backup
utilities. Sed- Scripts, Operation, Addresses, Commands, Applications, awk- Execution,
Fields and Records, Scripts, Operation, Patterns, Actions, Associative Arrays, String and
Mathematical functions, System commands in awk, Applications. Shell programming
with Bourne again shell(bash) - Introduction, shell responsibilities, pipes and
Redirection, here documents, running a shell script, the shell as a programming language,
shell meta characters, file name substitution, shell variables, command substitution, shell
commands, the environment, quoting, test command, control structures, arithmetic in
shell, shell script examples, interrupt processing, functions, debugging shell scripts.
UNIT II
Files and Directories- File Concept, File types, File System Structure, file metadata-
Inodes, kernel support for files, system calls for file I/O operations- open, create, read,
write, close, lseek, dup2,file status information-stat family, file and record locking-lockf
and fcntl functions, file permissions - chmod, fchmod, file ownership-chown, lchown,
fchown, links soft links and hard links – symlink, link, unlink. Directories - Creating,
removing and changing Directories- mkdir, rmdir, chdir, obtaining current working
directory-getcwd, Directory contents, Scanning Directories-opendir, readdir, closedir,
rewinddir, seekdir, telldir functions.
UNIT III
Process – Process concept, Kernel support for process, process identification, process
hierarchy, process states, process control - process creation, waiting for a process, process
termination, zombie process, orphan process, system call interface for process
management-fork, vfork, exit, wait, waitpid, exec family, system, I/O redirection Signals
– Introduction to signals, Signal generation and handling, Kernel support for signals,
Signal function, unreliable signals, reliable signals, kill, raise , alarm, pause, abort, sleep
functions.
UNIT IV
Inter process Communication - Introduction to IPC, IPC between processes on a single
computer system, IPC between processes on different systems, pipes-creation, IPC between
related processes using unnamed pipes, FIFOs-creation, IPC between unrelated processes
using FIFOs (Named pipes),differences between unnamed and named pipes, popen and pclose
library functions. Message Queues- Kernel support for messages, APIs for message queues,
client/server example. Semaphores - Kernel support for semaphores, APIs for semaphores, file
locking with semaphores.
UNIT V
Shared Memory- Kernel support for shared memory, APIs for shared memory, shared
memory example. Sockets- Introduction to Berkeley Sockets, IPC over a network, Client-
Server model, Socket address structures (Unix domain and Internet domain), Socket system
calls for connection oriented protocol and connectionless protocol, example-client/server
programs-Single Server- Client connection, Multiple simultaneous clients, Comparison of IPC
mechanisms.
TEXT BOOKS:
1. Unix System Programming using C++, T.Chan, PHI.
2. Unix Concepts and Applications, 4th Edition, Sumitabha Das, TMH,2006.
3. Unix Network Programming, W.R.Stevens, PHI
REFERENCE BOOKS:
1. Linux System Programming, Robert Love, O’Reilly, SPD, rp-2007.
2. Unix for programmers and users, 3rd Edition, Graham Glass, King Ables, Pearson2003,
3. Advanced Programming in the Unix environment, 2nd Edition, W.R.Stevens, Pearson.
4. System Programming with C and Unix, A.Hoover, Pearson
To develop the skills necessary for Unix systems programming including file system
To make effective use of Unix utilities and Shell scripting language such as bash.
To develop the basic skills required to write network programs using Sockets.
Course Outcomes
INDEX
01-84
IntroductionandLinuxUtilities
I
Shell programming with Bourne 85–109
againshell(bash)
II 110-129
FilesandDirectories
130-149
III Process
Signals 150-161
InterprocessCommunication 162-169
IV
MessageQueues, Semaphores 170-178
SharedMemory 179-183
V
Sockets 184-193
IntroductiontoLinux:
Linux is a Unix-like computer operating system assembled under the model of free and
opensource software development and distribution. The defining component of Linux is the
Linuxkernel,anoperating systemkernelfirstreleased5 October1991byLinusTorvalds.
LinuxwasoriginallydevelopedasafreeoperatingsystemforIntelx86-basedpersonalcomputers. It
has since been ported to more computer hardware platforms than any otheroperating system. It
is a leading operating system on servers and other big iron systems such
asmainframecomputersandsupercomputersmorethan90%oftoday's500fastestsupercomputersrun
somevariantof Linux,includingthe10fastest.Linux alsorunsonembedded systems (devices
where the operating system is typically built into the firmware andhighly tailored tothe
system)such asmobilephones,tabletcomputers,networkrouters,televisions and
videogameconsoles; the Android system in wide use on mobile devices isbuiltontheLinux
kernel.
BasicFeatures
FollowingaresomeoftheimportantfeaturesofLinuxOperatingSystem.
Portable - Portability means software‘s can works on different types of hardware‘s
insame way. Linux kernel and application programs support their installation on any
kindofhardwareplatform.
OpenSource-Linux sourcecode isfreelyavailableanditiscommunity baseddevelopment
project. Multiple Teams works in collaboration to enhance the
capabilityofLinuxoperatingsystemanditiscontinuouslyevolving.
Multi-User - Linux is a multiuser system means multiple users can access
systemresourceslikememory/ram/applicationprogramsatsametime.
Multiprogramming-Linuxisa
multiprogrammingsystemmeansmultipleapplicationscanrunatsametime.
HierarchicalFileSystem-Linuxprovidesastandardfilestructureinwhichsystem
`
files/userfilesarearranged.
Shell - Linux provides a special interpreter program which can be used to
executecommands of the operating system. It can be used to do various types of
operations, callapplicationprogramsetc.
Security - Linux provides user security using authentication features like
passwordprotection/controlledaccesstospecificfiles/encryptionofdata.
LinuxAdvantages
1. Low cost: You don‘t need to spend time andmoney to obtain licenses since Linux andmuch
of its software come with the GNU General Public License. You can start to workimmediately
without worrying that your software may stop working anytime because the freetrial version
expires. Additionally, there are large repositories from which you can
freelydownloadhighqualitysoftwareforalmostanytaskyoucanthinkof.
2. Stability: Linuxdoesn‘tneedtoberebootedperiodicallytomaintainperformancelevels.It
doesn‘tfreeze up or slow down over time due to memory leaks andsuch. Continuousup-
times ofhundreds ofdays (uptoayearormore)are notuncommon.
3. Performance:Linuxprovidespersistenthighperformanceonworkstationsandonnetworks. It
can handle unusually large numbers of users simultaneously, and can make
oldcomputerssufficientlyresponsive tobeusefulagain.
4. Network friendliness: Linux was developed by a group of programmers over the
Internetand has therefore strong support for network functionality; client and server systems
can beeasily set up on any computer running Linux. It can perform tasks such as network
backupsfasterandmore reliablythanalternativesystems.
5. Flexibility:Linuxcanbeusedforhighperformanceserverapplications,desktopapplications,ande
commonfileformats.
7. Choice: The large number of Linux distributions gives you a choice. Each distribution
isdevelopedandsupportedbyadifferentorganization.Youcanpicktheoneyoulikebest;the
corefunctionalitiesarethesame;mostsoftwarerunsonmostdistributions.
8. Fast and easy installation: Most Linux distributions come with user-friendly
installationand setup programs. Popular Linux distributions come with tools that make
installation ofadditionalsoftwareveryuserfriendlyaswell.
9. Full useofharddisk:Linux continuesworkwell even when theharddiskisalmostfull.
10. Multi-tasking: Linux is designed to do many things at the same time; e.g., a
largeprintingjobinthebackgroundwon‘tslow downyourotherwork.
11. Security: Linux is one ofthe most secure operating systems. ―Walls‖and flexible file access
permission systems prevent access by unwanted visitors or viruses. Linux users have
tooptiontoselectandsafelydownloadsoftware,freeof charge,from onlinerepositoriescontaining
thousands of high quality packages. No purchase transactions requiring credit
cardnumbersorothersensitivepersonalinformationarenecessary.
12. Open Source: If you develop software that requires knowledge or modification of
theoperating system code, LINUX‘s source code is at your fingertips. Most Linux
applicationsare OpenSourceas well.
DifferencebetweenUNIXandLINUX
byvariousvendors.
stems.OSXisPowerPC(10.0-
10.5)/x86(10.4)/x64(10.5-10.8)
LinuxDistribution(OperatingSystem)Names
A few popular names:
1.RedhatEnterpriseLinux2
.Fedora Linux
3. DebianLinux
4. Suse Enterprise
Linux5.UbuntuLinux
Commonthingsbetween Linux&UNIX
Bothsharemanycommonapplicationssuchas:1.GUI,
file, and windows managers (KDE,
Gnome)2.Shells(ksh,csh,bash)
3. Various office applications such as
OpenOffice.org4.Development tools (perl, php, python, GNU
c/c++ compilers)5.Posixinterface
LayeredArchitecture:
LinuxSystemArchitectureisconsistsoffollowinglayers
Hardwarelayer-Hardwareconsistsofallperipheraldevices(RAM/ HDD/CPUetc).
Kernel-
CorecomponentofOperatingSystem,interactsdirectlywithhardware,provideslowlevelservi
ces toupperlayercomponents.
Shell -
Aninterfacetokernel,hidingcomplexityofkernel'sfunctionsfromusers.Takescommandsfro
muserandexecuteskernel'sfunctions.
Utilities- Utilityprogramsgivingusermostofthefunctionalitiesofanoperatingsystems.
LINUXFilesystem
Linuxfilestructurefilesaregroupedaccordingtopurpose.Ex:commands,datafiles,documentation.
Parts of a Unix directory tree are listed below. All directories are groupedunderthe
rootentry"/".Thatpartofthe directorytree isleftoutofthe below diagram.
1. /– Root
Everysinglefileand directorystartsfromtherootdirectory.
Onlyrootuserhaswriteprivilegeunderthisdirectory.
Pleasenotethat/rootisrootuser‘shomedirectory,whichisnotsameas/.
2. /bin–User Binaries
Containsbinaryexecutables.
Commonlinuxcommandsyouneedtouseinsingle-usermodesarelocatedunderthisdirectory.
Commandsusedbyallthe usersofthesystemare locatedhere.
Forexample:ps,ls,ping,grep,cp.
3. /sbin – SystemBinaries
Justlike/bin, /sbinalsocontainsbinaryexecutables.
But,thelinuxcommandslocatedunderthisdirectoryareusedtypicallybysystemaministrator,f
orsystemmaintenancepurpose.
Forexample: iptables,reboot,fdisk,ifconfig,swapon
4. /etc–ConfigurationFiles
Containsconfigurationfilesrequiredbyallprograms.
Thisalsocontainsstartupandshutdownshellscriptsusedtostart/stopindividualprograms.
Forexample:/etc/resolv.conf,/etc/logrotate.conf
5. /dev–DeviceFiles
Containsdevicefiles.
Theseincludeterminaldevices,usb, oranydeviceattachedtothesystem.
Forexample:/dev/tty1, /dev/usbmon0
6. /proc–ProcessInformation
Contains informationaboutsystemprocess.
Thisisapseudofilesystemcontainsinformationaboutrunningprocess.Forexample:
/proc/{pid}directorycontains informationabouttheprocesswiththatparticularpid.
Thisisavirtualfilesystemwithtextinformationaboutsystemresources.Forexample:
/proc/uptime
7. /var –VariableFiles
varstandsforvariablefiles.
Contentofthefilesthatareexpectedtogrowcanbefoundunder thisdirectory.
catlinuxcommandconcatenates filesandprintitonthestandardoutput.
SYNTAX:
TheSyntaxis
cat[OPTIONS][FILE]...
OPTIONS:
-A Showall.
-b Omitslinenumbersforblankspaceintheoutput.
-e A$ characterwillbeprinted attheend ofeach linepriortoanewline.
-E Displaysa$(dollar sign) attheend ofeach line.
-n Linenumbers foralltheoutputlines.
-s Iftheoutputhas multipleemptylinesitreplaces itwithoneemptyline.
-T Displaysthetabcharactersinthe output.
Non-printingcharacters(withtheexception oftabs,new-linesandform-feeds)
-v areprintedvisibly.
Example:
ToCreateanewfile:cat>
file1.txt
This command creates a new file file1.txt. After typing into the file press
control+d(^d)simultaneouslytoendthefile.
2. Todisplaya
file:catf
ile1.txt
Thiscommanddisplaysthedatainthefile.
3. Toconcatenateseveralfilesanddi
splay:catfile1.txtfile2.txt
The above cat command will concatenate the two files (file1.txt and file2.txt) and it
willdisplay the output in the screen. Sometimes the output may not fit the monitor screen.
Insuch situation you can printthose files in a nlewfile or display the file
usinglesscommand.
cat file1.txtfile2.txt |less
4. Toconcatenateseveralfilesandtotransfertheoutputtoanotherfile.catfile
1.txtfile2.txt>file3.txt
In the above example the output is redirected to new file file3.txt. The cat command
willcreate new filefile3.txtandstorethe concatenatedoutputintofile3.txt.
rm COMMAND:
rmlinuxcommandisusedtoremove/deletethefilefromthedirectory.
SYNTAX:
TheSyntaxis
rm[options..] [file|directory]
OPTIONS:
-f Removeallfiles inadirectorywithoutpromptingtheuser.
Interactive.Withthisoption, rmpromptsforconfirmationbeforeremoving
-i
anyfiles.
Recursively remove directories and subdirectories in the argument list.
Thedirectorywillbeemptiedoffilesandremoved.Theuserisnormallypromptedfo
-r(or)-R
rremovalofanywrite-protectedfiles whichthedirectory
contains.
EXAMPLE:
1. ToRemove/Deleteafile:rm
file1.txt
Herermcommandwillremove/deletethefilefile1.txt.
2. To delete a directory
tree:rm-irtmp
This rm command recursively removes the contents of all subdirectories of the
tmpdirectory, prompting you regarding the removal of each file, and then removes the
tmpdirectoryitself.
3. Toremovemore
filesatoncermfile1.txtfile2.tx
t
rmcommandremoves file1.txtandfile2.txtfilesatthesametime.
cdCOMMAND:
cdcommandisusedtochangethedirectory.
SYNTAX:
TheSyntaxis
cd[directory|~|./|../|-]
OPTIONS:
-L Usethephysicaldirectorystructure.
-P Forcessymbolic links.
EXAMPLE:
1. cdlinux-command
This command will take you to the sub-directory(linux-command) from its
parentdirectory.
2. cd..
Thiswillchangetotheparent-directoryfromthecurrentworking directory/sub-directory.
3. cd~
Thiscommandwillmovetotheuser'shomedirectorywhichis"/home/username".
cpCOMMAND:
cp command copy files from one location to another. If the destination is an existing file,
thenthe file is overwritten; if the destination is an existing directory, the file is copied into
Linux Programming Page 13
Department of CSE
thedirectory(thedirectoryisnotoverwritten).
SYNTAX:
TheSyntaxis
cp[OPTIONS]...SOURCEDEST
cp[OPTIONS]...SOURCE...DIRECTORY
cp[OPTIONS]...--target-directory=DIRECTORYSOURCE...
OPTIONS:
-a sameas-dpR.
--backup[=CONTROL] makeabackupofeachexistingdestination file
-b like--backupbutdoesnotacceptanargument.
ifanexistingdestinationfilecannotbeopened,removeitandtry
-f
again.
-p sameas--preserve=mode,ownership,timestamps.
preserve the specified attributes
-
(default:mode,ownership,timestamps)andsecurityc
preserve[=ATTR_LIST]
ontexts,ifpossible
additionalattributes: links,all.
--no-
don'tpreservethespecifiedattribute.
preserve=ATTR_LIST
--parents appendsourcepathtoDIRECTORY.
EXAMPLE:
Copytwo
files:
cp
file1file2
Theabovecpcommandcopiesthecontentoffile1.phptofile2.php.
1. Tobackupthecopiedfi
le:cp-
bfile1.phpfile2.php
Backup offile1.php willbecreated with'~'symbolasfile2.php~.
2. Copyfolderandsubfolde
rs: cp -R scriptsscripts1
Theabovecpcommand copythefolderand subfoldersfromscriptstoscripts1.
lsCOMMAND:
lscommandliststhe filesanddirectoriesundercurrentworkingdirectory.
SYNTAX:
TheSynta
xis
ls[OPTIONS]...[FILE]
OPTIONS:
Listsallthefiles,directoriesandtheirmode, Numberoflinks, ownerofthe
-l
file,filesize,Modifieddateandtimeand filename.
-t Lists inorderoflastmodificationtime.
-a Listsallentries includinghiddenfiles.
-d Listsdirectoryfilesinsteadofcontents.
-p Putsslashat theend ofeachdirectories.
-u Listinorderoflastaccesstime.
-i Displayinode information.
-ltr Listfilesorderbydate.
-lSr Listfilesorderbyfilesize.
EXAMPLE:
Displayrootdirectorycontents:ls
/
liststhecontentsofrootdirectory.
1. Displayhiddenfilesanddirectories:ls
-a
listsallentriesincludinghiddenfilesanddirectories.
2. Display inode
information:ls-i
7373073book.gif
7373074clock.gif
7373082globe.gif
7373078pencil.gif
7373080child.gif
23
7373081email.gif
7373076indigo.gif
Theabovecommanddisplaysfilenamewithinodevalue.
lnCOMMAND:
ln command is used to create link to a file (or) directory. It helps to provide soft link
fordesiredfiles.Inode willbe differentforsourceanddestination.
SYNTAX:
TheSynt
axis
ln[options]existingfile(ordirectory)namenewfile(ordirectory)name
OPTIONS:
Linkfileswithoutquestioningtheuser, evenifthemodeoftargetforbids
-f
writing.Thisisthedefaultifthestandardinputisnotaterminal.
-n Doesnotoverwriteexistingfiles.
-s Usedtocreatesoftlinks.
EXAMPLE:
1. ln-sfile1.txtfile2.txt
Createsasymboliclinkto'file1.txt'withthenameof'file2.txt'.Hereinodefor'file1.txt'and'fi
le2.txt'willbedifferent.
2. ln-sniminimi1
Createsasymboliclinkto'nimi'withthenameof'nimi1'.
chownCOMMAND:
chowncommandisused tochangetheowner / userofthefileordirectory.Thisisanadmin
command,rootuseronlycanchange the ownerofafile ordirectory.
SYNTAX:
TheSyntaxis
chown[options]newownerfilename/directoryname
OPTIONS:
Changethepermissiononfilesthatareinthesubdirectoriesofthe directory
-R
thatyouarecurrentlyin.
-c Changethepermissionforeachfile.
Preventschownfromdisplayingerrormessageswhenitisunableto change
-f
theownershipofafile.
EXAMPLE:
1. chownhioxtest.txt
Theowner ofthe 'test.txt'fileisroot,Changetonewuser hiox.
2. chown-R hioxtest
The owner of the 'test' directory is root, With -R option the files and
subdirectoriesuseralsogets changed.
3. chown-c hioxcalc.txt
Herechangetheownerforthespecific'calc.txt'fileonly.
SecurityByFilePermissionsC
hmodCommand:
chmodcommandallowsyoutoalter/Changeaccessrightstofilesanddirectories.
FilePermissionisgivenforusers,groupandothersas,
User
Group
Others
Permission 000
SYNTAX:
TheSyntaxis
chmod[options][MODE]FileName
FilePermission
# FilePermission
0 none
1 executeonly
2 writeonly
3 writeandexecute
4 readonly
5 read and execute
6 readand write
7 setallpermissions
OPTIONS:
-c Displaysnamesofonlythosefileswhosepermissionsarebeing changed
-f Suppressmost errormessages
-R Changefilesanddirectoriesrecursively
-v Outputversioninformationandexit.
EXAMPLE:
1. Toviewyourfileswithwhatpermissiontheyare:ls-
alt
Thiscommandisusedtoviewyourfileswithwhatpermissiontheyare.
2. Tomakeafilereadableandwritablebythegroupandothers.chmo
d066file1.txt
3. Toalloweveryonetoread,write,andexecutethefilechm
od777file1.txt
mkdirCOMMAND:
mkdircommandisusedtocreateoneormoredirectories.
SYNTAX:
TheSyntaxis
mkdir[options]directories
OPTIONS:
-m Settheaccess modeforthenewdirectories.
-p Createinterveningparentdirectories iftheydon'texist.
-v Printhelpmessageforeachdirectorycreated.
EXAMPLE:
1. Create
directory:mkdirt
est
Theabovecommandisused tocreatethedirectory'test'.
2. Createdirectoryandsetpermissions:m
kdir-m666test
Theabovecommandisusedtocreatethedirectory'test'andsetthereadandwritepermission.
rmdirCOMMAND:
rmdircommand isusedtodelete/removeadirectoryand itssubdirectories.
SYNTAX:
TheSyntaxis
rmdir[options..]Directory
OPTIONS:
Allowuserstoremovethedirectorydirnameanditsparentdirectorieswhich
-p
becomeempty.
EXAMPLE:
1. To delete/remove a
directoryrmdirtmp
rmdir command willremove/deletethedirectorytmpifthedirectoryisempty.
2. To delete a directory
tree:rm-irtmp
Thiscommandrecursivelyremovesthecontentsofallsubdirectoriesofthetmpdirectory,
prompting you regarding the removal of each file, and then removes the
tmpdirectoryitself.
mv COMMAND:
mv command which is short for move. It is used to move/rename file from one directory
toanother. mv command is different from cp command as it completely removes the file from
thesource and moves to the directory specified, where cp command just copies the content from
onefile toanother.
SYNTAX:
TheSyntaxis
mv[-f][-i]oldnamenewname
OPTIONS:
Thiswillnotpromptbeforeoverwriting(equivalent to--reply=yes).mv -fwill
-f
movethefile(s)withoutpromptingevenifitiswritingoveranexistingtarget.
-i Promptsbeforeoverwritinganotherfile.
EXAMPLE:
1. ToRename/Moveafile:m
v file1.txtfile2.txt
Thiscommandrenamesfile1.txtasfile2.txt
2. To move a
directorymvhscripts
tmp
In the above line mv command moves all the files, directories and sub-directories
fromhscripts folder/directory to tmp directory if the tmp directory already exists. If there
is notmpdirectoryitrename'sthe hscripts directoryas tmpdirectory.
3. ToMovemultiplefiles/Morefilesintoanotherdirectorymvf
ile1.txttmp/file2.txtnewdir
This command moves the files file1.txt from the current directory and file2.txt from
thetmpfolder/directorytonewdir.
diffCOMMAND:
diffcommandisusedtofinddifferencesbetweentwofiles.
SYNTAX:
TheSyntaxis
diff[options..]from-fileto-file
OPTIONS:
-a Treatallfilesastextandcomparethemline-by-line.
-b Ignorechangesinamountofwhitespace.
-c Usethecontextoutputformat.
-e Makeoutputthatisavalidedscript.
Useheuristicstospeedhandlingoflargefilesthathavenumerousscattered
-H
smallchanges.
-i Ignorechanges incase;considerupper-andlower-caselettersequivalent.
Linux Programming Page 21
Department of CSE
-r Whencomparingdirectories,recursivelycompareanysubdirectoriesfound.
-s Reportwhentwofilesarethesame.
-w Ignorewhitespacewhencomparinglines.
-y Usethesidebysideoutputformat.
EXAMPLE:
Letscreatetwofilesfile1.txtandfile2.txtandletithavethe followingdata.
shipwithfriendshipHioxindia.com<
Thethird line(withfriendship)infile2.txthasmoreblankspaces,butstillthe-bignores
the blank space and does not show changes in the particular line, -y printout the
resultside byside.
OPTIONS:
Change thepermissionon filesthatare inthesubdirectoriesofthedirectory
-R
thatyouarecurrentlyin.
-c Changethepermissionforeachfile.
-f Force.Donotreporterrors.
Hioxindia.com <
EXAMPLE:
1. chgrphioxtest.txt
Thegroupof'test.txt'fileisroot,Changetonewgrouphiox.
2. chgrp -Rhioxtest
The group of 'test' directory is root. With -R, the files and its subdirectories also
changestonewgrouphiox.
3. chgrp-chioxcalc.txt
Theyabovecommandisusedtochangethegroupfor thespecificfile('calc.txt')only.
Aboutwc
Shortforwordcount,wcdisplaysacountoflines,words,andcharacters inafile.
Syntax
wc [-c |-m|-C] [-l][-w][file...]
-c Countbytes.
-m Countcharacters.
-C Sameas-m.
-l Countlines.
-w Countwordsdelimitedbywhitespacecharactersornewlinecharacters.Delimitingchara
ctersareExtendedUnixCode(EUC)charactersfromanycodesetdefinedbyiswspace()
File Nameoffiletowordcount.
Examples
wcmyfile.txt-Displaysinformationaboutthefilemyfile.txt. Belowisanexampleoftheoutput.
5 13
57myfile.txt5=Lin
es
13=Words
57 =
CharactersAbo
ut split
Splitafileintopieces.
Syntax
split[-linecount|-llinecount][-asuffixlength][file[name] ]split-
bn[k|m] [ -asuffixlength] [file[name]]
- Use suffixlength letters to form the suffix portion of the filenames of thesplitfile.
asuffixlengt If -a is not specified, the default suffix length is 2. If the sum of the
h nameoperandandthesuffixlengthoption-argument
wouldcreateafilenameexceeding
NAME_MAXbytes, anerrorwillresult;splitwillexitwithadiagnosticmessage
andnofileswillbecreated.
File The path name of the ordinary file to be split. If no input file is given or file is -
,the standardinputwillbeused.
name The prefix to be used for each of the files resulting from the split operation. If
noname argument is given, x will be used as the prefix of the output files.
Thecombinedlengthofthebasename ofprefixand suffixlengthcannotexceed
NAME_MAXbytes;seeOPTIONS.
Examples
split-b22newfile.txtnew-
wouldsplitthefile"newfile.txt"intothreeseparatefilescallednewaa,newabandnewaceachfilethesize
of22.
split-l300file.txtnew-
wouldsplitthefile"newfile.txt"intofilesbeginningwiththename"new"eachcontaining300lines
oftexteach
Aboutsettimeand touch
Changefileaccessandmodificationtime.
Syntax
touch[-a] [-c][-m][-r ref_file|-ttime ]
filesettime [-fref_file] file
-a Changetheaccesstimeoffile.Donotchangethemodificationtimeunless-mis
alsospecified.
-c Donotcreateaspecifiedfileifitdoesnotexist.Donotwriteanydiagnosticmessagesconcer
ningthis condition.
-m Changethemodificationtimeoffile.Donotchangetheaccesstimeunless-
aisalsospecified.
-rref_file Usethecorrespondingtimesofthefilenamedbyref_fileinsteadofthecurrenttime.
-ttime Use the specified time instead of the current time. time will be a decimal
numberoftheform:
[[CC]YY]MMDDhhmm[.SS]
-fref_file Usethecorrespondingtimesofthefilenamedbyref_fileinsteadofthecurrenttime.
Examples
settimemyfile.txt
Setsthefilemyfile.txtasthecurrenttime/date.
touchnewfile.txt
Creates a file known as "newfile.txt", if the file does not already exist. If the file already
existsthe accessed/modificationtimeis updatedforthe filenewfile.txt
About comm
Selectorrejectlinescommontotwofiles.
Syntax
comm[-1][-2][-3]file1file2
-1 Suppresstheoutputcolumnoflinesuniqueto file1.
-2 Suppresstheoutputcolumnoflinesuniquetofile2.
-3 Suppresstheoutputcolumnoflinesduplicatedinfile1andfile2.
file1 Nameofthefirstfiletocompare.
file2 Nameofthesecondfiletocompare.
Examples
commmyfile1.txtmyfile2.txt
Theaboveexamplewouldcomparethetwofiles myfile1.txtandmyfile2.txt.
Processutilities:
psCommand:
pscommandisusedtoreporttheprocessstatus. psistheshortnameforProcessStatus.
SYNTAX:
TheSyntaxisp
s[options]
OPTIONS:
Listinformationaboutallprocessesmostfrequentlyrequested:allthose
-a
exceptprocessgroupleadersandprocessesnotassociatedwithaterminal..
-Aor e Listinformationforallprocesses.
-d Listinformationaboutallprocessesexceptsessionleaders.
-e Listinformationabouteveryprocessnowrunning.
-f Generatesafulllisting.
-j PrintsessionIDandprocessgroupID.
-l Generatealonglisting.
EXAMPLE:
1. ps
Output:
PIDTTY TIMECMD
2540pts/1 00:00:00bash
2621pts/1 00:00:00ps
Intheaboveexample,typingpsalonewouldlistthecurrentrunningprocesses.
2. ps -f
Output:
UID PIDPPIDCSTIMETTY TIMECMD
nirmala 25402536015:31pts/1 00:00:00bash
nirmala 26392540015:51pts/1 00:00:00 ps -
fDisplaysfullinformationaboutcurrentlyrunningprocesses.
killCOMMAND:
killcommandisusedto killthebackground process.
SYNTAX:
TheSyntaxis
kill[-s] [-l]%pid
OPTIONS:
Specifythesignaltosend.Thesignal maybegivenasasignalname or
-s
number.
Writeallvaluesofsignalsupportedbytheimplementation,ifnooperandis
-l
given.
-pid Processidorjobid.
-9 Forcetokillaprocess.
EXAMPLE:
StepbyStepprocess:
Openaprocessmusicplayer.x
mms
pressctrl+zto stoptheprocess.
To knowgroupidorjobid
ofthebackgroundtask.jobs-l
Itwill listthebackgroundjobswithitsjobidas,
xmms
3956kmail
3467
To killajoborprocess.
kill3956
killcommandkillsorterminatesthebackgroundprocessxmms.
Aboutnice
Invokesacommandwithanalteredschedulingpriority.
Syntax
nice[-increment|-nincrement]command[argument...]
argument Anystringtobesuppliedasanargumentwheninvokingcommand.
Examples
nice+13picomyfile.txt-runsthepicocommandonmyfile.txtwithanincrementof+13.Aboutat
Schedulesacommandtoberanataparticulartime,suchasaprintjoblateatnight.
Syntax
at executescommandsataspecifiedtime.
atq liststheuser'spendingjobs,unlesstheuseristhesuperuser;inthatcase,everybody'sjobsarelisted.The
formatoftheoutputlines(oneforeachjob)is:Jobnumber, date,hour,job
class.
atrm deletesjobs,identifiedbytheirjobnumber.
at [-c |-k|-s][-ffilename][-qqueuename][-m]-ttime[date][-l][-r]
-c Cshell.csh(1)isusedtoexecutetheat-job.
-k Kornshell.ksh(1)isusedto executetheat-job.
-s Bourneshell.sh(1) isusedtoexecutetheat-job.
-ffilename Specifiesthefilethatcontainsthecommandtorun.
-m Sendsmailoncethecommand hasbeenrun.
-ttime Specifiesatwhattimeyouwantthecommand
toberan.Formathh:mm.am/pmindication can also follow the time otherwise a 24-
hour clock is used. A
timezonenameofGMT,UCTorZULU(caseinsensitive)canfollowtospecifythattheti
meisinCoordinatedUniversalTime.Othertimezonescanbespecifiedusingthe TZ
environment variable. The below quick times can also beentered:midnight
-Indicates the time 12:00 am
(00:00).noon - Indicates the time
12:00 pm.now - Indicates the current day and time.Invoking at-
now will submitsubmitanat-jobforpotentiallyimmediate execution.
date Specifiesthedateyouwishittoberanon.Formatmonth,date,year.Thefollowing
quick days can also be entered:
-l Liststhecommandsthathavebeensettorun.
-r Cancelsthecommandthatyouhavesetinthepast.
Examples
at -m 01:35 < atjob = Run the commands listed in the 'atjob' file at 1:35AM, in addition
alloutput that is generated from job mail to the user running the task. When this command has
beensuccessfullyenteryoushould receive a promptsimilartothebelow example.
Commands will be executed using /bin/cshjob1072250520.aatWedDec2400:22:002003
at -l = This command will list each of the scheduled jobs as seen
below.1072250520.a WedDec2400:22:002003
at-r1072250520.a=Deletesthejobjustcreated.or
atrm23= Deletesjob23.
Ifyouwishtocreateajobthatisrepeatedyou
couldmodifythefilethatexecutesthecommandswithanothercommand
thatrecreatesthejoborbetteryetuse thecrontabcommand.
Note: Performingjusttheatcommandatthepromptwillgiveyouanerror "GarbledTime",thisisa
standarderrormessageifnoswitchortime settingisgiven.
Diskutilities:
du (abbreviated from disk usage) is a standard Unix program used to estimate filespaceusage—
spaceusedunderaparticulardirectoryorfilesonafile system.
dutakesasingleargument,specifyingapathnamefordutowork;ifitisnotspecified,thecurrentdire
ctoryisused.TheSUSmandatesforduthefollowingoptions:
-a, displayanentryforeachfile(andnotdirectory)containedinthecurrentdirectory
-H,calculatediskusageforlinkreferencesspecifiedonthecommandline
-k,showsizesas multiplesof1024bytes,not512-byte
-L,calculatediskusageforlinkreferencesanywhere
-s,reportonlythesumoftheusageinthecurrentdirectory,notfor eachfile
-x,onlytraversefilesanddirectoriesonthedeviceonwhichthepathnameargumentisspecified.
Other Unix and Unix-like operating systems may add extra options. For example, BSD and
GNUdu specify a -h option, displaying disk usage in a format easier to read by the user, adding
unitswiththeappropriateSIprefix‘
$du-sk*
152304directoryOne
1856548directoryTwo
$du-sh*
149MdirectoryOne
1.8GdirectoryTwo
disk usage of all subdirectories and files including hidden files within the current
directory(sortedbyfilesize):
$du-sk.[!.]**|sort-n
disk usage of all subdirectories and files including hidden files within the current
directory(sortedbyreversefilesize):
nrTheweight
ofdirectories:
$du-d1-c-h
dfcommand:ReportfilesystemdiskspaceusageDf
command examples - to check free disk
spaceType df-hordf-ktolistfree diskspace:
$df-
hOR
$df –
kOutpu
t:
Filesystem Size UsedAvailUse%Mountedon
/dev/sdb1 20G9.2G 9.6G 49%/
varrun 393M144k 393M 1%/var/runvarlock
393M0393M0%/var/lockprocbususb
393M123k393M1%/proc/bus/usbudev
393M 123k393M1%/dev
devshm 393M 0 393M0%/dev/shm
lrm 393M 35M359M9%/lib/modules/2.6.20-15-generic/volatile
/dev/sdb5 29G5.4G 22G20%/media/docs
/dev/sdb3 30G5.9G 23G21%/media/isomp3s
/dev/sda1 8.5G4.3G4.3G51%/media/xp1
/dev/sda2 12G6.5G5.2G56%/media/xp2
/dev/sdc1 40G 3.1G 35G9%/media/backup
ducommandexamples
dushowshowmuchspaceoneoremorefilesor directories isusing.
$du-
sh103M
-s option summarize the space a directory is using and -h option provides "Human-
readable"output.
Networkingcommands:
These are most useful commands in my list while working on Linux server , this enables you
toquickly troubleshoot connection issues e.g. whether other system is connected or not ,
whetherother host is responding or not and while working for FIX connectivity for advanced
tradingsystemthis tools saves quitealotoftime .
This article is in continuation of my article How to work fast in Unix and Unix
CommandtutorialsandExamplesforbeginners.
• findinghost/domainnameandIPaddress-hostname
• testnetworkconnection–ping
• gettingnetworkconfiguration–ifconfig
• Networkconnections,routingtables,interfacestatistics–netstat
• queryDNSlookupname–nslookup
• communicatewithotherhostname –telnet
• outingstepsthatpacketstaketogettonetworkhost–traceroute
• viewuserinformation–finger
ping
It sends packets of information to the user-defined source. If the packets are received,
thedestinationdevice sendspacketsback.Pingcanbeusedfortwopurposes
1. Toensurethatanetworkconnectioncanbeestablished.
2. Timinginformationastothespeedoftheconnection.
Ifyoudoping www.yahoo.comitwilldisplayitsIP address.Usectrl+Cto stopthetest.
ifconfig
Viewnetworkconfiguration,itdisplaysthecurrentnetworkadapterconfiguration.Itishandyto
determineifyouaregettingtransmit(TX)orreceive(RX) errors.
netstat
Mostusefulandveryversatileforfindingconnectiontoandfromthehost.Youcanfindoutallthemulticast
groups(network)subscribedbythishostbyissuing"netstat-g"
netstat -nap|grepportwilldisplayprocessidofapplicationwhichisusingthatportnetstat-
aor netstat–allwilldisplayallconnections includingTCP andUDPnetstat--
tcpornetstat–twilldisplay onlyTCPconnection
netstat --udp or netstat –u will display only
UDPconnectionnetstat -g will display all multicast network
subscribed by this host.slookup
If you know the IP address it will display hostname. To find all the IP addresses for a
givendomain name, the command nslookup is used. You must have a connection to the
internet forthisutilitytobeuseful.
E.g.nslookupblogger.com
YoucanalsousenslookuptoconverthostnametoIPAddressandfromIPAddressfromhostname.
traceroute
Ahandyutilitytoviewthenumberofhopsandresponsetimetogettoaremotesystemorwebsiteistracerout
e.Again youneedan internetconnectionto make use ofthistool.
finger
Viewuserinformation,displaysauser‘sloginname,realname,terminalnameandwritestatus.thisisprett
yoldunixcommandandrarelyusednow days.
telnet
Connectsdestinationhostviatelnetprotocol,iftelnetconnection establishon
Linux Programming Page 37
Department of CSE
10Mostimportantlinuxnetworkingcommands
Linux is most powerful operating system which often needs to use commands to explore
iteffectively.Some of the commands are restricted to normal user groups as they are powerful
andhasmorefunctionalityinvolvedinit.Herewesummarizedmostinterestingandusefulnetworkingco
mmandswhicheverylinuxuserare supposedtobe familiarwithit.
1. Arp manipulates the kernel‘s ARP cache in various ways. The primary options are
clearinganaddressmappingentryandmanuallysettingupone.Fordebuggingpurposes,thearpprogram
also allows a complete dump of the ARP cache.ARP displays the IP address assigned
toparticularETHcardandmacaddress
[fasil@smashtech]# arp
Address HWtypeHWaddress Flags Iface
59.36.13.1 ether C Masketh0
[fasil@smashtech~]#/sbin/ifconfig
eth0 UPBROADCASTRUNNINGMULTICASTMTU:1500Metric:1
RXpackets:126341errors:0dropped:0overruns:0frame:0T
X packets:44441 errors:0 dropped:0 overruns:0
carrier:0collisions:0txqueuelen:1000
3. Netstat prints information about the networking subsystem. The type of information which
isusually printed by netstat are Print network connections, routing tables, interface
statistics,masquerade connections,andmulticast.
[fasil@smashtech~]#netstat
ActiveInternet connections(w/o servers)
ProtoRecv-QSend-QLocalAddress ForeignAddress
Statetcp 0 0 .230.87:https
ESTABLISHED
ActiveUNIXdomainsockets(w/oservers)
ProtoRefCntFlags Type State I-NodePath
unix 10 [] DGRAM 4970/dev/log
unix 2 [] DGRAM 6625@/var/run/hal/hotplug_socket
unix 2 [] DGRAM 2952@udevd
unix 2 [] DGRAM 100564
unix 3 [] STREAM CONNECTED 62438/tmp/.X11-unix/X0
4. pingcommandisusedtochecktheconnectivityofasystemtoanetwork.Whenever
thereisprobleminnetworkconnectivitywe use pingtoensurethe systemis connectedtonetwork.
[root@smashtech~]#ping google.com
PING google.com(74.125.45.100)56(84)bytesofdata.
64 bytes from yx-in-f100.google.com (74.125.45.100): icmp_seq=0 ttl=241 time=295
ms64 bytes from yx-in-f100.google.com (74.125.45.100): icmp_seq=1 ttl=241 time=277
ms64bytes fromyx-in-f100.google.com(74.125.45.100):icmp_seq=2ttl=241time=277ms
---google.compingstatistics---
3packetstransmitted, 3received,0%packetloss,time6332ms
rttmin/avg/max/mdev=277.041/283.387/295.903/8.860ms,pipe2
5. Nslookup is a program to query Internet domain name servers. Nslookup has two
modes:interactive and non-interactive. Interactive mode allows the user to query name
servers forinformation about various hosts and domains or to print a list of hosts in a domain.
Non-interactivemodeisusedtoprintjust thenameandrequestedinformationforahostordomain.
[fasil@smashtech~]#nslookupgoogle.comS
erver: serverip
Address: gatewayip3
Non-authoritativeanswer:
Name:
google.com
Address:209.85.171.100
Name:
google.com
Address:74.125.45.100
Name:
google.com
Address:74.125.67.100
6. dig (domain information groper) is a flexible tool for interrogating DNS name servers.
Itperforms DNS lookups and displays the answers that are returned from the name server(s)
thatwere queried. Most DNS administrators use dig to troubleshoot DNS problems because of
itsflexibility, ease of use and clarity of output. Other lookup tools tend to have less
functionalitythandig.
;;QUESTION SECTION:
;google.com. IN A
;;ANSWERSECTION:
google.com. 122 IN A 74.125.45.100
google.com. 122 IN A 74.125.67.100
google.com. 122 IN A 209.85.171.100
;;AUTHORITYSECTION:
google.com. 326567 IN NS ns3.google.com.
google.com. 326567 IN NS ns4.google.com.
google.com. 326567 IN NS ns1.google.com.
google.com. 326567 IN NS ns2.google.com.
;;ADDITIONALSECTION:
ns1.google.com. 152216 IN A 216.239.32.10
ns2.google.com. 152216 IN A 216.239.34.10
ns3.google.com. 152216 IN A 216.239.36.10
ns4.google.com. 152216 IN A 216.239.38.10
;;Querytime: 92msec
;;SERVER:172.29.36.1#53(172.29.36.1)
;;WHEN:ThuMar514:38:452009
;;MSGSIZErcvd: 212
7. Route manipulates the IP routing tables. Its primary use is to set up static routes to
specifichosts or networks via an interface after it has been configured with the ifconfig
program.Whenthe add or del options are used, route modifies the routing tables. Without
these options, routedisplaysthecurrentcontentsofthe routingtables.
[fasil@smashtech~]#route
KernelIProutingtable
Destination Gateway Genmask FlagsMetricRef UseIface
Filters:
more COMMAND:
morecommandisusedtodisplaytextintheterminalscreen.Itallowsonlybackwardmovement.
SYNTAX:
TheSyntaxis
more[options]filename
OPTIONS:
-c Clearscreenbeforedisplaying.
-e Exitimmediatelyafterwritingthe lastlineofthe lastfile intheargumentlist.
-n Specifyhowmanylinesareprintedinthescreenforagivenfile.
+n Startsupthe filefromthegivennumber.
EXAMPLE:
1. more-cindex.php
Clearsthescreenbeforeprintingthefile.
2. more-3index.php
Printsfirstthreelinesofthegivenfile.PressEntertodisplaythefilelinebyline.
headCOMMAND:
headcommandisusedtodisplay thefirsttenlinesofafile,andalsospecifieshowmanylinestodisplay.
SYNTAX:
TheSyntaxis
head[options]filename
OPTIONS:
-n Tospecifyhowmanylinesyouwanttodisplay.
-nnumber Thenumberoption-argumentmustbeadecimalintegerwhosesignaffects
thelocationinthefile,measuredinlines.
Thenumberoption-argumentmustbeadecimalintegerwhosesignaffects
-cnumber
thelocationinthe file,measuredinbytes.
EXAMPLE:
1. headindex.php
Thiscommandprintsthefirst10linesof'index.php'.
2. head-5index.php
Theheadcommand displaysthefirst5linesof'index.php'.
3. head-c5index.php
Theabovecommanddisplaysthefirst5charactersof'index.php'.
tailCOMMAND:
tailcommandisusedtodisplaythelastorbottom partofthefile.Bydefaultitdisplayslast10lines
ofafile.
SYNTAX:
TheSyntaxis
tail[options]filename
OPTIONS:
-l Tospecifytheunitsoflines.
-b To specifytheunitsofblocks.
-n Tospecifyhowmanylinesyouwanttodisplay.
Thenumberoption-argumentmust beadecimalintegerwhosesignaffectsthe
-cnumber
locationinthe file,measured inbytes.
Thenumberoption-argumentmust beadecimalintegerwhosesignaffectsthe
-nnumber
locationinthefile,measuredinlines.
EXAMPLE:
1. tailindex.php
Itdisplaysthelast10linesof'index.php'.
2. tail-2index.php
Itdisplaysthelast2linesof'index.php'.
3. tail-n5index.php
Itdisplaysthelast5linesof'index.php'.
4. tail-c5index.php
Itdisplaysthelast5charactersof'index.php'.
cut COMMAND:
cut command is used to cut out selected fields of each line of a file. The cut command
usesdelimiterstodeterminewhere tosplitfields.
SYNTAX:
TheSyntaxisc
ut[options]
OPTIONS:
-c Specifiescharacterpositions.
-b Specifiesbytepositions.
-dflags Specifiesthedelimitersand fields.
EXAMPLE:
1. cut-c1-3text.txt
Output:
Thi
Cutthefirstthreelettersfromtheaboveline.
2. cut-d,-f1,2text.txt
Output:
Thisis,anexampleprogram
Theabovecommandisusedtosplitthefieldsusingdelimiterandcutthefirsttwofields.
paste COMMAND:
paste command is used to paste the content from one file to another file. It is also used to
setcolumnformatforeachline.
SYNTAX:
The Syntax
ispaste[opt
ions]
OPTIONS:
-s Pasteonefileatatimeinsteadofinparallel.
-d ReusecharactersfromLISTinstead ofTABs.
EXAMPLE:
1. pastetest.txt>test1.txt
Pastethecontent from'test.txt'fileto'test1.txt'file.
2. ls|paste----
Listallfilesanddirectoriesinfourcolumnsforeachline.
sortCOMMAND:
sortcommandisusedtosortthelinesinatextfile.
SYNTAX:
TheSyntaxis
sort[options]filename
OPTIONS:
-r Sortsinreverseorder.
-u Iflineisduplicateddisplayonlyonce.
-ofilename Sendssortedoutputtoafile.
EXAMPLE:
1. sorttest.txt
Sortsthe'test.txt'fileandprintsresultinthescreen.
2. sort -rtest.txt
Sortsthe'test.txt'fileinreverseorderandprintsresultinthescreen.
Aboutuniq
Reportorfilteroutrepeatedlines inafile.
Syntax
uniq[-c|-d|-u][ -ffields][-schar ][-n] [+m] [input_file[output_file]]
theinput.
-d Suppressthewritingoflinesthatarenotrepeatedintheinput.
-u Suppressthewriting oflinesthatarerepeatedintheinput.
-ffields Ignorethefirstfieldsfieldsoneachinputlinewhendoingcomparisons,where
fieldsisapositivedecimalinteger.Afieldisthemaximalstringmatchedbythebasic
regularexpression:
[[:blank:]]*[^[:blank:]]*
Iffieldsspecifiesmorefieldsthanappearonaninputline,anullstringwillbeusedforcomparis
on.
-n Equivalentto -ffieldswithfieldssetton.
+m Equivalentto-scharswithcharssettom.
input_file Apathnameoftheinputfile.Ifinput_fileisnotspecified,oriftheinput_fileis-
,thestandardinputwillbeused.
output_file Apathnameoftheoutputfile.Ifoutput_fileisnotspecified,thestandardoutputwillbe
used.Theresultsareunspecifiedifthefilenamedbyoutput_fileisthe
file namedbyinput_file.
Examples
uniq myfile1.txt > myfile2.txt - Removes duplicate lines in the first file1.txt and outputs
theresultstothesecondfile.
Abouttr
Translatecharacters.
Syntax
tr[-c][-d][-s][string1][string2]
-c Complementthesetofcharactersspecifiedbystring1.
-d Deletealloccurrencesofinputcharactersthatarespecifiedbystring1.
-s Replaceinstancesofrepeated characterswithasinglecharacter.
string1 Firststringorcharactertobechanged.
string2 Secondstringorcharactertochangethestring1.
Examples
echo "12345678 9247" | tr 123456789 computerh - this example takes an echo response
of'12345678 9247' and pipes it through the tr replacing the appropriate numbers with the letters.
Inthisexampleitwouldreturncomputer hope.
tr -cd '\11\12\40-\176' < myfile1 > myfile2 - this example would take the file myfile1 and
stripallnonprintablecharactersandtakethatresults tomyfile2.
TextprocessingutilitiesandBackuputilities:Textp
rocessingutilities:
cat:concatenate filesand printonthestandard
outputUsage:cat[OPTION][FILE]...
file2.txtcatn
file1.txt
echo:displayalineoftextUsage:
...eg.echoIlove India
echo $HOME
filesUsage:wc [OPTION]...[FILE]...
eg.wcfile1.txt
wc L
file1.txt
filesUsage:sort[OPTION]...
[FILE]...
eg.sortfile1.txts
ortr
file1.txt
GeneralCommands:
dateCOMMAND:
datecommandprintsthedateandtime.
SYNTAX:
TheSyntaxis
date[options][+format][date]
OPTIONS:
Slowlyadjust thetimebysss.fffseconds(fffrepresentsfractionsofasecond).
-a
Thisadjustmentcanbepositiveornegative.Onlysystemadmin/superuser
canadjustthetime.
-u Display(orset)thedateinGreenwichMeanTime(GMT-universaltime).
Format:
%a Abbreviatedweekday(Tue).
%A Fullweekday(Tuesday).
%b Abbreviatedmonthname(Jan).
%B Fullmonthname(January).
%c Country-specificdateandtimeformat..
%D Dateintheformat%m/%d/%y.
%j Juliandayofyear(001-366).
%n Insertanew line.
%p Stringtoindicatea.m.orp.m.
%t Tabspace.
%V Weeknumberinyear(01-52);startweekonMonday.
EXAMPLE:
datecommand
date
TheabovecommandwillprintWed Jul23 10:52:34IST2008
1. Tousetabspace:
date+"Dateis%D%t Timeis%T"
The above command will remove space and print
asDate is07/23/08Timeis10:52:34
2. Toknowtheweeknumberoftheyear,dat
e-V
Theabovecommandwillprint30
3. To setthedate,
date-s"10/08/200811:37:23"
TheabovecommandwillprintWedOct0811:37:23IST 2008
whoCOMMAND:
who command can list the names of users currently logged in, their terminal, the time
theyhavebeen loggedin,andthename ofthe hostfromwhichtheyhaveloggedin.
SYNTAX:
TheSyntaxis
who[options][file]
OPTIONS:
Printtheusernameoftheinvokinguser,The'am'and'i'mustbespace
ami
separated.
-b Printstimeoflastsystemboot.
-d printdeadprocesses.
-H Printcolumnheadingsabovetheoutput.
-i IncludeidletimeasHOURS:MINUTES.Anidletimeof. indicatesactivity
withinthelastminute.
-m Sameaswhoami.
-q Printsonlytheusernamesandtheuser count/totalnoofusersloggedin.
-T,-w Includeuser'smessagestatusintheoutput.
EXAMPLE:
1. who–Uh
Output:
NAME LINE TIME IDLE PIDCOMMENT
hiox ttyp3 Jul1011:08 . 4578
Thissampleoutputwasproducedat11a.m.The"."indiacatesactivitywithinthelastminute.
2. whoami
whoamicommand printstheuser name.
echoCOMMAND:
echocommandprintsthegiveninputstringtostandardoutput.
SYNTAX:
TheSyntaxis
echo[options..][string]
OPTIONS:
-n donotoutputthetrailingnewline
-e enableinterpretationofthebackslash-escapedcharacterslistedbelow
-E disableinterpretationofthosesequencesinSTRINGs
Without-E,thefollowingsequencesarerecognizedandinterpolated:
thecharacterwhoseASCIIcodeis NNN
\NNN
(octal)
\a alert(BEL)
\\ backslash
\b backspace
\c suppresstrailingnewline
\f formfeed
\n newline
\r carriagereturn
\t horizontaltab
\v verticaltab
EXAMPLE:
echocommand
echo"hscriptsHioxIndia"
TheabovecommandwillprintashscriptsHioxIndia
1. Tousebackspace:
echo-e"hscripts\bHiox\bIndia"
TheabovecommandwillremovespaceandprintashscriptsHioxIndia
2. Tousetabspaceinechocommandech
o-e"hscripts\tHiox\tIndia"
Theabovecommandwillprintashscripts Hiox India
passwdCOMMAND:
passwdcommandisusedtochangeyourpassword.
SYNTAX:
The Syntax
ispasswd[o
ptions]
OPTIONS:
-a Showpasswordattributesforallentries.
-l Lockspasswordentryforname.
Deletespasswordforname.Theloginnamewillnotbepromptedfor
-d
password.
Forcetheusertochangepasswordatthenextloginbyexpiring the password
-f
forname.
EXAMPLE:
1.passwd
Enteringjustpasswdwouldallowyoutochangethepassword.Afterenteringpasswdyouwillreceiv
ethefollowingthree prompts:
CurrentPassword:
NewPassword:
ConfirmNewPassword:
Each of these prompts must be entered correctly for the password to be
successfullychanged.
pwdCOMMAND:
pwd - Print Working Directory. pwd command prints the full filename of the current
workingdirectory.
SYNTAX:
TheSyntaxispw
d[options]
OPTIONS:
-P Thepathnameprintedwillnotcontainsymboliclinks.
-L Thepathnameprintedmaycontainsymboliclinks.
EXAMPLE:
SYNTAX:
TheSyntaxis
cal[options][month][year]
OPTIONS:
-1 Displayssinglemonthas output.
-3 Displaysprev/current/nextmonthoutput.
-s Displayssundayasthefirstdayofthe week.
-m DisplaysMondayasthefirstdayofthe week.
-j DisplaysJuliandates(daysone-based,numberedfromJanuary1).
-y Displaysacalendarforthecurrentyear.
EXAMPLE:
1.cal
Output:
September2008
SuMoTuWeThFrSa123
456
78910111213
14151617181920
21222324252627
282930
calcommanddisplaysthecurrentmonthcalendar.
2.cal-352008
Output:
April2008 May2008 June2008
SuMoTuWe ThFrSa SuMoTuWe ThFrSaSuMoTuWe ThFrSa12345
1231234567
678910111245678910891011121314
131415161718191112131415161715161718192021
202122232425261819202122232422232425262728
27282930 252627282930312930
Herethecalcommanddisplaysthecalendar ofApril,MayandJunemonthofyear2008.
loginCommand
Signsinto anewsystem.
Syntax
login[ -p][-ddevice] [-hhostname |terminal| -rhostname ] [name[environ] ]
-p Usedtopassenvironmentvariablestotheloginshell.
-ddevice login accepts a device option, device. device is taken to be the path name of
theTTY port login is to operate on. The use of the device option can be expected
toimprove login performance, since login will not need to call ttyname. The -
doption isavailableonlytouserswhoseUIDand effectiveUIDareroot.Any
otherattempttouse -dwillcauselogintoquietlyexit.
- Usedbyin.telnetdtopassinformationabouttheremotehostandterminaltype.
hhostname|te
rminal
-rhostname Used by in.rlogind to pass information
aboutthe remotehost.
Examples
logincomputerhope.com-Wouldattempttologintothecomputerhopedomain.
unamecommand
Printnameofcurrentsystem.
Syntax
uname[-a][-i][-m][-n][-p][-r][-s][-v][-X][-Ssystemname]
-a Printbasicinformationcurrentlyavailablefromthesystem.
-i Printthenameofthehardwareimplementation(platform).
-p Printthecurrenthost'sISAorprocessortype.
-r Printtheoperatingsystemreleaselevel.
-v Printtheoperatingsystemversion.
systemname,node, release,version,machine,andnumberofCPUs.
BusType,Serial,andUsers(setto"unknown"inSolaris)
OEM#andOrigin#(setto0and1,respectively)
Examples
uname-arv
List the basic system information, OS release, and OS version as shown
below.SunOS hope5.7 Generic_106541-08sun4msparc SUNW,SPARCstation-
10
uname -p
DisplaytheLinuxplatform.
SED:
Whatissed?
Anon-interactivestreameditor
Interpretssedinstructionsandperformsactions
Use sedto:
Automaticallyperformedits onfile(s)
Simplifydoingthesameeditsonmultiplefiles
Writeconversionprograms
SedCommandSyntax(SedScripts):
SedOperation
How Does sedWork?
sedreadslineofinput
lineof inputiscopiedintoatemporarybuffercalledpatternspace
editingcommandsareapplied
subsequentcommandsareappliedtolineinthepatternspace,nottheoriginalinputline
oncefinished,lineissenttooutput(unless–noptionwasused)
lineisremovedfrompatternspace
sedreadsnextlineofinput,untilendoffileNote
:inputfileisunchanged
sedinstructionformat(SedAddresses):
addressdetermineswhichlinesintheinputfilearetobeprocessedbythecommand(s)
ifnoaddressisspecified,thenthecommandisappliedtoeachinputline
addresstypes:
Single-Lineaddress
Set-of-Linesaddress
Rangeaddress
Nested
addressSingle-LineAddress
Specifiesonlyonelineintheinputfile
special:dollarsign($)denoteslastlineofinputfile
Examples:
showonlyline3
sed -n -e'3p' input-file
showonlylastline
sed -n -e'$p' input-file
subst itute―endif‖wit h―fi‖online10
sed-e'10s/endif/fi/'input-file
Set-of-LinesAddress
use regularexpressiontomatchlines
writtenbetweentwoslashes
processonlylinesthatmatch
maymatchseverallines
linesmayormaynotbeconsecutives
Examples:
sed-e‘/key/s/more/other/’input-
filesed-n-e‘/r..t/p’input-file
RangeAddress
DefinesasetofconsecutivelinesFo
rmat:
start-addr,end-addr
(inclusive)
Examples:
10,50 line-number,line-
number10,/R.E/ line-
number,/RegExp/
/R.E./,10 /RegExp/,line-number
/R.E./,/R.E/
/RegExp/,/RegExp/
Example:RangeAddress
%sed-n-e‘/^BEGIN$/,/^END$/p’input-file
PrintlinesbetweenBEGINandEND,inclusive
BEGIN
Line 1 of
inputLine 2 of
inputLine3 of
inputEND
Line 4 of
inputLine 5 of
inputNestedAd
dress
Nested address contained within another
addressExample:
printblanklinesbetweenline20and30
20,30{
/^$/p
}
Addresswith!
addresswithanexclamationpoint(!):
instructionwillbeappliedtoalllinesthatdonotmatchtheaddressExample
:
printlinesthatdonotcontain―obso lete‖
sed-e‘/obsolete/!p’input-
filesedcommands
LineNumber
line
numbercommand(=)writesthecurrentlinenumberbeforeeachmatched/outputlineExamples:
sed-e'/Two-thirds-
Linux Programming Page 65
Department of CSE
time/='tuition.datased-e '/^[0-9][0-
9]/='inventory
modifycommands
InsertCommand:i
addsoneormorelines directlytotheoutputbeforetheaddress:
inserted―text‖never appearsinsed‘spatternspace
cannot be used with a range address; can only be used with the single-line and set-
of-linesaddresstypes
Syntax:
[address]i\t
ext
AppendCommand:a
addsoneormorelinesdirectlytotheoutputaftertheaddress:
Similartotheinsertcommand(i),appendcannotbeusedwitharangeaddress.
Appended―text‖doesnotappearin sed‘spatternspace.
Syntax:
[address]a\t
ext
ChangeCommand:c
replacesanentirematchedlinewithnewtext
acceptsfouraddresstypes:
single-line,set-of-line,range,andnestedaddresses.
Syntax:
[address1[,address2]]
c\
text
DeleteCommand:d
deletestheentirepatternspace
commandsfollowingthedeletecommandareignoredsincethedeletedtextisno
longerinthepatternspace
Syntax:
[address1[,address2]]
dSubstitute Command
(s)Syntax:
[addr1][,addr2]s/search/replace/[flags]
replacestextselectedbysearchstringwithreplacementstring
searchstringcanberegularexpression
flags:
global(g),i.e.replacealloccurrences
specificsubstitutioncount(integer),default1Re
gularExpressions:usewithsed
SubstitutionBackReferences
Example:Replacement String&
$catdatafile
PatriciaHemenway 4.0 .7 4 17
$sed-e‘s/[0-9][0-9]$/&.5/’datafile
TransformCommand(y)Syn
tax:
[addr1][,addr2]y/a/b/
translatesonecharacter'a'toanother'b'
cannot useregularexpressionmetacharacters
cannotindicatearangeofcharacters
similarto―tr‖command
Example:
$sed-e‘1,10y/abcd/wxyz/’datafile
sedi/o commands
Input(next)Command:nandN
Forcessedtoreadthenextinputline
Copiesthecontentsofthe patternspacetooutput
Deletesthecurrentlineinthepatternspace
Refillsitwiththenextinputline
Continueprocessing
N(uppercase)Command
addsthenextinputlinetothecurrentcontentsofthepatternspace
usefulwhenapplyingpatternstotwoormorelinesatthesametimeOutput
Command:pandP
Print Command(p)
copiestheentirecontentsofthepatternspacetooutput
willprintsamelinetwiceunlesstheoption―–n‖isused
Printcommand:P
printsonlythefirstlineofthepattern space
printsthecontentsofthepatternspaceuptoandincludinganewlinecharacter
anytextfollowingthe
firstnewlineisnotprintedListCommand(l)
Thelistcommand:l
showsspecialcharacters(e.g.tab,etc)
Theoctaldumpcommand(od-
c)canbeusedtoproducesimilarresultHoldSpace
temporarystoragearea
used tosavethecontentsofthepatternspace
4commandsthatcanbeusedtomovetextbackandforthbetweenthepatternspaceandthe
holdspace:
h,H
g,G
Filecommands
allowstoreadandwritefrom/tofilewhileprocessingstandardinput
read:rcommand
write: w
commandReadFilecomm
andSyntax:rfilename
queuethe contentsoffilename tobereadandinserted intothe output streamat
theendofthecurrentcycle,orwhenthenextinputlineis read
iffilenamecannotberead,itistreatedasifitwereanemptyfile,withoutanyerrorindicat
ion
singleaddressonly
Write Filecommand
Syntax:wfilename
Writethepatternspacetofilename
Thefilenamewillbecreated(ortruncated)beforethefirstinputlineisread
allwcommandswhichrefer
tothesamefilenameareoutputthroughthesameFILEstream
BranchCommand(b)
ChangetheregularflowofthecommandsinthescriptfileSynta
x:[addr1][,addr2]b[label]
Branch(unconditionally)to‗label‘orendofscript
If―label‖issupplied,execut ionresumes atthelinefo llowing:label;otherwise,
controlpasses totheendofthescript
Branchlabel
:mylabel
Example:Thequit(q)
CommandSyntax:[addr]q
Quit(exitsed)whenaddrisencountered.
Example:Displaythefirst50linesandquit
%sed-e’50q’datafile
Sameas:
%sed-n -e‘1,50p’datafile
%head-50datafile
Awk
Whatisawk?
createdby:Aho,Weinberger,andKernighan
scriptinglanguageusedformanipulatingdataandgeneratingreports
versionsofawk
scansafilelinebyline
splitseachinput lineintofields
comparesinputline/fieldstopattern
performsaction(s)onmatchedlines
Usefulfor:
transformdatafiles
produce formattedreports
Programmingconstructs:
formatoutputlines
arithmeticandstringoperations
conditionals and
loopsThe Command:awk
BasicawkSyntax
awk[options]‘script’file(s)
awk[options]–fscriptfilefile(s)
Options:
-F tochangeinputfieldseparator
-f
tonamescriptfileB
asic awkProgram
Example:
awk'/for/'testfile
printsalllines containingstring―for‖intest file
Basic Terminology:inputfile
Afieldisaunitofdatainaline
Eachfieldisseparatedfromtheotherfields bythefieldseparator
defaultfieldseparatoriswhitespace
Arecord isthecollectionoffieldsinaline
A data file is made up of
recordsExample InputFile
Buffers
awksupportstwotypesofbuffers:rec
ordandfield
fieldbuffer:
oneforeachfields inthecurrentrecord.
names: $1,$2,…
recordbuffer:
$0 holds the entire
recordSomeSystemVariables
FS Fieldseparator(default=whitespace)
RS Recordseparator(default=\n)
NF Numberoffields incurrentrecord
NR Numberofthecurrent record
OFS Outputfieldseparator(default=space)
(default=\n)FILENAME Currentfilename
Example:RecordsandFields
%catemps
%awk'{printNR,$0}' emps
Example:SpaceasFieldSeparator
%catemps
%awk
'{printNR,$1,$2,$5}'emps1TomJone
s 543354
2 Mary Adams28765
3 SallyChang650000
4 BillyBlack336500
Example:Colonas FieldSeparator
%catem2
TomJones:4424:5/12/66:543354
Mary
Adams:5346:11/4/63:28765Sally
Chang:1654:7/22/54:650000Billy
Black:1683:9/23/44:336500
em2TomJones 4424
awkScripts
awkscriptsaredividedintothreemajorparts:
commentlinesstartwith#
BEGIN:pre-processing
performsprocessingthatmustbecompletedbeforethefile
processingstarts(i.e.,before awk startsreadingrecordsfromtheinputfile)
usefulforinitializationtaskssuchastoinitializevariablesandtocreatereporthea
dings
BODY:Processing
containsmainprocessinglogictobeappliedtoinputrecords
likealoopthatprocessesinputdataonerecordatatime:
ifafilecontains100records,the bodywillbe
executed100times,oneforeachrecord
END:post-processing
containslogictobeexecutedafterallinput datahavebeenprocessed
logicsuchasprintingreportgrandtotalshouldbeperformedinthispartofthescrip
t
Pattern/Action Syntax
CategoriesofPatterns
ExpressionPatterntypes
match
entire inputrecord
o regularexpressionenclosedby‗/‘s
explicitpattern-matchingexpressions
o ~(match),!~(notmatch)
o expressionoperators
arithmetic
relational
logical
%catemployees2
TomJones:4424:5/12/66:543354
Mary
Adams:5346:11/4/63:28765Sally
Chang:1654:7/22/54:650000
BillyBlack:1683:9/23/44:336500
%awk–F:'/00$/'employees2Sally
Chang:1654:7/22/54:650000Billy
Black:1683:9/23/44:336500
Example:explicitmatch
%catdatafile
%awk'$5 ~ /\.[7-9]+/'datafile
Examples:matchingwithREs
datafilenorthwestNW
southwest
SWsouthernS
OnorthNO
centralCT
%awk'/^[ns]/{print$1}'datafileno
rthwest
southwest
southerns
outheastn
ortheastn
orth
ArithmeticOperators
+ Add x+y
- Subtract x–y
* Multiply x*y
/ Divide x/y
% Modulus x%y
^ Exponential x^y
Example:
%awk'$3*$4>500{print$0}'file
RelationalOperators
Operator Meaning Example
== Equalto x==y
!= Notequalto x!=y
~ Matchedbyregexp x~ /y/
!~ Notmatchedbyreqexp x!~/y/
LogicalOperators
Operator Meaning Example
|| LogicalOR a ||b
! NOT !a
Examples:
%awk'($2>5)&&($2<=15) {print$0}'file
%awk'$3==100||$4>50'file
RangePatterns
Matchesrangesofconsecutive
inputlinesSyntax:
pattern1, pattern2{action}
patterncanbeanysimple pattern
pattern1turns actionon
pattern2turnsactionoff
69
Department of CSE
RangePatternExample
awkActions
awkexpressions
Expressionisevaluatedandreturnsvalue
consistsofanycombinationofnumericandstringconstants,variables,operators,fu
nctions,andregularexpressions
Caninvolvevariables
As part ofexpressionevaluation
As target ofassignmentawkvariables
Ausercandefineanynumberofvariables withinanawkscript
Thevariablescanbenumbers,strings,orarrays
Variablenamesstartwithaletter, followedbyletters,digits,andunderscore
Variablescomeintoexistencethefirsttimetheyarereferenced;therefore,theydonotneedtob
edeclaredbeforeuse
Allvariablesareinitiallycreatedasstringsandinitializedtoanullstring―‖awkVaria
bles
Format
variable=expression
70
Department of CSE
Examples:
%awk'$1~/Tom/
{wage=$3*$4;printwage}' filename
%awk'$4=="CA" {$4="California";print$0}' filename
awkassignment operators
= assignresultofright-hand-
sideexpressiontoleft-hand-sidevariable
++ Add1tovariable
-- Subtract1fromvariable
+= Assignresultofaddition
-= Assignresultofsubtraction
*= Assignresultofmultiplication
/= Assignresultofdivision
%= Assignresultofmodulo
^=
AssignresultofexponentiationA
wkexample:
File:grades
john8592789488
andrea 8990759086
jasper8488809284
awkscript:average
#averagefivegrades
{total=$2+$3+$4+$5+$6avg=t
otal/5
print$1,avg}
Runas:
awk–faveragegrades
71
Department of CSE
OutputStatements
72
Department of CSE
printeasyand simpleoutput
printfs
print
printf
formatted(similartoCprintf)formats
tring(similar toCsprintf)
Function:print
Writestostandardoutput
OutputisterminatedbyORS
defaultORSisnewline
Ifcalledwithnoparameter,itwillprint$0
PrintedparametersareseparatedbyOFS,
defaultOFS isblank
Printcontrolcharactersareallowed:
\n\f\a\t\\ …printexample
%awk
'{print}'gradesjohn859
2789488
andrea 8990759086
%awk'{print$0}'gradesjo
hn8592789488
andrea 8990759086
% awk '{print($0)}'
gradesjohn8592789488
andrea 8990759086
73
Department of CSE
Redirectingprintoutput
74
Department of CSE
Printoutput
goestostandardoutputunless redirectedvia:
> ―file‖
>>―file‖
|―command‖
willopenfileorcommandonlyonce
subsequent redirections append to already open
streamprintExample
%catfilej
ohn85
andrea89
jasper84
%awk'{print$1,$2|"sort"}'gradesandr
ea89
jasper84
john85
%awk'{print$1,$2|"sort–
k2"}'gradesjasper84
john85
andrea89
%date
WedNov 1914:40:07CST2008
75
Department of CSE
%date|
awk'{print"Month:"$2"\nYear:",$6}'Mo
0nth:Nov
Year:2008
printf:FormattingoutputSyn
tax:
printf(format-string,var1,var2,…)
workslike Cprintf
eachformatspecifierin―format-string‖ requires argumentofmatchingtype
Formatspecifiers
%d%i decimalinteger
%c singlecharacter
%s stringofcharacters
%f floatingpointnumber
%o octalnumber
%x hexadecimalnumber
%e scientific floatingpointnotation
Formatspecifierexamples
Formatspecifiermodifiers
between―%‖andletter
76
Department of CSE
%10s
%7d
%10.4f
%-20s
meaning:
widthoffield,fieldisprintedrightjustified
precision:numberofdigitsafterdecimalpoint
―-‖willleftjust ifysprintf:Formattingtext
Syntax:
sprintf(format-string,var1,var2,…)
Example:
{
text = sprintf("1: %d – 2: %d", $1,
$2)printtext
awkArray
awkallows one-
dimensionalarraystostore
stringsornumbers
indexcanbenumberorstring
arrayneednotbedeclared
77
Department of CSE
itssize
itselements
78
Department of CSE
arrayelementsarecreatedwhenfirstused
init ializedto0or―‖Ar
raysinawk
Syntax:
arrayName[index]=value
Examples:
list[1]="one"list[2]
="three"
Illustration:AssociativeArrays
awkarrayscanusestring asindex
Awkbuiltinsplitfunctionss
plit(string,array, fieldsep)
dividesstringintopiecesseparatedbyfieldsep,andstoresthepiecesinarray
ifthefieldsepisomitted, thevalueofFSisused.
Example:
split("auto-da-fe",a,"-")
79
Department of CSE
setsthecontentsofthearrayaasfollows:
a[1]="auto"
a[2]="da"
a[3]="fe"
Example:processsalesdata
inputfile:
output:
80
Department of CSE
summaryofcategorysalesIllustration:processeachinputline
Illustration:processeachinput line
Summary:awkprogram
81
Department of CSE
Example:completeprogram
%catsales.awk
{
deptSales[$2]+=$3
}
END {
for(xindeptSales)
printx,deptSales[x]
}
% awk –f sales.awk
salesawkbuiltinfunctionst
olower(string)
123")returns"mixedcase 123"
toupper(string)
case.awk Example:listofproducts
103:sway
bar:49.991
01:propeller:104.99104:fis
hing
line:0.9911
3:premium
82
Department of CSE
fishbait:1.00106:cup
holder:2.49
107:cooler:14.89
83
Department of CSE
112:boat
cover:120.00109:trans
om:199.00
110:pulley:9.88
105:mirror:4.99
108:wheel:49.99
111:lock:31.00
102:trailerhitch:97.95a
wkExample:outputMari
nePartsRUsMaincatalo
Part-idname price
======================================
84
Department of CSE
85
Department of CSE
======================================
Cataloghas13partsawk
Example:
completeBEGIN {
FS=":"
Us"print"Maincatalog"
print"Part-id\tname\t\t\tprice"
print"======================================"
printf("%3d\t%-
20s\t%6.2f\n",$1,$2,$3)count++
END {
print"======================================"
Applications:
Awkcontrolstructures
Conditional
□ if-else
Repetition
□ for
withcounter
witharrayindex
□ while
□ do-while
□ also:break,continuei
fStatement
Syntax:
if(conditionalexpression)s
tatement-1
else
statement-2
Example:
if(NR<3)
print $2
else
print$3
87
Department of CSE
for
Loop
Syntax:
for(initialization;limit-
test;update)statement
Example:
for(i=1;i<=NR;i++)
total
+=$icount
++
arraysSyntax:
for(varinarray)st
atement
Example:
for(xindeptSales)
printx,deptSales[x]
}
88
Department of CSE
While
Loop
Syntax:
89
Department of CSE
while(logicalexpression)s
tatement
Example:
i=1
while(i<=NF)
printi,$ii
++
do-
whileLoopSyn
tax:
do
statement
while(condition)
statementisexecutedatleast once,evenif
conditionisfalseatthebeginningExample:
i=1d
o{
print$0i
++
90
Department of CSE
}while (i <=10)
91
Department of CSE
loop controlstatements
break
exits loop
continue
skipsrestofcurrentiteration,continueswithnextiteration
ShellProgramming
The shell has similarities to the DOS command processor Command.com (actually Dos
wasdesign as a poor copy of UNIX shell), it's actually much more powerful, really a
programminglanguageinits ownright.
A shell is always available on even the most basic UNIX installation. You have to go through
theshell to get other programs to run. You can write programs using the shell. You use the shell
toadministrate yourUNIXsystem.Forexample:
ls-al|more
is a short shell program to get a long listing of the present directory and route the output
throughthe morecommand.
What isaShell?
A shell is a program that acts as the interface between you and the UNIX system, allowing
youtoentercommandsfortheoperatingsystemtoexecute.
Herearesomecommonshells.
92
Department of CSE
Introduction- WorkingwithBourneShell
• The Bourne shell, or sh, was the default Unix shell of Unix Version 7. It was
developedbyStephenBourne,ofAT&TBellLaboratories.
• A Unix shell, also called "the command line", provides the traditional user interface
forthe Unix operating system and for Unix-like systems. Users direct the operation of
thecomputerbyenteringcommandinputastextforashelltoexecute.
• Therearemanydifferentshellsinuse.Theyare
– Bourneshell(sh)
– Cshell(csh)
– Korn shell
(ksh)BourneAgainshell(bash)
• Whenweissueacommandtheshellisthefirstagencytoacquiretheinformation.Itaccepts
and interprets user requests. The shell examines &rebuilds the commands&leaves
the execution work to kernel. The kernel handles the h/w on behalf
ofthesecommands &allprocessesinthesystem.
• The shell is generally sleeping. It wakes up when an input is keyed in at the prompt.
Thisinputisactuallyinputtotheprogramthatrepresentstheshell.
•
Shellresponsibilities
1. ProgramExecution
2. VariableandFilenameSubstitution
3. I/ORedirection
4. PipelineHookup
5. EnvironmentControl
6. InterpretedProgrammingLanguage
1.ProgramExecution:
• Theshellisresponsiblefortheexecutionofallprogramsthatyou
requestfromyourterminal.
• Each time you type in a line to the shell, the shell analyzes the line and
thendetermineswhattodo.
93
Department of CSE
• The line that is typed to the shell is known more formallyas the command line. The
shellscans this command line and determines the name of the program to be executed
andwhatargumentstopasstotheprogram.
2. VariableandFilenameSubstitution:
• Like any other programming language, the shell lets you assign values to
variables.Whenever you specify one of these variables on the command line, preceded
by adollarsign,theshellsubstitutes thevalueassignedtothevariableatthatpoint.
3. I/ORedirection:
• It is the shell's responsibility to take care of input and output redirection on the
commandline.Itscansthecommandlinefor
theoccurrenceofthespecialredirectioncharacters<,
>,or >>.
4. PipelineHookup:
• Just as the shell scans the command line looking for redirection characters, it also
looksfor the pipe character |. For each such character that it finds, it connects the
standardoutput from the command preceding the | to the standard input of the one
following the|.Ittheninitiates executionofbothprograms.
5. EnvironmentControl:
• The shell provides certain commands that let you customize your environment.
Yourenvironmentincludeshomedirectory,thecharactersthattheshelldisplaystopromptyou
94
Department of CSE
totypeinacommand,andalistofthedirectoriestobesearchedwheneveryourequestthataprogra
mbeexecuted.
6. InterpretedProgrammingLanguage:
PipesandRedirection
Pipesconnectprocessestogether.TheinputandoutputofUNIXprogramscanberedirected.
RedirectingOutput
The>operator isusedtoredirectoutputofaprogram.Forexample:ls-
l>lsoutput.txt
redirectstheoutputofthelistcommandfromthescreentothefilelsoutput.txt.
To0appendtoafile,usethe>>operator.
ps>>lsoutput.txt
RedirectingInput
example:more <killout.txt
Pipes
We can connect processes together using the pipe operator ( | ). For example, the
followingprogrammeansrunthepsprogram,sortitsoutput,and saveitinthefile pssort.out
HereDocuments
95
Department of CSE
Aheredocumentisaspecialwayofpassinginputtoacommandfromashellscript.Thedocumentstar
tsandendswiththe sameleaderafter<<.Forexample:
#!/bin/sh
HowIt Works
Itexecutestheheredocumentasifitwereinputcommands.
RunningaShellScript
InteractivePrograms
LanguageCreatingaScript
Tocreateashellscriptfirstuseatexteditortocreateafilecontainingthecommands.Forexample,typ
e thefollowingcommandsandsave themasfirst.sh
96
Department of CSE
Note:commandsstart witha#.
Theline
#!/bin/sh
isspecialandtellsthesystemtousethe /bin/shprogramtoexecutethisprogram.
Thecommand
exit0
Causesthescriptprogramtoexitandreturnavalueof0,whichmeanstherewerenoterrors.
MakingaScript Executable
There are two ways to execute the script. 1) invoke the shell with the name of the script file as
aparameter,thus:
/bin/shfirst.sh
Or2)changethemodeofthe scripttoexecutableandthenafterexecuteitbyjusttypingitsname.
chmod+xfirst.shfi
rst.sh
Actually,youmayneedtotype:
./first.sh
tomakethefileexecuteunlesthepathvariablehas your directoryinit.
ShellSyntax
ThemodernUNIXshellcanbeusedtowritequitelarge, structuredprograms.
Shellmetacharacters
The shell consists of large no. of metacharacters. These characters plays vital role in
Unixprogramming.
Typesofmetacharacters:
97
Department of CSE
1.Filesubstitution2.
I/Oredirection3.Pro
cessexecution
4. Quoting
metacharacters5.Position
al parameters6.Special
characters7.Command
substitutionFilenamesub
stitution:
Thesemetacharactersareusedtomatchthefilenamesinadirectory.
Metacharacter significance
? matchesasinglecharacter
[ijk] matchesasinglecharactereitheri,j,k
[!ijk] matchesasinglecharacterthatisnotanI,j,k
ShellVariables
Variablesaregenerallycreatedwhenyoufirstusethem.Bydefault,allvariablesareconsideredandstored
asstrings.Variable names are casesensitive.
Ucandefine&usevariablesbothinthecommandlineandshellscripts.Thesevariablesare
calledshellvariables.
98
Department of CSE
99
Department of CSE
VariablesinUnixareoftwotypes.
1) User-definedvariables:
Generalizedform:
variable=value.
Eg:$x=10
$echo$x
10
Toremoveavariableuseunset.
$unsetx
Allshellvariablesareinitializedtonullstringsbydefault.Toexplicitlysetnullvaluesuse
x= or x=‗‘ or x=―‖
Toassignmultiwordstringstoavariableuse
$msg=‗uhaveamail‘
2) EnvironmentVariables
Theyareinitializedwhentheshellscriptstartsandnormallycapitalizedtodis
tinguishthemfromuser-definedvariablesinscripts
Todisplayallvariablesinthelocalshellandtheirvalues,typethesetcommand
Theunsetcommandremovesthevariablefromthecurrentshellandsubshell
EnvironmentVariables Description
$HOME Homedirectory
$PATH Listofdirectoriestosearchforcommands
$PS1 Commandprompt
$PS2 Secondaryprompt
$SHELL Currentloginshell
$0 Nameoftheshellscript
$# No. ofparameterspassed
100
Department of CSE
$$ ProcessIDoftheshellscript
CommandsubstitutionandShellcommands:
read:
$readname
variablename.printf:
$printf"Thisisanumber:%d\n"10Thi
sisanumber:10
Printfsupportsconversionspecificationcharacterslike %d,%s,%x
,%o….Exitstatusofacommand:
o Everycommandreturnsavalueafterexecution.Thisvalueiscalledtheexitstatusor
returnvalueofacommand.
o Thisvalueissaidtobetrueifthecommandexecutessuccessfullyandfalseifitfails.
o Thereisspecialparameterusedbytheshellitisthe$?.Itstorestheexitstatusofaco
mmand.
exit:
o Theexitstatementisusedtoprematurelyterminateaprogram.Whenthisstatement is
encountered in a script, execution is halted and control is returned tothe
callingprogram-inmostcases the shell.
o U don‘t need to place exit at the end of every shell script because the
shellknowswhenscriptexecutioniscomplete.
set:
Setisusedtoproducethelistofcurrentlydefinedvariables.
$set
101
Department of CSE
Setisused toassignvaluestothepositionalparameters.
$setwelcometoUnix
Thedo-nothing(:)Command
Itisanullcommand.
In some older shell scripts, colon was used at the start of a line to introduce
acomment,butmodernscriptsuses #now.
expr:
Theexprcommandevaluatesitsarguments asanexpression:
$expr8+6
$x=`expr12/4`
$echo$x3
export:
Thereisawaytomakethevalueofavariableknown
toasubshell,andthat'sbyexportingitwiththeexportcommand.Theformatofthiscomm
andis
exportvariables
where variables is the list of variable names that you want exported. For any
subshellsthatgetexecutedfromthatpointon,thevalueoftheexportedvariableswillbepasse
ddowntothe subshell.
eval:
evalscansthecommandlinetwicebeforeexecutingit.Generalformforevalisevalco
mmand-line
Eg:
$catlast
evalecho\$$#
$lastonetwothreefourfou
r
${n}
${n} mustbeused.Sotodirectlyaccessargument10,youmustwrite
102
Department of CSE
${10}
Shiftcommand:
The shift command allows u to effectively left shift your positional parameters. If u
executethe command
Shift
whateverwaspreviouslystoredinside$2willbeassignedto$1,whateverwaspreviouslystoredin$
3 willbeassignedto$2,and soon.Theold valueof$1willbeirretrievablylost.
TheEnvironment-EnvironmentVariables
Itcreatesthevariablesalutation,displaysitsvalue,andsomeparametervariables.
• Whenashellstarts, somevariablesareinitializedfromvaluesintheenvironment.Hereis
asampleofsomeofthem.
ParameterVariables
• If yourscriptisinvokedwithparameters,someadditionalvariables arecreated.
Quoting
Normally,parametersareseparated bywhite
space,suchasaspace.Singlequotmarkscanbeusedtoenclosevaluescontaining
space(s).Typethefollowingintoafilecalledquot.sh
103
Department of CSE
makesuretomakeitexecutablebytypingthecommand:
< chmod
a+xquot.shT
heresultsofexecuting
104
Department of CSE
thefileis:
HowIt Works
The variable myvar is created and assigned the string Hi there. The content of the variable
isdisplyed using the echo $. Double quotes don't effect echoing the value. Single quotes
andbackslashdo.
Thetest,or[]Command
Here is how to check for the existance of the file fred.c using the test and using the
[]command.
You can even place the then on the same line as the if, if youu add a semicolon before
thewordthen.
Herearetheconditontypesthatcanbeusedwiththetestcommand.Therearestringcomparison.
Therearearithmeticcomparison.
105
Department of CSE
Therearefileconditions.
ControlStructures
if
Theifstatementisvarysimilarotherprogramminglanguages exceptitends
withafi.ifcondition
then
statements
else
statements
fi
elif
theelifisbetterknownas"elseif".Itreplacestheelsepartofanifstatementwithanotherifstatement.Youc
antryitoutbyusingthe followingscript.
106
Department of CSE
#!/bin/sh
echo"Is
itmorning?Pleaseansweryesorno"readtimeofd
ay
if[$ti0meofday="yes"]the
n
echo"Goodmorning"eli
f [ $timeofday = "no" ];
thenecho"Goodafternoon"
else
echo"Sorry,$timeofdaynotrecognized.Enteryesorno
" exit1fi
exit0
HowIt Works
Theabovedoesasecondtestonthevariabletimeofdayifitisn'tequaltoyes.
AProblemwithVariables
Ifavariableissettonull, thestatement
if[$timeofday="yes"]if[
lookslike
="yes" ]
107
Department of CSE
which is illegal. This problem can be fixed by using double quotes around the variable
name.if["$timeofday" ="yes" ]
.
for
Theforconstructisusedforloopingthrougharangeof values,whichcanbeanysetofstrings.The
syntaxis:
forvariableinvaluesd
o
statements
done
Tryoutthefollowingscript:
#!/bin/sh
HowIt Works
Theaboveexamplecreatesthevariablefooandassignsitadifferentvalueeachtimearoundtheforloop.
HowIt Works
Here is another script which uses the $(command) syntax to expand a list to chap3.txt,
chap4.txt,andchap5.txtandprintthefiles.
#!/bin/sh
forfilein $(ls
chap[345].txt);dolpr$file
done0
while
whileconditiondo
statements
done
108
Department of CSE
Hereisawhilloop
thatloops20times.#!/bin/
sh
foo=1
while["$foo"-le20]do
doneexit0
HowIt Works
echo"Herewegoagain" foo=$(($foo+1))
Theabovescriptusesthe[ ]commandtotestfoofor<=thevalue20.Thelinefoo=$(($fo0o+1))
incrementsthevalueoffooeachtimetheloopexecutes..
until
Theuntilstatementloopsuntilaconditionbecomestrue!Itssyntaxis:
untilcondition
do
statements
done
Here isascriptusinguntil.
#!/bin/sh
untilwho|grep "$1">/dev/nulldo
Sl0eep60
done
echo-e\\a
echo"****$1hasjustlooggedin****"exit
case
Thecasestatementallowsthetestingofavariableformorethenonevalue.Thecasestatementendswiththe
wordesac.Itssyntaxis:
casevariablein
pattern[|pattern]...)statements;;pat
tern[|pattern]...) statements;;
...
esac
109
Department of CSE
Hereisasamplescriptusingacasestatement:
#!/bin/sh
echo"Is
itmorning?Pleaseansweryesorno"readtimeofd
ay
case"$timeofday" in
"yes") echo "Good
Morning";;"no" ) echo "Good
Afternoon";;0"y" ) echo "Good
Morning";;"n")echo"GoodAfter
noon";;
*)echo"Soory,answernotrecognized";;
esace
xit0
Thevalueinthevaraibletimeofdayiscomparedtovariousstrings.Whenamatchismade,theassociatedec
hocommandisexecuted.
Here is a case where multiple strings are tested at a time, to do the some
action.case "$timeofday"in
"yes"| "y"| "yes"| "YES") echo
"goodMorning";;"n"*|"N"*)<echo"GoodAfternoon
";;
*)<echo"Sorry, answernotrecognized";;
0esac
HowIt Works
Theabovehassever stringstestedforeachpossiblestatement.
case.case "$timeofday"in
"yes"|"y"|"Yes"|"YES")echo"
GoodMorning"
echo "Upbrightandearlythismorning"
;;
H [nN]*)
o
w
It *)
W
o
r
k
esac s
110
Department of CSE
s
echo"GoodAfternoon" e
;; a
n
e s
c w
h e
o r
y
" e
S s
o o
r r
r n
y o
, o
"
a e
n x
s i
w t
e 1
r ;;
n
o
t
r
e
c
o
g
n
i
z
e
d
"
e
c
h
o
"
P
l
e
a
111
Department of CSE
When a match is found to the variable value of timeofday, all the statements up to the ;;
areexecuted.
Arithmeticinshell
The $((...)) is a better alternative to the expr command, which allows simple
arithmeticcommandstobeprocessed.
x=$(($x+1))
ParameterExpansion
expansion.#!/bin/sh
foriin12do
my_secret_process${i}_tmp
done
Herearesomeoftheparameter expansion
HowIt Works
Thetryitoutexerciseusesparameterexpansiontodemonstratehowparameter expansionworks.
ShellScriptExamplesEx
ample
#!/bin/sh
echo"Is
itmorning?(Answeryesorno)"readtimeofd
ay
112
Department of CSE
if[$timeofday="yes"];then
113
Department of CSE
echo "GoodMorning"
else
echo"Goodafternoon"
fiexit
elif-DoingfurtherChecks
#!/bin/sh
echo"Is
itmorning?Pleaseansweryesorno"readtimeofd
ay
if[$timeofday="yes"];thenecho"G
ood Morning"
elif[$timeofday="no"
];thenecho
"Goodafternoon"
else
echo"Wronganswer!Enteryesorno"
exit1
fi exit0
InterruptProcessing-trap
Thetrapcommandisusedforsecifyingtheactionstotakeonreceiptofsignals.Itsyntaxis:trapcommand
signal
Herearesomeofthesignals.
114
Department of CSE
HowIt Works
Thetry itoutsectionhasyoutypeinashellscripttotestthetrapcommand.Itcreatesafileand
115
Department of CSE
keepssayingthatitexistsuntilyouucauseacontrol-Cinterrupt.Itdoesitallagain.
Functions
is:function_name (){
statements
}0
Here isasample functionanditsexecution.
#!/bin/sh
foo(){
echo"Functionfooisexecuting"
}
116
Department of CSE
echo"scriptstarting"f
oo
echo"scriptended"e
xit0
HowIt Works
When the above script runs, it defines the funcion foo, then script echos script starting, then
itrunsthefunctionsfoowhichechosFunctionfooisexecuting,thenitechoscriptended.
Hereisanothersamplescriptwithafunctioninit.Saveitasmy_name#!/bin/sh
yes_or_no(){
echo"Parametersare
$*"whiletrue
do
echo-n"Enteryes
orno"readx
0case"$x" in
y|yes)return0;;n|n
o)return1;;
*) echo"Answeryesorno"
esac
done
}
echo"Originalparametersare$*"
ifyes_or_no"ISyournaem$1"the
n
echo"Hi$1"
else
echo"Nevermind"
fi
0exit0
HowIt Works
117
Department of CSE
Whenmy_nameisexecutewiththestatement:
my_nameRickandNeil
. givestheoutputof:
Original parameters are Rick and
NeilParametersareIsyournameRickEnt
eryes orno
no
Nevermind
Commands
You can execute normal command and built-in commands from a shell script. Built-
incommandsaredefinedandonlyruninside ofthescript.
break
Itisusedtoescapefromanenclosingfor,whileoruntilloopbeforethecontrollingconditionhasbeenmet.
The: Command
Thecoloncommandisanullcommand.Itcanbeusedfor analiasfortrue..
Continue
The continue command makes the enclosing for, while, or until loop continue at the
nextiteration.
TheCommand
Thedotcommandexecutesthecommandinthecurrentshell:
.shell_script
echo
Theechocommandsimplyoutputsastringtothestandardoutputdevicefollowedbyanewlinecharacter.
Eval
exec
exitn
118
Department of CSE
The exit command causes the script to exit with exit code n. An exit code of 0 means
success.Here aresomeothercodes.
export
Theexportcommandmakesthevariablenamedasitsparameteravailableinsubshells.
expr
Theexprcommandevaluates itsargumentsasanexpression.
0x= `expr$x+ 1`
Herearesomeofitsexpressionevaluations
printf
printf"formatstring"parameter1parameter2...
Herearesomecharactersand formatspecifiers.
119
Department of CSE
return
Thereturncommandcausesfunctionstoreturn. Itcanhaveavalueparameterwhichitreturns.
set
Thesetcommandsetstheparametervariables fortheshell.
shift
The shift command moves all the parameters variables down by one, so $2 becomes $1,
$3becomes$2,andsoon.
unset
Theunsetcommandremovesvariablesorfunctionsfromtheenvironment.
CommandExecution
The result of $(command) is simply the output string from the command, which is then
availabletothescript.
DebuggingShellScripts
Whenanerroroccursinascript,theshellprintsoutthelinenumberwithanerror.Youcanusethe
setcommandtosetvariousshelloption.Here are some ofthem.
120
Department of CSE
UnitII
FilesandDirectories
UNIXFileStructure
InUNIX,everythingisafile.
Programs can use disk files, serial ports, printers and other devices in the exactly the same
wayastheywoulduseafile.
Directories, too,arespecialsortsoffiles.
Filetypes
Most files on a UNIX system are regular files or directories, but there are additional types
offiles:
Theargumenttoeachofdifferentfiletypesisdefinedasfollows_
121
Department of CSE
Macro Typeoffile
S_ISREG() Regularfile
S_ISDIR() Directoryfile
S_ISCHR() Characterspecialfile
S_ISBLK() Blockspecialfile
S_ISFIFO() PipeorFIFO
S_ISLNK() Symboliclink
S_ISSOCK() Socket
FileSystemStructure
Filesarearrangedindirectories,whichalsocontainsubdirectories.
122
Department of CSE
Filesand Devices
Even hardware devices are represented (mapped) by files in UNIX. For example, as root,
youmountaCD-ROMdrive as afile,
$mount-tiso9660/dev/hdc/mnt/cd_rom
$cd/mnt/cd_rom
/dev/console-thisdevicerepresentsthesystemconsole.
/dev/tty - This special fileis an alias (logical device)for controllingterminal
(keyboardandscreen,orwindow)ofaprocess.
/dev/null-Thisisthenulldevice.Alloutputwrittentothisdeviceisdiscarded.
FileMetadata Inodes
• Astructurethatismaintainedinaseparateareaoftheharddisk.
• Fileattributesarestoredintheinode.
• Everyfileisassociatedwithatablecalledtheinode.
• The inodeisaccessedbytheinodenumber.
• Inodecontainsthefollowingattributesofafile:filetype,filepermissions,no.oflinks
UIDoftheowner,GIDofthegroupowner,filesizedateandtimeoflastmodification,lastaccess,change.
Fileattributes
Attribute valuemeaning
Filetype typeofthefile
Accesspermission
fileaccesspermissionforowner,groupandothersHar
dlinkcount no.ofhardlinksofafile.
UID fileowner userID.
GID thefilegroupID.
Filesize filesize inbytes.
Inodenumber systeminodenumber ofthefile.
FilesystemIDfilesystemIDwherethefileisstored.
123
Department of CSE
KernelSupportForFiles:
UNIX supports the sharing of open files between different processes. Kernel has three
datastructures are used and the relationship among them determines the effect one process has
onanotherwithregardtofilesharing.
1. Every process has an entry in the process table. Within each process table entry is a
tableofopenfiledescriptors,whichistakenasavector,withone entryperdescriptor.
Associatedwitheach filedescriptorare
a. Thefiledescriptorflags.
b. Apointertoafiletableentry.
2. Thekernelmaintainsafiletableforallopenfiles.Eachfiletableentrycontains
a. Thefilestatusflagsforthe file(read,write,append,sync,nonblocking,etc.),
b. Thecurrentfileoffset,
c. Apointertothev-nodetableentryforthefile.
3. Each open file (or device) has a v-node structure. The v-node contains information
aboutthe type of file and pointers to functions that operate on thefile. For most files the v-
node also contains thei-node forthe file.Thisinformation is readfrom disk when
thefileisopened,sothatallthe pertinentinformation aboutthefileisreadilyavailable.
The arrangementof these three tables for a single process thathas two differentfiles openone
file is open on standard input (file descriptor 0) and the otheris open standard output(file
descriptor1).
Here, thefirstprocess has thefile open descriptor 3 and the second process has file opendescriptor
4. Each process that opens the file gets its own file table entry, but only a single v-node table
entry. One reason each process gets its own file table entry is so that each process
hasitsowncurrentoffsetforthefile.
After each ‗write‘ is complete, the current file offset in the file table entry is
incrementedby the number of bytes written. If this causes the current file offset to exceed
the currentfile size, the current file size, in the i-node table the entry is to the current file
offset(Ex:fileis extended).
IfafileisopenedwithO_APPENDflag,acorrespondingflagissetinthefilestatusflagsofthefileta
bleentry.Eachtimea‗write‘isperformedforafilewiththisappendflag
124
Department of CSE
set,thecurrentfileoffsetinthefiletableentryis firstsettothecurrentfilesizefromthei-
nodetableentry. This forcesevery‗write‘tobeappendedtothecurrentendoffile.
The ‗lseek‘ function only modifies the current offset in the file table entry. No I/O
tableplace.
If a file is positioned to its current end of file using lseek, all that happens is the
currentfileoffsetinthefile tableentryissetto the current file sizefromthei-nodetableentry.
It is possible for more than a descriptor entry to point to the same file table only. The
filedescriptor flag is linked with a single descriptor in a single process, while file status flags
aredescriptorsinanyprocessthatpointtogivenfile tableentry.
SystemCallsandDeviceDrivers
System calls are provided by UNIX to access and control files and
Thesystemcallsto accessthedevicedriversinclude:
LibraryFunctions
Toprovideahigherlevelinterfacetodeviceanddiskfiles,UNIIXprovidesanumberofstandardlibraries.
125
Department of CSE
Low-levelFileAccess
Each running program, called a process, has associated with it a number of file
descriptors.Whenaprogramstarts,itusuallyhasthree
ofthesedescriptorsalreadyopened.Theseare:
The write system call arranges for the first nbytes bytes from buf to be written to the
fileassociatedwiththefiledescriptorfildes.
Withthisknowledge,let'swriteourfirstprogram,simple_write.c:
Hereishowtoruntheprogramanditsoutput.
$
simple_write
Hereissomedata
$
read
The read system call reads up to nbytes of data from the file associated with the
filedecriptorfildes andplaces theminthedataareabuf.
126
Department of CSE
This program, simple_read.c, copies the first 128 bytes of the standard input to the
standardoutput.
127
Department of CSE
Ifyouruntheprogram,youshouldsee:
$echohellothere|simple_readhel
lothere
$simple_read<draft1.txtFi
les
open
Tocreateanewfiledescriptorweneedtousetheopensystemcall.
Thenameofthefileordeviceto be openedispassed
asaparameter,path,andtheoflagsparameter
isusedtospecifyactionstobetakenonopeningthefile.
TheoflagsarespecifiedasabitwiseORofamandatoryfileaccessmodeandotheroptionalmodes.Theop
encall mustspecifyone ofthe followingfile accessmodes:
128
Department of CSE
The call may also include a combination (bitwise OR) of the following optional modes
inthe oflagsparameter:
InitialPermissions
When we create a file using the O_CREAT flag with open, we must use the three
parameterform. mode, the third parameter, is made form a bitwise OR of the flags defined in
the headerfilesys/stat.h.Theseare:
Forexample
Has the effect of creating a file called myfile, with read permission for the owner and
executepermissionforothers,andonlythosepermissions.
umask
129
Department of CSE
Theumaskisasystemvariablethatencodesamaskforfilepermissionstobeusedwhen afileiscreated.
Youcanchangethevariablebyexecutingtheumaskcommandtosupplyanewvalue.
Thevalueisathree-digitoctalvalue.EachdigitistheresultsofANDingvaluesfrom1,2,or4.
Values for each digit are ANDed together; so digit 2 will have 2 & 1, giving 3.
Theresultingumaskis 032.
close
118
Department of CSE
Weuseclosetoterminatetheassociationbetweenafiledescriptor,fildes,andits file.
ioctl
ioctlisabitofarag-
bagofthings.Itprovidesaninterfaceforcontrollingthebehaviorofdevices,theirdescriptors
andconfiguringunderlyingservices.
ioctl performs the function indicated by cmd on the object referenced by the descriptor
fildes.TryItOut-AFile CopyProgram
Runningtheprogramwillgivethefollowing:
119
Department of CSE
We used the UNIX time facility to measure how long the program takes to run. It took 2 and
onehalfminutes tocopythe1Mbfile.
Wecanimprovebycopyinginlargerblocks.Hereistheimprovedcopy_block.cprogram.
Nowtrytheprogram,firstremovingtheoldoutputfile:
OtherSystemCallsforManagingFiles
lseek
120
Department of CSE
The lseek system call sets the read/write pointer of a file descriptor, fildes. You use it to
setwhereinthefilethe nextreadorwritewilloccur.
The offset parameter is used to specify the position and the whence parameter specifies how
theoffsetis used.
whencecan beoneofthefollowing:
dup anddup2
The dup system calls provide a way of duplicating a file descriptor, giving two or more,
differentdescriptorsthataccess thesamefile.
FileStatusInformation-StatFamily:fstat, statandlstat
The fstat system call returns status information about the file associated with an open
filedescriptor.
Themembersofthestructure,stat,mayvarybetweenUNIXsystems, butwillinclude:
121
Department of CSE
Othermode flagsinclude:
Maskstointerpretthest_modeflags include:
122
Department of CSE
Therearesomemacrosdefinedtohelpwithdeterminingfiletypes.Theseinclude:
Totestthatafiledoesn'trepresentadirectoryandhasexecutepermissonsetfortheownerandnootherper
missions,we canusethetest:
Fileandrecordlocking-fcntlfunction
• Filelockingisapplicableonlyforregularfiles.
• Itallowsaprocesstoimposealockonafilesothatotherprocessescannotmodifythefile untilitis
unlockedbytheprocess.
• Writelock:it preventsotherprocessesfromsettinganyoverlappingread/writelocksonthe
lockedregionofafile.
• Readlock:it preventsotherprocessesfromsettinganyoverlappingwritelocks
onthelockedregionofafile.
• Writelockisalsocalledaexclusivelockandreadlockisalsocalledasharedlock.
• fcntlAPIcan beusedtoimposereadorwritelocksoneitherasegmentoranentirefile.
• Functionprototype:
#include<fcntl.h>
intfcntl(intfdesc,intcmd_flag,….);
• Allfilelockssetbyaprocesswillbeunlockedwhentheprocessterminates.
123
Department of CSE
FilePermission-chmod
You can change the permissions on a file or directory using the chmod system call. Tis forms
thebasisofthechmodshellprogram.
chown
Asuperuser canchangetheownerofafileusing thechownsystemcall.
Links-softlinkandhardlink
Soft link(symbolic links):Refer to a symbolic path indicating the abstract location of
anotherfile.
Usedto providealternativemeansofreferencingfiles.
Usersmaycreatelinksforfilesusinglncommandbyspecifying–soption.
hardlinks:Referto thespecificlocationofphysicaldata.
Ahard linkisaUNIXpathname forafile.
Mostofthefileshaveonlyonehardlink.Howeverusers
maycreateadditionalhardlinksforfilesusinglncommand.
Limitations:
Userscannotcreatehardlinksfordirectoriesunlesstheyhavesuperuserprivileges.
Userscannotcreatehardlinksonafilesystemthatreferences filesonadifferentsystems.
unlink,link,symlink
Wecanremoveafileusingunlink.
Theunlinksystemcalldecrementsthelinkcounton
124
Department of CSE
Thesymlinkcreatesasymboliclinktoanexistingfile.
Directories
Adirectoryisafilethatholdstheinodesandnamesofother files.
mkdir,rmdir
Themkdirsystemcallmakesanewdirectorywithpathasitsname.
The rmdirsystemcallremovesanemptydirectory.
chdir
CurrentWorkingDirectory- getcwd
125
Department of CSE
ScanningDirectories
The directory functions are declared in a header file, dirent.h. They use a structure, DIR, as
abasisfordirectorymanipulation.
Herearethesefunctions:
opendir
readdir
Thedirentstructurecontainingdirectoryentrydetailsincludedthefollowingentries:
telldir
126
Department of CSE
seekdir
The seekdirfunctionsetsthedirectoryentrypointerinthedirectorystreamgivenbydirp.closedir
TryIt Out-ADirectoryScanningProgram
127
Department of CSE
2. Nowwemoveontothemainfunction:
After some initial error checking, using opendir, to see that the directory exists, printdir
makesa call to chdir to the directory specified. While the entries returned by readdir aren't null,
theprogram checks to see whether the entry is a directory. If it isn't, it prints the file entry
withindentationdepth.
128
Department of CSE
Theprogramproducesoutputlikethis(edited forbrevity):HowItWorks
Hereisonewaytomaketheprogrammoregeneral.
Youcanrunitusingthecommand:
$printdir /usr/local|more
129
Department of CSE
UNIT-III
Processes and signals form a fundamental part of the UNIX operating environment,
controllingalmostallactivities performedbyaUNIX computersystem.
What isaProcess
TheX/OpenSpecification
definesaprocessasanaddressspaceandsinglethreadofcontrolthatexecuteswithinthataddressspace
anditsrequiredsystemresources.
Aprocessis,essentially,arunningprogram.
Layoutofa Cprogram
Hereishowacoupleofprocessesmightbearrangedwithintheoperationgsystem.
The program code that will be executed by the grep command is stored in a disk
130
Department of CSE
Aprocesshasitsownstackspace.
131
Department of CSE
Imageinmain memory
TheUNIXprocesstablemaybethoughofasadatastructuredescribingallof theprocessesthatare
currentlyloaded.
ViewingProcesses
The PID column gives the PIDs, the TTY column shows which terminal started the
process,theSTATcolumn showsthecurrent status,TIMEgivestheCPUtimeused sofarand
the COMMAND column shows the command used to start the
process.Let'stakea closerlookatsomeofthese:
The initial login was performed on virtual console number one (v01). The shell is running
bash.Its status is s, which means sleeping. Thiis is because it's waiting for the X Windows
sytem tofinish.
X Windows was started by the command startx. It won't finished until we exit from X. It too
issleeping.
The fvwm is a window manager for X, allowing other programs to be started and windows to
bearrangedonthescreen.
This process represents a window in the X Windows system. The shell, bash, is running in
thenewwindow.Thewindowisrunning onanewpseudoterminal(/dev/ptyp0)abbreviatedpp0.
132
Department of CSE
ThisistheEMACSeditorsessionstartedfromtheshellmentionedabove.Itusesthepseudoterminal.
This is a clock program started by the window manager. It's in the middle of a one-minute
waitbetweenupdatesoftheclockhands.
Processenvironment
Let'slookatsomeotherprocessesrunningonthisLinuxsystem.Theoutputhasbeenabbreviatedfor
clarity:
Herewecansee oneveryimportantprocessindeed:
In general, each process is started by another, known as its parent process. A process so
startedisknownas achildprocess.
WhenUNIXstarts,itrunsasingleprogram,theprimeancestrorandprocessnumberone:init.
One such example is the login procedure init starts the getty program once for each terminal
thatwecanusetolongin.
Theseareshowninthepsoutputlikethis:
133
Department of CSE
134
Department of CSE
When interacting with your server through a shell session, there are many pieces of
informationthat your shell compiles to determine its behavior and access to resources. Some of
these settingsarecontainedwithinconfigurationsettingsandothersare determinedbyuserinput.
One way that the shell keeps track of all of these settings and details is through an area
itmaintainscalledtheenvironment.Theenvironmentisanareathattheshellbuildseverytimethatitstartsa
sessionthatcontainsvariablesthatdefinesystemproperties.
In this guide, we will discuss how to interact with the environment and read or set
environmentalandshellvariablesinteractivelyand throughconfiguration files.Wewillbeusing
anUbuntu
12.04VPSasanexample, butthesedetailsshouldberelevantonanyLinuxsystem.
Every time a shell session spawns, a process takes place to gather and compile information
thatshould be available to the shell process and its child processes. It obtains the data for
thesesettingsfromavarietyofdifferentfiles andsettings onthe system.
Basically the environment provides a medium through which the shell process can get or
setsettingsand,inturn,pass these ontoitschildprocesses.
EnvironmentList
The environment is implemented as strings that represent key-value pairs. If multiple values
arepassed, they are typically separated by colon (:) characters. Each pair will generally will
looksomethinglikethis:
KEY=value1:value2:...
Ifthevaluecontainssignificantwhite-space,quotationsareused:
KEY="valuewithspaces"
The keys in these scenarios are variables. They can be one of two types, environmental
variablesorshellvariables.
Environmental variables are variables that are defined for the current shell and are inherited
byanychildshellsorprocesses.Environmentalvariablesareusedtopassinformationintoprocessesthata
respawnedfromtheshell.
Shell variables arevariables thatare contained exclusively within theshell in which they wereset
or defined. They are often used to keep track of ephemeral data, like the current
workingdirectory.
135
Department of CSE
By convention, these types of variables are usually defined using all capital letters. This
helpsusersdistinguishenvironmentalvariableswithinothercontexts.
Environmentvariables-getenv, setenv
Every process has an environment block that contains a set of environment variables and
theirvalues. There are two types of environment variables: user environment variables (set for
eachuser)andsystemenvironmentvariables(setforeveryone).
By default, a child process inherits the environment variables of its parent process.
Programsstarted by the command processor inherit the command processor's environment
variables. Tospecify a different environment for a child process, create a new environment block
and pass apointertoitas a parametertotheCreateProcess function.
The command processor provides the set command to display its environment block or to
createnewenvironmentvariables.You can alsoviewormodify theenvironmentvariablesbyselecting
System from the Control Panel, selectingAdvanced system settings,
andclickingEnvironmentVariables.
Eachenvironmentblockcontainstheenvironmentvariablesinthefollowingformat:
Var1=Value1\0
Var2=Value2\0
Var3=Value3\0
...
VarN=ValueN\0\0
Thenameofanenvironmentvariablecannotincludeanequalsign(=).
The GetEnvironmentStrings function returns a pointer to the environment block of the
callingprocess. This should be treated as a read-only block; donotmodify itdirectly. Instead,
usetheSetEnvironmentVariablefunctiontochangeanenvironmentvariable.Whenyouarefinished
withtheenvironmentblockobtainedfromGetEnvironmentStrings,calltheFreeEnvironmentStringsfu
nction
tofreetheblock.CallingSetEnvironmentVariablehasnoeffectonthesystemenvironmentvariables.
Kernelsupportforprocess
The kernel runs the show, i.e. it manages all the operations in a Unix flavored environment.
Thekernel architecture must support the primary Unix requirements. These requirements fall in
twocategories namely, functions for process management and functions for file management
(filesincludedevicefiles).ProcessmanagemententailsallocationofresourcesincludingCPU,memory,
and offers services that processes may need. The file management in itself involveshandling all
the files required by processes, communication with device drives and regulatingtransmission of
data to and from peripherals. The kernel operation gives the user processes a
feelofsynchronousoperation,hidingallunderlyingasynchronisminperipheralandhardwareoperation
s (like the time slicing by clock). In summary, we can say that the kernel handles
thefollowingoperations :
1. Itisresponsibleforschedulingrunningofuserandotherprocesses.
136
Department of CSE
2. Itisresponsibleforallocatingmemory.
4. Itisresponsibleformovingdatatoandfromtheperipherals.
5. itreceivesservicerequestsfromtheprocesses andhonorsthem.
ProcessIdentification:
Every process has a unique process ID, a non-negative integer. There are two special
processes.Process ID0 is usually the schedule process and is often known as the ‗swapper‘. No
program ondisk corresponds to this process – it is part of the kernel and is known as a system
process,process ID1 is usually the ‗init‘ process and is invoked by the kernel at the end of the
bootstrapprocedure.Theprogramfilesforthisprocessloss/etc/initinolderversionofUNIX andis
/sbin/initisnewerversion.‗init‘usuallyreadsthesystemdependentinitializationfilesandbringsthe
system to a certain state. The ‗init‘ process never dies. ‗init‘ becomes the parent process
ofanyorphanedchildprocess.
Processcontrol
Onefurther psoutputexampleistheentryforthepscommanditself:
This indicatesthatprocess192isinarunstate(R)andisexecutingthecommandps-ax.
We can set the process priority using nice and adjust it using renice, which reduce the priority
ofa process by10.Highpriorityjobs havenegative values.
Usingtheps-
l(forlongoutput),wecanviewthepriorityofprocesses.Thevalueweareinterestedinisshowninthe
NI(nice)column:
Here we can see that the oclock program is running with a default nice value. If it had
beenstatedwiththecommand,
itwouldhavebeenallocatedanicevalueof+10.
Wecanchangethe priorityofaruinningprocessbyusingtherenicecommand,
137
Department of CSE
So that now the clock program will be scheduled to run less often. We can see the modified
nicevalue withtheps again:
Notice that the status column now also contains N, to indicate that the nice value has
changedfromthedefault.
ProcessCreationStartingNewProcesses
We can cause a program to run from inside another program and thereby create a new process
byusingthesystem.libraryfunction.
The system function runs the command passed to it as string and waits for it to
complete.The commandisexecutedasifthecommand,
hasbeengiventoashell.
TryItOut-system
1. Wecanusesystemtowriteaprogramtorunps forus.
138
Department of CSE
2. When we compile and run this program, system.c, we get the following:
3. Thesystemfunctionusesashelltostartthedesiredprogram.
Now,whenwecompileandrunthisversionofthe program,weget:
HowIt Works
139
Department of CSE
Inthefirstexample,theprogramcallssystemwiththestring"ps-ax",whichexecutes
thepsprogram.Ourprogramreturnsfromthecalltosystemwhenthepscommandisfinished.
In the second example, the call to system returns as soon as the shell command finishes.
Theshellreturnsassoonasthepsprogramisstarted,justaswouldhappenifwe hadtyped,
atashellprompt.
ReplacingaProcessImage
There is a whole family of related functions grouped under the exec heading. They differ in
thewaythattheystartprocesses andpresentprogramarguments.
If we wish to use an exec function to start the ps program as in our previous examples, we
havethe followingchoices:
139
Department of CSE
TryIt Out-exclp
Now, when we run this program, pexec.c, we get the usual ps output, but no Done. message
atall.
Notealsothatthereisnoreferencetoaprocesscalledpexecintheoutput:
HowIt Works
Theprogramprintsitsfirstmessageandthencallsexeclp,whichsearchesthedirectoriesgivenbythePAT
Henvironmentvariableforaprogramcalledps.
It then executes this program in place of our pexec program, starting it as if we had given
theshellcommand:
Waiting foraProcess
Wecanarrangefortheparentprocesstowaituntilthechildfinishesbeforecontinuingbycallingwait
.
The wait system call causes a parent process to pause until one of its child processes dies or
isstopped.
Wecaninterrogatethestatusinformationusingmacrosdefinedinsys/wait.h. Theseinclude:
140
Department of CSE
TryItOut-wait
1. Let'smodifyourprogramslightlysowecanwaitfor andexaminethechildprocess
exitstatus.Callthenewprogramwait.c.
141
Department of CSE
2. This section of the program waits for the child process to finish:
142
Department of CSE
HowIt Works
The parent process uses the wait system call to suspend its own execution until
statusinformationbecomesavailablefora childprocess.
ZombieProcesses
Whenachildprocessterminates,anassociationwithitsparentsurvivesuntiltheparentinturneithertermin
ates normallyorcallswait.
Thisterminatedchildprocessisknownasazombieprocess.
TryIt Out-Zombies
fork2.c is jsut the same as fork.c, except that the number of messages printed by the child
andpaentporcessesis reversed.
Herearetherelevantlinesofcode:
143
Department of CSE
HowIt Works
Ifweruntheaboveprogramwithfork2&andthencallthepsprogramafterthechildhasfinishedbutbefor
etheparenthasfinished,we'llsee a linelike this:
There'sanothersystemcallthatyoucanusetowailforchildprocesses.It'scalledwaitpidandyoucanuseittowaitfo
ra specificprocesstoterminate.
Ifwewanttohaveaparentprocessregularlycheckwhetheraspecificchildprocesshadterminated,w
e coulduse thecall,
OrphanProcess
• Whentheparentdiesfirstthe childbecomesOrphan.
• Thekernelclearstheprocesstableslotfortheparent.
Systemcallinterfaceforprocessmanagement
Inadditiontotheprocess ID,thereareotheridentifiersforeveryprocess.
Thefollowingf
unctionsreturntheseidentifiers
#incldue<sys/types.h>
#include<unistd.h>
pid_tgetpid(void); Returns:processIDofcallingprocess
processuid_tgetuid(void); Returns:realuserIDofcallingprocess
144
Department of CSE
uid_tgeteuid(void);
Returns:effectiveuserIDofcallingprocessgid_t
getgid(void);
Returns:realgroupIDofcallingprocessgid_tget
egid(void); Returns:effectivegroupIDofcallingprocess
forkFunction
TheonlywayanewprocessiscreatedbytheUNIXkerneliswhenanexistingprocesscallstheforkfunction
.
#include<sys/types.h>
#include<unistd.h>pi
d_tfork(void);
Return: 0ischild,processIDofchildinparent,-1onerror
The new process created by fork is called child process. This is called once, but return twice
thatis the return value in the child is 0, while the return value in the parent is the process ID of
thenew child. The reason the child‘s process ID is returned to the parent is because a process
canhave more than one child, so there is no function that allows a process to obtain the process
IDsof its children. The reason fork return 0 to the child is because a process can have only a
singleparent,sothatchildcanalways callgetppidtoobtaintheprocessIDofitsparent.
Both the child and parent contain executing with the instruction that follows the call to fork.
Thechildis copy of the parent.For example,the child gets a copy of the parent‘s data
space,heapand stack. This is a copy for the child the parent and children don‘t share these
portions ofmemory.Oftenthe parentandchildshare thetextsegment,ifitisread-only.
Therearetwousersforfork:
1. When a process wants to duplicate itself so that the parent and child can each
executedifferent sections of code at the same time. This is common for network servers_
theparentwaitsfora service requestsfrom a client.When the requestarrives,the parentcalls
fork and lets the child handle the request. The parent goes back to waiting for
thenextservicerequesttoarrive.
145
Department of CSE
When a process wants to execute a different program, this is common for shells. In
thiscase the childdoesanexec rightafteritreturnsfromthefork.
146
Department of CSE
vforkFunction
Thefunction vfork has the same calling sequence and share return values as fork. But
thesemantics of the two functions differ. vforkis intended to create a new process when
thepurpose of the new process is to exec a new program. vfork creates the new process, just
likefork, without fully copying the address space of the parent into the child, since the child
won‘treference theaddress space– the childjustcalls exec rightafter thevfork. Instead,
whilethechild is running, until it calls either exec or exit, the child runs in the address space of
the parent.This optimization provides an efficiency gain on some paged virtual memory
implementations ofUNIX.
Another difference between the two functions is that vfork guarantees that the child runs
first,untiltheparentresumes.
exitFunction
Therearethreewaysforaprocesstoterminatenormally,andtwoformsofabnormaltermination.
1. Normaltermination:
a. Executingareturnfromthemainfunction.Thisisequivalenttocallingexit
b. Callingtheexitfunction
c. Callingthe_exitfunction
2. Abnormaltermination
a. Callingabort:ItgeneratestheSIGABRTsignal
b. When the process receives certain signals. The signal can be generated by
theprocessitself
Regardless of how a process terminates, the same code in the kernel is eventually executed.
Thiskernel code closes all the open descriptors for the process, releases the memory that it was
using,andthelike.
For any of the preceding cases we want the terminating process to be able to notify its parent how
itterminated. For the exit and _exit functions this is done by passing an exit status as the argument
tothesetwofunctions.Inthecaseofanabnormalterminationhowever,thekernelgeneratesatermination
status to indicate the reason for the abnormal termination. In any case, the parent of theprocess can
obtain the termination status from either the wait or waitpid function.The exit status isconverted into
a termination status by the kernel when _exit is finally called. If the child
terminatednormally,thentheparentcanobtaintheexitstatusofthechild.
147
Department of CSE
If the parent terminates before the child, then init process becomes the parent process of
anyprocess, whose parent terminates; that is the process has been inherited by init. Whenever
aprocess terminates the kernel goes through all active processes to see if the terminating process
isthe parent of any process that still exists.If so, the parent process ID of the still existing
processischangedtobe 1toassumethateveryprocesshasa parentprocess.
When a child terminates before the parent, and if the child completely disappeared, the
parentwouldn‘t be able to fetch its termination status, when the parent is ready to seek if the
child hadterminated. But parent get this information by calling wait and waitpid, which is
maintained bythe kernel.
waitandwaitpid Functions
When a process terminates, either normally or abnormally, the parent is notified by the
kernelsending the parent SIGCHLD signal. Since the termination of a child is an asynchronous
event,this signal is the asynchronous notification from the kernel to the parent. The default action
forthis signal is to be ignored. A parentmay want for one of its children to terminate and
thenacceptitchild‘sterminationcodebyexecutingwait.
Aprocessthatcallswaitandwaitpidcan
IftheprocessiscallingwaitbecauseitreceivedSIGCHLDsignal,weexpectwaittoreturnimmediately.B
ut,ifwecallitatanyrandompointintime,itcanblock.
#include<sys/types.h>
#include<sys/wait.h>
options);Bothreturn:processIDifOK,oor-
148
Department of CSE
1onerror
149
Department of CSE
Thedifferencebetweenthesetwofunctions is
1. waitcanblockthecaller untilachildprocessterminates,whilewaitpid
hasanoptionthatpreventsitfromblocking.
2. waitpiddoesnotwaitforthefirstchildtoterminate,ithasa
numberofoptionsthatcontrolwhichprocessitwaits for.
If a child has already terminated and is a zombie, wait returns immediately with that
child‘sstatus.Otherwise,it blocks the calleruntil a child terminates:if the callerblocks
andhasmultiple children, wait returns when one terminates, we can know this process by PID
return bythefunction.
For both functions, the argument statloc is pointer to an integer. If this argument is not a
nullpointer, the termination status of the terminated process is stored in the location pointed to by
theargument.
If we have more than one child, wait returns on termination of any of the children. A
functionthatwaits fora specificprocessis waitpidfunction.
Theinterpretationofthepidargumentforwaitpiddependsonitsvalue:
however, it‘s also possible to get an error if the specified process or process group does not
existorisnotachildofthecallingprocess.
The options argumentlets us further control the operation of waitpid. This argumentis either 0or
isconstructedfromthebitwiseOR ofthefollowingconstants.
WNOHANG
waitpidwillnotblinkifachildspecifiedbypidisnotimmediatelyavaila
ble.Inthiscase,the returnvalueis0.
150
Department of CSE
151
Department of CSE
statushasnotbeenupdatedsinceithas stopped,isreturned
Thewaitpidfunctionprovidesthesefeaturesthatarenotprovidedbythewaitfunctionare:
1. waitpidletsustowaitforoneparticularprocess
2. waitpid providesa non-blockingversionofwait
3. waitpidsupportsjobcontrol(wittheWUNTRACEDoption)
execFunction
The fork function can create a new process that then causes another program to be executed
bycalling one of the exec functions. When a process calls one of the exec functions, that process
iscompletely replaced by thenew program and the new program starts executing atits
mainfunction. The process ID doesn‘t change across an exec because a new process is not
created.execmerelyreplacesthecurrentprocesswitha brandnew programfromdisk.
Therearesixdifferentexecfunctions.ThesesixfunctionsroundouttheUNIXcontrolprimitives. With
fork we can create new processes, and with the exec functions we can initiatenew programs. The
exit function and the two wait functions handle termination and waiting
fortermination.Thesearetheonlyprocess controlprimitives weneed.
#include<unistd.h>
intexecl(constchar*pathname,constchar *arg0, .
../*(char*)0*/intexecv(constchar*pathname,char*constargv[]);
intexecle(constchar*pathname, constchar*arg0,.../*(char*)0,charenvp[]*/);
0*/);intexecvp(constchar*filename,char*constargv[]);
Allsixreturns:-1onerror,no returnonsuccess.
The first difference in these functions is that the first four take a pathname argument, while
thelast twotake a filename argument.Whenafilename argumentisspecified:
iffilenamecontainsaslash,itistakenasapathname.
Otherwise,theexecutablefileisasearchedforindirectoriesspecifiedbythePATH
152
Department of CSE
environmentvariable.
The PATH variable contains a list of directories (called path prefixes) that are separated
bycolors.Forexample,the name=value environmentstring
PATH=/bin:/usr/bin:usr/local/bin/:.
Specifiesfourdirectoriestosearch,wherelastoneiscurrentworkingdirectory.
If either of the two functions, execlp or execvp finds an executable file using one of the
pathprefixes, but the file is not a machine executable that was generated by the link editor, it
assumesthefileisa shellscriptand triesto invoke/bin/shwith filenameasinput to theshell.
The next difference concerns the passing of argument list. The function execl, execlp and
execlerequire each of the command-line arguments to the new program to be specified as
separatearguments. The end of the argumentshouldbe a null pointer. For the other three
functionsexecv, execvp andexecve, we have tobuild an array of pointers to the arguments, and
theaddressofthisarrayis the argumenttothese threefunctions.
The final difference is the passing of the environment list to the new program. The two
functionsexecle and execve allow us to pass a pointer to an array of pointer to an array of pointer
to anarray of pointers to the environment strings. The other four functions, however, use the
environvariableinthe calling processtocopythe existingenvironmentforthe newprogram.
DifferencesBetweenThreadsandProcesses
UNIXprocessescancooperate;theycansendeachothermessagesandtheycaninterruptoneanother.
There is a class of process known as a thread which are distinct from processes in that they
areseparate executionstreams withinasingleprocess.
Signals
A signal is an event generated by the UNIX system in response to some condition, upon
receiptofwhichaprocessmayinturntakesomeaction.
Signalnamesaredefinedintheheaderfilesignal.h.TheyallbeginwithSIG andinclude:
153
Department of CSE
Additionalsignalsinclude:
If the shell and terminal driver are configured normally, typing the interrupt character (Ctrl-C)
atthe keyboard will result in the SIGINT signal being sent to the foreground process. This
willcause theprogramtoterminate.
Wecanhandlesignalsusingthesignallibraryfunction.
154
Department of CSE
Thesignalfunctionitselfreturnsafunctionofthesametype,whichisthepreviousvalueofthefunctionsetu
ptohandle this signal,orone ofthese towspecialvalues:
Signalgeneration &Handling
1. We'llstartbywritingthefunctionwhichreactstothesignalwhichispassedintheparametersig.
Let'scallitouch:
2. The main function has to intercept the SIGINT signal generated when we type Ctrl-
C.Fortherest ofthetime,itjustsitsinaninfiniteloop,printingamessageonceasecond:
3. Whiletheprogramisrunning, typingCtrl-Ccausesittoreactandthencontinue.
Whenwe typeCtrl-Cagain,theprogramends:
155
Department of CSE
HowIt Works
The program arranges for the function ouch to be called when we type Ctrl-C, which
givesthe SIGINTsignal.
Kernelsupport forSignals-SendingSignals
Aprocessmaysendasignaltoitselfbycallingraise.
Signalsprovideuswithausefulalarmclockfacility.
The alarm function call can be used by a process to schedule a SIGALRM signal at some
timeinthefuture.
156
Department of CSE
Try ItOut-AnAlarmClock
1. Inalarm.c, thefirstfunction,ding,simulatesanalarmclock:
2. Inmain,wetellthechildprocesstowaitforfivesecondsbeforesendingaSIGALRMsignaltoitsparen
t:
3. Theparentprocess arranges
tocatchSIGALRMwithacalltosignalandthenwaitsfortheinevitable.
157
Department of CSE
When we run this program, it pauses for five seconds while it waits for the simulated
alarmclock.
This program introduces a new function, pause, which simply causes the program to
suspendexecutionuntilasignaloccurs.
It'sdeclaredas,
HowItWorks
The alarm clock simulation program starts a new process via fork. This child process sleeps
forfive seconds andthensends aSIGALRMtoitsparent.
ARobustSignalsInterface
Thesigactionstructure,usedtodefinetheactionstobetakenonreceiptofthesignalspecifiedbysig,isdef
inedinsignal.handhasatleastthefollowingmembers:
TryIt Out-sigaction
MakethechangesshownbelowsothatSIGINTisintercepted bysigaction.Callthenew
158
Department of CSE
programctrlc2.c.
Running the program, we get a message when we type Ctrl-C because SIGINT is
handledrepeated;ybysigaction.
TypeCtrl-\toterminatetheprogram.
159
Department of CSE
HowIt Works
The program calls sigaction instead of signal to set the signal handler for Ctrl-C (SIGINT)
tothe functionouch.
SignalSets
Theheaderfilesignal.hdefinesthetypesigset_tandfunctionsusedtomanipulatesetsofsignals.
Thefunctionsigismemberdetermineswhetherthegivensignalisamemberofasignalset.
Theprocesssignalmaskissetorexaminedbycallingthefunctionsigprocmask.
sigprocmask can change the process signal mask in a number of ways according
tothe howargument.
160
Department of CSE
Thehowargumentcanbeoneof:
A program
candeterminewhichofitsblockedsignalsarpendingbycallingthefunctionsigpending.
A process can suspend execution until the delivery of one of a set of signals
bycallingsigsuspend.
SignalFunctions
Thesystemcallsrelatedtosignalsareexplainedinthefollowingsections.
Unreliablesignals
The signals could get lost – a signal could occur and the process would never know about
it.Here, the process has little control over a signal,it could catch the signal or ignore it,
butblockingofasignalisnotpossible.
Reliablesignals
Linux supports both POSIX reliable signals (hereinafter "standard signals") and POSIX real-
timesignals.
Signaldispositions
161
Department of CSE
Each signal has a current disposition, which determines how the process behaves when it
isdeliveredthesignal.
Theentriesinthe "Action"column ofthe tablesbelowspecifythedefaultdispositionforeachsignal.
killand raiseFunctions
Thekillfunctionsendsasignaltoaprocessoragroupofprocesses.Theraisefunctionallowsaprocesstosen
da signaltoitself.
#include<sys/types.h>
#include<signal.h>
intkill(pid_tpid,intsigno);int
raise(intsigno);
Bothreturn:0ifOK,-1onerror
Therearefourdifferentconditionsforthepidargumenttokill:
pid>0ThesignalissenttotheprocesswhoseprocessIDispid.
pid =0
ThesignalissenttoallprocesseswhoseprocessgroupIDequalstheprocessgroupID
ofthesenderandforwhichthesenderhaspermissiontosendthesignal.
pid = -1 unspecified.
alarmand pauseFunctions
The alarm function allows us to geta timer that will expire ata specified timein thefuture.When
the timer expires, the SIGALRM signal is generated. If we ignore or don‘t catch
thissignal,itsdefaultactionistoterminatetheprocess.
#include<unistd.h>
unsignedintalarm(unsignedintseconds);
The seconds value is the number of clock seconds in the future when the signal should
begenerated. There is only one of the alarm clocks per process. If, when we call alarm, there is
apreviously registered alarm clock for the process that has not yet expired, the number of
secondsleft for that alarm clock to return as the value of this function. That previously registered
alarmclockis replacedbythe newvalue.
If there is a previously registered alarm clock for the process that has not yet expired and if
theseconds value is 0, the previous alarm clock is cancelled. The number of seconds left for
thatpreviousalarmclockis stillreturnedas the valueofthefunction.
AlthoughthedefaultactionforSIGALRMisterminatingtheprocess,mostprocessesuseanalarmclock
catchthis signal.
163
Department of CSE
caught.#include<unistd.h>
int pause(void);
Theonlytimepausereturnsisifasignalhandlerisexecutedandthathandlerreturns.Inthatcase,pauseret
urns-1witherrnosettoEINTR.
abortFunction
termination.#include<stdlib.h>
voidabort(void);
Thisfunctionneverreturns.
Thisfunctionsends
theSIGABRTsignaltotheprocess.Aprocessshouldnotignorethissignal.abortoverrides the
blockingorignoringofthe signalbythe process.
sleepFunction
#include<unistd.h>
unsignedintsleep(unsignedintseconds);Re
turns:0or numberofunsleptseconds.
This functioncausesthecallingprocesstobesuspendeduntileither:
164
Department of CSE
Ifalarmisused,however,therecanbei
nteractionbetweenthe twofunctions.
165
Department of CSE
UnitIV
IntroductiontoIPC
TherearetwofundamentallydifferentapproachesinIPC:
processesareresidingonthesamecomputer
processesareresidingondifferentcomputers
The first case is easier to implement because processes can share memory either in the user
spaceorinthe systemspace.Thisisequallytrue foruniprocessorsandmultiprocessors.
In the second case the computers do not share physical memory, they are connected via
I/Odevice(forexampleserialcommunicationorEthernet).Thereforetheprocessesresidingindifferentc
omputerscannotuse memoryasameansforcommunication.
IPCbetweenprocessesonaSingleSystem
MostofthischapterisfocusedonIPConasinglecomputersystem,includingfourgeneralapproaches:
Sharedmemory
Messages
Pipes
Sockets
The synchronization objects considered in the previous chapter normally work across the
processboundaries(onasinglecomputersystem).Thereisoneadditionnecessaryhowever:thesynchron
ization objects must be named. The handles are generally private to the process,
whiletheobjectnames,likefilenames,are globalandknowntoallprocesses.
h=init_CS("xxx");
h=init_semaphore(20,"xxx");h
=init_event("xxx");
h=init_condition("xxx");
h=init_message_buffer(100,"xxx");
IPCbetweenprocessesondifferentsystems
IPCbetweenprocessesondifferentsystems
IPCisInterProcessCommunication,moreofatechniquetosharedataacrossdifferentprocesses
within one machine, in such a way that data passing binds the coupling of different
processes.Thefirst,isusingmemorymappingtechniques,whereamemorymapiscreated,andothers
166
Department of CSE
PIPES:
A pipe is a serial communication device (i.e., the data is read in the order in which it
waswritten),whichallowsaunidirectionalcommunication.Thedatawrittentoendisreadbackfromthe
otherend.
The pipe is mainly used to communicate between two threads in a single process or
betweenparentandchildprocess.Pipescanonlyconnecttherelatedprocess.Inshell,thesymbolcanbeus
edtocreateapipe.
In pipes the capacity of data is limited. (i.e.) If the writing process is faster than the
readingprocess which consumes the data, the pipe cannot store the data. In this situation the
writerprocess will block until more capacity becomes available.Alsoif the reading process tries
toread data when there is no data to read, it will be blocked until the data becomes available.
Bythis,pipesautomaticallysynchronizethetwoprocess.
Creatingpipes:
Thepipe()functionprovidesameansofpassingdatabetweentwoprogramsandalsoallowstoreadandwrit
ethedata.
#include<unistd.h>
intpipe(intfile_descriptor[2]);
pipe()function is passed with an array of file descriptors. It will fill the array with new
filedescriptors and returns zero. On error, returns -1 and sets the errno to indicate the reason
offailure.
The file descriptors are connected in a way that is data written to file_ descriptor[1] can be
readbackfromthefile_descriptor[0].
(Note:Asthisusesfiledescriptorsandnotthefilestreams,wemustusereadandwritesystemcallstoaccess
thedata.)
Pipes are implemented in file system. Pipes are basically files with only two file offsets: one
forreading another for writing. Writing to a pipe and reading from a pipe is strictly in FIFO
manner.(Therefore pipesare alsocalledFIFOs).
For efficiency, pipes are in-core files, i.e. they reside in memory instead on disk, as any
otherglobal data structure. Therefore pipes must be restricted in size, i.e. number of pipe blocks
mustbe limited. (In UNIX the limitation is that pipes use only direct blocks.)Since the pipes have
alimited size and the FIFO access discipline, the reading and writing processes
aresynchronizedin a similar manner as in case of message buffers. The access functions for pipes
are the same asforfiles:WriteFile()andReadFile().
167
Department of CSE
Pipesusedasstandardinputandoutput:
We can invoke the standard programs, ones that don‘t expect a file descriptor as a
parameter.#include<unistd.h>
int dup(int file_descriptor);
intdup2(intfile_descriptor_1,int
file_descriptor_2);
The purpose of dup call is to open a new file descriptor, which will refer to the same file as
anexisting file descriptor. In case of dup, the value of the new file descriptor is the lowest
numberavailable. In dup2 itis same as, or the first available descriptor greater than the
parameterfile_descriptor_2.
We can pass data between process by first closing the file descriptor 0 and call is made to
dup.By this the new file descriptor will have the number 0.As the new descriptor is the
duplicate ofan existing one, standard input is changed to have the access. So we have created
two filedescriptorsforsamefile orpipe,oneofthemwillbethe standardinput.
(Note: The same operation can be performed by using the fcntl() function. But compared to
thisdupanddup2aremoreefficient)
//pipes.c#include<
unistd.h>#include<
stdlib.h>#include<
stdio.h>
#include<string.h>
intmain()
{
intdata_processed;in
tfile_pipes[2];
constcharsome_data[]="123";pi
d_tfork_result;
if(pipe(file_pipes)==0)
{
fork_result=fork();if(fork
_result==(pid_t)-1)
{
fprintf(stderr,"forkfailure");
exit(EXIT_FAILURE);
}
if(fork_result==(pid_t)0)
{
close(0);dup(file_pipes[0])
;close(file_pipes[0]);
168
Department of CSE
close(file_pipes[1]);execlp("od",
"od","-
c",(char*)0);exit(EXIT_FAILU
RE);
}
else
{
close(file_pipes[0]);data_processed=write(file_pipe
s[1],some_data,strlen(some_data));close(file_pipes[
1]);
printf("%d-wrote%dbytes\n",(int)getpid(),data_processed);
}
}exit(EXIT_SUCCESS);
}
The program creates a pipe and then forks. Now both parent and child process will have its
ownfiledescriptorsforreadingand writing.Thereforetotallytherearefourfiledescriptors.
The child process will close its standard input with close(0) and calls duo(file_pipes[0]).
Thiswill duplicate the file descriptor associated with the read end. Then child closes its original
filedescriptor.Aschild willneverwrite,italsoclosesthe writefile descriptor,
file_pipes[1]. Now there is only one file descriptor 0 associated with the pipe that is
standardinput.Next,childusestheexec toinvoke anyprogramthatreadsstandardinput.
Theodcommandwillwaitforthedatatobeavailablefromtheuserterminal.
Since the parent never read the pipe, it starts by closing the read end that is file_pipe[0].
Whenwriting process of data has been finished, the write end of the parent is closed and exited. As
thereare no file descriptor open to write to pipe, the od command will be able to read the three
byteswrittento pipe,meanwhilethereading processwillreturn0 bytesindicating theend ofthefile.
Namedpipes.
Unnamedpipes(Anonymouspipes)
Namedpipes(FIFOs)
Similartopipes,butallowsforcommunicationbetweenunrelatedprocesses.This
isdonebynamingthecommunicationchannelandmakingitpermanent.
Likepipe,FIFOistheunidirectionaldatastream.
FIFOcreation:
intmkfifo (const char*pathname,mode_tmode);
- makesaFIFOspecialfilewithnamepathname.
(modespecifiestheFIFO'spermissions, ascommoninUNIX-likefilesystems).
- AFIFOspecialfileissimilar toapipe,exceptthatitiscreatedinadifferentway.Insteadofbeingananonymous
communicationschannel,aFIFOspecialfileisentered intothefilesystembycallingmkfifo()
OnceaFIFOspecialfilehasbeencreated,anyprocesscanopenitforreadingorwriting,inthesame wayas
anordinaryfile.
169
Department of CSE
A First-in, first-out(FIFO) file is a pipe that has a name in the filesystem. It is also called
asmedpipes.
CreationofFIFO:
WecancreateaFIFOfromthecommandlineandwithinaprogram.
(Note:Themknodcommandisavailableonlynolderversions,youcanmakeuseofmkfifoinnewversions.
)
TocreateFIFOwithintheprogramwecanusetwosystemcalls.Theyare,#include<sys/types.h>
#include<sys/stat.h>
intmkfifo(constchar
*filename,mode_tmode);
intmknod(constchar*filename,mod
e_t mode|S_IFIFO,(dev_t)0);
IfwewanttousethemknodfunctionwehavetouseORingprocessoffileaccessmodewithS_IFIFOandt
hedev_tvalueof0.Insteadofusingthiswecanusethesimplemkfifofunction.
AccessingFIFO:
LetusfirstdiscusshowtoaccessFIFOincommandlineusingfilecommmands.Theusefulfeatureofnam
edpipesis,astheyappearinthefilesystem,wecanusethemincommands.
WecanreadfromtheFIFO(empty)
$cat</tmp/my_fifo
Now,letuswritetotheFIFO.
$echo "Simple!!!">/tmp/my_fifo
(Note:Thesetwocommandsshouldbeexecutedindifferentterminalsbecausefirstcommandwillbewait
ingforsomedatatoappearintheFIFO.)
FIFO can also be accessed as like a file in the program using low-level I/O functions or C
libraryI/Ofunctions.
The only difference between opening a regular file and FIFO is the use of open_flag with
theoptionO_NONBLOCK. The only restriction is that we can‘t open FIFO for reading and
writingwithO_RDWRmode.
//fifo1.c
#include<unistd.h>#in
clude<stdlib.h>#includ
e<stdio.h>#include<str
ing.h>#include<fcntl.h
>#include<limits.h>
170
Department of CSE
#include<sys/types.h>
#include<sys/stat.h>
#defineFIFO_NAME"/tmp/my_fifo"#d
efineBUFFER_SIZEPIPE_BUF#define
TEN_MEG(1024* 1024* 10)
intmain()
{
intpipe_fd;i
ntres;
intopen_mode=O_WRONLY;in
tbytes_sent= 0;
charbuffer[BUFFER_SIZE+1];
if(access(FIFO_NAME,F_OK)==-
1){res= mkfifo(FIFO_NAME,0777);
if(res!=0){
fprintf(stderr, "Couldnotcreatefifo
%s\n",FIFO_NAME);exit(EXIT_FAILURE);
}
}
printf("Process%d openingFIFO
O_WRONLY\n",getpid());pipe_fd=
open(FIFO_NAME,open_mode);
printf("Process%d result%d\n",getpid(),pipe_fd);
if(pipe_fd!=-1){
while(bytes_sent<TEN_MEG){
res=write(pipe_fd,buffer,BUFFER_SIZE);if(re
s ==-1){
fprintf(stderr,"Writeerroronpipe\n");ex
it(EXIT_FAILURE);
}
}
(void)close(pipe_fd);
}
else{exit(EXIT_FAILURE);
}
printf("Process%dfinished\n",getpid());e
xit(EXIT_SUCCESS);
}
//fifo2.c
#include<unistd.h>#in
clude<stdlib.h>#includ
e<stdio.h>#include<str
ing.h>#include<fcntl.h
>#include<limits.h>#i
nclude
<sys/types.h>#include
<sys/stat.h>
#define
171
Department of CSE
FIFO_NAME"/tmp/my_fifo"#defineB
UFFER_SIZEPIPE_BUFint
172
Department of CSE
main()
{
int pipe_fd;
intres;
int open_mode = O_RDONLY;
charbuffer[BUFFER_SIZE+1];intby
tes_read=0;
memset(buffer,'\0',sizeof(buffer));
printf("Process%d openingFIFO
O_RDONLY\n",getpid());pipe_fd=
open(FIFO_NAME,open_mode);
printf("Process%dresult%d\n", getpid(),
pipe_fd);if(pipe_fd!=-1){
do{
res=
read(pipe_fd,buffer,BUFFER_SIZE);bytes_
read+=res;
}while(res>0);(void)close(
pipe_fd);
}
else {
exit(EXIT_FAILURE);
}
printf("Process %d finished, %d bytes read\n", getpid(),
bytes_read);exit(EXIT_SUCCESS);
}
Bothfifo1.candfifo2.cprogramsusetheFIFOinblockingmode.
First fifo1.c is executed .It blocks and waits for reader to open the named pipe. Now
writerunblocks and starts writing data to pipe. At the same time, the reader starts reading data
from thepipe.
Unnamedpipes(AnonymousPipes)
Anonymous pipes don't have names, therefore they can be used only between related
processeswhichcaninheritthefilehandles (file descriptors).
Anonymous pipes are typically used to "pipe" two programs: standard output from one
programis redirected to the pipe input (write handle), while the standard input of the second
program isredirected to from the pipe output (read handle). The pipe is created by the parent
(usually theloginshell),and thepipehandlesarepassed tochildrenthroughtheinheritancemechanism.
Anonymous pipes cannot be used across a network. Also, anonymous pipes are unidirectional-
inorder to communicate two related processes in both directions, two anonymous pipes must
becreated.
//*******************************************************************
//Thisprogramimplementspipingofprogramsp1.exeandp2.exe
//throughananonymouspipe.Theprogramcreatestwo child processes
173
Department of CSE
//(whichexecuteimagesp1.exeandp2.exe) andapipe,thenpasses
//thepipehandlestothechildren.
174
Department of CSE
//
//Theprogramis invokedas:pipep1p2(nocommandlinearguments)
//*******************************************************************
#include <windows.h>
#include<iostream.h>
intmain(int argc,char*argv[])
{
//Createanonymous
(unnamed)pipeSECURITY_ATTRI
BUTESsa;
sa.nLength=sizeof(SECURITY_ATTRIBUTES);sa.lpSecurity
Descriptor=0;
sa.bInheritHandle=TRUE;//Handlesareinheritable(defaultisFALSE)HAN
DLErh,wh;
//Readandwritehandlesofthepipeif(!CreatePip
e(&rh,&wh,&sa,0))
{
cout<<"Couldn'tcreatepipe "<<GetLastError()<<
endl;return(1);
}
//Createthefirstchildprocessp1PROC
ESS_INFORMATIONpi1;STARTU
PINFO si1;
GetStartupInfo(&si1); //Getdefaultstartupstructure
si1.hStdInput=GetStdHandle(STD_INPUT_HANDLE);si1.hStd
Output=wh; // Std output of p1 is input to the
pipesi1.dwFlags=STARTF_USESTDHANDLES;
CreateProcess(argv[1], // Name of the p1's image (without
".exe."0,0,0,
TRUE, //Eachopeninheritablehandleofthe
//parentwillbeinheritedbythechild
0,0,0,
&si1,&pi1);
CloseHandle(wh); //Pipehandlenolongerneeded
//Createthesecondchildprocessp2PROCE
SS_INFORMATION
pi2;
STARTUPINFO si2;
GetStartupInfo(&si2);//Getdefaultstartupstructuresi2.hStdInput=rh
; // Std input of p2 is otput from the
pipesi2.hStdOutput=GetStdHandle(STD_OUTPUT_HANDLE);si
2.dwFlags=STARTF_USESTDHANDLES;
CreateProcess(0,argv[2], //
Nameofthep1'simage(without".exe."0,0,
TRUE, //Eachopeninheritablehandleofthe
//parentwillbeinheritedbythechild
0,0,0,
&si2,&pi2);WaitForSingleObject(pi1.hProcess,
INFINITE);CloseHandle(pi1.hProcess);WaitFo
rSingleObject(pi2.hProcess,INFINITE);CloseH
andle(pi2.hProcess);
175
Department of CSE
CloseHandle(rh);
176
Department of CSE
return(0);
}
Comment:
In order to communicate two processes (P1 and P2) through anonymous pipes by redirecting
thestandard I/O, the processes don't have to be aware of the existence of pipes, i.e. their sources
andimagesdon'thavetobemodified.
Pipeprocessing:(popen&pcloselibraryfunctions)
The process of passing data between two programs can be done with the help of popen()
andpclose()functions.
#include<stdio.h>
FILE*popen(constchar*command
,constchar*open-mode);
intpclose(FILE*stream_to_close);
popen():
The popen function allows a program to invoke another program as a new process and
eitherwrite the data to it or to read from it. The parameter command is the name of the program
to run.The open_mode parameter specifies in which mode it is to be invoked, it can be only
either "r" or"w".Onfailurepopen()returnsaNULLpointer.Ifyouwanttoperformbi-
directionalcommunicationyouhave tousetwopipes.
pclose():
By using pclose(), we can close the filestream associated with popen() after the process
startedby it has been finished. The pclose() will return the exit code of the process, which is to
beclosed. If the process was already executed a wait statement before calling pclose, the exit
statuswill be lost because the process has been finished. After closing the filestream, pclose()
will waitforthechildprocess toterminate.
Messagequeue:
This isaneasy wayofpassingmessagebetweentwoprocess.Itprovidesaway ofsending a block of
data from one process to another. The main advantage of using this is, eachblock of data is
considered to have a type, and a receiving process receives the blocks of
datahavingdifferenttypevaluesindependently.
Creationandaccessingofamessagequeue:
Youcancreateandaccessamessagequeueusingthemsgget()function.#inclu
de<sys/msg.h>
int msgget(key_tkey,int msgflg);
The first parameter is the key value, which specifies the particular message queue. The
specialconstant IPC_PRIVATE will create a private queue. But on some Linux systems the
messagequeuemaynotactuallybe private.
Thesecondparameteristheflagvalue, whichtakesninepermissionflags.
177
Department of CSE
Addingamessage:
The msgsnd() function allows to add a message to a message
queue.#include<sys/msg.h>
int msgsnd(intmsqid,constvoid*msg_ptr,size_tmsg_sz,intmsgflg);
The second parameter is the pointer to the message to be sent. The third parameter is the size
ofthe message pointed to by msg_ptr. The fourth parameter, is the flag value controls what
happensif either the current message queue is full or within the limit. On success, the function
returns 0and a copy of the message data has been taken and placed on the message queue, on
failure -1 isreturned.
Retrievingamessage:
Thesmirch()functionretrievesmessage
fromthemessagequeue.#include<sys/msg.h>
intmsgsnd(intmsqid,constvoid*msg_ptr
,size_tmsg_sz,longintmsgtype
,intmsgflg);T
hesecondparameterisapointertothemessagetobereceived.
The fourth parameter allows a simple form of reception priority. If its value is 0,the
firstavailable message in the queue is retreived. If it is greater than 0,the first message type
isretrived. If it is less than 0,the first message that has a type the same a or less than the
absolutevalue ofmsgtypeis retrieved.
On success, msgrcv returns the number on bytes placed in the receive buffer, the message
iscopied into the user-allocated buffer and the data is deleted from the message queue. It returns
-1onerror.
Controllingthemessagequeue:
Thisisverysimilar thatofcontrolfunctionofsharedmemory.
#include<sys/msg.h>
intmsgctl(intmsgid,intcommand,st
ructmsqid_ds*buf);
Thesecondparametertakesthe valuesasgivenbelow:
2.) IPC_SET - If the process has the permission to do so, this sets the values associated with
themessage queue tothose providedinthemsgid_dsdatastructure.
3.) IPC_RMID-Deletesthemessagequeue.
(Note: If the message queue is deleted while the process is writing in a msgsnd or
msgrcvfunction,thesendorreceivefunctionwillfail.
178
Department of CSE
Client/serverExample:
//msgq1.c#include
<stdlib.h>#include
<stdio.h>#include
<string.h>#include
<errno.h>#include
<unistd.h>
#include<sys/msg.h>
structmy_msg_st
{
long int
my_msg_type;char
some_text[BUFSIZ];
};
intmain()
{
int running =
1;intmsgid;
structmy_msg_stsome_data;l
ong intmsg_to_receive=0;
msgid=msgget((key_t)1234,0
666|IPC_CREAT);
if(msgid==-1)
{
fprintf(stderr,"failedtoget:\n");e
xit(EXIT_FAILURE);
}
while(running)
{
if(msgrcv(msgid,(void*)&some_data,
BUFSIZ,msg_to_receive,0)==-1)
{
fprintf(stderr,"failedtoreceive:\n");e
xit(EXIT_FAILURE);
}
printf("You Wrote:
179
Department of CSE
%s",some_data.some
_text);
180
Department of CSE
if(strncmp(some_data.some_text,"end",3)
== 0)
{
running = 0;
}
}
if(msgctl(msgid,IPC_RMID,0)==-1)
{
fprintf(stderr,"failedtodelete\n");e
xit(EXIT_FAILURE);
}exit(EXIT_SUCCESS
);
}
//msgq2.c
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include
<unistd.h>
#include<sys/msg.h>#defi
neMAX_TEXT 512
structmy_msg_st
{
longintmy_msg_type;
charsome_text[MAX_TEXT];
};
intmain()
{
intrunning=1;
structmy_msg_stsome_data;i
ntmsgid;
charbuffer[BUFSIZ];
msgid=msgget((key_t)1234,0
666|IPC_CREAT);
if(msgid==-1)
{
fprintf(stderr,"failedtocreate:\n");e
xit(EXIT_FAILURE);
}
while(running)
{
printf("EnterSomeText:");fgets(buffer,
BUFSIZ,stdin);some_data.my_msg_ty
pe=1;strcpy(some_data.some_text,buff
181
Department of CSE
er);
182
Department of CSE
if(msgsnd(msgid,(void*)&some_data,MAX_TEXT,0)==-1)
{
fprintf(stderr,"msgsndfailed\n");e
xit(EXIT_FAILURE);
}
if(strncmp(buffer,"end", 3)==0)
{
running = 0;
}
}exit(EXIT_SUCCESS
);
}
The msgq1.c program will create the message queue using msgget() function.The
msgid identifier is returned by the msgget().The message are received from the
queueusingmsgrcv()functionuntil
thestring"end"isencountered.Thenthequeueisdeletedusingmsgctl()function.
Semaphore:
While we are using threads in our programs in multi-user systems, multiprocessing system, or
acombination of two, we may often discover critical sections in the code. This is the section
wherewehave toensurethata singleprocesshasexclusiveaccesstotheresource.
It is counter variable, which takes only the positive numbers and upon which programs can
onlyact atomically. The positive number is the value indicating the number of units of the
sharedresourcesareavailableforsharing.
Thecommonformofsemaphoreisthebinarysemaphore,whichwillcontrolasingleresource,andits
valueisinitializedto0.
Creationofsemaphore:
The shmget() function creates a new semaphore or obtains the semaphore key of an
existingsemaphore.
#include<sys/sem.h>intshmget(ke
y_tkey,intnum_sems,intsem_flags)
;
The first parameter, key, is an integral value used to allow unrelated process to access the
samesemaphore. The semaphore key is used only by semget. All others use the identifier return
by thesemget(). There is a special key value IPC_PRIVATE whichallows to create the
semaphore andto be accessed only by the creating
183
Department of CSE
process.
184
Department of CSE
Thesecondparameteristhenumberofsemaphoresrequired,itisalmostalways1.Thethird
parameteristhe setofflags.The ninebitsarethe permissionsforthe semaphore.
Onsuccessitwillreturnapositivevaluewhichistheidentifierusedbytheothersemaphorefunctions.On
error,itreturns-1.
Changingthevalue:
Thefunctionsemop()isusedforchangingthevalueofthesemaphore.
#include<sys/sem.h>
intsemop(intsem_id,structsembuf
*sem_ops,size_tnum-_sem_ops);
Thefirstparameteristheshmidistheidentifierreturnedbythesemget().
Thesecondparameteristhepointertoanarrayofstructure.Thestructuremaycontainatleastthe
followingmembers:
structsembuf{s
hort
sem_num;short
sem_op;shortse
m_flg;
}
The first member is the semaphore number, usually 0 unless it is an array of semaphore.
Thesem_op is the value by which the semaphore should be changed. Generally it takes -
1,which isoperation to wait for a semaphore and +1, which is the operation to signal the
availability ofsemaphore.
The third parameter, is the flag which is usually set to SET_UNDO. If the process
terminateswithout releasingthesemaphore,thisallowstoreleaseitautomatically.
Controllingthesemaphore:
Thesemctl()functionallowsdirectcontrolofsemaphoreinformation.
#include<sys/sem.h>
intsemctl(intsem_id,intsem_num,intc
ommand,.../*unionsemunarg*/);
Thethirdparameteristhecommand,whichdefinestheactiontobetaken.Therearetwocommonval
ues:
Filelockingwithsemaphores
//sem.c
#include<unistd.h>#in
clude<stdlib.h>#inclu
de<stdio.h>#include<
185
Department of CSE
sys/sem.h>
186
Department of CSE
#include<sys/ipc.h>#i
nclude<sys/types.h>u
nionsemun
{
intval;
struct semid_ds
*buf;unsigned
short*array;
};
staticvoid
del_semvalue(void);s
taticint
set_semvalue(void);s
taticint
semaphore_p(void);s
taticint
semaphore_v(void);s
taticint sem_id;
//
staticintset_semvalue()
{
unionsemunsem_union;s
em_union.val= 1;
if (semctl(sem_id, 0, SETVAL, sem_union) == -1)
return(0);return(1);
}
//
staticvoiddel_semvalue()
{
unionsemunsem_union;
if(semctl(sem_id,0,IPC_RMID, sem_union) ==-
1)fprintf(stderr,"Failedtodeletesemaphore\n");
}
//
staticintsemaphore_p()
{
struct sembuf
sem_b;sem_b.sem_n
um =
0;sem_b.sem_op=-
1;/*P()*/sem_b.sem_flg=SEM_U
NDO;
if(semop(sem_id,&sem_b,1)==-1)
{
fprintf(stderr,
"semaphore_pfailed\n");return(0);
}
return(1);
}
187
Department of CSE
//
staticintsemaphore_v()
{
struct sembuf
sem_b;sem_b.sem_n
um =
0;sem_b.sem_op=1;/
*V()*/sem_b.sem_flg=SEM_UND
O;
188
Department of CSE
for(i= 0;i<10;i++)
{
if (!semaphore_p())
exit(EXIT_FAILURE);printf("%c",op_char);
fflush(stdout);pause_time
=rand()%3;sleep(pause_t
ime);
printf("%c",op_char);fflush(stdout);
if (!semaphore_v())
exit(EXIT_FAILURE);pause_time=rand()
%2;sleep(pause_time);
}
printf("\n%d -
finished\n",getpid());if(argc>1)
{
sleep(10);del_s
emvalue();
}exit(EXIT_SUCCESS
);
}
thecommandIPC_RMIDinthesemctl()function.Thefunctionsemaphore_p()changesthe
190
Department of CSE
Inthefunctionsemaphore_v(),thesemopmemberofthestructuresembufissetto1.Bythisthesemphore
becomes available for the other processes because it isreleased.
191
Department of CSE
UNITV
SharedMemory:
Shared memory is a highly efficient way of data sharing between the running programs. It
allowstwo unrelated processes to access the same logical memory. It is the fastest form of IPC
becauseallprocessessharethesamepieceofmemory.Italsoavoidscopyingdataunnecessarily.
As kernel does not synchronize the processes, it should be handled by the user. Semaphore
canalsobe usedtosynchronize the access tosharedmemory.
Usageofsharedmemory:
To use the shared memory, first of all one process should allocate the segment, and then
eachprocess desiring to access the segment should attach the segment. After accessing the
segment,eachprocessshould detach it.Itisalsonecessaryto deallocatethesegmentwithoutfail.
Allocating the shared memory causes virtual pages to be created. It is important to note
thatallocatingtheexistingsegmentwouldnotcreatenewpages,butwillreturntheidentifierfortheexistin
gpages.
All the shared memory segments are allocated as the integral multiples of the system's page
size,whichis thenumberofbytesinapageofmemory.
Unixkernelsupportforsharedmemory
There is a shared memory table in the kernel address space that keeps track of all
sharedmemoryregions createdinthesystem.
Eachentryofthetables storethefollowingdata:
1. Name
2. CreatoruserIDandgroupID.
3. Assignedowneruser IDandgroupID.
4. Read-writeaccess permissionoftheregion.
5. Thetimewhenthelastprocessattachedtotheregion.
6. Thetimewhenthelastprocessdetachedfromtheregion.
7. Thetimewhenthelastprocesschangedcontroldataoftheregion.
8. Thesize,inno.ofbytesoftheregion.
UNIXAPIsforsharedmemoryshmget
Openandcreateasharedmemory.
Functionprototype:
#include<sys/types.h>
#include<sys/ipc.h>#i
nclude<sys/shm.h>
192
Department of CSE
int shmget(key_tkey,intsize,intflag);
Functionreturnsapositivedescriptorifitsucceedsor-1ifitfails.
Shmat
Attachasharedmemorytoaprocessvirtualaddressspace.
Functionprototype:
#include<sys/types.h>
#include<sys/ipc.h>#i
nclude<sys/shm.h>
void*shmat(intshmid,void*addr,intflag);
Shmdt
Detachasharedmemoryfromtheprocess virtualaddressspace.
Functionprototype:
#include<sys/types.h>#
include<sys/ipc.h>#incl
ude<sys/shm.h>intshm
dt (void*addr);
Functionreturns0ifitsucceedsor-1ifitfails.
Shmctl
#include<sys/types.h>
#include<sys/ipc.h>#i
nclude<sys/shm.h>
intshmctl(intshmid,intcmd, structshmid_ds*buf);
Functionreturns0ifitsucceedsor-1ifitfails.
193
Department of CSE
SharedmemoryExample
//shmry1.c
#include<unistd.h>#i
nclude<stdlib.h>#inc
lude<stdio.h>#includ
e<string.h>#include
<sys/shm.h>
#defineTEXT_SZ2048s
tructshared_use_st
{
intwritten_by_you;
charsome_text[TEXT_SZ];
};
intmain()
{
intrunning=1;
void *shared_memory = (void
*)0;structshared_use_st*shared_st
uff;intshmid;
srand( (unsigned int)getpid()
);shmid = shmget(
(key_t)1234,sizeof(structshare
d_use_st),0666|IPC_CREAT);
if(shmid==-1)
{
fprintf(stderr,"shmgetfailed\n");
exit(EXIT_FAILURE);
}
shared_memory=shmat(shmid,(void*)0,0);if(
shared_memory==(void*)-1)
{
fprintf(stderr, "shmat
failed\n");exit(EXIT_
FAILURE);
}
printf("Memory Attached at
%x\n",(int)shared_memory);
shared_stuff=(structshared_use_st*)sh
ared_memory;
shared_stuff->written_by_you =
0;while
(running)
194
Department of CSE
195
Department of CSE
if(shared_stuff->written_by_you)
{
printf("YouWrote:%s",sh
ared_stuff->some_text);
sleep(rand()%4);
shared_stuff->written_by_you=0;
if(strncmp(shared_stuff-
>some_text,"end",3)==0)
{
running= 0;
}
}
}
if(shmdt(shared_memory) ==-1)
{
fprintf(stderr,"shmdtfailed\n");e
xit(EXIT_FAILURE);
}
if(shmctl(shmid,IPC_RMID,0)==-1)
{
fprintf(stderr,"failedtodelete\n");
exit(EXIT_FAILURE);
}exit(EXIT_SUCCESS
);
//shmry2.c
#include<unistd.h>
#include<stdlib.h>
#include<stdio.h>#
include<string.h>
#include<sys/shm.h>
#defineTEXT_SZ2048s
tructshared_use_st
{
intwritten_by_you;
charsome_text[TEXT_SZ];
};
intmain()
{
intrunning=1
196
Department of CSE
shared_memory=shmat(shmid,(voi
d*)0,0);
if(shared_memory==(void*)-1)
{
fprintf(stderr, "shmat
failed\n");exit(EXIT_
FAILURE);
}
printf("MemoryAttachedat%x\n",(int)shared_memory);shared_
stuff=(structshared_use_st*)shared_memory;while(running)
{
while(shared_stuff->written_by_you==1)
{
sleep(1);
printf("waitingforclient .... \n");
}
printf("EnterSomeText:");fgets
(buffer,BUFSIZ,stdin);
strncpy(shared_stuff-
>some_text,buffer,TEXT_SZ);
shared_stuff->written_by_you=1;
if(strncmp(buffer,"end", 3)==0)
{
running = 0;
}
}
if(shmdt(shared_memory) ==-1)
{
fprintf(stderr, "shmdt
failed\n");exit(EXIT_FAILURE
);
}exit(EXIT_SUCCESS
);
197
Department of CSE
The shmry1.c program will create the segment using shmget() function and returns the
identifiershmid.Thenthatsegmentisattached toitsaddressspaceusingshmat()function.
The shmry2.c program gets and attaches to the same memory segment. This is possible with
thehelp of same key value 1234 used in the shmget() function. If the written_by_you text is set,
theprocess will wait until the previous process reads it. When the flag is cleared, the data is
writtenand sets the flag. This program too will use the string "end" to terminate. Then the
segment isdetached.
Sockets
For example,you can obtain theWWW page from aWeb server using theTelnetprogrambecause
they both use sockets for network communications.To open a connection to a WWWserver at
www.codesourcery.com, use telnet www.codesourcery.com 80.The magic constant 80specifies a
connection to the Web server programming running www.codesourcery.com insteadof some
other process.Try typing GET / after the connection is established.This sends a messagethrough
the socket to the Web server, which replies by sending the home page‘s HTML
sourceandthenclosingthe connection—forexample:
% telnet www.codesourcery.com
80Trying206.168.99.1...
Connectedtomerlin.codesourcery.com(206.168.99.1).E
scapecharacteris‗^]‘.
GET/
<html>
<head>
<metahttp-equiv=‖Content-Type‖content=‖text/html;charset=iso-8859-1‖>
...
3. Note that only Windows NT can create a named pipe;Windows 9x programs can form
onlyclientconnections.
4. Usually,you‘dusetelnettoconnectaTelnetserverforremotelogins.Butyoucanalsousetelnettoco
nnectto aserverofadifferentkind and thentypecommentsdirectlyatit.
IntroductiontoBerkeleysockets
Berkeley sockets (or BSD sockets) is a computing library with an application
programminginterface(API)forinternetsocketsandUnixdomainsockets,usedforinter-
processcommunication(IPC).
198
Department of CSE
ThislistisasummaryoffunctionsormethodsprovidedbytheBerkeleysocketsAPIlibrary:
socket() creates a new socket of a certain socket type, identified by an integer
number,andallocates systemresources toit.
bind()istypicallyusedontheserver
side,andassociatesasocketwithasocketaddressstructure,i.e.a
specifiedlocalportnumberand IPaddress.
listen()isusedontheserverside,andcausesaboundTCPsockettoenterlisteningstate.
connect() is used on the client side, and assigns a free local port number to a socket.
Incase of a TCP socket, it causes an attempt to establish a new TCP connection.accept()
isused on the server side. It accepts a received incoming attempt to create a new
TCPconnection from the remote client, and creates a new socket associated with the
socketaddresspairofthisconnection.
send()andrecv(),orwrite() andread(),or sendto()andrecvfrom(),
areusedforsendingandreceivingdatato/fromaremotesocket.
close() causes the system to release resources allocated to a socket. In case of TCP,
theconnectionisterminated.
gethostbyname()andgethostbyaddr()areusedtoresolvehostnamesandaddresses.IPv4only.
select()isusedtopend,waitingfor oneor moreofaprovidedlistof socketsto
bereadytoread,readytowrite,orthathaveerrors.
poll()isusedtocheckonthestateofasocketinasetofsockets.Thesetcanbetestedtoseeif
anysocketcanbewrittento,readfromorifanerroroccurred.
getsockopt()isusedtoretrievethecurrentvalueofaparticularsocketoptionforthespecifie
dsocket.
setsockopt()isusedtosetaparticularsocket optionforthespecifiedsocket.
IPCovera networkSocketConcepts
Whenyoucreateasocket, youmustspecifythreeparameters:
communicationstyle,
namespace,
protocol.
Acommunicationstylecontrolshowthesockettreatstransmitted dataandspecifies
the number of communication partners.When data is sent through a socket, it is ackaged
intochunkscalledpackets.The communicationstyle determineshowthese
packetsarehandled and howtheyareaddressedfromthesendertothereceiver.
Connection styles guarantee delivery of all packets in the order they were sent.
Ifpackets are lost or reordered by problems in the network, the receiver
automaticallyrequeststheirretransmissionfromthe sender.
Aconnection-stylesocketis likeatelephonecall:Theaddressesofthesender
and receiver are fixed at the beginning of the communication when the
connectionisestablished.
Datagramstylesdonotguaranteedeliveryorarrivalorder.Packetsmaybelostorreorder
ed in transit due to network errors or other conditions. Each
packetmustbelabeledwithitsdestinationandisnotguaranteedtobedelivered.Thesyste
m
199
Department of CSE
A socket namespace specifies how socket addresses are written. A socket address identifies
oneendo fasocketconnection.Forexample,socketaddressesint he―localnamespace‖areordinary
filenames.In―Internetnamespace,‖asocketaddressiscomposedoftheInternetaddress(also
knownasanInternetProtocoladdressorIPaddress)ofahostattachedtothenetworkandaport
number.The port numberdistinguishesamongmultiple socketsonthesamehost.
Aprotocolspecifieshowdataistransmitted.SomeprotocolsareTCP/IP,theprimarynetworking
protocols used by the Internet; the AppleTalk network protocol; and the UNIX
localcommunicationprotocol.Notallcombinationsofstyles,namespaces,andprotocolsaresupported.
Client-serverdatagramsocket—example
ToexperimentwithdatagramsocketsintheUNIXdomainwewillwriteaclient/serverapplicationw
here:
the client takes a number of arguments on its command line and send them to the
serverusingseparatedatagrams
foreachdatagramreceived,theserverconvertsittouppercaseandsenditbacktotheclient
theclientprintsserverrepliestostandardoutput
Forthistoworkwewillneedtobindallinvolved socketstopathnames.
Client-serverdatagramsocketexample—protocol
#include<ctype.h>#include
<sys/un.h>
#include<sys/socket.h>#in
clude<unistd.h>#include"h
elpers.h"
#defineSRV_SOCK_PATH"/tmp/uc_srv_socket"#define
CLI_SOCK_PATH " /tmp/ uc_cl i_socket .%ld "#define
MSG_LEN10
#include"uc proto.h"
int main( int argc , char *argv [ ] )
{struct sockaddr_un srv_addr , cl i_addr
;intsrv_fd,i;
ssize_t bytes
;socklen_t len;
charbuf[MSG_LEN] ;
i f ( ( srv_fd = socket (AF_UNIX , SOCK_DGRAM, 0) ) <
0)err_sys(" socketerror");
memset(&srv_addr,0,sizeof(structsockaddr_un));srv_a
ddr.sun_family=AF_UNIX ;
strncpy(srv_addr.sun_path,SRV_SOCK_PATH,size
of( srv_addr.sun_path) 1);
i
f(access(srv_addr.sun_path,F_OK)==0)unlink(
srv_addr.sun_path);
if(bind (srv_fd,( structsockaddr*)&srv_addr ,
200
Department of CSE
sizeof(structsockaddr_un))<0)err_
sys(" binderror");
for(;;){
len= sizeof(structsockaddr_un);
i f ( ( bytes = recvfrom( srv_fd , buf , MSG_LEN,
0,( structsockaddr * ) &cl i_addr , &len ) ) <
1)err_sys(" recvfromerror" );
printf("serverreceived%ldbytesfrom%s\n",(long)b
ytes,cli_addr.sun_path);
for(i=0;i<bytes ;i++)
buf[i]=
toupper((unsignedchar)buf[i]);if(sendto(srv_fd,
buf,bytes ,0,
(structsockaddr*)&cli_addr,len)!=bytes)err_sys("sen
dtoerror");
}
}
#include"uc proto.h"
int main( int argc , char *argv [ ] )
{struct sockaddr_un srv_addr , cl i_addr
;intsrv_fd,i;
size_tlen;ssize_t
bytes;
charresp[MSG_LEN];if
(argc<2)
err_quit("Usage :uc c lientMSG...");
i f ( ( srv_fd = socket (AF_UNIX , SOCK_DGRAM, 0) ) <
0)err_sys(" socketerror");
memset(&cli_addr,0,sizeof(structsockaddr_un));cli_ad
dr.sun_family=AF_UNIX ;
snpr int f ( cl i_addr . sun_path , sizeof ( cl i_addr . sun_path )
,CLI_SOCK_PATH,(long)getpid());
if(bind ( srv_fd ,(structsockaddr*
)&cli_addr,sizeof(structsockaddr_un))== 1)
err_sys("binderror");
Notes:
the server is persistent and processes one datagram at a time, no matter the client rocess,
i.e.thereisnonotionofconnectionmessageslargerthan10 bytesare silentlytruncated
UNIXdomainSockets:
We now want to give an example of stream sockets. To do so, we can longer remain in
theabstract of general sockets, but we need to pick a domain. We pick the UNIX domain. In
theUNIX domain, addresses are pathnames. Thecorresponding Cstructure is
sockaddr_un:structsockaddr_un{
sa_fami ly_tsun_family;/* = AF_UNIX*/
201
Department of CSE
charsun_path[108];/*socketpathname,NU
LL terminated*/
}
Thefieldsun_pathcontainsaregularpathname,pointingtoaspecialfileof
typesocket(.pipe)whichwillbecreatedatbindtime.
During communication the file will have no content, it is used only as a rendez-vous
pointbetweenprocesses.
Internet-DomainSockets
UNIX-domain sockets can be used only for communication between two processes on the
samecomputer. Internet-domain sockets, on the other hand, may be used to connect processes
ondifferentmachines connectedbya network.
Sockets connecting processes through the Internet use the Internet namespace represented
byPF_INET.ThemostcommonprotocolsareTCP/IP.TheInternetProtocol(IP),alow-levelprotocol,
moves packets through the Internet, splitting and rejoining the packets, if necessary.
Itguaranteesonly―best-effort‖delivery,sopacketsmayvanishorbereorderedduringtransport. Every
participating computer is specified using a unique IP number.The Transmission ControlProtocol
(TCP), layered on top of IP, provides reliable connection-ordered transport. It permitstelephone-
likeconnectionstobeestablishedbetweencomputersandensuresthatdataisdeliveredreliablyandinorde
r.
DNSNames
Becauseitiseasiertoremembernamesthannumbers,theDomainNameService(DNS)associates names
such as www.codesourcery.com with computers‘ unique IP numbers. DNS isimplemented by a
worldwide hierarchy of name servers, but you don‘t need to understand DNSprotocolstouse
Internethostnamesinyourprograms.
Internet socket addresses contain two parts: a machine and a port number.This information
isstoredinastructsockaddr_invariable.Setthesin_familyfieldtoAF_INETtoindicatethatthisis an
Internet namespace address.The sin_addr field stores the Internet address of the desiredmachine
as a 32-bit integer IP number.A port number distinguishes a given machine‘s differentsockets.
Because different machines store multibyte values in different byte orders, use htons
toconverttheportnumberto
network byte order. See the man page for ip for more information.To convert human-
readablehostnames, either numbers in standard dot notation (such as 10.0.0.1) or DNS names
(such aswww.codesourcery.com)into 32-bitIP numbers,you can use gethostbyname.This returns
apointertothe structhostentstructure;theh_addrfield containsthe host‘sIPnumber.
SystemCalls
Socketsaremoreflexiblethanpreviouslydiscussedcommunicationtechniques.Thesearethesy
stemcallsinvolvingsockets:
socket—
Createsasocketcloses—
Destroysasocket
connect—Creates
aconnectionbetweentwosocketsbind—Labelsa
serversocketwithanaddress
202
Department of CSE
listen—Configuresasockettoacceptconditions accept—
Acceptsaconnectionandcreatesanewsocketforthe
connectionSocketsarerepresentedbyfiledescriptors.
CreatingandDestroyingSockets
Sockets are IPC objects that allow to exchange data between processes
running:eitheronthe samemachine (host),orondifferentonesoveranetwork.
The UNIX socket API first appeared in 1983 with BSD 4.2. It has been finally standardized
forthe first time in POSIX.1g (2000), but has been ubiquitous to every UNIX implementation
sincethe 80s.
The socket API is best discussed in a network programming course,which this oneis not. Wewill
only address enough general socketconcepts to describe how to use a specific socket
family:UNIXdomainsockets.
ConnectionOrientedProtocol
203
Department of CSE
ConnectionlessProtocol
Client-serversetup
Let‘s consider a typical client-server application scenario — no matter if they are located on
thesame ordifferenthosts.
Socketsareusedasfollows:
eachapplication:createasocket
idea:co mmunicat ionbetweenthetwoapplicat ionswillflowthroughanimaginary―pipe‖that
willconnectthetwosocketstogether
server:binditssockettoawell-knownaddress
wehavedonethesametosetuprendez-vouspointsforother IPCobjects.
e.g.FIFOs
client:locateserversocket(viait swell-knownaddress)and―init iatecommunicat ion‖1wit hthe server.
Socketoptions:
Inordertotellthesocket togettheinformationaboutthepacketdestination,weshouldcallsetsockopt().
setsockopt()andgetsockopt()-
setandgetoptionsonasocket.Bothmethods
return0onsuccessand-1onerror.
Prototype:intsetsockopt(int
sockfd,intlevel,intoptname,...There
aretwolevelsofsocketoptions:
To manipulate options at the sockets API
level:SOL_SOCKET
Tomanipulateoptionsataprotocollevel,thatprotocolnumbershouldbeused;
204
Department of CSE
forexample,for UDPitisIPPROTO_UDPorSOL_UDP(bothareequal17)
;seeinclude/linux/in.handinclude/linux/socket.h
● SOL_IPis0.
● There are currently 19 Linux socket options and one another on option for
BSDcompatibility.
● ThereisanoptioncalledIP_PKTINFO.
WewillsettheIP_PKTINFOoptiononasocketinthefollowingexample.
// from
/usr/include/bits/in.h#defineI
P_PKTINFO8/*bool*/
/*StructureusedforIP_PKTINFO.*/structin
_pktinfo
{
intipi_ifindex;/*Interfaceindex*/
structin_addripi_spec_dst; /*Routingdestination address
*/structin_addripi_addr;/* Headerdestinationaddress*/
};
constint on=1;
sockfd=socket(AF_INET,SOCK_DGRAM,0);
if (setsockopt(sockfd, SOL_IP, IP_PKTINFO,
&on,sizeof(on))<0)
perror("setsockopt");
...
...
...
Whencallingrecvmsg(),wewillparsethemsghrlikethis:
for(cmptr=CMSG_FIRSTHDR(&msg);cmptr!=NULL;cmptr=CMSG_NXTHDR(&msg,cm
ptr))
{
if(cmptr->cmsg_level==SOL_IP &&cmptr->cmsg_type==IP_PKTINFO)
{
pktinfo=(structin_pktinfo*)CMSG_DATA(cmptr);
printf("destination=%s\n", inet_ntop(AF_INET,&pktinfo->ipi_addr,str,sizeof(str)));
}
}
Inthekernel,thiscallsip_cmsg_recv()innet/ipv4/
ip_sockglue.c. (which eventually
callsip_cmsg_recv_pktinfo()).
● You can
inthiswayretrieveotherfieldsoftheipheader:ForgettingtheT
TL:
● setsockopt(sockfd,SOL_IP,IP_RECVTTL,&on,sizeof(on))<0).
● But: cmsg_type ==
IP_TTL.Forgettingip_options:
● setsockopt()withIP_OPTIONS.
205
Department of CSE
fcntlsystemcalls
Thefcntlsystemcallprovides furtherwaystomanipulatelowlevelfiledescriptors.
descriptors.Thecall,
returnsanewfiledescriptorwithanumerical valueequaltoorgreaterthantheintegernewfd.Thecall,
returnsthefiledescriptorflagsasdefinedinfcntl.h.Thec
all,
isusedtosetthefiledescriptorflags, usuallyjustFD_CLOEXEC.The
calls,
respectivelygetandsetthefilestatusflagsandaccessmodes.
ComparisionofIPC mechanisms.
IPCmechanismsaremianly5types
1. pipes:itisrelateddataonlysendfromonepipeoutputisgivingtoanotherpipeinputtoshareresousespi
peareuseddrawback:itis onlyrelatedprocessonlycommunicated
2. messagequeues:messagequeuesareunrelatedprocessarealsocommunicatewithmessagequeue
s.
3. sockets:socketsalsoipcitiscomunicateclientsandserver
206
Department of CSE
withsocketsystemcallsconnectionorientedandconnectionlessalso
4. PIPE:Onlytworelated (eg: parent&child)processesscan
becommunicated.Datareadingwouldbefirstinfirstoutmanner.
Named PIPE or FIFO : Only two processes (can be related or unrelated) can communicate.
DatareadfromFIFOisfirstinfirstoutmanner.
5. MessageQueues:Anynumberofprocessescanread/writefrom/tothe
queue.Datacanbereadselectively.(neednotbeinFIFOmanner)
6. Shared Memory: Part of process's memory is shared to other processes. other processes
canread or write into this shared memory area based on the permissions. Accessing Shared
memoryisfasterthananyotherIPCmechanismasthisdoesnotinvolveanykernellevelswitching(Shared
memoryresidesonusermemoryarea).
7. Semaphore: Semaphores are used for process synchronisation. This can't be used for bulk
datatransferbetweenprocesses.
207