Python Introduction:

Python is a powerful high-level, object-oriented programming language created by Guido

van Rossum and first released in 1991. Python's name is derived from the television series
Monty Python's Flying Circus, and it is common to use Monty Python reference in example
code. This language is now one of the most popular languages in existence. Since 2003,
Python has consistently ranked in the top ten most popular programming languages as
measured by the TIOBE Programming Community Index.
Python is general purpose, dynamic, high level and interpreted programming
language. It supports Object Oriented programming approach to develop
applications. It is simple and easy to learn and provides lots of high-level data
structures.
Its syntax is very clean, with an emphasis on readability and uses Standard English
keywords. A basic understanding of any of the programming languages is a plus. Moreover,
an experienced developer in any programming language can pick up Python very quickly.
Python is a cross-platform programming language, meaning, it runs on multiple platforms
like Windows, Mac OS X, Linux and UNIX and has even been ported to the Java and .NET
virtual machines.

Python interpreters are available for many operating systems, allowing Python code to run
on a wide variety of systems. Most Python implementations (including CPython) include a
read–eval–print loop (REPL), meaning they can function as a command line interpreter, for
which the user enters statements sequentially and receives the results immediately. The
design of Python offers some support for functional programming in the Lisp tradition.
Python is indeed an exciting and powerful language. It has the right combination of
performance and features that make writing programs in Python both fun and easy.

Features of Python:
Open Source:
It is open source so you can freely download and use.

High-level and general purpose language.

Interpreted Language:
Python uses an interpreter to run the source code.

Easy to Learn:
Python is a beginner-friendly programming language that’s easy to learn, regardless of your
experience/knowledge with the language. If you’re a newcomer who’s looking to learn his or
her first language, give Python a try. Its simplistic syntax makes learning fun and easy, which
is a rare trait among most programming languages.

Platform Independent:
It is platform independent programming language, its code easily run on any platform such
as Windows, Linux, Unix, Macintosh etc. Thus, Python is a portable language.
Strong and Dynamic Typing Language:
A language is dynamically typed if the type is associated with run-time values, and not
named variables/fields/etc. It means user doesn’t need to provide type of a variable
explicitly.
Variable declaration:
a = 10
b = ‘hello’
c = 20.5
A language is strongly typed where it checks the type of a variable before performing an
operation.
Object-Oriented and Procedure-Oriented:
Python supports both procedure-oriented and object-oriented functions. Procedure-oriented
means the language is designed around procedures and functions that are reused
throughout the language. In comparison, object-oriented means the language is built around
objects that combine data and functionality.
Indentation:
Many of the high-level programming languages like C, C++, C# use braces { } to mark a
block of code. Python does it via indentation. A code block which represents the body of a
function or a loop begins with the indentation and ends with the first un-indented line.
Python style guidelines (PEP 8) states that you should keep indent size of four. However,
Google has its unique style guideline which limits indenting up to two spaces. So you too can
choose a different style, but we recommend to follow the PEP8.
Indentation contributes heavily in making Python code readable.
Code Quality:
Python code is highly readable which makes it more reusable and maintainable. It has broad
support for advanced software engineering paradigms such as object-oriented (OO) and
functional programming.
Developer Productivity:
Python has a clean and elegant coding style. It uses an English-like syntax and is
dynamically typed. So, you never declare a variable. A simple assignment binds a name to
an object of any type. Python code is significantly smaller than the equivalent C++/Java
code. It implies there is less to type, limited to debug, and fewer to maintain. Unlike compiled
languages, Python programs don't need to compile and link which further boosts the
developer speed.
Code Portability:
Since Python is an interpreted language, so the interpreter has to manage the task of
portability. Also, Python's interpreter is smart enough to execute your program on different
platforms to produce the same output. So, you never need to change a line in your code.
Built-In and External Libraries:
Python comes with a huge standard library and also has huge third-party library called as
PYPI (Python Package Indexing).
Component Integration:
Some applications need interaction across different components to support the end to end
workflows. One such component could be a Python script while other be a program written in
languages like Java/C++ or any other technology.
Python Installation:
https://www.python.org is the official website for python. Download python from
https://www.python.org/downloads/. Based on operating system you use.
Download required python version and install the same.

Install Python on Windows:

After downloading the required version of python double click on the exe file and install it.
On Windows machines, the Python installation is usually placed
in C:\Users\UserName\AppData\Local\Programs\Python\Python(Version), though you can
change this when you’re running the installer.

Also check the check box Add Python 3.6 to PATH to implicitly add python environment
variables to system path.
Python is by default in most of the Linux, Unix and Mac based OS.
After successful installation, open python from Windows Start, All Programs and you will see
a folder named Python<version_no>. Open it and run IDLE.
Python standard library will be located at <python_folder_loc>/Lib.
Python third-party packages installed using pip will be located at
<python_folder_loc>/Lib/site-packages.
Install Third-Party Packages:
Third party packages for python are located at a central repository called PYPI (Python
Package Index).
To install third-party packages, from PYPI to our physical environment, python provides a
utility called pip(package installer for python) which come by default with latest versions of
python and will be located at <python_folder_loc>/Scripts.
PIP Commands:
Install: to install package.
pip install <package_name> (pip install django)
pip install <package_name> == <version_no> (pip install django==1.10)
Uninstall: to uninstall package.
pip uninstall <package_name>
Freeze: Output installed packages in requirements format.
feedparser==5.1.3
wsgiref==0.1.2
django==1.4.2

Installing PIP on Ubuntu:

By default Ubuntu comes with both the versions (2.7 and 3.5) of Python.
We have to install pip manually using following commands:
For Python2:
$ sudo apt-get install python-pip python-dev build-essential
$ sudo pip install --upgrade pip
For Python3:
$ sudo apt-get install python3-pip
$ sudo pip3 install --upgrade pip3
Using PIP we can install packages in a similar way like windows as mentioned above.
Python’s traditional runtime execution model:
Source code (.py) you type is translated to byte code (.pyc), which is then run by the Python
Virtual Machine(PVM). Your code is automatically compiled, but then it is interpreted.
Create python file using .py extension and run it on terminal (command line)
“python <file_name.py>”
Python has an interactive prompt to check code snippets. Just type python on command
prompt. It will open python prompt. This prompt is used to check the code snippets and it
works like a calculator.

$ python
Python 3.6.1+ (default, Jun 8 2017, 12:24:27)
[GCC 5.4.1 20170519] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print(“Hello world.”)
Hello world.
>>> (1 + 4) * 2
10

Source code extension is .py Byte code extension is .pyc (compiled python code)

Virtual Environments for Python:

Virtualenv is a tool which allows you to create isolated environments for your python
application and allows you to install python libraries in that isolated environment instead of
installing them globally.
To Install virtualenv just type the below command in the shell: pip install virtualenv.
Create virtualenv:
From command line run virtualenv <any_name> # Creates isolated virtualenv environment in
the given name.
You can specify executable path while creating virtualenv.
Virtualenv --python=<executable location> <any_name>.
Activate virtualenv: Move to virtualenv folder cd to Scripts then type <activate>
Deactivate virtualenv: To come out of virtualenv just type <deactivate>.

Virtualenv on Ubuntu:
Install virtualenv:
sudo pip3 install virtualenv
Create a virtual environment
virtualenv venv # you can use any name instead of venv
You can also use a Python interpreter of your choice
virtualenv -p /usr/bin/python2.7 venv
Using python3:
virtualenv –p python3 venv
python3 -m venv myenv

Active your virtual environment:

source venv/bin/activate
Variables:
A variable is nothing but a reserved memory location to store values. In other words a
variable in a program gives data to the computer to work on.

Python is dynamic typing language. You do not need to declare the type of a variable.
Typically speaking python has names not variables. A name is just a label for an object. In
python each object can have lots of names. Every variable in Python is an object.

Creating variable using assignment.

The variable is always assigned with the equal sign, followed by the value of the variable.

x = 123 # integer
f = 3.14 # float
s = "hello" # string
l = [0, 1, 2] # list
t = (0, 1, 2) # tuple

Assignment is done with the equal sign:

x = 1
y = 2
You can perform the same assignment on a single line with mulitple assignments:
x, y = 1, 2
There is no limit on the number of multiple assignments but the number of variables and
targets must be identical. Note the following statement, which is valid:
>>> x, y, z = "ABC"
>>> x, y, z
('A', 'B', 'C')
The string on the left has been split into 3 elements because its length is indeed 3, like the
number of targets on the left.
It is also possible to assign a single value to multiple variables:
>>> x, y, z = 0
>>> x, y, z
(0, 0, 0)

Conventions and rules to follow while creating variables:

 Variable names can starts with any letter, or special character "_".
 Always use lowercase with words separated by underscores (snake case) as
necessary to improve readability for long variable names.
 Cannot use keywords, built-in functions as variable names.
 Variable names are case sensitive.
 Please do refer PEP-8 https://www.python.org/dev/peps/pep-0008/ for more details.

Valid names: Invalid names:

abc = 10 9abc = 10
_abc = ‘hello’ *abc = 20
abc3 = 100
Keywords:

Keywords are the reserved words in Python. These are reserved words and we cannot use
a keyword as variable name, function name or any other identifier. Keywords in Python
are case sensitive.
To list down all available keywords in python just type following statements on python
prompt.
import keyword
keyword.kwlist

Memory Management:
Python, just like other high-level programming languages such as Java, Ruby or Javascript,
manages memory automatically.

Every object we create in python has three things:

 Type
 Value
 Reference Count (Every Python object contains a reference count — a number that
shows how many names are tagged to this object.)

Reference Counting:

All objects are reference-counted. An object’s reference count is increased whenever it’s
assigned to a new name or placed in a container such as a list, tuple, or dictionary, as shown
here:

a = 10 # Creates an object with value 10

b=a # Increases reference count on 10
c = []
c.append(b) # Increases reference count on 10

This example creates a single object containing the value 10. ‘a’ is merely a name that refers
to the newly created object. When b is assigned a, b becomes a new name for the same
object and the object’s reference count increases. Likewise, when you place b into a list, the
object’s reference count increases again. Throughout the example, only one object contains
10. All other operations are simply creating new references to the object.

Garbage Collection:

A way for a program to automatically release memory when the object taking up that space is
no longer in use.
An object’s reference count is decreased by the del statement or whenever a reference goes
out of scope (or is reassigned). del statement doesn’t delete objects. It removes the name as
reference to that object and reduces the reference count by 1.

Here’s an example:

del a # Decrease reference count of 10

b = 20 # Decrease reference count of 10
c[0] = 2.0 # Decrease reference count of 10

When an object’s reference count reaches zero, it is garbage-collected.

The current reference count of an object can be obtained using the sys.getrefcount() function.

For example:

>>> a = 10
>>> import sys
>>> sys.getrefcount(a)
7

In many cases, the reference count is much higher than you might guess. For immutable data
such as numbers and strings, the interpreter aggressively shares objects between different
parts of the program in order to conserve memory.

Unused objects are moved to garbage collector using destructor called __del__. Runs before
an object is removed from the memory.

Operators in Python:
Operator: An operator is a symbol which specifies a specific action.
Operand: An operand is a data item on which operator acts.

Python language supports the following types of operators.

1. Arithmetic Operators
2. Comparison (Relational) Operators
3. Assignment Operators
4. Logical Operators
5. Bitwise Operators
6. Membership Operators
7. Identity Operators

Arithmetic Operators:
Assume variable a holds 10 and variable b holds 20, then −

Operator Description Example

+ Addition Adds values on either side of the operator. a + b = 30

 - Subtraction Subtracts right hand operand from left hand operand. a – b = -10

* Multiplies values on either side of the operator a * b = 200

Multiplication

/ Division Divides left hand operand by right hand operand b/a=2

% Modulus Divides left hand operand by right hand operand and returns remainder b%a=0

** Exponent Performs exponential (power) calculation on operators a**b =10 to

the power
20

// Floor Division - The division of operands where the result is the 9//2 = 4 and
quotient in which the digits after the decimal point are removed. But if 9.0//2.0 =
one of the operands is negative, the result is floored, i.e., rounded away 4.0, -11//3 =
from zero (towards negative infinity) − -4

Comparison Operators:
These operators compare the values on either sides of them and decide the relation among
them. They are also called Relational operators.
Assume variable a holds 10 and variable b holds 20, then

Operator Description Example

== If the values of two operands are equal, then the condition becomes (a == b) is
true. not true.

!= If values of two operands are not equal, then condition becomes true. (a != b) is
true.

<> If values of two operands are not equal, then condition becomes true. (a <> b) is
true. This is
similar to !=
operator.

> If the value of left operand is greater than the value of right operand, (a > b) is not
then condition becomes true. true.
 < If the value of left operand is less than the value of right operand, then (a < b) is
condition becomes true. true.

>= If the value of left operand is greater than or equal to the value of right (a >= b) is
operand, then condition becomes true. not true.

<= If the value of left operand is less than or equal to the value of right (a <= b) is
operand, then condition becomes true. true.

Assignment Operators:
Assume variable a holds 10 and variable b holds 20, then −

Operator Description Example

= Assigns values from right side operands to left side operand c=a+b
assigns value of
a + b into c

+= Add AND It adds right operand to the left operand and assign the result c += a is
to left operand equivalent to c =
c+a

-= Subtract It subtracts right operand from the left operand and assign the c -= a is
AND result to left operand equivalent to c =
c–a

*= Multiply It multiplies right operand with the left operand and assign the c *= a is
AND result to left operand equivalent to c =
c*a

/= Divide AND It divides left operand with the right operand and assign the c /= a is
result to left operand equivalent to c =
c / ac /= a is
equivalent to c =
c/a

%= Modulus It takes modulus using two operands and assign the result to c %= a is
AND left operand equivalent to c =
c%a
 **= Exponent Performs exponential (power) calculation on operators and c **= a is
AND assign value to the left operand equivalent to c =
c ** a

//= Floor It performs floor division on operators and assign value to the c //= a is
Division left operand equivalent to c =
c // a

Logical Operators:

Operator Description Example

and True if both the operands are true x and y

Or True if either of the operands is true x or y

not True if operand is false (complements the operand) not x

Bitwise Operators:
Bitwise operator works on bits and performs bit by bit operation. Assume if a = 60; and b =
13; Now in binary format they will be as follows −

a = 0011 1100
b = 0000 1101
a&b = 0000 1100
a|b = 0011 1101
a^b = 0011 0001
~a = 1100 0011

Operator Description Example

& Binary AND Operator copies a bit to the result if it exists in both (a & b) (means
operands 0000 1100)
 | Binary OR It copies a bit if it exists in either operand. (a | b) = 61
(means 0011
1101)

^ Binary XOR It copies the bit if it is set in one operand but not both. (a ^ b) = 49
(means 0011
0001)

~ Binary Ones (~a ) = -61

Complement (means 1100
0011 in 2's
It is unary and has the effect of 'flipping' bits. complement
form due to a
signed binary
number.

<< Binary Left Shift The left operands value is moved left by the number of a << 2 = 240
bits specified by the right operand. (means 1111
0000)

>> Binary Right Shift The left operands value is moved right by the number of a >> 2 = 15
bits specified by the right operand. (means 0000
1111)

Membership Operators:
Python’s membership operators test for membership in a sequence, such as strings, lists, or
tuples. There are two membership operators as explained below −

Operator Description Example

In Evaluates to true if it finds a variable in the specified sequence and x in y, here in

false otherwise. results in a 1 if x
is a member of
sequence y.

not in Evaluates to true if it does not finds a variable in the specified x not in y, here
sequence and false otherwise. not in results in a
1 if x is not a
member of
sequence y.
Identity Operators:
Identity operators compare the memory locations of two objects. There are two Identity
operators explained below:

Operator Description Example

Is Evaluates to true if the variables on either side of the operator point x is y,

to the same object and false otherwise. here is results in
1 if id(x) equals
id(y).

is not Evaluates to false if the variables on either side of the operator point x is not y, here is
to the same object and true otherwise. not results in 1.

Builtin functions in python:

Python language provides many built-in functions in handy for the ease of development. To
list down all the built-in functions available in python type below statement on python prompt.
dir(__builtins__).
dir(): Returns the attributes of the object or module.

dir(list) or dir([])

['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__',

'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__',
'__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__',
'__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear',
'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

help(): Returns the python built in documentation about the object.

type(): Returns the type of object.

type(10)  <class 'int'>

type(‘python’)  <class 'str'>

id(): Returns objects reference id.

a = 10
id(a)  124567

s = “hello”
id(s)  144563208

b = 10
id(b)  124567

input(): Accepts user input during runtime, reads and returns a line of string.

>>> input("Enter a number")

Enter a number7
‘7’

Note that this returns the input as a string. If we want to take 7 as an integer, we need to
apply the int() function to it.

>>> int(input("Enter a number"))

Enter a number7
7

Interview Questions:
What is Python?
Python is a high-level, interpreted, interactive and object-oriented programming
language. Python is designed to be highly readable. It uses English keywords frequently
where as other languages use punctuation, and it has fewer syntactical constructions than
other languages.
Python is general purpose, dynamic, high level and interpreted programming
language. It supports Object Oriented programming approach to develop applications. It is
simple and easy to learn and provides lots of high-level data structures.
List Features of Python Language?
1. It’s a general purpose, high-level, interpreted, interactive and dynamic typing
programming language.
2. It’s open source, platform independent, and case sensitive programming language.
3. It uses indentation to separate code blocks.
4. It has a huge standard and third-party library.
5. It can be used as a scripting language or can be compiled to byte-code for building
large applications.
6. It supports functional and structured programming methods as well as OOP.
7. It provides very high-level dynamic data types and supports dynamic type checking.
8. It supports automatic memory management and garbage collection.
9. Supports database, GUI, networking and sockets programming.
10. It can be easily integrated with C, C++, and Java.

Is python compiled based or interpreted based language?

Python is a general purpose programming language which supports OOP Object oriented
programming principles as supported by C++, Java, etc.

Python programs are written to files with extension .py . These python source code files are
compiled to byte code (python specific representation and not binary code), platform
independent form stored in .pyc files. This byte code helps in startup speed optimization.
These byte code are then subjected to Python Virtual Machine PVM where one by one
instructions are read and executed. It is an interpreted language.
How to write and run a program in python?

Open a file, write your statements and save it using .py extension. i.e <filename.py>.
Run your program from command line using python filename.py.

How Python is interpreted?

Python language is an interpreted language. Python program runs directly from the source
code. It converts the source code that is written by the programmer into an intermediate
language, which is again translated into machine language that has to be executed.

How to list all the built-ins available in python?

List all the built-ins in python using dir(__builtins__).

Describe all the helper utilities in python?

Function Description
dir() Returns the attributes of the object or module.
help() Returns the python built in documentation about the object.
type() Returns the type of object.
id() Returns objects reference id.
input() Accepts user input during runtime.
__doc__ Returns the doc-string of object or module.

Explain about python library?

Python has a huge standard library and it’s located at <python folder>/Lib.
Also it has a huge third-party library and it’s located at <python folder>/Lib/site-packages.

What is PEP 8?

PEP 8 is a coding convention, a set of recommendation, about how to write your Python
code more readable.

How to install and maintain third-party packages in python?

Third-party packages for python is located at central repository called PYPI (Python Package
Index). Packages can be installed using a helper utility called PIP (Package Installer for
Python).

Install package:
pip install <package_name> # installs latest version of package.
pip install <package_name == <version>> # install specific version of package.

How is memory managed in Python?

Python memory is managed by Python private heap space. All Python objects and
data structures are located in a private heap. The programmer does not have an access to
this private heap and interpreter takes care of this Python private heap.
The allocation of Python heap space for Python objects is done by Python memory
manager. The core API gives access to some tools for the programmer to code.
Python also have an inbuilt garbage collector, which recycle all the unused memory and
frees the memory and makes it available to the heap space.
Explain how garbage collection works in python?

Python have an automatic memory allocation and deallocation. It can be handled by memory
manager and garbage collector respectively. Every object in python has three things, id, type
and reference count. If the reference count of an object becomes zero, then interpreter will
move that object into garbage.

Differentiate between .py and .pyc files?

Both .py and .pyc files holds the byte code. “.pyc” is a compiled version of Python file. This
file is automatically generated by Python to improve performance. The .pyc file is having
byte code which is platform independent and can be executed on any operating system that
supports .pyc format.
Note: there is no difference in speed when program is read from .pyc or .py file; the only
difference is the load time.

Different types of operators in Python?

1. Arithmetic Operators
2. Comparison (Relational) Operators
3. Assignment Operators
4. Logical Operators
5. Bitwise Operators
6. Membership Operators
7. Identity Operators

How long can an identifier be in Python?

In Python, an identifier can be of any length. Apart from that, there are certain rules we must
follow to name one:
1. It can only begin with an underscore or a character from A-Z or a-z.
2. The rest of it can contain anything from the following: A-Z/a-z/_/0-9.
3. Python is case-sensitive, as we discussed in the previous question.
4. Keywords cannot be used as identifiers.
Explain the //, %, and ** operators in Python.
The // operator performs floor division. It will return the integer part of the result on division.
>>> 7//2  3
Normal division would return 3.5 here.
Similarly, ** performs exponentiation. a**b returns the value of a raised to the power b.
>>> 2**10  1024
Finally, % is for modulus. This gives us the value left after the highest achievable division.
>>> 13%7  6
>>> 3.5%1.5  0.5
Explain logical operators in Python.
We have three logical operators- and, or, not.
>>> False and True  False
>>> 7<7 or True  True
>>> not 2==2  False
What are membership operators?
With the operators ‘in’ and ‘not in’, we can confirm if a value is a member in another.
>>> 'me' in 'disappointment'  True
>>> 'us' not in 'disappointment'  True
Explain identity operators in Python.
The operators ‘is’ and ‘is not’ tell us if two values have the same identity.
>>> 10 is '10'  False
>>> True is not False  True
How can you declare multiple assignments in one statement?
There are two ways to do this:
>>> a,b,c=3,4,5 # This assigns 3, 4, and 5 to a, b, and c respectively
>>> a=b=c=3 # This assigns 3 to a, b, and c
What is the PYTHONPATH variable?
PYTHONPATH is the variable that tells the interpreter where to locate the module files
imported into a program. Hence, it must include the Python source library directory and the
directories containing Python source code. You can manually set PYTHONPATH, but
usually, the Python installer will preset it.
How do you take input in Python?
For taking input from user, we have the function raw_input() and input() in Python 2. Python
3 there is only input() function available.
The input() function takes, as argument, the text to be displayed for the task:
a = input(“Enter any value: ”) # Input always take the given input as string.
Data Types and Data Structures:
A data type, in programming, is a classification that specifies which type of value a variable
has. When working with data, we need ways to store it in variables so we can manipulate it.

Three of the most common data types used in programming: Numbers, Strings and
Booleans. We assigned those data types to variables one-by-one, like so:

x = 3 # numbers
a = "Python” # strings
t = True # Boolean

Datatypes available in python:

Numbers (integer, float and complex numbers)

Strings ( single quote(‘’), double quotes(“”), triple single quotes(‘’’ ’’’), triple double quotes.
Boolean (True, False)

Data Structures are efficient ways to store data, so that we can access and
manipulate data accordingly.

Data structures in python:

1. List
2. Tuple
3. Dictionary
4. Set
5. Frozenset

Everything in Python is an object. And all objects in Python can be either mutable or
immutable.

Mutable: An object that can be changed after it is created during run-time. It means you can
change their content without changing their identity. Mutable objects in python are list,
dictionary and set.

Immutable: An object that can’t be changed after it is created is called an immutable object.
Immutable objects in python are int, float, complex, string, tuple and frozen set.

1. Python handles mutable and immutable objects differently.

2. Immutable are quicker to access than mutable objects.
3. Mutable objects are great to use when you need to change the size of the object,
example list, dict etc.
4. Immutables are used when you need to ensure that the object you made will always
stay the same.
5. Immutable objects are fundamentally expensive to “change”, because doing so
involves creating a copy. Changing mutable objects is cheap.
Numbers (Immutable):
Numbers are one of the most prominent Python data types. Unlike many languages which
have only integers and floats, Python introduces complex as a new type of numbers.

The numbers in Python are classified using the following keywords.

int, float, and complex.

Integers and floating points are separated by the presence or absence of a decimal point. 5
is integer whereas 5.0 is a floating point number.
Complex numbers are written in the form, x + yj, where x is the real part and y is the
imaginary part.

a = 1  int
b = 10.5  float
c = 3+4j  complex

While integers can be of any length, a floating point number is accurate only up to 15
decimal places (the 16th place is inaccurate).
Numbers we deal with everyday are decimal (base 10) number system. But computer
programmers (generally embedded programmer) need to work with binary (base 2),
hexadecimal (base 16) and octal (base 8) number systems.
In Python, we can represent these numbers by appropriately placing a prefix before that
number. Following table lists these prefix

Number System Prefix

Binary '0b' or '0B'
Octal '0o' or '0O'
Hexadecimal '0x' or '0X'

print(0b1101011)  Output: 107

print(0xFB + 0b10)  Output: 253 (251 + 2)
print(0o15)  Output: 13

List (Mutable):
 List is one of the most frequently used and very versatile datatype used in Python.
 It’s an ordered collection of heterogeneous elements enclosed using square brackets
[ ].
 It is a mutable type means we can add or remove elements from the list.
 It maintains insertion order and are similar to arrays.
 List elements can be accessed using indexing, which starts from 0.

Creating List:
Creating a list is as simple as putting different comma-separated values in square brackets.

a = [1, 2, 3, 4]
b = ['a', 'b', 'c', 'd']
c = ['one', 'two', 'three', 'four']
Accessing List Elements:
Elements from a list can be accessed using indexing. Python indexes from 0 rather than 1.

l = [1, 2, 3, 4]
l[0]  1
l[3]  4
l[5]  Throws an index error.
d = [1, 2, 'three', 'four']
d[1]  2
d[2]  ‘three’

Python supports two types of indexing, positive and negative indexing.

Positive 0 1 2 3 4 5 6 7
index

Values 9 14 12 19 16 18 24 15

Negative -8 -7 -6 -5 -4 -3 -2 -1
index

Indexing:

a = [9, 14, 12, 19, 16, 18, 24, 15]

a[2]  12
a[5]  18
a[-1]  15
a[-4]  16
a[8]  throws error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range.

Slicing:
To retrieve a portion or part of data from a given sequence or list. A sub-list created by
specifying start/end indexes. Lists will always traverse through forward direction in case of
slicing.
Syntax:

list[start: end] # end is exclusive

list[start:] # to end of list
list[:end] # from start of list to exclusive end.
list[start: end: step] # every step'th value and default step is 1.

Examples:

a[:5]  [9, 14, 12, 19, 16] # from the beginning of list till 5th index
(excluding 5th index)
a[2:]  [12, 19, 16, 18, 24, 15] # from 2nd index to till the end.
a[2:6]  It will print elements from indexes 2 to 6(excluding 6) [12, 19, 16, 18]
default step 1.
a[2:8:2]  It will print elements from indexes 2 to 8(excluding 8) [19, 18, 15]
with step 2.
a[-1:-5]  [ ] # returns an empty list.
a[-5:-2]  [19, 16, 18].

Update list elements:

a[3] = 10  [9, 14, 12, 10, 16, 18, 24, 15]. It will update 3rd index value 19
to 10.
a[3:6] = [20, 30]  [9, 14, 12, 20, 30, 24, 15]

How to reverse a list using slicing:

a[::-1] # here step is -1 which will read a list from backward direction.

Basic List Operations:

Lists respond to the + and * operators much like strings; they mean concatenation and
repetition here too, except that the result is a new list, not a string.

In fact, lists respond to all of the general sequence operations we used on strings in the prior
chapter.
Python Expression Results Description

len([1, 2, 3]) 3 Length

[1, 2, 3] + [4, 5, 6] [1, 2, 3, 4, 5, 6] Concatenation

[1, 2, 3] * 2 [1, 2, 3, 1, 2, 3] Repetition

3 in [1, 2, 3] True Membership

for x in [1,2,3] : print (x,end = ' ') 123 Iteration

List Methods:
dir(list) will display all the methods we can perform on list object.

['__add__', '__class__', '__contains__', '__delattr__', '__delitem__',

'__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
'__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__',
'__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__',
'__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__',
'__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop',
'remove', 'reverse', 'sort']

The methods or attributes which start with __ (double underscore) and endswith __ (double
underscore) are called as magic methods.

Method Description

append(object) Appends the object at the end of the list

clear() Removes all the elements from the list keeping the structure.

copy() Creates another copy of list.

index(object) Returns the index value of the object.

count(object) It returns the number of times an object is repeated in list.

pop()/pop(index) Removes the last object or the specified indexed object.

It displays the popped object.

insert(index, object) Insert an object at the given index.

extend(sequence) It adds the sequence to existing list.

remove(object) It removes the object from the given List.

reverse() Reverse the position of all the elements of a list.

sort() It is used to sort the elements of the List.

append:
Appends adds its argument as a single element to the end of a list.

a = [ ]
a.append(5)  [5]
a.append(‘hello’)  [5, ‘hello’]
a.append([1,2,3])  [5, ‘hello’, [1,2,3]]

clear:
clear() will remove all items from the list. But the structure remains same.

a = ['one', 'two', 'three', 'four', 'five']

a.clear()  [ ]

copy: It will create another copy of list.

a = [1, 10, 5, 7]
b = a.copy()  [1, 10, 5, 7]

index(object):

data = [10,'abc','python',123.5]
data.index('python')  2
data.index('abc')  1

count(object): It gives the no. of occurrences of a given element.

data = [10, 'abc', 'python', 123.5, 100, 'rahul', 'b', 100]

data.count('abc')  1
data.count(100)  2

pop()/pop(int):

data = [10,'abc','python',123.5,786]
data.pop()  786 # removes last element from list
data.pop(1)  'abc' # removes element from the specified index.

insert(index,object):

data = ['abc',123,10.5,'a']
data.insert(2,'hello')  ['abc', 123, 'hello', 10.5, 'a']

extend(sequence):

Extend method can extend only iterables(list, tuple, string, dict)

Numbers are not iterables.

data1 = ['abc',123,10.5,'a']
data2 = ['ram',541]
data1.extend(data2)  ['abc', 123, 10.5, 'a', 'ram', 541]

remove(object):
Removes an element from the list, if the element doesn’t exist it throws an error.

data = ['abc', 123, 10.5, 'a', 'xyz']

data.remove(123)  ['abc', 10.5, 'a', 'xyz']

data.remove(10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

reverse():

list1 = [10,20,30,40,50]
list1.reverse()  [50, 40, 30, 20, 10]

sort():

In Python2 sort will work on both homogeneous and heterogeneous elements.

In Python3 sort will work on homogeneous elements.

list1 = [10, 30, 20, 90, 60]

list1.sort()  [10, 20, 30, 60, 90] # default sort order is ascending order.
list1.sort(reverse=True)  [90, 60, 30, 20, 10] # sorts in descending order.

Accessing nested lists:

a = [1, 10, 2, [20, 6, 7], 8]

a[3]  [20, 6, 7]
a[3][2]  7
a[3] = 20  [1, 10, 2, 20, 8]

Tuple (Immutable):
 It’s an ordered collection of heterogeneous elements enclosed using parentheses ( ).
 It is an immutable type means we can’t add or remove elements in tuple.
 Tuple elements can be accessed using indexing, which starts from 0.
 Default return type in python is tuple.

Tuple is similar to list. Only the difference is that list is enclosed between square bracket, tuple
between parenthesis and List is mutable type whereas Tuple is an immutable type.

t = ( ) # empty tuple
t = 1, 2, 3, 4  (1, 2, 3, 4) # default type is tuple.
t = (1,) # one element tuple.

NOTE: If Parenthesis is not given with a sequence, it is by default treated as Tuple.

Accessing Tuple:

Tuple can be accessed in the same way as list using indexing and slicing.

data1=(1, 2, 3, 4)
data2=('x', 'y', 'z')
data1[0]  1
data1[0:2]  (1, 2)
data2[-3:-1]  (‘x’, ‘y’)
data1[0:]  (1, 2, 3, 4)
data2[:2]  (‘x’, ‘y’)

Advantages of tuples over lists:

1. Processing of Tuples are faster than Lists.
2. It makes the data safe as Tuples are immutable and hence cannot be changed.
3. Tuples are used for String formatting.
4. Tuples are used whenever you want to return multiple results from a function.

We can’t modify elements in tuple using assignment, due to its immutable nature. Only two
methods we can perform on tuple which are count and index.
dir(tuple):

['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__',

'__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__',
'__getslice__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
'__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__rmul__', '__setattr__', '__sizeof__', '__str__',
'__subclasshook__', 'count', 'index']

Adding tuples:

t1 = (1, 2, 3)
t2 = (7, 8, 9)
t1 + t2  (1, 2, 3, 7, 8, 9)

Strings (Immutable):
1. Strings are sequences of characters.
2. Python strings are "immutable" which means they cannot be changed after they are
created.
3. To create a string, put the sequence of characters inside either single quotes, double
quotes, or triple quotes and then assign it to a variable.

s = 'Hello World!'  single quote strings

s = "Hello World!"  double quote strings
s = ‘’’ Welcome to Dreamwin Technologies’’’  triple single quotes
s = """Sunday  triple double quotes.
Monday
Tuesday"""

Access characters in a string:

To access characters from String, use indexing or slicing. Strings supports both positive and
negative indexing. String index starts from 0.

Indexing:
To retrieve a single character from a string.
s[0]  ‘P’
s[3]  ‘H’
s[-1]  ‘N’
s[-4]  ‘T’
Slicing:
To retrieve a portion or part of data from a given sequence or string.
There can be many forms to slice a string. As string can be accessed or indexed from both
the direction and hence string can also be sliced from both the direction that is left and right.

Syntax:

<string_name>[start:end: step]  It will exclude end index and the default step
is 1.
<string_name>[:endIndex]
<string_name>[startIndex:]

Note: startIndex in string slice is inclusive whereas endIndex is exclusive.

Note: Srtings will always traverse through forward direction in case of slicing.

data ="dreamwin"
data[0:6]  ‘dreamw’
data[2:7]  ‘eamwi’
data[:5]  ‘dream’
data[2:]  ‘eamwin’
data[-1: -6]  ‘ ’ # returns an empty string
data[-6: -1]  ‘eamwi’
data[-1:]  ‘n’
data[: -5]  ‘dre’
data[2:8:2]  ‘emi’
data[:8:2]  ‘demi’

Modifying/Deleting Python Strings.

Python Strings are by design immutable. It suggests that once a String binds to a variable; it
can’t be modified. If you want to update the String simply re-assign a new String value to the
same variable.

sample_str = 'Python String'

sample_str[2] = 'a'
# TypeError: 'str' object does not support item assignment
sample_str = 'Programming String'
print (sample_str)  ‘Programming String’

Similarly, we cannot modify the Strings by deleting some characters from it. Instead, we can
remove the Strings altogether by using ‘del’ command.

sample_str = "Python is the best scripting language."

del sample_str[1]
# TypeError: 'str' object doesn't support item deletion

del sample_str
print (sample_str)
# NameError: name 'sample_str' is not defined

Python has several built-in methods associated with the string data type. These methods let
us easily modify and manipulate strings.
List down all string methods using dir(str). Below are the methods we can perform on string
object.
['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__',
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__',
'__getnewargs__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
'__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count',
'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index',
'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower',
'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust',
'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex',
'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith',
'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

String Methods:
Capitalize: Converts the first character of a string into Upper case.

s = "welcome to dreamwin"
s.capitalize()  'Welcome to dreamwin'

Casefold(): Returns casefolded string. Converts Uppercase to lowercase.

s='DreamWin Technologies'
s.casefold()  'dreamwin technologies'

Center: Returns a new copy of the string after centering it in a field of length width

s = 'dreamwin'
s.center(20)  ' dreamwin '
s.center(20, '*')  '******dreamwin******'

Rjust: Returns a new copy of the string justified to left in field of length width.

s = "Dreamwin"
s.rjust(10)  ' Dreamwin'
s.rjust(5)  'Dreamwin'
s.rjust(12, '*')  '****Dreamwin'

Ljust: Returns a new copy of the string justified to left in field of length width.

s = "Dreamwin"
s.ljust(10)  'Dreamwin '
s.ljust(5)  'Dreamwin'
s.ljust(12, '*')  'Dreamwin****'
Endswith: endswith() method returns True if a string ends with the specified prefix(string). If
not, it returns False

s = 'dreamwin'
s.endswith('n')  True
s.endswith('h')  False
s.endswith('win')  True

Expand tabs:

s = 'xyz\t12345\tabc'
s.expandtabs()  'xyz 12345 abc'
s.expandtabs(2)  xyz 12345 abc
s.expandtabs(3)  xyz 12345 abc
s.expandtabs(4)  xyz 12345 abc
s.expandtabs(5)  xyz 12345 abc

Startswith: startswith() method returns True if a string starts with the specified prefix(string).
If not, it returns False.

s = "dreamwin tech"
s.startswith('dream')  True
s.startswith('tech')  False

Count: Counts the occurrences of a substring.

s = "Python is awesome"
s.count('s')  2
s.count('w')  1
s.count('on')  1
s.count('x')  0

Find: finds the substring and returns the first occurrences index if exists. If not find return -1.

s = 'Python is awesome'
s.find('is')  7
s.find('e')  12
s.find('x')  -1

Rfind: finds the substring from the right side and returns the first occurrences index if exists.
If not find returns -1.

s = 'Python is awesome'
s.rfind('is')  7
s.rfind('e')  16
s.rfind('x')  -1

Index: Returns the index of a given substring.

s = 'Python is awesome'
s.index('is')  7
s.index('e')  12
s.index('x')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: substring not found

Rindex: Returns the index of a given substring from the right side occurrence.

s = 'Python is awesome'
s.rindex('is')  7
s.index('e')  16
s.index('x')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: substring not found

Format: The format() reads the type of arguments passed to it and formats it according to
the format codes defined in the string.

format() method takes any number of parameters. But, is divided into two types of
parameters:

 Positional parameters - list of parameters that can be accessed with index of

parameter inside curly braces {index}
 Keyword parameters - list of parameters of type key=value, that can be accessed
with key of parameter inside curly braces {key}

"There are {} people in {} classroom".format(5, 'python')

'There are 5 people in python classroom'

"{1} and {0} are two courses in dreamwin technologies".format('RPA', 'Python')

'Python and RPA are two courses in dreamwin technologies'
"{a} and {b} are two courses in dreamwin technologies".format(a='RPA', b='Python')
'RPA and Python are two courses in dreamwin technologies'

"{a} and {} are two courses in dreamwin technologies".format('Python', a='RPA')

'RPA and Python are two courses in dreamwin technologies'

Join:
The join() method provides a flexible way to concatenate string. It concatenates each
element of an iterable (such as list, string and tuple) to the string and returns the
concatenated string.

The join() method returns a string concatenated with the elements of an iterable. If the
iterable contains any non-string values, it raises a TypeError exception.

Syntax:

String.join(iterable) # accepts only one iterable argument.

Examples:
l = ['1', '2', '3', '4']
','.join(l)  1,2,3,4
''.join(l)  1234
t = ('1', '2', '3', '4')
','.join(t)  1,2,3,4
''.join(t)  1234

s1 = 'abc'
s2 = '123'
""" Each character of s2 is concatenated to the front of s1"""
s1.join(s2))  1abc2abc3abc
""" Each character of s1 is concatenated to the front of s2"""
s2.join(s1))  a123b123c123

Split: splits the given string based on delimiter. Default delimiter in whitespace.

s = 'hello welcome to python'

s.split()  ['hello', 'welcome', 'to', 'python']
s.split(maxsplit=2)  ['hello', 'welcome', 'to python']
s = 'hello welcome, to, python'
s.split(',')  ['hello welcome', ' to', ' python']

Rsplit: rsplit() method splits string from the right at the specified separator and returns a list
of strings.

s = 'hello welcome to python'

s.rsplit()  ['hello', 'welcome', 'to', 'python']

s = 'hello welcome, to, python'

s.rsplit(',')  ['hello welcome', ' to', ' python']
s.split(':')  ['hello welcome to python']

Strip: strip() method returns a copy of the string with both leading and trailing characters
removed (based on the string argument passed).
Lstrip: lstrip() method returns a copy of the string with leading characters removed (based on
the string argument passed).
Rstrip: rstrip() method returns a copy of the string with trailing characters removed (based on
the string argument passed).
Strip methods by default removes white spaces in a string, if you don’t provide any
arguments.

s = " hello welcome to python "

s.strip()  'hello welcome to python'
s.rstrip()  ' hello welcome to python'
s.lstrip()  'hello welcome to python '
s.strip(',')  ' hello welcome to python '

s = "Python is awesome"
s.strip('Py')  'thon is awesome'
Dictionary: (Mutable Type)
1. A dictionary is an unordered collection of key-value pairs.
2. It’s a mutable type. We can add elements and remove elements from dictionary.
3. It’s a built-in mapping type in Python where keys map to values. These key-value
pairs provide an intuitive way to store data.
4. Only immutable types (numbers, strings, tuples) can be used as dictionary keys.
Values can be of any type.
5. It is also known as Associative arrays or hash tables.
Why Need A Dictionary?
The dictionary solves the problem of efficiently storing a large data set. Python has made the
dictionary object highly optimized for retrieving data.
Creating a Dictionary
1. Python syntax for creating dictionaries use curly or flower braces { } where each item
appears as a pair of keys and values.
2. The key and the value is separated by a colon (:). This pair is known as item.
3. Items are separated from each other by a comma (,). Different items are enclosed
within a curly brace and this forms Dictionary.
4. Keys must be unique in dictionary while values may not be.
General syntax of dictionary is as follows:

d = {'name': 'dreamwin', 'place': 'bangalore', 'started': 'Dec 2017'}

type(d)  <class 'dict'>
d  {'started': 'Dec 2017', 'name': 'dreamwin', 'place': 'bangalore'}

Adding items into dictionary during run-time:

dt = { }
dt[‘lang’] = ‘Python’
dt[‘year’] = 1990
dt[‘author’] = ‘Guido Van Rossum’
print(dt)  {'author': 'Guido Van Rossum', 'year': 1990, 'lang': 'Python'}

Accessing Dictionaries Elements With Keys:

Dictionaries act like a database. Here, we don’t use a number to get a particular index value
as we do with a list. Instead, we replace it with a key and then use the key to fetch its value.
Consider dictionary d = {'name': 'dreamwin', 'place': 'bangalore', 'started': 'Dec 2017'}

>>> d['name']  'dreamwin'

>>> d['place']  'bangalore'
>>> d['started']  'Dec 2017'
>>> d['course']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'course'

Modifying a Dictionary (Add/Update/Delete):

Dictionary are mutable. We can add new items or change the value of existing items using
assignment operator.
If the key is already present, value gets updated, else a new key: value pair is added to the
dictionary.

Add or update items into dictionary:

>>> c = {'name': 'dreamwin tech', 'place': 'bangalore', 'start': 2017}

>>> c['area'] = 'marathahalli'
>>> c  {'name': 'dreamwin tech', 'area': 'marathahalli', 'place':
'bangalore', 'start': 2017}
>>> c['course'] = ['python', 'datascience']
>>> c
{'course': ['python', 'datascience'], 'name': 'dreamwin tech', 'area':
'marathahalli', 'place': 'bangalore', 'start': 2017}
>>> c['course'] = 'django'
>>> c
{'course': 'django', 'name': 'dreamwin tech', 'area': 'marathahalli', 'place':
'bangalore', 'start': 2017}

Delete or remove elements from dictionary:

1. We can remove a particular item in a dictionary by using the method pop(). This method
removes as item with the provided key and returns the value.
2. The method, popitem() can be used to remove and return an arbitrary item (key, value)
form the dictionary. All the items can be removed at once using the clear() method.
3. We can also use the del keyword to remove individual items or the entire dictionary itself.

>>> squares = {1:1, 2:4, 3:9, 4:16, 5:25}

>>> squares.pop(2)  4
>>> squares  {1: 1, 3: 9, 4: 16, 5: 25}
>>> squares.popitem()  (1, 1)
>>> squares  {3: 9, 4: 16, 5: 25}
>>> del squares[5]
>>> squares  {3: 9, 4: 16}
>>> del squares
>>> squares
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'squares' is not defined

Dictionary methods:
Dictionary methods can be listed using dir(dict):
['__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__',
'__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__',
'__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear',
'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']

Clear: The clear() method removes all items from the dictionary.
>>> d = {'started': 'Dec 2017', 'name': 'dreamwin', 10: 'ten', 'place':
'bangalore', 'courses': ['Python', 'Datascience', 'Django']}
>>> d.clear()
>>> d  {}

You can also remove all elements from the dictionary by assigning empty dictionary { }.
However, there is a difference between calling clear() and assigning { } if there is another
variable referencing the dictionary.

>>> d = {10: "ten", 2: "twenty"}

>>> d1 = d # creating another reference
>>> d.clear()
>>> d  { }
>>> d1  { }
>>> d = {10: "ten", 2: "twenty"}
>>> d1 = d
>>> d = {}
>>> d  { }
>>> d1  {10: 'ten', 2: 'twenty'}

Copy: They copy() method returns a shallow copy of the dictionary. It doesn't modify the
original dictionary. It creates a new dictionary.

>>> d = {10: "ten", 20: "twenty", 30: 'thirty'}

>>> d1 = d.copy()
>>> d1  {10: 'ten', 20: 'twenty', 30: 'thirty'}

Fromkeys: The fromkeys() method creates a new dictionary from the given sequence of
elements with a value provided by the user.

>>> {}.fromkeys('python')
{'h': None, 't': None, 'p': None, 'o': None, 'n': None, 'y': None}
>>> {}.fromkeys('python', 10)
{'h': 10, 't': 10, 'p': 10, 'o': 10, 'n': 10, 'y': 10}

Get:
The get() method takes maximum of two parameters:
key - key to be searched in the dictionary
value (optional) - Value to be returned if the key is not found. The default value is None.
The get() method returns:
the value for the specified key if key is in dictionary.
None if the key is not found and value is not specified.
value if the key is not found and value is specified.

>>> d = {10: 'ten', 20: 'twenty', 30: 'thirty'}

>>> d.get(10)  'ten'
>>> d.get(40)  None
>>> d.get(40, 'forty')  'forty'

Keys: The keys() returns a view object that displays a list of all the keys.
>>> d = {'started': 'Dec 2017', 'name': 'dreamwin', 10: 'ten', 'place':
'bangalore', 'courses': ['Python', 'Datascience', 'Django']}
>>> d.keys()  dict_keys(['started', 'name', 10, 'place', 'courses'])

Values: The values() method returns a view object that displays a list of all values in a given
dictionary.

>>> d = {'started': 'Dec 2017', 'name': 'dreamwin', 10: 'ten', 'place':

'bangalore', 'courses': ['Python', 'Datascience', 'Django']}
>>> d.values()
dict_values(['Dec 2017', 'dreamwin', 'ten', 'bangalore', ['Python', 'Datascience',
'Django']])

Items: The items() method returns a view object that displays a list of a given dictionary's
(key, value) tuple pair.

>>> d = {'started': 'Dec 2017', 'name': 'dreamwin', 10: 'ten', 'place':

'bangalore', 'courses': ['Python', 'Datascience', 'Django']}
>>> d.items()
dict_items([('started', 'Dec 2017'), ('name', 'dreamwin'), (10, 'ten'), ('place',
'bangalore'), ('courses', ['Python', 'Datascience', 'Django'])])

Popitem: The popitem() returns and removes an arbitrary element (key, value) pair from the
dictionary.

>>> d = {'name': 'virat', 'age': 22, 'salary': 35000}

>>> d.popitem()  ('age', 22)
>>> d  {'name': 'virat', 'salary': 35000}

Pop: The pop() method removes and returns an element from a dictionary having the given
key.

>>> d = {'name': 'virat', 'age': 22, 'salary': 35000}

>>> d.pop('age')  22
>>> d
{'name': 'virat', 'salary': 35000}

Setdefault: The setdefault() method returns the value of a key (if the key is in dictionary). If
not, it inserts key with a value to the dictionary.
The setdefault() takes maximum of two parameters:
1. key - key to be searched in the dictionary.
2. default_value (optional) - key with a value default_value is inserted to the dictionary if
key is not in the dictionary.
3. If not provided, the default_value will be None.

>>> c = {'name': 'dreamwin tech', 'place': 'bangalore', 'start': 2017}

>>> c.setdefault('course')
>>> c  {'course': None, 'name': 'dreamwin tech', 'place': 'bangalore',
'start': 2017}
>>> c.setdefault('area', 'marathahalli')  'marathahalli'
>>> c  {'course': None, 'name': 'dreamwin tech', 'area': 'marathahalli',
'place': 'bangalore', 'start': 2017}
Update: The update() method updates the dictionary with the elements from the another
dictionary object or from an iterable of key/value pairs.

The update() method takes either a dictionary or an iterable object of key/value pairs
(generally tuples).
If update() is called without passing parameters, the dictionary remains unchanged.

>>> c = {'name': 'dreamwin tech', 'place': 'bangalore', 'start': 2017}

>>> c.update({'area': 'marathahalli'})
>>> c
{'name': 'dreamwin tech', 'area': 'marathahalli', 'place': 'bangalore', 'start':
2017}
>>> c.update(course = 'python')
>>> c
{'course': 'python', 'name': 'dreamwin tech', 'area': 'marathahalli', 'place':
'bangalore', 'start': 2017}

Sets: (Mutable Type)

1. A set is an unordered collection of items. Every element is unique (no duplicates) and
must be immutable (which cannot be changed).
2. Its definition starts with enclosing braces { } having its items separated by commas
inside.
3. However, the set itself is mutable. We can add or remove items from it.
4. Sets can be used to perform mathematical set operations like union, intersection,
symmetric difference etc.

Why Need A Set?

The set type has a significant advantage over a list. It implements a highly optimized method
that checks whether the container hosts a specific element or not. The mechanism used
here is based on a data structure known as a hash table.

Creating A Set:

To create a set, call the built-in set() function with a sequence or any iterable object.

>>> s = set("Python data types")

>>> type(s)  <class 'set'>
>>> s  {'e', 'y', 't', 'o', ' ', 'd', 's', 'P', 'p', 'n', 'h', 'a'}

Another simpler way is to specify the elements enclosed in curly braces {}.

>>> another_set = {'red', 'green', 'black'}

>>> type(another_set)  <class 'set'>
>>> another_set  {'red', 'green', 'black'}

Note: Creating an empty set is a bit tricky. Empty curly braces { } will make an empty
dictionary in Python. To make a set without any elements we use the set() function without
any argument.
>>> a = { }
>>> type(a)  <class 'dict'>
>>> s = set()
>>> type(s)  <class 'set'>

Add / Update a set in Python:

Sets are mutable. But since they are unordered, indexing have no meaning.
We cannot access or change an element of set using indexing or slicing. Set does not
support it.
We can add single element using the add() method and multiple elements using the update()
method.
The update() method can take tuples, lists, strings or other sets as its argument. In all cases,
duplicates are avoided.

>>> s = {1, 2, 'dreamwin'} # creating a set

>>> s  {1, 2, 'dreamwin'}
>>> s[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'set' object does not support indexing

# using add we can add only one element at a time.

>>> s.add(5)
>>> s.add('python')
>>> s  {1, 2, 'dreamwin', 5, 'python'}

# add multiple elements using update.

>>> s.update([7, 8, 9])
>>> s  {1, 2, 5, 7, 8, 9, 'dreamwin', 'python'}
>>> s.update([100, 20], ('one', 'two'), {30, 40})
>>> s  {1, 2, 100, 5, 7, 8, 9, 40, 'dreamwin', 'one', 20, 'two', 'python',
30}

Remove elements from set:

A particular item can be removed from set using methods, discard() and remove().
The only difference between the two is that, while using discard() if the item does not exist in
the set, it remains unchanged. But remove() will raise an error in such condition.

>>> my_set = set('dreamwin tech')

>>> my_set  {'d', 'm', 'a', 'c', 'h', 't', 'w', 'e', ' ', 'i', 'r', 'n'}
>>> my_set.discard('h')
>>> my_set  {'d', 'm', 'a', 'c', 't', 'w', 'e', ' ', 'i', 'r', 'n'}
>>> my_set.remove('i')
>>> my_set  {'d', 'm', 'a', 'c', 't', 'w', 'e', ' ', 'r', 'n'}
>>> my_set.discard(10)  None
>>> my_set.remove('p')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'p'

Similarly, we can remove and return an item using the pop() method.
Set being unordered, there is no way of determining which item will be popped. It is
completely arbitrary.
We can also remove all items from a set using clear().

>>> my_set = set('dreamwin tech')

>>> my_set.pop()  'd'
>>> my_set.pop()  'm'
# clears elements from set
>>> my_set.clear()
>>> my_set  set() # empty set

List all set operations using dir(set).

['__and__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__',

'__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__iand__', '__init__',
'__ior__', '__isub__', '__iter__', '__ixor__', '__le__', '__len__', '__lt__', '__ne__', '__new__',
'__or__', '__rand__', '__reduce__', '__reduce_ex__', '__repr__', '__ror__', '__rsub__',
'__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__xor__',
'add', 'clear', 'copy', 'difference', 'difference_update', 'discard', 'intersection',
'intersection_update', 'isdisjoint', 'issubset', 'issuperset', 'pop', 'remove',
'symmetric_difference', 'symmetric_difference_update', 'union', 'update']

Python Set Operations:

Sets can be used to carry out mathematical set operations like union, intersection, difference
and symmetric difference. We can do this with operators or methods.

Let us consider the following two sets for the following operations.

>>> A = {1, 2, 3, 4, 5}
>>> B = {4, 5, 6, 7, 8}

Union: Union of A and B is a set of all elements from both sets.Union is performed using |
operator. Same can be accomplished using the method union().

# use union on set A

>>> A.union(B)
{1, 2, 3, 4, 5, 6, 7, 8}
>>> A | B
{1, 2, 3, 4, 5, 6, 7, 8}
# use union function on B
>>> B.union(A)
{1, 2, 3, 4, 5, 6, 7, 8}

Intersection:
Intersection of A and B is a set of elements that are common in both sets. Intersection is
performed using & operator. Same can be accomplished using the method intersection().

# use intersection function on A

>>> A.intersection(B)
{4, 5}
>>> A & B
{4, 5}
# use intersection function on B
>>> B.intersection(A)
{4, 5}

Difference:

Difference of A and B (A - B) is a set of elements that are only in A but not in B. Similarly, B -
A is a set of element in B but not in A.
Difference is performed using - operator. Same can be accomplished using the method
difference().

# use difference function on A

>>> A.difference(B)
{1, 2, 3}
>>> A – B
{1, 2, 3}
# use - operator on B
>>> B – A
{8, 6, 7}
# use difference function on B
>>> B.difference(A)
{8, 6, 7}

Symmetric Difference:

Symmetric Difference of A and B is a set of elements in both A and B except those that are
common in both.
Symmetric difference is performed using ^ operator. Same can be accomplished using the
method symmetric_difference().

# use symmetric_difference function on A

>>> A.symmetric_difference(B)
{1, 2, 3, 6, 7, 8}
>>> A ^ B
{1, 2, 3, 6, 7, 8}
# use symmetric_difference function on B
>>> B.symmetric_difference(A)
{1, 2, 3, 6, 7, 8}

Difference_update:
The difference_update() updates the set calling difference_update() method with the
difference of sets.
Symmetric_difference_update:
The symmetric_difference_update() method updates the set calling the
symmetric_difference_update() with the symmetric difference of sets.
isdisjoint(): Return True if two sets have a null intersection
issubset(): Return True if another set contains this set
issuperset(): Return True if this set contains another set

FrozenSet: (Immutable)
Frozenset is a new class that has the characteristics of a set, but its elements cannot be
changed once assigned. While tuples are immutable lists, frozensets are immutable sets.
Sets being mutable are unhashable, so they can't be used as dictionary keys. On the other
hand, frozensets are hashable and can be used as keys to a dictionary.
Frozensets can be created using the function frozenset().
This datatype supports methods like copy(), difference(), intersection(), isdisjoint(),
issubset(), issuperset(), symmetric_difference() and union(). Being immutable it does not
have method that add or remove elements.

Interview Questions:
Datatypes in Python?
1. Numbers (int, float, complex)
2. String
Data structures in python?
1. List
2. Tuple
3. Dictionary
4. Set
5. Frozenset

Mutable vs Immutable?
A mutable object can change its state or contents during run-time and immutable objects
cannot.

Mutable objects: list, dict, set, byte array

Immutable objects: int, float, complex, string, tuple, frozen set [note: immutable version of
set], bytes

1. Python handles mutable and immutable objects differently.

2. Immutable are quicker to access than mutable objects.
3. Mutable objects are great to use when you need to change the size of the object,
example list, dict etc.. Immutables are used when you need to ensure that the object
you made will always stay the same.
4. Immutable objects are fundamentally expensive to “change”, because doing so
involves creating a copy. Changing mutable objects is cheap.

List vs Tuple?
List is a collection of heterogeneous elements and enclosed using square brackets []. It’s a
mutable type.
Tuple is a collection of heterogeneous elements and enclosed using parentheses (). It’s an
immutable type.
When do you use list vs tuple?

List and Tuple are both ordered containers.

Tuple: If you want an ordered container of constant elements use tuple as tuples are
immutable objects.
List: If you want an ordered container of arbitrary data or elements use lists. Lists are
mutable objects.

The main difference between list and tuple is you can change the list but you cannot change
the tuple. Tuple can be used as keys in mapping where list is not.

What is used to represent Strings in Python? Is double quotes used for String representation
or single quotes used for String representation in Python?

Using Single Quotes (‘ ’)

You can specify strings using single quotes such as ‘Hello welcome to python’. All white
space i.e. spaces and tabs are preserved as-is.

Using Double Quotes (“ ”)

Strings in double quotes work exactly the same way as strings in single quotes.
Eg:
S = “What’s your name”

Using Triple Quotes ()

You can specify multi-line strings using triple quotes. You can use single quotes and double
quotes freely within the triple quotes. An example is
Triple single quotes are used to define multi-line strings or multi-line comments.

’’’This is a multi-line string.

This is the first line.
This is the second line.’’’

Triple double quotes are used to define doc strings.

What is slicing in Python?

Slicing in Python is a mechanism to select a range of items from Sequence types like
strings, list, tuple, etc.
Example of slicing:

>>> l=[1,2,3,4,5]
>>> l[1:3]
[2, 3]
>>> l[1:-2]
[2, 3]
>>> l[-3:-1] # negative indexes in slicing
[3, 4]

>>> s="Hello World"

>>> s[1:3]
'el'
>>> s[:-5]
'Hello '
>>> s[-5:]
'World'

What is a negative index in python?

Python list items can be accessed with positive or negative numbers (also known as index).
For instance our list is of size n, then for positive index 0 is the first index, 1 second, last
index will be n-1. For negative index, -n is the first index, -(n-1) second ... A negative index
accesses elements from the end of the list counting backwards.
An example to show negative index in python.

>>> a= [1, 2, 3]
>>> print a[-3]
1
>>> print a[-2]
2
>>> print a[-1]
3

What is the difference Between List And Tuple?

List Tuple
List objects are mutable objects Tuple objects are immutable Objects
Applying iterations on list objects takes Applying iterations on tuple Objects
longer time takes less time
If the frequent operation is insertion or If the frequent operation is retrieval of
deletion of the elements then it is the elements then it is recommended
recommended to use list to use tuple
Tuple can be used as a key for the
List can’t be used as a ‘key’ for the
dictionary if the tuple is storing only
dictionary
immutable elements

What is the Dictionary?

Dictionary objects can be created by using curly braces {} or by calling dictionary function
Dictionary objects are mutable objects
Dictionary represents key value base
Each key value pair of Dictionary is known as a item
Dictionary keys must be immutable
Dictionary values can be mutable or immutable
Duplicate keys are not allowed but values can be duplicate
Insertion order is not preserved
Heterogeneous keys and heterogeneous values are allowed.

d = {}
d[‘name’] = ‘Python’
d[‘year’] = 1990

What does this code output?

def f(x,l=[]):
for i in range(x):
 l.append(i*i)
print(l)
f(2)
f(3,[3,2,1])
f(3)
Answer
[0, 1]
[3, 2, 1, 0, 1, 4]
[0, 1, 0, 1, 4]
Explain set?
Set is an unordered collection of unique and immutable or hash able elements. It’s a mutable
type.
S = {}
s.add(10)  {10}

set is used to remove duplicates from an iterable. One can perform mathematical set
operations like union, intersection, difference..
Explain list methods append vs extend?
Append and extend are list methods, which are used to add elements at the end of the list.
Append: using append we can add any elements at the end of the list.
l=[]
l.append(10)  [10]
l.append(‘hello’)  [10, ‘hello’]
l.append([1, 2, 3])  [10, ‘hello’, [1, 2, 3]]
Extend: using extend we can add only iterables into a list. i.e except integers we can add
any type.
l.append(‘hell’)  [‘h’, ‘e’, ‘l’, ‘l’, ‘o’]
l.append([1,2,3])  [1, 2, 3]
Difference between sort and sorted?
Sort is a list method, which will does the in-place sorting. It means it changes the original list
object.
A = [5, 2, 4, 6, 1, 3]
a.sort()
print(a)  [1, 2, 3, 4, 5, 6]

Sorted is a built-in function, upon sort it creates new object. It will not modify the original list
object.

sorted(a)  [1, 2, 3, 4, 5, 6]
print(a)  [5, 2, 4, 6, 1, 3]
Explain split() and join() function.
Python’s split() function helps to split a string into list of substrings.
join() function does exactly the opposite. Given a list of values you can make a string out of it
based on the given delimiter.
a = ‘hello welcome to python’
a.split()  [‘hello’, ‘welcome’, ‘to’, ‘python’]
‘ ’.join(a)  ‘hello welcome to python’
What is the difference between Xrange and Range?

range() and xrange() are two functions that could be used to iterate a certain number of times
in for loops in Python. In Python 3, there is no xrange , but the range function behaves like
xrange If you want to write code that will run on both Python 2 and Python 3, you should use
range().
range() – This returns a list of numbers created using range() function.
xrange() – This function returns the generator object that can be used to display numbers
only by looping. Only particular range is displayed on demand and hence called “lazy
evaluation“.
Both are implemented in different ways and have different characteristics associated with
them. The points of comparisons are:
 Return Type
 Memory
 Operation Usage
 Speed

range() returns – the list as return type.

xrange() returns – xrange() object.

Difference Between set and dictionary:

Dictionary is an unordered collection of key-value pairs and its enclosed using flower or curly
braces {}. It’s a mutable type.
d = {1: ‘one’, 2: ‘two’}
Set is an unordered collection of unique and immutable (hashable) elements, its enclosed
using curly braces {} same like dictionary. It’s a mutable type.
s = {1, 2, ‘hello’, (3, 4, 5)}

What is the usage of help() and dir() function in Python?

Help() and dir() both functions are accessible from the Python interpreter and used for viewing
a consolidated dump of built-in functions.

Help() function: The help() function is used to display the documentation string and also
facilitates you to see the help related to modules, keywords, attributes, etc.

Dir() function: The dir() function is used to display the methods and attributes of an object.

What Are Different Methods To Copy An Object In Python?

There are two ways to copy objects in Python.

 copy.copy() function
 It makes a copy of the file from source to destination.
 It’ll return a shallow copy of the parameter.
 copy.deepcopy() function
 It also produces the copy of an object from the source to destination.
 It’ll return a deep copy of the parameter that you can pass to the function.

Difference between shallow copy vs deepcopy?

Python defines a module which allows to deep copy or shallow copy mutable object
using the inbuilt functions present in the module “copy“.
Assignment statements in Python do not copy objects, they create bindings between a
target and an object. For collections that are mutable or contain mutable items, a copy is
sometimes needed so one can change one copy without changing the other.

In case of deep copy, a copy of object is copied in other object. It means that any
changes made to a copy of object do not reflect in the original object.
In python, this is implemented using “deepcopy()” function.

In case of shallow copy, a reference of object is copied in other object. It means that any
changes made to a copy of object do reflect in the original object.
In python, this is implemented using “copy()” function.
Decision Making or Conditional Statements:
Decision making is one of the most important concepts of computer programming. It require
that the developer specify one or more conditions to be evaluated or tested by the program,
along with a statement or statements to be executed if the condition is determined to be true,
and optionally, other statements to be executed if the condition is determined to be false.
Python programming language assumes any non-zero and non-null values as TRUE, and if
it is either zero or null, then it is assumed as FALSE value.
Python uses indentation to separate expressions and statements.

In Python:

 True and non-empty things are true. Number 1 is true.

 False, None, zero, and empty things are false.

Following evaluates to false:

 False → A builtin literal expression.

 None → A builtin literal expression.
 0 → Zero.
 0.0 → Zero, float.
 " " → Empty string.
 [ ] → Empty list.
 ( ) → Empty tuple.
 { } → Empty dictionary.
 set([]) → Empty set.
 frozenset([]) → Empty frozen set.

Python programming language provides following types of decision making statements.

1. if statements
2. if....else statements
3. if..elif..else statements
4. nested if statements

Python if statements:

Syntax:

if <condition>:
Body of if

The Python if statement is a statement which is used to test

specified condition. We can use if statement to perform
conditional operations in our Python application.

The if statement executes only when specified condition is true.

We can pass any valid expression into the if parentheses

x = 10
if x == 10:
print("Welcome to Dreamwin")
o/p: Welcome to Dreamwin

Python if..else statements:

Syntax:
if <condition>:
Body of if
else:
Body of else

The if..else statement evaluates test expression and will execute body of if only when test
condition is True. If the condition is False, body of else is executed. Indentation is used to
separate the blocks.
x = 3
if x < 10:
print("x smaller than 10")
else:
print("x is bigger than 10 or equal")

o/p: x smaller than 10

Python if..elif..else statements.

Syntax:
if <condition>:
Body of if
elif <condition>:
Body of elif
else:
Body of else

The elif is short for else if. It allows us to check for multiple expressions.
If the condition for if is False, it checks the condition of the next elif block and so on.

If all the conditions are False, body of else is executed. Only one block among the
several if...elif...else blocks is executed according to the condition.
The if block can have only one else block. But it can have multiple elif blocks.

x = 30
y = 20

if x < y:
print("x smaller than y")

elif x == y:
print("x equals to y")

else:
print("x is bigger than y")

o/p: x is bigger than y

Python Nested if statements
We can have a if...elif...else statement inside another if...elif...else statement. This is called
nesting in computer programming.
Any number of these statements can be nested inside one another. Indentation is the only
way to figure out the level of nesting. This can get confusing, so must be avoided if we can

num = 10
if num >= 0:
if num == 0:
print("Zero")
else:
print("Positive number")
else:
print("Negative number")
o/p: Positive number

Python Loop Statements:

Loops are one of the most important features in programming. Loops offer a quick and easy
way to do something repeatedly. It can execute a block of code a number of times.

The for loop in Python is used to iterate over a sequence (list, tuple, string, dictionary, set,
frozenset and files) or other iterable objects. Iterating over a sequence is called traversal.

Syntax of for Loop:

for val in sequence or iterable:

Body of for

Here, val is the variable that takes the value of the item inside the sequence on each
iteration.
Loop continues until we reach the last item in the sequence. It iterates based on the length
of the sequence or iterable. The body of for loop is separated from the rest of the code using
indentation of for loop.

Iterating on list:

days = ['Monday','Tuesday','Wednesday','Thrusday','Friday']
for day in days:
print(day)

o/p:
Monday
Tuesday
Wednesday
Thrusday
Friday
Iterating on string:

day = “Monday”
for d in day:
print(d)
o/p:
M
o
n
d
a
y

Iterating on tuple:

t = ("Python", "for", "beginners")

for i in t:
print(i)
o/p:
Python
for
beginners

Iterating on dictionary:

d = {1: 'one', 2: 'two', 10: 'ten'}

for i in d:
print(i)

o/p:
1
2
10

Iterating on set:

s = {'one', 'two', 'ten'}

for i in s:
print(i)

o/p:
ten
one
two
Loop Control Statements:

These statements are used to change execution from its normal sequence.

Control Statements Description

Break statement It is used to exit a while loop or a for loop. It terminates the looping &
transfers execution to the statement next to the loop.

Continue statement It causes the looping to skip the rest part of its body & start re-testing its
condition.

Pass statement It is used in Python to when a statement is required syntactically and the
programmer does not want to execute any code block or command.

Break and Continue in Python:

In Python, break and continue statements can alter the flow of a normal loop.
Loops iterate over a block of code until test expression is false, but sometimes we wish to
terminate the current iteration or even the whole loop without checking test expression.

The break and continue statements are used in these cases.

Python break statement:

The break statement terminates the loop containing it. Control of the program flows to the
statement immediately after the body of the loop.
If break statement is inside a nested loop (loop inside another loop), break will terminate the
innermost loop.

for val in "Dreamwin":

if val == 'm':
break
else:
print(val)
print("outside for loop")

o/p:
D
r
e
a
outside for loop

Python continue statement:

The continue statement is used to skip the rest of the code inside a loop for the current
iteration only. Loop does not terminate but continues on with the next iteration.
for val in "Dreamwin":
if val == "i":
continue
print(val)
print("The end")
o/p:
D
r
e
a
m
w
n
The end

Pass statement Python:

In Python programming, pass is a null statement. The difference between a comment and
pass statement in Python is that, while the interpreter ignores a comment entirely, pass is
not ignored. It is used as a placeholder for future implementation of functions, loops, etc.

However, nothing happens when pass is executed. It results into no operation (NOP).
Suppose we have a loop or a function that is not implemented yet, but we want to implement
it in the future. They cannot have an empty body. The interpreter would complain. So, we
use the pass statement to construct a body that does nothing.

for i in range(10):
pass

def hello():
pass

For-else in python:

Well, this might seem to be very unusual, but we do have an optional else block in for
statement in Python. This else block gets executed only when break statement is not
executed. If there is no break statement in the code, else block will always execute.

s = "Dreamwin"
for i in s:
print(i)
else:
print('Iterated over everything.')

o/p:
D
r
e
a
m
w
i
n
Iterated over everything.

For-else with break statement:

s = "Dreamwin"
for i in s:
if i == 'm':
break
print(i)
else:
print('Iterated over everything.')

o/p:
D
r
e
a

While Loop Python:

The while loop in Python is used to iterate over a block of code as long as the test
expression (condition) is true.
We generally use this loop when we don't know beforehand, the number of times to iterate

Syntax:

while <condition>:
Body of while

In while loop, test expression is checked first. The body of the loop is entered only if the
condition evaluates to True. After one iteration, the test expression is checked again. This
process continues until the condition evaluates to False.

In Python, the body of the while loop is determined through indentation. Body starts with
indentation and the first un-indented line marks the end.
Python interprets any non-zero value as True. None and 0 are interpreted as False.

count = 0
while count < 5:
print(count)
count += 1
o/p:
0 Note: while will keep printing until count is less than 5.
1
2
3
4

Else clause on Python while statement:

The else clause is only executed when your while condition becomes false. If you break out
of the loop, or if an exception is raised, it won't be executed.
while <condition>:
Body of while
else:
Body of else

Example:

count = 0
while count < 5:
print(count)
count += 1
else:
print('Inside else block of while loop.')
o/p:
0
1
2
3
4
Inside else block of while loop.

Example:

x = 5
while x < =10:
print(x)
x = x + 1
if x==7:
break
else:
print("Inside Else")
o/p:
5
6

Python Infinite Loop:

We can program an infinite loop using while statement. If the condition of while loop is
always True, we get an infinite loop.

# Press Ctrl + c to exit from loop

while True:
print ("This is an infinite Loop")
Python Functions
Functions are an essential part of the Python programming language: you might have
already encountered and used some of the many fantastic functions that are built-in in the
Python language or that come with its library ecosystem.

In Python, function is a group of related statements that perform a specific task. Functions
help break our program into smaller and modular chunks. As our program grows larger and
larger, functions make it more organized and manageable. Furthermore, it avoids repetition
and makes code reusable.

There are three types of functions in Python:

Built-in functions: Functions that readily come with Python are called built-in functions.
Function such as help() to ask for help, min() to get the minimum value, print() to print an
object to the terminal,…
User-Defined Functions (UDFs), Functions that we define ourselves to do certain specific
task are referred as user-defined functions.
Anonymous functions, which are also called lambda functions because they are not
declared with the standard def keyword.

User-Defined Functions (UDFs):

 Keyword def marks the start of function header.
 A function name to uniquely identify it. Function naming follows the same rules of
writing identifiers in Python.
 Parameters (arguments) through which we pass values to a function. They are
optional.
 A colon (:) to mark the end of function header.
 Optional documentation string (docstring) to describe what the function does.
 One or more valid python statements that make up the function body. Statements
must have same indentation level (usually 4 spaces).
 An optional return statement to return a value from the function.

User defined function:

#function definition without any arguments

def hello():
print("Hello World")
return

#Calling python function

hello()

The return Statement:

Note that as you’re printing something in your UDF hello(), you don’t really need to
return it. There won’t be any difference between the function above and this one:

def hello(): # function definition without any arguments

print("Hello World")
hello() # function calling

However, if you want to continue to work with the result of your function and try out
some operations on it, you will need to use the return statement to actually return a
value, such as a String, an integer, list etc.

The return statement is used to exit a function and go back to the place from where it
was called.

def hello():
print("Hello World")
return("hello")
def hello_noreturn():
print("Hello World")

# hello function result has to be assign to a variable to make use of it in our

code.
a = hello()
print(a)

o/p:
Hello World
hello

hello_noreturn()

o/p:
Hello World

Note: functions immediately exit when they come across a return statement, even if it means
that they won’t return any value

Return multiple values:

Another thing that is worth mentioning when you’re working with the return statement is
the fact that you can use it to return multiple values. Default return type is tuple. To do
this,

def addition(a,b):
c = a + b
return a, b, c
val = addition(3,4)
print(val) # (3, 4, 7)

Function Arguments in Python:

Earlier, you learned about the difference between parameters and arguments. In short,
arguments are the things which are given to any function or method call, while the
function or method code refers to the arguments by their parameter names.

There are four types of arguments that Python UDFs can take:
1. Default arguments
 2. Required arguments (or) positional arguments
3. Variable number of arguments
4. Keyword arguments

Default Arguments:

Default arguments are those that take a default value if no argument value is passed
during the function call. You can assign this default value by with the assignment
operator =, just like in the following example:

# Define `hello()` function

def hello(a, b = 2):
return a + b
# Call `hello()` with only `a` parameter
hello(a=1) o/p: 3
# Call `hello()` with `a` and `b` parameters
hello(a=1, b=3) o/p: 4

Required or Positional Arguments:

These arguments need to be passed during the function call and in exactly the right
order, just like in the following example:

# Define `hello()` with required arguments

def hello(a, b):

return a + b

hello(4,5) # o/p: 9
hello(4) # It will throw an error.

You need arguments that map to the a as well as the b parameters to call the function
without getting any errors.

Variable Number of Arguments:

It is denoted using * (single star <var name>).

In cases where you don’t know the exact number of arguments that you want to pass to
a function, you can use the following syntax with *args:
The return type of *args is tuple.

# Define `hello()` function to accept a variable number of arguments

def hello(*args):
return args

hello(1,4,5) # o/p: (1,4,5)

hello() # o/p: ()

The asterisk (*) is placed before the variable name that holds the values of all non-keyword
variable arguments.
Note: try replacing *args with another name that includes the asterisk. You’ll see that the
above code keeps working!

Keyword Arguments:

It is denoted using **kwargs (double star <var name>).

In case where you don’t know the exact number of keyword arguments that you want to
pass to a function, you can use the following syntax with **kwargs:
The return type of **kwargs is dictionary.

# Define `hello()` function to accept a variable number of arguments

def hello(**kwargs):
return kwargs

# Calling function
hello(name=”python”, year=1990) # o/p: {‘name’: ’python’, ‘year’: 1990}
hello() # o/p: {}

The asterisk (**) is placed before the variable name that holds the values of all keyword
arguments.

Note: try replacing **kwargs with another name that includes the double asterisk. You’ll see
that the above code keeps working!

def hello(x, y=20, *args, **kwargs): # function definition

print(x, y)
print(“args: ”,args)
print(“kwargs: ”, kwargs)

# function calling
hello(10, 50, [1,2,3], "hello", 10.5, {1:2}, name="xyz",val=20)
o/p:
(10, 50)
args:([1, 2, 3], 'hello', 10.5, {1: 2})
kwargs: {'name': 'xyz', 'val': 20}

Anonymous Functions:

Python supports the creation of anonymous functions (i.e. functions that are not bound to a
name) at runtime, using a construct called "lambda".
lambda operator or lambda function is used for creating small, one-time and anonymous
function objects in Python.
This is not exactly the same as lambda in functional programming languages, but it is a very
powerful concept that's well integrated into Python and is often used in conjunction with
typical functional concepts like filter(), map() and reduce().

Like def, the lambda creates a function to be called later. But it returns the function instead
of assigning it to a name. This is why lambdas are known as anonymous functions. In
practice, they are used as a way to inline a function definition, or to defer execution of a
code.
Lambda’s Syntax: lambda arguments : expression

User-defined function:

def func(x):
return x ** 3
print(func(5)) # 125

Using Lambda:

lamb = lambda x: x ** 3
print(lamb(5)) # 125

val = lambda x, y : x + y
val(5, 10) # 15

In the above lambda function, x and y are arguments and x + y is an expression.

lambda is an expression, not a statement:

Because of this, a lambda can appear in places where a def is not allowed. For example,
places like inside a list literal, or a function call's arguments.
As an expression, lambda returns a value that can optionally be assigned a name. In
contrast, the def statement always assigns the new function to the name in the header,
instead of returning is as a result.

lambda's body is a single expression, not a block of statements:

The lambda's body is similar to what we'd put in a def body's return statement. We simply
type the result as an expression instead of explicitly returning it. Because it is limited to an
expression, a lambda is less general that a def. lambda is designed for coding simple
functions, and def handles larger tasks.

f = lambda x, y, z: x + y + z
f(2, 30, 400) # o/p: 432

a = (lambda a = 'hello', b = ' Welcome to', c = ' python': a + b + c)

a('Hello', ' Java') # 'Hello Welcome to Java'

Docstrings:

The first string after the function header is called the docstring and is short for
documentation string. It is used to explain in brief, what a function does.
Although optional, documentation is a good programming practice. Its best practice to
document every action that we do in code.

In the below example, we have a docstring immediately below the function header. We
generally use triple quotes so that docstring can extend up to multiple lines. This string is
available to us as __doc__ attribute of the function.

def hello(name):
"""This function greets to
the person passed in as
parameter"""
 print("Hello, " + name + ". Good morning!")

hello("Virat")
o/p: ("Hello, Virat. Good morning!")

Variable scopes in Python:

Namespaces help us uniquely identify all the names inside a program. However, this doesn't
imply that we can use a variable name anywhere we want. A name also has a scope that
defines the parts of the program where you could use that name without using any prefix.
Just like namespaces, there are also multiple scopes in a program. Here is a list of some
scopes that can exist during the execution of a program.

 A local scope, which is the innermost scope that contains a list of local names
available in the current function.
 A scope of all the enclosing functions. The search for a name starts from the nearest
enclosing scope and moves outwards.
 A module level scope that contains all the global names from the current module.
 The outermost scope that contains a list of all the built-in names. This scope is
searched last to find the name that you referenced.

Scope of a variable is the portion of a program where the variable is recognized. Parameters
and variables defined inside a function is not visible from outside. Hence, they have a local
scope.
Lifetime of a variable is the period throughout which the variable exits in the memory. The
lifetime of variables inside a function is as long as the function executes.
They are destroyed once we return from the function. Hence, a function does not remember
the value of a variable from its previous calls.

Two types of variable python has.

1. Local variable (local scope)
2. Global variable (global scope)

Global variables are the one that are defined and declared outside a function and we can to
use them inside a function.

# This function uses global variable “s”

def f():
print s

# Global scope
s = "Dreamwin Technologies"
f()

O/p: Dreamwin Technologies

If a variable with same name is defined inside the scope of function as well then it will print
the value given inside the function only and not the global value.
# This function has a variable with name same as s.

def f():
s = "Me too."
Print(s)
# Global scope
s = "I love python."
f()
print(s)

Output:
Me too.
I love python.

The variable s is defined as the string “I love python.”, before we call the function f(). The
only statement in f() is the “print(s)” statement. As there is no local s, the value from the
global s will be used.

The question is, what will happen, if we change the value of s inside of the function f()? Will it
affect the global s as well? We test it in the following piece of code:

def f():
print(s)
# this program will NOT show error if we comment below line
s = "Me too."
print(s)
# Global scope

s = "I love Python."

f()
print(s)
Output: UnboundLocalError: local variable 's' referenced before assignment.

To make the above program work, we need to use “global” keyword. We only need to use
global keyword in a function if we want to do assignments / change them. global is not
needed for printing and accessing.
Why? Python “assumes” that we want a local variable due to the assignment to s inside of
f(), so the first print statement throws this error message. Any variable which is changed or
created inside of a function is local, if it hasn’t been declared as a global variable.
To tell Python, that we want to use the global variable, we have to use the keyword “global”,
as can be seen in the following example:

# This function modifies global variable 's'

def f():
global s
print(s)
s = "Look for Python programming."
print(s)
# Global Scope
s = "Python is great!"
f()
print(s)

Output:
Python is great!
Look for Python programming.
Look for Python programming.

Example:

a = 1
# Uses global because there is no local 'a'
def f():
print('Inside f() : ', a_

# Variable 'a' is redefined as a local

def g():
a = 2
print('Inside g() : ',a)

# Uses global keyword to modify global 'a'

def h():
global a
a = 3
print('Inside h() : ',a)

# Global scope
Print('global : ',a)
f()
print('global : ',a)
g()
Print('global : ',a)
h()
print('global : ',a)

Output:
global : 1
Inside f() : 1
global : 1
Inside g() : 2
global : 1
Inside h() : 3
global : 3

Function Utilities:
Python has a very good function utilities to apply a function of any iterable.
map, filter and reduce are the function utilities in python.

Map:

map(function_object, iterable1, iterable2,...)

map functions expects a function object and any number of iterables like list, dictionary, etc.
It executes the function_object for each element in the sequence and returns a list of the
elements modified by the function object.

In Python3, map function returns an iterator or map object which gets lazily evaluated. It
means it returns an object instead of returning list of values.

def multiply2(x):
return x * 2

map(multiply2, [1, 2, 3, 4]) # Output [2, 4, 6, 8]

In the above example, map executes multiply2 function for each element in the list i.e. 1, 2,
3, 4 and returns [2, 4, 6, 8]
Let’s see how we can write the above code using map and lambda. Just one line of code
and that’s it.

#In Python 2
map(lambda x : x*2, [1, 2, 3, 4]) #Output [2, 4, 6, 8]

# Python 3
a = map(lambda x : x*2, [1, 2, 3, 4]) # returns a map object
# use a magic method __next__ to get values from the object.
a.__next__() # 2
a.__next__() # 4
list(a) # [4, 8] it will create a list and append remaining values from the
object.

Examples:

dict_a = [{'name': 'python', 'points': 10}, {'name': 'java', 'points': 8}]

list(map(lambda x : x['name'], dict_a)) # Output: ['python', 'java']

list(map(lambda x : x['points']*10, dict_a)) # Output: [100, 80]

list(map(lambda x : x['name'] == "python", dict_a)) # Output: [True, False]

# also we can use map on more that one iterable

list(map(lambda x, y: x + y, [3, 4, 5], [6, 7, 8])) # [9, 11, 13]

Filter:

In simple words, the filter() method filters the given iterable with the help of a function that
tests each element in the iterable to be true or not.

syntax:
filter(function, iterable)

The filter() function in Python takes in a function and a iterable as arguments. The function is
called with all the items in the list and a new list is returned which contains items for which
the function evaluats to True.
In Python3, filter function returns an iterator or filter object which gets lazily evaluated. It
means it returns an object instead of returning list of values.

f = [0,1,1,2,3,5,8,13,21,34,55]
odd_numbers = list(filter(lambda x: x % 2, f)) # [1, 1, 3, 5, 13, 21, 55]
even_numbers = list(filter(lambda x: x % 2 == 0, f)) #[0, 2, 8, 34]

Recursion:

Recursion is the process of defining something in terms of itself.

We know that in Python, a function can call other functions. It is even possible for the
function to call itself. These type of construct are termed as recursive functions.

Following is an example of recursive function to find the factorial of an integer.

Factorial of a number is the product of all the integers from 1 to that number. For example,
the factorial of 6 (denoted as 6!) is 1*2*3*4*5*6 = 720.

def calc_factorial(x):
"""This is a recursive function
to find the factorial of an integer"""
if x == 1:
return 1
else:
return (x * calc_factorial(x-1))
num = 4
print("The factorial of", num, "is", calc_factorial(num))

In the above example, calc_factorial() is a recursive functions as it calls itself.

When we call this function with a positive integer, it will recursively call itself by decreasing
the number.
Each function call multiples the number with the factorial of number 1 until the number is
equal to one. This recursive call can be explained in the following steps.

calc_factorial(4) # 1st call with 4

4 * calc_factorial(3) # 2nd call with 3
4 * 3 * calc_factorial(2) # 3rd call with 2
4 * 3 * 2 * calc_factorial(1) # 4th call with 1
4 * 3 * 2 * 1 # return from 4th call as number=1
4 * 3 * 2 # return from 3rd call
4 * 6 # return from 2nd call
24 # return from 1st call

Our recursion ends when the number reduces to 1. This is called the base condition.
Every recursive function must have a base condition that stops the recursion or else the
function calls itself infinitely.

Advantages of Recursion:

1. Recursive functions make the code look clean and elegant.

2. A complex task can be broken down into simpler sub-problems using recursion.
3. Sequence generation is easier with recursion than using some nested iteration.
Disadvantages of Recursion:

1. Sometimes the logic behind recursion is hard to follow through.

2. Recursive calls are expensive (inefficient) as they take up a lot of memory and time.
3. Recursive functions are hard to debug.

Closures:

Actually, a closure is a function in Python which allows you to do things that aren’t possible
in another language (like C/C++ or Java) with functions.

In Python everything is treated like a first-class citizen, so they can be passed around just as
normal variables. So are functions too. You probably have seen code like this already:
>>> def adder(a, b):
... return a + b

>>> def caller(fun):

... print(fun(2, 4))
... print(fun(3, 5))
...
>>> caller(adder)
6
8

In this example above we created a function caller which calls another function which is
passed to it as an argument.

Because we can pass around functions as arguments we can return functions too from
function calls. Let's consider the following example:
>>> def contains_factory(x):
... def contains(lst):
... return x in lst
... return contains
...
>>> contains_15 = contains_factory(15)

After running this example, contains_15 has the type of a function:

<function contains_factory.<locals>.contains at 0x101d78b70>

And these functions are closures.

What makes closures special?

If you think that closures are ordinary functions you might have missed the key difference
from the previous example.

Call the contains_15 with some lists / iterables to test the functionality:
>>> contains_15([1,2,3,4,5])
False
>>> contains_15([13, 14, 15, 16, 17])
True
>>> contains_15(range(1, 20))
True
The key point of closures are that they remember their context in which they were
created. In the example above contains_15 remembers, that the contains_factory function
was called with the value 15. And this 15 is used for later in the contains function as the
variable x when you provide multiple iterables to look-up x in them.

Another interesting fact is, that the closure remains existing even if the original creator
function (in the example case it is contains_factory) is deleted:
>>> del contains_factory
>>> contains_factory(42)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'contains_factory' is not defined
>>> contains_15(range(14, 20))
True

How to create a closure?

After this introduction on closures you should know how to create closures in Python. But for
the sake of brevity let's sum things up:
 We have to create a nested function (a function inside another function).
 This nested function has to refer to a variable defined inside the enclosing function.
 The enclosing function has to return the nested function

Pretty simple, isn't it?

The second point is simple in reality, optional -- but if you do not reference a variable from
the enclosing function there is not much sense to create a nested function and return it --
you simply define a function. You could do this in the normal scope too.

Let's create another interesting closure: a counter. The idea behind the counter is that in
some cases you just want to count interactions. In those cases you define a global variable
(most of the time called counter) and increment it at the right place when an interaction
occurs. We can replace this global variable and the incrementation by defining a closure and
call this closure every time we want to count something.
>>> def counter_factory():
... count = 0 # here I create the variable to increment
... def counter():
... nonlocal count
... count += 1
... return count
... return counter
...

The example above creates a function which will generate a counter closure every time it is
invoked -- and the counter starts from 0 and increases every time you call the closure.
>>> counter1 = counter_factory()
>>> counter1()
1
>>> counter1()
2
>>> counter2 = counter_factory()
>>> counter2()
1
>>> counter1()
3
>>> counter2()
2

Well, this solution is not quite what we want because we return the value of count every time
with the function invocation. This means that if we want to verify that there were no
interactions then we have to do something like this:
counter1() == 1

And this requires thinking and remembering. And we are humans that make errors.

What are closures good for?

Even if closures seem pretty interesting (a function returning another function which knows
its creation context!) there is another question: where can we utilize closures to make the
best of them?

Here are a few uses for closures:

 Eliminating global variables
 Replacing hard-coded constants
 Providing consistent function signatures

Late binding of closures

A source of problems is the way Python binds the variables in closures. For example you
write:

>>> def create_adders():

... adders = []
... for i in range(10):
... def adder(x):
... return i + x
... adders.append(adder)
... return adders
...
>>> for adder in create_adders():
... print(adder(1))
...

You may expect the following output:

2
3
4
5
6
7
8
9
10
11

In reality you get:

10
10
10
10
10
10
10
10
10
10

What happens? You create 10 functions which add 1 to the number 9. This is because of the
late-binding of closures: the variables used inside the closures are looked up at the time the
inner function is called. Whenever the adder function is called the inner loop over the
range(10) is finished and the variable i has the value 9 -- so you end up with 10 for each 10
functions as result.

To solve this problem you could add an additional parameter to your closures which will
maintain the right state:
>>> def create_adders():
... adders = []
... for i in range(10):
... def adder(x, i=i):
... return i + x
... adders.append(adder)
... return adders

Here we added i as a local variable to the closure with a default value of i. This later i comes
from the range and sets the value of the closure i to 0 to 9 respectively. This solves the
problem if we run the example:
>>> for adder in create_adders():
... print(adder(1))
...
1
2
3
4
5
6
7
8
9
10
Exception Handling
Errors or mistakes in a program are often referred to as bugs. They are almost always the
fault of the programmer. The process of finding and eliminating errors is called debugging.
An Error is something that most of the time you cannot handle it. Errors are unchecked
exception and the developer is not required to do anything with these. Errors normally tend
to signal the end of your program, it typically cannot be recovered from and should cause
you exit from current program. It should not be caught or handled.
Errors can be classified into three major groups:

1. Syntax errors
2. Runtime errors
3. Logical errors
Syntax errors:

Python will find these kinds of errors when it tries to parse your program, and exit with an
error message without running anything. Syntax errors are mistakes in the use of the Python
language, and are analogous to spelling or grammar mistakes in a language like English.
Common Python syntax errors include:
leaving out a keyword.
putting a keyword in the wrong place.
leaving out a symbol, such as a colon, comma or brackets.
misspelling a keyword.
incorrect indentation.
empty block.

if mark >= 50 # colon missing for if statement.

print("You passed!")

if arriving:
print("Hi!")
esle: # invalid keyword esle.
print("Bye!")

if flag: # invalid indentation

print("Python is Awesome!")

Runtime errors:
If a program is syntactically correct – that is, free of syntax errors – it will be run by the
Python interpreter. However, the program may exit unexpectedly during execution if it
encounters a runtime error – a problem which was not detected when the program was
parsed, but is only revealed when a particular line is executed. When a program comes to a
halt because of a runtime error, we say that it has crashed.
Some examples of Python runtime errors:
division by zero.
performing an operation on incompatible types.
using an identifier which has not been defined.
accessing a list element, dictionary value or object attribute which doesn’t exist.
trying to access a file which doesn’t exist.

Logical errors:
Logical errors are the most difficult to fix. They occur when the program runs without
crashing, but produces an incorrect result. The error is caused by a mistake in the program’s
logic. You won’t get an error message, because no syntax or runtime error has occurred.
You will have to find the problem on your own by reviewing all the relevant parts of your
code – although some tools can flag suspicious code which looks like it could cause
unexpected behaviour.
Sometimes there can be absolutely nothing wrong with your Python implementation of an
algorithm – the algorithm itself can be incorrect. However, more frequently these kinds of
errors are caused by programmer carelessness.

Here are some examples of mistakes which lead to logical errors:

using the wrong variable name.
indenting a block to the wrong level.
using integer division instead of floating-point division.
getting operator precedence wrong.
making a mistake in a boolean expression.
off-by-one, and other numerical errors
If you misspell an identifier name, you may get a runtime error or a logical error, depending
on whether the misspelled name is defined.
All the Errors are Exceptions but the reverse is not true. In general Errors are which nobody
can control or guess when it happened, on the other hand Exception can be guessed and
can be handled.

Exception:

All the runtime errors that we have encountered are called exceptions in
Python. Python uses them to indicate that something exceptional has occurred, and
the program cannot continue unless it is handled.
An Exception is an unwanted event that interrupts the normal flow of the program.
This could be a programming error attempting to divide by zero, attempting to invoke
a method on an object that does not define the method, or passing an invalid
argument to a method. When an exception occurs program execution gets terminated.
In such cases we get a system generated error message.

Why an exception occurs:

There can be several reasons that can cause a program to throw exception. For example,
when we try to read a file or get input from a user, there is a chance that something
unexpected will happen – the file may have been moved or deleted, and the user may enter
data which is not in the right format.
All exceptions are subclasses of the Exception class. A try-except statement can be used to
wrap entire programs or just particular portions of code to trap and identify errors. If an error
occurs within the try statement, an exception is raised, and the code under the except
statement is executed. For example, it is not possible to divide by zero. If we try to do this, a
ZeroDivisionError is raised and the script is interrupted.

List all built-in exceptions in python using dir(__builtins__).

['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BufferError',

'BytesWarning', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception',
'False', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError',
'ImportWarning', 'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt',
'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented', 'NotImplementedError',
'OSError', 'OverflowError', 'PendingDeprecationWarning', 'ReferenceError', 'RuntimeError',
'RuntimeWarning', 'StandardError', 'StopIteration', 'SyntaxError', 'SyntaxWarning',
'SystemError', 'SystemExit', 'TabError', 'True', 'TypeError', 'UnboundLocalError',
'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError',
'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__debug__',
'__doc__', '__import__', '__name__', '__package__', 'abs', 'all', 'any', 'apply', 'basestring',
'bin', 'bool', 'buffer', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'cmp', 'coerce',
'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval',
'execfile', 'exit', 'file', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash',
'help', 'hex', 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals',
'long', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print',
'property', 'quit', 'range', 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set', 'setattr',
'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'unichr', 'unicode', 'vars',
'xrange', 'zip']

Syntax:

try:
<source code>
except:
<code to handle if exception occurs>

The above try..except will handle all the built-in exceptions. Also single try statement can
have multiple except statements.

Single except statement will handle all generic exceptions that python supports. If we want to
handle a specific exception then user need to provide an argument to except statement.

Except statement with generic exceptions:

try:
<source code>
except:
<code to handle if any exception occurs>

Except statement with single exceptions:

try:
<source code>
except NameError:
<code to handle if NameError exception occurs>

Except statement with multiple exceptions:

try:
<source code>
except (NameError, KeyError, MemoryError):
<code to handle if any of the above error occurs>
except:
<code to handle if any exception occurs>

Pyhton provides optional clauses called else and finally.

else: This block will get execute if no exception occurs in the source code and its syntax is
as follows:

try:
<source code>
except:
<code to handle if any exception occurs>
else:
<If no exception then execute this block>

try:
a=10/0
except:
print "Arithmetic Exception"
else:
print "Successfully Done"

o/p: Arithmetic Exception

finally: This block will get execute irrespective of exception occurs or not and its syntax is as
follows.

try:
<source code>
except:
<code to handle if any exception occurs>
else:
<if no exception then execute this block>
finally:
<execute this block irrespective of exception>

try:
a=10/5
print(a)
except:
print("Arithmetic Exception")
else:
print("Successfully Done")
finally:
print("Finally")

o/p:
Successfully Done
Finally

Raise exception:

In Python programming, exceptions are raised when corresponding errors occur at run time,
but we can forcefully raise it using the keyword raise.
We can also optionally pass in value to the exception to clarify why that exception was
raised.

>>> raise KeyboardInterrupt

Traceback (most recent call last):
KeyboardInterrupt
>>> raise MemoryError("This is an argument")
Traceback (most recent call last):
MemoryError: This is an argument
try:
a = int(input("Enter a positive integer: "))
if a <= 0:
raise ValueError("That is not a positive number!"
except ValueError as ve:
print(ve)
o/p: Enter a positive integer: -2 That is not a positive number!

Creating Custom Exceptions:

Python has many built-in exceptions which forces your program to output an error when
something in it goes wrong.

However, sometimes you may need to create custom exceptions that serves your purpose.

In Python, users can define such exceptions by creating a new class. This exception class
has to be derived, either directly or indirectly, from Exception class. Most of the built-in
exceptions are also derived form this class.

class UnderAge(Exception):
pass

def verify_age(age):
if int(age) < 18:
raise UnderAge
else:
print('Age: '+str(age))

# main program
verify_age(23) # won't raise exception
verify_age(17) # will raise exception
User defined Exceptions:

# define Python user-defined exceptions

class Error(Exception):
"""Base class for other exceptions"""
pass

class ValueTooSmallError(Error):
"""Raised when the input value is too small"""
pass

class ValueTooLargeError(Error):
"""Raised when the input value is too large"""
pass

# our main program

# user guesses a number until he/she gets it right

# you need to guess this number

number = 10

while True:
try:
i_num = int(input("Enter a number: "))
if i_num < number:
raise ValueTooSmallError
elif i_num > number:
raise ValueTooLargeError
break

except ValueTooSmallError:
print("This value is too small, try again!")
print()

except ValueTooLargeError:
print("This value is too large, try again!")
print()

print("Congratulations! You guessed it correctly.")

Packages and Modules
Names and Namespaces:
Namespaces:
Name (also called identifier) is simply a name given to objects. Everything in Python is
an object. Name is a way to access the underlying object.
For example, when we do the assignment a = 2, here 2 is an object stored in memory
and a is the name we associate it with. We can get the address (in RAM) of some object
through the built-in function, id(). Let's check it.

A namespace is basically a system to make sure that all the names in a program are unique
and can be used without any conflict. You might already know that everything in Python—
like strings, lists, functions, etc.—is an object. Another interesting fact is that Python
implements namespaces as dictionaries. There is a name-to-object mapping, with the
names as keys and the objects as values. Multiple namespaces can use the same name
and map it to a different object. Here are a few examples of namespaces:

Local Namespace: This namespace includes local names inside a function. This
namespace is created when a function is called, and it only lasts until the function returns.

Global Namespace: This namespace includes names from various imported modules that
you are using in a project. It is created when the module is included in the project, and it
lasts until the script ends.

Built-in Namespace: This namespace includes built-in functions and built-in exception
names.

A file containing Python code, for e.g.: abc.py, is called a module and its module name would
be abc. Thus module is simplify a python code where classes, variables and functions are
defined.

We use modules to break down large programs into small manageable and organized files.
Furthermore, modules provide reusability of code. We can define our most used functions in
a module and import it, instead of copying their definitions into different programs.

Python provides the following advantages for using module:

1) Reusability: Module can be used in some other python code. Hence it provides the facility
of code reusability.
2) Categorization: Similar type of attributes can be placed in one module.
Let us create a module. Type the following and save it as abc.py.

# Python Module abc

def add(a, b):
"""This program adds two
numbers and return the result"""

result = a + b
return result

def hello():
return "Hello World"

Here, we have defined a functions add() inside a module named abc. The function takes in
two numbers and returns their sum.

Now, if I want to make use of the functions inside module abc in some other file (xyz.py), we
have to import the module (abc.py) in module (xyz.py).

In Python, modules are accessed by using the import statement. When you do this, you
execute the code of the module, keeping the scopes of the definitions so that your current
file(s) can make use of these.

Python interpreter will look for the module in sys.modules (dictionary) first, if it’s not available
then it will search in all the paths which are under sys.path (list).

When Python imports a module called hello for example, the interpreter will first search for a
built-in module called hello. If a built-in module is not found, the Python interpreter will then
search for a file named hello.py in a list of directories that it receives from
the sys.path variable. sys.path is initialized from these locations:

1. The directory containing the input script (or the current directory).
2. PYTHONPATH (a list of directory names, with the same syntax as the shell variable
PATH).
3. The installation-dependent default.

Also, Python imports are case-sensitive. import Xyz is not the same as import xyz. While
importing a module extension(.py) is not required.

Python supports different ways of importing a module.

# Python Module foo

def bar(a, b):
return "Hey bar!"

def baz():
return "Hey baz!"

1. import foo.

The is the basic Python import. The statement import foo looks for a foo module, loads it into
memory and creates a module object called foo.
If there is a bar object (which could be anything from a function to a submodule) it can be
accessed like a member: foo.bar. You can also import several modules in one line by doing
import foo, bar, but it is considered good practice to put each import in a single line.

2. import foo.bar

This statement imports only bar attribute from foo module.

3. from foo import bar

This statement imports bar from foo module. It could be a function definition, a class or even
a submodule (which make foo a package). Notice that if bar is a submodule of foo, this
statement acts as if we simply imported bar (if it was in Python’s search path). This means
that a bar object is created, and its type is 'module'. No foo object is created in this
statement.
Multiple members of foo can be imported in the same line like so

4. from foo import bar, baz

The meaning of this is pretty intuitive: it imports both bar and baz from the module foo. bar
and baz aren’t neccessarily the same types: baz could be a submodule and bar could be
function, for that matter. Unlike importing unrelated modules, it’s perfectly acceptable to
import everything from one module in the same line.

5. from foo import *

Sometimes foo contains so many things, that it becomes cumbersome to import them
manually. Instead, you can just import * to import them all at the same time. Don’t do this
unless you know what you’re doing! It may seem convenient to just import * instead of
specific members, but it is considered bad practice.

def list():
raise RuntimeError("I'm rubber, you're glue!")

When you do import *, this list definition will override the global, built-in list type and you’ll get
very, very unexpected errors. So it’s always better to know exactly what you’re importing. If
you’re importing too much stuff from a certain package, you can either just suck it up or just
import the package itself (import foo) and use the foo qualifier for every use. An interesting
good use for import * is in Django settings file hierarchy. It’s convenient there because you
actually do want to manipulate the global namespace with imported settings.

6. from foo import bar as hello (also called as aliasing)

This one is far less common than what we covered so far, but still well known. It acts like
from foo import bar, except instead of creating a bar object, it creates a hello module with the
same meaning. There are two main reasons to use this kind of statement: the first is when
you’re importing two similarly named objects from two different modules. You then use
import as to differentiate them, like so:

from xml import Parser as XmlParser

from json import Parser as JsonParser
The other reason, which I’ve seen used a few times is when you import a long-named
function (or class) and use it extensively throughout your code and want to shorten its name.

7. from .foo import bar

Well, this escalated quickly.

This one is pretty rare and a lot of people are completely unaware of it. The only difference
in this statement is that it uses a modified search path for modules. Namely, instead of
searching the entire PYTHONPATH, it searches in the directory where the importing file
lives.
So if you have two files called fizz.py and foo.py, you can use this import in fizz, and it will
import the correct file, even if you have another foo module in your PYTHONPATH. What is
this good for? Well, sometime you create modules with generic names like common, but you
might also have a common package in the base of your project. Instead of giving different
names, you can explicitly import the one closest to you.
You can also use this method to load modules from an ancestor in the directory tree by
putting several dots. For example, from ..foo import Foo will search one directory up, from
...foo import Foo will search two directories up, etc.

8. reload(foo)

This statement does exactly what it looks like. It reloads the foo module. It’s pretty useful
when you have a console open playing with a bit of code you’re tweaking and want to
continue without resetting your interpreter.

Note: If you used from foo import bar, it’s not enough to reload foo for bar to update. You
need to both reload foo and call from foo import bar again.

Standalone Mode:

Python files can also be designed as both importable modules and top-level scripts. That is,
when run, the Python module will run as a stand-alone program, and when imported, it will
act as a importable module containing code definitions.

This is easily done using the attribute __name__ , which is automatically built into every
module. If the module is run as a top-level script the __name__ attribute will equal to the
string "__main__", otherwise if imported, it will contain the name of the actual module.

# hello.py
multiply = 3
def print_hi():
print("Hi!" * multiply)

# Stand-alone script part

if __name__ == '__main__':
print_hi()

The above 'hello.py' file defines a function, which will be exposed to the client when it's
imported. If the file is run as a stand-alone program, the same function is called
automatically. The basics, is that when 'hello.py' is imported it won't run the code nested
under the if __name__ == '__main__' statement.
Packages:

Similar files are kept in the same directory, for example, we may keep all the songs in the
"music" directory. Analogous to this, Python has packages for directories and modules for
files.

As our application program grows larger in size with a lot of modules, we place similar
modules in one package and different modules in different packages. This makes a project
(program) easy to manage and conceptually clear.

Similar, as a directory can contain sub-directories and files, a Python package can have sub-
packages and modules.

A directory must contain a file named __init__.py in order for Python to consider it as a
package. This file can be left empty but we generally place the initialization code for that
package in this file.

Any directory with a file named __init__.py is a Python package. This file can be
empty. This does NOT apply to Python 3.3 and above, thanks to the adoption of
implicit namespace packages. Basically, Python 3.3+ treats all folders as packages,
so empty __init__.py files are no longer necessary and can be omitted.

Here is an example. Suppose we are developing a game, one possible organization of

packages and modules could be as shown in the figure below.

Importing module from a package:

We can import modules from packages using the dot (.) operator. For example, if want to
import the start module in the above example, it is done as follows.

import Game.Level.start
Now if this module contains a function named select_difficulty(), we must use the full name
to reference it.

Game.Level.start.select_difficulty(2)

If this construct seems lengthy, we can import the module without the package prefix as
follows.

from Game.Level import start

We can now call the function simply as follows.

start.select_difficulty(2)

Yet another way of importing just the required function (or class or variable) form a module
within a package would be as follows.

from Game.Level.start import select_difficulty

Now we can directly call this function.

select_difficulty(2)

Although easier, this method is not recommended. Using the full namespace avoids
confusion and prevents two same identifier names from colliding.

While importing packages, Python looks in the list of directories defined in sys.path, similar
as for module search path.

Absolute vs. Relative Import

An absolute import uses the full path (starting from the project’s root folder) to the desired
module to import.
A relative import uses the relative path (starting from the path of the current module) to the
desired module to import. There are two types of relative imports:

an explicit relative import follows the format from .<module/package> import X, where
<module/package> is prefixed by dots . that indicate how many directories upwards to
traverse. A single dot . corresponds to the current directory; two dots .. indicate one folder
up; etc.

An implicit relative import is written as if the current directoy is part of sys.path. Implicit
relative imports are only supported in Python 2. They are NOT SUPPORTED IN PYTHON 3.
The Python documentation says the following about Python 3’s handling of relative imports:
The only acceptable syntax for relative imports is from .[module] import name. All import
forms not starting with . are interpreted as absolute imports.

For example, suppose we are running start.py which imports a1 which in turn imports other,
a2, and sa1. Then the import statements in a1.py would look as follows:

absolute imports:
import other
import packA.a2
import packA.subA.sa1
explicit relative imports:
import other
from . import a2
from .subA import sa1
implicit relative imports (NOT SUPPORTED IN PYTHON 3):
import other
import a2
import subA.sa1
Comprehensions
Comprehensions are elegant and efficient way of creating new objects and it allow you to
create iterable objects with a for loop with less code.
Comprehensions are of three types in python:

1. List Comprehension
2. Dictionary Comprehension
3. Set Comprehension
List Comprehension:
List comprehension is a complete substitute to for loops, lambda function as well as the
functions map(), filter() and reduce().
A list comprehension is a syntactic construct which creates a list based on existing list. List
comprehensions provide a concise way to create lists. It is a common requirement to make
new lists where each element is the result of some operations applied to each member of
another sequence or iterable, or to create a subsequence of those elements that satisfy a
certain condition.
List comprehension can even be easier to understand and use in practice.
A list comprehension can be used to:

1. transform a list
2. filter a list
Syntax:

L = [expression for variable in sequence [if condition]]

Examples:

nums = [1, 2, 3, 4]
squares = []
for i in squares:
squares.append(i * i)
print(squares) # [1, 4, 9, 16]

# using list comprehension

squares = [n * n for n in nums]
print(squares) # [1, 4, 9, 16]

a = [1, 2, 3, 4, 5, 6]
b = [e * 2 for e in a]
print(b) # [2, 4, 6, 8, 10, 12]

y = “welcome to the world of python”

new_list = [len(x) for x in y.split()]
print(new_list) # [7, 2, 3, 5, 2, 6]
List Comprehension with if condition:

nums = [x ** 2 for x in range(10) if x % 2 == 0]

print(nums) # [0, 4, 16, 36, 64]

d = ["hello", 'python', 'java', 'devops', "ruby"]

x = [(i, len(i)) for i in d if len(i) > 4]
print(x) # [('hello', 5), ('python', 6), ('devops', 6)]

List Comprehension with multiple if conditions:

we can create list comprehension with multiple if conditions.

number_list = [x for x in range(100) if x % 3 == 0 if x % 5 == 0]

print(number_list) # [0, 15, 30, 45, 60, 75, 90]

a = [9, 2, 18, 14, 22, 11, 7, 19, 23]

b = [e for e in a if e > 10 if e < 20]
print(b) # [18, 14, 11, 19]

List Comprehension with multiple for loops:

# creates a cartesian product using list comprehension.

a = [1, 2, 3]
b = ['A', 'B', 'C']
c = [str(i) + j for i in a for j in b]
print(c) # ['1A', '1B', '1C', '2A', '2B', '2C', '3A', '3B', '3C']

# find common numbers in two lists

list_a = [1, 2, 3, 4, 7]
list_b = [2, 6, 5, 4, 3]
common_num = [i for i in list_a for j in list_b if i == j]
print(common_num) # Output: [2, 3, 4]

Nested Loops in List Comprehension:

# Transpose of Matrix using Nested Loops.

matrix = [[1, 2],[3,4],[5,6],[7,8]]

transposed = []
for i in range(2):
transposed_row = []
for row in matrix:
transposed_row.append(row[i])
transposed.append(transposed_row)

print(transposed) # [[1,3,5,7],[2,4,6,8]]
# Transpose of Matrix using List Comprehension.

transpose = [[row[i] for row in matrix] for i in range(2)]

print (transpose) # [[1,3,5,7],[2,4,6,8]]
Dictionary Comprehensions:
Same like list comprehensions, Python now supports dictionary comprehensions, which
allow you to express the creation of dictionaries at runtime using a similarly concise syntax.
It is used to create a dictionary elements (key, value) in an elegant and efficient way.
A dictionary comprehension takes the form {key: value for (key, value) in iterable}. This
syntax was introduced in Python 3 and backported as far as Python 2.7, so you should be
able to use it regardless of which version of Python you have installed.
Syntax:

d = {key: value for (key, value) in iterable}

Examples:

a_list = [2, 4, 3, 5]
squares = {i:i*i for i in a_list}
print(squares) # {2: 4, 4: 16, 3: 9, 5: 25}

# dictionary comprehension to count values of same alphabets.

d = {'a':10, 'b': 34, 'A': 7, 'Z':3}
out = {k.lower(): d.get(k.lower(), 0) + d.get(k.upper(), 0) for k in d}
print(out) # {'a': 17, 'z': 3, 'b': 34}
# reverse keys and values in dictionary.
a_dict = {'a': 1, 'b': 2, 'c': 3}
out_dict = {value:key for key, value in a_dict.items()}
print(out_dict) # {1: 'a', 2: 'b', 3: 'c'}

{value:key for key, value in a_dict.items()}

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <dictcomp>
TypeError: unhashable type: 'list'

Set Comprehensions:
Python also supports comprehensions for set. Set comprehensions are similar to list
comprehensions, but it uses curly braces ({ }) to create new set.
Set comprehensions allow sets to be constructed using the same principles as list
comprehensions, the only difference is that resulting sequence is a set.
Python3 introduces set comprehensions.
Syntax:

S = {expression for variable in sequence [if condition]}

Examples:

nums = {n**2 for n in range(10)}

print(nums) # {0, 1, 64, 4, 36, 9, 16, 49, 81, 25}
nums = {n**2 for n in range(10) if n % 2 == 0}
print(nums) # {0, 64, 4, 36, 16, 25}
names = [ 'Bob', 'JOHN', 'alice', 'bob', 'ALICE', 'J', 'Bob' ]
out = { n[0].upper() + n[1:].lower() for n in names if len(n) > 1 }
print(out) # {'Bob', 'John', 'Alice'}

Collections:
Python’s collections module has specialized container datatypes that can be used to replace
Python’s general purpose containers (dict, tuple, list, and set). We will be studying the
following parts of this fun module:

• Counter
• defaultdict
• deque
• namedtuple
• OrderedDict
• ChainMap
All the above listed methods are part of collections module. To make use of these we have
to import them from collections module.

Counter:
The collections module also provides us with a neat little tool that supports convenient and
fast tallies. This tool is called Counter. You can run it against most iterables.

A counter is a dictionary-like object designed to keep tallies. With a counter, the key is the
item to be counted and value is the count. You could certainly use a regular dictionary to
keep a count, but a counter provides much more control.
A counter object ends up looking just like a dictionary and even contains a dictionary
interface.
One thing to note is that if you try to access a key that doesn't exist, the counter will
return 0rather than raising a KeyError as a standard dictionary would.

from collections import Counter

c = Counter('superfluous')
print(c) # Counter({'u': 3, 's': 2, 'e': 1, 'l': 1, 'f': 1, 'o': 1, 'r': 1,
'p': 1})

c['u'] # 3
c['z'] # 0

Counters come with a brilliant set of methods that will make your life easier if you learn how
to use them.

# to get most common elements from counter object

Counter('abracadabra').most_common() # [('a', 5), ('r', 2), ('b', 2),
('c', 1), ('d', 1)]
Counter('abracadabra').most_common(2) # [('a', 5), ('r', 2)]

# method to get elements from counter.

d = Counter('abracadabra')
list(d.elements())# ['c', 'r', 'r', 'b', 'b', 'd', 'a', 'a', 'a', 'a', 'a']

Defaultdict:

defaultdict is a dictionary like object which provides all methods provided by dictionary but
takes first argument (default_factory) as default data type for the dictionary.
defaultdict allows us to specify the default type of the value. This is simpler and faster than
using a regular dict with dict.setdefault.
The defaultdict is a subclass of Python’s dict that accepts a default_factory as its primary
argument. The default_factory is usually a Python type, such as int or list, but you can also
use a function or a lambda too.

Examples:

# defaultdict using list as a default type for value

from collections import defaultdict
s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
d = defaultdict(list)
for k, v in s:
d[k].append(v)
print(d) # defaultdict(<class 'list'>, {'yellow': [1, 3], 'red': [1],
'blue': [2, 4]})
d.items() # dict_items([('blue', [2, 4]), ('red', [1]), ('yellow', [1,
3])])

players = ["sachin cricket", "Federer tennis", "Nadal tennis", "virat

cricket", "Messi football"]
d = defdaultdict(list)
for i in players:
k, v = i.split()
d[v].append(k)

print(d) # defaultdict(<class 'list'>, {'cricket': ['sachin', 'virat'],

'football': ['ronaldo'], 'tennis': ['nadal', 'federer']})

# defaultdict using int as a default type for value

sentence = "The red for jumped over the fence and ran to the zoo for food"
words = sentence.split()
reg_dict = {}
for word in words:
if word in reg_dict:
reg_dict[word] += 1
else:
reg_dict[word] = 1
print(reg_dict) # {'The': 1, 'and': 1, 'fence': 1, 'food': 1, 'for': 2,
'jumped': 1, 'over': 1, 'ran': 1, 'red': 1, 'the': 2, 'to': 1, 'zoo': 1}
# using defaultdict!

from collections import defaultdict

words = sentence.split()
d = defaultdict(int)
for word in words:
d[word] += 1
print(d) # defaultdict(<class 'int'>, {'The': 1, 'and': 1, 'fence': 1,
'food': 1, 'for': 2, 'jumped': 1, 'over': 1, 'ran': 1, 'red': 1, 'the': 2,
'to': 1, 'zoo': 1})

You will notice right away that the code is much simpler. The defaultdict will automatically
assign zero as the value to any key it doesn’t already have in it. We add one so it makes
more sense and it will also increment if the word appears multiple times in the sentence.

deque:

deque stands for "double-ended queue" and is used as a stack or queue. Although lists offer
many of the same operations, they are not optimized for variable-length operations.

When to use deque:

Basically if you're structuring the data in a way that requires quickly appending to either end
or retrieving from either end then you would want to use a deque. For instance, if you're
creating a queue of objects that need to be processed and you want to process them in the
order they arrived, you would want to append new objects to one end and pop objects off of
the other end for processing.

queue = deque()
# append values to wait for processing
queue.appendleft("first")
queue.appendleft("second")
queue.appendleft("third")

# pop values when ready

process(queue.pop()) # would process "first"
# add values while processing
queue.appendleft("fourth")
# what does the queue look like now?
queue # deque(['fourth', 'third', 'second'])

As you can see we're adding items to the left and popping them from the right. Deque
provides four commonly used methods for appending and popping from either side of the
queue: append, appendleft, pop, and popleft.

In the above example we started with an empty deque, but we can also create a deque from
another iterable.

numbers = [0, 1, 2, 3, 5, 7, 11, 13]

queue = deque(numbers)
print(queue) # deque([0, 1, 2, 3, 5, 7, 11, 13])

namedtuple:

A namedtuple is a ... named tuple. When you use a standard tuple it's difficult to convey the
meaning of each position of the tuple. A named tuple is just like a normal tuple, but it allows
you to give names to each position making the code more readable and self-documenting.
Also with a namedtuple you can access the positions by name as well as index.
To instantiate we pass in the name of the type we want to create. Then we pass in a list of
field names.

from collections import namedtuple

# creating named tuple
Parts = namedtuple('Parts', 'id_num desc cost amount')
# creating an object to named tuple class Parts
auto_parts = Parts(id_num='1234', desc='Ford Engine', cost=1200.00, amount=10)
print(auto_parts.id_num) # 1234
print(auto_parts.amount) # 10

Here we import namedtuple from the collections module. Then we called namedtuple, which
will return a new subclass of a tuple but with named fields. So basically we just created a
new tuple class. you will note that we have a strange string as our second argument. This is
a space delimited list of properties that we want to create.

Now that we have our shiny new class, let’s create an instance of it! As you can see above,
we do that as our very next step when we create the auto_parts object. Now we can access
the various items in our auto_parts using dot notation because they are now properties of
our Parts class.

One of the benefits of using a namedtuple over a regular tuple is that you no longer have to
keep track of each item’s index because now each item is named and accessed via a class
property. Here’s the difference in code:

auto_parts = ('1234', 'Ford Engine', 1200.00, 10)

auto_parts[2] # 1200.0

id_num, desc, cost, amount = auto_parts

id_num # '1234'

You can convert a dictionary to a namedtuple using the double-start-operator.

c = {'x': 30, 'y': 45}

coordinate = namedtuple('Coordinate', ['x', 'y'])
d = coordinate(**c)
print(d) # coordinate(x=30, y=45)
print(d.x) # 30

OrderedDict:
OrderedDicts act just like regular dictionaries except they remember the order that items
were added. This matters primarily when you are iterating over the OrderedDict as the order
will reflect the order in which the keys were added.
As the name implies, this dictionary keeps track of the order of the keys as they are added. If
you create a regular dict, you will note that it is an unordered data collection.

A regular dictionary doesn't care about order:

d = {}
d[1] = 'one'
d[2] = 'two'
d[10] = 'ten'
print(d) # {1: 'one', , 10: 'ten', 2: 'two'}

from collections import OrderedDict

d = OrderedDict()
d[1] = 'one'
d[2] = 'two'
d[10] = 'ten'
print(d) # {1: 'one', 2: 'two', 10: 'ten'}

ChainMap
A ChainMap is a class that provides the ability to link multiple mappings together such that
they end up being a single unit. If you look at the documentation, you will notice that it
accepts **maps*, which means that a ChainMap will accept any number of mappings or
dictionaries and turn them into a single view that you can update. Let’s look at an example
so you can see how this works:

from collections import ChainMap

car_parts = {'hood': 500, 'engine': 5000, 'front_door': 750}
car_options = {'A/C': 1000, 'Turbo': 2500, 'rollbar': 300}
car_accessories = {'cover': 100, 'hood_ornament': 150, 'seat_cover': 99}
car_pricing = ChainMap(car_accessories, car_options, car_parts)
car_pricing['hood']  500

Here we import ChainMap from our collections module. Next we create three dictionaries.
Then we create an instance of our ChainMap by passing in the three dictionaries that we just
created
Iterators and Generators:
Iterator is an object which allows a programmer to traverse through all the elements of a
collection, regardless of its specific implementation.

Iterable: Any python object on which we can traverse or perform a loop.

Iteration: The process of traversing through an iterable is called an iteration.

Iterator:
In Python, an iterator is an object which implements the iterator protocol. The iterator
protocol consists of two methods. The __iter__() method, which must return the iterator
object, and the __next__ method, which returns the next element from a sequence.

Iterators have several advantages:

1. Cleaner code
2. Iterators can work with infinite sequences
3. Iterators save resources

Iterators are everywhere in Python. They are elegantly implemented within for loops,
comprehensions, generators etc. but hidden in plain sight.
Iterator in Python is simply an object that can be iterated upon. An object which will return
data, one element at a time.

Technically speaking, Python iterator object must implement two special

methods, __iter__()and __next__(), collectively called the iterator protocol.
An object is called iterable if we can get an iterator from it. Most of built-in containers in
Python like: list, tuple, string etc. are iterables.

The iter() function (which in turn calls the __iter__() method) returns an iterator from them.
__iter__ returns the iterator object itself. This is used in for and in statements.
__next__ method returns the next value from the iterator. If there is no more items to return
then it should raise StopIteration exception.

# iterating over a string object

s = "Dreamwin"
for e in str:
print(e, end=" ")

print() # D r e a m w i n
# create an iterator from iterable using iter() built-in function.
it = iter(s)
# In Python2 use next(it) where as in python 3 use it.__next__()
print(it.__next__())  D
print(it.__next__())  r
print(it.__next__())  e
print(list(it))  ['a', 'm', 'w', 'i', 'n']
print(it.__next__())
Traceback (most recent call last):
File "jdoodle.py", line 13, in <module>
 print(it.__next__())
StopIteration
Command exited with non-zero status 1

Lazy Evaluation:
In programming language theory, lazy evaluation or call-by-need is an evaluation strategy
which delays the evaluation of an expression until its value is needed (non-strict evaluation)
and which also avoids repeated evaluations (evaluated only once).
A really cool implication of lazy evaluation, is the possibility of creating an infinite (yes,
infinite) sequence of numbers.
Benefits:
1. Performance increases
2. The ability to construct potentially infinite data structures
3. The ability to define control flow
Therefore, each number in a sequence is evaluated on demand rather than immediately.
This is a very useful feature when dealing with large sequences of numbers, and can save
valuable execution time.
In python3 most of the built-in functions (map, filter, zip, reduce, range..) give an iterator
object instead of values.

# In python2
range(10)  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] # It returns list of values
xrange(10)  xrange(0, 10) # This returns an xrange iterator object.

# In python3 there is no xrange. Range behaves like an xrange.

a = range(10)  range(0, 10)
list(a)  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] # converts the values from iterator
into list.

Creating custom iterator:

Building an iterator from scratch is easy in Python. We just have to implement the methods
__iter__() and __next__().
The __iter__() method returns the iterator object itself. If required, some initialization can be
performed.
The __next__() method must return the next item in the sequence. On reaching the end, and
in subsequent calls, it must raise StopIteration.
Below iterator will give us next power of 2 in each iteration. Power exponent starts from zero
up to a user set number.

class PowTwo:

"""Class to implement an iterator

of powers of two"""
def __init__(self, max = 0):
self.max = max
 def __iter__(self):
self.n = 0
return self
def __next__(self):
if self.n <= self.max:
result = 2 ** self.n
self.n += 1
return result
else:
raise StopIteration

Now we can create an iterator and iterate through it as follows.

a = PowTwo(4)
i = iter(a)
i.__next__()  1
i.__next__()  2
i.__next__()  4
i.__next__()  8
i.__next__()  16
i.__next__()
Traceback (most recent call last):
StopIteration

We can also use a for loop to iterate over our iterator class.

for i in PowTwo(5):
print(i)
1
2
4
8
16
32

Generators:

There is a lot of overhead in building an iterator in Python; we have to implement a class

with __iter__() and __next__() method, keep track of internal states, raise StopIteration
when there was no values to be returned etc.
This is both lengthy and counter intuitive. Generator comes into rescue in such situations.
Python generators are a simple way of creating iterators. All the overhead we mentioned
above are automatically handled by generators in Python.
Simply speaking, a generator is a function that returns an object (iterator) which we can
iterate over (one value at a time).
Generators in Python:

1. Are defined with the def keyword

2. Use the yield keyword
3. May use several yield keywords
4. Return an iterator

Creating Generator:

It is fairly simple to create a generator in Python. It is as easy as defining a normal function

with yield statement instead of a return statement.
If a function contains at least one yield statement (it may contain other yield or return
statements), it becomes a generator function. Both yield and return will return some value
from a function.
The difference is that, while a return statement terminates a function entirely, yield statement
pauses the function saving all its states and later continues from there on successive calls.

Differences between Generator function and a Normal function

 Generator function contains one or more yield statement.

 When called, it returns an object (iterator) but does not start execution immediately.
 Methods like __iter__() and __next__() are implemented automatically. So we can
iterate through the items using next() in python2 and __next__() in python3.
 Once the function yields, the function is paused and the control is transferred to the
caller.
 Local variables and their states are remembered between successive calls.
 Finally, when the function terminates, StopIteration is raised automatically on further
calls.

Here is an example to illustrate all of the points stated above. We have a generator function
named my_gen() with several yield statements.

def my_gen():
yield "hello"
yield "Python"
yield 10

a = my_gen()
print(a)  <generator object my_gen at 0x7fa03d779460>
print(a.__next__())  hello
print(a.__next__())  1
print(a.__next__())  Python
print(a.__next__())  2
print(a.__next__())  10
print(a.__next__())  3
print(a.__next__())

Traceback (most recent call last):

File "gen.py", line 11, in <module>
print(next(a))
StopIteration

One interesting thing to note in the above example is that, the value of variable n is
remembered between each call.
Unlike normal functions, the local variables are not destroyed when the function yields.
Furthermore, the generator object can be iterated only once.
To restart the process we need to create another generator object using something like a =
my_gen().

Difference Between return and yield:

The keyword return returns a value from a function, at which time the function then loses its
local state. Thus, the next time we call that function, it starts over from its first statement.
On the other hand, yield maintains the state between function calls, and resumes from
where it left off when we call the next() method again. So if yield is called in the generator,
then the next time the same generator is called we'll pick right back up after the last yield
statement

Python Generator Expression:

Simple generators can be easily created on the fly using generator expressions. It makes
building generators easy. Same as lambda function creates an anonymous function,
generator expression creates an anonymous generator function.
The syntax for generator expression is similar to that of a list comprehension in Python. But
the square brackets are replaced with round parentheses ().
The major difference between a list comprehension and a generator expression is that while
list comprehension produces the entire list, generator expression produces one item at a
time.
They are kind of lazy, producing items only when asked for. For this reason, a generator
expression is much more memory efficient than an equivalent list comprehension.

# Initialize the list

my_list = [1, 3, 6, 10]
# square each term using list comprehension
[x**2 for x in my_list]  # [1, 9, 36, 100]
# same thing can be done using generator expression
val = (x**2 for x in my_list)
print(val)  <generator object <genexpr> at 0x0000000002EBDAF8>
print(a.__next__())  1
print(a.__next__())  9
print(a.__next__())  36
print(a.__next__())  100
print(a.__next__())  StopIteration

Advantages of Generator:

1. Easy to Implement
Generators can be implemented in a clear and concise way as compared to their iterator
class counterpart. Following is an example to implement a sequence of power of 2's using
iterator class.

class PowTwo:
def __init__(self, max = 0):
self.max = max
 def __iter__(self):
self.n = 0
return self

def __next__(self):
if self.n > self.max:
raise StopIteration
result = 2 ** self.n
self.n += 1
return result

This was lengthy. Now lets do the same using a generator function.

def PowTwoGen(max = 0):

n = 0
while n < max:
yield 2 ** n
n += 1

Since, generators keep track of details automatically, it was concise and much cleaner in
implementation.
2. Memory Efficient
A normal function to return a sequence will create the entire sequence in memory before
returning the result. This is an overkill if the number of items in the sequence is very large.
Generator implementation of such sequence is memory friendly and is preferred since it only
produces one item at a time.
3. Represent Infinite Stream
Generators are excellent medium to represent an infinite stream of data. Infinite streams
cannot be stored in memory and since generators produce only one item at a time, it can
represent infinite stream of data.
The following example can generate all the even numbers (at least in theory).

def all_even():
n = 0
while True:
yield n
n += 2
File Handling:
File is a named location on disk to store related information. It is used to permanently store
data in a non-volatile memory (e.g. hard disk).

Since, random access memory (RAM) is volatile which loses its data when computer is
turned off, we use files for future use of the data.

When we want to read from or write to a file we need to open it first. When we are done, it
needs to be closed, so that resources that are tied with the file are freed.

Hence, in Python, a file operation takes place in the following order.

1. Open a file
2. Read or write (perform operation)
3. Close the file

File Types:

1. Text Files.
2. Binary Files

A file whose contents can be viewed using a text editor is called a text file. A text file is
simply a sequence of ASCII or Unicode characters. Python programs, HTML source code
are some of the example of text files.

A binary file stores the data in the same way as stored in the memory. The mp3 files, image
files, word documents are some of the examples of binary files. You can't read a binary file
using a text editor.

Opening a File:

Before you perform any operation on a file, you must the open it. Python has a built-in
function open() to open a file. This function returns a file object, also called a handle, as it is
used to read or modify the file accordingly.

fileobject = open(filename, mode) # open a file.

f = open("employee.txt") # open file in current directory

f = open("C:/Python33/README.txt", "rt") # specifying full path

We can specify the mode while opening a file. In mode, we specify whether we want to read
'r', write 'w' or append 'a' to the file. We also specify if we want to open the file in text mode
or binary mode.
The default is reading in text mode. In this mode, we get strings when reading from the file.

On the other hand, binary mode returns bytes and this is the mode to be used when dealing
with non-text files like image or exe files.
File Modes:

Mode Description

'r' Open a file for reading. (default)

Open a file for writing. Creates a new file if it does not exist or truncates the file if
'w'
it exists.

'x' Open a file for exclusive creation. If the file already exists, the operation fails.

Open for appending at the end of the file without truncating it. Creates a new file
'a'
if it does not exist.

't' Open in text mode. (default)

'b' Open in binary mode.

'+' Open a file for updating (reading and writing)

Different file modes:

f = open("test.txt") # equivalent to 'r' or 'rt'

f = open("test.txt",'w') # write in text mode
f = open("img.bmp",'r+b') # read and write in binary mode

Assume a file named welcome.txt contains below text.

This is my first file

This file
contains three lines

File data can be read in three ways:

Read: The read() function reads the specified number of bytes from the file. If the number of
bytes is not specified, it reads the whole file.

f = open("welcome.txt") # equivalent to 'r' or 'rt'

data = f.read() # reads entire content from a file and stores in a string
data = f.read(5) # reads only first five characters from file.
data = f.read(10) # reads other ten characters from file.

f.read(4)  'This' # read the first 4 data

f.read(4)  ' is ' # read the next 4 data
f.read() # read in the rest till end of file
'my first file\nThis file\ncontains three lines\n'
f.read()  ' ' # further reading returns empty sting
The code above read five characters from the file and prints them to the console. Then it
reads and prints another ten characters. The second operation continues reading from the
position where the first has ended.

Readline:

The readline() method reads a line from the file. A trailing newline character is kept in the
string. When the function reaches the end of file, it returns an empty string.

we can use readline() method to read individual lines of a file. This method reads a file till
the newline, including the newline character.

f = open("welcome.txt") # equivalent to 'r' or 'rt'

f.readline()  'This is my first file\n'
f.readline()  'This file\n'
f.readline()  'contains three lines\n'
f.readline()  ''

Readlines:

readlines() method returns a list of remaining lines of the entire file. All these reading method
return empty values when end of file (EOF) is reached.

f = open("welcome.txt") # equivalent to 'r' or 'rt'

f.readlines()
['This is my first file\n', 'This file\n', 'contains three lines\n']

Reading a file using for loop:

We can read a file line-by-line using a for loop. This is both efficient and fast.

f = open("welcome.txt") # equivalent to 'r' or 'rt'

for line in f:
print(line, end = '')

This is my first file

This file
contains three lines

Writing into file: (w)

In order to write into a file in Python, we need to open it in write 'w', append 'a' or exclusive
creation 'x' mode.
We need to be careful with the 'w' mode as it will overwrite into the file if it already exists. All
previous data are erased.
Writing a string or sequence of bytes (for binary files) is done using write() method. This
method returns the number of characters written to the file.

f = open("welcome.txt", "w") # creates and open a file in write mode.

f.write("my first file\n")
f.write("This file\n\n")
f.write("contains three lines\n")
f.close() # closing a file.
This program will create a new file named 'welcome.txt' if it does not exist. If it does exist, it
is overwritten.

Writelines:

Using writelines we can write multiple lines into file through list of lines.

f = open("welcome.txt", "w")
f.writelines(['one\n', 'two\n', 'three\n'])
f.close()

Append: (a)

Using append mode we can write content into file. It will create a new file named
'welcome.txt' if it does not exist. If it does exist, it is append the content into the same file at
the end.

f = open("welcome.txt", "a") # creates and open a file in append mode.

f.write("my first file\n")
f.write("This file\n\n")
f.write("contains three lines\n")
f.close() # closing a file.

Closing a file:

Once we are done working with the file or we want to open the file in some other mode,
we should close the file using close() method of the file object. Closing a file will free
up the resources that were tied with the file.
Python has a garbage collector to clean up unreferenced objects but, we must not rely on it
to close the file.

f = open("welcome.txt")
# perform file operations
f.close()

This method is not entirely safe. If an exception occurs when we are performing some
operation with the file, the code exits without closing the file. A safer way is to use
a try...finally block.

try:
f = open("welcome.txt")
# perform file operations
finally:
f.close()

This way, we are guaranteed that the file is properly closed even if an exception is raised,
causing program flow to stop.
The best way to do this is using the with statement. This ensures that the file is closed when
the block inside with is exited. We don't need to explicitly call the close() method. It is done
internally.
File Positions:

A file position is a place in the file from where we read data.

Tell: tell() method gives the current position in a file.
Seek: seek() method moves the position in a file.

File methods Overview:

Method Description
Reads the specified number of characters from the file and returns them as
read([num])
string. If num is omitted then it reads the entire file.
readline() Reads a single line and returns it as a string.
readlines() Reads the content of a file line by line and returns them as a list of strings.
Writes the string argument to the file and returns the number of characters
write(str)
written to the file.
seek(offset,
Moves the file pointer to the given offset from the origin.
origin)
tell() Returns the current position of the file pointer.
close() Closes the file

Reading large files:

The read() and readlines() methods work great with small files. But what if your file has
thousands or millions of lines in it? In such cases using read() or readlines() may result in
memory hogs. A better approach would be to use loops and read file data in small chunks.

f = open("welcome.txt", "r")
chunk = 10 # specify chunk size
data = ""

# keep looping until there is data in the file

while True:
data = f.read(chunk)
print(data, end="")
# if end of file is reached, break out of the while loop
if data == "":
break
f.close()

Here we are using an infinite loop to iterate over the contents of the file. As soon as the end
of file is reached, the read() method returns an empty string (" "), if condition evaluates to
true and break statement causes the loop to terminate.
Working with files using with statement:

Python also provides a nice shortcut for file handling using the with statement called context
manager. The following is the general form of the with statement when used with files.

with open(filename, mode) as file_object:

# body of with statement
# perform the file operations here

The best thing about this shortcut is that it automatically closes the file without requiring any
work on your part. The statements inside the body of the with statement must be equally
indented otherwise you will get an error. The scope of file_object variable is only limited to
the body of the with statement. If you try to call read() or write() method on it outside the
block you will get an error.
The following examples show how we can use the with statement to read and write data to
and from the file.

with open("welcome.txt", "r") as f:

for line in f:
print(line, end="")

Writing content into file using with:

with open("random.txt", "w") as f:

f.write("ONE D\n")
f.write("TWO D\n")
f.write("THREE D\n")
f.write("FOUR D\n")
Object Oriented Programming in Python
There are two common programming paradigms in use:

1. Procedural Programming.
2. Object Oriented Programming.

Procedural Programming:

Procedural programming uses series of steps to tell computer what to do. Procedural
programming extensively uses procedures, we can think of procedures as functions which
perform a specific tasks, such as calculate the incentive of an employee, save the data to
the database, run backup and so on.
The central idea behind procedural programming is to create reusable functions to operate
on data. There is nothing wrong with this approach, but as the program grows it becomes
difficult to manage. All the programs we have written so far were procedural as they
extensively rely on procedures/functions to perform various tasks.

Object Oriented Programming:

The Object Oriented Programming revolves around objects instead of procedures. An object
is an entity which contains data as well as procedures that operates on the data. The data
and procedures inside an object is known as attributes and methods respectively.
Before we create objects, we first have to define a class. A class is simply a template where
we define attributes and methods. When we define a class, we essentially create a new data
type. It is important to note that class is just a blueprint, it does nothing by itself. To use a
class we have to create objects that are based upon that class.
An object is also known as instance of a class or class instance or just instance. The
process of creating object from a class is known as instantiating a class and we can create
as many objects as needed.

There are some basic programming concepts in OOP:

1. Inheritance
2. Polymorphism
3. Encapsulation
4. Abstraction

The inheritance is a way to form new classes using classes that have already been defined.
The polymorphism is the process of using an operator or function in different ways for
different data input. The encapsulation hides the implementation details of a class from other
objects. The abstraction is simplifying complex reality by modelling classes appropriate to
the problem.

Everything in Python is an object. Objects are basic building blocks of a Python OOP
program.
import sys

def function():
pass

print(type(1))  <class 'int'>

print(type(""))  <class 'str'>
print(type([]))  <class 'list'>
print(type({}))  <class 'dict'>
print(type(()))  <class 'tuple'>
print(type(object))  <class 'type'>
print(type(function))  <class 'function'>
print(type(sys))  <class 'module'>

In this example we show that all these entities are in fact objects. The type() function returns
the type of the object specified. Integers, strings, lists, dictionaries, tuples, functions, and
modules are Python objects.

Class:

Class can be defined as a collection of objects. It is a logical entity that has some specific
attributes and methods. An object is the fundamental concept of OOP but classes provide an
ability to define similar type of objects.

For example: if you have an employee class then it should contain an attribute and method
i.e. an email id, name, age, salary etc.

The user defined objects are created using the class keyword. The class is a blueprint that
defines a nature of a future object. From classes we construct instances or objects.

Syntax for defining a class:

Defining a class is simple, all you have to do is use the keyword class followed by the name
that you want to give your class, and then a colon symbol : . It is standard approach to start
the name of class with a capital letter and then follow the camel case style.
The class definition is included, starting from the next line and it should be indented, as
shown in the code below. Also, a class can have variables and member functions in it.

Syntax:

class MyClass:
<statement-1>
<statement-N>

Object:

Object is an entity that has state and behaviour. It may be anything which is physical and
logical.

1. Objects are the basic run-time entities in an object-oriented system.

2. They may represent a person, a place, a bank account, a table of data or any item
that the program must handle.
3. When a program is executed the objects interact by sending messages to one
another.
4. Objects have two components:
Data (i.e., attributes)
Behaviours (i.e., methods)

Creating object for a class:

Class is mere a blueprint or a template. No storage is assigned when we define a class.

Objects are instances of class, which holds the data variables declared in the class and the
member functions work on these class objects.
To create an object, all we have to do is:

myObject = MyClass()
myObject = MyClass(arguments) # In case if __init__ expects any arguments

where, MyClass is the name of the class and myObject is the object variable.
Do you remember how we were able to initialize a list by writing myList = list(). Similar is the
case here, we have created a user-defined data type(a class) called MyClass, and in order
to inform python that myObject variable will be of that datatype, we make a call to this class.
In python, the object creation part is divided into two parts:

1. Object Creation or Constructor (__new__)

2. Object Initialisation or Initializer (__init__)

Object Creation:
Object creation is controlled by a static class method with the name __new__ and which is
also generally called as a Constructor. Hence when you call DreamWin(), to create an object
of the class MyClass, then the __new__ method of this class is called. Python defines this
function for every class by default, although you can always do that explicitly too, to play
around with object creation.
class DreamWin:
var = "Welcome to DreamWin" # class variable

obj = DreamWin()
print(obj)  <__main__.DreamWin object at 0x0000000001E3DE48>
print(obj.var)  Welcome to DreamWin

In the above code an object named obj got created for a class DreamWin using __new__
method.

Object Initialisation:
Object initialisation is controlled by an instance method with the name __init__ and which is
also generally called as a Initializer. Although, both __new__ and __init__ together forms a
constructor.
Once the object is created, you can make sure that every variable in the object is correctly
initialised by defining an __init__ method in your class, which pretty much means init-iate.
# A Sample class with init method
class Person:

# init method or Initializer

def __init__(self, name):
self.name = name

# Sample Method
def say_hi(self):
print('Hello, my name is', self.name)
p = Person('DreamWin') # object creation and Instantiation happens here.
p.say_hi()  Hello, my name is DreamWin

Pictorial representation of object creation in python:

Self:

Class methods have only one specific difference from ordinary functions - they must have
an extra first name that has to be added to the beginning of the parameter list, but you do
not give a value for this parameter when you call the method, Python will provide it. This
particular variable refers to the object itself, and by convention, it is given the name self.

Although, you can give any name for this parameter, it is strongly recommended that you
use the name self.

self is nothing but an object itself. If we have a method which takes no arguments, then we still
have to have one argument – the self.

Method:

If we define a function in class with self as first parameter then function will become method.
In the above example say_hi is the method of Person class.
Accessing attributes and methods:

Once we have an object of a class, we can use it to access object's attribute (or instance
variable) and methods using the following syntax:

object.attribute # syntax to access attributes

object.method(arguments) # syntax to access instance methods

Class and Instance Variables (Or attributes):

In Python, instance variables are variables whose value is assigned inside a constructor or
method with self. Class variables are variables whose value is assigned in class.
There are two types of fields - class variables and object variables which are classified
depending on whether the class or the object owns the variables respectively.

Class Variables: Class variables are shared - they can be accessed by all instances of that
class. There is only one copy of the class variable and when any one object makes a change
to a class variable, that change will be seen by all the other instances.

Instance Variables: Object variables are owned by each individual object/instance of the
class. In this case, each object has its own copy of the field i.e. they are not shared and are
not related in any way to the field by the same name in a different instance.

# Python program to show that the variables with a value assigned in class
declaration, are class variables and variables inside methods and constructors are
instance variables.

# Class for Computer Science Student

class CSStudent:

# Class Variable
stream = 'cse'

# The init method or constructor

def __init__(self, roll):

# Instance Variable
self.roll = roll

# Objects of CSStudent class

a = CSStudent(101)
b = CSStudent(102)

print(a.stream) # prints "cse"

print(b.stream) # prints "cse"
print(a.roll) # prints 101

# Class variables can be accessed using class name also

print(CSStudent.stream) # prints "cse"

Bound vs Unbound Methods:

In Python, we can call methods in two ways. There are bounded and unbounded method
calls.
class Methods():

def __init__(self):
self.name = 'Methods'

def getName(self):
return self.name

m = Methods()
print(m.getName())
print(Methods.getName(m))

Bound Method: A method which can be called using an instance (self) is called Bound
Method.
print(m.getName())  Methods

This is the bounded method call. The Python interpreter automatically pairs the m instance
with the self parameter.

Unbound Method: A method which can be called using class is called an unbound method.
It means this method call is not bound to an instance.
print(Methods.getName(m))  Methods

And this is the unbounded method call. The instance object is explicitly given to the
getName() method.
Example:
class BankAccount:

def __init__(self, balance):

self.balance = balance

def make_deposit(self, amount):

self.balance += amount

def make_withdrawal(self, amount):

if self.balance < amount:
print("Error: Not enough funds")
else:
print("Successfully withdrawn $", amount, sep="")
self.balance -= amount

def get_balance(self):
return self.balance

my_account = BankAccount(5000) # Create my bank account with $5000

print("Current Balance: $", my_account.get_balance(), sep="")

print("Withdrawing $10000 ...")

my_account.make_withdrawal(10000)

print("Lets try withdrawing $1000 ...")

my_account.make_withdrawal(1000)
print("Now Current Balance: $", my_account.get_balance(), sep="")

print("Depositing $2000 ...")

my_account.make_deposit(2000)
print("Now Current Balance: $", my_account.get_balance(), sep="")

Old and New Style Class

Python 3 by default has new classes. While, if you are using Python 2, you still have two
options. New style classes are the ones whose first parent inherits from Python root ‘object’
class. New style classes were introduced in python 2.2, so if by any chance you are using
Python 2.1 or earlier, you are bound to use old-style classes.

#Old style class

class OldStyleClass: pass

#New style class

class NewStyleClass(object): pass

Object Oriented Concepts:

Inheritance:

Inheritance is a mechanism which allows us to create a new class known as child class
(derived class or sub class), that is based upon an existing class the parent class (super
class or Base class), by adding new attributes and methods on top of the existing class.
When you do so, the child class inherits attributes and methods of the parent class.

Inheritance really shines when you want to create classes that are very similar. All you need
to do is to write the code for the things that they have common in one class the parent class.
And then write code for things that are very specific in a different class the child class. This
saves you from duplicating a lot of code.

Object is root of all classes. In Python 3.x, “class Test(object)” and “class Test” are same.
In Python 2.x, “class Test(object)” creates a class with object as parent (called new style
class) and “class Test” creates old style class (without object parent).

Syntax:

class ParentClass:
# body of ParentClass
# method1
# method2

class ChildClass(ParentClass):
# body of ChildClass
# method 1
# method 2
# A Python program to demonstrate inheritance

# Base or Super class. Note object in bracket. # (Generally, object is made

ancestor of all classes)
# In Python 3.x "class Person" is equivalent to "class Person(object)"

class Person(object):

# Constructor
def __init__(self, name):
self.name = name

# To get name
def getName(self):
return self.name

# To check if this person is employee

def isEmployee(self):
return False

# Inherited or Sub class (Note Person in bracket)

class Employee(Person):

# Here we return true

def isEmployee(self):
return True

# Driver code
emp = Person("Shiva") # An Object of Person
print(emp.getName(), emp.isEmployee())  Shiva False

emp = Employee("Raj") # An Object of Employee

print(emp.getName(), emp.isEmployee())  Raj True

Python has different types of inheritances:

1. Multilevel Inheritance
2. Multiple Inheritance
Multilevel Inheritance:

Multilevel inheritance is also possible in Python unlike other programming languages. You
can inherit a derived class from another derived class. This is known as multilevel
inheritance. In Python, multilevel inheritance can be done at any depth.

In multilevel inheritance, features of the base class and the derived class is inherited into the
new derived class.

class Base:
pass

class Derived1(Base):
pass
class Derived2(Derived1):
pass

Example:
class Animal:
def (self):
print('Eating...')
class Dog(Animal):
def bark(self):
print('Barking...')
class BabyDog(Dog):
def weep(self):
print('Weeping...')
d=BabyDog() # created an instance to topmost derived class
d.eat()  Eating…
d.bark()  Barking…
d.weep()  Weeping…

Multiple Inheritance:

A class can be derived from more than one base classes in Python. This is called multiple
inheritance. In multiple inheritance, the features of all the base classes are inherited into the
derived class. The syntax for multiple inheritance is similar to single inheritance.

Syntax:
class Base1:
pass

class Base2:
pass
class MultiDerived(Base1, Base2):
pass

Here, MultiDerived is derived from classes Base1 and Base2.

Example:

# Python example to show working of multiple inheritance

class Base1:
def __init__(self):
self.str1 = "Base1 init"
print("Base1")
class Base2:
def __init__(self):
self.str2 = "Base2 init"
print("Base2")
class Derived(Base1, Base2):
def __init__(self):
# Calling constructors of Base1 and Base2 class
Base1.__init__(self)
Base2.__init__(self)
print("Derived"
def printStrs(self):
print(self.str1, self.str2)
ob = Derived()
ob.printStrs()

Python has two useful functions:

1. Isinstance
2. Issubclass

One can use issubclass() or isinstance() functions to check a relationships of two classes
and instances.

Isinstance: The isinstance function helps you to check whether "something" is an object of a
certain class or not.

Issubclass: The issubclass function checks whether this class is the child of another class or
not.

# Python example to check if a class is subclass of another

class Base(object):
pass # Empty Class

class Derived(Base):
pass # Empty Class

print(issubclass(Derived, Base))  True

print(issubclass(Base, Derived))  False

d = Derived()
b = Base()

# b is not an instance of Derived

print(isinstance(b, Derived))  False

# But d is an instance of Base

print(isinstance(d, Base))  True

Accessing parent class (Base Class) members in child class (Derived Class):

# Python example to show that base class members can be accessed in derived class
using base class name.
class Base:
# Constructor
def __init__(self, x):
self.x = x
class Derived(Base):
# Constructor
def __init__(self, x, y):
super(Derived, self).__init__(x) # or super().__init__(x)
self.y = y
def printXY(self):
print(self.x, self.y) # print(self.x, self.y) will also work

d = Derived(10, 20)
d.printXY()  10 20
Polymorphism:

Polymorphism means “Poly mean Multiple” and “Morph means Forms”. It is one feature
of Object Oriented Paradigm having ability of taking more than one form.

Polymorphism is of two types:

Compile-time or Static Polymorphism (Overloading)

Run-time or Dynamic Polymorphism (Overriding)

Compile-time polymorphism (or static binding): the polymorphism exhibited at compile time
is called static polymorphism.

Run-time Polymorphism (Dynamic Polymorphism): the polymorphism exhibited at run time is

called dynamic polymorphism. This means when a method is bound to the method body at
the time of running the program dynamically.

Overloading:

Method Overloading:

When two or more methods (functions) in the same class have the same name but different
parameters is called method overloading.

Python does not supports method overloading in traditional way as supported by other OOP
languages. We may overload the methods but can only use the latest defined method.

class A:
# First product method. Takes two argument and print their product
def product(self, a, b):
p = a * b
print(p)
# Second product method takes three argument and print their product
def product(self, a, b, c):
p = a * b *c
print(p)
# Uncommenting the below line shows an error
obj = A()
obj.product(4, 5)
# This line will call the second product method
obj.product(4, 5, 5)

In the above code we have defined two product method, but we can only use the second
product method, as python does not supports method overloading. We may define many
method of same name and different argument but we can only use the latest defined
method. Calling the other method will produce an error. Like here calling will produce an
error as the latest defined product method takes three arguments.

In Python instead of defining two functions, we can achieve this using variable no of
arguments (*args) and key-word arguments (**kwargs).
# Function to take multiple arguments
def add(datatype, *args):
# if datatype is int initialize answer as 0
if datatype =='int':
answer = 0
# if datatype is str initialize answer as ''
if datatype =='str':
answer = ''

# Traverse through the arguments

for x in args:
# This will do addition if the arguments are int. Or concatenation if the
arguments are st.
answer = answer + x
print(answer)
# Integer
add('int', 5, 6)
# String
add('str', 'Hi ', 'Geeks')

Operator Overloading:

Python operators work for built-in classes. But same operator behaves differently with
different types. For example, the + operator will, perform arithmetic addition on two numbers,
merge two lists and concatenate two strings.
This feature in Python, that allows same operator to have different meaning according to the
context is called operator overloading.

# First product method. Takes two argument and print their product
5 + 10  15 # Addition
'Hello' + 'python'  Hellopython # Concatenation.
[1, 2, 3] + [4, 5, 6]  [1, 2, 3, 4, 5, 6] # Merge two lists.

So what happens when we use them with objects of a user-defined class? Let us consider
the following class, which tries to simulate a point in 2-D coordinate system.

class Point:
def __init__(self, x = 0, y = 0):
self.x = x
self.y = y

p1 = Point(2, 3)
p2 = Point(-1, 2)
p1 + p2
Traceback (most recent call last):
TypeError: unsupported operand type(s) for +: 'Point' and 'Point'

TypeError was raised since Python didn't know how to add two Point objects together.
However, the good news is that we can teach this to Python through operator overloading.
But first, let's get a notion about special functions.
Special Functions in Python:

Class functions that begins with double underscore (__) and ends with double underscores
(__) are called special functions in Python. This is because, well, they are not ordinary.
The __init__() function we defined above, is one of them. It gets called every time we create
a new object of that class. There are a ton of special functions in Python.
Using special functions, we can make our class compatible with built-in functions.

p1 = Point(2,3)
print(p1)
<__main__.Point object at 0x00000000031F8CC0>

That did not print well. But if we define __str__() method in our class, we can control how it
gets printed. So, let's add this to our class.

class Point:
def __init__(self, x = 0, y = 0):
self.x = x
self.y = y
def __str__(self):
return "({0},{1})".format(self.x,self.y)

Now let's try the print() function again.

p1 = Point(2,3)
print(p1)  (2,3)

That's better. Turns out, that this same method is invoked when we use the built-in
function str() or format().

str(p1)  '(2,3)'
format(p1)  '(2,3)'

So, when you do str(p1) or format(p1), Python is internally doing p1.__str__(). Hence the
name, special functions. Ok, now back to operator overloading.

Overloading the + Operator in Python:

To overload the + sign, we will need to implement __add__() function in the class. With great
power comes great responsibility. We can do whatever we like, inside this function. But it is
sensible to return a Point object of the coordinate sum.

class Point:
def __init__(self, x = 0, y = 0):
self.x = x
self.y = y

def __str__(self):
return "({0},{1})".format(self.x,self.y)
 def __add__(self,other):
x = self.x + other.x
y = self.y + other.y
return Point(x,y)
p1 = Point(2,3)
p2 = Point(-1,2)
print(p1 + p2)  (1,5)

What actually happens is that, when you do p1 + p2, Python will call p1.__add__(p2) which
in turn is Point.__add__(p1,p2). Similarly, we can overload other operators as well. The
special function that we need to implement is tabulated below.

Operator Expression Internally

Addition p1 + p2 p1.__add__(p2)

Subtraction p1 - p2 p1.__sub__(p2)

Multiplication p1 * p2 p1.__mul__(p2)

Power p1 ** p2 p1.__pow__(p2)

Division p1 / p2 p1.__truediv__(p2)

Floor Division p1 // p2 p1.__floordiv__(p2)

Remainder (modulo) p1 % p2 p1.__mod__(p2)

Bitwise Left Shift p1 << p2 p1.__lshift__(p2)

Bitwise Right Shift p1 >> p2 p1.__rshift__(p2)

Bitwise AND p1 & p2 p1.__and__(p2)

Bitwise OR p1 | p2 p1.__or__(p2)

Bitwise XOR p1 ^ p2 p1.__xor__(p2)

Bitwise NOT ~p1 p1.__invert__()

Overriding:

Method Overriding:
Overriding is the ability of a class (child or derived class) to change the implementation of a
method provided by one of its ancestors (parent or base class).

Override means having two methods with the same name but doing different tasks. It means
that one of the methods overrides the other. If there is any method in the superclass and a
method with the same name in a subclass, then by executing the method, the method of the
corresponding class will be executed.

A subclass may change the functionality of a Python method in the superclass. It does so by
redefining it. This is termed python method overriding.

class A:
def sayhi(self):
print("I'm in A")

class B(A):
def sayhi(self):
print("I'm in B")
obj = B()
obj.sayhi()  "I'm in B"

In the above code we have created two classes with same name and having same number
of arguments. I have created an object to class B, and if I call a method sayhi it will call B’s
class method.

If I want to make use of class A’s sayhi method attributes also we can use Super().

In Python, super() built-in has some major use cases:

 Allows us to avoid using base class explicitly

 For single inheritance using to refer parent classes
 Working with Multiple Inheritance, in multiple inheritance its very useful in during
dynamic execution.

Super() can be used in new-style classes only. In python 2 class has to be inherit from object
to make use of super().

For python3 the syntax is like below:

super().method_name(args)
super(subClass, instance).method_name(args) (this can be used in both python 2 and 3.)

In case of single inheritance, it allows us to refer base class by super().

class Mammal(object):
def __init__(self, mammalName):
print(mammalName, 'is a warm-blooded animal.')
class Dog(Mammal):
def __init__(self):
print('Dog has four legs.')
super().__init__('Dog')
d1 = Dog()
o/p: Dog has four legs.
o/p: Dog is a warm-blooded animal.
The super() builtin returns a proxy object, a substitute object that has ability to call method of
the base class via delegation. This is called indirection (ability to reference base object with
super())

Since the indirection is computed at the runtime, we can use point to different base class at
different time (if we need to).

class Animal:
def __init__(self, animalName):
print(animalName, 'is an animal.')
class Mammal(Animal):
def __init__(self, mammalName):
print(mammalName, 'is a warm-blooded animal.')
super().__init__(mammalName)
class NonWingedMammal(Mammal):
def __init__(self, NonWingedMammalName):
print(NonWingedMammalName, "can't fly.")
super().__init__(NonWingedMammalName)
class NonMarineMammal(Mammal):
def __init__(self, NonMarineMammalName):
print(NonMarineMammalName, "can't swim.")
super().__init__(NonMarineMammalName)
class Dog(NonMarineMammal, NonWingedMammal):
def __init__(self):
print('Dog has 4 legs.');
super().__init__('Dog')
d = Dog()
print('')
bat = NonMarineMammal('Bat')

Dog has 4 legs.

Dog can't swim.
Dog can't fly.
Dog is a warm-blooded animal.
Dog is an animal.
Bat can't swim.
Bat is a warm-blooded animal.
Bat is an animal.

Method Resolution Order (MRO):

In Python, a class can inherit features and attributes from multiple classes and thus,
implements multiple inheritance. MRO or Method Resolution Order is the hierarchy in which
base classes are searched when looking for a method in the parent class.

It's the order in which method should be inherited in the presence of multiple inheritance.
You can view the MRO by using __mro__ attribute.

MRO of a class can be viewed using class_name.mro() or class_name.__mro__, and it works

only on new style classes.
Class_name.mro() returns a searching order in list, whereas class_name.__mro__ returns in
tuple.

Dog.__mro__

(<class 'Dog'>, <class 'NonMarineMammal'>, <class 'NonWingedMammal'>, <class

'Mammal'>, <class 'Animal'>, <class 'object'>)

Here is how MRO is calculated in Python:

 A method in the derived calls is always called before the method of the base class.
In our example, Dog class is called before NonMarineMammal or NoneWingedMammal.
These two classes are called before Mammal which is called before Animal,
and Animal class is called before object.
 If there are multiple parents like Dog(NonMarineMammal, NonWingedMammal),
method of NonMarineMammal is invoked first because it appears first.

O = Object
class F(O): pass
class E(O): pass
class D(O): pass
class C(D,F): pass
class B(D,E): pass

class A(B,C): pass

print(A.__mro__)

(__main__.A, __main__.B, __main__.C, __main__.D, __main__.E, __main__.F,

object)

Old style classes use DLR or depth-first left to right algorithm whereas new style classes use C3
Linearization algorithm for method resolution while doing multiple inheritance.

Encapsulation: Hiding Information

Encapsulation is a mechanism that restricts direct access to objects’ data and methods. But
at the same time, it facilitates operation on that data (objects’ methods).

“Encapsulation can be used to hide data members and members function. Under this
definition, encapsulation means that the internal representation of an object is
generally hidden from view outside of the object’s definition.” 

All internal representation of an object is hidden from the outside. Only the object can
interact with its internal data.

Access Modifiers:
Before you can take advantage of encapsulation, you have to understand how Python
restricts access to the data stored in variables and methods.

Python has different levels of restriction that control how data can be accessed and from
where. Variables and methods can be public, private, or protected. Those designations are
made by the number of underscores before the variable or method.

Public:

Every variable and method that we define inside a class are by default public. Public
variables and methods can be freely modified and run from anywhere, either inside or
outside of the class. To create a public variable or method, don't use any underscores.

Private:

The private designation only allows a variable or method to be accessed from within its own
class or object. You cannot modify the value of a private variable from outside of a class.
Private variables and methods are preceded by two underscores.

Python does not have the private keyword, unlike some other object oriented languages, but
encapsulation can be done. Instead, it relies on the convention: a class variable that should
not directly be accessed should be prefixed with an underscore.

class Dreamwin:
def __init__(self):
self.a = 10 # public variable
self._b = 20 # protected variable
self.__c = 30 # private variable
obj = Dreamwin()
print(obj.a)  10
print(obj._b)  20
print(obj.__c)
Traceback (most recent call last):
File "test.py", line 10, in <module>
print(obj.__c)
AttributeError: 'Dreamwin' object has no attribute '__c'

A single underscore: Protected variable, it should not be accessed directly. But nothing
stops you from doing that (except convention).
A double underscore: Private variable, harder to access but still possible.
Both are still accessible: Python has private variables by convention.

Getters and setters:

The interfaces that are used for interacting with encapsulated variables are generally
referred to as setter and getter methods because they are used to set and retrieve the
values of variables. Because methods exist within a class or object, they are able to access
and modify private variables, while you will not be able to do so from outside the class.

Private variables are intended to be changed using getter and setter methods. These
provide indirect access to them:
class Dreamwin:
def __init__(self):
self.__version = 22
def getVersion(self):
print(self.__version)
def setVersion(self, version):
self.__version = version
obj = Dreamwin()
obj.getVersion()  22
obj.setVersion(23)
obj.getVersion()  23
print(obj.__version)
Traceback (most recent call last):
AttributeError: 'Dreamwin' object has no attribute '__version'

Nothing in Python is truly private; internally, the names of private methods and attributes
are mangled and unmangled on the fly to make them seem inaccessible by their given
names. These can be accessed using _classname__variable or method.

# A Python program to demonstrate that hidden members can be accessed outside a

class

class MyClass:
# Hidden member of MyClass
__hidden_var = 10
# hidden method
def __hello(self):
print('Dreamwin')
obj = MyClass()
# Accessing private variable outside a class using class name as prefix
print(obj._MyClass__hidden_var)  10
print(myObject._MyClass__hiddenVariable)  Dreamwin

Abstraction or Data Hiding:

Data abstraction refers to the process of providing only the essential information to the
outside world or users and hiding their background details. Data Abstraction or data hiding
will only represent the important information in the program that user needs to know without
presenting the details behind it.

At a higher level, Abstraction is a process of hiding the implementation details and showing
only functionality to the user. It only indicates important things to the user and hides the
internal details, ie. While sending SMS, you just type the text and send the message. Here,
you do not care about the internal processing of the message delivery.

Abstract Class:
A class which is declared “abstract” is called as an abstract class. It can have abstract
methods as well as concrete methods. A normal class cannot have abstract methods.

Abstract Method:
A method without a body is known as an Abstract Method. It must be declared in an abstract
class.

Rules of Abstract Method:

 Abstract methods do not have an implementation; it only has method signature.

 Abstract classes can contain abstract methods: methods without an implementation.
 If a class is using an abstract method they must be declared abstract. The opposite
cannot be true. This means that an abstract class does not necessarily have an
abstract method.
 Objects cannot be created from an abstract class. A subclass can implement an
abstract class. If a regular class inherits an abstract class, then that class must
implement all the abstract methods of the abstract parent.

Advantages of Abstraction:

 The main benefit of using an abstract class is that it allows you to group several
related classes as siblings.
 Abstraction helps to reduce the complexity of the design and implementation process
of software.

When to use Abstract Methods & Abstract Class:

Abstract methods are mostly declared where two or more subclasses are also doing the
same thing in different ways through different implementations. It also extends the same
Abstract class and offers different implementations of the abstract methods.

Abstract classes help to describe generic types of behaviours and object-oriented

programming class hierarchy. It also describes subclasses to offer implementation details of
the abstract class.

Abstraction can be achieved using Abstract Base Class (abc) module in Python.

Syntax:

from abc import abstractmethod, ABCMeta

Python2:

class MyClass(object):
__metaclass__ = ABCMeta
For Python 3+ the right class declaration is the following:
class MyClass(metaclass=ABCMeta):
pass
Starting from Python 3.4 we can just subclass the ABC class:
from abc import ABC
class MyClass(ABC):
pass

Create an abstract class: AbstractAnimal. In the abstract class we only define the methods
without an implementation.
You can then create concrete classes: classes containing an implementation. Let’s create a
class Duck which implements the AbstractAnimal. We use the same methods, but now add
an implementation.

import abc
class AbstractAnimal(metaclass=ABCMeta):

@abstractmethod
def walk(self):
pass

@abstractmethod
def talk(self):
pass
class Duck(AbstractAnimal):
def __init__(self, name):
print('duck created.')
self.name = name
def walk(self):
print('walks')
def talk(self):
print('quack')
obj = Duck('duck1')
obj.talk()  quack
obj.walk()  walks

If a class is defined as Abstract class and it has abstract methods, then we can’t instantiate
(create an object) that class. Abstract Base Classes (ABCs) ensure that derived classes
implement particular methods from the base class at instantiation time.

from abc import ABCMeta, abstractmethod

class Animal(metaclass=ABCMeta):

@abstractmethod
def say_something(self): pass
class Cat(Animal):
def say_something(self):
return "Miauuu!"
a = Animal()

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Animal with abstract methods
say_something

If we forget to implement one of the abstract methods, Python will throw an error.

from abc import ABCMeta, abstractmethod

class Base(metaclass=ABCMeta):
@abstractmethod
def foo(self):
pass
@abstractmethod
def bar(self):
pass

class Concrete(Base):
def foo(self):
pass
# We forget to declare bar() again...
c = Concrete()
TypeError: "Can't instantiate abstract class Concrete with abstract methods bar"
Decorators:
Everything is an object in python, even functions. A function can be assigned to a variable,
passed to another function and can be returned from another function.

First Class Objects

In Python, functions are first-class objects. This means that functions can be passed
around, and used as arguments, just like any other value (e.g, string, int, float).

def foo(bar):
return bar + 1
print(foo)
print(foo(2))
print(type(foo))
def call_foo_with_arg(foo, arg): # foo is passed as an argument to call_foo_with
return foo(arg)
print(call_foo_with_arg(foo, 3))

Nested Functions:

Because of the first-class nature of functions in Python, you can define functions inside
other functions. Such functions are called nested functions.

def parent():
print("Printing from the parent() function.")

def first_child():
return "Printing from the first_child() function."
def second_child():
return "Printing from the second_child() function."
print(first_child())
print(second_child())
parent()
Printing from the parent() function.
Printing from the first_child() function.
Printing from the second_child() function
First_child()

Traceback (most recent call last):

File "decorator03.py", line 15, in <module>
first_child()
NameError: name 'first_child' is not defined
Whenever you call parent(), the sibling functions, first_child() and second_child() are also
called AND because of scope, both of the sibling functions are not available (e.g., cannot be
called) outside of the parent function.
Python provides many powerful tools for writing clean, dynamic code. One of these tools is
the ability to write function decorators, which allow you to dynamically change the behaviour
or extend the functionality of existing functions without changing the function body.
Python decorator are the function that receive a function as an argument and return another
function as return value.
Functions can also create other functions — this is how we create closures.
Closures allow new functions to be created that rely on runtime variables in the program.
The example below should help clear this up:

def outer(word):
def inner():
print(word)
return inner

ab = outer('Welcome to DreamWin')
ab()  'Welcome to Dreamwin'

Decorators as Closures:

Decorators also use closures. They create new functions that “close over” the original
function.

Functions and methods are called callable as they can be called. In fact, any object which
implements the special method __call__() is termed callable. So, in the most basic sense, a
decorator is a callable that returns a callable.

Basically, a decorator takes in a function, adds some functionality and returns it.

def decorator(wrapped):

def inner():
print("I got decorated")
wrapped()

return inner
def ordinary(a):
print("I am an ordinary function")
ordinary()  I am an ordinary function

# let's decorate this ordinary function

pretty = decorator(ordinary)
pretty()
I got decorated
I am an ordinary function
decorator creates a function inner, which is a closure. The inner function in this case is very
simple — it just calls the wrapped function with all of the arguments it receives.

In the example shown above, decorator() is a decorator. In the assignment step.

pretty = decorator(ordinary)

The function ordinary() got decorated and the returned function was given the name pretty.

We can see that the decorator function added some new functionality to the original function.
This is similar to packing a gift. The decorator acts as a wrapper. The nature of the object
that got decorated (actual gift inside) does not alter. But now, it looks pretty (since it got
decorated).

Generally, we decorate a function and reassign it as,

ordinary = decorator(ordinary).

This is a common construct and for this reason, Python has a syntax to simplify this.
We can use the @ symbol along with the name of the decorator function and place it above
the definition of the function to be decorated. For example,

@decorator
def ordinary():
print("I am ordinary")

is equivalent to

def ordinary():
print("I am ordinary")
ordinary = decorator(ordinary)

This is just a syntactic sugar to implement decorators.

Decorators can be of two types.

Function Based Decorators

Class Based Decorators

Function Decorator:

def smart_divide(func):
def inner(a, b):
print("I am going to divide",a,"and",b)
if b == 0:
print("Whoops! cannot divide")
return
return func(a, b)
return inner
@smart_divide
def divide(a, b):
return a/b
divide(2, 5)
I am going to divide 2 and 5
0.4
divide(2, 0)
I am going to divide 2 and 0
Whoops! cannot divide

Function Decorator with Arguments:

def decorator(arg1, arg2):

def real_decorator(function):
def wrapper(*args, **kwargs):
print("Congratulations. You decorated a function that does something
with %s and %s" % (arg1, arg2))
function(*args, **kwargs)
return wrapper
return real_decorator
@decorator("arg1", "arg2") # decorator("arg1", "arg2")(print_args)(1, 2, 3)
def print_args(*args):
for arg in args:
print(arg)
print_args(1, 2, 3)
Congratulations. You decorated a function that does something with arg1 and arg2
1
2
3

def speak(word='moo'):
def decorator(func):
def decorated(*args, **kwargs):
print(word)
return func(*args, **kwargs)
return decorated
return decorator
@speak('quack')
def i_am_a(kind):
print("I am a {kind}".format(kind=kind))
i_am_a("duck")
quack
I am a duck

Class Decorator:

class ClassBasedDecorator:
def __init__(self, func_to_decorate):
print("INIT ClassBasedDecorator")
 self.func_to_decorate = func_to_decorate

def __call__(self, *args, **kwargs):

print("CALL ClassBasedDecorator")
return self.func_to_decorate(*args, **kwargs)
@ClassBasedDecorator
def print_more_args(*args):
for arg in args:
print(arg)
print_more_args(1, 2, 3)
print_more_args(1, 2, 3)
INIT ClassBasedDecorator
CALL ClassBasedDecorator
1
2
3
CALL ClassBasedDecorator
1
2
3

Class decorator with Arguments:

If we wanted to add arguments to the decorator the structure of the class changes, you’ll
note that the function to decorate is now a parameter with the call method.

class ClassBasedDecoratorWithParams:

def __init__(self, arg1, arg2):

print("INIT ClassBasedDecoratorWithParams")
print(arg1)
print(arg2)
def __call__(self, fn, *args, **kwargs):
print("CALL ClassBasedDecoratorWithParams")
def new_func(*args, **kwargs):
print("Function has been decorated. Congratulations.")
return fn(*args, **kwargs)
return new_func
@ClassBasedDecoratorWithParams("Welcome", "Python")
def print_args_again(*args):
for arg in args:
print arg
print_args_again(1, 2, 3)
print_args_again(4, 5, 6)

INIT ClassBasedDecoratorWithParams
Welcome
Python
CALL ClassBasedDecoratorWithParams
Function has been decorated. Congratulations.
1
2
3
Function has been decorated. Congratulations.
4
5
6

You can also see now that __call__ is only called once.

Method Decorator
Method decorators allow overriding class properties by decorating, without having to find the
calling function.

def method_decorator(method):
def inner(city_instance):
if city_instance.name == "SFO":
print("Its a cool place to live in.")
else:
method(city_instance)
return inner

class City(object):
def __init__(self, name):
self.name = name

@method_decorator
def print_test(self):
print(self.name)

obj = City("SFO")
obj.print_test()  Its a cool place to live in.

In the snippet shown above, we decorate the class method print_test. The
method_decorator prints the name of the city, if the name of city instance is not SFO.
Python Logging:
Logging is a means of tracking events that happen when some software runs. The
software’s developer adds logging calls to their code to indicate that certain events have
occurred. An event is described by a descriptive message which can optionally contain
variable data (i.e. data that is potentially different for each occurrence of the event). Events
also have an importance which the developer ascribes to the event; the importance can also
be called the level or severity.

print is not a good idea

Although logging is important, not all developers know how to use them correctly. I saw
some developers insert print statements when developing and remove those statements
when it is finished. It may looks like this

Print('Start reading database')

records = model.read_recrods()
print('# records', records)
print('Updating record ...')
model.update_records(records)
print('done')

It works when the program is a simple script, but for complex systems, you better not to use
this approach.

Logging provides a set of convenience functions for simple logging usage. These
are debug(), info(), warning(), error() and critical(). To determine when to use logging, see
the table below, which states, for each of a set of common tasks, the best tool to use for it.

Task you want to perform The best tool for the task
Display console output for ordinary usage
print()
of a command line script or program
Report events that occur during normal
logging.info() (or logging.debug() for very
operation of a program (e.g. for status
detailed output for diagnostic purposes)
monitoring or fault investigation)
warnings.warn() in library code if the issue
is avoidable and the client application
Issue a warning regarding a particular should be modified to eliminate the warning
runtime event logging.warning() if there is nothing the
client application can do about the
situation, but the event should still be noted
Report an error regarding a particular
Raise an exception
runtime event
Report suppression of an error without logging.error(), logging.exception() or loggi
raising an exception (e.g. error handler in a ng.critical() as appropriate for the specific
long-running server process) error and application domain
The logging functions are named after the level or severity of the events they are used to
track. The standard levels and their applicability are described below (in increasing order of
severity):
Level When it’s used
DEBUG Detailed information, typically of interest only when diagnosing problems.
INFO Confirmation that things are working as expected.
An indication that something unexpected happened, or indicative of some
WARNING problem in the near future (e.g. ‘disk space low’). The software is still
working as expected.
Due to a more serious problem, the software has not been able to perform
ERROR
some function.
A serious error, indicating that the program itself may be unable to continue
CRITICAL
running.

So, how do you do logging correctly? It’s easy, use the standard Python logging module.
Thanks to Python community, logging is a standard module, it was well designed to be easy-
to-use and very flexible. You can use the logging system like this

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.info('Start reading database')
# read database here
records = {'john': 55, 'tom': 66}
logger.debug('Records: %s', records)
logger.info('Updating records ...')
# update records here
logger.info('Finish updating records')

INFO:__main__:Start reading database

INFO:__main__:Updating records ...
INFO:__main__:Finish updating records

logging.basicConfig(level=logging.DEBUG)

INFO:__main__:Start reading database

DEBUG:__main__:Records: {'john': 55, 'tom': 66}
INFO:__main__:Updating records ...
INFO:__main__:Finish updating records

Benefits over print:

 You can control message level and filter out not important ones
 You can decide where and how to output later.

Logger:

The logging library takes a modular approach and offers several categories of components:
loggers, handlers, filters, and formatters.
 1. Loggers expose the interface that application code directly uses.
2. Handlers send the log records (created by loggers) to the appropriate destination.
3. Filters provide a finer grained facility for determining which log records to output.
4. Formatters specify the layout of log records in the final output.
Log event information is passed between loggers, handlers, filters and formatters in
a LogRecord instance.

Logging is performed by calling methods on instances of the Logger class (hereafter

called loggers). Each instance has a name, and they are conceptually arranged in a
namespace hierarchy using dots (periods) as separators.
For example, a logger named ‘scan’ is the parent of loggers ‘scan.text’, ‘scan.html’ and
‘scan.pdf’. Logger names can be anything you want, and indicate the area of an application
in which a logged message originates.
A good convention to use when naming loggers is to use a module-level logger, in each
module which uses logging, named as follows:

import logging
log = logging.getLogger(__name__)
def do_something():
log.debug("Doing something!")

That is all there is to it. In Python, __name__ contains the full name of the current module,
so this will simply work in any module.

Create a module-level logger instance:

logger = logging.getLogger(__name__)

Handlers:

Handlers emit the log records into any output. They take log records and handle them in the
function of what they were built for.

As an example, a FileHandler will take a log record and append it to a file.

The standard logging module already comes with multiple built-in handlers like:

1. Multiple file handlers (TimeRotated, SizeRotated, Watched) that can write to

files.
2. StreamHandler can target a stream like stdout or stderr.
3. SMTPHandler sends log records via email
4. SocketHandler sends LogRecords to a streaming socket
5. SyslogHandler, NTEventHandler, HTTPHandler, MemoryHandler, and
others.

import logging

# create the logging instance for logging to file only

logger = logging.getLogger('SmartfileTest')

# create the handler for the main logger

file_logger = logging.FileHandler('test.log')
# finally, add the handler to the base logger
logger.addHandler(file_logger)

# log some stuff!

logger.debug("This is a debug message!")
logger.info("This is an info message!")
logger.warning("This is a warning message!")

# now we can add the console logging.

console = logging.StreamHandler()
console.setLevel(logging.INFO)
logger.addHandler(console)

This is an info message!

This is a warning message!

The above code will log the log records into both file and console.

Formatter:

Formatters take a LogRecord, and do a msg % args type thing.

The basic one is something like:

"%(asctime)s %(name)-19s %(levelname)-5s - %(message)s"

...which, while it doesn't display everything that can be present in a LogRecord, serves most
immediate needs.
You may wish to think about parseablility, for example using an unusual divider
The members of LogRecord are mostly interesting when writing custom handlers.

A LogRecord's most interesting members:

name - logger name

levelno - numeric level
module – module name
levelname - name of level
pathname - absolute pathname of the source file
lineno - line number in that file with the log call
msg - message (format string)
args - args from logging call. A tuple that combines with msg.

Developer can create his own members while formatting the log message. To achieve this
we have to provide a key-word argument extra to the log record.

FORMAT = "%(asctime)-15s %(clientip)s %(user)-8s %(message)s"

logging.basicConfig(format=FORMAT)
d = {'clientip':'192.168.0.1','user':'dreamwin'}
logging.warning("Protocol problem: %s", "connection reset", extra=d)

In the above code, clientip and user are the custom members for formatter. Below is the
message format.

2018-02-08 22:20:02,165 192.168.0.1 dreamwin Protocol problem: connection reset

Logging in an Application
There are at least three ways to configure a logger:
Using code:
Pro: complete control over the configuration.
Con: modifications require a change to source code.
Using an INI-formatted file:
Pro: possible to update configuration while running using the function
logging.config.listen() to listen on a socket.
Con: less control (e.g. custom subclassed filters or loggers) than possible when
configuring a logger in code.
Using a dictionary or a JSON-formatted file:
Pro: in addition to updating while running, it is possible to load from a file using the
json module.
Con: less control than when configuring a logger in code.
Configuration directly in code:

import logging
logger = logging.getLogger()
handler = logging.StreamHandler()
formatter = logging.Formatter(
'%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)
logger.debug('often makes a very good meal of %s', 'visiting tourists')

Configuration via an INI file.

Let us say the file is named logging_config.ini.

[loggers]
keys=root

[handlers]
keys=stream_handler

[formatters]
keys=formatter
[logger_root]
level=DEBUG
handlers=stream_handler
[handler_stream_handler]
class=StreamHandler
level=DEBUG
formatter=formatter
args=(sys.stderr,)

[formatter_formatter]
format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s
Then use logging.config.fileConfig() in the code:

import logging
from logging.config import fileConfig
fileConfig('logging_config.ini')
logger = logging.getLogger()
logger.debug('often makes a very good meal of %s', 'visiting tourists')

Configuration via Dictionary:

import logging
from logging.config import dictConfig

logging_config = dict(
version = 1,
formatters = {
'f': {'format':
'%(asctime)s %(name)-12s %(levelname)-8s %(message)s'}
},

handlers = {
'h': {'class': 'logging.StreamHandler',
'formatter': 'f',
'level': logging.DEBUG}
},

root = {
'handlers': ['h'],
'level': logging.DEBUG,
},
)
dictConfig(logging_config)
logger = logging.getLogger()
logger.debug('often makes a very good meal of %s', 'visiting tourists')
Regular Expressions:
Regular Expressions (sometimes shortened to regexp, regex, or re) are a tool for matching
patterns in text. It is extremely useful for extracting information from text such as code, files,
log, spreadsheets or even documents.
While using the regular expression the first thing is to recognize is that everything is
essentially a character, and we are writing patterns to match a specific sequence of
characters also referred as string. Ascii or latin letters are those that are on your keyboards
and Unicode is used to match the foreign text. It includes digits and punctuation and all
special characters like $#@!%, etc.

For instance, a regular expression could tell a program to search for specific text from the
string and then to print out the result accordingly. Expression can include

 Text matching
 Repetition
 Branching
 Pattern-composition etc.

Regular expressions comes with a set of meta characters for pattern matching. Below are
the list of meta characters. Everything in regex is a character. Even . is also a character

^ Matches the beginning of the string.

$ Matches the end of the string.

. Matches any character except newline.
? Matches zero or one occurrence.
* Any number of occurrences (including 0 occurrences).
+ One or more occurrences.
| Means OR (Matches with any of the characters separated by it.)

{} Indicate number of occurrences of a preceding RE to match.

{x} - Repeat exactly x number of times.

{x,} - Repeat at least x times or more.
{x, y} - Repeat at least x times but no more than y times.
() Enclose a group of Res
[] Represent a character class.
[abc] - Matches a or b or c.
[a-zA-Z0-9] – Matches alpha-numeric characters.
[^...] Matches any single character not in brackets
\ Used to drop the special meaning of character following it (discussed below)
Special sequences in regular expressions:

Element Description
. This element matches any character except \n
\d This matches any digit [0-9]
\D This matches non-digit characters [^0-9]
\s This matches whitespace character [ \t\n\r\f\v]
\S This matches non-whitespace character [^ \t\n\r\f\v]
\w This matches alphanumeric character [a-zA-Z0-9_]

\W This matches any non-alphanumeric character [^a-zA-Z0-9]

Python has a built-in library called re for dealing with regular expressions. re module provides
a handy methods to perform pattern matching.

Based on the regular expressions, Python offers two different primitive operations. The
match method checks for a match only at the beginning of the string while search checks for
a match anywhere in the string.
Search:
search(pattern, string, flags=0)
With this function, you scan through the given string/sequence looking for the first location
where the regular expression produces a match. It returns a corresponding match object if
found, else returns None if no position in the string matches the pattern.

import re
pattern = "cookie"
sequence = "Cake and cookie"
data = re.search(pattern, sequence)  <_sre.SRE_Match object; span=(9, 15),
match='cookie'>
print(data.group())  'cookie'

Match:
match(pattern, string, flags=0)
Returns a corresponding match object if zero or more characters at the beginning of string
match the pattern. Else it returns None, if the string does not match the given pattern.

import re
pattern = "C"
sequence1 = "IceCream"

# No match since "C" is not at the start of "IceCream"

re.match(pattern, sequence1)

sequence2 = "Cake"
re.match(pattern, sequence2).group()  'C'
search() versus match()
The match() function checks for a match only at the beginning of the string (by default)
whereas the search() function checks for a match anywhere in the string.
Compile:
compile(pattern, flags=0)
Compiles a regular expression pattern into a regular expression object. When you need to
use an expression several times in a single program, using the compile() function to save the
resulting regular expression object for reuse is more efficient. This is because the compiled
versions of the most recent patterns passed to compile() and the module-level matching
functions are cached.

import re
pattern = re.compile(r"cookie")
sequence = "Cake and cookie"
pattern.search(sequence).group()  'cookie'

# This is equivalent to:

re.search(pattern, sequence).group()  'cookie'

Group Extraction:
The "group" feature of a regular expression allows you to pick out parts of the matching text.
Suppose for the emails problem that we want to extract the username and host separately.
To do this, add parenthesis ( ) around the username and host in the pattern, like this: r'([\w.-
]+)@([\w.-]+)'. In this case, the parenthesis do not change what the pattern will match,
instead they establish logical "groups" inside of the match text. On a successful search,
match.group(1) is the match text corresponding to the 1st left parenthesis, and
match.group(2) is the text corresponding to the 2nd left parenthesis. The plain match.group()
is still the whole match text as usual.

str = 'purple alice-b@google.com monkey dishwasher'

match = re.search('([\w.-]+)@([\w.-]+)', str)
if match:
print(match.group())  'alice-b@google.com' (the whole match)
print(match.groups())  ('alice-b', 'google.com')
print(match.group(1))  'alice-b' (the username, group 1)
print(match.group(2))  'google.com' (the host, group 2)

Findall:
findall(pattern, string, flags=0)
Finds all the possible matches in the entire sequence and returns them as a list of strings.
Each returned string represents one match.

import re
email_address = "Please contact us at: support@google.com, xyz@google.com
hello@yahoo.co.in"

#'addresses' is a list that stores all the possible match

addresses = re.findall(r'[\w\.-]+@[\w\.-]+', email_address)
o/p: ['support@google.com', 'xyz@google.com', 'hello@yahoo.co.in']
# \d is equivalent to [0-9].
p = re.compile('\d')
print(p.findall("I went to him at 11 A.M. on 4th July 1886"))
# \d+ will match a group on [0-9], group of one or greater size
p = re.compile('\d+')
print(p.findall("I went to him at 11 A.M. on 4th July 1886"))
Output:
['1', '1', '4', '1', '8', '8', '6']
['11', '4', '1886']

# '*' replaces the no. of occurrence of a character.

p = re.compile('ab*')
print(p.findall("ababbaabbb"))

o/p: ['ab', 'abb', 'a', 'abbb']

Split:
re.split(pattern, string, maxsplit=0, flags=0)
The First parameter, pattern denotes the regular expression, string is the given string in
which pattern will be searched for and in which splitting occurs, maxsplit if not provided is
considered to be zero ‘0’, and if any nonzero value is provided, then at most that many splits
occurs.
If maxsplit = 1, then the string will split once only, resulting in a list of length 2. The flags are
very useful and can help to shorten code, they are not necessary parameters, eg: flags =
re.IGNORECASE, In this split, case will be ignored.

text = """John Doe

Jane Doe
Jin Du
Chin Doe"""

import re
results = re.split(r"\n+", text)
print(results)  ['Jane Doe', 'Jane Doe', 'Jin Du', 'Chin Doe']

# splitting using multiple delimiters.

a = "Hello, welcome to! python"
re.split(", |!", a)  ['Hello', 'welcome to', ' python']
re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)  ['0', '3', '9']

# Splitting will occurs only once, at '12', returned list will have length 2
print(re.split('\d+', 'On 12th Jan 2016, at 11:02 AM', 1))
o/p: ['On ', 'th Jan 2016, at 11:02 AM']
# 'Boy' and 'boy' will be treated same when flags = re.IGNORECASE
print(re.split('[a-f]+', 'Aey, Boy oh boy, come here', flags = re.IGNORECASE))
o/p: ['', 'y, ', 'oy oh ', 'oy, ', 'om', ' h', 'r', '']

print(re.split('[a-f]+', 'Aey, Boy oh boy, come here'))

o/p: ['A', 'y, Boy oh ', 'oy, ', 'om', ' h', 'r', '']
Sub:
sub(pattern, repl, string, count=0, flags=0)
This is the substitute function. It returns the string obtained by replacing or substituting the
leftmost non-overlapping occurrences of pattern in string by the replacement repl. If the
pattern is not found then the string is returned unchanged.

import re

pattern = re.compile(r"[0-9]+")
result = pattern.sub("_", "there is only 1 thing 2 do")
print(result)  there is only _ thing _ do

email_address = "Please contact us at: xyz@googlcom"

new_email_address = re.sub(r'([\w\.-]+)@([\w\.-]+)', r'support@google.com',
email_address)
print(new_email_address)  Please contact us at: support@datacamp.com

subn:
subn(pattern, repl, string, count=0, flags=0)
subn() is similar to sub() in all ways, except in its way to providing output. It returns a tuple
with count of total of replacement and the new string rather than just the string.

import re
print(re.subn('ub', 'ab' , 'Subject has Uber booked already'))
o/p: ('Sabject has Uber booked already', 1)
t = re.subn('ub', 'ab' , 'Subject has Uber booked already', flags = re.IGNORECASE)
print(t)  ('Sabject has aber booked already', 2)
print(len(t))  2
# This will give same output as sub() would have
print(t[0])  'Sabject has Uber booked already'

Escape:
re.escape(string)
Return string with all non-alphanumerics backslashed, this is useful if you want to match an
arbitrary literal string that may have regular expression metacharacters in it.

import re
# escape() returns a string with BackSlash '\', before every Non-Alphanumeric
Character
# In 1st case only ' ', is not alphanumeric
# In 2nd case, ' ', caret '^', '-', '[]', '\' are not alphanumeric
print(re.escape("This is Awseome even 1 AM"))
o/p: This\ is\ Awseome\ even\ 1\ AM
print(re.escape("I Asked what is this [a-9], he said \t ^WoW"))
o/p: I\ Asked\ what\ is\ this\ \[a\-9\]\,\ he\ said\ \ \ \^WoW
Regex Flags:

Syntax long syntax meaning

re.I re.IGNORECASE ignore case.
re.M re.MULTILINE make begin/end {^, $} consider each line.
re.S re.DOTALL make . match newline too.
re.U re.UNICODE make {\w, \W, \b, \B} follow Unicode rules.
re.L re.LOCALE make {\w, \W, \b, \B} follow locale.
re.X re.VERBOSE allow comment in regex.

To specify more than one of them, use | operator to connect them. For example,
re.search(pattern,string,flags=re.IGNORECASE|re.MULTILINE|re.UNICODE).
re.IGNORECASE or re.I: Indicates case-insensitive matching.
re.MULTILINE or re.M:
When specified, the pattern character ^ match the beginning of the string and the beginning
of each line (immediately following each newline); and the pattern character $ match at the
end of the string and at the end of each line (immediately preceding each newline).
Normally, ^ and $ only match at the beginning/end of the string.

# example of regex flag re.MULTILINE

import re

ss = """abc
def
ghi"""
r1 = re.findall(r"^\w", ss)  ['a']
r2 = re.findall(r"^\w", ss, flags = re.MULTILINE)  ['a', 'd', 'g']

re.DOTALL or re.S:
Make the dot character . match any character, including a newline. Without this flag, a dot
will match anything except a newline.

# example of regex flag re.DOTALL

import re
ss = """once upon a time,
there lived a king"""
r1 = re.findall(r".+", ss) # ['once upon a time,', 'there lived a king']
r2 = re.findall(r".+", ss, re.DOTALL)  ['once upon a time,\nthere lived a
king']

re.UNICODE or re.U:
Make the pattern characters {\w, \W, \b, \B} dependent on the Unicode character properties
database. For Example:

import re
x1 = re.search(r"\w+", u"♥αβγ!", re.U)
x2 = re.search(r"\w+", u"♥αβγ!")
if x1:
print(x1.group().encode("utf8"))  αβγ
else:
 print("no match")
print(x2)  None

Note that Unicode string can be in the pattern string. Just be sure to use the Unicode prefix u
to the pattern string.
re.LOCALE or re.L:
Make the word pattern {\w, \W} and boundary pattern {\b, \B}, dependent on the current
locale.

re.VERBOSE or re.X:
This flag changes the regex syntax, to allow you to add annotations in regex. Whitespace
within the pattern is ignored, except when in a character class or preceded by an unescaped
backslash, and, when a line contains a # neither in a character class or preceded by an
unescaped backslash, all characters from the leftmost such # through the end of the line are
ignored.

# example of the regex re.VERBOSE flag

import re

# matching a decimal number

p1 = re.compile(r"""\d + # the integral part
\. # the decimal point
\d * # some fractional digits""", re.X)
p2 = re.compile(r"\d+\.\d*") # pattern p2 is same as p1
r1 = re.findall(p1, u"a3.45")
r2 = re.findall(p2, u"a3.45")
print r1[0].encode("utf8")  3.45
print r2[0].encode("utf8")  3.45

Regex Examples:

ruby?  Match "rub" or "ruby": the y is optional

ruby*  Match "rub" plus 0 or more ys
ruby+  Match "rub" plus 1 or more ys
\d{3}  Match exactly 3 digits
\d{3,}  Match 3 or more digits
\d{3,5}  Match 3, 4, or 5 digits
[Pp]ython  Match "Python" or "python"
rub[ye]  Match "ruby" or "rube"
[aeiou]  Match any one lowercase vowel
[0-9]  Match any digit; same as [0123456789]
[a-z]  Match any lowercase ASCII letter
[A-Z]  Match any uppercase ASCII letter
[a-zA-Z0-9]  Match any of the above
[^aeiou]  Match anything other than a lowercase vowel
[^0-9]  Match anything other than a digit
Database connections using Python:
There are two types of databases currently using for application development.

1. RDBMS
2. NoSQL (Not Only SQL)
RDBMS:
RDBMS Database is a relational database. It is the standard language for relational
database management systems. Data is stored in the form of rows and columns in RDBMS.
The relations among tables are also stored in the form of the table SQL (Structured query
Language) is a programming language used to perform tasks such as update data on a
database, or to retrieve data from a database. Some common relational database
management systems that use SQL are: Oracle, Sybase, Microsoft SQL Server, Access,
etc.

1. SQL databases are table based databases.

2. Data store in rows and columns.
3. Each row contains a unique instance of data for the categories defined by the
columns.
4. Provide facility primary key, to uniquely identify the rows.
5. Well maintained relationship between data tables.
6. It is completely a structured way of storing data.
7. The amount of data stored in RDBMS depends on physical memory of the system or
in other words it is vertically scalable.
8. RDBMS databases are table based databases This means that SQL databases
represent data in form of tables which consists of n number of rows of data
9. RDBMS have predefined schemas.
10. For defining and manipulating the data RDBMS use structured query language i.e.
SQL which is very powerful.
11. ACID properties(Atomicity, Consistency, Isolation, Durability).

RDBMS database examples: MySql, Oracle, Sqlite, Postgres and MS-SQL.

NoSQL:

1. It is completely a unstructured way of storing data.

2. While in Nosql there is no limit you can scale it horizontally.
3. Work on only open source development models.
4. NoSQL databases are document based, key-value pairs, graph databases or
wide-column stores.whereas NoSQL databases are the collection of key-value
pair, documents, graph databases or wide-column stores which do not have
standard schema definitions which it needs to adhere to.
5. NoSql have dynamic schema with the unstructured data.
6. It uses UnQL i.e. unstructured query language and focused on collection of
documents and vary from database to database.
7. Follow Brewers CAP theorem (Consistency, Availability and Partition tolerance).
NoSQL database examples: MongoDB, BigTable, Redis, RavenDb, Cassandra, Hbase,
Neo4j and CouchDb

Python provides packages to interact with both RDBMS and NoSQL. In this tutorial we will
learn about PostgreSQL database and how to interact with postgresql database with python.
PostgreSQL is a powerful, open source object-relational database system. It is a multi-user
database management system. It runs on multiple platforms including Linux, FreeBSD,
Solaris, Microsoft Windows and Mac OS X. PostgreSQL is developed by the PostgreSQL
Global Development Group.
For installing PostgreSQL in windows please follow this link :
http://www.postgresqltutorial.com/install-postgresql/
After installing PostgreSQL db open psql shell from command line.
List all databases using : \l
Connect to database using \c <db_name>.
List all tables using: \dt
create a database.
Create database testdb;

After installing PostgreSQL database install python package also. psycopg2 is the package
to interact with PostgreSQL database using python.
Install package through pip: pip install psycopg2
Connect to Database:

import psycopg2

# creates a connection to database.

conn = psycopg2.connect(database="testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432") # or we can provide host as
localhost also.

Create a table:

import psycopg2

conn = psycopg2.connect(database = "testdb", user = "postgres", password =

"pass123", host = "127.0.0.1", port = "5432")
cur = conn.cursor()
cur.execute('''CREATE TABLE COMPANY
(ID INT PRIMARY KEY NOT NULL,
NAME TEXT NOT NULL,
AGE INT NOT NULL,
ADDRESS CHAR(50),
SALARY REAL);''')

conn.commit()
conn.close()
Insert data into created table:

import psycopg2
conn = psycopg2.connect(database = "testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432")
cur = conn.cursor()
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \
VALUES (1, 'Paul', 32, 'California', 20000.00 )");

cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \

VALUES (2, 'Allen', 25, 'Texas', 15000.00 )");
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \
VALUES (3, 'Teddy', 23, 'Norway', 20000.00 )");
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \
VALUES (4, 'Mark', 25, 'Rich-Mond ', 65000.00 )");
conn.commit()
conn.close()

Insert multiple records at once:

import psycopg2
import sys
cars = ((1, 'Audi', 52642),(2, 'Mercedes', 57127),(3, 'Skoda', 9000),
(4, 'Volvo', 29000), (5, 'Bentley', 350000), (6, 'Citroen', 21000),
(7, 'Hummer', 41400), (8, 'Volkswagen', 21600))

try:
con = psycopg2.connect(database = "testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432")
cur = con.cursor()
cur.execute("DROP TABLE IF EXISTS Cars")
cur.execute("CREATE TABLE Cars(Id INT PRIMARY KEY, Name TEXT, Price INT)")
query = "INSERT INTO Cars (Id, Name, Price) VALUES (%s, %s, %s)"

cur.executemany(query, cars)
con.commit()

except psycopg2.DatabaseError, e:

if con:
con.rollback()

print 'Error %s' % e

sys.exit(1)

finally:

if con:
con.close()

select records from table:

import psycopg2
conn = psycopg2.connect(database = "testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432")
cur = conn.cursor()
cur.execute("SELECT id, name, address, salary from COMPANY")
rows = cur.fetchall() # It will fetch all records from the table.
rows = cur.fetchone() # It will fetch one record from the table.
conn.close()

Update records from the table:

import psycopg2
conn = psycopg2.connect(database = "testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432")

cur = conn.cursor()
cur.execute("UPDATE COMPANY set SALARY = 25000.00 where ID = 1")
conn.commit()
cur.execute("SELECT id, name, address, salary from COMPANY")
rows = cur.fetchall()
conn.close()

Delete rows from the table:

import psycopg2
conn = psycopg2.connect(database = "testdb", user = "postgres", password =
"pass123", host = "127.0.0.1", port = "5432")

cur = conn.cursor()
cur.execute("DELETE from COMPANY where ID=2;")
conn.commit()
cur.execute("SELECT id, name, address, salary from COMPANY")
rows = cur.fetchall()
conn.close()
Multi-Threading and Multi-Processing:
Before moving into multi-threading and multi-processing we should know about threads and
processes.

Process:
A process (sometimes called a heavyweight process) is a program in execution. Each
process has its own address space, memory, a data stack, and other auxiliary data to keep
track of execution. The operating system manages the execution of all processes on the
system, dividing the time fairly between all processes. Processes can
also fork or spawn new processes to perform other tasks, but each new process has its own
memory, data stack, etc., and cannot generally share information unless inter process
communication (IPC) is employed.
1. Created by the operating system to run programs.
2. Processes can have multiple threads
3. Two processes can execute code simultaneously in the same python program
4. Processes have more overhead than threads as opening and closing processes
takes more time
5. Sharing information between processes is slower than sharing between threads
as processes do not share memory space. In python they share information by
pickling data structures like arrays which requires IO time.

Threads:

Threads (sometimes called lightweight processes) are similar to processes except that they
all execute within the same process, and thus all share the same context. They can be
thought of as “mini-processes” running in parallel within a main process or “main thread.”

1. Threads are like mini-processes that live inside a process

2. They share memory space and efficiently read and write to the same variables
3. Two threads cannot execute code simultaneously in the same python program
(although there are workarounds*)
When we talk about implementing threads and processes, what we are really trying to
achieve is concurrency and/or parallelism.
Concurrency and parallelism are distinct concepts. Concurrency is concerned with managing
access to shared state from different threads, whereas parallelism is concerned with utilizing
multiple processors/cores to improve the performance of a computation.

When to use threads vs processes?

 Processes speed up Python operations that are CPU intensive because they benefit
from multiple cores and avoid the GIL.

 Threads are best for IO tasks or tasks involving external systems because threads can
combine their work more efficiently. Processes need to pickle their results to combine
them which takes time.

 Threads provide no benefit in python for CPU intensive tasks because of the GIL.
Note that the threads in Python work best with I/O operations, such as downloading
resources from the Internet or reading files and directories on your computer. If you
need to do something that will be CPU intensive, then you will want to look at
Python’s multiprocessing module instead. The reason for this is that Python has the
Global Interpreter Lock (GIL) that basically makes all threads run inside of one master
thread. Because of this, when you go to run multiple CPU intensive operations with
threads, you may find that it actually runs slower.

Multithreading:

In multithreading, multiple lightweight processes called threads are created which share the
same memory pool so precautions have to be taken, or two threads will write the same
memory at the same time. Multiple threads live in the same process in the same space, each
thread will do a specific task, have its own code, own stack memory, instruction pointer, and
share heap memory. If a thread has a memory leak it can damage the other threads and
parent process.
In Python, for multithreading, there is an inbuilt package ‘threading’. By creating objects of
Thread class of threading package, threads can be easily created.

# Python program to illustrate the concept of threading importing the threading

module

import threading

def print_cube(num):
""" function to print cube of given num """
print("Cube: {}".format(num * num * num))

def print_square(num):
""" function to print square of given num """
print("Square: {}".format(num * num))

if __name__ == "__main__":
# creating threads
t1 = threading.Thread(target=print_square, args=(10,))
t2 = threading.Thread(target=print_cube, args=(10,))

# starting thread 1
t1.start()
# starting thread 2
t2.start()

# wait until thread 1 is completely executed

t1.join()
# wait until thread 2 is completely executed
t2.join()

# both threads completely executed

print("Done!")

o/p:
Square: 100
Cube: 1000
Done!
The below code describes to format a thread using thread name.
# Python program to illustrate the concept of threading.

import threading
import os

def task1():
print("Task 1 assigned to thread: {}".format(threading.current_thread().name))
print("ID of process running task 1: {}".format(os.getpid()))

def task2():
print("Task 2 assigned to thread: {}".format(threading.current_thread().name))
print("ID of process running task 2: {}".format(os.getpid()))

if __name__ == "__main__":

# print ID of current process

print("ID of process running main program: {}".format(os.getpid()))

# print name of main thread

print("Main thread name: {}".format(threading.main_thread().name))

# creating threads
t1 = threading.Thread(target=task1, name='t1')
t2 = threading.Thread(target=task2, name='t2')

# starting threads
t1.start()
t2.start()

# wait until all threads finish

t1.join()
t2.join()

Locks and Synchronization:

When you have more than one thread, then you may find yourself needing to consider how
to avoid conflicts. What I mean by this is that you may have a use case where more than
one thread will need to access the same resource at the same time. If you don’t think about
these issues and plan accordingly, then you will end up with some issues that always
happen at the worst of times and usually in production.

The solution is to use locks. A lock is provided by Python’s threading module and can be
held by either a single thread or no thread at all.

Locks have 2 states: locked and unlocked. 2 methods are used to manipulate them: acquire
() and release (). Those are the rules:

 If the state is unlocked: a call to acquire () changes the state to locked.

 If the state is locked: a call to acquire () blocks until another thread calls release ().
 If the state is unlocked: a call to release () raises a RuntimeError exception.
 If the state is locked: a call to release () changes the state to unlocked ().
import threading

total = 0

def update_total(amount):
"""
Updates the total by the given amount
"""
global total
total += amount
print (total)
if __name__ == '__main__':
for i in range(10):
my_thread = threading.Thread(target=update_total, args=(5,))
my_thread.start()

Regardless, the issue here is that one thread might call update_total and before it’s done
updating it, another thread might call it and attempt to update it too. Depending on the order
of operations, the value might only get added to once.

Let’s add a lock to the function. There are two ways to do this. The first way would be to use
a try/finally as we want to ensure that the lock is always released.

import threading

total = 0
lock = threading.Lock()

def update_total(amount):
"""
Updates the total by the given amount
"""
global total
lock.acquire()
try:
total += amount
finally:
lock.release()
print (total)

if __name__ == '__main__':
for i in range(10):
my_thread = threading.Thread(target=update_total, args=(5,))
my_thread.start()

we no longer need the try/finally as the context manager that is provided by

the with statement does all of that for us.
Inter-Thread Communication:
Threads inside a process can be communicate each other. This communication can be done
using a module called Queue in python.
Queues are a great mechanism when we need to exchange information between threads as
it takes care of locking for us.

We are interested in the following 4 Queue methods:

  put: Put an item to the queue.
 get: Remove and return an item from the queue.
 task_done: Needs to be called each time an item has been processed.
 join: Blocks until all items have been processed.

Python Threading with Queue:

from threading import Thread

import Queue

def worker():
while True:
print("in while")
item = q.get() # remove item from queue
print(item)
print("task finished")

q = Queue.Queue() # defining the Queue

num_worker_threads =2

for i in range(num_worker_threads):
t = Thread(target=worker)
t.start()

for item in range(2):

q.put(item) # put item in queue

# wait till all the threads are done with execution.

for i in range(num_worker_threads):
t.join()

Queue with task_done:

import threading
import queue

def task(a):
q.get()
print("Executing task function")
#time.sleep(2)
print(a)
q.task_done()

q = queue.Queue()
for i in range(5):
t = threading.Thread(target=task, args=(i, ))
t.start()
q.put(t)

# wait till queue becomes empty

q.join()

print("Task Done")
MultiProcessing:
Python Multiprocessing is similar to multithreading, instead of creating threads
multiprocessing create sub-processes. By using subprocess multiprocessing evade the GIL.
(Global Interpreter Lock) It runs on both Unix and Windows.
The multiprocessing module was added to Python in version 2.6. It was originally defined in
PEP 371 The multiprocessing module allows you to spawn processes in much that same
manner than you can spawn threads with the threading module. The idea here is that
because you are now spawning processes, you can avoid the Global Interpreter Lock (GIL)
and take full advantages of multiple processors on a machine.
The multiprocessing package also includes some APIs that are not in the threading module
at all. We will start with the multiprocessing module’s Process class.

Creating a single process:

from multiprocessing import Process

def call_name(name):
print('hello ', name)

if __name__ == '__main__':
p = Process(target=call_name, args=('Dreamwin',))
p.start()

Creating multiple process:

from multiprocessing import Process, current_process

def worker(num):
print('process:', num)
print(current_process().name)
print(current_process().pid)

if __name__ == '__main__':
# creating new processess
p1 = Process(name ='worker1', target=worker, args=(1,))
p2 = Process(name ='service', target=worker, args=(2,))

# start new processess

p1.start()
p2.start()

O/p:
process: 1
worker1
12345
Process: 2
Service
67890

Multiprocessing Queue:

A simple way to pass the message between processes is to use a Queue. Unlike
Multithreading the Multiprocessing use own Queue.
from multiprocessing import Process, Queue
import time
import random

def producer(Q):
while True:
num = random.randint(1, 100)
time.sleep(1)
Q.put(num)

def consumer(Q):
while True:
time.sleep(1)
print Q.get()

if __name__ == '__main__':
Q = Queue(5)
p1 = Process(name ='worker1', target=producer,args=(Q,))
p2 = Process(name ='service', target=consumer,args=(Q,))
print "start"
p1.start()
p2.start()
Python Unit-testing:
Unit testing is a software testing method in which individual components of the program,
called units, are tested independently with all the required dependencies. Unit testing is
mostly done by the actual programmers, who write the programs for the units. In smaller
projects, it is done informally. In most of the very large-scale projects, unit testing is part of a
formal process of development with proper documentation and proper schedule/ efforts
allocated to it.

Test Automation:

Test automation is the automated execution and reporting of the outcome of test scenarios
and cases. In most large and complex projects, many phases of the testing process are
automated. Sometimes the effort of automating the tests is so huge that there is a separate
project for automation with a separate team dedicated to it, including a separate reporting
structure with separate management. There are several areas and phases of testing that can
be automated. Various tools like code libraries and third-party APIs are used for unit testing.
Sometimes, the code for unit testing is also generated in an automated way. Unit testing is a
prime candidate for automation.

The Benefits of Automated Unit Testing:

There are many reasons to automate unit tests. Let’s consider them one by one.

Time and effort:

As your codebase grows, the number of modules to be unit tested grows. Manual
testing occupies a lot of days of the typical programmer’s calendar. To reduce
manual testing efforts, you can automate test cases, which then can be automated
easily and quickly.

Accuracy:

Test case execution is a rote and boring activity. Humans can make mistakes. However, an
automated test suite will run and return correct results every time.

Early bug reporting:

Automating unit test cases gives you the distinct advantage of early reporting of bugs and
errors. When the automated test suites are run by the scheduler, once the code freezes due
to an error, all the logical bugs in the code are quickly discovered and reported, without
much human intervention needed.

Built-in support for unit testing:

There are many programming languages that provide built-in support for writing unit tests by
means of libraries dedicated to unit testing. Examples include Python, Java, and PHP.

Unittest is the batteries-included test module in the Python standard library. Its API will be
familiar to anyone who has used any of the JUnit/nUnit/CppUnit series of tools.
Creating test cases is accomplished by sub classing unittest.TestCase.
Python unittest module is used to test a unit of source code. Suppose, you need to test your
project. You know what kind of data the function will return. After writing huge code, you
need to check it whether the output is correct or not.

Normally, what we do is printing the output and match it with the reference output file or
check the output manually.

Python Unit Test Outcome & Basic Functions:

This unittest has 3 possible outcomes. They are mentioned below:

OK: If all test cases are passed, the output shows OK.
Failure: If any of test cases failed and raised an Assertion Error exception
Error: If any exception other than Assertion Error exception is raised.

There are several function under unittest module. They are listed below.

METHOD CHECKS THAT

assertEqual(a,b) a==b
assertNotEqual(a,b) a != b
assertTrue(x) bool(x) is True
assertFalse(x) bool(x) is False
assertIs(a,b) a is b
assertIsNot(a, b) a is not b
assertIsNone(x) x is None
assertIsNotNone(x) x is not None
assertIn(a, b) a in b
assertNotIn(a, b) a not in b
assertIsInstance(a, b) isinstance(a, b)
assertNotIsInstance(a, b) not isinstance(a, b)

import unittest
def fun(x):
return x + 1
class MyTest(unittest.TestCase):
def test(self):
self.assertEqual(fun(3), 4)
# test execution starts from here.
if __name__ == "__main__":
unittest.main()

At the bottom of the test file, we have this code:

if __name__ == '__main__':
unittest.main()

This allows us to run all of the test code just by running the file.

import unittest
class TestClass01(unittest.TestCase):
 def test_case01(self):
my_str = "DreamWin"
my_int = 999
self.assertTrue(isinstance(my_str, str))
self.assertTrue(isinstance(my_int, int))
def test_case02(self):
my_pi = 3.14
self.assertFalse(isinstance(my_pi, int))

if __name__ == '__main__':
unittest.main()

Running in normal mode:

python3 test_module01.py
It yields the following output:
..
----------------------------------------------------------------------
Ran 2 tests in 0.002s
OK

Running in verbose mode:

python3 test_module01.py -v
The verbose output is as follows:
test_case01 (__main__.TestClass01) ... ok
test_case02 (__main__.TestClass01) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.004s
OK

Running Without unittest.main():

Up until now, you have run the test modules with unittest.main(). Now you will see
how to run the test module without unittest.main().

import unittest
class TestClass07(unittest.TestCase):

def test_case01(self):
self.assertTrue("PYTHON".isupper())
print("\nIn test_case01()")
# Running a testcase without unittest.main()
python -m unittest test_module06 -v
The verbose output is as follows:
test_case01 (test_module06.TestClass07) ...
In test_case01()
ok----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
Version Control System:
About Version Control
What is “version control”, and why should you care? Version control is a system that records
changes to a file or set of files over time so that you can recall specific versions later. For the
examples in this book, you will use software source code as the files being version
controlled, though in reality you can do this with nearly any type of file on a computer.
If you are a graphic or web designer and want to keep every version of an image or layout
(which you would most certainly want to), a Version Control System (VCS) is a very wise
thing to use. It allows you to revert selected files back to a previous state, revert the entire
project back to a previous state, compare changes over time, see who last modified
something that might be causing a problem, who introduced an issue and when, and more.
Using a VCS also generally means that if you screw things up or lose files, you can easily
recover. In addition, you get all this for very little overhead.

Local Version Control Systems

Many people’s version-control method of choice is to copy files into another directory
(perhaps a time-stamped directory, if they’re clever). This approach is very common
because it is so simple, but it is also incredibly error prone. It is easy to forget which directory
you’re in and accidentally write to the wrong file or copy over files you don’t mean to.
To deal with this issue, programmers long ago developed local VCSs that had a simple
database that kept all the changes to files under revision control.

One of the more popular VCS tools was a system called RCS, which is still distributed with
many computers today. RCS works by keeping patch sets (that is, the differences between
files) in a special format on disk; it can then re-create what any file looked like at any point in
time by adding up all the patches.

Centralized Version Control Systems:

The next major issue that people encounter is that they need to collaborate with developers
on other systems. To deal with this problem, Centralized Version Control Systems (CVCSs)
were developed. These systems, such as CVS, Subversion, and Perforce, have a single
server that contains all the versioned files, and a number of clients that check out files from
that central place. For many years, this has been the standard for version control.

This setup offers many advantages, especially over local VCSs. For example, everyone
knows to a certain degree what everyone else on the project is doing. Administrators have
fine-grained control over who can do what, and it’s far easier to administer a CVCS than it is
to deal with local databases on every client.
However, this setup also has some serious downsides. The most obvious is the single point
of failure that the centralized server represents. If that server goes down for an hour, then
during that hour nobody can collaborate at all or save versioned changes to anything they’re
working on. If the hard disk the central database is on becomes corrupted, and proper
backups haven’t been kept, you lose absolutely everything — the entire history of the project
except whatever single snapshots people happen to have on their local machines. Local
VCS systems suffer from this same problem — whenever you have the entire history of the
project in a single place, you risk losing everything.

Distributed Version Control Systems:

This is where Distributed Version Control Systems (DVCSs) step in. In a DVCS (such as Git,
Mercurial, Bazaar or Darcs), clients don’t just check out the latest snapshot of the files;
rather, they fully mirror the repository, including its full history. Thus, if any server dies, and
these systems were collaborating via that server, any of the client repositories can be copied
back up to the server to restore it. Every clone is really a full backup of all the data.

GIT and GITHub:

Git is a Version Control System developed by Linus Torvalds, sound familiar? Yes, you got
that right, the Father of the Linux Operating System. The Linux kernel is still maintained by
him.
Consider the Linux Kernel Project.

It has over 15 million lines of code.

About 3,500 new lines added every day.
Whenever a new kernel is released, about 1,000 developers are involved in the process.
Git was originally designed to help manage the Linux Kernel and make collaboration easy
from the beginning. If Git can effectively manage a project as large as the Linux Kernel, it
can manage your projects easily and effectively.
Furthermore, the architecture of Git is distributed Version Control as opposed to a
centralized, network access VCS. A centralized VCS requires a network connection to work
with and a central failure may result in all your work being destroyed. A distributed VCS such
as Git, does not require a network connection to interact with the repository. Each developer
has their own repository which makes it fast and easy to collaborate with.
Finally, on to GitHub. GitHub is easily the most popular website for sharing your projects with
collaborators or with the whole world. It's like a social network for your
projects. Repositories can be made public so that anyone can submit a commit and help
make your project better.
Git uses checksums to secure your data. This makes it impossible to make changes to the
data without Git getting a whiff of it. This functionality is built into Git at the lowest levels and
is integral to its philosophy. The basic idea is that you can't lose information in transit or have
files corrupted without Git being able to detect it. Git> uses the SHA-1 hash system which as
you may probably know, is a 40- character string composed of hex characters and is
calculated based on the content. Git extensively uses these values and stores files in
database by this hash and not by the name.
GitHub has been gaining popularity in the software development world and it is likely that
you would require to know Git if you are to be a collaborator on a project which is a reason
on its own to learn more about Git.
Basic and Common GIT Commands:

Getting & Creating Projects

Command Description

git init Initialize a local Git repository

git clone
ssh://git@github.com/[username]/[repo Create a local copy of a remote repository
sitory-name].git

Basic Snapshotting

Command Description

git status Check status

git add [file-name.txt] Add a file to the staging area

Add all new and changed files to the staging

git add -A
area

git commit -m "[commit message]" Commit changes

git rm -r [file-name.txt] Remove a file (or folder)

Branching & Merging:

Command Description

List branches (the asterisk denotes the current

git branch
branch)

git branch -a List all branches (local and remote)

git branch [branch name] Create a new branch

git branch -d [branch name] Delete a branch

git push origin --delete [branchName] Delete a remote branch

git checkout -b [branch name] Create a new branch and switch to it

git checkout -b [branch name]
origin/[branch name] Clone a remote branch and switch to it
 Command Description

git checkout [branch name] Switch to a branch

git checkout - Switch to the branch last checked out

git checkout -- [file-name.txt] Discard changes to a file

git merge [branch name] Merge a branch into the active branch
git merge [source branch] [target
branch] Merge a branch into a target branch

git stash Stash changes in a dirty working directory

git stash clear Remove all stashed entries

Sharing & Updating Projects

Command Description

git push origin [branch name] Push a branch to your remote repository

Push changes to remote repository (and

git push -u origin [branch name]
remember the branch)

Push changes to remote repository

git push
(remembered branch)
git push origin --delete [branch
name] Delete a remote branch

git pull Update local repository to the newest commit

git pull origin [branch name] Pull changes from remote repository

git remote add origin

ssh://git@github.com/[username]/[repo Add a remote repository
sitory-name].git

git remote set-url origin

ssh://git@github.com/[username]/[repo Set a repository's origin branch to SSH
sitory-name].git

Inspection & Comparison

 Command Description

git log View changes

git log --summary View changes (detailed)

git diff [source branch] [target
branch} Preview changes before merging