Python Notes 2020

Course contents of Python Programming Certification Course
Section Heading
1 Core Python Concepts
2 Automating (Daily Business) task using Python
3 Advanced Python Concepts
After this python course, I will be able to :

1. Code and Understand Python projects.
2. Understand any advanced level course on Machine Learning or
Deep Learning i.e. Neural Networks (using Python).
3. Understand and Perform Data Analytics to Explore data ,
Visualize data and make Predictions.
4. Automate your routine business task.
Python Programming(Core+ Advanced) By Rocky Sir 1
Section 1 Index
Core Python Concepts
Module 1 Overview of Python
Module 2 Data-types & Operators in Python
Module 3 Build in functions of Python Data-types
Module 4 Functions , Modules & Packages
Module 5 Reading/Writing from files
Module 6 Assertions & Exceptions

Section 2 Index
Automating (Daily Business) task
using Python
Module 7 Regex and its daily applications

Module 8 Web scrapping
Module 9 Reading & Writing : PDFs and Word Documents
Module 10 Reading excel sheet and doing data analysis
Module 11 CSV and JSON data Parsing
Module 12 Scheduling task and launching programs
Section 3 Index
Advanced Python Concepts
Module 13 Classes & Object Programming
Module 14 Inheritance, Overloading and Overriding
Module 15 Database programming
Module 16 Multithread Programming
Module 17 Introduction to Functional Programming(FP)
Internship A> Phone Number and Email Address Extractor
Project
Specification B> Combining Select Pages from Many PDFs
Module 1 : Overview of Python
 Python is a general-purpose interpreted, interactive, object-oriented, and high-
level programming language.
 It was created by Guido van Rossum

during 1985- 1990. Like Perl, Python
source code is also available under the
GNU General Public License (GPL).
 Python is named after a TV Show called

‘Monty Python’s Flying Circus’ and not
after Python-the snake.
Installing Step 1: Download the Python 3 Installer.
Python Step 2: Run the Installer.
For Windows
Step 1: Download the Python 3 Installer
 Open a browser window and navigate to the Download page for Windows at python.org.
 Underneath the heading at the top that says Python Releases for Windows, click on the link
for the Latest Python 3 Release - Python 3.x.x. (Prefer Python 3.6)
 Scroll to the bottom and select either Windows x86-64 executable installer for 64-bit
or Windows x86 executable installer for 32-bit.
Step 2: Run the Installer

 Once you have chosen and downloaded an installer, simply run it by double-clicking on the
downloaded file. A dialog should appear that looks something like this:
Important:
You want to be sure to
check the box that
says Add Python 3.x to
PATH as shown to ensure
that the interpreter will
be placed in your
execution path.
Then just click Install Now. That should be all there is to it. A few minutes later you should have
a working Python 3 installation on your system.
Installation steps
For any version https://realpython.com/installing-python/
of Linux or Mac
Check Python working or not ?
1> After Python installation on Windows, check for path either :
C:\Python36
or
C:\Users\(Your logged in User)\AppData\Local\Programs\Python\Python36
In the 2nd case, Cut the Python36 folder and place it in directly under Program files
C:\Program Files\Python36
2> Go to cmd prompt, change path to C:\Program Files\Python36 & type
> python
// this would open python in interactive mode
>>>
Type Some code, like : Interactive Mode Programming
>>> print("hi all")
// see the o/p. This confirms the python interpreter is correctly installed.
Now to come out of this interactive shell, type
>>> quit()
3> Now, create folder say "suven" where we would dump all our .py code files.
4> Now open your text editor say "Notepadd++". [We won't be using Jupyter notebooks or
anaconda or any other IDE for the time being.] Type the following code :
print("Hello World!")
print("Hello Again")
save your file as sctpl.py. In the folder “suven”
5> run sctpl.py like this : Script Mode Programming

> python suven/sctpl.py
Note :
• As on today, One of the best (and only) full-featured, dedicated IDEs for Python is PyCharm.
Website: https://www.jetbrains.com/pycharm/
• Although Jupyter notebooks are more popular among Python for Data Science Developers.

Check out chapter_1s.py file
We would 1. print() statement

learn : 2. how to give comments -> #
3. what's the use """ or '''
4. declaring variables of different types
5. two ways to format String -> {} and .format()
 print(f"I said: {x}")
 print(joke_evaluation.format(hilarious))
6. different types of escape sequences
7. order of MATH operations -> PEMDAS
8. use input() to get input
9. use cmd line arguments i.e. argv to get input
10. define and call functions
Note : In python, be careful of indentation error.

Quick Read theory topics
Python Identifiers
A Python identifier is a name used to identify a variable, function, class, module or other object.
An identifier starts with a letter A to Z or a to z or an underscore (_) followed by zero or more
letters, underscores and digits (0 to 9).
Python does not allow punctuation characters such as @, $, and % within identifiers. Python is a
case sensitive programming language. Thus, Suven and suven are two different identifiers in
Python.
Lines and Indentation

Python does not use braces({}) to indicate blocks of code for class and function definitions or
flow control. Blocks of code are denoted by line indentation, which is rigidly enforced.
The number of spaces in the indentation is variable, but all statements within the block must be
indented the same amount. For example −
if 1==2:
print("1 and 2 are same")
else:
print("1 and 2 are different")
Multi-Line Statements
Statements in Python typically end with a new line. Python, however, allows the use of the line
continuation character (\) to denote that the line should continue. For example −
total = var_one + \
var_two + \
var_three
Quotation in Python
Python accepts single ('), double (") and triple (''' or """) quotes to denote string literals, as long
as the same type of quote starts and ends the string.
The triple quotes are used to span the string across multiple lines. For example, all the following
are legal −
word = 'word'
sentence = "This is a sentence."
paragraph = """This is a paragraph. It is
made up of multiple lines and sentences."""

Comments in Python
A hash sign (#) that is not inside a string literal is the beginning of a comment. All characters
after the #, up to the end of the physical line, are part of the comment and the Python
interpreter ignores them.
# First comment
print ("Hello, Suven!") # second comment
'''
Following triple-quoted string is also ignored by Python This is a
interpreter and can be used as a multiline comments: multiline
comment.
'''
Multiple Statements on a Single Line

The semicolon ( ; ) allows multiple statements on a single line given that no statement starts a
new code block.
print("; acts as a separator"); print("I am second")

Getting input from the user : input()
The following line of the program displays the prompt asking input from the user.
print("How old are you?", end=' ')
age = input()
Getting input from the user : via command-line arguments

The Python sys module provides access to any command-line arguments via the sys.argv.
This serves two purposes −
 sys.argv is the list of command-line arguments.
 len(sys.argv) is the number of command-line arguments.
Here sys.argv[0] is the program i.e. the script name.

import sys
print('Number of arguments:', len(sys.argv), 'arguments.')
print('Argument List:', str(sys.argv))

Order of evaluation for Math Operations
PEMDAS which stands for Parentheses Exponents Multiplication Division Addition Subtraction.
The actual order is you do the multiplication and division (M&D) in one step, from left to right,
then you do the addition and subtraction in one step from left to right. So, you could rewrite
PEMDAS as PE(M&D)(A&S).
For example-
print("Now I will count the eggs:")
print(3 + 2 + 1 - 5 + 4 % 2 - 1 / 4 + 6)

Reserved Words
The following list shows the Python keywords. These are reserved words and you cannot use
them as constants or variables or any other identifier names. All the Python keywords contain
lowercase letters only.

Module 2 : Data-types & Operators in Python
Variables are nothing but reserved memory locations to store values. It means that when you
create a variable, you reserve some space in the memory.
Based on the data type of a variable, the interpreter allocates memory and decides what can be
stored in the reserved memory.
Python has six standard data types −

1. Numbers
2. String
3. List  [ … ]
4. Tuple  ( … )
5. Set  { … }
6. Dictionary  { key : Value pairs }

1. Python Numbers : (long)int, float, complex no.
We would
2. Python String
learn :
3. Python Lists -> [123, 'Technology']
4. Python Tuples -> (123, 'Technology')
5. Python Dictionary : holds key:value pair
6. Types of Operators
a) Arithmetic Operators
b) Comparison (Relational) Operators
c) Assignment Operators
d) Logical Operators
e) Bitwise Operators
f) Membership Operators
g) Identity Operators
7. Decision making
8. Loops
9. Iterator & Generator
10. Mini Project : web scrapping tool
11. Mini Project : print a simplified hotel bill
Python Numbers
Number data types store numeric values. Number objects are created when you assign a value
to them. For example −
var1 = 1
var2 = 10
You can also delete the reference to a number object by using the del statement. The syntax of
the del statement is −
del var1[,var2[,var3[....,varN]]]]
You can delete a single object or multiple objects by using the del statement.
For example −
del var
del var_a, var_b var1 = 1 #long-int
var2 = 10.5 #float
Python supports three different numerical types −
var3 = 10+5j #complex no.
 int (signed integers)
 float (floating point real values) print(var1)
 complex (complex numbers) print(var2)
print(var3)
#All integers in Python3 are represented as
long integers. Hence, there is no separate
number type as long.
Python Strings
Strings in Python are identified as a contiguous set of characters represented in the quotation
marks. Python allows either pair of single or double quotes. Subsets of strings can be taken
using the slice operator ([ ] and [:] ) with indexes starting at 0 to (last-1).
The plus (+) sign is the string concatenation operator and the asterisk (*) is the repetition
operator. For example −
str = 'Python@Suven Consultants'
print(str)
print(str[0])
#[m:n] called slice operator
print(str[2:7])
print(str[2:])
print(str[:8])
print(str * 2)
print(str + " Technology")

Python Lists
Lists are the most versatile of Python's compound data types. A list contains items separated by
commas and enclosed within square brackets ([]). To some extent, lists are similar to arrays in C.
One of the differences between them is that all the items belonging to a list can be of different
data type.
The values stored in a list can be accessed using the slice operator ([ ] and [:]) with indexes
starting at 0 in the beginning of the list and working their way to end -1. The plus (+) sign is the
list concatenation operator, and the asterisk (*) is the repetition operator. For example −
list = [ 'scikit', 123 , 2.23, 'suven', 72+3j ]
tinylist = [123, 'Technology']
print(list)
print(list[0])
print(list[1:3])
print(list[2:])

Python Tuples
A tuple is another sequence data type that is similar to the list.
The main difference between lists and tuples are − Lists are enclosed in brackets ( [ ] ) and their
elements and size can be changed, while tuples are enclosed in parentheses ( ( ) ) and cannot be
updated. Tuples can be thought of as read-only lists. For example −
tuple = ('scikit', 123 , 2.23, 'suven', 72+3j)
tinytuple = (123, 'Technology')
print(tuple)
print(tuple[0])
print(tuple[1:3])
print(tuple[2:])

Python Dictionary
Python's dictionaries are kind of hash-table type. They work like associative arrays or hashes
found in Perl and consist of key-value pairs. A dictionary key can be almost any Python type, but
are usually numbers or strings. Values, on the other hand, can be any arbitrary Python object.
Dictionaries are enclosed by curly braces ({ }) and values can be assigned and accessed using
square braces ([]). For example −
dict = {}
dict['campus'] = "IIT-B"
dict[400071]= "Chembur"
tinydict={'name':'suven','code':1234,'dept':'IT sales'}
print(dict['campus'])
print(dict[400071])
print(tinydict) #elements of dict are not ordered
print(tinydict.keys())
print(tinydict.values())

Python Set
 Set is a collection which is unordered and un-indexed.
 No duplicate members.
 Elements are enclosed in { ... }
 Cannot update elements. Can only add elements
thisset = {"Suven", "Rocky", "Lokesh", "ML", "Neural Networks", "DL"}

print(thisset)

Types of Operators
1. Arithmetic Operators
2. Comparison (Relational) Operators
3. Assignment Operators
4. Logical Operators
5. Bitwise Operators
6. Membership Operators
7. Identity Operators
1. Arithmetic
Operators

2. Comparison (Relational) Operators

3. Assignment Operators

4. Logical Operators

5. Bitwise Operators

6. Python Membership Operators
7. Python Identity Operators

Decision-making
Decision-making is the anticipation of conditions occurring during the execution of a program
and specified actions taken according to the conditions.
Python programming language assumes any non-zero and non-null values as TRUE, and
any zero or null values as FALSE value. Python programming language provides the following
types of decision-making statements.

people = 30
cars = 40
if cars > people:

print("We should take the cars.")
elif cars < people:
print("We should not take the cars.")
else:
print("We can't decide.")

Loops
In general, statements are executed sequentially − The first statement in a function is executed
first, followed by the second, and so on. There may be a situation when you need to execute a
block of code several number of times.
A loop statement allows us to execute a statement or group of statements multiple times.

the_count = [1, 2, 3, 4, 5]
fruits = ['apples', 'oranges', 'pears', 'apricots']
change = [1, 'rupees', 2, 'dollars', 3, 'euros']
for number in the_count:

print(f"This is count {number}")
for fruit in fruits:

print(f"A fruit of type: {fruit}")
for i in change:
print(f"I got {i}")

Lets code a simple mail scrapping tool
Web scraping is the term for using a program to download and process content from the
Web. For example, Google runs many web scraping programs to index web pages for its search
engine. We will code web scrapper later in the course.
Similarly we have a simple scraping tool , which can scrap (i.e. find all required details) from
some text file or string.
We will now code a simple mail scrapping tool to get all mail-ids from some string data.
import re
s = """
some text, lets say comments
of linked-in or fb post containing mail-ids
"""
match = re.findall(r'[\w\.-]+@[\w\.-]+', s)
print('\n\n')
for email in match:
print(email)
Loop Control Statements
The Loop control statements change the execution from its normal sequence. When the
execution leaves a scope, all automatic objects that were created in that scope are destroyed.
Python supports the following control statements.

Iterator and Generator
Iterator is an object which allows a programmer to traverse through all the elements of a
collection, regardless of its specific implementation. In Python, an iterator object implements
two methods, iter() and next().
String, List or Tuple objects can be used to create an Iterator.

import sys
list = [1,2,3,4]
it = iter(list) #this builds an iterator object
print(next(it)) #prints next available element in iterator
for x in it:
print(x, end=" ")

Generator
A generator is a function that produces or yields a sequence of values using yield method.
When a generator function is called, it returns a generator object without even beginning
execution of the function. When the next() method is called for the first time, the function starts
executing until it reaches the yield statement, which returns the yielded value. The yield keeps
track i.e. remembers the last execution and the second next() call continues from previous
value. import sys
def fibonacci(n): #generator function
Example
a, b, counter = 0, 1, 0
The following example
while True:
defines a generator, which
if(counter > n):
generates an iterator for all
return
the Fibonacci numbers.
yield a
a, b = b, a + b
counter += 1
f = fibonacci(5) #f is iterator object
while True:
try:
print(next(f), end=" ")
except StopIteration:
sys.exit()
Coding Exercise : 1
Loop through and print out all even numbers from the numbers list in the same order they are
received. Don't print any numbers that come after 237 in the sequence.
numbers = [ 951, 402, 984, 651, 360, 69, 408, 319, 601, 485, 980, 507, 725, 547, 544, 615, 83,
165, 141, 501, 263,617, 865, 575, 219, 390, 984, 592, 236, 105, 942, 941, 386, 462, 47, 418, 907,
344, 236, 375, 823, 566, 597, 978,328, 615, 953, 345, 399, 162, 758, 219, 918, 237, 412, 566,
826, 248, 866, 950, 626, 949, 687, 217, 815, 67, 104, 58, 512, 24, 892, 894, 767, 553, 81, 379,
843, 831, 445, 742, 717, 958, 609, 842, 451, 688, 753, 854, 685, 93, 857, 440, 380, 126, 721,
328, 753, 470, 743, 527 ]

Coding Exercise : 2
Code a script to Print a simplified hotel bill.
You are been given with following rates

service_tax = .05 #tax rate for A/c restaurants
Adult_Meal_Veg = 399 #buffet lunch cost-veg
Adult_Meal_non_veg = 499 #buffet lunch cost-non-veg
Child_Meal_Cost = 199 #for child below 12 years
limited_hard_drinks = 450 #per adult cost at the bar
and your script should ask for following inputs:

a> Customer Name
b> Number_Of_Adults_having_Veg_food
c> Number_Of_Adults_having_non_veg
d> Number_Of_Children
e> Number_Of_Adults_going_to_the_Bar

Module 3 : Build in functions of Python Data-types
We would 1. Number build-in functions

learn : 2. String build-in functions
3. List build-in functions
4. Tuple build-in functions
5. zip() and use of zip()
6. Properties of Dictionary Keys
7. dictionary build-in functions
8. Difference between shallow and deep copy
9. Date & time functions

Random Number Functions
Random numbers are used for games, simulations, testing, security, and privacy applications.
Python includes the following functions that are commonly used.

String Functions
Python includes the following built-in methods to manipulate strings −

String Functions… contd..

String
Functions…
contd..

String
Functions…
contd..

String
Functions…
contd..

String
Functions…
contd..

String
Functions…
contd..

List Functions
1 len(list) -> Gives the total length of the list.
2 max(list) -> Returns item from the list with max value.
3 min(list) -> Returns item from the list with min value.
4 list(seq) -> Converts a tuple into list.
Python also includes the following list methods −

more
list methods
.. Contd..

Tuple Functions
1 len(tuple) -> Gives the total length of the tuple.
2 max(tuple) -> Returns item from the tuple with max value.
3 min(tuple) -> Returns item from the tuple with min value.
4 tuple(seq) -> Converts a list into tuple.

Dictionary Methods
Python includes the following dictionary methods −

zip() and use of zip()
The purpose of zip() is to map the similar index of multiple containers so that they can
be used as a single entity.
name = [ "Akshay", "Neha", "Snehar", "Vipin" ]
roll_no = [ 4, 1, 3, 2 ]
marks = [ 40, 50, 60, 70 ]
# using zip() to map values

mapped = zip(name, roll_no, marks)
# converting values to print as set

mapped = set(mapped)
# printing resultant values

print("The zipped result is : ",end="")
print(mapped)

I.Q How do you compare 2 sequences ?
#--application of zip in comparing sequences

#-- 1st way :
a = (100, 0)
b = (99, 0)
print(tuple(i > j for i, j in zip(a,b)))
#-- another way :

#-- all() gives cumulative result
print(all(i > j for i, j in zip(a,b)))

imp Properties of Dictionary Keys
Dictionary values have no restrictions. They can be any arbitrary Python object, either standard
objects or user-defined objects. However, same is not true for the keys.
There are two important points to remember about dictionary keys −
(a) More than one entry per key is not allowed. This means no duplicate key is allowed. When
duplicate keys are encountered during assignment, the last assignment wins. For example −
dictt = {'Name':'SCTPL','Age':15,'Name':'Suven'}
print(dictt)
(b) Keys must be immutable. This means you can use strings, numbers or tuples as dictionary
keys but something like ['key'] is not allowed.
dict = {['Name']: 'Suven', 'Age': 15}
print("dict['Name']: ", dict['Name'])
#Note: keys should be hash-able. [list] are not hashable
imp What’s the Difference between shallow and deep copy ?
1> Shallow copy
a. A shallow copy means constructing a new collection object and then populating it with
references to the child objects found in the original.
b. In essence, a shallow copy is only one level deep. The copying process does not recurse and
therefore won’t create copies of the child objects themselves.
xs = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
ys = list(xs) #Makes a shallow copy. ys refers to same address space
print(ys)
print("--------------------")
xs.append(['new sublist'])
print("xs is : ", xs)
print("ys is : ", ys)
print("--------------------") Shallow copy means
both point to the
xs[1][0] = 'X'
same address space.
print("updated xs is : ", xs)
print("updated ys is : ", ys) Hence updates to xs
print("--------------------") would be seen in ys also.

2> Deep copy
A deep copy makes the copying process recursive. It means first constructing a new collection
object and then recursively populating it with copies of the child objects found in the original.
Copying an object this way walks the whole object tree to create a fully independent clone of
the original object and all of its children.
import copy
xs = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
zs = copy.deepcopy(xs) #all child objects are copied
print("zs is : ", zs)
print("--------------------")
xs[1][0] = 'X'
print("updated xs is : ", xs)
print("zs remains same : ", zs)

Short Coding Questions
1> What's the ans ? Type & Run this code.
a = {}
dict = a.fromkeys(['a', 'b', 'c', 'd'], 98)
print(a)
print(dict)
2> Complete the below code to get list of all keys and key:value pairs.
#-- complete the code , to get following output

3> How do check for a key ? (Hint : use in operator)
#-- complete the code, where we check
#-- Name as well Names is in dictt
4> Update the dictionary using update() ?

dict1 = {'Name':'Suven', 'course':'Eco and Stats'}
dict2 = {'Location':'IITB'}
#-- complete the code here to get following o/p --

5> Remove a key and print its value. use pop()
dict3 = {'Name':'Neha', 'course':'Web Tech'}
#-- complete the code here to get following o/p --
6> Type below script to see the difference of Dictionary Keys vs. List Indices.
d = {0: 'a', 1: 'b', 2: 'c', 3: 'd'}
print(d)
print(d[0]) #here 0 is the key not Index

print(d[2])
#irrespective of the position of the key, value is same

d = {1: 'b', 2: 'c', 3: 'd', 0: 'a'}
print(d[0])
7> Find type of dictionary ? Type and run below code .
d = {0: 'a', 1: 'b', 2: 'c', 3: 'd'}
print(type(d))
8> Type and run below code. Note each error.

d = {0: 'a', 1: 'b', 2: 'c', 3: 'd'}
print(d[-1]) #note the error
#print(d[0:2]) #note the error
#print(d.append('e')) #note the error

Date & Time Data structure
A Python program can handle date and time in several ways. Python's time and calendar
modules help track dates and times.
What is Tick?
 Time intervals are floating-point numbers in units of seconds. Particular instants in time are
expressed in seconds since 12:00am, January 1, 1970(epoch).
 There is a popular time module available in Python which provides functions for working
with times, and for converting between representations.
 The function time.time() returns the current system time in ticks since 12:00am, January 1,
1970(epoch).
import time; #importing time module.

ticks = time.time()
print ("Number of ticks since 12:00am, January 1, 1970:", ticks)

In Python, time is stored as time tuple
import time
print (time.localtime());
#--The above tuple is equivalent to struct_time structure. This structure has following
attributes −

9> Type and run below code. Note two ways of printing current date
and time in time tuple format .
import time
print (time.localtime());
print("---can do like this also---")
localtime = time.localtime(time.time())
print ("Local current time :", localtime)
10> Type and run below code, for Getting formatted time. Use asctime()
import time
localtime = time.asctime(time.localtime(time.time()))
print("Local current time :", localtime)

11> Getting calendar for a month. Type and run it.
import calendar
cal = calendar.month(2019, 3)
print("Here is the calendar:")
print(cal)

Module 4 : Functions , Modules & Packages
We would 1. Syntax of user-defined function

learn : 2. Pass by Reference vs. Pass by Value
3. Function Arguments
a> Required arguments
b> Keyword arguments
c> Default arguments
d> Variable-length arguments
4. use pointer notation(*) : to accept many arguments

5. Anonymous Functions : use lambda keyword not def keyword
6. Scope of Variables
a> Global variables
b> Local variables
We would learn, contd…
7. Concept of a module
8. from...import <name> Statement
9. from...import * Statement
10.Executing Modules as Scripts
11.Locating Modules
12.dir() Function
13.globals() and locals()
14.Packages
15.don't confuse between importing from a package w.r.t importing from a module
16.install packages

Function
A function is a block of organized, reusable code that is used to perform a single, related action.
Functions provide better modularity for your application and a high degree of code reusing.
As you already know, Python gives you many built-in functions like print(), etc. but you can also
create your own functions. These functions are called user-defined functions.
#-syntax of function
def functionname( parameters ):
"function_docstring is optional"
function_suite
return [expression]
def printme(str):
"This prints a passed string into this function"
print(str)
return
printme("hello from Suven")

Pass by Reference vs. Value
All parameters (arguments) in the Python language are passed by reference.
It means if you change what a parameter refers to within a function, the change also reflects
back in the calling function. For example −
def seeChanges( mylist ):
"This changes a passed list into this function"
print("Values before change: ", mylist)
mylist[2]=50
print("Values after change: ", mylist)
return
mylist = [10,20,30]
seeChanges( mylist )
print("Values outside the function: ", mylist)

Function Arguments
You can call a function by using the following types of formal arguments −
a> Required arguments
b> Keyword arguments
c> Default arguments
d> Variable-length arguments
A> Required Arguments

Required arguments are the arguments passed to a function in correct positional order. Here,
the number of arguments in the function call should match exactly with the function definition.
#-- here str is Required argument

def printme(str):
"This prints a passed string into this function"
print(str)
return
printme("hello from Suven")

B> Keyword Arguments
Keyword arguments are related to the function calls. When you use keyword arguments in a
function call, the caller identifies the arguments by the parameter name.
This allows you to skip arguments or place them out of order because the Python
interpreter is able to use the keywords provided to match the values with parameters.
For example-
def printinfo(company, url):

print("Company Name: ", company)
print("URL ", url)
return
#--Now you can call printinfo function

printinfo( url = 'https://suvenconsultants.com', company = "Suven Consultants" )

C> Default Arguments
A default argument is an argument that assumes a default value if a value is not provided in the
function call for that argument. The following example gives an idea on default arguments,
def printinfo(url,company ="SCTPL"):

print("Company Name: ", company)
print("URL ", url)
return
# Now you can call printinfo function

printinfo( url = 'https://suvenconsultants.com')
Rule : Default arguments always at last.

D> Variable-length Arguments
You may need to process a function for more arguments than you specified while defining the
function. These arguments are called variable-length arguments and are not named in the
function definition, unlike required and default arguments.
Syntax for a function with non-keyword variable arguments is given below −

def functionname([formal_args,] *var_args_tuple ):
"function_docstring"
function_suite
return [expression]
def printinfo( arg1, *vartuple ):

print("Output is: ")
print(arg1)
for var in vartuple:

print("--",var)
return
printinfo(10) #no variable arguments

printinfo(70,60,50) #with 2 variable arguments
The Anonymous Functions
These functions are called anonymous because they are not declared in the standard manner by
using the def keyword. You can use the lambda keyword to create small anonymous functions.
Simple rules w.r.t lambda

1. Lambda forms can take any number of arguments but return just one value in the form of an
expression. They cannot contain commands or multiple expressions.
2. An anonymous function cannot be a direct call to print because lambda requires an

expression.
3. Lambda functions have their own local namespace and cannot access variables other than
those in their parameter list and those in the global namespace.
Syntax :
lambda [arg1 [,arg2,.....argn]]:expression
#--code example
sum = lambda arg1, arg2: arg1 + arg2
#--Now you can call sum as a function
print("Value of total : ", sum(12,13))
Scope of Variables
All variables in a program may not be accessible at all locations in that program. This depends
on where you have declared a variable.
The scope of a variable determines the portion of the program where you can access a
particular identifier. There are two basic scopes of variables in Python −
 Global variables
 Local variables
Global vs. Local variables

Variables that are defined inside a function body have a local scope, and those defined
outside have a global scope.
This means that local variables can be accessed only inside the function in which they are
declared, whereas global variables can be accessed throughout the program body by all
functions. When you call a function, the variables declared inside it are brought into scope.

Following is a simple example of Global vs. Local variables
total = 0 # These are global variable.
gtotal = 0
def sum(arg1, arg2):
total = arg1 + arg2; # Here total is local variable.
global gtotal # for using global variables inside fn
gtotal = total
print("Inside the function local total : ", total)
return total
# Now call sum function
sum(10, 20)
print("Outside the function total : ", total)
print("Outside the function gtotal : ", gtotal)
#--local variables can be accessed only inside the function
in which they are declared.

Module
1. A module allows you to logically organize your Python code.
2. Grouping related code into a module makes the code easier to understand and use.
3. A module is a Python object with arbitrarily named attributes that you can bind and
reference.
4. Simply, a module is a file consisting of Python code.
5. A module can define functions, classes and variables. A module can also include runnable
code.
#-put this code into firstModule.py
def print_func(par):
print("Hello : ", par)
return
# Sctpl.py
#import the above module, like this :
import firstModule
firstModule.print_func("EY")

The from...import Statement
from statement lets you import specific attributes from a module into the current namespace.
Syntax is
from modname import name1[, name2[, ... nameN]]
#--importing fib() from fibonacci module

from fibonacci import fib
list = fib(100)
print(list)

The from...import * Statement
It is also possible to import all the names from a module into the current namespace by using
the following import statement −
from modname import *
This provides an easy way to import all the items from a module into the current namespace;
however, this statement should be used sparingly.
#--this would also work

from fibonacci import *
list = fib(100)
print(list)

Executing Modules as Scripts
1. Within a module, the module’s name (as a string) is available as the value of the global
variable __name__.
2. The code in the module will be executed, just as if you imported it, but with the __name__
set to "__main__".
def fib(n):
result = []
a, b = 0, 1
while b < n:
result.append(b)
a, b = b, a + b
return result
if __name__ == "__main__":
f = fib(100)
print(f)

Locating Modules
When you import a module, the Python interpreter searches for the module in the following
sequences −
1> The current directory.
2>If the module not found, Python then searches each directory in the environment variable
PYTHONPATH.
3> If all else fails, Python checks the default path.
import sys
print("default path is ", sys.path)

The dir( ) Function
1. The dir() built-in function returns a sorted list of strings containing the names defined by a
module.
2. The list contains the names of all the modules, variables and functions that are defined in a
module. Following is a simple example −
import math
content = dir(math)
print(content)

The globals() and locals() Functions
1. The globals() and locals() functions can be used to return the names in the global and local
namespaces depending on the location from where they are called.
2. If locals() is called from within a function, it will return all the names that can be accessed
locally from that function.
3. If globals() is called from within a function, it will return all the names that can be accessed
globally from that function.
4. The return type of both these functions is dictionary. Therefore, names can be extracted
using the keys() function.
total = 0 # These are global variable.
gtotal = 0
def test(arg1, arg2):
total = 10
print("locals are ", locals())
print("--------------------------")
print("globals are ", globals())
return total
# Now call sum function
test(10, 20)
Packages in Python
Packages are namespaces which contain multiple packages and modules themselves. They are
simply directories, but with a twist.
Each package in Python is a directory which MUST contain a special file called __init__.py. This
file can be empty, and it indicates that the directory it contains is a Python package, so it can be
imported the same way a module can be imported.
Follow these simple steps to make a package:
1> make a folder recruitment
2> code some .py files in it
3> code a empty __init__.py file and place it in the package
4> 2 ways to : use the package by writing
#-- This is the first way ----
#-- sctpl.py --
from recruitment import interview_prep
from recruitment import first_call
interview_prep.interviewprep()
first_call.firstcall()

#--another way : sctpl.py
#--Note : we must use the package prefix whenever we access the module
import recruitment.interview_prep
import recruitment.first_call
recruitment.interview_prep.interviewprep()
recruitment.first_call.firstcall()
#--be careful don't use * , like this

from recruitment import * #this won't work
#Why ?
because recruitment is a package not a module and *
means load all classes, functions and variables of a module
into the current script.

Coding Exercise 1:
Code a script to print an alphabetically sorted list of all
functions in the re module, which contain the word "find"
import re
# Your code goes here
# sample output

Coding Exercise 2:
Code a very basic calculator. It should have 4 functions
add, sub, mul and div.

What is __pycache__ ?
When you run a program in python, the interpreter compiles it to bytecode first and stores it in
the __pycache__ folder. If you look in __pycache__ folder you will find a bunch of files sharing
the names of the .py files in your project's folder, only their extensions will be either .pyc or
.pyo. These are bytecode-compiled and optimized bytecode-compiled versions of your
program's files, respectively.
As a programmer, you can largely just ignore it. All it does is make your program start a little
faster. When your scripts change, they will be recompiled, and if you delete the files or the
whole folder and run your program again, they will reappear.

Module 5 : Reading/Writing from files
1. Opening and Closing Files
We would 2. Reading from files
learn : 3. Writing to files
4. Check current position of file pointer
5. Copy from one file to another
6. Renaming file
7. Deleting file/s
8. Functions to operate with directories
a> mkdir() Method
b> chdir() Method
c> getcwd() Method
d> rmdir() Method.
Mini file project : Simple invoicing program
a> version 1 : using text file
b> version 2 : using csv file
Opening and Closing Files
Python provides basic functions and methods necessary to manipulate files by default. You can
do most of the file manipulation using a file object.
Before you can read or write a file, you have to open it using Python's built-in open() function.
syntax of open()
file object = open(file_name [, access_mode][, buffering])
 access_mode − The access_mode determines the mode in which the file has to be opened,
i.e., read, write, append, etc.
 If the buffering value is set to 0, no buffering takes place.

 If the buffering value is 1, line buffering is performed while accessing a file.
syntax of close()
fileObject.close();

access_mode : When Opening Files in read mode :
r
Opens a file for reading only. The file pointer is placed at the
beginning of the file. This is the default mode.
rb
Opens a file for reading only in binary format. The file pointer is
placed at the beginning of the file. This is the default mode.
r+
Opens a file for both reading and writing. The file pointer placed at
the beginning of the file.
rb+
Opens a file for both reading and writing in binary format. The file
pointer placed at the beginning of the file.

access_mode : When Opening Files in write mode :
w
Opens a file for writing only. Overwrites the file if the file exists.
If the file does not exist, creates a new file for writing.
wb
Opens a file for writing only in binary format. Overwrites the file if
the file exists. If the file does not exist, creates a new file for
writing.
w+
Opens a file for both writing and reading. Overwrites the existing file
if the file exists. If the file does not exist, creates a new file for
reading and writing.
wb+
Opens a file for both writing and reading in binary format. Overwrites
the existing file if the file exists. If the file does not exist,
creates a new file for reading and writing.

access_mode : When Opening Files in append mode :
a
Opens a file for appending. The file pointer is at the end of the file
if the file exists. That is, the file is in the append mode. If the
file does not exist, it creates a new file for writing.
ab
Opens a file for appending in binary format. The file pointer is at the
end of the file if the file exists. That is, the file is in the append
mode. If the file does not exist, it creates a new file for writing.
a+
Opens a file for both appending and reading. The file pointer is at the
end of the file if the file exists. The file opens in the append mode.
If the file does not exist, it creates a new file for reading and
writing.
ab+
Opens a file for both appending and reading in binary format. The file
pointer is at the end of the file if the file exists. The file opens in
the append mode. If the file does not exist, it creates a new file for
reading and writing.
The file Object Attributes
Once a file is opened and you have one file object, you can get various information related to
that file.
Here is a list of all the attributes related to a file object −

#-example prg
fptr = open("test.txt", "rb")
print("Name of the file: ", fptr.name)
print("Closed or not : ", fptr.closed)
print("Opening mode : ", fptr.mode)
fptr.close()

Reading and Writing Files
 for reading use fileObject.read([count]);
 for writing fileObject.write(string);
#-- program to r/w --
fptr = open(“suven/test.txt", "w")
fptr.write("Python is a \ngreat language")
fptr.close()
#opening again but in a different mode
fptr = open("suven/test.txt", "r+")
str = fptr.read(100) #read up to 100 chars
print("Read String is : ", str)
fptr.close()
print("---------------------------------")
#can open r/w mode also , like this
fptr = open("test.txt", "r+") #can use w+ also
fptr.write("first:I am writing.\nthen:I will read")
fptr.seek(0) #reset the ptr to start of file
str = fptr.read(100)
fptr.close()
File Positions
The tell() method tells you the current position within the file; in other words, the next read or
write will occur at that many bytes from the beginning of the file.
The seek(offset[, from]) method changes the current file position. The offset argument
indicates the number of bytes to be moved. The from argument specifies the reference position
from where the bytes are to be moved.
If from is set to 0, the beginning of the file is used as the reference

position. If it is set to 1, the current position is used as the
reference position. If it is set to 2 then the end of the file would be
taken as the reference position.
#Opens a file
fptr = open("test.txt", "r+")
str = fptr.read(5)
#Check current position

position = fptr.tell()
print("Current file position : ", position)

Coding Exercise 1:
Code a script to copy from one file to another
from sys import argv
from os.path import exists
script, from_file, to_file = argv
print(f"Copying from {from_file} to {to_file}")
#--your code goes here

Renaming and Deleting Files
Python os module provides methods that help you perform file-processing operations, such as
renaming and deleting files.
To use this module, you need to import it first and then you can call any related functions.
The rename() Method

The rename() method takes two arguments, the current filename and the new filename.
Syntax
os.rename(current_file_name, new_file_name)
#---Renaming and Deleting Files--

import os
# Rename a file from test1.txt to test2.txt

os.rename("test1.txt", "test2.txt")

The remove() Method
You can use the remove() method to delete files by supplying the name of the file to be deleted
as the argument.
Syntax
os.remove(file_name)
import os
from os.path import exists
#delete a file
filename = "suven/test2.txt"
os.remove(filename)
print("test2.txt existence :", exists(filename))

Directories in Python
All files are contained within various directories, and Python has no problem handling these too.
The os module has several methods that help you create, remove, and change directories.
1> mkdir() Method

2> chdir() Method
3> getcwd() Method
4> rmdir() Method.
#--Before removing a directory, all the contents in it
should be removed.
The mkdir() Method

You can use the mkdir() method of the os module to create directories in the current
directory. You need to supply an argument to this method, which contains the name of
the directory to be created.
Syntax
os.mkdir("newdir")

The chdir() Method
You can use the chdir() method to change the current directory. The chdir() method takes an
argument, which is the name of the directory that you want to make the current directory.
Syntax
os.chdir("newdir")
The getcwd() Method

The getcwd() method displays the current working directory.
Syntax
os.getcwd()
The rmdir() Method

The rmdir() method deletes the directory, which is passed as an argument in the method.
Before removing a directory, all the contents in it should be removed.
Syntax
os.rmdir('dirname')
Coding Exercise 2:
Write a script to do the following
1> create a directory "test" (only if it does not exist).
2> create a file inside the directory.
Coding Exercise 3:
#--this is version 2 of the above program
Write a script to do the following
1> create a directory "test" (only if it does not exist).
2> change the path to the new directory.
3> create a file inside the (above specified) directory.
Mini file project

Create a simple invoicing program, which should:
1> Help keep track of purchases people made from me.
2> Search the invoice file for a keyword (the invoice number)
3> Lets you enter a new invoice to a file named inv.txt
Optional : Recode the above project to r/w from csv.
#--use with keyword when dealing with file streams--
The with statement simplifies exception handling by
encapsulating common preparation and cleanup tasks in so-called
context managers.
For instance, the open statement is a context manager in itself,

which lets you open a file, keep it open as long as the
execution is in the context of the with statement where you used
it, and close it as soon as you leave the context, no matter
whether you have left it because of an exception or during
regular control flow. Hence resource acquired by the with
statement is released when you leave the with context.

Module 6 : Assertions & Exceptions
We would Assertions : when and why to use ?
learn :
About Exceptions
except Clause with no Exceptions
except Clause with multiple Exceptions
try-finally Clause
Argument of an Exception
Raising an Exception
User-Defined Exceptions
Coding Exercise

Python provides two very important features to handle any unexpected error in
your Python programs and to add debugging capabilities in them −
1. Assertions
2. Exception Handling
Assertions
Programmers often place assertions at the start of a function to check for valid input, and
after a function call to check for valid output.
Syntax
assert Expression[, Arguments]
def KelvinToFahrenheit(Temperature):
assert(Temperature >= 0),"Colder than absolute zero!"
return((Temperature-273)*1.8)+32
print(KelvinToFahrenheit(-5))

Note : If the assertion fails, Python uses ArgumentExpression as the argument for the
AssertionError. AssertionError exceptions can be caught and handled like any other
exception, using the try-except statement. If they are not handled, they will terminate
the program and produce a traceback.

What is Exception?
An exception is an event, which occurs during the execution of a program that disrupts the
normal flow of the program's instructions. In general, when a Python script encounters a
situation that it cannot cope with, it raises an exception. An exception is a Python object that
represents an error.
When a Python script raises an exception, it must either handle the exception immediately
otherwise it terminates and quits.
Handling an exception
If you have some suspicious code that may raise an exception, you can defend your program by
placing the suspicious code in a try: block. After the try: block, include an except:
statement, followed by a block of code which handles the problem as elegantly as possible.
Syntax try:
#You do your operations here
......................
except Exception-1:
#If there is Exception-1, then execute this block.
except Exception-2:
#If there is Exception-2, then execute this block.
......................
else:
#If there is no exception then execute this block.
The except Clause with No Exceptions
try:
......................
except:
#if there is any Exception, then execute this block.
else:
#if there is no exception then execute this block.
Note : This kind of a try-except statement catches all the

exceptions that occur. Using this kind of try-except
statement is not considered a good programming practice
though, because it catches all exceptions but does not
make the programmer identify the root cause of the problem
that may occur.
The except Clause with Multiple Exceptions
try:
......................
except(Exception1[, Exception2[,...ExceptionN]]]):
#if there is any exception from the given exception
list, then execute this block.
else:
#if there is no exception then execute this block.

The try-finally Clause
try:
#You do your operations here;
......................
#Due to any exception, this may be skipped
finally:
#This would always be executed.
......................
Note : You can use a finally: block along with a try: block. The finally: block is a place to
put any code that must execute, whether the try-block raised an exception or not.
Important Pointers:
1> can provide except clause(s), or a finally clause, but
not both.
2> cannot use else clause along with a finally clause.

Argument of an Exception
An exception can have an argument, which is a value that gives additional information
about the problem. The contents of the argument vary by exception. You capture an
exception's argument by supplying a variable in the except clause as follows −
try:
......................
except ExceptionType as Argument:
#You can print value of Argument here...
def temp_convert(var):
try:
return int(var)
except ValueError as A:
print("The argument does not contain numbers\n", A)
ans = temp_convert("sctpl")
print(ans)

Raising an Exception
You can raise exceptions in several ways by using the raise statement. The general syntax for
the raise statement is as follows.
def functionName(level):
if level<1:
raise Exception(level)
return level
try:
l = functionName(-10)
print ("level = ",l)
except Exception as e:
print("error in level argument", e.args[0])

User-Defined Exceptions
Python also allows you to create your own exceptions by deriving classes from the standard
built-in exceptions.
 We would learn this topic after we learn classes & objects.

 This topic is covered in classes & objects chapter

Standard built-in exceptions The following exceptions are used mostly
as base classes for other exceptions.
exception BaseException
The base class for all built-in exceptions. It is not meant to be directly inherited by user-
defined classes (for that, use Exception).
exception Exception
All built-in, non-system-exiting exceptions are derived from this class. All user-defined
exceptions should also be derived from this class.
exception ArithmeticError
The base class for those built-in exceptions that are raised for various arithmetic errors:
OverflowError, ZeroDivisionError, FloatingPointError.
exception BufferError
Raised when a buffer related operation cannot be performed.
exception LookupError
The base class for the exceptions that are raised when a key or index used on a mapping or
sequence is invalid: IndexError, KeyError.

Concrete exceptions
The following exceptions are the exceptions that are usually raised.
Must
exception AssertionError read
Raised when an assert statement fails. section
exception AttributeError
Raised when an attribute reference or assignment fails. (When an object does not
support attribute references or attribute assignments at all, TypeError is raised.)
exception EOFError
Raised when the input() function hits an end-of-file condition (EOF) without reading any
data.
exception ImportError
Raised when the import statement has trouble trying to load a module. Also raised when
the "from list" in from ... import has a name that cannot be found.
exception ModuleNotFoundError
A subclass of ImportError which is raised by import when a module could not be located.
It is also raised when None is found in sys.modules.
exception IndexError
Raised when a sequence subscript is out of range. (Slice indices are silently truncated to
fall in the allowed range; if an index is not an integer, TypeError is raised.)
exception KeyError
Raised when a mapping (dictionary) key is not found in the set of existing keys.
exception KeyboardInterrupt
Raised when the user hits the interrupt key (normally Control-C or Delete). During
execution, a check for interrupts is made regularly. The exception inherits from
BaseException so as to not be accidentally caught by code that catches Exception and thus
prevent the interpreter from exiting.
exception MemoryError
Raised when an operation runs out of memory but the situation may still be rescued (by
deleting some objects). The associated value is a string indicating what kind of (internal)
operation ran out of memory. Note that because of the underlying memory management
architecture (C’s malloc() function), the interpreter may not always be able to completely
recover from this situation; it nevertheless raises an exception so that a stack traceback
can be printed, in case a run-away program was the cause.

exception NameError
Raised when a local or global name is not found. This applies only to unqualified names. The
associated value is an error message that includes the name that could not be found.
exception NotImplementedError
This exception is derived from RuntimeError. In user defined base classes, abstract methods
should raise this exception when they require derived classes to override the method, or while
the class is being developed to indicate that the real implementation still needs to be added.
Note It should not be used to indicate that an operator or method is not meant to be supported
at all – in that case either leave the operator / method undefined or, if a subclass, set it to None.
exception OverflowError
Raised when the result of an arithmetic operation is too large to be represented. This cannot
occur for integers (which would rather raise MemoryError than give up).
exception RecursionError
This exception is derived from RuntimeError. It is raised when the interpreter detects that the
maximum recursion depth (see sys.getrecursionlimit()) is exceeded.
New in version 3.5: Previously, a plain RuntimeError was raised.

exception ReferenceError
This exception is raised when a weak reference proxy, created by the weakref.proxy() function, is
used to access an attribute of the referent after it has been garbage collected. For information
on weak references, refer the weakref module. (on official python docs)
exception RuntimeError
Raised when an error is detected that doesn’t fall in any of the other categories. The associated
value is a string indicating what precisely went wrong.
exception StopIteration
Raised by built-in function next() and an iterator’s __next__() method to signal that there are no
further items produced by the iterator.
Changed in version 3.7: for all code by default: a StopIteration error raised in a generator is
transformed into a RuntimeError.
exception SyntaxError
Raised when the parser encounters a syntax error. This may occur in an import statement, in a
call to the built-in functions exec() or eval(), or when reading the initial script or standard input
(also interactively). Instances of this class have attributes filename, lineno, offset and text for
easier access to the details. str() of the exception instance returns only the message.

exception IndentationError
Base class for syntax errors related to incorrect indentation. This is a subclass of SyntaxError.
exception TabError
Raised when indentation contains an inconsistent use of tabs and spaces. This is a subclass of
IndentationError.
exception SystemExit
This exception is raised by the sys.exit() function. It inherits from BaseException instead of
Exception so that it is not accidentally caught by code that catches Exception. This allows the
exception to properly propagate up and cause the interpreter to exit. When it is not handled,
the Python interpreter exits; no stack traceback is printed.
exception TypeError
Raised when an operation or function is applied to an object of inappropriate type. The
associated value is a string giving details about the type mismatch.
exception ValueError
Raised when an operation or function receives an argument that has the right type but an
inappropriate value, and the situation is not described by a more precise exception such as
IndexError.

exception ZeroDivisionError
Raised when the second argument of a division or modulo operation is zero. The associated
value is a string indicating the type of the operands and the operation.
Extra reading
Exception hierarchy : https://docs.python.org/3.6/library/exceptions.html

Coding Exercise : 1
Write script to read txt file and count lines, words and chars.
fname = "suven\\feed.txt"
num_lines = 0
num_words = 0
num_chars = 0
#your code here....

Coding Exercise : 2
Read txt file and get frequency of a keyword.
fname = "suven\\feedd.txt"
num_key_words = 0
key_word = "scraping"
try:
#your code here....

Section 2
Automating (Daily Business) task
using Python
Module 7 Regex and its daily applications

Module 8 Web scrapping
Module 9 Reading & Writing : PDFs and Word Documents
Module 10 Reading excel sheet and doing data analysis
Module 11 CSV and JSON data Parsing
Module 12 Scheduling task and launching programs
Module 7 : Regex and its daily applications
We would  Finding Patterns of Text Without Regex

learn :  Finding Patterns of Text with Regex (Regular Expressions)
 form groups by using ()
 Matching Multiple Groups with the Pipe |
 Optional Matching with the Question Mark -> (wo)?
 Matching Zero or More with the Star -> Bat(wo)*man
 Matching One or More with the Plus -> Bat(wo)+man
 Matching Specific Repetitions with Curly Brackets -> {1,3}
 Greedy and Non-greedy Matching
 findall() Method
 Using Character Classes : \d, \D, \w, \W, \s, \S
 Making Your Own Character Classes
 Many short coding exercises

Regex
You may be familiar with searching for text by pressing ctrl-F and typing in the words you're
looking for.
Regular expressions go one step further: They allow you to specify a pattern of text to search for.
You may not know a business's exact phone number, but if you live in India, you know it will be
two/three digits of state code , followed by a hyphen or space, and then eight more digits.
This is how you, as a human, know a phone number when you see it: 022-25277413 is a phone
number, but 2,225,277,413 is not.
A regular expression is a special sequence of characters

that helps you match or find other strings or sets of
strings, using a specialized syntax held in a pattern.
The module re provides full support for Perl-like

regular expressions in Python. The re module raises
the exception re.error if an error occurs while
compiling or using a regular expression.

Finding Patterns of Text Without Regex is a hard work, not smart work.
def isPhoneNumber(text):
if len(text) != 13:
return False
for i in range(0, 3):

if not text[i].isdecimal():
return False
If we call the function as :
if text[3] != '-':
return False isPhoneNumber('022-2527-7413')
for i in range(4,8): We get True.

if not text[i].isdecimal():
But its hard work as we are checking
return False
for every character position.
#more code here

Finding Patterns of Text with Regular Expressions
The regex \d\d\d-\d\d\d\d-\d\d\d\d is used by Python to

match the same text in the previous isPhoneNumber()
import re
phoneNumRegex = re.compile(r'\d\d\d-\d\d\d\d-\d\d\d\d')
mo = phoneNumRegex.search('My number is 022-2527-7413.')
#return value of search is match_object or NONE
if mo :
print('Phone number found: ' + mo.group())
Pointers to the above program

1> Passing a string value representing your regular expression to re.compile() returns a Regex
pattern object (or simply, a Regex object).
2> A Regex object’s search() method searches the string it is passed for any matches to the
regex. The search() method will return None if the regex pattern is not found in the string. If the
pattern is found, the search() method returns a Match object.
3> Match objects have a group() method that will return the actual matched text from the
searched string.
Steps for Coding any Regular Expression Script
1. Import the regex module with import re.
2. Create a Regex object with the re.compile() function.

(Remember to use a raw string.)
3. Pass the string you want to search into the Regex

object’s search() method. This returns a Match object.
4. Call the Match object’s group() method to return a

string of the actual matched text.

Grouping with Parentheses
Say you want to separate the area code from the rest of the phone number.
Adding parentheses will create groups in the regex: (\d\d\d)-(\d\d\d-\d\d\d\d).
Then you can use the group() match object method to grab the matching text from just
one group.
#--form groups by using ()

#--printing state code and the no.
import re
phoneNumRegex = re.compile(r'(\d\d\d)-(\d\d\d\d-\d\d\d\d)')
mo = phoneNumRegex.search('My number is 022-2527-7413.')
if mo :
print('State code is : ' + mo.group(1))
print('And the number is : ' + mo.group(2))
print('and the entire no. is : ', mo.group())

Matching Multiple Groups with the Pipe
The | character is called a pipe. You can use it anywhere you want to match one of many
expressions.
For example, the regular expression r'Batman|Tina Fey' will match either 'Batman' or
'Tina Fey'.
When both Batman and Tina Fey occur in the searched string, the first occurrence of
matching text will be returned as the Match object.

Optional Matching with the Question Mark
Sometimes there is a pattern that you want to match only optionally. That is, the regex
should find a match whether or not that bit of text is there.
The ? character flags the group that precedes it as an optional part of the pattern.

Matching Zero or More with the Star
The * (called the star or asterisk) means “match zero or more”—the group that precedes
the star can occur any number of times in the text. It can be completely absent or
repeated over and over again.
Note : If you need to match an actual star character, prefix the star in the regular
expression with a backslash, \*.
Matching One or More with the Plus
While * means “match zero or more,” the + (or plus) means “match one or more.” Unlike
the star, which does not require its group to appear in the matched string, the group
preceding a plus must appear at least once. It is not optional.

Matching Specific Repetitions with Curly Brackets
If you have a group that you want to repeat a specific number of times, follow the group in your
regex with a number in curly brackets.
For example, the regex (Ha){3} will match the string 'HaHaHa', but it will not match 'HaHa',
since the latter has only two repeats of the (Ha) group.
Instead of one number, you can specify a range by writing a minimum, a comma, and a
maximum in between the curly brackets. For example, the regex (Ha){3,5} will match 'HaHaHa',
'HaHaHaHa', and 'HaHaHaHaHa'.

Greedy and Nongreedy Matching
Since (Ha){3,5} can match three, four, or five instances of Ha in the string 'HaHaHaHaHa', you
may wonder why the Match object’s call to group() in the previous curly bracket example
returns 'HaHaHaHaHa' instead of the shorter possibilities. After all, 'HaHaHa' and 'HaHaHaHa'
are also valid matches of the regular expression (Ha){3,5}.
Python’s regular expressions are greedy by default, which means that in ambiguous situations
they will match the longest string possible. The nongreedy version of the curly brackets, which
matches the shortest string possible, has the closing curly bracket followed by a question mark.

The findall() Method
In addition to the search() method, Regex objects also have a findall() method.
While search() will return a Match object of the first matched text in the searched string, the
findall() method will return the strings of every match in the searched string.
If there are groups in the regular expression, then findall() will return a list of tuples. Each tuple
represents a found match, and its items are the matched strings for each group in the regex.

Character Classes
In the earlier phone number regex example, you learned that \d could stand for any
numeric digit. That is, \d is shorthand for the regular expression
(0|1|2|3|4|5|6|7|8|9).
There are many such shorthand character classes, as
Character classes are nice for shortening regular expressions. The character class [0-5] will
match only the numbers 0 to 5; this is much shorter than typing (0|1|2|3|4|5).

The regular expression \d+\s\w+ will match text that has one or more numeric digits
(\d+), followed by a whitespace character (\s), followed by one or more
letter/digit/underscore characters (\w+). The findall() method returns all matching
strings of the regex pattern in a list.

Making Your Own Character Classes
There are times when you want to match a set of characters but the shorthand character classes
(\d, \w, \s, and so on) are too broad.
You can define your own character class using square brackets. For example, the character class
[aeiouAEIOU] will match any vowel, both lowercase and uppercase. Enter the following into the
interactive shell:

Coding Exercise : 1 : Type and run this code :
import re
vowelRegex = re.compile(r'[aeiouAEIOU]')
mo = vowelRegex.findall('RoboCop eats baby food. BABY FOOD.')
if mo :
print(mo)

By placing a caret character (^) just after the character class’s opening bracket, you
can make a negative character class. A negative character class will match all the
characters that are not in the character class.
Coding Exercise : 2
Find all letters other than vowels. Using ^ makes a negative character class.
import re
#your code here

The Caret and Dollar Sign Characters
You can also use the caret symbol (^) at the start of a regex to indicate that a match must
occur at the beginning of the searched text. Likewise, you can put a dollar sign ($) at the
end of the regex to indicate the string must end with this regex pattern. And you can use
the ^ and $ together to indicate that the entire string must match the regex—that is, it’s not
enough for a match to be made on some subset of the string.

#--see the difference of using ^ without []
# using (^Hello) means string starts with Hello
import re
beginsWithHello = re.compile(r'^Hello')
mo1 = beginsWithHello.search('Hello world!')
if mo1 :
print(mo1)
mo2 = beginsWithHello.search('He said hello.')

if mo2 == None:
print(mo2)

#--using $ means ends with
import re
endsWithNumber = re.compile(r'\d$')
mo1 = endsWithNumber.search('Your number is 42')

if mo1 :
print(mo1)
mo2 = endsWithNumber.search('number is forty two.')

if mo2 == None:
print(mo2)

Coding Exercise : 5
Code script to find whether whole String Is Number or not.
#using ^(starts with), +(1 or more), $(ends with)
import re
wholeStringIsNum = re.compile(r'^\d+$')
#---your code here---
#check '1234567890'
#check '12345xyz67890'
#check '12 34567890‘

The Wildcard Character
The . (or dot) character in a regular expression is called a wildcard and will match any
character except for a newline. For example, enter the following into the interactive shell:
Coding Exercise : 6 : Type and run this code.

import re
atRegex = re.compile(r'.at')
mo1 = atRegex.findall('The cat in the hat sat on the flat
mat.')
if mo1 :
print(mo1)

Matching Everything with Dot-Star
Sometimes you will want to match everything and anything. For example, say you want to match
the string 'First Name:', followed by any and all text, followed by 'Last Name:', and then
followed by anything again. You can use the dot-star (.*) to stand in for that “anything.”
Remember that the dot character means “any single character except the newline,” and the
star character means “zero or more of the preceding character.”

import re
nameRegex = re.compile(r'First Name:(.*) Last Name:(.*)')
mo = nameRegex.search('First Name:Suven Last Name:Consultants')
if mo :
print(mo.group(1))
print(mo.group(2))

The dot-star uses greedy mode: It will always try to match as much text as possible.
To match any and all text in a nongreedy fashion, use the dot, star, and question mark (.*?).

import re
nongreedyRegex = re.compile(r'<.*?>')
mo1 = nongreedyRegex.search('<To serve paneer> for dinner.>')
if mo1 :
print(mo1.group())
greedyRegex = re.compile(r'<.*>')
mo2 = greedyRegex.search('<To serve paneer> for dinner.>')
if mo2 :
print(mo2.group())

Coding Exercise : 9 : Run this code.
import re
noNewlineRegex = re.compile('.*')
mo1 = noNewlineRegex.search('Serve the public trust.\nProtect
the innocent.\nUphold the law.')
The dot-star will match
if mo1 : everything except a
print("no new line :", mo1.group()) newline. By passing
re.DOTALL as the second
argument to re.compile(),
print("------------------------------") you can make the dot
character match all
characters, including the
newline character.
newlineRegex = re.compile('.*', re.DOTALL)
mo2 = newlineRegex.search('Serve the public trust.\nProtect the
innocent.\nUphold the law.')
if mo2 :
print(mo2.group())

Case-Insensitive Matching
Normally, regular expressions match text with the exact casing you specify. To make your
regex case-insensitive, you can pass re.IGNORECASE or re.I as a second argument to
re.compile().
import re
robocop = re.compile(r'robocop', re.I)
#cascaded all statements

print(robocop.search('RoboCop is part man, part machine,
all cop.').group())

Review of Regex Symbols

Substituting Strings with the sub() Method
Regular expressions can not only find text patterns but can also substitute new text in
place of those patterns. The sub() method for Regex objects is passed two arguments. The
first argument is a string to replace any matches.
The second is the string for the regular expression. The sub() method returns
a string with the substitutions applied.

import re
#-- \w -> any char except space
namesRegex = re.compile(r'Agent \w+')
mo = namesRegex.sub('CENSORED', 'Agent Alice gave the secret documents to Agent Bob.')
if mo :
print(mo)

Coding example : 12 : little complex example
Sometimes you may need to use the matched text itself as part of the substitution. In the first
argument to sub(), you can type \1, \2, \3, and so on, to mean “Enter the text of group 1, 2, 3,
and so on, in the substitution.”
For example, say you want to censor the names of the secret agents by showing just the first
letters of their names. To do this, you could use the regex Agent (\w)\w* and pass
r'\1****' as the first argument to sub().
The \1 in that string will be replaced by whatever text was matched by group 1— that is, the
(\w) group of the regular expression.
Run the above code “Hide the names of agents” as given in your workspace

Managing Complex Regexes
Regular expressions are fine if the text pattern you need to match is simple.
But matching complicated text patterns might require long, convoluted regular expressions. You
can mitigate this by telling the re.compile() function to ignore whitespace and comments inside
the regular expression string.
This “verbose mode” can be enabled by passing the variable re.VERBOSE as the second
argument to re.compile().
Now instead of a hard-to-read regular expression like this:
You can spread the regular

expression over multiple
lines with comments
like this:

Managing Complex Regexes
#--Coding ex: 13 : Regex to check a phone number
Run the above code as given in your workspace
#--Coding ex:14: Regex to check phone number, using VERBOSE


Explanation to the phoneRegex used in Coding Ex : 14:
The phone number begins with an optional area code, so the area code group is followed
with a question mark. Since the area code can be just three digits (that is, \d{3}) or three
digits within parentheses (that is, $\d{3}$), you should have a pipe joining those parts.
The phone number separator character can be a space (\s), hyphen (-), or period (.), so
these parts should also be joined by pipes.
The last part is an optional extension made up of any number of spaces followed by ext, x,
or ext., followed by two to five digits.

Module 8 : Web scrapping
Web scraping is the term for using a program to download and process content
from the Web. For example, Google runs many web scraping programs to index
web pages for its search engine.
We would  Modules needed and built-in in python for Web scrapping

learn :  open a web page from your script
 Downloading Files from the Web with the requests Module
 Checking for Errors
 Saving Downloaded Files to the Hard Drive
 Parsing HTML with the BeautifulSoup Module
 scraping a web page and finding all <a> tags
 Mini Project : Automate Google Search

Modules needed and built-in in python
1. webbrowser : Comes with Python and opens a browser to a specific page.
2. Requests : Downloads files and web pages from the Internet.
3. Beautiful Soup : Parses HTML, the format that web pages are written in.
4. Selenium : Launches and controls a web browser. Selenium is able to fill in forms and
simulate mouse clicks in this browser.
The webbrowser module’s open() function can launch a new browser to a specified
URL.
import webbrowser
webbrowser.open('https://suvenconsultants.com//')

Coding Exercise:
1> Gets a street address from the command line arguments
2> Open the web browser to the Google Maps page for the address.
import webbrowser, sys
if len(sys.argv) > 1:
#--Get address from command line.
#--- your code here ---
#try running the code with some address

#like suven consultants chembur mumbai

Downloading Files from the Web with the requests Module
The requests module lets you easily download files from the Web without having to worry
about complicated issues such as network errors, connection problems, and data
compression.
The requests module doesn’t come with Python, so you’ll have to install it first. From the
command line, run
pip install requests.
Try running this code :

import requests
res = requests.get('https://training.suven.net/')
#check the response object
print(type(res))
if res.status_code == requests.codes.ok :
len(res.text)
print(res.text[:251])

Explanation to above code :
The URL goes to a web page https://training.suven.net.
You can tell that the request for this web page succeeded by checking the status_code attribute
of the Response object. If it is equal to the value of requests.codes.ok, then everything went
fine well. (other status code is 404 called “Not Found.”)
If the request succeeded, the downloaded web page is stored as a string in the Response
object’s text variable. This variable holds a large string of len(res.text).
Finally, calling print(res.text[:251]) displays only the first 251 characters.

Checking for Errors
As you’ve seen, the Response object has a status_code attribute that can be checked against
requests.codes.ok to see whether the download succeeded. A simpler way to check for success
is to call the raise_for_status() method on the Response object.
This will raise an exception if there was an error downloading the file and will do nothing if the
download succeeded.
import requests
res = requests.get('http://suven.net/no_page.html')
try:
res.raise_for_status()
except Exception as exc:
print('There was a problem: %s' % (exc))
Note : Always call raise_for_status() after calling requests.get(). You want to be sure that
the download has actually worked before your program continues.

Saving Downloaded Files to the Hard Drive
You can save the web page to a file on your hard drive with the standard open() function
and write() method.
There are some slight differences, though.
First, you must open the file in write binary mode by passing the string 'wb' as the
second argument to open(). Even if the page is in plaintext , you need to write binary data
instead of text data in order to maintain the Unicode encoding of the text.

Explanation :
The iter_content() method returns “chunks” of the content on each iteration through the loop.
Each chunk is of the bytes data type, and can specify how many bytes each chunk will contain.
For review, here’s the complete process for downloading and saving a file:
1. Call requests.get() to download the file.
2. Call open() with 'wb' to create a new file in write binary mode.
3. Loop over the Response object’s iter_content() method.
4. Call write() on each iteration to write the content to the file.
5. Call close() to close the file.

Parsing HTML with the BeautifulSoup Module
Beautiful Soup is a module for extracting information from an HTML page (and is much
better for this purpose than regular expressions). The BeautifulSoup module’s name is bs4
(for Beautiful Soup, version 4). To install it, you will need to run pip install beautifulsoup4
from the command line.
While beautifulsoup4 is the name used for installation, to import Beautiful Soup you run
import bs4.
Coding Exercise:
Scrape a web page and find all <a> tags

Project : Automate Google Search :
Problem Definition : Whenever I search a topic on Google, I don’t look at just one search result
at a time. By middle-clicking a search result link, I open the first several links in a bunch of new
tabs to read later.
I search Google often enough that this workflow—opening my browser, searching for a topic,
and middle-clicking several links one by one—is tedious. It would be nice if I could simply type a
search term on the command line and have my computer automatically open a browser with all
the top search results in new tabs.
Write a script to do this.
Algorithm :
1> Get search keywords from the command line arguments.
2> Retrieves the search results page.
3> Open a browser tab for each result.

Implementation steps
This means your code will need to do the following:
a> Read the command line arguments from sys.argv.
b> Fetch the search result page with the requests module.
c> Find the links to each search result.
d> Call the webbrowser.open() function to open the web browser.
Refer version 1 as given in your workspace

Module 9 : Reading & Writing : PDFs and Word Documents
PDF and Word documents are binary files, which makes them much more complex than
plaintext files. In addition to text, they store lots of font, color, and layout information. If you
want your programs to read or write to PDFs or Word documents, you’ll need to do more than
simply pass their filenames to open().
Fortunately, there are Python modules that make it easy for you to interact with PDFs and
Word documents.

We would a) read and write to a .docx file use python-docx package
b) create a new doc and write some thing to it
learn :
c) change the style of a para from above doc
d) Mini project to read text, links, images* from docx using docxpy package
e) Mini project for extracting images use docx2txt
f) Reading from PDF doc using PyPDF2
g) creating a new combined pdf from 2 pdf docs
h) Rotating Pages of a pdf document
i) Add watermark to a pdf doc
j) Encrypting PDFs, i.e. password protecting
k) Decrypting a pdf doc and reading from it
Python can create and modify Word documents, which have the .docx file extension, with the
python-docx module.
You can install the module by running
pip install python-docx
#--this will install 2 packages : lxml and python-docx

Compared to plaintext, .docx files have a lot of structure. This structure is represented by
three different data types in Python-Docx.
1. At the highest level, a Document object represents the entire document.

2. The Document object contains a list of Paragraph objects for the paragraphs in the
document. (A new paragraph begins whenever the user presses enter or return while
typing in a Word document.)
3. Each of these Paragraph objects contains a list of one or more Run objects.
4. The single-sentence paragraph in below Figure has four runs.
5. The text in a Word document is more than just a string. It has font, size, color, and
other styling information associated with it. A style in Word is a collection of these
attributes.
6. A Run object is a contiguous run of text with the same style.
7. A new Run object is needed whenever the text style changes.

Lets understand python-docx through this simple example.
Note : demo.docx file is available in the workspace.

1) we open a .docx file in Python, call docx.Document(), and pass the filename demo.docx. This
will return a Document object, which has a paragraphs attribute that is a list of Paragraph
objects.
2) When we call len() on doc.paragraphs, it returns 7, which tells us that there are seven
Paragraph objects in this document.
3) Each of these Paragraph objects has a text attribute that contains a string of the text in that
paragraph (without the style information). Here, the first text attribute contains 'DocumentTitle'
4) and the second contains 'A plain paragraph with some bold and some italic'.
5) each Paragraph object also has a runs attribute that is a list of Run objects. Run objects also
have a text attribute, containing just the text in that particular run. Let’s look at the text
attributes in the second Paragraph object, 'A plain paragraph with some bold and some italic'.
Calling len() on this Paragraph object tells us that there are four Run objects.
6) The first run object contains 'A plain paragraph with some '.
7) Then, the text change to a bold style, so 'bold' starts a new Run object.
8) The text returns to an unbolded style after that, which results in a third Run object, ' and
some '.
9) Finally, the fourth and last Run object contains 'italic' in an italic style.
With Python-Docx, your Python programs will now be able to read the text from
a .docx file and use it just like any other string value.

w.r.t practice programs in the workspace :
first code : teaches you to read one or more elements from the .docx file.
To read the full text, understand this code :

Styling Paragraph and Run Objects Must
This theory part is important to understand before coding :
Read
a> Create .docx files.
b> Add Text content.
c> Read Styles into your code and change them.
In Word for Windows, you can see the styles by pressing

ctrl-alt-shift-S to display the Styles pane,
which looks like >>

For Word documents, there are three types of styles: Paragraph styles can be applied to
Paragraph objects, character styles can be applied to Run objects, and linked styles can be
applied to both kinds of objects.
You can give both Paragraph and Run objects styles by setting their style attribute to a string.
This string should be the name of a style. If style is set to None, then there will be no style
associated with the Paragraph or Run object.
The string values for the default Word styles are as follows:

When setting the style attribute, do not use spaces in the style name.
For example, while the style name may be Subtle Emphasis, you should set the style attribute
to the string value 'SubtleEmphasis' instead of 'Subtle Emphasis'.
Including spaces will cause Word to misread the style name and not apply it.
When using a linked style for a Run object, you will need to add 'Char' to the end of its name.
For example, to set the Quote linked style for a Paragraph object, you would use
paragraphObj.style = 'Quote', but for a Run object, you would use runObj.style = 'QuoteChar'.
Run Attributes
Runs can be further styled using text
attributes. Each attribute can be set to one
of three values: True (the attribute is always
enabled, no matter what other styles are
applied to the run), False (the attribute is
always disabled), or None (defaults to
whatever the run’s style is set to).
lists the text attributes that can be set on

Run objects. >>

Writing Word Documents
To create your own .docx file, call docx.Document() to return a new, blank Word
Document object. The add_paragraph() document method adds a new paragraph of
text to the document and returns a reference to the Paragraph object that was added.
When you’re done adding text, pass a filename string to the save() document method to
save the Document object to a file.
This will create a file named helloworld.docx in the current working directory

You can add paragraphs by calling the add_paragraph() method again with the new
paragraph’s text. Or to add text to the end of an existing paragraph, you can call the
paragraph’s add_run() method and pass it a string.
Note that the text This text is being added to the second paragraph. was added to the
Paragraph object in paraObj1, which was the second paragraph added to doc. The
add_paragraph() and add_run() functions return paragraph and Run objects,
respectively, to save you the trouble of extracting them as a separate step.
Keep in mind that as of Python-Docx, new Paragraph objects can be added only to
the end of the document, and new Run objects can be added only to the end of a
Paragraph object. The save() method can be called again to save the additional
changes you’ve made.
Adding Headings
Calling add_heading() adds a paragraph with one of the heading styles.
The arguments to add_heading() are a string of the heading text and an integer from 0
to 4. The integer 0 makes the heading the Title style, which is used for the top of the
document. Integers 1 to 4 are for various heading levels, with 1 being the main heading
and 4 the lowest subheading.

Limitation of python-docx
Conceptually, Word documents have two layers, a text layer and a drawing layer.
In the text layer, text objects are flowed from left to right and from top to bottom, starting a
new page when the prior one is filled. In the drawing layer, drawing objects, called shapes, are
placed at arbitrary positions. These are sometimes referred to as floating shapes.
A picture is a shape that can appear in either the text or drawing layer. When it appears in the
text layer it is called an inline shape, or more specifically, an inline picture.
Inline shapes are treated like a big text character (a character glyph). python-docx only supports
inline pictures. We can only add. Not get or remove or process the images.
Solution : to extract text, hyperlinks, images* use docxpy
pip install docxpy
From workspace :
Refer code that reads text, links, images* from docx using docxpy.

Limitation of docxpy
The same package docxpy can be used for extracting images. But code fails
most of the times. Hence for extracting images use docx2txt.
pip install docx2txt

Reading from PDF doc
pip install PyPDF2
The module we will use to work with PDFs is PyPDF2. To install it, run pip install PyPDF2
from the command line. This module name is case sensitive, so make sure the y is
lowercase and everything else is uppercase.
Note : PyPDF2 does not have a way to extract images, charts, or other media
from PDF documents, but it can extract text and return it as a Python string.
To understand capabilities of PyPDF2, Run the following codes from

the workspace :
 Opening a PDF doc and reading contents from it.
 Creating a new combined pdf from 2 pdf docs
 Rotating Pages of a pdf document
 Add watermark to a pdf doc
 Encrypting PDFs, i.e password protecting
 Decrypting a pdf doc and reading from it

Imp Note : In PyPDF2’s PdfFileReader object is used to read text data.
Similarly PdfFileWriter object, can create new PDF files. But PyPDF2 cannot write
arbitrary text to a PDF like Python can do with plaintext files.
Instead, PyPDF2’s PDF-writing capabilities are limited to copying pages from other
PDFs, rotating pages, overlaying pages, and encrypting files.

Module 10 : Reading excel sheet and doing data analysis
Excel is a popular and powerful spreadsheet application for Windows. The openpyxl
module allows your Python programs to read and modify Excel spreadsheet files.
For example, you might have the boring task of copying certain data from one spreadsheet and
pasting it into another one. Or you might have to go through thousands of rows and pick
out just a handful of them to make small edits based on some criteria. Or you might have to look
through hundreds of spreadsheets of department budgets, searching for any that are in the red.
These are exactly the sort of boring, mindless spreadsheet tasks that Python can do for you.

1. Getting Sheets from the Workbook
We would
2. Getting Cells from the Sheets
learn :
3. printing all values of a given column
4. printing a matrix of rows and columns
5. Project: Reading census spreadsheet file and calculate statistics

for each county.

Excel Documents
First, let’s go over some basic definitions:
1. An Excel spreadsheet document is called a workbook. A single workbook is saved in a file

with the .xlsx extension.
2. Each workbook can contain multiple sheets (also called worksheets). The sheet the user is
currently viewing (or last viewed before closing Excel) is called the active sheet.
3. Each sheet has columns (addressed by letters starting at A) and rows (addressed by numbers
starting at 1). A box at a particular column and row is called a cell.
4. Each cell can contain a number or text value. The grid of cells with data makes up a sheet.

Python does not come with OpenPyXL, so you’ll have to
install it.
pip install openpyxl
Run all codes from workspace
#---loading an example.xlsx file --

import openpyxl
wb = openpyxl.load_workbook('suven/example.xlsx')
print(type(wb))
#--wb is a Workbook object that represents the Excel file, just

like how a File object represents an opened text file.

#--Getting Sheets from the Workbook
import openpyxl
print("Sheet names : ", wb.get_sheet_names())
sheet = wb.get_sheet_by_name('dummy')
print("Sheet Spec : ", sheet)
print("Title of 3rd sheet :", sheet.title)
anotherSheet = wb.get_active_sheet()
print("Active sheet : ", anotherSheet)

#--Getting a Cell value from the Sheet--
import openpyxl
sheet = wb.get_sheet_by_name('Sheet1')
print("first row, first coln :", sheet['A1'])
print("-------------------")
print("value of 1st row, 1st coln :", sheet['A1'].value)
Note : You can also get a cell using the sheet’s cell() method and passing integers for its row
and column keyword arguments. The first row or column integer is 1, not 0.

#--print all values of a given column
for i in range(1, 8):
print(i, sheet.cell(row=i, column=2).value)
#--we are printing all values of column no. 2
Instead of printing we can process it, like :

1> copy to a txt file
2> apply regex to search for some pattern
3> do analytics like finding some k/w frequency

#--printing a matrix of rows and columns--
import openpyxl
sheet = wb.get_sheet_by_name('Sheet1')
print(tuple(sheet['A1':'C3']))
for rowOfCellObjects in sheet['A1':'C3']:

for cellObj in rowOfCellObjects:
print(cellObj.coordinate, cellObj.value)
print('--- END OF ROW ---')

Project: Reading Data from a Spreadsheet
---------------------------------------------------------------------------------------
Problem : Say you have a spreadsheet of data from the 2017 US Census and you have the
boring task of going through its thousands of rows to count both the total population and the
number of census tracts* for each country. *Each row represents a single census tract.
Even though Excel can calculate the sum of multiple selected cells, you’d still have to select the
cells for each of the 3,000-plus counties. Even if it takes just a few seconds to calculate a
county’s population by hand, this would take hours to do for the whole spreadsheet.
-----------------------------------------------------------------------------------------
Code a script : that can read from the census spreadsheet file and calculate statistics for
each county. Automate your work to finish within matter of seconds.
#--Refer Algorithm and Implementation steps in workspace file.

Module 11 : CSV and JSON data Parsing
You learned how to extract text from PDF and Word documents. These files were in a binary
format, which required special Python modules to access their data.
CSV and JSON files, on the other hand, are just plaintext files. You can view them in a text editor.
But Python also comes with the special csv and json modules, each providing functions to help
you work with these file formats.

We would a) Reading from a .csv file
learn : b) Access any particular cell entry using indexing
c) Reading Data from Reader Objects in a for Loop
d) Using Writer Object to write to a csv file
e) Project : Automate the task of removing first line from 100's of CSV files.
f) Reading JSON with the loads() Function
g) Writing JSON with the dumps() Function
h) Understanding serialization and de-serialization process
i) Project idea: Fetching Current Weather Data

About CSV and JSON
CSV stands for "comma-separated values," and CSV files are simplified spreadsheets stored as
plaintext files. Python’s csv module makes it easy to parse CSV files. CSV file data looks
like :
JSON (pronounced "JAY-saw" or "Jason“) is a format that stores information as JavaScript

source code in plaintext files. (JSON is short for JavaScript Object Notation.)
You don’t need to know the JavaScript programming language to use JSON files, but the JSON
format is useful to know because it’s used in many web applications. JSON data looks like :
{"president": {"name": "Ram Nath Kovind", "native": "Uttar

Pradesh"}}

Reading CSV files
Remember :
1. The advantage of CSV files is simplicity. CSV files are widely supported by many types of
programs, can be viewed in text editors, and are a straightforward way to represent
spreadsheet data.
2. The CSV format is exactly as advertised: It’s just a text file of comma-separated values.

Reader Objects
 To read data from a CSV file with the csv module, you need to create a Reader object.
 A Reader object lets you iterate over lines in the CSV file.
#--reading from a .csv file
import csv
exampleFile = open('suven/example.csv')
exampleReader = csv.reader(exampleFile)
exampleData = list(exampleReader)
print(exampleData)
Note 1: You don’t pass a filename string directly to the csv.reader() function.
Note 2: The most direct way to access the values in the Reader object is to convert it to a
plain Python list by passing it to list() . Using list() on this Reader object returns a list of
lists, which you can store in a variable like exampleData.

#--To access any particular cell entry using indexing
import csv
exampleData = list(exampleReader)
print("printing the first cell entry")

print("------------------------------")
print(exampleData[0][0], '\n')

Reading Data from Reader Objects in a for Loop
 For large CSV files, you’ll want to use the Reader object in a for loop.
 This avoids loading the entire file into memory at once.
#--Reading Data from Reader Objects in a for Loop

import csv
for row in exampleReader:

print('Row #' + str(exampleReader.line_num) + ' ' + str(row))
#--The print() function call prints the number of the current row and the contents of the
row. To get the row number, use the Reader object’s line_num variable, which contains
the number of the current line.
Note : The Reader object can be looped over only once. To

reread the CSV file, you must call csv.reader to create a
Reader object.

Writer Objects
 A Writer object lets you write data to a CSV file.
 To create a Writer object, you use the csv.writer() function.
#--Writer Object : lets write to a csv file
import csv
outputFile = open('suven/output.csv', 'w', newline='')

outputWriter = csv.writer(outputFile)
outputWriter.writerow(['Suven','Consultants','Python','IBM'])
outputWriter.writerow(['Hello, world!','eggs','bread','tea'])
outputWriter.writerow([1, 2, 3.141592, 4])
outputFile.close()
#--Note : if you forget to set the newline argument, the rows

in output.csv will be double-spaced. Try it.

Project :
You have the boring job of removing the first line from several hundred CSV files. Maybe you’ll
be feeding them into an automated process that requires just the data and not the headers at
the top of the columns. You could open each file in Excel, delete the first row, and resave the
file—but that would take hours. Write a program to do it instead.
Expected from the Program :

The program will need to open every file with the .csv extension in the current working
directory, read in the contents of the CSV file, and rewrite the contents without the first row to
a file of the same name. This will replace the old contents of the CSV file with the new, headless
contents.
At a high level, the program must do the following:

• Find all the CSV files in the current working directory.
• Read in the full contents of each file.
• Write out the contents, skipping the first line, to a new CSV file.

About JSON and its applications
JSON is useful to know, because many websites offer JSON content as a way for programs to
interact with the website. This is known as providing an application programming interface
(API).
Accessing an API is the same as accessing any other web page via a URL. The difference is that
the data returned by an API is formatted (with JSON, for example) for machines;
APIs aren’t easy for people to read.
Many websites make their data available in JSON format. Facebook, Twitter, Yahoo, Google,
Tumblr, Wikipedia, Flickr, Data.gov, Reddit, IMDb, Rotten Tomatoes, LinkedIn, and many other
popular sites offer APIs for programs to use. Some of these sites require registration, which is
almost always free.
You’ll have to find documentation for what URLs your program needs to request in order to get
the data you want, as well as the general format of the JSON data structures that are returned.
This documentation should be provided by whatever site is offering the API; if they have
a “Developers” page, look for the documentation there.

The JSON Module
Python’s JSON module handles all the details of translating between a string with JSON data and
Python values for the json.loads() and json.dumps() functions.
JSON can't store every kind of Python value. It can contain values of only the following data
types: strings, integers, floats, Booleans, lists, dictionaries, and NoneType.
JSON cannot represent Python-specific objects, such as File objects, CSV Reader or Writer
objects, Regex objects, or Selenium WebElement objects.

#--Reading JSON with the loads() Function
stringOfJsonData = '{"name": "Catty", "isCat": true,

"miceCaught": 0, "felineIQ": null}'
import json
jsonDataAsPythonValue = json.loads(stringOfJsonData)
print(jsonDataAsPythonValue)
Reading JSON with the loads() Function

To translate a string containing JSON data into a Python value, pass it to the json.loads()
function. (The name means “load string,” not “loads.”)

#--Writing JSON with the dumps() Function
The json.dumps() function (which means “dump string,” not “dumps”) will translate a
Python value into a string of JSON-formatted data.
pythonValue = {'isCat': True, 'miceCaught': 0, 'name':

'Catty','felineIQ': None}
import json
stringOfJsonData = json.dumps(pythonValue)
print(stringOfJsonData)
#--The process of encoding JSON is called serialization.
#--De-serialization is the reciprocal process of decoding data that has been stored
or delivered in the JSON standard.

#--reading json data and writing to a file
import json
data = {
"president": {
"name": "Ram Nath Kovind",
"native": "Uttar Pradesh"
}
}
with open("suven/data_file.json", "w") as write_file:

json.dump(data, write_file)

#--reading json data from a file and printing it.
import json
with open("suven/data_file.json", "r") as read_file:

str = read_file.read(1000)
print(json.loads(str))

#--formatting data in dumps()
import json
data = {
"president": {
"name": "Ram Nath Kovind",
"native": "Uttar Pradesh"
}
}
print(json.dumps(data))
print("--------------------------------")
#after formatting data

print(json.dumps(data, indent=4))

Project idea: Fetching Current Weather Data
Algorithmic Steps :
Reads the requested location from the command line.
• Downloads JSON weather data from OpenWeatherMap.org.
• Converts the string of JSON data to a Python data structure.
• Prints the weather for today and the next two days.
Coding Steps:
• Join strings in sys.argv to get the location.
• Call requests.get() to download the weather data.
• Call json.loads() to convert the JSON data to a Python data structure.
• Print the weather forecast.
#--Coding this project is optional.

Module 12 : Scheduling task and launching programs
Your computer’s clock can schedule programs to run code at some specified time and date or at
regular intervals. For example, your program could scrape a website every hour to check for
changes or do a CPU-intensive task at 4 am while you sleep. Python’s time and datetime
modules provide these functions.
You can also write programs that launch other programs on a schedule by using the sub-process
and threading modules.

We would a) Concept of Unix epoch (as a time reference)
learn : b) Application of time() function
 finding time taken to execute a piece of code
c) use of sleep(n secs)
d) use of datetime object
e) Pausing Until a Specific Date and time
f) Launching Other Programs from Python using Popen()
 poll() and wait()
Mini-Project 1: Super Stopwatch

Mini-Project 2: Simple Countdown Program
The time.time() Function
The Unix epoch is a time reference commonly used in programming: 12 am on January 1,
1970, Coordinated Universal Time (UTC).
The time.time() function returns the number of seconds since that moment as a float value.
This number is called an epoch timestamp.
Epoch timestamps can be used to profile code, that is, measure how long a piece of
code takes to run. If you call time.time() at the beginning of the code block you want to
measure and again at the end, you can subtract the first timestamp from the second to
find the elapsed time between those two calls.
# -- find time taken to execute a piece of code.

# -- refer code in workspace

The time.sleep() Function
If you need to pause your program for a while, call the time.sleep() function
and pass it the number of seconds you want your program to stay paused.
#--use of sleep(n secs)

#--try CTRL-C and you would see KeyboardInterrupt Exception
import time
for i in range(3):
print('Tick')
time.sleep(1)
print('Tock')
time.sleep(1)

The datetime Module
 The time module is useful for getting a Unix epoch timestamp to work with.
 But if you want to display a date in a more convenient format, or do arithmetic
with dates (for example, figuring out what date was 205 days ago or what date is 123 days
from now), you should use the datetime module.
 The datetime module has its own datetime data type.
 datetime values represent a specific moment in time.
import datetime
#--present datetime
print(datetime.datetime.now())
#--making a datetime object

dt = datetime.datetime(2018, 12, 21, 16, 29, 0)
print(dt.year, dt.month, dt.day)

print(dt.hour, dt.minute, dt.second)

A Unix epoch timestamp can be converted to a datetime object with the
datetime.datetime.fromtimestamp() function. The date and time of the datetime
object will be converted for the local time zone.
#--Pausing Until a Specific Date and time

import datetime
import time
traditionalDay = datetime.datetime(2020, 1, 2, 14, 03, 0)
while datetime.datetime.now() < traditionalDay:

time.sleep(1)
Choose a date + time > datetime.now()

Launching Other Programs from Python
Your Python program can start other programs on your computer with the Popen()
function in the built-in subprocess module. (The P in the name of the Popen()
function stands for process.) If you have multiple instances of an application open, each
of those instances is a separate process of the same program. For example, if you open
multiple windows of your web browser at the same time, each of those windows is a
different process of the web browser program.
#--opening calculator application

import subprocess
subprocess.Popen('C:\\Windows\\System32\\calc.exe')

poll() and wait()
The return value is a Popen object, which has two useful methods:
poll() and wait().
You can think of the poll() method as asking your friend if she’s finished running the code you
gave her.
The poll() method will return None if the process is still running at the time poll() is called.
If the program has terminated, it will return the process’s integer exit code.
An exit code is used to indicate whether the process terminated without errors (an exit code
of 0) or whether an error caused the process to terminate (a nonzero exit code —generally 1,
but it may vary depending on the program).
The wait() method is like waiting for your friend to finish working on her code before you keep
working on yours. The wait() method will block until the launched process has terminated.
This is helpful if you want your program to pause until the user finishes with the other program.
The return value of wait() is the process’s integer exit code.

Mini-Project 1: Super Stopwatch
Your super-stopwatch should
1> Track the amount of time elapsed between presses of the enter key.
2> With each key press starting a new "lap" on the timer.
3> Print the lap number, total time, and lap time.
Coding steps for super-stopwatch :

1. Find the current time by calling time.time() and store it as a timestamp at the start of the
program, as well as at the start of each lap.
2. Keep a lap counter and increment it every time the user presses enter.
3. Calculate the elapsed time by subtracting timestamps.
4. Handle the KeyboardInterrupt exception so the user can press ctrl-C to quit.

Mini-Project 2: Simple Countdown Program
Write a countdown program that plays an alarm at the end of the
countdown.
At a high level, here’s what your program will do:

 Count down from 10.
 Play a sound file (alarm.wav) when the countdown reaches zero.

 Pause for one second in between displaying each number in the countdown by calling
time.sleep().
 Call subprocess.Popen() to open the sound file with the default application.

Section 3 Index
Advanced Python Concepts
Module 13 Classes & Object Programming
Module 14 Inheritance, Overloading and Overriding
Module 15 Database programming
Module 16 Multithread Programming
Module 17 Introduction to Functional Programming(FP)
Internship A> Phone Number and Email Address Extractor
Project
Specification B> Combining Select Pages from Many PDFs
Module 13 : Classes & Object Programming

We would Prerequisite : Basic knowhow of OOP Terminology
learn : https://medium.freecodecamp.org/object-oriented-programming-concepts-
21bb035f7260
a) Syntax of Creating Class & Object
b) Why self is passed ?
c) Using getters and setters
d) Defining class method use annotation
e) Built-In Class Attributes
f) Class Inheritance
g) Overriding Methods
h) Overloading Methods
i) Overload the built-in method
j) Overloading Operators
k) Data Hiding
l) Destroying Objects (Garbage Collection)
Creating Classes
The class statement creates a new class definition. The name of the class immediately follows
the keyword class followed by a colon as follows −
#-- Syntax :
class ClassName:
'Optional class documentation string'
class_suite
#--The class has a documentation string, which can be accessed

via ClassName.__doc__.
#--The class_suite consists of all the component statements

defining class members, data attributes and functions.

#Example
class Employee:
'Common base class for all employees'
empCount = 0 #this is a class variable
def __init__(self, name, salary):

self.name = name
self.salary = salary
Employee.empCount += 1
def displayCount(self):
print("Total Employee %d" % Employee.empCount)
1. The first method __init__() is a special method, which is called class constructor or
initialization method that Python calls when you create a new instance of this class.
2. You declare other class methods like normal functions with the exception that the first
argument to each method is self. Python adds the self argument to the list for you; you do
not need to include it when you call the methods.

Important Notes on self :
1> self is a special parameter that will always be used as a reference back to the object itself.*
2> Remember , the first argument to each method is self.
*Why self is passed ? -- v.imp

Important: Unlike some other languages, when a new object is created in python, it does not
create a new set of instance methods to itself. So instance methods lie within the class object
without being created with each object initialization, nice way to save up memory. Recall that
python is a fully object oriented language and so a class is also an object. So it lives within the
memory.
class SomeClass: here we have created obj1 and

def __init__(self): are calling the instance method
self.arr = [] insert_to_arr() of SomeClass
while passing an argument 6.
def insert_to_arr(self, value): But now how does that method
self.arr.append(value) know "which object is calling me
and whose instance attributes
should be updated".
obj1 = SomeClass()
obj2 = SomeClass() Ans : self gets refr to obj1
obj1.insert_to_arr(6)
Instead of directly accessing attributes, use the following functions :
1. getattr(obj, name[, default]) #called getters
2. hasattr(obj,name)
3. setattr(obj,name,value) #called setters
4. delattr(obj, name)
class Employee:
def __init__(self, name, salary):
self.name = name
self.salary = salary
self.loc = None
#--This would create first object of Employee class

emp1 = Employee("Neetu", 23000)
print(getattr(emp1, "salary", 10000))
print(hasattr(emp1,"salary"))
#--change the value of salary attribute. setters have no return value

setattr(emp1,"salary","24500")
#--deleting a un-wanted attribute

print("deleting location attribute :",delattr(emp1, "loc"))

For defining class method use annotation
#-- for defining class method use annotation
class SomeClass:
def create_arr(self): # An instance method
self.arr = []
def insert_to_arr(self, value): #An instance method

self.arr.append(value)
@classmethod
def class_method(cls):
print("the class method was called")
#--we are calling the class method, not instance method

SomeClass.class_method()
#--Note: that the class calling becomes the parameter : cls

Built-In Class Attributes
Every Python class keeps following built-in attributes and they can be accessed using dot
operator like any other attribute
__dict__ − Dictionary containing the class's namespace.
__doc__ − Class documentation string or none, if undefined.
__name__ − Class name.
__module__ − Module name in which the class is defined. This attribute is "__main__" in
interactive mode.
__bases__ − A possibly empty tuple containing the base classes, in the order of their
occurrence in the base class list.
#--run program from workspace

Class Inheritance
Instead of starting from a scratch, you can create a class by deriving it from a pre-existing class
by listing the parent class in parentheses after the new class name.
The child class inherits the attributes of its parent class, and you can use those attributes as if
they were defined in the child class. A child class can also override data members and methods
from the parent.
#--Syntax :
class SubClassName (ParentClass1[, ParentClass2, ...]):
'Optional class documentation string'
class_suite
#--run program from workspace
In a similar way, you can drive a class from multiple parent classes as follows −
class A: # define your class A.....
class B: # define your class B.....
class C(A, B): # subclass of A and B.....
Overriding Methods
You can always override your parent class methods. One reason for overriding parent's methods
is that you may want special or different functionality in your subclass.
class Parent: # define parent class

def dummy(self):
print('Calling parent method')
class Child(Parent): # define child class

def dummy(self):
print('Calling child method')
c = Child() # instance of child

c.dummy() # child calls overridden method

Case Study : A Fraction Class
A very common example to show the details of implementing a user-defined class is to construct
a class to implement the abstract data type Fraction.
We know that Python has number of numeric classes for our use.
(Reference : https://docs.python.org/3/library/numeric.html ).
There are times, however, that it would be most appropriate to be able to create data objects
that "look like" fractions.
A fraction such as 3/5 consists of two parts. The top value, known as the numerator, can be any
integer. The bottom value, called the denominator, can be any integer greater than 0 (negative
fractions have a negative numerator).
Although it is possible to create a floating point approximation for any fraction, in this case we
would like to represent the fraction as an exact value.
The operations for the Fraction type will allow a Fraction data object to behave like any other
numeric value. We need to be able to add, subtract, multiply, and divide fractions. We also
want to be able to show fractions using the standard "slash" form, for example 3/5. In addition,
all fraction methods should return results in their lowest terms so that no matter what
computation is performed, we always end up with the most common form.

#--Implement the abstract data type "Fraction" e.g.: 3/5
#--version 1:
class Fraction:
def __init__(self,top,bottom):
self.num = top
self.den = bottom
myfraction = Fraction(3,5)
#--if you print the object

of Fraction class
print(myfraction)
#--it prints moduleName.className at <addr>

#--object always prints its address

#--version 2:
#--adding methods to print the object as string
class Fraction:
self.num = top
self.den = bottom
def show(self):
print(self.num,"/",self.den)
def __str__(self):
return str(self.num) + "/" + str(self.den)
myfraction = Fraction(2,5)
#--call the show()
myfraction.show()
#--print the object directly. It calls __str__ method
print(myfraction)

#--version 3.a:
#--check whether the fraction objects are equal or not
#--this code checks object references only
class Fraction:
self.num = top
self.den = bottom
def __str__(self):
return str(self.num)+"/"+str(self.den)
x = Fraction(1,3)
y = Fraction(1,3)
print(x == y) #shallow comparison or shallow equality
Understand the difference between

shallow comparison or shallow equality vs. Deep equality or Deep comparison

#--version 3.b:
#--this code checks object contents i.e. Deep comparison
#--overload __eq__ method
class Fraction:
self.num = top
self.den = bottom
def __str__(self):
return str(self.num)+"/"+str(self.den)
def __eq__(self, other):

firstnum = self.num * other.den
secondnum = other.num * self.den
return firstnum == secondnum
x = Fraction(1,3)
y = Fraction(1,3)
print(x == y)
We can override many other methods for our new Fraction class. Some of the most important
of these are the basic arithmetic operations. We would like to be able to create two Fraction
objects and then add them together using the standard “+” notation. At this point, if we try to
add two fractions, we get the following:
If you look closely at the error, you see that the problem is that the "+" operator does not
understand the Fraction operands. We can fix this by providing the Fraction class with a method
that overrides the addition method. In Python, this method is called __add__ and it requires
two parameters. The first, self, is always needed, and the second represents the other operand
in the expression. For example, would ask the Fraction object f1 to add the
Fraction object f2 to itself. This can be written in the standard notation, f1+f2.

#--version 4:
#--overload the built-in method __add__ to add 2 fractions
class Fraction:
#-- constructor , show and __str__ methods
def __add__(self,otherfraction):
newnum = self.num*otherfraction.den + self.den*otherfraction.num
newden = self.den * otherfraction.den
return Fraction(newnum,newden)
f_one = Fraction(1,4) Why this calculation ?

f_two = Fraction(1,5) Ans> see next slide
f_three = f_one + f_two
print(f_three)

Two fractions must have the same denominator to be added. The easiest way to make sure
they have the same denominator is to simply use the product of the two denominators as
a common denominator so that
The addition function returns a new Fraction object with the numerator and denominator
of the sum. We can use this method by writing a standard arithmetic expression involving
fractions, assigning the result of the addition, and then printing our result.

The best-known algorithm for finding a
greatest common divisor is Euclid’s Algorithm
Euclid’s Algorithm states that the greatest common

divisor of two integers m and n is n if n divides m
evenly.
However, if n does not divide m evenly, then the

answer is the greatest common divisor of n and the
remainder of m divided by n. We will simply provide
an iterative implementation here (see code).
Note that this implementation of the GCD algorithm

only works when the denominator is positive.

#--version 5:
#--after adding the fractions, the result should be shown in
reduced form
def gcd(m,n):
#--your code here
class Fraction:
#--your code here

Magic methods
When we override the built-in method __add__ to use + operator, we are able to work with
Fraction object as normal Numbers. Here __add__ is called magic method.
Their are many such

magic methods.

Coding Exercise : Overloading Magic Methods
#--Complete the code below
class Vector:
def __init__(self, a, b):
self.a = a
self.b = b
def __str__(self):
return 'Vector (%d, %d)' % (self.a, self.b)
#--Your code here.
v1 = Vector(2,10)
v2 = Vector(5,-2)
print("Addition : ", v1 + v2)
print("Subtraction : ", v1 - v2)
print("Multiplication : ", v1 * v2)
print("Division : ", v1/v2)

Data Hiding
An object's attributes may or may not be visible outside the class definition. You need to name
attributes with a double underscore prefix, and those attributes then will not be directly visible
to outsiders.
Python protects those members by internally changing the name to include the class
name. You can access such attributes as object._className__attrName.

Destroying Objects (Garbage Collection)
Python deletes unneeded objects (built-in types or class instances) automatically to free the
memory space. The process by which Python periodically reclaims blocks of memory that no
longer are in use is termed as Garbage Collection.
1. Python's garbage collector runs during program execution and is triggered when an
object's reference count reaches zero.
2. An object's reference count changes as the number of aliases that point to it
changes.
3. An object's reference count increases when it is assigned a new name or placed in
a container (list, tuple, or dictionary).
4. The object's reference count decreases when it is deleted with del, its reference is
reassigned, or its reference goes out of scope.
5. When an object's reference count reaches zero, Python collects it automatically.

Normally one will not notice when the garbage collector destroys an orphaned instance
and reclaims its space. However, a class can implement the special method __del__(),
called a destructor, that is invoked when the instance is about to be destroyed. This
method might be used to clean up any non-memory resources used by an instance.
Example
This __del__()
destructor prints
the class name of an
instance that is
about to be
destroyed −

Module 14 : Inheritance, Overloading and Overriding

We would 1. Inheritance : Examples
learn : 2. pass is keyword : needed when the child class is empty
3. issubclass() or isinstance() functions : to check relationship between
classes
4. Method Overriding in depth
5. Overriding and super()
6. Method Overloading in depth
7. Operator Overloading
8. Solving many coding questions on above concepts
9. Project : Build a inheritance tree of primary logical gates.

10. Multiple Inheritance
11. Be careful of the Diamond Problem
12. Coding questions on Multiple Inheritance

Inheritance
1. Children inherit characteristics from their parents.
2. Similarly, Python child classes can inherit characteristic data and behavior from a parent
class. These classes are often referred to as subclasses and superclasses.

The child class can remain empty
As for most OOP languages, in Python inheritance works through implicit delegation:
when the object cannot satisfy a request, it first tries to forward the request to its ancestors,
following the specific language rules in the case of multiple inheritance.
As you can see the Child class is empty, but since it inherits from Parent , Python takes
charge of routing all method calls. So you may use the get_value() method of Child
objects and everything works as expected.

Q> What is the Error on running the above code without
"pass" k/word ?
#--refer program in workspace
#--Remember get_value() is not a method of child-class !!
Q> Re-define (that is Override) the get_value() method in the

above program.
#--after Overriding
#--the Child class actually contains a get_value() method with a
different implementation (the id of the two functions are
different).

How to check a relationship between 2 classes ?
1> use issubclass() or isinstance() functions to check a relationships of two
classes and instances.
2> issubclass(sub, sup) boolean function returns True, if the given subclass sub is
indeed a subclass of the superclass sup.
3> isinstance(obj, Class) boolean function returns True, if obj is an instance of class
Class or is an instance of a subclass of Class

Method overriding in action
1. In Python method overriding occurs simply defining in the child class a method with
the same name of a method in the parent class.
2. When you define a method in the object you make the latter able to satisfy that
method call, so the implementations of its ancestors do not come in play.

Q> How to call super or original implementation of an Overridden method
in the child class ?
Ans> Use super().methodName(parameters)

Coding Exercise: 1
Extend the class FileReading to define a class FileReadingWithNoBlankLines. Complete your
code as below.
Hint : use strip() to strip out the blank lines.

Method Overloading
We can define method in such a way that there are multiple ways to call it. Given a single
method or function, we can call it with different number of parameters. Depending on the
function definition, it can be called with zero, one, two or more parameters.
Unlike Java : Here in Python we don't define many methods all with the same name, but
difference in no. of parameters for Overloading !!
Remember : You can't have two methods with the same name in Python.

#---Be careful don't overload like this :
class A:
def stackoverflow(self):
print('first method')
def stackoverflow(self, i):
print('second method', i)
ob=A()
ob.stackoverflow(2)
Coding Exercise : 2:
Correctly overload above stackoverflow() method in 2 ways.
#--complete the code in your work space

Method & Operator Overloading
 Many built-in operators are overloaded by overloading some magic methods.
 Same operator behaves differently with change of operands i.e. parameters, see OL1
#--example : OL1
#Adds the two numbers
print(1 + 2)
#Concatenates the two strings

print('Learning' + 'Python')
#Gives the product

print(3 * 2)
Kindly refer list of
#Repeats the string Magic methods from
print('Python' * 3) the previous Chapter

Each built-in function or operator has a special method corresponding to it. For example,
there’s __len__(), corresponding to len(), and __add__(), corresponding to the + operator.
By default, most of the built-ins and
operators will not work with objects
of your classes. Hence Overload
the special or magic or dunder methods
to make Operator’s work over
user-defined class objects.

Write codes to override or implement the following methods
__len__() , __str__() or __repr__(), __bool__(), __add__
to find :
a> len of the String
b> string representation of the object
c> representing the Object as Boolean : True or False
d> Making Your Objects Capable of Being Added Using +

Project :
Build a inheritance tree of primary logical gates.
Build a inheritance tree having the following :

1> top level class : Logic Gate
2> second level class : Binary Gate and Unary Gate
3> last level class :
Under Binary: And , OR Gate
Under Unary : Not Gate
#-- refer workspace for help

Multiple Inheritance
class derive-class(<base class 1>, <base class 2>, ..... <base
class n>):
<class - suite>

In multiple inheritance : be careful of Diamond Problem
class A:
def m(self):
print("m of A called")
class B(A):
def m(self):
print("m of B called")
class C(A):
def m(self):
print("m of C called") Coding Exercise : 4:
class D(C,B): Try running this code.
pass
Also try
x = D() class D(B,C):

x.m() pass

Q> Whats the o/p of this code ?
class A:
def m(self):
print("m of A called")
class B(A):
pass
class C(A):
def m(self):
print("m of C called")
class D(B,C):
pass
x = D()
x.m()

Coding Exercise 6:
Assume method m() is defined for all classes A, B(A), C(A) and
D(B,C).
Now what would be the o/p and learning on running the following
code ? #-- refer workspace for help
#-- making object of sub-sub-class

x = D()
B.m(x) # have to pass some object. self needs some object

C.m(x) # please note you are calling via class-name
x.m() # note you are calling via object now.
# Hence self gets reference of x
#--once again calling m()

x1 = B()
B.m(x1) # have to pass some object

C.m(x1)
x1.m()
Module 15 : Database Programming
Prerequisite : Basic knowhow of Database Terminology
We would https://www.studytonight.com/dbms/overview-of-dbms.php
learn :
a) Download and install Oracle
b) Making connection and fetch one or more records
c) format and print records column wise
d) print all records with their row count
e) using bind variables to fire dynamic queries
f) Different ways of giving bind parameters
g) Use preparedStatement
h) Performing DDL :
i) create a table from python in Oracle
j) drop a table
k) Performing DML :
l) Insert record in the above table
m) Update record
n) Deleting a record
o) Project
#--download Oracle from
refer steps from db.suven.net
#--install the package

pip install cx_Oracle
1. Among the core principles of Python's way of doing things there is a rule about having
high-level interfaces to APIs.
2. The Database API (in this case the Oracle API) is one example.
3. Using the cx_Oracle Python module from Computronix, you can take command over
the Oracle query model while maintaining compatibility with Python Database API
Specification v2.0.

Getting Started
 Before working with queries and cursors, a connection to the database needs to be
established.
 The credentials and data source names can be supplied in one of several ways, with similar
results.

Cursor Objects
You can define an arbitrary number of cursors using the cursor() method of the Connection
object. Simple programs will do fine with just a single cursor, which can be used over and over
again. Larger projects might however require several distinct cursors.
>>> cursor = db.cursor()

Application logic often requires clearly distinguishing the stages of processing a statement
issued against the database. This will help understand performance bottlenecks better and
allow writing faster, optimized code.
The three stages of processing a statement are:
1> Parse (optional)

cx_Oracle.Cursor.parse([statement])
Not really required to be called because SQL statements are automatically parsed at the Execute
stage. It can be used to validate statements before executing them. When an error is detected in
such a statement, a DatabaseError exception is raised with a corresponding error message, most
likely "ORA-00900: invalid SQL statement, ORA-01031: insufficient privileges or ORA-00921:
unexpected end of SQL command."

2> Execute
cx_Oracle.Cursor.execute(statement, [parameters])
This method can accept a single argument - a SQL statement - to be run directly against the
database. Bind variables assigned through the parameters can be specified as a dictionary,
sequence, or a set of keyword arguments. This method returns a list of variable objects if it is a
query, and None when it's not.
3> Fetch (optional)

Only used for queries (because DDL and DCL statements don't return results).
On a cursor that didn't execute a query, these methods will raise an InterfaceError exception.
cx_Oracle.Cursor.fetchall()
Fetches all remaining rows of the result set as a list of tuples. If no more rows are available, it
returns an empty list.
cx_Oracle.Cursor.fetchmany([rows_no])
Fetches the next rows_no rows from the database. In situations where rows_no is greater than
the number of fetched rows, it simply gets the remaining number of rows.
cx_Oracle.Cursor.fetchone() Fetches a single tuple from the database or none if no more rows
are available.
#--fetch all records
import cx_Oracle
con = cx_Oracle.connect('system/admin123@localhost:1521/XE')
cursor = con.cursor()
cursor.execute('SELECT * from employees_record')
print(cursor.fetchall())
con.close()
#-- Note : the o/p is not User friendly

#-- Soln: use pprint()
#-- Write code to try fetchmany() and fetchone()
import cx_Oracle
#--your code here

#---to print records , column wise so that its easier to read.
#--use pprint()
import cx_Oracle
from pprint import pprint
#--your code here

#--Cursor populates only when we first fetch records
from it.
Try this out, to prove the above point.
import cx_Oracle
print("No. of entries :", cursor.rowcount) #would print 0
Remember : cx_Oracle cursors are iterators. These powerful Python structures let you
iterate over sequences in a natural way that fetches subsequent items on demand only.
Costly database select operations naturally fit into this idea because the data only gets
fetched when needed. Instead of creating or fetching the whole result set, you iterate
until the desired value is found or another condition fulfilled.

#--to know all details about the column.
#--run below code.
#--accessing all column description info. of cursor objects
import cx_Oracle
pprint(cursor.description)
con.close()
Detailed information about data types is available through the description attribute of
cursor objects. The description is a list of 7-item tuples where each tuple consists of a
column name, column type, display size, internal size, precision, scale and whether
null is possible. Note that column information is only accessible for SQL statements
that are queries.

Datatypes
During the fetch stage, basic Oracle data types get mapped into their Python equivalents.
cx_Oracle maintains a separate set of data types that helps in this transition. The Oracle -
cx_Oracle - Python mappings are:

Important Note w.r.t datatypes
1. The above data types are usually transparent to the user except for cases
involving Large Objects.
2. Other data types that are not yet handled by cx_Oracle include XMLTYPE
and all complex types.
3. All queries involving columns of unsupported types will currently fail with a
NotSupportedError exception. You need to remove them from queries or
cast to a supported data type.

Bind Variable : Use and its 2 Types
As advertised by Oracle guru Tom Kyte, bind variables are core principles of database
development. They do not only make programs run faster but also protect against SQL injection
attacks. Consider the following queries:
SELECT * FROM emp_details_view WHERE department_id=50

 When run one-by-one, each need to be parsed separately which adds extra overhead to your
application.
 By using bind variables you can tell Oracle to parse a query only once.
cx_Oracle supports binding variables by name or by position.

Binding variables by name
Passing bind variables by name requires the parameters argument of the execute method
to be a dictionary or a set of keyword arguments.
#--query1 and query2 below are equivalent:
When using named bind variables you can check the currently assigned ones using the
bindnames() method of the cursor:
>>> print cursor.bindnames()

['DEPT_ID', 'SAL']
#--run codes from workspace

Binding variables by Position
Passing by position is similar but you need to be careful about naming. Variable names
are arbitrary so it's easy to mess up queries this way.
positional_params = (100, 10000) #use tuple to give values

cursor.execute('SELECT * from employees_record where
department_id= :1 and salary> :2', positional_params)
#--or
#-----can also give direct values
cursor.execute('SELECT * from employees_record where
department_id= :1 and salary> :2',(100, 10000))

Q> How to use the bind variables ?
#--Use bind variables with preparedStatement
positional_params = (100, 10000) #use tuple to give values
cursor.prepare('SELECT * from employees_record where

department_id= :d')
#None->indicates I don't have a new query. Use preparedStatement

r = cursor.execute(None, {'d':100})
pprint(r.fetchall())
Important Pointers
1> When binding, you can first prepare the statement and then execute None with
changed parameters
2> Any number of executions can be involved for prepared statements.
3> Each time you can execute the preparedStatement with a different bind value

Create a table from python in Oracle.
import cx_Oracle
#--Drop table if it already made.

#cursor.execute("DROP TABLE Professional")
# Create table as per requirement
#-- your code here --

Insert the record in the above table.
Note : Insert , update and delete are DML operations
import cx_Oracle
con = cx_Oracle.connect('system', 'admin123',

'localhost:1521/XE')
#-- your code here

Update a record in the above table.
#---Update a record
import cx_Oracle
con = cx_Oracle.connect('system', 'admin123', 'localhost:1521/XE')
#-- your code here

Delete a record in the above table.
#---Delete a record
import cx_Oracle
con = cx_Oracle.connect('system', 'admin123', 'localhost:1521/XE')
#-- your code here

Module 16 : Multithread Programming
We would 1. Use of threads

learn : 2. Creating 2 or more threads
3. Joining threads
4. Synchronizing Threads using threading.Lock()

a) -> acquire() and release()
5. Multi-threaded Priority Queue

a) -> understanding synchronized consumption of data
6. Coding Exercises

What is a Thread and What is multithreading ?
1) Multiprocessing and multithreading is a way of achieving multitasking.
2) In multithreading, the concept of threads is used.
Let us first understand the concept of thread in computer architecture.
In computing, a process is an instance of a computer program that is being executed.

Any process has 3 basic components:
1. An executable program.
2. The associated data needed by the program (variables, work space, buffers, etc.)
3. The execution context of the program (State of process)
A thread is an entity within a process that can be scheduled for execution. Also, it is the smallest
unit of processing that can be performed in an OS (Operating System).
In simple words, a thread is a sequence of such instructions within a program that can be
executed independently of other code. For simplicity, you can assume that a thread is simply a
subset of a process!

Running several threads is similar to running several different programs concurrently, but with
the following benefits −
1. Multiple threads within a process share the same data space with the main thread and can
therefore share information or communicate with each other more easily than if they were
separate processes.
2. Threads are sometimes called light-weight processes and they do not require much memory
overhead; they are cheaper than processes.
A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer
that keeps track of where within its context is it currently running.
1. It can be pre-empted (interrupted).
2. It can temporarily be put on hold (also known as sleeping) while other threads are running -
this is called yielding.
There are two different kind of threads −

 kernel thread
 user thread
Kernel Threads are a part of the operating system, while the User-space threads are not
implemented in the kernel.

A thread contains all its information in a Thread Control Block (TCB):
• Thread Identifier: Unique id (TID) is assigned to every new
thread.
• Stack pointer: Points to thread’s stack in the process. Stack

contains the local variables under thread’s scope.
• Program counter: a register which stores the address of the

instruction currently being executed by thread.
• Thread state: can be running, ready, waiting, start or done.
• Thread’s register set: registers assigned to thread for

computations.
• Parent process Pointer: A pointer to the Process control

block (PCB) of the process that the thread lives on.

Consider the diagram to
understand the relation between
process and its thread:

Multiple threads can exist within one process where:
 Each thread contains its own register set and local variables (stored in stack).
 All thread of a process share global variables (stored in heap) and the program code.
Consider the diagram below to understand how multiple threads exist in memory:

There are two modules which support the usage of threads in Python3 −
1. _thread
2. threading
 The thread module has been "deprecated" for quite a long time.
 Users are encouraged to use the threading module instead.
 Hence, in Python 3, the module "thread" is not available anymore.
 However, it has been renamed to "_thread" for backwards compatibilities in Python3.
#--run the first thread program from workspace

#--Here we are
1> creating 2 threads
2> finding active threads by activeCount()
3> joining the main thread with the 2 child threads using join()

The threading module exposes all the methods of the thread module and provides some
additional methods −
1. threading.activeCount() − Returns the number of thread objects that are active.
2. threading.currentThread() − Returns details of the current thread.
3. threading.enumerate() − Returns a list of all thread objects that are currently active.
In addition to the methods, the threading module has the Thread class that implements
threading. The methods provided by the Thread class are as follows −
1. run() − The run() method is the entry point for a thread.

2. start() − The start() method starts a thread by calling the run method.
3. join([time]) − The join() waits for threads to terminate.
4. isAlive() − The isAlive() method checks whether a thread is still executing.
5. getName() − The getName() method returns the name of a thread.
6. setName() − The setName() method sets the name of a thread.

threading.currentThread() − Returns details of the current thread
#--creating threads and printing its details
import threading
class myThread (threading.Thread):

def __init__(self):
threading.Thread.__init__(self)
def run(self):
print(threading.currentThread())
Note:
# Create new threads What does the no. in the o/p of
thread1 = myThread() currentThread() indicate?
thread2 = myThread()
# Start new Threads That's the actual Id of the thread.
thread1.start()
thread2.start()

#--Running 2 threads with a delay
#--for delay use module time
#--time.sleep(delay_in_secs)
import threading
import time
class myThread (threading.Thread):
def __init__(self, threadID, name, counter):
#--refer in workspace--
def run(self):
#--refer in workspace--
def print_time(threadName, delay, counter):

while counter:
if exitFlag:
threadName.exit()
time.sleep(delay)
print ("%s: %s" % (threadName, time.ctime(time.time())))
counter -= 1
Synchronizing Threads
The threading module provided with Python includes a simple-to-implement locking mechanism
that allows you to synchronize threads. A new lock is created by calling the Lock() method,
which returns the new lock.
The acquire(blocking) method of the new lock object is used to force the threads to run
synchronously. The optional blocking parameter enables you to control whether the thread
waits to acquire the lock.
If blocking is set to 0, the thread returns immediately with a 0 value if the lock cannot be
acquired and with a 1 if the lock was acquired. If blocking is set to 1, the thread blocks and
wait for the lock to be released.
The release() method of the new lock object is used to release the lock when it is no longer
required.
#--Refer code from workspace

#--Synchronizing Threads using threading.Lock()

Simple Points to remember about Lock :
1. Locks are perhaps the simplest synchronization primitives in Python.
2. A Lock has only two states : locked and unlocked.
3. It is created in the unlocked state and has two principal methods - acquire() and
release().
4. The acquire() method locks the Lock and blocks execution until the release()
method in some other co-routine sets it to unlocked.
5. The release() method should only be called in the locked state, it sets the state to
unlocked and returns immediately.
6. If release() is called in the unlocked state, a RunTimeError is raised.
List of synchronization primitives in Python, defined in it’s standard threading.py module :

Locks, RLocks, Semaphores, Events, Conditions and Barriers

Quick Read on RLocks
The standard Lock doesn’t know which thread is currently
holding the lock. If the lock is held, any thread that
attempts to acquire it will block, even if the same thread
itself is already holding the lock.
In such cases, RLock (re-entrant lock) is used.

Quick Read on Semaphores
Semaphores are simply advanced counters.
An acquire() call to a semaphore will block only after a
number of threads have acquire()ed it. The associated
counter decreases per acquire() call, and increases per
release() call.
A ValueError will occur if release() calls try to increment
the counter beyond it’s assigned maximum value (which is the
number of threads that can acquire() the semaphore before
blocking occurs).

Multithreaded Priority Queue
The Queue module allows you to create a new queue object that can hold a specific number of
items. There are following methods to control the Queue −
1. get() − The get() removes and returns an item from the queue.
2. put() − The put adds item to a queue.
3. qsize() − The qsize() returns the number of items that are currently in the queue.
4. empty() − The empty( ) returns True if queue is empty; otherwise, False.
5. full() − the full() returns True if queue is full; otherwise, False.
#--Refer code from workspace

#--understanding synchronized consumption of data

#--Space for noting points w.r.t Multithreaded Priority Queue Program

Another way of Thread programming :
1. Without creating a sub-class
2. Without explicitly coding run()
import threading
def print_cube(num):
print("Cube: {}".format(num * num * num))
def print_square(num):
print("Square: {}".format(num * num))
if __name__ == "__main__":
# creating thread
t1 = threading.Thread(target=print_square, args=(10,))
t2 = threading.Thread(target=print_cube, args=(10,))
# starting thread 1
t1.start()
# starting thread 2
t2.start() #--explanation on next slide

Important Pointers:
t1 = threading.Thread(target=print_square, args=(10,))
#--Actually the threading module is based on _thread module.
#--when we create a thread , internally, below method is called
_thread.start_new_thread(function, args[, kwargs] )
#--here target indicates the function we are calling
#--args is an iterable tuple, hence we provide a trailing ,(comma)

Consider the diagram below for
a better understanding of how
above program works:
main starts both the threads:

t1 -> print_cube(num)
t2 -> print_square(num)

 Every thread belongs to a process.
 Which means that we can always find to which process our thread belongs.
Code a script to find the process Id to which a thread belongs.
import threading
import os
def task1():
print("Task 1 assigned to thread:
{}".format(threading.current_thread().name))
print("ID of process running task 1: {}".format(os.getpid()))
#--------your code here---------------------
if __name__ == "__main__":
# print ID of current process
print("ID of process running main program: {}".format(os.getpid()))
# print name of main thread

print("Main thread name: {}".format(threading.main_thread().name))
# creating and starting threads

#----------------your code here------------------
Refer this diagram
for better
understanding

Find output
import threading o/p :
def worker():
"""thread worker function"""
print('Worker')
return
threads = []
for i in range(5):
t = threading.Thread(target=worker)
t.start()
Q1> Is the threads array used in any way in the above program ?
A>

Find output
import threading
def worker(num):
"""thread worker function"""
print('Worker: %s' % num)
return
threads = []
for i in range(5):
t = threading.Thread(target=worker, args=(i,))
threads.append(t)
t.start()
o/p : Q2> Is the threads array is used in any way

in the above program ?
A>

Find output
import threading
import time
def worker():
print(threading.currentThread().getName(), 'Starting')
time.sleep(2)
print(threading.currentThread().getName(), 'Exiting')
def my_service():
print(threading.currentThread().getName(), 'Starting')
time.sleep(3)
print(threading.currentThread().getName(), 'Exiting')
t = threading.Thread(name='my_service', target=my_service)
w = threading.Thread(name='worker', target=worker)
w2 = threading.Thread(target=worker) # uses default name
w.start()
w2.start()
t.start()

o/p :
Q3> Is the sequence of o/p’s always the same or dependent on scheduling of

the threads ?
A>

Find output
class MyThread(threading.Thread):
def run(self):
print('running')
return
for i in range(5):
t = MyThread()
t.start()
Q4> Which class is MyThread extending from ?

A>

Find output
class MyThread(threading.Thread):
def run(self):
print('running')
return
for i in range(5):
t = MyThread()
t.start()
Q4> Which class is MyThread extending from ?

A>

Write your observations
#--Refer Workspace
Q5> Run this program. Write your observations
#-- A Timer starts its work after a delay, and can be
canceled at any point within that delay time period.
observations : o/p :

Events
The Event synchronization primitive acts as a simple communicator
between threads.
They are based on an internal flag which threads can set() or
clear(). Other threads can wait() for the internal flag to be set().
The wait() method blocks until the flag becomes true.
Conditions
A Condition object is simply a more advanced version of the Event object. It too acts as
a communicator between threads and can be used to notify() other threads about a change
in the state of the program. For example, it can be used to signal the availability of a
resource for consumption.
Other threads must also acquire() the condition (and thus its related lock) before wait()ing
for the condition to be satisfied.
Also, a thread should release() a Condition once it has completed the related actions, so
that other threads can acquire the condition for their purposes.
<Q6> code demonstrates the implementation of a simple producer-consumer problem
with the help of the Condition object.

Write your observations #--Self
Research
#--Refer Workspace on
Producer
Q6> Run this program. Write your observations. Consumer
#--Its a simple solution to producer consumer problem. Problem
observations : o/p :
Additional Reading on Synchronization Primitives:

https://hackernoon.com/synchronization-primitives-in-python-564f89fee732

#--w.r.t Q6 answer the following questions
Q6.1> Explain the purpose of making Condition object ?
#--condition = threading.Condition()
Q6.2> What's the role of wait() call ?

#--cond.wait()
Q6.3> What's the difference between notify() and nofityall() ?

Module 17 : (Introduction to) Functional Programming
We would 1. Two different styles of programming namely :

learn : a) OOP
b) Functional
2. Functional Programming Elements

a) lambda expression
b) map function
c) filter function
d) reduce function

Python supports two different styles of programming :
1> object-oriented ( this is most used in Python Frameworks – like Django)
2> functional ( crisply used during machine learning programming )
1. As an object-oriented language we use classes(Pre-defined or
User defined), and make objects of them. Till now 95% code
examples in this course of Python(Core + advanced) is using
OOP.
2. Languages like Java can make it hard to move away from

object-oriented thinking, but Python makes it easy. Another
approach of programming in Python is Functional programming.
Functional programming gets its name from writing functions

which provides the main source of logic in a program.

Comparing object-oriented to functional
The easiest way to introduce functional programming is to compare it to something we’re
already aware of: object-oriented programming.
Suppose we wanted to create a line counter class that takes in a file, reads each line, then
counts the total amount of lines in the file. Using a class, it could look something like the
following:

OOP concepts :
1. Methods and Properties.
2. The properties set and retrieve the state of the object, and
3. The methods manipulate that state.
OO Pointers w.r.t Program :

1> Here read() and count() are methods.
2> and lines[] is the property.
3> property values become the state of the object.
4> property values are manipulated by methods.
#-- properties , methods and state change is

explained on next slide

For both these concepts to work, the object’s state must change over time. This change of
state is evident in the lines property after calling the read() method.
#--Case 1
#--example_file.txt contains 100 lines.
lc = LineCounter('example_file.txt')
print(lc.lines)
>> [] The lc object must read the
print(lc.count()) file to set the lines property.
>> 0
#--Case 2
lc.read()
#--The lc.lines property has been changed.
#--This is called changing the state of the lc object.
print(lc.lines)
>> [['Hello world!', ...]]
print(lc.count())
>> 100

Moving towards Functional programming
The alternative is to build the line counter as a series of independent (and pure*) functions.
[after this below program we will understand difference between Pure and Impure functions]
When we only use functions, we are applying a functional approach

to programming which is , called functional programming.

Very Important Point w.r.t Functional Programming
1. The concepts behind functional programming requires functions to be stateless, and rely
only on their given inputs to produce an output.
2. The functions that meet the above criteria are called pure functions.
3. Here’s an example to highlight the difference between pure functions, and non-pure:
# Create a global variable `A`.
A = 5
def impure_sum(b):
# Adds two numbers, but uses the global `A` variable.
return b + A
def pure_sum(a, b):
# Adds two numbers, using ONLY the local function inputs.
return a + b
1. The benefit of using pure functions
print(impure_sum(6)) over impure (non-pure) functions is the
>> 11 reduction of side effects.
2. Side effects occur when there are
print(pure_sum(4, 6)) changes performed within a function’s
>> 10 operation that are outside its scope.

Functional Programming Elements
The Lambda Expression
Instead of the def syntax for function declaration, we can use a lambda expression to
write Python functions. The lambda syntax closely follows the def syntax, but it’s not a 1-
to-1 mapping.
Note : When assigning the lambda expression to a variable, it acts

exactly like a Python function, and can be called using the function
call syntax: new_add().

The Map Function
While the ability to pass in functions as arguments is not unique to Python, it is a recent
development in programming languages. Functions that allow for this type of behavior are
called first-class functions. Any language that contains first-class functions can be written in a
functional style.
There are a set of important first-class functions that are commonly used within the functional
paradigm. These functions take in a Python iterable, and, like sorted(), apply a function for each
element in the list.
We will examine three functions: map(), filter() and reduce()
All follow the general form :

function_name(function_to_apply, iterable_of_elements).

• The map() function takes in an iterable (i.e. list), and creates a new iterable object,
a special map object.
• The new object has the first-class function applied to every element.
#--Refer map example from workspace

#--map() example: where we are adding some constant value to each member of the list.

The Filter Function
The filter() function takes 2 parameters :

1> evaluate function
2> an iterable object like list
#--Refer filter The return value is a Map which contains only
example from those values, for which evaluate function
workspace returns true.

The Reduce Function
1. The last function we’ll look at is the reduce() function from the functools package.
2. The reduce() function takes in an iterable, and then reduces the iterable to a single value.
3. Reduce is different from filter() and map(), because reduce() takes in a function that has two
input values.

Internship Project Specifications
Project: Phone Number and Email Address Extractor
Problem Definition :
1. Say you have the boring task of finding every phone number
and email address in a comments section of a popular blog.
2. If you manually scroll through the page, you might end up
searching for a long time. But if you had a program that
could search the text in your clipboard for phone numbers
and email addresses, you could simply press ctrl-A to select
all the text, press ctrl-C to copy it to a .txt file, and
then run your program.
3. It would fetch just the phone numbers and email addresses it
finds and write them to another .txt file.
4. Code such a script.

Internship Project Specifications
Project: Combining Select Pages from Many PDFs
Problem Definition :
Say you have task of merging several dozen PDF documents into a
single PDF file. Each of them has a cover sheet as the first
page, but you don’t want the cover sheet repeated in the final
result.
You are expected to code a Python program to customize which

pages you want in the combined PDF.
At a high level, here’s what your code should do:

1> Find all PDF files in the current working directory.
2> Sort the filenames so the PDFs are added in order.
3> Write each page, excluding the first page, of each PDF to the
output file.

After this python course, you should be able to :
1. Code and Understand Python projects.
2. Understand any advanced level course on Machine Learning

or Deep Learning i.e. Neural Networks (using Python).
3. Understand and Perform Data Analytics to Explore data ,

Visualize data and make Predictions.
4. Automate your routine business task.

Contact Details & Subject Syllabus Downloads
Can download Syllabus of any course from
1 https://datascience.suven.net
2 Call us on 9870014450 for any inquiries.
Chat with Rocky Sir on 9892544177 for

3 resolving any confusion.
Enroll at any training Centre :

4 Thane, Dadar, Chembur & Borivali
5 For Updates , Connect with Rocky Sir on Linked-In

Python Notes 2020

Uploaded by

Python Notes 2020

Uploaded by

Course contents of Python Programming Certification Course

After this python course, I will be able to :

Python Programming(Core+ Advanced) By Rocky Sir 2

Module 7 Regex and its daily applications

 It was created by Guido van Rossum

 Python is named after a TV Show called

Step 2: Run the Installer

save your file as sctpl.py. In the folder “suven”

5> run sctpl.py like this : Script Mode Programming

Python Programming(Core+ Advanced) By Rocky Sir 9

We would 1. print() statement

Note : In python, be careful of indentation error.

Python Programming(Core+ Advanced) By Rocky Sir 10

Lines and Indentation

Python Programming(Core+ Advanced) By Rocky Sir 12

Multiple Statements on a Single Line

print("; acts as a separator"); print("I am second")

Python Programming(Core+ Advanced) By Rocky Sir 13

Getting input from the user : via command-line arguments

Here sys.argv[0] is the program i.e. the script name.

Python Programming(Core+ Advanced) By Rocky Sir 14

Python Programming(Core+ Advanced) By Rocky Sir 15

Python Programming(Core+ Advanced) By Rocky Sir 16

Python has six standard data types −

Python Programming(Core+ Advanced) By Rocky Sir 17

Python Programming(Core+ Advanced) By Rocky Sir 20

Python Programming(Core+ Advanced) By Rocky Sir 21

Python Programming(Core+ Advanced) By Rocky Sir 22

Python Programming(Core+ Advanced) By Rocky Sir 23

thisset = {"Suven", "Rocky", "Lokesh", "ML", "Neural Networks", "DL"}

Python Programming(Core+ Advanced) By Rocky Sir 24

Python Programming(Core+ Advanced) By Rocky Sir 25

Python Programming(Core+ Advanced) By Rocky Sir 26

Python Programming(Core+ Advanced) By Rocky Sir 27

Python Programming(Core+ Advanced) By Rocky Sir 28

Python Programming(Core+ Advanced) By Rocky Sir 29

7. Python Identity Operators

Python Programming(Core+ Advanced) By Rocky Sir 30

Python Programming(Core+ Advanced) By Rocky Sir 31

if cars > people:

Python Programming(Core+ Advanced) By Rocky Sir 32

A loop statement allows us to execute a statement or group of statements multiple times.

Python Programming(Core+ Advanced) By Rocky Sir 33

for number in the_count:

for fruit in fruits:

Python Programming(Core+ Advanced) By Rocky Sir 34

Python supports the following control statements.

Python Programming(Core+ Advanced) By Rocky Sir 36

String, List or Tuple objects can be used to create an Iterator.

Python Programming(Core+ Advanced) By Rocky Sir 37

Python Programming(Core+ Advanced) By Rocky Sir 39

You are been given with following rates

and your script should ask for following inputs:

Python Programming(Core+ Advanced) By Rocky Sir 40

Check out chapter_3s.py file

We would 1. Number build-in functions

Python Programming(Core+ Advanced) By Rocky Sir 41

Python Programming(Core+ Advanced) By Rocky Sir 42

Python Programming(Core+ Advanced) By Rocky Sir 43

Python Programming(Core+ Advanced) By Rocky Sir 44

Python Programming(Core+ Advanced) By Rocky Sir 45

Python Programming(Core+ Advanced) By Rocky Sir 46

Python Programming(Core+ Advanced) By Rocky Sir 47

Python Programming(Core+ Advanced) By Rocky Sir 48

Python Programming(Core+ Advanced) By Rocky Sir 49

Python also includes the following list methods −

Python Programming(Core+ Advanced) By Rocky Sir 50

Python Programming(Core+ Advanced) By Rocky Sir 51

Python Programming(Core+ Advanced) By Rocky Sir 52

Python Programming(Core+ Advanced) By Rocky Sir 53

# using zip() to map values

# converting values to print as set

# printing resultant values

Python Programming(Core+ Advanced) By Rocky Sir 55

#--application of zip in comparing sequences

#-- another way :