Notes For Python Part I
Notes For Python Part I
What is Python?
Anaconda includes both the core Python interpreter and standard libraries as
well as most modules required for data analysis.
1 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
What is Python?
2 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Anaconda Navigator
3 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Anaconda
4 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Anaconda Navigator
The standard Python interactive console is very basic and does not support
useful features such as tab completion. The QtConsole version of IPython,
transforms the console into a highly productive environment.
The jupyter notebook is a simple and useful method to share code with others.
The primary method for using notebooks is through a web interface, which
allows creation, deletion, export and interactive editing of notebooks.
5 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Anaconda Navigator
6 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
One of the installed module is numpy. To access the squared root function of
numpy, one of the following can be used:
import numpy # first option for importing numpy
numpy . sqrt (3) # return the squared root of 3
import numpy as np # second option for importing numpy
np . sqrt (3) # return the squared root of 3
from numpy import * # third option for importing numpy
sqrt (3) # return the squared root of 3
In the last case, (from numpy import*), all functions in numpy become
available.
Any name following a dot is called an attribute of the object to the left of the
dot:
attributes can contain auxiliary information regarding to the object
attributes can act like functions, called methods
7 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
8 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Python, by default, only has access to a small number of built-in types and
functions. The vast majority of functions are located in modules, and before a
function can be accessed, the module which contains the function must be
imported.
Some important modules are
numpy, scipy, matplotlib, seaborn, pandas and statsmodels.
Numpy provides a set of array and matrix data types which are essential for
statistics, econometrics and data analysis.
Scipy contains a large number of routines needed for analysis of data. The
most important include a wide range of random number generators, linear
algebra routines, and optimizers. scipy depends on numpy.
9 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Pandas provides fast, flexible, and expressive data structures designed to make
working with “relational” or “labeled” data both easy and intuitive.
Statsmodels provides classes and functions for the estimation of many different
statistical models, as well as for conducting statistical tests, and statistical data
exploration.
10 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Boolean
String
List
Tuple
Dictionary
Set
Range
11 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Numeric
The Boolean data type is used to represent true and false, using the reserved
keywords True and False.
In : x = True In : bool (1) In : bool (0)
In : type ( x ) Out : True Out : False
Out : bool
Strings are delimited using single quotes (’ ’) or double quotes (" ").
In : x = ' This is a string '
In : type ( x )
Out : str
In : print ( x )
Out : This is a string
13 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Slicing Strings
Substrings within a string can be accessed using slicing. Slicing uses [ ] to
contain the indices of the characters in a string. Suppose that s has n
characters. Python uses 0−based indices.
14 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Lists
Basic list slicing is identical to slicing strings, and operations such as x[:],
x[1:], x[:1] and x[3:] can all be used.
15 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Slicing lists
16 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Slicing lists
A number of functions are available for manipulating lists. The most useful are
In : x = [0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9]
In : x . append (0)
In : x
Out : [0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 0]
In : x . extend ([11 ,12 ,13])
In : x
Out : [0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 0 , 11 , 12 , 13]
In : x . pop (1)
Out : 1
In : x
Out : [0 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 0 , 11 , 12 , 13]
In : x . remove (0)
In : x
Out : [2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 0 , 11 , 12 , 13]
17 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Tuples (tuple)
18 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Dictionary (dict)
19 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Sets
Sets are collections which contain all unique elements of a collection. set and
frozenset only differ in that the latter is immutable.
In : x = set ([ ' MSFT ' , ' GOOG ' , ' AAPL ' , ' HPQ ' , ' MSFT ' ])
In : x
Out : { ' AAPL ' , ' GOOG ' , ' HPQ ' , ' MSFT ' }
In : x . add ( ' CSCO ' )
In : x
Out : { ' AAPL ' , ' CSCO ' , ' GOOG ' , ' HPQ ' , ' MSFT ' }
In : y = set ([ ' XOM ' , ' GOOG ' ])
In : x . intersection ( y )
Out : { ' GOOG ' }
In : x = x . union ( y )
In : x
Out : { ' AAPL ' , ' CSCO ' , ' GOOG ' , ' HPQ ' , ' MSFT ' , ' XOM ' }
range
21 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Arrays
Arrays are the base (homogeneous) data type in NumPy, are in similar to lists
or tuples since they both contain collections of elements.
The main data structure for multidimensional arrays in NumPy is the ndarray
class.
In addition to the data stored in the array, this data structure also contains
important metadata about the array, such as its shape, size, data type, and
other attributes.
Attribute Description
shape A tuple that contains the number of elements (i.e., the length) for each dimension
(axis) of the array.
size The total number elements in the array.
ndim Number of dimensions (axes).
nbytes Number of bytes used to store the data.
dtype The data type of the elements in the array.
22 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Arrays
Arrays are initialized from lists (or tuples) using array(). Higher
dimensional arrays can be initialized by nesting lists or tuples.
In : import numpy as np
In : x = [0.0 , 1 , 2 , 3 , 4]
In : y = np . array (x , dtype = int )
In : y
Out : array ([0. , 1. , 2. , 3. , 4.])
In : type ( y )
Out : numpy . ndarray
In : y = np . array ([[0.0 , 1 , 2 , 3 , 4] , [5 , 6 , 7 , 8 , 9]])
In : y
Out :
array ([[0. , 1. , 2. , 3. , 4.] ,
[5. , 6. , 7. , 8. , 9.]])
In : np . shape ( y )
Out : (2 , 5)
In : y = np . array ([[[1 ,2] ,[3 ,4]] ,[[5 ,6] ,[7 ,8]]])
In : y
Out :
array ([[[1 , 2] ,
[3 , 4]] ,
[[5 , 6] ,
[7 , 8]]])
In : np . shape ( y )
Out : (2 , 2 , 2)
23 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Arrays
24 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Matrices
25 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
26 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
27 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Four methods are available for accessing elements contained within an array:
scalar selection, slicing, numerical indexing and logical (or Boolean) indexing
# scalar selection
In : x = np . array ([1.0 ,2.0 ,3.0 ,4.0 ,5.0])
In : x [0]
In : x = np . array ([[1.0 ,2 ,3] ,[4 ,5 ,6]])
In : x [1 ,2]
In : type ( x [1 , 2])
In : x = np . array ([1.0 ,2.0 ,3.0 ,4.0 ,5.0])
In : x [0] = -5
# slicing
In : x = np . array ([1.0 ,2.0 ,3.0 ,4.0 ,5.0])
In : y = x [:]
In : y = x [:2]
In : y = x [1::2] # starting from 1 every second element
# 2 dimensional array
In : y = np . array ([[0.0 , 1 , 2 , 3 , 4] ,[5 , 6 , 7 , 8 , 9]])
In : y [:1 , :] # Row 0 , all columns
In : y [: , :1] # all rows , column 0
In : y [:1 , 0:3] # Row 0 , columns 0 to 2
In : y [:1][: , 0:3] # Same as previous
In : y [: , 3:] # All rows , columns 3 and 4
# 3 dimensional array
In : y = np . array ([[[1.0 ,2] ,[3 ,4]] ,[[5 ,6] ,[7 ,8]]])
In : y [0 , : , :] # Panel 0
In : y [1 , : , :] # Panel 1
In : y [0 , 0 , 0] # Row 0 and column 0 of Panel 0
In : y [1 , 1 , 0] # Row 1 and column 0 of Panel 1
28 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
When elements in a view are assigned new values, the values of the original
array are therefore also updated.
In : f = lambda m , n : n + 10* m
In : A = np . fromfunction (f , (6 , 6) , dtype = int )
In : A
Out : array ([[ 0 , 1 , 2 , 3 , 4 , 5] ,
[10 , 11 , 12 , 13 , 14 , 15] ,
[20 , 21 , 22 , 23 , 24 , 25] ,
[30 , 31 , 32 , 33 , 34 , 35] ,
[40 , 41 , 42 , 43 , 44 , 45] ,
[50 , 51 , 52 , 53 , 54 , 55]])
In : B = A [1:5 , 1:5]
In : B [: ,:] = 0
In : A
Out : array ([[ 0 , 1 , 2 , 3 , 4 , 5] ,
[10 , 0 , 0 , 0 , 0 , 15] ,
[20 , 0 , 0 , 0 , 0 , 25] ,
[30 , 0 , 0 , 0 , 0 , 35] ,
[40 , 0 , 0 , 0 , 0 , 45] ,
[50 , 51 , 52 , 53 , 54 , 55]])
When a copy rather than a view is needed, the view can be copied explicitly by
using the copy method of the ndarray instance.
In : C = B [1:3 , 1:3]. copy ()
In : C [: ,:] = 1
In : C
29 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
With fancy indexing, an array can be indexed with another NumPy array, a
Python list, or a sequence of integers, whose values select elements in the
indexed array.
In : A = np . linspace (0 , 1 , 11)
In : A [ np . array ([0 , 2 , 4])]
Out : array ([ 0. , 0.2 , 0.4])
In : A [[0 , 2 , 4]]
Out : array ([ 0. , 0.2 , 0.4])
Unlike arrays created by using slices, the arrays returned using fancy indexing
and Boolean-valued indexing are not views but rather new independent arrays.
30 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
31 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
32 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Special Arrays
M, N = 5, 5
x = np . ones (( M , N )) # M by N array of 1 s
x = np . ones (( M ,M , N )) # 3 D array
x = np . zeros (( M , N )) # M by N array of 0 s
x = np . zeros (( M ,M , N )) # 3 D array of 0 s
x = np . empty (( M , N )) # M by N empty array
x = np . empty (( N ,N ,N , N )) # 4 D empty array
x = np . empty (( M , N ) , dtype = ' float32 ' ) # 32 - bit floats ( single precision )
x = np . full (( M , N ) , c ) # dtype of c
x = np . eye ( N ) # N by N identity array
x = np . eye (N , k = 1) # N by N identity array above the diagonal
x = np . diag ( np . arange (0 , 20 , 5)) # diagonal array
33 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
34 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
35 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
36 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
In : X = np . array ([[1.0 ,2.0 ,3.0] ,[3.0 ,3.0 ,4.0] ,[1.0 ,1.0 ,4.0]])
In : y = np . array ([[1.0] ,[2.0] ,[3.0]])
In : np . linalg . solve (X , y )
Out :
array ([[ 0.625] ,
[ -1.125] ,
[ 0.875]])
In : x = np . matrix ([[1 ,.5] ,[.5 ,1]])
In : C = np . linalg . cholesky ( x )
In : C * C . T - x
Out :
matrix ([[ 0.00000000 e +00 , 0.00000000 e +00] ,
[ 0.00000000 e +00 , -1.11022302 e -16]])
37 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Basic functions
Some functions from numpy are given in the following tables.
38 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Basic functions
39 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
40 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Logical operators
41 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Logical operators
42 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Logical operators
In : x = np . array ([[1 ,2] ,[ -3 , -4]])
In : x > 0
Out :
array ([[ True , True ] ,
[ False , False ]])
In : x == -3
Out :
array ([[ False , False ] ,
[ True , False ]])
In : x = np . arange (2.0 ,4)
In : y = x >= 0
In : y
Out : array ([ True , True ])
In : z = x <2
In : z
Out : array ([ False , False ])
In : np . logical_and (y , z )
Out : array ([ False , False ])
In : y & z
Out : array ([ False , False ])
In : ~( y & z )
Out : array ([ True , True ])
In : np . logical_not ( y & z )
Out : array ([ True , True ])
In : import math
In : x = np . array ([4 , math . pi , math . inf , math . inf / math . inf ])
In : x
Out : array ([4. , 3.14159265 , inf , nan ])
In : np . isnan ( x )
Out : array ([ False , False , False , True ])
In : np . isinf ( x )
Out : array ([ False , False , True , False ])
43 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Control Structures
Python uses white space changes to indicate the start and end of flow control
blocks, and so indentation (4 spaces) matters.
if...elif...else
for
while
try...except
List, tuple, dictionary and set comprehension
44 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
if...elif...else
The structure of if...elif...else is in the following form:
if logical_1:
Code to run if logical_1
elif logical_2:
Code to run if logical_2 and not logical_1
elif logical_3:
Code to run if logical_3 and not logical_1 or logical_2
...
...
else:
Code to run if all previous logicals are false
Some examples
In : y = 5
In : if y <5:
...: y += 1 # this is equivalent to y = y +1
...: else :
...: y -= 1 # this is equivalent to y =y -1
In : y
Out : 4
In : x = 5
In : if x <5:
...: x = x + 1
...: elif x >5:
...: x = x - 1
...: else :
...: x = x * 2
In : x
Out : 10
45 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
for loop
46 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
for loop
Some examples
In : x = np . zeros ((5 ,5))
...: for i in range ( np . size (x ,0)):
...: for j in range ( np . size (x ,1)): # nested for loop
...: if i < j :
...: x [i , j ]= i + j
...: else :
...: x [i , j ]= i - j
...:
...: x
Out :
array ([[0. , 1. , 2. , 3. , 4.] ,
[1. , 0. , 3. , 4. , 5.] ,
[2. , 1. , 0. , 5. , 6.] ,
[3. , 2. , 1. , 0. , 7.] ,
[4. , 3. , 2. , 1. , 0.]])
for loops can be used with 2 items when the iterable is wrapped in
enumerate.
In : x = np . linspace (0 ,5 ,3)
...: for i , j in enumerate ( x ):
...: print ( ' i is : ' , i )
...: print ( ' j is : ' , j )
Out :
i is : 0
j is : 0.0
i is : 1
j is : 2.5
i is : 2
j is : 5.0
47 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
while loop
The generic structure of a while loop is
while logical:
Code to run
Update logical
Some examples
In : count = 0
...: i = 1
...: while i <10:
...: count += i
...: i += 1 # equivalent to i = i +1
...:
...: count
Out : 45
In : mu = np . abs (100* np . random . randn (1))
...: index = 1
...: while np . abs ( mu ) > .0001:
...: mu = ( mu + np . random . randn (1))/ index
...: index = index +1
...:
In : mu
Out : array ([ -7.44593523 e -05])
In : index
Out : 162
List comprehension
A simple list can be used to convert a for loop which includes an append into
a single line statement.
In : x = np . arange (3.0)
...: y = []
...: for i in range ( len ( x )):
...: y . append ( np . exp ( x [ i ]))
...:
...: y
Out : [1.0 , 2.718281828459045 , 7.38905609893065]
In : z = [ np . exp ( x [ i ]) for i in range ( len ( x ))] # list comprehension saves 2 lines .
...: z
Out : [1.0 , 2.718281828459045 , 7.38905609893065]
In : x = np . arange (5.0)
...: y = []
...: for i in range ( len ( x )):
...: if np . floor ( i /2)== i /2:
...: y . append ( x [ i ]**2)
...:
...: y
Out : [0.0 , 4.0 , 16.0]
In : z = [ x [ i ]**2 for i in range ( len ( x )) if np . floor ( i /2)== i /2]
...: z
Out : [0.0 , 4.0 , 16.0]
49 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
List comprehension
50 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Functions
To declare a function:
def function_name(arguments):
"""docstring"""
statement(s)
return expression_list
Functions are declared using the def keyword, and the value produced is
returned using the return keyword.
docstring is use to describe what function does. It can be accessed by calling
help(function_name).
0
Consider the Lp distance between two vectors x = (x1 , . . . , xn ) and
0 Pn p 1/p
y = (y1 , . . . , yn ) defined by dp (x, y) = i=1 |xi − yi | .
def lp_norm (x ,y , p ):
"""
This function calculates L_p distance between two vectors
"""
d = ( np . sum ( np . abs (x - y )** p ))**(1/ p )
return d
51 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Functions
The multiple outputs can be returned in tuple form as in the following example.
In : def l1_l2_norm (x , y ):
...: d1 = np . sum ( np . abs (x - y ))
...: d2 = ( np . sum ( np . abs (x - y )**2))**(1/2)
...: return ( d1 , d2 )
In : x = np . random . randn (10)
In : y = np . random . randn (10)
In : # Using 1 output returns a tuple
In : z = l1_l2_norm (x , y )
In : print ( " The L1 distance is " , z [0])
Out : The L1 distance is 5.83427433121689
In : print ( " The L2 distance is " , z [1])
Out : The L2 distance is 2.137922607625371
52 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
Default values are set in the function declaration using the syntax input =
default.
In : def lp_norm (x , y , p = 2):
...: d = ( np . sum ( np . abs (x - y )** p ))**(1/ p )
...: return d
# Call the function
In : x = np . random . randn (10)
In : y = np . random . randn (10)
# Inputs with default values can be ignored
In : l2 = lp_norm (x , y )
In : l1 = lp_norm (x , y , 1)
In : print ( " The l1 and l2 distances are " , l1 , l2 )
Out : The l1 and l2 distances are 10 .201 240 563 996 286 3.911381835158071
In : print ( " Is the default value overridden ? " , sum ( abs (x - y )) == l1 )
Out : Is the default value overridden ? True
53 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
54 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
# Using kwargs
def lp_norm (x , y , p = 2 , ** kwargs ):
d = ( np . sum ( np . abs (x - y )** p ))**(1/ p )
for key , value in kwargs . items ():
print ( ' The value of ' , key , ' is ' , value )
return d
# Call the function
In : x = np . random . randn (10)
In : y = np . random . randn (10)
In : lp = lp_norm (x , y , kword1 = 1 , kword2 = 3.2)
Out :
The value of kword1 is 1
The value of kword2 is 3.2
In : lp
Out : 4 . 1 2 3858213836834
In : lp = lp_norm (x , y , kword1 = 1 , kword2 = 3.2 , p = 1)
The value of kword1 is 1
The value of kword2 is 3.2
In : lp
Out : 9 . 2 1 6016436616611
55 / 56
Intro to Python Built-in Data Types Arrays&Matrices Basic Functions Control Structures Function
56 / 56