Mastering Python 50 Specific Tips For Writing Better Code

Mastering Python: 50 Specific Tips
for Writing Better Code
- Dane Olsen
ISBN: 9798865196815
Ziyob Publishers.
Mastering Python: 50 Specific Tips
for Writing Better Code
Practical Strategies for Writing High-Quality Python Code
Copyright © 2023 Ziyob Publishers

All rights are reserved for this book, and no part of it may be
reproduced, stored in a retrieval system, or transmitted in any form
or by any means without prior written permission from the publisher.
The only exception is for brief quotations used in critical articles or
reviews.
While every effort has been made to ensure the accuracy of the
information presented in this book, it is provided without any
warranty, either express or implied. The author, Ziyob Publishers, and
its dealers and distributors will not be held liable for any damages,
whether direct or indirect, caused or alleged to be caused by this
book.
Ziyob Publishers has attempted to provide accurate trademark
information for all the companies and products mentioned in this book
by using capitalization. However, the accuracy of this information
cannot be guaranteed.
This book was first published in October 2023 by Ziyob Publishers,
and more information can be found at:
www.ziyob.com
Please note that the images used in this book are borrowed, and
Ziyob Publishers does not hold the copyright for them. For inquiries
about the photos, you can contact:
contact@ziyob.com
About Author:
Dane Olsen
Dane Olsen is an experienced software engineer and Python
enthusiast with over a decade of experience developing applications
across a range of industries. As a technology consultant and
educator, he has helped countless developers and businesses adopt
best practices for Python programming.
In "Mastering Python: 50 Specific Tips for Writing Better Code", Dane
distills his extensive experience into a comprehensive guide for
Python developers of all levels. With a focus on practical, real-world
examples, Dane provides insights and best practices for every stage
of the Python development process, from writing clean and efficient
code to designing effective algorithms and data structures.
Dane is a frequent speaker at industry conferences and events,
where he shares his insights on Python programming and software
development. He is also a contributor to open-source projects,
including popular Python libraries, and an active member of the
Python community.
In addition to his work with Python, Dane has extensive experience
with other modern application development technologies, including
web development frameworks and machine learning libraries. He
holds a degree in computer science from a top-ranked university and
is passionate about using technology to solve real-world problems.
Table of Contents
Chapter 1:
Introduction
1. The Zen of Python
2. Pythonic thinking
Chapter 2:
Pythonic thinking
1. Know your data structures
Tuples
Lists
Dictionaries
Sets
Arrays
Queues
Stacks
Heaps
Trees
Graphs
2. Write expressive code
Choosing good names

Avoiding magic numbers and strings
Using list comprehensions and generator expressions
Leveraging built-in functions
Using the with statement
Using decorators
Writing context managers
3. Take advantage of Python's features

Using named tuples
Leveraging closures
Using properties
Using descriptors
Using metaclasses
4. Writing idiomatic Python
Writing Pythonic loops

Using enumerate and zip
Using the ternary operator
Using multiple assignment
Using the walrus operator
Using context managers
Chapter 3:
Functions
1. Function basics
Function arguments and return values

Documenting functions
Writing doctests
Writing function annotations
Using default arguments
Using keyword arguments
Using *args and **kwargs
2. Function design
Writing pure functions

Writing functions with side effects
Writing functions that modify mutable arguments
Using the @staticmethod and @classmethod decorators
Using partial functions
3. Function decorators and closures
Writing simple decorators

Writing decorators that take arguments
Writing class decorators
Using closures
Using functools.partial
Chapter 4:
Classes and Objects
1. Class basics
Creating and using classes
Defining instance methods
Using instance variables
Understanding class vs instance data
Using slots for memory optimization
Understanding class inheritance
Using multiple inheritance
2. Class design
Writing clean, readable classes

Writing classes with a single responsibility
Using composition over inheritance
Using abstract base classes
Writing metaclasses
3. Advanced class topics

Using descriptors to customize attribute access
Using properties to control attribute access
Using the super function
Using slots to optimize memory usage
Chapter 5:
Concurrency and Parallelism
1. Threads and Processes
Understanding the Global Interpreter Lock (GIL)

Using threads for I/O-bound tasks
Using processes for CPU-bound tasks
Using multiprocessing
Using concurrent.futures
2. Coroutines and asyncio
Understanding coroutines
Using asyncio for I/O-bound tasks
Using asyncio for CPU-bound tasks
Using asyncio with third-party libraries
Debugging asyncio code
Chapter 6:
Built-in Modules
1. Collections
Using namedtuple
Using deque
Using defaultdict
Using OrderedDict
Using Counter
Using ChainMap
Using UserDict
Using UserList
Using UserString
2. Itertools
Using count, cycle, and repeat
Using chain, tee, and zip_longest
Using islice, dropwhile, and takewhile
Using groupby
Using starmap and product
3. File and Directory Access

Using os and os.path
Using pathlib
Using shutil
Using glob
4. Dates and Times
Using datetime
Using time
Using timedelta
Using pytz
Using dateutil
5. Serialization and Persistence
Using json
Using pickle
Using shelve
Using dbm
Using SQLite
6. Testing and Debugging
Writing unit tests

Using pytest
Debugging with pdb
Debugging with logging
Using assertions
Chapter 7:
Collaboration and Development
1. Code Quality
Using linters
Using type checkers
Using code formatters
Using docstring conventions
Writing maintainable code
2. Code Reviews
Conducting effective code reviews
Giving and receiving feedback
Improving code quality through reviews
3. Collaboration Tools
Using version control with Git

Using GitHub for collaboration
Using continuous integration
Using code coverage tools
4. Documentation and Packaging
Writing documentation
Using Sphinx
Packaging Python projects
Distributing Python packages
Managing dependencies
Chapter 1:
Introduction
Python is a popular, high-level programming language that is widely
used for web development, scientific computing, artificial intelligence,
data analysis, and many other applications. It is a versatile and
powerful language that offers a lot of flexibility and ease of use to
developers. However, like any other programming language, writing
effective and efficient Python code requires a good understanding of
the language's features and best practices.
"Effective Python: 50 Specific Ways to Write Better Python" is a
comprehensive guide that focuses on providing readers with specific
tips and techniques to improve their Python coding skills. The book
covers a wide range of topics, including data structures, functions,
classes, concurrency, testing, and debugging. Each topic is
presented in a clear and concise manner, with practical examples and
explanations that help readers understand the concepts better.
The book is divided into 50 chapters, each of which covers a specific
aspect of Python programming. The chapters are organized in a
logical and progressive order, with each chapter building upon the
previous one. This makes it easy for readers to follow along and
learn at their own pace.
One of the strengths of the book is its focus on practical examples.
The author, Brett Slatkin, is an experienced Python developer who
has worked at Google for many years. He draws upon his experience
to provide readers with real-world examples that illustrate the
concepts he is explaining. This makes it easy for readers to
understand how the concepts apply to real-world programming
situations.
Another strength of the book is its emphasis on best practices. The
author provides readers with tips and techniques that are widely
accepted as best practices within the Python community. This helps
readers to write code that is more efficient, more maintainable, and
easier to understand.
One of the unique features of the book is its focus on Python 3.
Python 3 is the latest version of the language, and it has many new
features and improvements over Python 2. The author recognizes that
many developers still use Python 2, but he encourages readers to
move to Python 3, as it is a more modern and robust language.
Overall, "Effective Python: 50 Specific Ways to Write Better Python"
is an excellent resource for anyone who wants to improve their
Python coding skills. Whether you are a beginner or an experienced
developer, this book provides valuable insights and techniques that
can help you write better Python code. It is a must-read for anyone
who wants to become a more proficient Python programmer.
The Zen of Python

The Zen of Python is a collection of guiding principles for writing code
in the Python programming language. It was created by Tim Peters, a
well-known Python developer and contributor, and is included as an
Easter egg in the Python interpreter. The Zen of Python provides a
set of rules and guidelines that promote readability, simplicity, and
clarity in Python code.
The Zen of Python provides guidance on several aspects of Python
programming. Let's take a closer look at some of the principles:
"Beautiful is better than ugly": This principle encourages
developers to write code that is visually appealing and easy to
read. This can be achieved by using descriptive variable
names, commenting the code where necessary, and adhering
to a consistent coding style.
"Explicit is better than implicit": This principle encourages

developers to be clear and concise in their code. It's better to
be explicit about what the code does, even if it means writing
a few extra lines of code.
"Simple is better than complex": This principle encourages

developers to write code that is easy to understand and
maintain. This can be achieved by breaking down complex
tasks into smaller, simpler functions or modules.
"Readability counts": This principle emphasizes the importance

of writing code that is easy to read and understand. This can
be achieved by using consistent indentation, commenting the
code where necessary, and adhering to a consistent coding
style.
Let's take a look at some sample code that demonstrates the

principles of the Zen of Python:
# Example 1: Beautiful is better than ugly.

# Instead of using single-letter variable
names, use descriptive names.
# Also, use comments to explain what the
code does.
# Bad code
a=5
b=7
c=a+b
print(c)
# Good code
num1 = 5
num2 = 7
sum = num1 + num2 # Calculate the sum of
num1 and num2
print(sum)
# Example 2: Explicit is better than implicit.
# Instead of using implicit variables or
functions, be explicit.
# Bad code
lst = [1, 2, 3, 4, 5]
result = filter(lambda x: x % 2 == 0, lst)
print(list(result))
# Good code
def is_even(num):
return num % 2 == 0
numbers = [1, 2, 3, 4, 5]
even_numbers = filter(is_even, numbers)
print(list(even_numbers))
# Example 3: Simple is better than complex.
# Instead of writing complex code
Pythonic thinking
Pythonic thinking refers to writing code that is idiomatic and natural to
the Python language. It involves using the language's features and
syntax in a way that is efficient, elegant, and easy to read. In this
note, we will discuss some key principles of Pythonic thinking and
demonstrate them with suitable code examples.
Using list comprehensions instead of loops:
List comprehensions are a concise and efficient way to create new
lists by applying a function to each element of an existing list. They
are more Pythonic than using for-loops with append statements to
create a new list. Here is an example:
# Using for loop to create a new list

squares = []
for i in range(10):
squares.append(i**2)
print(squares)
# Using list comprehension to create a new
list
squares = [i**2 for i in range(10)]
print(squares)
Output:
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Using built-in functions and modules:
Python provides many built-in functions and modules that make it
easy to perform common tasks. It is more Pythonic to use these
functions and modules instead of reinventing the wheel. Here is an
example:
# Using built-in function sum() to sum a list

of numbers
numbers = [1, 2, 3, 4, 5]
total = sum(numbers)
print(total)
# Using built-in module math to calculate the
square root of a number
import math
sqrt = math.sqrt(16)
print(sqrt)
Output:
15
4.0
Using generator expressions instead of list comprehensions:
Generator expressions are a memory-efficient way to generate
values on-the-fly. They are more Pythonic than list comprehensions
when you are working with large datasets. Here is an example:
# Using list comprehension to create a list of

squares
squares = [i**2 for i in range(1000000)]
print(len(squares))
# Using generator expression to generate
squares on-the-fly
squares = (i**2 for i in range(1000000))
print(len(squares))
Output:
1000000
<generator object <genexpr> at
0x7f9367040b30>
Using context managers for resource management:
Context managers provide a convenient way to manage resources
such as files, sockets, and database connections. They are more
Pythonic than using try/finally blocks to ensure that resources are
properly released. Here is an example:
# Using try/finally block to manage file

resources
try:
f = open('myfile.txt', 'w')
f.write('Hello, World!')
finally:
f.close()
# Using context manager to manage file
resources
with open('myfile.txt', 'w') as f:
f.write('Hello, World!')
Using the Python standard library:
Python has a rich standard library that provides many useful modules
for various tasks. It is more Pythonic to use these modules instead of
third-party libraries when possible. Here is an example:
# Using the built-in module datetime to work

with dates and times
import datetime
today = datetime.datetime.today()
print(today)
Output:
2023-03-13 13:44:55.881958
Chapter 2:
Pythonic thinking
Python is a popular high-level programming language known for its

simplicity, readability, and ease of use. It is widely used in various
fields, from web development to data science, and has a large and
supportive community. One of the key aspects that make Python
unique is the concept of "Pythonic thinking."
Pythonic thinking refers to the mindset of writing code in a way that
aligns with the core principles and design philosophy of Python. It is a
set of guidelines that encourage developers to write clean, concise,
and efficient code that is easy to read and maintain. Pythonic code is
not only efficient, but it is also elegant and easy to understand.
The concept of Pythonic thinking is deeply rooted in the Python
community, and it is often considered a way of life for Python
developers. It is not just about writing code, but also about
understanding the essence of Python and its design philosophy.
One of the core principles of Pythonic thinking is "Readability counts."
Python code is designed to be easy to read and understand, even by
non-programmers. This is achieved through the use of simple and
clear syntax, meaningful variable names, and well-structured code.
Python's design philosophy emphasizes the importance of writing
code that is easy to read and understand, even by someone who has
never seen it before.
Another key principle of Pythonic thinking is "Don't repeat yourself"
(DRY). This principle encourages developers to write code that is
reusable and modular. In other words, instead of writing the same
code over and over again, Python developers are encouraged to
write code that can be reused in different parts of the program. This
not only saves time but also reduces the chances of introducing
errors in the code.
Pythonic thinking also emphasizes the importance of simplicity.
Python code is designed to be simple and straightforward. Python
developers are encouraged to write code that is as simple as
possible, without sacrificing functionality. This not only makes the
code easier to read and understand, but it also makes it easier to
maintain and modify.
Pythonic thinking also encourages the use of built-in functions and
libraries. Python has a large number of built-in functions and libraries
that can be used to perform common tasks. By using these built-in
functions and libraries, Python developers can save time and avoid
reinventing the wheel.
Finally, Pythonic thinking encourages the use of idiomatic Python
code. Idiomatic Python code is code that is written in a way that is
consistent with the core principles and design philosophy of Python.
Python developers are encouraged to write code that is not only
efficient but also follows the conventions and style of the Python
community.
In summary, Pythonic thinking is a way of approaching programming
in Python that emphasizes simplicity, readability, and efficiency. It is a
mindset that encourages developers to write code that is easy to
read, maintain, and understand. By following the core principles of
Pythonic thinking, Python developers can write code that is not only
efficient but also elegant and easy to understand.
Know your data structures

Tuples
In Pythonic thinking, it is essential to know the data structures

available in the Python programming language and how to use them
effectively. In this note, we will discuss tuples, one of the most
commonly used data structures in Python, and provide code
examples to demonstrate their usage.
A tuple is an ordered collection of elements that can be of any data
type. However, tuples are immutable, which means their elements
cannot be changed once they are created. Tuples are typically used
for grouping related data together.
Here are some examples of tuples and how to use them:
Creating a tuple:
Tuples can be created using parentheses or the tuple() function.
# Creating a tuple using parentheses

mytuple = (1, 2, 3, 4, 5)
# Creating a tuple using the tuple() function
mytuple = tuple([1, 2, 3, 4, 5])
Accessing tuple elements:
Tuple elements can be accessed using indexing. Indexing starts at 0
for the first element.
# Accessing tuple elements

mytuple = (1, 2, 3, 4, 5)
print(mytuple[0]) # Output: 1
print(mytuple[2]) # Output: 3
Slicing a tuple:
Tuples can be sliced using the same syntax as lists.
# Slicing a tuple
mytuple = (1, 2, 3, 4, 5)
print(mytuple[1:3]) # Output: (2, 3)
Unpacking a tuple:
Tuples can be unpacked into multiple variables.
# Unpacking a tuple
mytuple = (1, 2, 3)
a, b, c = mytuple
print(a) # Output: 1
print(b) # Output: 2
print(c) # Output: 3
Concatenating tuples:
Tuples can be concatenated using the + operator.
# Concatenating tuples
tuple1 = (1, 2, 3)
tuple2 = (4, 5, 6)
newtuple = tuple1 + tuple2
print(newtuple) # Output: (1, 2, 3, 4, 5, 6)
Using tuples as keys in dictionaries
Since tuples are immutable, they can be used
as keys in dictionaries.
python
Copy code
# Using tuples as keys in dictionaries
mydict = {(1, 2): 'value1', (3, 4): 'value2'}
print(mydict[(1, 2)]) # Output: 'value1'
print(mydict[(3, 4)]) # Output: 'value2'
In summary, tuples are an essential data structure in Pythonic
thinking, and they can be used for a wide range of applications. They
are particularly useful for grouping related data together, and their
immutability makes them ideal for use as keys in dictionaries or as
elements in sets.
Lists
In Pythonic thinking, it is crucial to know the available data structures
and how to use them effectively. One of the most commonly used
data structures in Python is the list. A list is an ordered collection of
elements that can be of any data type. Lists are mutable, which
means their elements can be changed once they are created. Lists
are typically used for storing data that can be modified or changed.
Here are some examples of lists and how to use them:
Creating a list:
Lists can be created using square brackets [] or the list() function.
# Creating a list using square brackets

mylist = [1, 2, 3, 4, 5]
# Creating a list using the list() function
mylist = list([1, 2, 3, 4, 5])
Accessing list elements:
List elements can be accessed using indexing. Indexing starts at 0 for
the first element.
# Accessing list elements

mylist = [1, 2, 3, 4, 5]
print(mylist[0]) # Output: 1
print(mylist[2]) # Output: 3
Slicing a list:
Lists can be sliced using the syntax start:end. Slicing a list returns a
new list containing the selected elements.
# Slicing a list
mylist = [1, 2, 3, 4, 5]
print(mylist[1:3]) # Output: [2, 3]
Modifying list elements:
Since lists are mutable, their elements can be modified.
# Modifying list elements

mylist = [1, 2, 3, 4, 5]
mylist[2] = 7
print(mylist) # Output: [1, 2, 7, 4, 5]
Adding elements to a list:
Elements can be added to a list using the append() method or the
extend() method to add multiple elements at once.
# Adding elements to a list

mylist = [1, 2, 3]
mylist.append(4)
print(mylist) # Output: [1, 2, 3, 4]
mylist.extend([5, 6])
print(mylist) # Output: [1, 2, 3, 4, 5, 6]
Removing elements from a list:
Elements can be removed from a list using the remove() method or
the pop() method to remove an element at a specific index.
# Removing elements from a list

mylist = [1, 2, 3, 4, 5]
mylist.remove(3)
print(mylist) # Output: [1, 2, 4, 5]
mylist.pop(2)
print(mylist) # Output: [1, 2, 5]
Sorting a list:
Lists can be sorted using the sort() method or the sorted() function.
# Sorting a list
mylist = [4, 2, 3, 1, 5]
mylist.sort()
sortedlist = sorted(mylist, reverse=True)
print(sortedlist) # Output: [5, 4, 3, 2, 1]
Dictionaries
In Pythonic thinking, it is essential to know the available data

structures and how to use them effectively. One of the most
commonly used data structures in Python is the dictionary. A
dictionary is an unordered collection of key-value pairs, where each
key is unique. Dictionaries are mutable, which means their elements
can be changed once they are created. Dictionaries are typically
used for storing data that can be looked up using a key.
Here are some examples of dictionaries and how to use them:
Creating a dictionary:
Dictionaries can be created using curly braces {} or the dict()
function.
# Creating a dictionary using curly braces

mydict = {'apple': 1, 'banana': 2, 'orange': 3}
# Creating a dictionary using the dict()
function
mydict = dict(apple=1, banana=2, orange=3)
Accessing dictionary elements:
Dictionary elements can be accessed using the key. If the key does
not exist, a KeyError will be raised.
# Accessing dictionary elements

print(mydict['apple']) # Output: 1
print(mydict['watermelon']) # Raises
KeyError: 'watermelon'
Modifying dictionary elements:
Since dictionaries are mutable, their elements can be modified.
# Modifying dictionary elements

mydict['orange'] = 4
print(mydict) # Output: {'apple': 1, 'banana':
2, 'orange': 4}
Adding elements to a dictionary:

Elements can be added to a dictionary using the key-value syntax.
# Adding elements to a dictionary

mydict = {'apple': 1, 'banana': 2}
mydict['orange'] = 3
2, 'orange': 3}
Removing elements from a dictionary:
Elements can be removed from a dictionary using the del keyword or
the pop() method to remove an element using its key.
# Removing elements from a dictionary

del mydict['orange']
2}
mydict.pop('banana')
print(mydict) # Output: {'apple': 1}
Checking if a key exists in a dictionary:
To check if a key exists in a dictionary, you can use the in keyword.
# Checking if a key exists in a dictionary

print('banana' in mydict) # Output: True
print('watermelon' in mydict) # Output: False
Iterating over a dictionary:
To iterate over a dictionary, you can use the items() method to get
the key-value pairs or the keys() method to get the keys.
# Iterating over a dictionary
for key, value in mydict.items():
print(key, value)
for key in mydict.keys():
print(key)
for value in mydict.values():
print(value)
Sets
In Pythonic thinking, it is important to know the available data

structures and how to use them effectively. One of the commonly
used data structures in Python is the set. A set is an unordered
collection of unique elements. Sets are mutable, which means their
elements can be changed once they are created. Sets are typically
used for operations that require finding the intersection, union, or
difference between two collections.
Here are some examples of sets and how to use them:
Creating a set:
Sets can be created using curly braces {} or the set() function.
# Creating a set using curly braces

myset = {1, 2, 3, 4}
# Creating a set using the set() function
myset = set([1, 2, 3, 4])
Accessing set elements:
Set elements can be accessed using a for loop or the in keyword.
# Accessing set elements

myset = {1, 2, 3, 4}
for element in myset:
print(element)
print(1 in myset) # Output: True
print(5 in myset) # Output: False
Modifying set elements:

Since sets are mutable, their elements can be modified.
# Modifying set elements

myset = {1, 2, 3, 4}
myset.add(5)
print(myset) # Output: {1, 2, 3, 4, 5}
myset.remove(4)
print(myset) # Output: {1, 2, 3, 5}
Combining sets:
Sets can be combined using union(), intersection(), and difference()
methods.
# Combining sets
set1 = {1, 2, 3, 4}
set2 = {3, 4, 5, 6}
union_set = set1.union(set2)
print(union_set) # Output: {1, 2, 3, 4, 5, 6}
intersection_set = set1.intersection(set2)
print(intersection_set) # Output: {3, 4}
difference_set = set1.difference(set2)
print(difference_set) # Output: {1, 2}
Checking if a set is a subset or superset:
To check if a set is a subset or superset of another set, you can use
the issubset() and issuperset() methods.
# Checking if a set is a subset or superset

set1 = {1, 2, 3}
set2 = {1, 2, 3, 4, 5}
print(set1.issubset(set2)) # Output: True
print(set2.issuperset(set1)) # Output: True
Removing duplicate elements:

Sets can be used to remove duplicate elements from a list.
# Removing duplicate elements from a list

mylist = [1, 2, 2, 3, 3, 4, 5, 5]
myset = set(mylist)
print(myset) # Output: {1, 2, 3, 4, 5}
In summary, sets are a useful data structure in Python for operations
that require finding the intersection, union, or difference between two
collections.
Arrays
In Pythonic thinking, it is important to know the available data

structures and how to use them effectively. One of the commonly
used data structures in Python is the array. An array is a collection of
elements of the same data type, arranged in contiguous memory
locations. Arrays are typically used for operations that require
efficient element-wise computations, such as linear algebra
operations.
Here are some examples of arrays and how to use them:
Creating an array:
Arrays can be created using the array() function from the array
module.
import array as arr

# Creating an array of integers
myarr = arr.array('i', [1, 2, 3, 4, 5])
# Creating an array of floats
myarr = arr.array('f', [1.0, 2.0, 3.0, 4.0, 5.0])
Accessing array elements:
Array elements can be accessed using indexing, just like in a list.
# Accessing array elements

myarr = arr.array('i', [1, 2, 3, 4, 5])
print(myarr[0]) # Output: 1
print(myarr[4]) # Output: 5
Modifying array elements:
Array elements can be modified by assigning new values to their
corresponding indices.
# Modifying array elements

myarr = arr.array('i', [1, 2, 3, 4, 5])
myarr[0] = 10
print(myarr) # Output: array('i', [10, 2, 3, 4, 5])
Performing element-wise computations:
Arrays can be used to perform element-wise computations efficiently
using NumPy, a powerful Python library for scientific computing.
import numpy as np
# Performing element-wise computations
myarr = arr.array('f', [1.0, 2.0, 3.0, 4.0, 5.0])
myarr = np.square(myarr)
print(myarr) # Output: array([ 1., 4., 9., 16.,
25.], dtype=float32)
Converting arrays to lists:
Arrays can be converted to lists using the tolist() method.
# Converting arrays to lists

myarr = arr.array('i', [1, 2, 3, 4, 5])
mylist = myarr.tolist()
In summary, arrays are a useful data structure in Python for
operations that require efficient element-wise computations, such as
linear algebra operations. They can be created using the array()
function from the array module, accessed and modified using
indexing, and converted to lists using the tolist() method. To perform
more advanced computations with arrays, NumPy is a powerful
library that is widely used in scientific computing.
Queues
Queues are a fundamental data structure in computer science that

are used to store a collection of elements in a first-in, first-out (FIFO)
order. They are commonly used in algorithms for tasks such as
breadth-first search, job scheduling, and message passing. In Python,
queues can be implemented using the built-in deque class from the
collections module or using the queue module.
Here are some examples of using queues in Python:
Creating a queue:
To create a queue in Python, we can use the deque class from the
collections module or the Queue class from the queue module.
from collections import deque

# or
from queue import Queue
# Creating a queue using deque
myqueue = deque()
# Creating a queue using Queue
myqueue = Queue()
Adding elements to a queue:
We can add elements to a queue using the append() method of the
deque class or the put() method of the Queue class.
# Adding elements to a queue

myqueue = deque()
myqueue.append(1)
myqueue.append(2)
myqueue.append(3)
# or
myqueue = Queue()
myqueue.put(1)
myqueue.put(2)
myqueue.put(3)
Removing elements from a queue:

We can remove elements from a queue using the popleft() method of
the deque class or the get() method of the Queue class.
# Removing elements from a queue

myqueue = deque([1, 2, 3])
myqueue.popleft() # Output: 1
# or
myqueue = Queue()
myqueue.put(1)
myqueue.put(2)
myqueue.put(3)
myqueue.get() # Output: 1
Checking the size of a queue:
We can check the size of a queue using the len() function.
# Checking the size of a queue

myqueue = deque([1, 2, 3])
print(len(myqueue)) # Output: 3
In summary, queues are a useful data structure in Python for tasks
that require a collection of elements to be processed in a first-in, first-
out order. They can be implemented using the built-in deque class
from the collections module or using the Queue class from the queue
module. To add elements to a queue, we can use the append()
method of the deque class or the put() method of the Queue class.
To remove elements from a queue, we can use the popleft() method
of the deque class or the get() method of the Queue class. Finally,
we can check the size of a queue using the len() function.
Stacks
Stacks are a fundamental data structure in computer science that are

used to store a collection of elements in a last-in, first-out (LIFO)
order. They are commonly used in algorithms for tasks such as
expression evaluation, function call management, and undo/redo
operations. In Python, stacks can be implemented using the built-in
list class.
Here are some examples of using stacks in Python:
Creating a stack:
To create a stack in Python, we can use an empty list.
# Creating a stack
mystack = []
Adding elements to a stack:
We can add elements to a stack using the append() method of the list
class.
# Adding elements to a stack

mystack = []
mystack.append(1)
mystack.append(2)
mystack.append(3)
Removing elements from a stack:
We can remove elements from a stack using the pop() method of the
list class.
# Removing elements from a stack

mystack = [1, 2, 3]
mystack.pop() # Output: 3
mystack.pop() # Output: 2
Checking the size of a stack:
We can check the size of a stack using the len() function.
# Checking the size of a stack

mystack = [1, 2, 3]
print(len(mystack)) # Output: 3
In summary, stacks are a useful data structure in Python for tasks
that require a collection of elements to be processed in a last-in, first-
out order. They can be implemented using an empty list. To add
elements to a stack, we can use the append() method of the list
class. To remove elements from a stack, we can use the pop()
method of the list class. Finally, we can check the size of a stack
using the len() function.
Heaps
Heaps are a fundamental data structure in computer science that are

used to efficiently maintain the minimum (or maximum) element in a
collection of elements. In Python, heaps can be implemented using
the built-in heapq module.
Here are some examples of using heaps in Python:
Creating a heap:
To create a heap in Python, we can use the heapify() function of the
heapq module to convert a list into a heap.
import heapq
# Creating a heap
myheap = [3, 1, 4, 1, 5, 9, 2, 6, 5]
heapq.heapify(myheap)
Alternatively, we can use the heappush() function of the heapq
module to add elements to an empty heap.
import heapq
# Creating a heap
myheap = []
heapq.heappush(myheap, 3)
Getting the minimum element from a heap:
To get the minimum element from a heap, we can use the heappop()
function of the heapq module.
import heapq
# Getting the minimum element from a heap
myheap = [3, 1, 4, 1, 5, 9, 2, 6, 5]
print(heapq.heappop(myheap)) # Output: 1
print(heapq.heappop(myheap)) # Output: 1
Adding elements to a heap:
We can add elements to a heap using the heappush() function of the
heapq module.
import heapq
# Adding elements to a heap
myheap = [3, 1, 4, 1, 5, 9, 2, 6, 5]
print(myheap) # Output: [0, 1, 2, 3, 5, 9, 4, 6,
5, 7]
Checking the size of a heap:
We can check the size of a heap using the len() function.
import heapq
# Checking the size of a heap
myheap = [3, 1, 4, 1, 5, 9, 2, 6, 5]
print(len(myheap)) # Output: 9
In summary, heaps are a useful data structure in Python for efficiently
maintaining the minimum (or maximum) element in a collection of
elements. They can be implemented using the heapq module. To
create a heap, we can use the heapify() function of the heapq module
to convert a list into a heap, or we can use the heappush() function to
add elements to an empty heap. To get the minimum element from a
heap, we can use the heappop() function. Finally, we can check the
size of a heap using the len() function.
Trees
Trees are a fundamental data structure in computer science that are

used to represent hierarchical relationships between elements. In
Python, trees can be implemented using classes and objects.
Here is an example of using trees in Python:
Creating a tree:
To create a tree in Python, we can define a class for the nodes of the
tree, and use objects of this class to represent the nodes.
class Node:
def __init__(self, data):
self.data = data
self.left = None
self.right = None
# Creating a tree
root = Node(1)
root.left = Node(2)
root.right = Node(3)
root.left.left = Node(4)
root.left.right = Node(5)
Traversing a tree:
To traverse a tree in Python, we can use recursive functions to visit
each node in the tree in a specific order. Here are three common
ways to traverse a tree:
Inorder traversal: Visit the left subtree, then the current node, then
the right subtree.
def inorder(node):
if node is not None:
inorder(node.left)
print(node.data)
inorder(node.right)
# Inorder traversal of the tree
inorder(root)
Preorder traversal: Visit the current node, then the left subtree, then
the right subtree.
def preorder(node):
print(node.data)
preorder(node.left)
preorder(node.right)
# Preorder traversal of the tree
preorder(root)
Postorder traversal: Visit the left subtree, then the right subtree, then
the current node.
def postorder(node):
postorder(node.left)
postorder(node.right)
print(node.data)
# Postorder traversal of the tree
postorder(root)
Finding elements in a tree:
To find an element in a tree in Python, we can use a recursive
function to traverse the tree and search for the element.
def find(node, data):

if node is None:
return False
elif node.data == data:
return True
elif data < node.data:
return find(node.left, data)
else:
return find(node.right, data)
# Finding elements in the tree
print(find(root, 2)) # Output: True
print(find(root, 6)) # Output: False
In summary, trees are a useful data structure in Python for
representing hierarchical relationships between elements. They can
be implemented using classes and objects. To traverse a tree, we
can use recursive functions to visit each node in the tree in a specific
order. To find an element in a tree, we can use a recursive function to
traverse the tree and search for
the element.
Graphs
Graphs are an important data structure in computer science that are

used to represent relationships between objects. A graph consists of
a set of vertices (or nodes) and a set of edges that connect pairs of
vertices. In Python, graphs can be implemented using classes and
objects.
Here is an example of using graphs in Python:
Creating a graph:
To create a graph in Python, we can define a class for the nodes of
the graph, and use objects of this class to represent the nodes. Each
node can have a list of neighbors that represents the edges
connecting it to other nodes.
class Node:
def __init__(self, data):
self.data = data
self.neighbors = []
# Creating a graph
A = Node('A')
B = Node('B')
C = Node('C')
D = Node('D')
E = Node('E')
F = Node('F')
G = Node('G')
H = Node('H')
A.neighbors = [B, C, D]
B.neighbors = [A, E]
C.neighbors = [A, F]
D.neighbors = [A, G, H]
E.neighbors = [B]
F.neighbors = [C]
G.neighbors = [D, H]
H.neighbors = [D, G]
Traversing a graph:
To traverse a graph in Python, we can use a recursive function to visit
each node in the graph in a specific order. Here are two common
ways to traverse a graph:
Depth-first search (DFS): Visit the current node, then recursively visit
each of its neighbors.
def dfs(node, visited):

visited.add(node)
print(node.data)
for neighbor in node.neighbors:
if neighbor not in visited:
dfs(neighbor, visited)
# DFS traversal of the graph
visited = set()
dfs(A, visited)
Breadth-first search (BFS): Visit all nodes at a given distance from
the starting node, then all nodes at the next distance, and so on.
def bfs(node):
visited = set()
queue = [node]
visited.add(node)
while queue:
curr_node = queue.pop(0)
print(curr_node.data)
for neighbor in curr_node.neighbors:
visited.add(neighbor)
queue.append(neighbor)
# BFS traversal of the graph
bfs(A)
Finding paths in a graph:
To find a path between two nodes in a graph in Python, we can use a
recursive function to traverse the graph and search for the path. We
can use either DFS or BFS to perform the traversal.
def find_path(start_node, end_node, visited,

path):
visited.add(start_node)
path.append(start_node)
if start_node == end_node:
return path
for neighbor in start_node.neighbors:
result = find_path(neighbor, end_node,
visited, path)
if result:
return result
path.pop()
# Finding a path in the graph
visited = set()
path = []
result = find_path(A, H, visited, path)
if result:
print('Path found:', [node.data for node in
result])
else:
print('Path not found')
In summary, graphs are a versatile data structure in Python for
representing relationships between objects. They can be
implemented using classes and objects. To traverse a graph, we can
use either DFS or BFS to visit each node in the graph in a specific
order. To find a path between two nodes in a graph, we can use a
recursive function to traverse the graph and search for the path.
Write expressive code

Choosing good names
In Pythonic thinking, writing expressive and readable code is

important for maintaining code quality and making it easier for others
to understand and work with. One key aspect of this is choosing
good variable and function names that are descriptive and meaningful.
In this note, we will explore some best practices for choosing good
names in Python, along with some examples of suitable code.
Use descriptive names: Names should be clear and descriptive,
making it easier for other developers to understand what they
represent. For example, instead of naming a variable "data", consider
naming it "user_data" or "sales_data".
Follow naming conventions: Python has some established naming
conventions that are widely used, such as using lowercase letters for
variables and using underscores to separate words in a name.
Following these conventions can make your code more readable and
easier to understand for other developers.
Avoid abbreviations: While it may be tempting to use abbreviations to
save space, it can actually make your code harder to understand. For
example, instead of using "usr" for "user", use the full word "user".
Be consistent: Consistency in naming conventions is important for
maintaining readability and making your code easy to understand. If
you choose to use a certain naming convention, make sure to apply it
consistently throughout your code.
Use meaningful function names: Function names should be descriptive
and indicate what the function does. For example, if a function
calculates the average of a set of numbers, consider naming it
"calculate_average" instead of just "average".
Use comments sparingly: While comments can be helpful for
providing context to code, they should not be relied upon to make up
for poorly named variables or functions. Instead, focus on choosing
descriptive names that communicate the purpose of the code.
Here is an example of code that follows these best practices for
choosing good names:
# calculate the average of a list of numbers

def calculate_average(numbers_list):
sum = 0
for number in numbers_list:
sum += number
return sum / len(numbers_list)
# get user data from database
def get_user_data(user_id):
query = "SELECT * FROM users WHERE id
= %s"
result = execute_query(query, user_id)
return result
# generate a random password for a user
def generate_password():
alphabet =
"abcdefghijklmnopqrstuvwxyz0123456789"
password = ""
for i in range(8):
password += random.choice(alphabet)
return password
In summary, choosing good names is an important aspect of writing
expressive and readable code in Python. By following best practices
such as using descriptive names, following naming conventions,
avoiding abbreviations, being consistent, using meaningful function
names, and using comments sparingly, you can create code that is
easy for other developers to understand and work with.
Avoiding magic numbers and strings
When writing code, it is important to avoid the use of magic numbers

and strings. Magic numbers and strings are hard-coded values that
appear throughout your code and have no clear meaning or
explanation. These values can make your code difficult to understand
and maintain, and can lead to errors and bugs. In this note, we will
explore some best practices for avoiding magic numbers and strings
in Python, along with some examples of suitable code.
Define constants: Instead of using hard-coded values throughout your
code, define constants to hold these values. This makes it easier to
change these values in the future, and makes your code more self-
documenting. For example:
# define a constant for the number of days in

a week
DAYS_IN_WEEK = 7
# use the constant instead of a hard-coded
value
for i in range(DAYS_IN_WEEK):
...
Use named variables: Instead of using hard-coded strings throughout
your code, define variables to hold these values. This makes your
code more self-documenting and helps prevent typos. For example:
# define variables for column names

NAME_COLUMN = "name"
AGE_COLUMN = "age"
# use the variables instead of hard-coded
strings
if column == NAME_COLUMN:
...
Use enums: Enums are a type of constant that represent a fixed set
of values. They are especially useful when working with a limited set
of options, and make your code more self-documenting. For example:
# define an enum for days of the week

from enum import Enum
class Weekday(Enum):
MONDAY = 1
TUESDAY = 2
WEDNESDAY = 3
THURSDAY = 4
FRIDAY = 5
SATURDAY = 6
SUNDAY = 7
# use the enum instead of a hard-coded value
day = Weekday.MONDAY
if day == Weekday.SATURDAY:
...
Use configuration files: Instead of hard-coding values in your code,
you can store them in configuration files. This makes it easier to
change these values without modifying your code, and makes your
code more modular. For example:
# load configuration from a file

import configparser
config = configparser.ConfigParser()
config.read("config.ini")
# use the configuration values in your code
if config.getboolean("debug", "enabled"):
...
In summary, avoiding magic numbers and strings is an important
aspect of writing expressive and maintainable code in Python. By
defining constants, using named variables, using enums, and using
configuration files, you can make your code more self-documenting
and easier to understand and maintain.
Using list comprehensions and generator expressions
List comprehensions and generator expressions are powerful

features in Python that allow you to create new lists and generators
from existing ones in a concise and expressive way. In this note, we
will explore how to use list comprehensions and generator
expressions to write expressive code in Python, along with some
examples of suitable code.
List comprehensions: List comprehensions allow you to create new
lists by iterating over an existing list and applying a function or
conditional statement to each element. The resulting list is created in
a single line of code, making it concise and expressive. For example:
# create a list of squared numbers

numbers = [1, 2, 3, 4, 5]
squares = [x ** 2 for x in numbers]
# create a list of even numbers
evens = [x for x in numbers if x % 2 == 0]
Generator expressions: Generator expressions are similar to list
comprehensions, but instead of creating a new list, they create a
generator that yields each element as needed. This can be more
memory-efficient than creating a new list, especially for large
datasets. For example:
# create a generator of squared numbers

numbers = [1, 2, 3, 4, 5]
squares = (x ** 2 for x in numbers)
# create a generator of even numbers
evens = (x for x in numbers if x % 2 == 0)
Nested list comprehensions: Nested list comprehensions allow you to
create more complex lists by iterating over multiple lists and applying
a function or conditional statement to each element. This can be
useful for creating matrices or performing more complex calculations.
For example:
# create a matrix of zeros

rows = 3
cols = 3
matrix = [[0 for j in range(cols)] for i in
range(rows)]
# create a list of all pairs of numbers from
two lists
list1 = [1, 2, 3]
list2 = [4, 5, 6]
pairs = [(x, y) for x in list1 for y in list2]
In summary, list comprehensions and generator expressions are
powerful features in Python that allow you to create new lists and
generators from existing ones in a concise and expressive way. By
using these features, you can write more expressive and efficient
code in Python.
Leveraging built-in functions
Python has a vast collection of built-in functions that can perform a

wide variety of operations. Leveraging these functions can help you
write more expressive code that is concise and easy to read. In this
note, we will explore some of the built-in functions that you can use to
write more expressive code, along with suitable codes.
map(): The map() function applies a function to each element of an
iterable and returns a new iterable with the results. This can be a
useful way to apply a function to every element of a list without using
a loop. For example:
# create a list of squared numbers using

map()
numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, numbers))
filter(): The filter() function returns an iterable containing the elements
from an iterable that satisfy a given condition. This can be a useful
way to filter a list based on a condition without using a loop. For
example:
# create a list of even numbers using filter()

numbers = [1, 2, 3, 4, 5]
even_numbers = list(filter(lambda x: x % 2 ==
0, numbers))
reduce(): The reduce() function applies a function to the first two
elements of an iterable, then applies the same function to the result
and the next element, and so on, until all elements have been
processed. This can be a useful way to aggregate a list of values
without using a loop. For example:
# calculate the sum of a list using reduce()

from functools import reduce
numbers = [1, 2, 3, 4, 5]
total = reduce(lambda x, y: x + y, numbers)
zip(): The zip() function takes multiple iterables and returns an iterable
of tuples containing the elements from each iterable at corresponding
positions. This can be a useful way to combine multiple lists into a
single list without using a loop. For example:
# create a list of tuples using zip()
names = ["Alice", "Bob", "Charlie"]
ages = [25, 30, 35]
people = list(zip(names, ages))
enumerate(): The enumerate() function takes an iterable and returns
an iterable of tuples containing the index and element of each item.
This can be a useful way to iterate over a list while keeping track of
the index without using a loop counter. For example:
# iterate over a list using enumerate()

fruits = ["apple", "banana", "orange"]
for index, fruit in enumerate(fruits):
print(f"{index}: {fruit}")
In summary, leveraging built-in functions can help you write more
expressive code in Python by providing a concise and easy-to-read
way to perform common operations. By using these functions, you
can avoid writing repetitive or verbose code, leading to more efficient
and maintainable code.
Using the with statement
Python is a programming language that provides a lot of built-in tools

and constructs to help developers write more expressive and
maintainable code. One such construct is the with statement, which
allows for the automatic management of resources such as file
handles, database connections, and network sockets. In this article,
we will explore the with statement and see how it can help us write
more expressive and robust code.
The with statement is used to define a block of code that will be
executed in the context of a resource, such as a file or a network
connection. The most common use case for with is to manage file
handles. Consider the following code snippet that opens a file, reads
its contents, and then closes the file:
file = open('example.txt', 'r')

contents = file.read()
file.close()
This code works fine, but there are a few issues with it. First, if an
exception is raised while reading the file, the close method will not be
called, and the file handle will remain open, potentially causing a
resource leak. Second, the code is not very expressive - it is not
immediately clear what the purpose of the code is.
Now, let's rewrite the code using the with statement:
with open('example.txt', 'r') as file:

contents = file.read()
This code is much more expressive - it is clear that we are reading
the contents of a file. In addition, the with statement automatically
takes care of closing the file handle, even if an exception is raised
while reading the file. This ensures that we don't leak resources and
that our code is more robust.
The with statement can also be used with other resources, such as
database connections and network sockets. Here is an example of
using with to manage a database connection:
import sqlite3
with sqlite3.connect('example.db') as conn:
cursor = conn.cursor()
cursor.execute('SELECT * FROM
customers')
results = cursor.fetchall()
In this code, the with statement is used to create a database
connection, which is automatically closed when the block of code
inside the with statement completes. This ensures that we don't leak
database connections and that our code is more robust.
The with statement is a powerful tool that can help us write more
expressive and robust code. By using with to manage resources such
as file handles, database connections, and network sockets, we can
ensure that our code is more maintainable and less error-prone.
Using decorators
Python provides a feature called decorators, which allows

programmers to modify the behavior of a function or class without
changing its source code. Decorators are a powerful tool for writing
expressive code and can be used to simplify complex tasks.
In Python, a decorator is a callable object that takes another function
or class as its argument and returns a new function or class. This
new function or class can then be used in place of the original function
or class.
Here's an example of a decorator that logs the time taken by a
function to execute:
import time
def timing_decorator(func):
def wrapper(*args, **kwargs):
start_time = time.time()
result = func(*args, **kwargs)
end_time = time.time()
print(f"{func.__name__} took {end_time -
start_time} seconds to run.")
return result
return wrapper
@timing_decorator
def long_running_function():
# simulate a long running function
time.sleep(2)
long_running_function()
In this example, the timing_decorator function takes a function as an
argument, creates a new function called wrapper, and returns it. The
wrapper function uses the time module to measure the time taken by
the original function to execute and prints it to the console.
The @timing_decorator syntax is a shorthand way of applying the
decorator to the long_running_function function. It is equivalent to
calling long_running_function =
timing_decorator(long_running_function).
Decorators can also be used to add functionality to a class. Here's an
example of a decorator that adds a log method to a class:
def add_logging(cls):
def log(self, message):
print(f"{cls.__name__}: {message}")
cls.log = log
return cls
@add_logging
class MyClass:
pass
obj = MyClass()
obj.log("Hello, world!")
In this example, the add_logging function takes a class as an
argument, defines a new log method that prints a message to the
console, adds the log method to the class, and returns the class. The
@add_logging syntax is a shorthand way of applying the decorator to
the MyClass class.
Decorators are a powerful tool for writing expressive code in Python.
They allow programmers to modify the behavior of a function or class
without changing its source code and can be used to simplify complex
tasks.
Writing context managers
When writing code in Python, it's important to consider not only its
functionality but also its readability and maintainability. One way to
achieve this is by using context managers. Context managers are
objects that help manage resources, such as files, locks, and network
connections, by defining the setup and teardown logic for the
resource.
A context manager is implemented as a class that defines the
methods __enter__() and __exit__():
__enter__() is called at the beginning of a with block and
returns the resource that will be managed.
__exit__() is called at the end of the with block and handles
any cleanup logic that needs to be performed.
Here's an example of a simple context manager that opens and

closes a file:
class File:
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
def __enter__(self):
self.file = open(self.filename, self.mode)
return self.file
def __exit__(self, exc_type, exc_value,
traceback):
self.file.close()
with File('example.txt', 'w') as f:
f.write('Hello, world!')
In this example, the File class defines the __enter__() and __exit__()
methods to open and close a file. The with statement is used to
automatically call these methods and ensure that the file is properly
closed when the block is exited.
Context managers can also be used to manage resources other than
files, such as network connections or database transactions. Here's
an example of a context manager that wraps a database transaction:
import sqlite3
class Transaction:
def __init__(self, db):
self.db = db
def __enter__(self):
self.conn = sqlite3.connect(self.db)
self.cursor = self.conn.cursor()
return self.cursor
def __exit__(self, exc_type, exc_value,
traceback):
if exc_type is None:
self.conn.commit()
else:
self.conn.rollback()
self.cursor.close()
self.conn.close()
with Transaction('example.db') as cursor:
cursor.execute('CREATE TABLE IF NOT
EXISTS users (id INTEGER PRIMARY KEY,
name TEXT)')
In this example, the Transaction class defines the __enter__() and
__exit__() methods to open and close a database connection and
transaction. The with statement is used to automatically call these
methods and ensure that the transaction is properly committed or
rolled back when the block is exited.
In summary, context managers are a powerful tool for managing
resources in Python. They provide a clean and readable way to
ensure that resources are properly setup and cleaned up, which can
lead to more maintainable and bug-free code.
Take advantage of Python's
features
Using named tuples
Python's named tuples are a convenient and efficient way of creating

lightweight, immutable objects with named fields. They are essentially
a subclass of tuples that have named fields, making them more
readable and self-documenting. Named tuples are commonly used to
represent data structures in Python, and they can be used to replace
dictionary objects in some scenarios where the key is always a
string.
In this note, we will discuss how to take advantage of Python's
named tuples feature, including its syntax and how it can be used to
enhance code readability.
Syntax
The syntax for defining a named tuple in Python is straightforward.
Here is an example:
from collections import namedtuple

# Define a named tuple called 'Person' with
three fields: 'name', 'age', and 'gender'
Person = namedtuple('Person', ['name', 'age',
'gender'])
# Create an instance of the named tuple
person1 = Person(name='Alice', age=25,
gender='female')
In the example above, we first import the namedtuple class from the
collections module. Then we define a named tuple called 'Person' with
three fields: 'name', 'age', and 'gender'. The first argument to the
namedtuple function is the name of the tuple, and the second
argument is a list of field names. We can then create an instance of
the named tuple by passing in the field values as keyword arguments.
Code Example
Here is an example that demonstrates how named tuples can be
used to improve the readability of code:

# Define a named tuple called 'Point' with
two fields: 'x' and 'y'
Point = namedtuple('Point', ['x', 'y'])
# Define a function that calculates the
distance between two points
def distance(p1, p2):
dx = p1.x - p2.x
dy = p1.y - p2.y
return (dx**2 + dy**2) ** 0.5
# Create two points
p1 = Point(x=1, y=2)
p2 = Point(x=4, y=6)
# Calculate the distance between the two
points
d = distance(p1, p2)
print(f"The distance between {p1} and {p2} is
{d:.2f}.")
In the example above, we define a named tuple called 'Point' with two
fields: 'x' and 'y'. We then define a function called 'distance' that takes
two Point objects as arguments and calculates the distance between
them using the Pythagorean theorem. Finally, we create two Point
objects and use them to call the distance function, which returns the
distance between the two points. The result is then printed using an f-
string that makes use of the Point objects' str method.
Python's named tuples are a powerful and useful feature that can
greatly enhance the readability and organization of code. By using
named tuples, developers can create lightweight and self-
documenting objects with named fields, making the code more
readable and less error-prone.
Leveraging closures
Python's closures are a powerful and useful feature that allows

developers to create functions that can access and manipulate
variables from the enclosing scope. Closures are essentially functions
that remember the values of the variables in their lexical scope, even
when the outer function has returned.
closures feature, including its syntax and how it can be used to
enhance code modularity and reusability.
Syntax
The syntax for defining a closure in Python is straightforward. Here is
an example:
def outer_function(x):
def inner_function(y):
return x + y
return inner_function
# Create a closure by calling the outer
function
closure = outer_function(10)
# Call the closure
result = closure(5)
In the example above, we define a function called 'outer_function' that
takes an argument 'x' and returns another function called
'inner_function'. The inner function takes an argument 'y' and returns
the sum of 'x' and 'y'. When we call 'outer_function', it returns
'inner_function', which we assign to a variable called 'closure'. We can
then call the closure by passing in an argument 'y' and storing the
result in a variable called 'result'.
Code Example
Here is an example that demonstrates how closures can be used to
improve code modularity and reusability:
def make_multiplier(x):
def multiplier(y):
return x * y
return multiplier
# Create two closures using the
make_multiplier function
double = make_multiplier(2)
triple = make_multiplier(3)
# Use the closures to multiply some numbers
print(double(5)) # Output: 10
print(triple(5)) # Output: 15
In the example above, we define a function called 'make_multiplier'
that takes an argument 'x' and returns another function called
'multiplier'. The inner function takes an argument 'y' and returns the
product of 'x' and 'y'. We then create two closures using the
'make_multiplier' function, one that doubles the input and one that
triples the input. We can then use these closures to multiply some
numbers.
Python's closures are a powerful and flexible feature that allows
developers to create functions that can access and manipulate
variables from the enclosing scope. By using closures, developers
can create reusable and modular code that can be easily customized
to suit different use cases. Closures are an important tool in the
Python programmer's toolbox, and should be used judiciously to
enhance code readability, modularity, and reusability.
Using properties
Python's properties are a useful feature that allow developers to

define methods that look like simple attributes, while still providing the
functionality of a method. Properties can be used to validate or
sanitize input, calculate derived values, or trigger side effects, all
while providing a clean and intuitive interface for accessing and
modifying object state.
properties feature, including its syntax and how it can be used to
enhance code readability and maintainability.
Syntax
The syntax for defining a property in Python is straightforward. Here
is an example:
class Rectangle:
def __init__(self, width, height):
self._width = width
self._height = height
@property
def width(self):
return self._width
@width.setter
def width(self, value):
if value <= 0:
raise ValueError("Width must be
positive")
self._width = value
@property
def height(self):
return self._height
@height.setter
def height(self, value):
if value <= 0:
raise ValueError("Height must be
positive")
self._height = value
@property
def area(self):
return self._width * self._height
In the example above, we define a class called 'Rectangle' that has
two private instance variables '_width' and '_height'. We then define
three properties using the '@property' decorator: 'width', 'height', and
'area'. Each property has a getter method, which simply returns the
value of the corresponding instance variable. We also define two
setter methods for 'width' and 'height', which validate that the new
value is positive before setting the corresponding instance variable.
Finally, we define a 'area' property that calculates the area of the
rectangle by multiplying 'width' and 'height'.
Code Example
Here is an example that demonstrates how properties can be used to
enhance code readability and maintainability:
class Temperature:
def __init__(self, celsius):
self._celsius = celsius
@property
def celsius(self):
return self._celsius
@celsius.setter
def celsius(self, value):
if value < -273.15:
raise ValueError("Temperature cannot
be below absolute zero")
self._celsius = value
@property
def fahrenheit(self):
return self._celsius * 9 / 5 + 32
@fahrenheit.setter
def fahrenheit(self, value):
self._celsius = (value - 32) * 5 / 9
In the example above, we define a class called 'Temperature' that has
a private instance variable '_celsius'. We define two properties using
the '@property' decorator: 'celsius' and 'fahrenheit'. The 'celsius'
property has a getter method that simply returns the value of
'_celsius', and a setter method that validates that the new value is not
below absolute zero (-273.15 Celsius). The 'fahrenheit' property has
a getter method that calculates the Fahrenheit equivalent of the
current Celsius temperature, and a setter method that sets the
Celsius temperature based on the Fahrenheit input.
Python's properties are a useful and powerful feature that can be
used to enhance code readability and maintainability. By defining
methods that look like simple attributes, developers can provide clean
and intuitive interfaces for accessing and modifying object state, while
still providing the flexibility and functionality of a method. Properties
are an important tool in the Python programmer's toolbox, and should
be used judiciously to enhance code readability and maintainability.
Using descriptors
Python's descriptors are a powerful feature that allows for the

creation of objects that behave like variables, but with custom
behavior when they are accessed or assigned. They provide a way to
add custom behavior to attributes in classes, allowing for increased
flexibility and extensibility in Python code.
descriptors feature, including its syntax and how it can be used to
enhance code functionality.
Syntax
The syntax for defining a descriptor in Python is straightforward. Here
is an example:
class Descriptor:
def __get__(self, instance, owner):
print("Getting the attribute")
return instance._value
def __set__(self, instance, value):
print("Setting the attribute")
instance._value = value
class MyClass:
def __init__(self, value):
self._value = value
x = Descriptor()
In the example above, we define a class called 'Descriptor' that has
two special methods: 'get' and 'set'. These methods are called by the
interpreter when an attribute is accessed or assigned in an instance
of the class that uses the descriptor. We then define a class called
'MyClass' that has an instance variable '_value' and a descriptor
called 'x'. When the 'x' attribute is accessed or assigned, the
corresponding 'get' and 'set' methods of the descriptor are called.
Code Example
Here is an example that demonstrates how descriptors can be used
to enhance code functionality:
class PositiveNumber:
if value < 0:
raise ValueError("Value must be
positive")
class MyClass:
x = PositiveNumber()
def __init__(self, x):
self.x = x
In the example above, we define a descriptor called 'PositiveNumber'
that ensures that any value assigned to the attribute it is used on
must be positive. We then define a class called 'MyClass' that uses
the 'PositiveNumber' descriptor on the 'x' attribute. When an instance
of 'MyClass' is created, it initializes the 'x' attribute to the value
passed in to the constructor, but if that value is negative, a
'ValueError' is raised.
Using metaclasses
Metaclasses are a powerful feature of Python that allow you to

modify the behavior of a class when it is defined. Metaclasses can be
used to customize the way classes are constructed, add or modify
class attributes, and perform other advanced operations.
To take advantage of Python's metaclass feature, we can create our
own custom metaclasses that define how classes are created and
behave. Let's take a look at an example:
class MyMeta(type):
def __new__(cls, name, bases, attrs):
print("Creating class:", name)
return super().__new__(cls, name, bases,
attrs)
class MyClass(metaclass=MyMeta):
pass
In this example, we define a custom metaclass MyMeta that will be
used to create the class MyClass. The __new__ method of the
metaclass is called when MyClass is defined, and it prints a message
indicating that the class is being created.
To use the custom metaclass, we pass it as the metaclass argument
when defining the class, as shown in the MyClass definition.
Now, let's look at another example that demonstrates the ability of
metaclasses to modify class attributes:
class MyMeta(type):
attrs['my_attribute'] = 42
attrs)
pass
print(MyClass.my_attribute) # Output: 42
In this example, the MyMeta metaclass adds an attribute
my_attribute to the class MyClass. When we access this attribute on
an instance of MyClass, we get the value 42.
These are just a few examples of how you can use metaclasses in
Python. With metaclasses, you have the power to customize the
behavior of classes in many different ways, so feel free to experiment
and see what you can do!
Writing idiomatic Python

Writing Pythonic loops
Python is a powerful and versatile programming language, known for

its readability and expressiveness. One of the key features that
makes Python stand out is its ability to write clean, concise and
Pythonic code. In this note, we'll focus on writing Pythonic loops,
which are loops that are written in a way that is idiomatic to the
Python language. We'll cover some common scenarios and best
practices for writing Pythonic loops.
Looping through a list:
A common scenario in Python is to loop through a list of items. Here's
an example of a non-Pythonic way of doing this:
my_list = [1, 2, 3, 4, 5]
for i in range(len(my_list)):
print(my_list[i])
In this code, we're using the range function to generate a sequence
of integers that correspond to the indices of the list. We then use the
index to access each item in the list. While this code works, it's not
very Pythonic. A better way to write this code would be:
my_list = [1, 2, 3, 4, 5]
for item in my_list:
print(item)
In this code, we're directly iterating over the items in the list using a
for loop. This is the Pythonic way of looping through a list.
Looping through a dictionary:
Another common scenario is to loop through a dictionary. Here's an
example of a non-Pythonic way of doing this:
my_dict = {'a': 1, 'b': 2, 'c': 3}

for key in my_dict:
value = my_dict[key]
print(key, value)
In this code, we're iterating over the keys in the dictionary and then
using the key to access the corresponding value. While this code
works, it's not very Pythonic. A better way to write this code would
be:
my_dict = {'a': 1, 'b': 2, 'c': 3}

for key, value in my_dict.items():
print(key, value)
In this code, we're using the items method of the dictionary to directly
iterate over the key-value pairs. This is the Pythonic way of looping
through a dictionary.
Looping with a condition:
Sometimes, we want to loop through a list or dictionary and only
process items that meet a certain condition. Here's an example of a
non-Pythonic way of doing this:
my_list = [1, 2, 3, 4, 5]
for i in range(len(my_list)):
if my_list[i] > 2:
print(my_list[i])
In this code, we're using the range function to generate a sequence
of integers that correspond to the indices of the list. We then use the
index to access each item in the list and check if it meets a condition.
While this code works, it's not very Pythonic. A better way to write
this code would be:
my_list = [1, 2, 3, 4, 5]
if item > 2:
print(item)
In this code, we're directly iterating over the items in the list using a
for loop and checking the condition using an if statement. This is the
Pythonic way of looping with a condition.
Using enumerate and zip
Python provides two built-in functions, enumerate and zip, that are
very useful for looping over sequences and iterating over multiple
sequences at the same time. In this note, we'll focus on how to use
enumerate and zip effectively in your Python code.
Using enumerate:
The enumerate function is used to iterate over a sequence and keep
track of the index of the current item. Here's an example of how to
use enumerate:
my_list = ['apple', 'banana', 'orange']

for i, item in enumerate(my_list):
print(i, item)
In this code, we're using enumerate to loop through the my_list
sequence and keep track of the index and the current item. The
output of this code would be:
0 apple
1 banana
2 orange
This is a very common pattern in Python code, especially when you
need to access the index of the current item.
Using zip:
The zip function is used to iterate over multiple sequences at the
same time, combining their corresponding items into tuples. Here's an
example of how to use zip:
list_a = [1, 2, 3]
list_b = ['a', 'b', 'c']
for item_a, item_b in zip(list_a, list_b):
print(item_a, item_b)
In this code, we're using zip to loop through both list_a and list_b at
the same time, combining their corresponding items into tuples. The
output of this code would be:
1a
2b
3c
This is a very useful feature of Python when you need to iterate over
multiple sequences and process their corresponding items at the
same time.
Using enumerate and zip together:
One powerful pattern in Python is to use enumerate and zip together
to iterate over a sequence and its corresponding indices at the same
time. Here's an example of how to use enumerate and zip together:
my_list = ['apple', 'banana', 'orange']

for i, item in enumerate(zip(my_list,
range(len(my_list)))):
print(i, item[0], item[1])
In this code, we're using enumerate and zip together to loop through
my_list and its corresponding indices. The zip function is used to
combine my_list with a sequence of integers that correspond to the
indices of the list. The output of this code would be:
0 apple 0
1 banana 1
2 orange 2
This is a powerful pattern in Python that allows you to iterate over a
sequence and its corresponding indices at the same time, without
having to generate the sequence of integers using range.
Enumerate and zip are powerful built-in functions in Python that can
make your code more concise and readable. They are especially
useful when you need to access the index of the current item in a
sequence, or when you need to iterate over multiple sequences at the
same time.
Using the ternary operator
The ternary operator in Python is a powerful tool that allows you to

write concise, readable code. It is a shorthand way to write an if-else
statement and is often used to make code more readable by reducing
the amount of boilerplate code. In this note, we'll discuss how to write
idiomatic Python using the ternary operator.
Basic syntax:
The syntax of the ternary operator is as follows:
<expression_if_true> if <condition> else

<expression_if_false>
Here, <condition> is the boolean expression
that you want to evaluate, and
<expression_if_true> and
<expression_if_false> are the expressions
that will be returned if the condition is true or
false, respectively.
Writing idiomatic Python:
In Python, it's important to write code that is easy to read and
understand. When using the ternary operator, it's important to use it
in a way that is clear and concise.
One way to use the ternary operator in a clear and concise way is to
use it to assign a value to a variable. For example:
x = 10
y = 20
max_num = x if x > y else y
In this code, we're using the ternary operator to assign the maximum
value of x and y to the max_num variable. This code is concise and
easy to read.
Another way to use the ternary operator in a clear and concise way is
to use it to conditionally execute code. For example:
x = 10
y = 20
result = x * 2 if x > y else y * 2
In this code, we're using the ternary operator to conditionally execute
the x * 2 expression if x is greater than y, and the y * 2 expression if y
is greater than or equal to x. This code is also concise and easy to
read.
Examples of idiomatic Python:
Here are some examples of using the ternary operator in idiomatic
Python code:
# Example 1: Check if a value is in a list

my_list = [1, 2, 3, 4, 5]
if 6 in my_list:
index = my_list.index(6)
else:
index = -1
# This code can be written more
idiomatically using the ternary operator:
index = my_list.index(6) if 6 in my_list else -1
# Example 2: Set a variable to a default value
if it is None
my_var = None
if my_var is None:
my_var = "default_value"
my_var = my_var if my_var is not None else
"default_value"
# Example 3: Check if a variable is empty
my_var = ""
if len(my_var) == 0:
is_empty = True
else:
is_empty = False
is_empty = True if len(my_var) == 0 else
False
In each of these examples, we're using the ternary operator to write

more concise and readable code.
Using the ternary operator in Python can help you write concise and
readable code. When using the ternary operator, it's important to use
it in a way that is clear and concise. By following the examples in this
note, you can learn how to write idiomatic Python using the ternary
operator.
Using multiple assignment
Multiple assignment is a powerful feature in Python that allows you to

assign multiple variables at once. It can make your code more
readable and concise by reducing the amount of boilerplate code. In
this note, we'll discuss how to write idiomatic Python using multiple
assignment.
Basic syntax:
The syntax for multiple assignment in Python is as follows:
a, b = 10, 20
In this code, we're assigning the values 10 and 20 to the variables a
and b, respectively. This code is equivalent to writing:
a = 10
b = 20
Using multiple assignment can help you reduce the amount of code
you need to write and make your code more readable.
When using multiple assignment in Python, it's important to use it in a
way that is clear and concise. Here are some tips for writing idiomatic
Python using multiple assignment:
Use tuple packing and unpacking: Python allows you to pack multiple
values into a tuple and then unpack them into variables using multiple
assignment. For example:
my_tuple = (10, 20, 30)

a, b, c = my_tuple
In this code, we're packing the values 10, 20, and 30 into the
my_tuple tuple, and then unpacking them into the variables a, b, and
c, respectively. This code is equivalent to writing:
my_tuple = (10, 20, 30)

a = my_tuple[0]
b = my_tuple[1]
c = my_tuple[2]
Using tuple packing and unpacking can make your code more concise
and readable.
Use multiple assignment with functions that return multiple values:
Many functions in Python return multiple values as tuples. For
example, the divmod() function returns the quotient and remainder of
a division operation as a tuple. You can use multiple assignment to
assign these values to separate variables. For example:
quotient, remainder = divmod(10, 3)

In this code, we're using multiple assignment to assign the quotient
and remainder of the division operation 10 / 3 to the variables
quotient and remainder, respectively.
Use multiple assignment to swap variable values: In Python, you can
swap the values of two variables using multiple assignment. For
example:
a, b = b, a
In this code, we're swapping the values of a and b. This code is
equivalent to writing:
temp = a
a=b
b = temp
Using multiple assignment to swap variable values can make your
code more concise and readable.
Examples of idiomatic Python:
Here are some examples of using multiple assignment in idiomatic
Python code:
# Example 1: Unpack a tuple returned by a

function
def get_numbers():
return 10, 20, 30
a, b, c = get_numbers()
# Example 2: Swap variable values

x, y = 10, 20
x, y = y, x
# Example 3: Assign default values to

multiple variables
x, y = None, None
x = x or 10
y = y or 20
In each of these examples, we're using multiple assignment to write
more concise and readable code.
Using the walrus operator
The walrus operator, also known as the assignment expression, is a

new feature introduced in Python 3.8 that allows you to assign values
to variables as part of an expression. It can be used to write more
concise and readable code in certain situations. In this note, we'll
discuss how to write idiomatic Python using the walrus operator.
Basic syntax:
The syntax for the walrus operator in Python is as follows:
variable := expression
In this code, we're assigning the result of the expression to the
variable using the walrus operator. The := symbol is the walrus
operator.
When using the walrus operator in Python, it's important to use it in a
way that is clear and concise. Here are some tips for writing idiomatic
Python using the walrus operator:
Use it in list comprehensions: The walrus operator can be useful in list
comprehensions when you want to filter the list based on a condition
and then use the filtered list in the same expression. For example:
numbers = [1, 2, 3, 4, 5]
squares = [x ** 2 for x in numbers if (y := x **
2) > 10]
In this code, we're using the walrus operator to assign the result of x
** 2 to the variable y, and then using y in the same expression to filter
the list based on the condition y > 10.
Use it to simplify if-else statements: The walrus operator can also be
used to simplify if-else statements when you need to assign a value
to a variable based on a condition. For example:
name = input("What is your name? ")

greeting = f"Hello, {name}" if (name :=
name.strip()) else "Hello, Stranger"
In this code, we're using the walrus operator to assign the result of
name.strip() to the variable name, and then using name in the same
expression to determine the value of the greeting variable based on
whether or not name is empty after stripping.
Using context managers
Python context managers provide a convenient way to manage

resources and ensure proper clean-up, even in the presence of
exceptions or other errors. In this note, we will discuss how to use
context managers in Python to write more idiomatic and readable
code.
A context manager is an object that defines the methods __enter__
and __exit__. The __enter__ method is called when the context
manager is entered, and the __exit__ method is called when the
context manager is exited. The with statement is used to invoke the
context manager.
Here's an example of using a context manager to open a file:
with open("example.txt", "r") as f:

contents = f.read()
In this example, the open function returns a file object, which is used
as a context manager in the with statement. The __enter__ method
of the file object is called when the with statement is executed, and
the file is opened for reading. The __exit__ method of the file object
is called when the with block is exited, and the file is closed.
Here's another example of using a context manager to lock a
resource:
import threading
lock = threading.Lock()
with lock:
# do some thread-safe operation here
In this example, the threading.Lock object is used as a context
manager in the with statement. The __enter__ method of the lock is
called when the with statement is executed, and the lock is acquired.
The __exit__ method of the lock is called when the with block is
exited, and the lock is released.
Now, let's look at some tips for writing more idiomatic Python code
using context managers:
Use the with statement whenever possible to ensure proper

clean-up of resources.
Use context managers provided by the standard library

whenever possible, such as open, threading.Lock,
contextlib.suppress, etc.
Use the contextlib.ContextDecorator class to create context

managers that can be used as function decorators.
Use the contextlib.ExitStack class to manage multiple

context managers.
Here's an example of using contextlib.ContextDecorator to create a

context manager that measures the time taken by a function:
import contextlib
import time
@contextlib.ContextDecorator
def timeit(func):
start = time.time()
yield
end = time.time()
print(f"{func.__name__} took {end - start}
seconds")
In this example, the timeit function is a context manager that
measures the time taken by a function. The function is passed as an
argument to the timeit function, and the yield statement is used to
indicate the point where the function should be executed. When the
with block is exited, the time taken by the function is printed to the
console.
Finally, here's an example of using contextlib.ExitStack to manage
multiple context managers:
import contextlib
class DatabaseConnection:
def __init__(self, database_url):
self.database_url = database_url
def connect(self):
# connect to database here
pass
def disconnect(self):
# disconnect from database here
pass
class HttpConnection:
def __init__(self, http_url):
self.http_url = http_url
def connect(self):
# connect to http server here
pass
def disconnect(self):
# disconnect from http server here
pass
Chapter 3:
Functions
Functions are an essential part of programming and are used in
almost every programming language. Functions are a set of
instructions that perform a specific task or set of tasks. They help in
organizing the code and make it easier to read, understand, and
maintain. In Python, functions are defined using the "def" keyword
and are an integral part of the language.
Python functions are powerful and flexible, allowing developers to
perform complex operations with ease. They can be used to
encapsulate code, which makes it reusable and reduces the amount
of code that needs to be written. This, in turn, reduces the chances of
introducing errors in the code.
Functions in Python can be simple or complex, depending on the task
they perform. Simple functions perform a single task, while complex
functions perform a set of tasks. Regardless of their complexity,
Python functions are easy to define, use, and understand.
One of the key features of Python functions is that they can be called
multiple times from different parts of the code. This makes it easy to
reuse code and avoid repetition. Functions in Python can also take
parameters, which allows them to be customized based on the needs
of the program.
Python functions can also return values, which makes it possible to
use them in complex operations. The return statement is used to
return a value from a function. The returned value can be used in
other parts of the program, making it possible to perform complex
operations with ease.
Python also has built-in functions that can be used without defining
them. These functions are part of the Python standard library and can
be used to perform common tasks. Some examples of built-in
functions include print(), len(), and input().
In Python, functions can also be defined inside other functions. These
are called nested functions and are used when a function performs a
specific task that is used only in the context of the main function.
Nested functions make it easy to organize code and make it more
readable.
Another important feature of functions in Python is recursion.
Recursion is a technique where a function calls itself, either directly or
indirectly. This technique is used when a function needs to perform a
specific task repeatedly.
In summary, functions are an essential part of programming in
Python. They are used to perform a specific task or set of tasks and
help in organizing the code, making it easier to read, understand, and
maintain. Python functions are powerful and flexible, allowing
developers to perform complex operations with ease. They can be
called multiple times from different parts of the code, take
parameters, return values, and can be defined inside other functions.
By mastering functions in Python, developers can write code that is
more efficient, flexible, and reusable.
Function basics
Function arguments and return values
In Python, function arguments and return values are an essential part

of the language, allowing for the creation of reusable code that can
be easily called with different inputs and produce different outputs. In
this note, we will discuss function arguments and return values in
Python, including their types and how they can be used.
Function Arguments in Python:
In Python, there are four types of function arguments:
Positional Arguments
Keyword Arguments
Default Arguments
Variable-length Arguments
Positional Arguments:
Positional arguments are the most basic type of function argument.
These are the arguments that are passed to a function in the order
they are defined in the function definition.
def add_numbers(x, y):

return x + y
result = add_numbers(3, 5)
print(result) # Output: 8
In the example above, x and y are the two positional arguments
passed to the add_numbers function. The order of the arguments is
important, so if we were to switch the order of the arguments, we
would get a different result.
Keyword Arguments:
Keyword arguments are a way to pass arguments to a function using
their parameter names. This allows us to pass arguments in any
order, as long as we specify the parameter names.
def subtract_numbers(x, y):

return x - y
result = subtract_numbers(x=10, y=3)
In the example above, we are using keyword arguments to pass x
and y to the subtract_numbers function. We can also mix positional
and keyword arguments in the same function call:
result = subtract_numbers(10, y=3)

Default Arguments:
Default arguments are a way to specify a default value for a function
parameter. If the argument is not passed when the function is called,
the default value is used instead.
def greet(name, greeting="Hello"):

print(f"{greeting}, {name}!")
greet("John") # Output: Hello, John!
greet("Mary", "Hi") # Output: Hi, Mary!
In the example above, we are using a default argument to specify the
default greeting if one is not provided. If we pass a greeting
argument, it will override the default value.
Variable-length Arguments:
Variable-length arguments allow a function to accept an arbitrary
number of arguments. There are two types of variable-length
arguments in Python:
*args: This allows a function to accept an arbitrary number of
positional arguments.
**kwargs: This allows a function to accept an arbitrary number of
keyword arguments.
def multiply_numbers(*args):
result = 1
for arg in args:
result *= arg
return result
result = multiply_numbers(2, 3, 4)
def print_values(**kwargs):
for key, value in kwargs.items():
print(f"{key} = {value}")
print_values(name="John", age=30,
city="New York")
# Output:
# name = John
# age = 30
# city = New York
In the example above, we are using *args to accept an arbitrary
number of positional arguments in the multiply_numbers function, and
**kwargs to accept an arbitrary number of keyword arguments in the
print_values function.
Documenting functions
Documenting functions in Python is an important aspect of writing

clean and maintainable code. A well-documented function helps other
developers understand its purpose, inputs, outputs, and any potential
side effects. In this note, we will discuss how to document functions
in Python, including the use of docstrings and annotations.
Docstrings in Python:
Docstrings are a way to document functions, modules, and classes in
Python. A docstring is a string that appears as the first statement in a
module, function, or class definition. Docstrings can be accessed
using the built-in help() function or by typing the function name
followed by a question mark in an interactive Python session.
There are two types of docstrings in Python: one-line docstrings and
multi-line docstrings. One-line docstrings are used for simple
functions and consist of a single line of text enclosed in triple quotes.

"""Add two numbers and return the
result."""
return x + y
Multi-line docstrings are used for more complex functions and consist
of a brief summary followed by a more detailed description of the
function's purpose, inputs, outputs, and any side effects. Multi-line
docstrings are enclosed in triple quotes and can span multiple lines.
def greet(name):
"""
Greet the given name.
This function takes a name as input and prints a greeting message to
the console.
It does not return any value.
Args:
name (str): The name to greet.
Returns:
None
"""
print(f"Hello, {name}!")
Annotations in Python:
Function annotations are another way to document functions in
Python. Annotations are optional metadata that can be added to
function arguments and return values using the colon syntax.
def add_numbers(x: int, y: int) -> int:

"""Add two integers and return the
result."""
return x + y
In the example above, we are using annotations to specify that the x
and y arguments should be integers, and that the function should
return an integer.
Annotations can also be used to specify default argument values and
variable-length arguments.
def greet(name: str = "World", *args: str,

**kwargs: str) -> None:
"""
Greet the given name and print any additional arguments and
keyword arguments.
This function takes a name as input and prints a greeting message to
the console.
It can also take additional positional and keyword arguments, which
will be printed.
Args:
name (str): The name to greet. Defaults
to "World".
args (str): Additional positional
arguments.
kwargs (str): Additional keyword
arguments.
Returns:
None
"""
if args:
print("Additional arguments:")
for arg in args:
print(f"- {arg}")
if kwargs:
print("Additional keyword arguments:")
print(f"- {key}: {value}")
In the example above, we are using annotations to specify that the
name argument should be a string with a default value of "World", and
that *args should be an arbitrary number of positional string
arguments, and **kwargs should be an arbitrary number of keyword
string arguments.
Documenting functions in Python is an essential part of writing clean
and maintainable code. Docstrings and annotations are powerful tools
that can help other developers understand the purpose, inputs,
outputs, and any potential side effects of a function. By following best
practices for function documentation, we can make our code more
accessible and easier to maintain.
Writing doctests
Writing doctests in Python is a way to test functions by embedding

test cases in their docstrings. This can help ensure that the function
works as intended and also serves as a form of documentation. In
this note, we will discuss how to write doctests in Python, including
the syntax and best practices.
Syntax:
Doctests are written in the docstring of a function and consist of a
series of input/output pairs. Each input/output pair consists of a
prompt, a function call, and the expected output. The prompt is a
string that describes the test case and the function call is the
expression to be evaluated.
Here is an example of a simple function with a doctest:

"""
Add two numbers and return the result.
>>> add_numbers(2, 3)
5
>>> add_numbers(-1, 1)
0
"""
return x + y
In the example above, we have defined a function add_numbers that
takes two numbers and returns their sum. We have also included two
doctests in the function's docstring. The first doctest checks that
add_numbers(2, 3) returns 5 and the second doctest checks that
add_numbers(-1, 1) returns 0.
Running Doctests:
Doctests can be run using the built-in doctest module in Python. To
run doctests for a module, simply call doctest.testmod() at the
bottom of the module.
import doctest
"""
Add two numbers and return the result.
>>> add_numbers(2, 3)
5
>>> add_numbers(-1, 1)
0
"""
return x + y
if __name__ == '__main__':
doctest.testmod()
In the example above, we have imported the doctest module and
called doctest.testmod() at the bottom of the module to run the
doctests. When the module is executed, the testmod() function will
search for doctests in the module's docstrings and execute them. If
all tests pass, nothing will be printed to the console. If a test fails, an
error message will be printed to the console.
Best Practices:
When writing doctests in Python, it is important to follow best
practices to ensure that the tests are effective and maintainable.
Here are some tips:
Write doctests for all functions and methods.
Include only one input/output pair per test case.
Use descriptive prompts that describe the input and
expected output.
Avoid using variables in prompts that are not defined in the
function.
Use assert statements for more complex test cases.
Use triple quotes for multi-line prompts and output.
Writing doctests in Python is an effective way to test functions and

document their behavior. By following best practices and including
doctests for all functions, we can ensure that our code is more
reliable and maintainable.
Writing function annotations
Function annotations in Python are used to specify the expected data

types of function arguments and return values. These annotations are
not enforced by the interpreter, but they can be used by other tools
such as linters, type checkers, and IDEs to provide better code
completion, type checking, and documentation.
To write function annotations in Python, you can use the colon syntax
to indicate the expected data type. For example, to specify that the
function add expects two integers as arguments and returns an
integer, you can write:
def add(x: int, y: int) -> int:
return x + y
In this example, the int before the argument name and after the ->
arrow indicate that x and y should be integers, and that the return
value should also be an integer.
You can use any data type as a function annotation, including built-in
types like int, float, str, bool, and None, as well as user-defined types
and even generic types from the typing module.
Here are some more examples:
def greet(name: str) -> str:

return f"Hello, {name}!"
def divide(x: float, y: float) -> float:
return x / y
def repeat_string(s: str, n: int) -> str:
return s * n
def process_data(data: List[Dict[str, Any]]) ->
Dict[str, Any]:
# process data and return result
In this last example, we use the List and Dict types from the typing
module to specify that the data argument should be a list of
dictionaries with string keys and values of any type, and that the
return value should be a dictionary with string keys and values of any
type.
It's worth noting that function annotations are optional in Python, and
they do not affect the behavior of the function in any way. However,
they can be very useful for providing documentation and improving
code quality, especially when working on larger projects with multiple
developers.
In addition to function annotations, you can also use type hints to
specify the expected types of variables and attributes. For example:
name: str = "Alice"

age: int = 30
class Person:
def __init__(self, name: str, age: int):
self.name = name
self.age = age
p = Person("Bob", 25)
In this example, we use type hints to specify that the name variable
should be a string and the age variable should be an integer, and we
use the same annotations in the Person class constructor to specify
the expected types of the name and age attributes.
Overall, using function annotations and type hints can help make your
code more readable, maintainable, and reliable. While they are not
required in Python, they are a powerful tool for improving code quality
and reducing errors.
Using default arguments
In Python, you can define default arguments for function parameters.

Default arguments are values that are automatically assigned to a
parameter if no argument is provided for that parameter. This allows
you to write functions that are more flexible and can handle different
scenarios with minimal changes to the code.
To use default arguments in Python, you simply need to provide a
default value for a parameter when defining the function. For
example, consider the following function that adds two numbers:
def add(x, y):

return x + y
This function works fine when you call it with two arguments:
>>> add(2, 3)
5
But what if you want to use the same function to add only one number
to a fixed value? You could modify the function to take a default value
for y:
def add(x, y=0):

return x + y
Now, if you call add with only one argument, it will add that argument
to 0 (the default value of y):
>>> add(2)
2
And if you call add with two arguments, it will add them together as
before:
>>> add(2, 3)
5
You can also use default arguments to make a function more flexible
by allowing the user to customize some behavior without having to
modify the function code. For example, consider the following function
that prints a message:
def greet(name, greeting="Hello"):

This function takes a name argument and a greeting argument that
defaults to "Hello". If you call the function with only a name argument,
it will print "Hello, {name}!":
>>> greet("Alice")
Hello, Alice!
But you can also provide a custom greeting:
>>> greet("Bob", "Good morning")

Good morning, Bob!
Using default arguments in Python can help you write more flexible
and reusable functions, and make your code easier to read and
maintain. However, you should be careful when using mutable objects
(such as lists or dictionaries) as default arguments, as their values
can be modified across multiple calls to the same function, leading to
unexpected behavior.
Using keyword arguments
In Python, you can use keyword arguments to pass arguments to a

function in any order you like. Keyword arguments are a way to
specify which argument corresponds to which parameter by using the
parameter name as a keyword when calling the function.
To use keyword arguments in Python, you simply need to provide the
argument name and its corresponding value when calling the function.
For example, consider the following function that takes two
arguments:
def greet(name, greeting):

To call this function with positional arguments, you need to provide the
arguments in the correct order:
>>> greet("Alice", "Hello")

Hello, Alice!
But you can also call this function with keyword arguments, specifying
the argument names explicitly:
>>> greet(name="Bob", greeting="Good

morning")
Good morning, Bob!
When using keyword arguments, you can provide the arguments in
any order you like:
>>> greet(greeting="Hi", name="Charlie")

Hi, Charlie!
Using keyword arguments can be especially useful when calling
functions that have many parameters or when you want to provide
default values for some of the parameters. For example, consider the
following function that takes three arguments, with the third argument
having a default value:
def divide(x, y, precision=2):

result = x / y
return round(result, precision)
If you call this function with only two arguments, it will use the default
value for precision:
>>> divide(10, 3)
3.33
But you can also provide a custom value for precision by using a
keyword argument:
>>> divide(10, 3, precision=4)

3.3333
Keyword arguments can help make your code more readable and
easier to maintain, especially when working with functions that have
many parameters or complex argument lists. By using keyword
arguments, you can make it clear which argument corresponds to
which parameter, and you can provide default values for some of the
parameters without having to modify the function code.
Using *args and **kwargs
In Python, you can use *args and **kwargs to define functions that
can accept an arbitrary number of arguments and keyword
arguments, respectively. These features can be particularly useful
when you don't know ahead of time how many arguments you will
need to pass to a function, or when you want to provide a flexible API
that can handle a variety of use cases.
*args is used to pass a variable number of positional arguments to a
function. When you use *args in a function definition, it tells Python to
collect any remaining positional arguments into a tuple. For example:
def my_function(*args):
for arg in args:
print(arg)
In this example, the function my_function() accepts any number of
positional arguments and prints each argument to the console. You
can call the function with any number of arguments:
my_function(1, 2, 3) # prints 1 2 3
my_function('a', 'b', 'c', 'd') # prints a b c d
my_function() # prints nothing
**kwargs is used to pass a variable number of keyword arguments to
a function. When you use **kwargs in a function definition, it tells
Python to collect any remaining keyword arguments into a dictionary.
For example:
def my_function(**kwargs):
print(f"{key}: {value}")
In this example, the function my_function() accepts any number of
keyword arguments and prints each argument key-value pair to the
console. You can call the function with any number of keyword
arguments:
my_function(name="Alice", age=25) # prints

name: Alice age: 25
my_function(city="Boston", state="MA",
country="USA") # prints city: Boston state:
MA country: USA
You can also use *args and **kwargs together to create functions
that can accept both positional and keyword arguments:
def my_function(*args, **kwargs):

for arg in args:
print(arg)
print(f"{key}: {value}")
In this example, the function my_function() accepts both positional
and keyword arguments and prints each argument to the console.
You can call the function with any combination of positional and
keyword arguments:
my_function(1, 2, 3, name="Alice", age=25) #
prints 1 2 3 name: Alice age: 25
my_function('a', 'b', city="Boston",
state="MA", country="USA") # prints a b city:
Boston state: MA country: USA
my_function(name="Bob") # prints name:
Bob
In summary, using *args and **kwargs in Python can help you create
functions that are more flexible and can handle a wider range of
inputs. These features can be especially useful when you don't know
ahead of time how many arguments you will need to pass to a
function, or when you want to provide a flexible API that can handle a
variety of use cases.
Function design
Writing pure functions
Python functions can be classified into two categories: pure functions

and impure functions. Pure functions are functions that don't cause
side effects and always produce the same output for the same input.
Pure functions are predictable, easy to test, and don't rely on any
external state. In this note, we'll discuss how to write pure functions in
Python and provide some sample code to illustrate the concept.
Avoid modifying global state:
A pure function should not modify any global state or modify any
variables outside of its scope. This includes modifying global variables
or objects that are passed by reference to the function.
# Impure function that modifies a global

variable
count = 0
def impure_add_one():
global count
count += 1
# Pure function that doesn't modify any

global state
def pure_add_one(num):
return num + 1
Avoid modifying input parameters:
A pure function should not modify its input parameters. This means
that the function should create a new object or copy the input object if
it needs to modify it.
# Impure function that modifies an input
parameter
def impure_append_list(item, lst):
lst.append(item)
# Pure function that creates a new list and

doesn't modify the input
def pure_append_list(item, lst):
return lst + [item]
Avoid relying on external state:
A pure function should not rely on any external state that could
change its behavior. This includes reading from global variables or
accessing data from external sources like files or databases.
# Impure function that relies on external state

def impure_get_current_time():
return datetime.datetime.now()
# Pure function that takes a time parameter
and doesn't rely on external state
def pure_format_time(time):
return time.strftime("%Y-%m-%d
%H:%M:%S")
Return a value:
A pure function should always return a value. This value should be
determined solely by the input parameters and not by any external
state.
# Impure function that prints a value instead

of returning it
def impure_print_hello(name):
print("Hello, " + name)
# Pure function that returns a greeting string

def pure_get_greeting(name):
return "Hello, " + name
Here's an example of a pure function that calculates the area of a
rectangle:
def calculate_area(length, width):

return length * width
This function takes two input parameters (length and width) and
returns their product (the area of the rectangle). It doesn't modify any
input parameters, global state, or rely on any external state.
In summary, writing pure functions in Python requires avoiding
modifying global state, input parameters, or relying on external state.
By following these principles, we can create functions that are
predictable, easy to test, and don't have any unintended side effects.
Writing functions with side effects
In Python, functions with side effects are those that modify state
outside of their own scope. These side effects can take many forms,
such as modifying global variables, changing the state of an object, or
interacting with external systems like databases or files. While pure
functions are often preferred in functional programming, there are
cases where side effects are necessary to achieve a specific
functionality. In this note, we'll discuss how to write functions with side
effects in Python and provide some sample code to illustrate the
concept.
Use global variables with care:

Global variables are variables that are declared outside of any
function and are accessible from any part of the program. Functions
that modify global variables can be useful, but they can also introduce
unexpected behavior and make code hard to maintain.
# Global variable
count = 0
# Function that modifies the global variable

def increment_count():
global count
count += 1
increment_count()
print(count) # Output: 1
In this example, we have a global variable count and a function
increment_count that modifies it. When we call increment_count, the
value of count is incremented by 1. However, using global variables
can make it difficult to track where changes are made and can
introduce bugs when different parts of the program modify the same
variable.
Modify object state with methods:
Object-oriented programming in Python allows us to modify object
state with methods. A method is a function that is associated with a
specific object or class and can modify its internal state.
class BankAccount:
def __init__(self, balance):
self.balance = balance
def deposit(self, amount):

self.balance += amount
def withdraw(self, amount):

self.balance -= amount
account = BankAccount(100)
account.deposit(50)
print(account.balance) # Output: 150
account.withdraw(25)
print(account.balance) # Output: 125
In this example, we have a class BankAccount that has methods
deposit and withdraw that modify the balance attribute. When we
create an instance of BankAccount, we can deposit or withdraw
funds by calling the respective methods. This allows us to
encapsulate the state of the object and provides a clean interface for
interacting with it.
Interact with external systems using libraries:
Sometimes we need to interact with external systems like databases,
files, or web services to achieve a specific functionality. In Python, we
can use libraries and modules to abstract away the details of these
interactions and provide a clean interface for our functions.
import requests
def fetch_data(url):
response = requests.get(url)
return response.content
In this example, we have a function fetch_data that uses the requests
library to make an HTTP request to a given URL and return the
response content. By using a library, we can hide the complexity of
making network requests and provide a simple function for our code
to interact with.
In summary, functions with side effects in Python can be useful for
achieving specific functionality, but care should be taken to minimize
their impact on the rest of the program. By using global variables with
care, modifying object state with methods, and interacting with
external systems using libraries, we can write functions that are
easier to reason about and maintain.
Writing functions that modify mutable arguments
In Python, mutable arguments are those that can be modified in

place, such as lists, dictionaries, and sets. When we pass a mutable
argument to a function, the function can modify it and these
modifications persist outside of the function's scope. However,
modifying mutable arguments can introduce unexpected behavior and
make code hard to maintain. In this note, we'll discuss how to write
functions that modify mutable arguments in Python and provide some
sample code to illustrate the concept.
Modify arguments in place:
One way to modify mutable arguments in a function is to modify them
in place. This means that we modify the original object directly
instead of creating a new object.
def add_item_to_list(item, lst):

lst.append(item)
my_list = [1, 2, 3]
add_item_to_list(4, my_list)
print(my_list) # Output: [1, 2, 3, 4]
In this example, we have a function add_item_to_list that takes an
item and a list and appends the item to the list. When we call this
function with 4 and my_list, the list is modified in place and the new
value [1, 2, 3, 4] is printed.
Return a new object:
Another way to modify mutable arguments in a function is to create a
new object and return it. This approach can be useful when we want
to preserve the original object and create a modified copy.
def reverse_list(lst):
return lst[::-1]
my_list = [1, 2, 3]
reversed_list = reverse_list(my_list)
print(my_list) # Output: [1, 2, 3]
print(reversed_list) # Output: [3, 2, 1]
In this example, we have a function reverse_list that takes a list and
returns a reversed copy of the list. When we call this function with
my_list, the original list is not modified, but a new reversed list is
created and returned.
Combine both approaches:
In some cases, it can be useful to combine both approaches and
modify the original object in place and return a new copy.
def remove_duplicates(lst):
unique_lst = list(set(lst))
lst.clear()
lst.extend(unique_lst)
my_list = [1, 2, 2, 3, 3, 3]
remove_duplicates(my_list)
print(my_list) # Output: [1, 2, 3]
In this example, we have a function remove_duplicates that takes a
list, creates a new unique list, clears the original list, and extends it
with the unique values. When we call this function with my_list, the
original list is modified in place and the new value [1, 2, 3] is printed.
In summary, when writing functions that modify mutable arguments in
Python, it's important to consider whether we want to modify the
original object in place, return a new object, or use a combination of
both approaches. By following best practices and being intentional
about our approach, we can write functions that are easier to reason
about and maintain.
Using the @staticmethod and @classmethod
decorators
Python is a powerful object-oriented programming language that

provides two built-in decorators @staticmethod and @classmethod to
create static and class methods, respectively. These decorators can
be used to define methods that are associated with a class instead of
an instance of the class.
Static Method:
A static method is a method that belongs to a class rather than an
instance of the class. This means that a static method can be called
on the class itself, without the need to create an object of the class.
Static methods are useful for creating utility functions that do not
require access to the instance or class variables.
The syntax for defining a static method using the @staticmethod
decorator is as follows:
class MyClass:
@staticmethod
def my_static_method(arg1, arg2, ...):
# function body
Here, the @staticmethod decorator is used to define a static method
named my_static_method() that takes any number of arguments.
Here's an example of a static method that calculates the factorial of a
number:
class Math:
@staticmethod
def factorial(n):
if n == 0:
return 1
else:
return n * Math.factorial(n-1)
print(Math.factorial(5)) # Output: 120

In this example, the factorial() method is defined as a static method
using the @staticmethod decorator. This method can be called
directly on the Math class, without the need to create an object of the
class.
Class Method:
A class method is a method that belongs to a class rather than an
instance of the class, but unlike a static method, it can access and
modify the class variables. Class methods are useful for creating
factory methods that return an instance of the class with specific
attributes.
The syntax for defining a class method using the @classmethod
decorator is as follows:
class MyClass:
@classmethod
def my_class_method(cls, arg1, arg2, ...):
# function body
Here, the @classmethod decorator is used to define a class method
named my_class_method() that takes any number of arguments,
including the cls parameter, which refers to the class itself.
Here's an example of a class method that creates an instance of a
class with specific attributes:
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
@classmethod
def from_birth_year(cls, name, birth_year):
age = datetime.date.today().year -
birth_year
return cls(name, age)
person = Person.from_birth_year('Alice',
1990)
print(person.age) # Output: 33
In this example, the from_birth_year() method is defined as a class
method using the @classmethod decorator. This method takes the
class (cls) as the first parameter, followed by the name and
birth_year parameters. The method calculates the age based on the
birth year, and returns an instance of the class with the name and age
attributes set.
Note that the cls parameter is used instead of the class name to
create the instance of the class, which makes the method more
flexible and easier to maintain.
The @staticmethod and @classmethod decorators are powerful
features in Python that allow us to define methods that are
associated with a class rather than an instance of the class. They are
useful for creating utility functions and factory methods, respectively.
Using partial functions
In Python, a partial function is a way of fixing a certain number of

arguments to a function, creating a new function with the remaining
arguments. This can be useful when we have a function that takes too
many arguments, and we want to simplify its use by fixing some of
them.
The functools module in Python provides the partial() function that
allows us to create a partial function from an existing function.
Creating a partial function:
To create a partial function, we need to import the partial function
from the functools module and call it with the original function and the
arguments to be fixed as its arguments. The resulting partial function
can be called with the remaining arguments, and it will automatically
pass the fixed arguments to the original function.
Here's an example of creating a partial function:
from functools import partial
def multiply(x, y):

return x * y
double = partial(multiply, 2)
In this example, we define a multiply() function that takes two
arguments and returns their product. We then create a partial function
called double by calling the partial() function with the multiply()
function and the argument 2. The resulting partial function fixes the x
argument to 2, and can be called with the y argument to double a
number.
Passing additional arguments to a partial function:

We can also pass additional arguments to a partial function when we
call it, and they will be appended to the fixed arguments in the order
they are passed.
Here's an example of passing additional arguments to a partial
function:
def multiply(x, y, z):

return x * y * z
double = partial(multiply, 2)
triple = partial(multiply, z=3)
print(double(5, 2)) # Output: 20
print(triple(5, 2)) # Output: 30
In this example, we define a multiply() function that takes three
arguments and returns their product. We create two partial functions,
double and triple, that fix the x and z arguments to 2 and 3,
respectively. We can then call the partial functions with the remaining
arguments, which will be appended to the fixed arguments.
Using partial functions with lambda functions:
We can also create partial functions using lambda functions. This can
be useful when we have a simple function that we want to partially
apply without defining a separate function.
Here's an example of using a lambda function to create a partial
function:
double = partial(lambda x, y: x * y, 2)
In this example, we define a lambda function that takes two
arguments and returns their product. We then create a partial function
called double by calling the partial() function with the lambda function
and the argument 2. The resulting partial function fixes the x argument
to 2, and can be called with the y argument to double a number.
Partial functions in Python provide a powerful way of fixing arguments
to an existing function, creating a new function that is easier to use.
We can create partial functions using the functools module and the
partial() function, and pass additional arguments to them when we
call them. We can also use lambda functions to create partial
functions without defining a separate function.
Function decorators and closures

Writing simple decorators
In Python, a decorator is a function that takes another function as

input and returns a modified version of that function. Decorators can
be used to modify the behavior of a function without changing its
source code. They are a powerful tool in Python that can help simplify
code and make it more modular.
Defining a Simple Decorator:
To define a simple decorator, we use the @ symbol followed by the
decorator function name before the function we want to modify. The
decorator function takes the original function as input, modifies it in
some way, and returns the modified function.
Here's an example of a simple decorator that adds a greeting before
a function is called:
def greeting_decorator(func):
def wrapper():
print("Hello!")
func()
return wrapper
@greeting_decorator
def say_hello():
print("Welcome to my program!")
say_hello()
In this example, we define a greeting_decorator() function that takes
the original function func as input, defines a new function wrapper()
that adds a greeting before calling func(), and returns wrapper(). We
then decorate the say_hello() function with @greeting_decorator,
which modifies it by adding a greeting before it is called.
When we call say_hello(), the output will be:
Hello!
Welcome to my program!
Passing Arguments to a Decorator:
We can also modify our decorator to accept arguments. This can be
useful when we want to modify the behavior of a function based on
some external parameter.
Here's an example of a decorator that accepts an argument:
def repeat(num):
def decorator(func):
for i in range(num):
func(*args, **kwargs)
return wrapper
return decorator
@repeat(3)
def say_hello(name):
say_hello("John")
In this example, we define a repeat() function that takes an argument
num, defines a decorator function that takes the original function func
as input, defines a new function wrapper() that repeats the call to
func() num times, and returns wrapper(). We then decorate the
say_hello() function with @repeat(3), which modifies it by repeating
the call to the function three times.
When we call say_hello("John"), the output will be:
Hello, John!
Hello, John!
Hello, John!
Using Multiple Decorators:
We can also use multiple decorators to modify a function. In this
case, the decorators are applied in order from top to bottom.
Here's an example of using multiple decorators:
def bold_decorator(func):
return f"{func(*args, **kwargs)}"
return wrapper
def italic_decorator(func):
return f"{func(*args, **kwargs)}"
return wrapper
@bold_decorator
@italic_decorator
def say_hello():
return "Hello!"
print(say_hello())
In this example, we define two decorators, bold_decorator() and
italic_decorator(), that modify the output of the original function by
adding HTML tags. We then decorate the say_hello() function with
@bold_decorator and @italic_decorator, which modifies it by adding
both decorators in sequence.
When we call say_hello(), the output will be:
Hello!
Writing decorators that take arguments
In Python, decorators are functions that take a function as input and

return a modified function as output. Decorators can be used to
modify the behavior of a function without changing its source code. In
some cases, we may want to write a decorator that takes
arguments. In this case, we need to define a function that takes the
arguments and returns a decorator function that takes the original
function as input.
Defining a Decorator that Takes Arguments:
To define a decorator that takes arguments, we define a function that
takes the arguments and returns a decorator function that takes the
original function as input. The decorator function then defines a new
function that modifies the original function in some way and returns
the modified function.
Here's an example of a decorator that takes arguments:
def repeat(num):
for i in range(num):
return wrapper
return decorator
@repeat(3)
say_hello("John")
In this example, we define a repeat() function that takes an argument
num, defines a decorator function that takes the original function func
as input, defines a new function wrapper() that repeats the call to
func() num times, and returns wrapper(). We then decorate the
say_hello() function with @repeat(3), which modifies it by repeating
the call to the function three times.
Hello, John!
Hello, John!
Hello, John!
Passing Arguments to the Decorator:
In some cases, we may want to pass arguments to the decorator
itself. In this case, we need to define a function that takes the
arguments and returns the decorator function.
Here's an example of a decorator that takes arguments:
def greeting_decorator(greeting):
print(greeting)
return wrapper
return decorator
@greeting_decorator("Welcome!")
say_hello("John")
In this example, we define a greeting_decorator() function that takes
an argument greeting, defines a decorator function that takes the
original function func as input, defines a new function wrapper() that
adds the greeting before calling func(), and returns wrapper(). We
then decorate the say_hello() function with
@greeting_decorator("Welcome!"), which modifies it by adding a
greeting before it is called.
Welcome!
Hello, John!
Python decorators are a powerful feature of the language that allow

you to modify or enhance the behavior of functions or classes without
changing their source code. They are essentially functions that take
another function or class as an argument, modify it, and return the
modified version.
In this note, we will focus on writing class decorators in Python. Class
decorators work in a similar way to function decorators, but they take
a class as an argument instead of a function.
To write a class decorator in Python, you define a function that takes
a class as an argument and returns a modified version of the class.
The modified class can have additional methods or attributes, or it
can modify the behavior of existing methods.
Here is an example of a simple class decorator that adds a "version"
attribute to a class:
def add_version(cls):
cls.version = "1.0"
return cls
@add_version
class MyClass:
pass
print(MyClass.version) # Output: 1.0

In the example above, the add_version function takes a class cls as
an argument, adds a version attribute to it, and returns the modified
class. The @add_version decorator is then applied to the MyClass
class, which modifies it by adding the version attribute.
You can also chain multiple class decorators together to modify a
class:
def add_version(cls):
cls.version = "1.0"
return cls
def add_author(cls):
cls.author = "John Doe"
return cls
@add_version
@add_author
class MyClass:
pass
print(MyClass.version) # Output: 1.0

print(MyClass.author) # Output: John Doe
In the example above, the add_version and add_author class
decorators are chained together using the @ symbol. When the
MyClass class is defined, both decorators are applied in the order
they appear, resulting in a class that has both a version and author
attribute.
Class decorators can also modify the behavior of methods in a class.
For example, the following class decorator logs the execution time of
all methods in a class:
import time
def log_execution_time(cls):
for name, value in vars(cls).items():
if callable(value):
def new_func(*args, **kwargs):
start_time = time.time()
result = value(*args, **kwargs)
end_time = time.time()
print(f"Execution time of {name}:
{end_time - start_time}")
return result
setattr(cls, name, new_func)
return cls
@log_execution_time
class MyClass:
def method1(self):
time.sleep(1)
def method2(self):
time.sleep(2)
my_obj = MyClass()
my_obj.method1() # Output: Execution time
of method1: 1.000123
my_obj.method2() # Output: Execution time
of method2: 2.000234
In the example above, the log_execution_time function takes a class
cls as an argument and loops through all its attributes. If an attribute
is a method, a new function is created that logs the execution time of
the method using the time module. The setattr function is then used to
replace the original method with the new function.
The @log_execution_time decorator is then applied to the MyClass
class, which modifies it by replacing its methods with versions that log
their execution time.
Using closures
Closures are a powerful feature of Python that allow you to create

functions with persistent state. A closure is a function that remembers
the values of variables that were in scope at the time it was defined.
This makes it possible to create functions that have a "memory" and
can retain information between calls.
To create a closure in Python, you define a function inside another
function and return it. The inner function has access to the variables in
the outer function's scope, even after the outer function has
completed execution. Here is an example:
def outer_function(x):
def inner_function(y):
return x + y
closure = outer_function(10)
print(closure(5)) # Output: 15
In the example above, the outer_function takes a parameter x and
defines an inner function inner_function that takes another parameter
y. The inner function returns the sum of x and y.
When the outer_function is called with an argument of 10, it returns
the inner_function. This creates a closure that remembers the value
of x as 10. The closure is then assigned to the variable closure.
When the closure is called with an argument of 5, it invokes the
inner_function with y equal to 5 and returns the sum of x and y, which
is 15.
Closures are often used to create functions with persistent state. For
example, you can create a counter function using a closure like this:
def counter():
count = 0
def inner_function():
nonlocal count
count += 1
return count
my_counter = counter()
print(my_counter()) # Output: 1
In the example above, the counter function defines an inner function
inner_function that has access to a variable count in the outer
function's scope. The inner function increments the value of count
each time it is called and returns it.
When the counter function is called, it returns the inner_function. This
creates a closure that remembers the value of count as 0. The
closure is then assigned to the variable my_counter.
Each time my_counter is called, it invokes the inner_function and
returns the current value of count. Because the closure persists
between calls, the value of count is incremented each time and the
counter function behaves as expected.
In summary, closures are a powerful feature of Python that allow you
to create functions with persistent state. They are created by defining
a function inside another function and returning it. The inner function
has access to the variables in the outer function's scope, even after
the outer function has completed execution. Closures are often used
to create functions with persistent state, such as counters or
memoization functions.
Using functools.partial
functools.partial is a Python built-in module that allows you to create

a new function with some of the parameters of an existing function
already "filled in". It is a useful tool for making functions more flexible
and reusable.
To use functools.partial, you first need to import it:

Once you have imported partial, you can use it to create a new
function based on an existing function. Here is an example:
def multiply(x, y):

return x * y
double = partial(multiply, y=2)
In the example above, the multiply function takes two parameters x
and y and returns their product. The partial function is used to create
a new function double based on multiply, with y set to 2. This means
that double only takes one parameter x, and always multiplies it by 2.
When double is called with an argument of 5, it invokes the multiply
function with x equal to 5 and y equal to 2, and returns the product of
the two, which is 10.
partial can also be used to fill in multiple parameters of a function.
Here is an example:
def power(base, exponent):

return base ** exponent
square = partial(power, exponent=2)

cube = partial(power, exponent=3)
print(square(5)) # Output: 25
print(cube(5)) # Output: 125
In the example above, the power function takes two parameters base
and exponent and returns base raised to the power of exponent. The
partial function is used to create two new functions square and cube
based on power, with exponent set to 2 and 3, respectively.
When square is called with an argument of 5, it invokes the power
function with base equal to 5 and exponent equal to 2, and returns the
square of 5, which is 25. Similarly, when cube is called with an
argument of 5, it invokes the power function with base equal to 5 and
exponent equal to 3, and returns the cube of 5, which is 125.
In summary, functools.partial is a powerful tool for making functions
more flexible and reusable by allowing you to create new functions
based on existing ones with some parameters already filled in. It is
particularly useful for creating functions that are similar but have
different default values for some parameters.
Chapter 4:
Classes and Objects
Python is an object-oriented programming language that is widely
used by developers all over the world. It is a high-level language that
is simple to read and write, making it ideal for beginners. Python is
known for its strong support for object-oriented programming, which
is a programming paradigm that focuses on creating objects, which
are instances of classes, to represent real-world entities.
Classes and objects are the building blocks of object-oriented
programming in Python. They allow developers to create reusable
and maintainable code that is easy to read and understand. A class is
a blueprint for creating objects, while an object is an instance of a
class. Python provides a simple syntax for creating and using classes
and objects, which makes it easy for developers to write object-
oriented code.
In this chapter, we will explore the basics of classes and objects in
Python. We will start by defining classes and objects and explain how
they are related. We will then look at how to create objects from
classes, and how to use them to perform various tasks. We will also
cover the different attributes and methods that can be defined in a
class and how to access them from an object.
We will also examine how inheritance works in Python, which is the
mechanism for creating new classes from existing ones. Inheritance
allows developers to reuse code and create new classes that inherit
the attributes and methods of existing classes. We will explore the
different types of inheritance, including single inheritance and multiple
inheritance, and explain how to use them in your code.
Additionally, we will look at some advanced topics related to classes
and objects in Python. We will discuss the concept of encapsulation,
which is the practice of hiding data and methods within a class to
protect them from external access. We will also cover the concept of
polymorphism, which allows objects of different classes to be used
interchangeably.
Finally, we will provide some practical examples of how to use
classes and objects in real-world scenarios. We will demonstrate how
to create classes for representing common objects such as bank
accounts and cars, and show how to use them to perform various
operations. We will also provide examples of how to use inheritance
to create new classes that inherit attributes and methods from
existing classes.
By the end of this chapter, you will have a thorough understanding of
classes and objects in Python, and how they can be used to create
reusable and maintainable code. You will also have a strong
understanding of how to use inheritance, encapsulation, and
polymorphism to create powerful and flexible programs. Whether you
are a beginner or an experienced developer, this chapter will provide
you with the knowledge and skills needed to create high-quality
object-oriented code in Python.
Class basics
Creating and using classes
In Python, a class is a blueprint for creating objects with a set of

attributes and methods. It is a way of organizing and structuring
code, and it allows you to create custom data types that can be used
throughout your code. In this note, we will cover the basics of
creating and using classes in Python.
Creating a Class:
To create a class in Python, you use the class keyword followed by
the name of the class. Here's an example:
class Person:
pass
This creates a simple class called Person with no attributes or
methods. We use the pass keyword to indicate that there is no code
to execute in the class definition.
Attributes:
Attributes are variables that are associated with a class. They can
hold values that are specific to an instance of the class. To define
attributes in a class, you create variables inside the class definition.
Here's an example:
class Person:
self.name = name
self.age = age
In this example, we define a Person class with two attributes: name
and age. We use the special __init__ method to initialize these
attributes with values passed in when an instance of the class is
created.
Methods:
Methods are functions that are associated with a class. They can
perform actions on the attributes of an instance of the class. To
define methods in a class, you create functions inside the class
definition. Here's an example:
class Person:
self.name = name
self.age = age
def greet(self):
print(f"Hello, my name is {self.name}
and I am {self.age} years old.")
In this example, we define a greet method that prints a greeting using
the name and age attributes of an instance of the Person class.
Using a Class:
To use a class in Python, you first need to create an instance of the
class. This is done by calling the class like a function. Here's an
example:
person1 = Person("Alice", 25)

This creates an instance of the Person class with the name attribute
set to "Alice" and the age attribute set to 25.
Once you have an instance of the class, you can access its attributes
and methods using dot notation. Here's an example:
person1.greet()
This calls the greet method on the person1 instance of the Person
class and prints the greeting.
Full Example:
Here's a complete example that demonstrates creating and using a
Person class in Python:
class Person:
self.name = name
self.age = age
def greet(self):

person1.greet()
person2 = Person("Bob", 30)

person2.greet()
Output:
Hello, my name is Alice and I am 25 years

old.
Hello, my name is Bob and I am 30 years old.
In this note, we have covered the basics of creating and using
classes in Python. Classes are an important feature of object-
oriented programming, and they allow you to create custom data
types that can be used throughout your code. By defining attributes
and methods in a class, you can create powerful and flexible code
that can be easily
Defining instance methods
In Python, an instance method is a method that is defined within a

class and can be called on instances of that class. Instance methods
are used to perform actions or operations on the attributes of an
object. In this note, we will cover the basics of defining instance
methods in Python.
Defining Instance Methods:
To define an instance method in Python, you first need to create a
class. Here's an example:
class Person:
self.name = name
self.age = age
def greet(self):
In this example, we define a Person class with two attributes: name
and age. We then define an instance method called greet that prints a
greeting using the name and age attributes of an instance of the
Person class.
The self parameter:
When defining an instance method in Python, you need to include the
self parameter as the first parameter. This parameter refers to the
instance of the class on which the method is being called. It is used to
access the attributes and other methods of the object.
Calling Instance Methods:
To call an instance method in Python, you first need to create an
instance of the class. Here's an example:

person1.greet()
set to "Alice" and the age attribute set to 25. It then calls the greet
method on the person1 instance of the Person class and prints the
greeting.
Instance Methods with Parameters:
Instance methods can also take parameters, just like regular
functions in Python. Here's an example:
class Person:
self.name = name
self.age = age
def greet(self, greeting):

print(f"{greeting}, my name is
{self.name} and I am {self.age} years old.")

person1.greet("Hi")
In this example, we define an instance method called greet that takes
a greeting parameter. When the method is called, it prints the
greeting along with the name and age attributes of the object.
In this note, we have covered the basics of defining and using
instance methods in Python. Instance methods are an essential part
of object-oriented programming, and they allow you to perform
actions or operations on the attributes of an object. By defining
instance methods in a class, you can create powerful and flexible
code that can be easily reused throughout your program.
Using instance variables
In Python, instance variables are variables that are defined within a
class and are associated with instances of that class. Instance
variables hold unique values for each instance of the class, and they
are used to store and manipulate data that is specific to each object.
In this note, we will cover the basics of using instance variables in
Python.
Defining Instance Variables:
To define an instance variable in Python, you first need to create a
class. Here's an example:
class Person:
self.name = name
self.age = age
In this example, we define a Person class with two instance
variables: name and age. These variables are defined within the
__init__ method using the self parameter. The self parameter refers
to the instance of the class on which the method is being called.
Accessing Instance Variables:
To access an instance variable in Python, you can use dot notation.
Here's an example:

print(person1.name)
print(person1.age)
set to "Alice" and the age attribute set to 25. It then prints the value
of name and age using dot notation.
Modifying Instance Variables:
To modify an instance variable in Python, you can use dot notation to
access the variable and assign a new value. Here's an example:

person1.age = 26
print(person1.age)
set to "Alice" and the age attribute set to 25. It then modifies the
value of age to 26 using dot notation and prints the new value.
Instance Variables with Default Values:
Instance variables can also have default values, just like function
parameters in Python. Here's an example:
class Person:
def __init__(self, name, age=18):
self.name = name
self.age = age
In this example, we define a Person class with an age instance
variable that has a default value of 18. If no value is provided for age
when an instance of the class is created, it will default to 18.
In this note, we have covered the basics of using instance variables in
Python. Instance variables are an essential part of object-oriented
programming, and they allow you to store and manipulate data that is
specific to each object. By defining and using instance variables in a
class, you can create powerful and flexible code that can be easily
reused throughout your program.
Understanding class vs instance data
In Python, classes can have both class data and instance data. Class
data is shared among all instances of the class, while instance data is
unique to each instance. Understanding the difference between these
two types of data is essential for writing effective object-oriented
code in Python. In this note, we will cover the basics of class data vs
instance data in Python, with suitable codes.
Class Data:
Class data is data that is shared among all instances of a class. It is
defined within the class but outside of any methods. Here's an
example:
class Person:
count = 0
def __init__(self, name):

self.name = name
Person.count += 1
In this example, we define a Person class with a class variable count.
The count variable is shared among all instances of the Person class.
We also define an __init__ method that increments the count variable
each time a new instance of the Person class is created.
Instance Data:
Instance data is data that is unique to each instance of a class. It is
defined within the __init__ method using the self parameter. Here's an
example:
class Person:
self.name = name
self.age = age
In this example, we define a Person class with instance variables
name and age. These variables are unique to each instance of the
Person class.
Accessing Class Data and Instance Data:
To access class data, you can use dot notation with the class name.
To access instance data, you can use dot notation with the instance
name. Here's an example:

print(Person.count) # Output: 2
print(person1.name) # Output: "Alice"

print(person1.age) # Output: 25
print(person2.name) # Output: "Bob"

This creates two instances of the Person class and prints the value of
count, name, and age using dot notation.
Modifying Class Data and Instance Data:
To modify class data, you can use dot notation with the class name
and assign a new value. To modify instance data, you can use dot
notation with the instance name and assign a new value. Here's an
example:

Person.count = 3 # Modifying class data

person1.age = 26 # Modifying instance data
print(Person.count) # Output: 3
This modifies the value of count for the Person class and the value of
age for person1.
In this note, we have covered the basics of class data vs instance
data in Python. Class data is shared among all instances of a class,
while instance data is unique to each instance. By understanding the
difference between these two types of data, you can write more
effective and flexible object-oriented code in Python.
Using slots for memory optimization
In Python, every object is created with a dictionary that stores all of

its attributes. While this is convenient, it can also be memory-intensive
if you are creating a large number of objects. In situations where
memory is limited, you may want to optimize the memory usage of
your objects. One way to do this is by using slots. In this note, we will
cover the basics of using slots for memory optimization in Python,
with suitable codes.
What are Slots?
Slots are a way to tell Python that a class will have a fixed set of
attributes, so it doesn't need to create a dictionary for each instance.
Instead, it allocates a fixed amount of memory for the attributes. This
can significantly reduce the memory usage of your objects, especially
if you are creating a large number of instances.
Using Slots:
To use slots, you need to define a class attribute called __slots__ as
a sequence of strings that represent the names of the attributes.
Here's an example:
class Person:
__slots__ = ['name', 'age']

self.name = name
self.age = age
In this example, we define a Person class with slots for the name and
age attributes. This tells Python that each instance of the Person
class will only have these two attributes, and it can allocate memory
accordingly.
Benefits of Using Slots:
Using slots has several benefits:
Memory optimization: Slots can significantly reduce the memory
usage of your objects, especially if you are creating a large number
of instances.
Faster attribute access: Since slots allocate memory for each
attribute, attribute access is faster than with a dictionary-based
approach.
Prevents dynamic attribute creation: With slots, you cannot
dynamically add new attributes to an instance. This can help prevent
bugs caused by typos or other mistakes.
Limitations of Using Slots:
Using slots also has some limitations:
You must specify all attributes in advance: Since slots allocate
memory for each attribute, you must specify all attributes in advance.
This can make your code less flexible if you need to add new
attributes later.
Inheritance issues: If you subclass a class with slots, the subclass
must also have slots that include all of the attributes of the parent
class.
In this note, we have covered the basics of using slots for memory
optimization in Python. Slots are a way to tell Python that a class will
have a fixed set of attributes, so it can allocate a fixed amount of
memory for each instance. While slots can significantly reduce the
memory usage of your objects and improve attribute access speed,
they also have some limitations. By understanding the benefits and
limitations of slots, you can decide whether or not to use them in your
code.
Understanding class inheritance
In Python, classes can inherit attributes and methods from other

classes. This is called class inheritance and it allows you to create
new classes that are variations of existing classes. In this note, we
will cover the basics of understanding class inheritance in Python,
with suitable codes.
What is Class Inheritance?
Class inheritance is the process of creating a new class that inherits
properties (attributes and methods) from an existing class. The
existing class is called the parent class or the superclass, and the
new class is called the child class or the subclass. In Python, a
subclass can inherit attributes and methods from a single parent class
or from multiple parent classes.
Syntax of Class Inheritance:
To create a subclass, you need to define a new class and specify the
parent class(es) in parentheses after the class name. Here's an
example:
class Parent:
def __init__(self):
self.x = 1
def parent_method(self):
print("Parent method called.")
class Child(Parent):
pass
In this example, we define a Parent class with an __init__ method
and a parent_method. We then define a Child class that inherits from
the Parent class by specifying it in parentheses after the class name.
Since the Child class doesn't have any attributes or methods of its
own, we simply use the pass statement.
Overriding Parent Methods:
In addition to inheriting attributes and methods from the parent class,
a subclass can also override methods of the parent class. To do this,
you define a method with the same name in the subclass. Here's an
example:
class Parent:
def __init__(self):
self.x = 1
print("Parent method called.")
print("Child method called.")
In this example, we define a Parent class with a parent_method. We
then define a Child class that overrides the parent_method by defining
a new method with the same name. When we call parent_method on
an instance of the Child class, the child method will be called instead
of the parent method.
Multiple Inheritance:
In Python, a subclass can inherit from multiple parent classes. To do
this, you specify all of the parent classes in parentheses after the
class name, separated by commas. Here's an example:
class Parent1:
def __init__(self):
self.x = 1
def parent1_method(self):
print("Parent1 method called.")
class Parent2:
def __init__(self):
self.y = 2
def parent2_method(self):
class Child(Parent1, Parent2):

pass
In this example, we define two parent classes, Parent1 and Parent2,
with their own attributes and methods. We then define a Child class
that inherits from both Parent1 and Parent2. Since the Child class
doesn't have any attributes or methods of its own, we simply use the
pass statement.
In this note, we have covered the basics of understanding class
inheritance in Python. Class inheritance allows you to create new
classes that inherit attributes and methods from existing classes, and
it can help you create more modular and reusable code. By
understanding how to create subclasses, override parent methods,
and inherit from multiple parent classes, you can use class inheritance
to create more complex and powerful programs.
Using multiple inheritance
In Python, multiple inheritance is the process of creating a new class

that inherits properties (attributes and methods) from multiple parent
classes. In this note, we will cover the basics of using multiple
inheritance in Python, with suitable codes.
What is Multiple Inheritance?
Multiple inheritance is a type of class inheritance where a subclass
can inherit attributes and methods from multiple parent classes. In
Python, you can specify multiple parent classes in the parentheses
after the class name.
Syntax of Multiple Inheritance:
To create a subclass with multiple inheritance, you need to define a
new class and specify the parent classes in parentheses after the
class name, separated by commas. Here's an example:
class Parent1:
def method1(self):
class Parent2:
def method2(self):

pass
each with their own methods. We then define a Child class that
inherits from both Parent1 and Parent2 by specifying them in
parentheses after the class name, separated by commas. Since the
Child class doesn't have any methods of its own, we simply use the
pass statement.
Method Resolution Order (MRO):
When a subclass inherits from multiple parent classes, Python
determines the order in which it searches for methods in the parent
classes. This is called the Method Resolution Order (MRO). The
MRO is important because it determines which method will be called
if two or more parent classes have methods with the same name.
In Python 3, the MRO is determined using the C3 linearization
algorithm, which guarantees that the method resolution order is
consistent and respects local precedence ordering and monotonicity.
You can access the MRO of a class using the mro() method.
class Parent1:
def method(self):
class Parent2:
def method(self):

pass
print(Child.mro()) # prints [<class
'__main__.Child'>, <class
'__main__.Parent1'>, <class
'__main__.Parent2'>, <class 'object'>]
each with their own method. We then define a Child class that inherits
from both Parent1 and Parent2. When we call Child.mro(), we get the
method resolution order, which shows that Python will first look for
methods in Child, then Parent1, then Parent2, and finally object.
Diamond Inheritance:
In multiple inheritance, a situation can arise where a subclass inherits
from two parent classes that both inherit from the same grandparent
class. This is called diamond inheritance and it can lead to ambiguity
in method resolution. To resolve this ambiguity, Python uses the C3
linearization algorithm to determine the order in which methods are
searched for.
class Grandparent:
def method(self):
print("Grandparent method called.")
class Parent1(Grandparent):
pass
class Parent2(Grandparent):
pass

pass
c = Child()
c.method() # prints "Grandparent method
called."
In this example, we define a Grandparent class with a method. We
then define two parent classes, Parent1 and Parent2, both of which
inherit from Grandparent.
Class design
Writing clean, readable classes
When writing classes in Python, it's important to focus not only on

functionality but also on readability and maintainability of the code. In
this note, we will discuss some best practices for writing clean and
readable classes in Python, with suitable codes.
Best Practices for Writing Clean and Readable Classes in Python:
Use descriptive names for classes and methods: It's important to use
descriptive names for classes and methods that accurately reflect
their purpose. This makes it easier for other developers to
understand what the code does without having to read through the
entire implementation.
class Student:
self.name = name
self.age = age
def get_name(self):
return self.name
def get_age(self):
return self.age
In this example, we define a Student class with name and age
attributes and get_name() and get_age() methods. The names of the
class and methods clearly indicate their purpose.
Follow the Single Responsibility Principle (SRP): A class should have
only one responsibility and should be focused on that responsibility.
This makes the code easier to understand and maintain.
class Calculator:
def add(self, x, y):
return x + y
def subtract(self, x, y):

return x - y
In this example, we define a Calculator class with add() and
subtract() methods. The class has only one responsibility, which is to
perform arithmetic operations.
Use comments to explain complex logic: Sometimes, complex logic is
necessary in a class. In such cases, it's a good practice to use
comments to explain what the code does and why it does it.
class ShoppingCart:
def __init__(self):
self.items = []
def add_item(self, item):

"""
Add an item to the shopping cart.
If the item already exists in the cart,

increase the quantity
by 1. Otherwise, add a new item to the
cart.
"""
for i in self.items:
if i['name'] == item['name']:
i['quantity'] += 1
return
self.items.append(item)
In this example, we define a ShoppingCart class with an add_item()
method. The method contains complex logic for adding an item to the
cart. We use a comment to explain the logic and make it easier to
understand.
Avoid global variables: Global variables can make code harder to
read and maintain. It's a good practice to avoid them in classes.
class Car:
def __init__(self, make, model, year):
self.make = make
self.model = model
self.year = year
def get_make(self):
return self.make
def get_model(self):
return self.model
def get_year(self):
return self.year
In this example, we define a Car class with make, model, and year
attributes and get_make(), get_model(), and get_year() methods. We
don't use any global variables in the class.
Follow the Python style guide (PEP 8): The Python community has
established a style guide called PEP 8 that provides guidelines for
writing Python code. Following the style guide can make code more
consistent and easier to read for other developers.
class Rectangle:
def __init__(self, length, width):
self.length = length
self.width = width
def get_area(self):
return self.length * self.width
def get_perimeter(self
Writing classes with a single responsibility
The Single Responsibility Principle (SRP) is an important design
principle in object-oriented programming. According to the SRP, a
class should have only one responsibility and should be focused on
that responsibility. This makes the code easier to understand and
maintain. In this note, we will discuss how to write classes with a
single responsibility in Python, with suitable codes.
Best Practices for Writing Classes with a Single Responsibility in

Python:
Identify the class's responsibility: The first step in writing a class with
a single responsibility is to identify what that responsibility is. A class
should have one clear responsibility and should be focused on that
responsibility.
class Circle:
def __init__(self, radius):
self.radius = radius
def get_area(self):
return 3.14 * self.radius ** 2
def get_circumference(self):
return 2 * 3.14 * self.radius
In this example, we define a Circle class with radius attribute and
get_area() and get_circumference() methods. The class's
responsibility is to calculate the area and circumference of a circle.
Separate concerns into different classes: If a class has multiple
responsibilities, it's a good practice to separate those responsibilities
into different classes. This makes the code easier to understand and
maintain.
class Employee:
def __init__(self, name, salary):
self.name = name
self.salary = salary
class Payroll:
def calculate_payroll(self, employees):
for employee in employees:
print(f'{employee.name}:
{employee.salary}')
In this example, we define an Employee class with name and salary
attributes and a Payroll class with a calculate_payroll() method. The
Employee class is responsible for storing employee information, while
the Payroll class is responsible for calculating employee pay.
Avoid adding unrelated functionality: When writing a class, it's
important to avoid adding unrelated functionality. This can make the
class harder to understand and maintain.
class Email:
def __init__(self, subject, body):
self.subject = subject
self.body = body
def send_email(self, recipient):

# code to send email
pass
def encrypt_email(self):
# code to encrypt email
pass
In this example, we define an Email class with subject and body
attributes and send_email() and encrypt_email() methods. The
send_email() method is related to the class's responsibility of sending
emails, but the encrypt_email() method is not. It's a good practice to
remove the unrelated functionality from the class.
Keep methods short and focused: Methods should be short and
focused on a specific task. This makes the code easier to read and
understand.
class ShoppingCart:
def __init__(self):
self.items = []
def add_item(self, item):

i['quantity'] += 1
return
self.items.append(item)
def remove_item(self, item):
i['quantity'] -= 1
if i['quantity'] == 0:
self.items.remove(i)
return
In this example, we define a ShoppingCart class with add_item() and
remove_item() methods. Both methods are short and focused on a
specific task, making the code easier to read and understand.
Using composition over inheritance
Inheritance and composition are two common approaches to

designing object-oriented systems. Inheritance involves creating a
subclass that inherits the behavior of its superclass. Composition
involves creating objects that contain other objects. In this note, we
will discuss using composition over inheritance in Python, with suitable
codes.
Benefits of Using Composition:
Code reuse: Composition allows for code reuse without creating a
tightly coupled hierarchy of classes.
Flexibility: Composition provides greater flexibility in designing
objects. Objects can be composed of different objects to achieve a
specific behavior.
Simplified class hierarchies: Composition can simplify class
hierarchies by avoiding deep inheritance chains.
Using Composition in Python:
Here's an example of using composition in Python to create a Car
class that has an Engine object and a Transmission object.
class Engine:
def __init__(self, horsepower):
self.horsepower = horsepower
def start(self):
print("Engine started")
def stop(self):
print("Engine stopped")
class Transmission:
def __init__(self, num_gears):
self.num_gears = num_gears
def shift_up(self):
print("Shifted up")
def shift_down(self):
print("Shifted down")
class Car:
def __init__(self, engine, transmission):
self.engine = engine
self.transmission = transmission
def start(self):
self.engine.start()
def stop(self):
self.engine.stop()
def shift_up(self):
self.transmission.shift_up()
def shift_down(self):
self.transmission.shift_down()
In this example, the Engine and Transmission classes are composed
into the Car class. The Car class has a start() and stop() method that
call the corresponding methods on the Engine object, and a shift_up()
and shift_down() method that call the corresponding methods on the
Transmission object.
Advantages of Composition over Inheritance:
Reduced coupling: Composition reduces coupling between classes,
making it easier to modify the behavior of a class without affecting
other classes.
Increased flexibility: With composition, the behavior of an object can
be changed at runtime by swapping out its constituent objects.
Simplified testing: Composition simplifies testing, as individual
components can be tested in isolation.
Composition is a powerful technique for building flexible and
maintainable object-oriented systems. By using composition instead
of inheritance, we can create classes that are more modular, more
flexible, and easier to maintain.
Using abstract base classes
In Python, an abstract base class (ABC) is a class that cannot be

instantiated and is meant to serve as a blueprint for other classes.
ABCs define abstract methods, which are methods that must be
implemented by any concrete subclasses. In this note, we will discuss
using abstract base classes in Python, with suitable codes.
Creating an Abstract Base Class:
In Python, we can create an abstract base class by importing the abc
module and using the ABC class as a base class. We can then define
abstract methods using the @abstractmethod decorator.
Here's an example of an abstract base class for a Shape:
import abc
class Shape(metaclass=abc.ABCMeta):
@abc.abstractmethod
def area(self):
pass
@abc.abstractmethod
def perimeter(self):
pass
In this example, we define an abstract base class Shape that has two
abstract methods area() and perimeter(). Any concrete subclass of
Shape must implement these two methods.
Creating a Concrete Subclass:
To create a concrete subclass of an abstract base class, we simply
inherit from the abstract base class and implement its abstract
methods. Here's an example of a Rectangle class that inherits from
Shape:
class Rectangle(Shape):
def __init__(self, length, width):
self.length = length
self.width = width
def area(self):
return self.length * self.width
def perimeter(self):
return 2 * (self.length + self.width)
In this example, we create a Rectangle class that inherits from Shape
and implements the area() and perimeter() methods.
Using the Abstract Base Class:
Once we have defined the abstract base class and concrete
subclasses, we can use them in our code. Here's an example of how
to use the Shape and Rectangle classes:
def print_shape_info(shape):
print(f"Area: {shape.area()}")
print(f"Perimeter: {shape.perimeter()}")
rectangle = Rectangle(5, 10)

print_shape_info(rectangle)
In this example, we define a function print_shape_info() that takes a
Shape object and prints its area and perimeter. We then create a
Rectangle object and pass it to print_shape_info().
Advantages of Using Abstract Base Classes:
Enforces implementation of methods: Abstract base classes enforce
the implementation of specific methods in concrete subclasses,
making it easier to write correct and maintainable code.
Defines a common interface: Abstract base classes define a common
interface for related classes, making it easier to write code that
works with multiple objects.
Encourages polymorphism: Abstract base classes encourage the use
of polymorphism, making it easier to write code that can handle
objects of different types.
Abstract base classes are a powerful tool for designing maintainable
and extensible object-oriented systems in Python. By defining a
common interface for related classes, enforcing the implementation of
specific methods, and encouraging polymorphism, abstract base
classes make it easier to write correct and maintainable code.
Writing metaclasses
In Python, a metaclass is a class that defines the behavior of other

classes. When we create a class, Python uses a metaclass to define
its behavior. In this note, we will discuss how to write metaclasses in
Python, with suitable codes.
Creating a Metaclass:
In Python, we can create a metaclass by defining a class that inherits
from type. The type class is the built-in metaclass in Python, and it is
responsible for creating all classes.
Here's an example of a simple metaclass:
class MyMeta(type):
print(f"Creating class {name} with bases
{bases} and attrs {attrs}")
attrs)
In this example, we define a metaclass MyMeta that inherits from
type. The __new__() method is called when a new class is created,
and it takes four arguments:
cls: The metaclass itself
name: The name of the new class
bases: A tuple of the base classes for the new class
attrs: A dictionary of the attributes and methods of the new class
When we create a new class using MyMeta as the metaclass, the
__new__() method will be called and will print out the class name,
base classes, and attributes.
Using the Metaclass:
Once we have defined the metaclass, we can use it to create new
classes. Here's an example of how to use MyMeta to create a new
class:
x = 42
In this example, we create a new class MyClass and specify MyMeta
as the metaclass. When we create the class, the __new__() method
of MyMeta will be called, and it will print out information about the
new class.
Advantages of Using Metaclasses:
Customizing class creation: Metaclasses allow us to customize the
way classes are created, giving us fine-grained control over class
behavior.
Enforcing constraints: Metaclasses allow us to enforce constraints on
classes, such as requiring certain attributes or methods to be
present.
Automatic registration: Metaclasses can be used to automatically
register classes in a registry or database, simplifying the
management of large codebases.
Metaclasses are a powerful tool for customizing class creation in
Python. By defining a metaclass
and using it to create new classes, we can customize class behavior,
enforce constraints, and simplify the management of large
codebases. However, metaclasses should be used with care, as they
can make code harder to understand and maintain if used improperly.
Advanced class topics

Using descriptors to customize attribute access
In Python, descriptors are a way to customize the behavior of
attribute access. They allow us to define how attributes are
accessed, set, or deleted on an object. In this note, we will discuss
how to use descriptors to customize attribute access in Python, with
suitable codes.
Creating a Descriptor:
To create a descriptor, we need to define a class with one or more of
the following methods:
__get__(self, instance, owner): This method is called when
the descriptor's value is accessed using dot notation.
instance is the instance of the class that contains the
descriptor, and owner is the class itself.
__set__(self, instance, value): This method is called when

the descriptor's value is set using dot notation.
__delete__(self, instance): This method is called when the

descriptor's value is deleted using del statement.
Here's an example of a simple descriptor:
class MyDescriptor:
print("Getting the value")

print("Setting the value")
def __delete__(self, instance):
print("Deleting the value")
del instance._value
In this example, we define a descriptor MyDescriptor that has
__get__(), __set__(), and __delete__() methods. The __get__()
method prints a message and returns the value of _value attribute of
the instance. The __set__() method prints a message and sets the
value of _value attribute of the instance. The __delete__() method
prints a message and deletes the _value attribute of the instance.
Using the Descriptor:
Once we have defined the descriptor, we can use it to customize
attribute access on a class. Here's an example of how to use
MyDescriptor to customize attribute access on a class:
class MyClass:
self._value = value
x = MyDescriptor()
In this example, we define a class MyClass that has an attribute x
that is an instance of MyDescriptor. When we access, set, or delete
the x attribute using dot notation, the corresponding method of
MyDescriptor will be called.
Advantages of Using Descriptors:
Reusable code: Descriptors can be reused across multiple classes,
making it easier to write DRY (Don't Repeat Yourself) code.
Customizable behavior: Descriptors allow us to customize the
behavior of attribute access, making it possible to enforce constraints
or perform custom operations when an attribute is accessed, set, or
deleted.
Easy to use: Descriptors are easy to use, requiring only a few lines
of code to define and use.
Descriptors are a powerful tool for customizing attribute access in
Python. By defining a descriptor and using it to customize attribute
access on a class, we can enforce constraints, perform custom
operations, and write reusable code. However, descriptors should be
used with care, as they can make code harder to understand and
maintain if used improperly.
Using properties to control attribute access
In Python, descriptors are a way to define customized behavior for

accessing and setting attributes of an object. A descriptor is an
object that defines one or more of the following methods: __get__(),
__set__(), and __delete__(). These methods allow you to control
how an attribute is accessed, modified, or deleted on an instance of a
class.
Using descriptors can be useful in many scenarios. For example, you
can use descriptors to:
Validate data before it is stored in an attribute
Convert data to a different format before it is stored in an
attribute
Create read-only or write-only attributes
Implement computed attributes that are calculated on the fly
Implement lazy attributes that are only calculated when
needed
To define a descriptor, you need to create a class that defines one or

more of the descriptor methods. For example, to create a descriptor
that validates data before it is stored in an attribute, you can define a
class like this:
class PositiveNumber:
def __set_name__(self, owner, name):
self.name = name

if value < 0:
raise ValueError(f"{self.name} must be
positive")
instance.__dict__[self.name] = value

return instance.__dict__[self.name]
This descriptor ensures that a number attribute is always positive.
The __set_name__() method is called when the descriptor is
assigned to a class attribute, and it sets the name of the attribute.
The __set__() method is called when the attribute is set, and it
checks if the value is positive before setting it. The __get__() method
is called when the attribute is accessed, and it returns the value of
the attribute.
To use the descriptor, you need to define a class that uses it as an

attribute:
class MyClass:
x = PositiveNumber()
Now, when you create an instance of MyClass and set the x attribute
to a negative value, an error will be raised:
>>> obj = MyClass()

>>> obj.x = -10
ValueError: x must be positive
You can also define a descriptor that converts data to a different
format before it is stored in an attribute. For example, you can define
a descriptor that stores a string as uppercase:
class UppercaseString:
self.name = name

instance.__dict__[self.name] =
str(value).upper()

return instance.__dict__[self.name]
To use this descriptor, you can define a class like this:
class MyOtherClass:
name = UppercaseString()
Now, when you create an instance of MyOtherClass and set the
name attribute to a lowercase string, it will be stored as uppercase:
>>> obj2 = MyOtherClass()
>>> obj2.name = "john"
>>> obj2.name
'JOHN'
Descriptors can also be used to create read-only or write-only
attributes. To create a read-only attribute, you can define a descriptor
that only implements the __get__() method:
class ReadOnly:
self.name = name

raise AttributeError(f"{self.name} is read-
only")

return instance.__dict
In Python, class decorators are a way to modify or extend the

behavior of a class without changing its code. A class decorator is a
function that takes a class as its argument and returns a new class
that has the same name as the original class. The new class can
inherit from the original class or not, and can add new methods,
attributes, or modify existing ones.
To create a class decorator, you define a function that takes a class
as an argument and returns a new class. The new class can inherit
from the original class or not, and can add new methods, attributes,
or modify existing ones.
For example, let's create a class decorator that adds a count
attribute to a class that keeps track of the number of instances
created from the class:
def count_instances(cls):
class CountedClass(cls):
count = 0
def __init__(self, *args, **kwargs):

super().__init__(*args, **kwargs)
CountedClass.count += 1
return CountedClass
The count_instances decorator takes a class as an argument and
returns a new class that inherits from the original class. The new
class has a count attribute that is initialized to 0, and a modified
__init__() method that increments the count attribute every time an
instance of the class is created.
To use the decorator, you just need to apply it to a class:
@count_instances
class MyClass:
self.value = value
Now, every time you create an instance of MyClass, the count
attribute will be incremented:
>>> obj1 = MyClass(10)

>>> MyClass.count
3
You can also create a class decorator that modifies the behavior of a
method. For example, let's create a class decorator that adds logging
to all methods of a class:
def log_methods(cls):
for name, method in cls.__dict__.items():
if callable(method):
def logged_method(self, *args,
**kwargs):
print(f"Calling method {name}")
return method(self, *args, **kwargs)
setattr(cls, name, logged_method)
return cls
The log_methods decorator iterates over all the attributes of the
class, and if an attribute is a method, it replaces it with a new method
that logs the name of the method before calling it.
To use the decorator, you just need to apply it to a class:
@log_methods
class MyOtherClass:
def method1(self):
print("Method 1")
def method2(self):
print("Method 2")
Using the super function
In Python, the super() function is used to call a method from a parent

class, which allows us to extend or override the behavior of the
parent class in the child class.
The super() function is commonly used in object-oriented
programming to call the constructor or methods of the parent class,
while still allowing the child class to customize its behavior.
To use the super() function, we call it with two arguments: the first
argument is the child class, and the second argument is an instance
of the child class.
For example, let's consider the following class hierarchy:
class Parent:
self.name = name
def greet(self):
print(f"Hello, {self.name}!")
def greet(self):
super().greet()
print("I'm a child!")
In this example, we have a Parent class with an __init__() method
and a greet() method, and a Child class that inherits from Parent and
overrides the greet() method.
In the Child class, we call the greet() method of the Parent class
using super().greet(). This will call the greet() method of the parent
class, and then print "I'm a child!".
Let's create an instance of the Child class and call the greet()
method:
>>> child = Child("John")

>>> child.greet()
Hello, John!
I'm a child!
As you can see, the super() function allowed us to call the greet()
method of the Parent class, while still allowing the Child class to
customize its behavior.
Another example of using the super() function is to call the
constructor of the parent class from the child class:
class Parent:
self.name = name
super().__init__(name)
self.age = age
In this example, we have a Parent class with an __init__() method

that takes a name argument, and a Child class that inherits from
Parent and overrides the __init__() method to take an additional age
argument.
In the Child class, we call the __init__() method of the Parent class
using super().__init__(name). This will call the __init__() method of
the parent class with the name argument, and then we can initialize
the age attribute in the child class.
Let's create an instance of the Child class and print its attributes:
>>> child = Child("John", 10)

>>> print(child.name, child.age)
John 10
As you can see, the super() function allowed us to call the constructor
of the parent class from the child class, while still allowing the child
class to customize its behavior.
Using slots to optimize memory usage
In Python, every object has a dictionary that stores its attributes.

While this is convenient, it can also lead to high memory usage,
especially when creating large numbers of objects.
To optimize memory usage in Python, we can use slots. Slots are a
mechanism that allows us to explicitly declare the attributes that an
object can have. This allows Python to allocate memory for these
attributes directly in the object, rather than in a dictionary.
To use slots, we define a list of attribute names as class variables in
our class, like this:
class Person:
__slots__ = ['name', 'age']

self.name = name
self.age = age
In this example, we have defined a Person class with two attributes:
name and age. We have also defined the __slots__ attribute as a list
of strings containing the attribute names.
When we create an instance of this class, Python will allocate
memory for the name and age attributes directly in the object, rather
than in a dictionary. This can lead to significant memory savings when
creating large numbers of objects.
Let's create a few instances of this class and check their memory
usage:
import sys
p1 = Person("Alice", 25)
p2 = Person("Bob", 30)
p3 = Person("Charlie", 35)
print(sys.getsizeof(p1)) # prints 56
As you can see, the memory usage of each object is only 56 bytes,
which is much smaller than the size of a typical Python object.
Note that slots have some limitations. Once we define a class with
slots, we can only assign attributes to the slots that we have defined.
If we try to assign a new attribute, we will get an AttributeError.
Additionally, we cannot use properties or other dynamic attributes in
classes with slots.
Overall, slots can be a useful tool for optimizing memory usage in
Python, especially when creating large numbers of objects with a
fixed set of attributes. However, it's important to carefully consider
the limitations of slots before using them in a project.
Chapter 5:
Concurrency and Parallelism
Concurrency and parallelism are essential concepts in modern

programming, as they allow developers to take advantage of modern
hardware to write programs that are more efficient and responsive.
Python is a popular language that supports both concurrency and
parallelism, making it a powerful tool for creating high-performance
applications.
Concurrency refers to the ability of a program to handle multiple
tasks at the same time. In other words, it is the ability of a program
to perform more than one task simultaneously. This is achieved
through the use of threads, which are lightweight processes that can
run independently of each other within the same program.
Parallelism, on the other hand, refers to the ability of a program to
use multiple processors or cores to perform tasks simultaneously.
This is achieved through the use of processes, which are separate
instances of a program that can run independently of each other.
In this chapter, we will explore the basics of concurrency and
parallelism in Python. We will start by defining the concepts of
threads and processes and explain how they are related to
concurrency and parallelism. We will then look at how to create and
manage threads and processes in Python, and how to communicate
between them.
We will also examine some of the challenges of concurrency and
parallelism, such as race conditions and deadlocks, and discuss
techniques for avoiding them. We will explore how to use locks,
semaphores, and other synchronization primitives to coordinate
access to shared resources and avoid conflicts between threads and
processes.
Additionally, we will look at some advanced topics related to
concurrency and parallelism in Python. We will discuss the concept of
asynchronous programming, which allows programs to perform non-
blocking I/O operations and handle large numbers of concurrent
connections efficiently.
Threads and Processes

Understanding the Global Interpreter Lock (GIL)
In Python, the Global Interpreter Lock (GIL) is a mechanism that

ensures only one thread executes Python bytecode at a time. This
means that even in a multithreaded application, only one thread can
execute Python code at any given moment.
The purpose of the GIL is to ensure thread safety in Python. Because
of the way Python's memory management works, allowing multiple
threads to execute Python bytecode simultaneously could lead to
data corruption and other issues.
While the GIL is an important feature for ensuring the safety of
multithreaded Python applications, it can also be a bottleneck in
applications that require high levels of parallelism. Because only one
thread can execute Python bytecode at a time, applications that
spend a lot of time executing Python code may not see significant
performance improvements from using multiple threads.
Let's take a look at an example that demonstrates the GIL in action:
import threading
x=0
def increment():
global x
for i in range(1000000):
x += 1
threads = []
for i in range(10):
t = threading.Thread(target=increment)
threads.append(t)
for t in threads:
t.start()
for t in threads:
t.join()
print(x)
In this example, we define a function increment that simply increments
a global variable x by 1, 1000000 times. We then create 10 threads
and start them, each of which calls the increment function.
If we run this code, we might expect the final value of x to be
10000000 (10 threads * 1000000 increments per thread). However,
due to the GIL, the actual value of x will be less than this. On my
machine, the output is typically around 8-9 million.
While the GIL can be a bottleneck in applications that require high
levels of parallelism, it's important to note that it only affects the
execution of Python bytecode. If your application spends a lot of time
waiting for I/O (such as network requests or disk reads), you may
still see significant performance improvements from using multiple
threads.
In summary, the Global Interpreter Lock is an important feature of
Python that ensures thread safety. While it can be a bottleneck in
certain types of applications, it's important to carefully consider the
tradeoffs between performance and safety when designing
multithreaded Python applications.
Using threads for I/O-bound tasks
In Python, threads can be used to improve the performance of I/O-

bound tasks. An I/O-bound task is one that spends a significant
amount of time waiting for input/output operations to complete, such
as reading from a file or making a network request. By using threads,
we can allow the main thread to continue executing other tasks while
the I/O-bound task is waiting for I/O operations to complete.
Let's take a look at an example that demonstrates how threads can
be used to improve the performance of an I/O-bound task:
import threading
import requests
def download_url(url):
response = requests.get(url)
print(f"Downloaded
{len(response.content)} bytes from {url}")
urls = [
"https://www.example.com",
"https://www.python.org",
"https://www.google.com",
"https://www.github.com",
"https://www.stackoverflow.com",
]
threads = []
for url in urls:
t = threading.Thread(target=download_url,
args=(url,))
threads.append(t)
t.start()
for t in threads:
t.join()
print("All downloads complete!")
In this example, we define a function download_url that uses the
requests library to download the contents of a URL. We then create a
list of URLs to download and create a thread for each URL, passing
the URL as an argument to the download_url function.
We then start each thread and wait for them to complete using the
join method. Finally, we
print a message indicating that all downloads are complete.
When we run this code, we should see that the downloads are
performed concurrently in multiple threads. Because each download
operation spends most of its time waiting for I/O operations to
complete, using threads allows us to download multiple URLs in
parallel without significantly impacting the performance of the main
thread.
It's important to note that while threads can be used to improve the
performance of I/O-bound tasks, they may not be suitable for tasks
that are CPU-bound (i.e., tasks that spend most of their time
performing calculations rather than waiting for I/O operations to
complete). In such cases, using multiprocessing or other techniques
may be more appropriate. Additionally, care should be taken when
using threads to ensure that shared resources (such as file handles
or database connections) are accessed safely from multiple threads.
Using processes for CPU-bound tasks
In Python, processes can be used to improve the performance of

CPU-bound tasks. A CPU-bound task is one that spends most of its
time performing calculations or other CPU-intensive operations, rather
than waiting for I/O operations to complete. By using multiple
processes, we can perform these calculations in parallel, taking
advantage of multiple CPU cores.
Let's take a look at an example that demonstrates how processes
can be used to improve the performance of a CPU-bound task:
from multiprocessing import Pool

def square(x):
return x * x
if __name__ == '__main__':
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
with Pool() as pool:
results = pool.map(square, numbers)
print(results)
In this example, we define a function square that calculates the
square of a given number. We then create a list of numbers to square
and use the Pool class from the multiprocessing module to create a
pool of worker processes. We then use the map method of the pool
to apply the square function to each number in the list.
The map method applies the function to each element of the input list
in parallel, using multiple worker processes. The results are returned
as a list in the order they were submitted.
When we run this code, we should see that the squares of the
numbers are calculated in parallel using multiple processes. Because
each calculation is CPU-bound and can be performed independently
of the others, using multiple processes allows us to perform these
calculations in parallel, taking advantage of multiple CPU cores.
It's important to note that while processes can be used to improve
the performance of CPU-bound tasks, they come with some
overhead compared to using threads. Creating new processes is
more expensive than creating new threads, and interprocess
communication (IPC) can be more complex than interthread
communication. Additionally, care should be taken when using
processes to ensure that shared resources (such as memory or
database connections) are accessed safely from multiple processes.
Using multiprocessing
Python's multiprocessing module allows us to spawn multiple

processes in order to execute code concurrently. This can be useful
for CPU-bound tasks that need to take advantage of multiple CPU
cores. In this subtopic, we will explore how to use the multiprocessing
module in Python to spawn processes and execute code concurrently.
Here is an example that demonstrates how to use the multiprocessing
module:
import multiprocessing
def worker(num):
"""Worker function"""
print(f'Worker {num} executing')
return
if __name__ == '__main__':
jobs = []
for i in range(5):
p=
multiprocessing.Process(target=worker,
args=(i,))
jobs.append(p)
p.start()
In this example, we define a function called worker that prints a
message indicating that it is executing. We then create a list of
processes and use a for loop to create five instances of the Process
class, passing the worker function and an argument to identify the
worker as arguments to the constructor.
We then append each process to the jobs list and start it by calling
the start method. When we run this code, we should see five
messages printed to the console indicating that each worker is
executing concurrently in its own process.
We can also use the Pool class from the multiprocessing module to
create a pool of worker processes. The Pool class provides a
convenient way to create a fixed number of worker processes and
distribute tasks to them. Here is an example that demonstrates how
to use the Pool class:
import multiprocessing
def worker(num):
return
if __name__ == '__main__':
with multiprocessing.Pool(processes=5)
as pool:
pool.map(worker, range(5))
In this example, we define the same worker function as before. We
then create a Pool object with five processes by passing
processes=5 as an argument to the constructor.
We then use the map method of the Pool object to apply the worker
function to each element in the range from 0 to 4. The map method
distributes the work across the processes in the pool and returns the
results as a list.
When we run this code, we should see five messages printed to the
console indicating that each worker is executing concurrently in its
own process.
The multiprocessing module provides a powerful way to spawn
multiple processes and ecute code concurrently. By using the
Process class or the Pool class, we can take advantage of multiple
CPU cores to perform CPU-bound tasks in parallel. However, care
should be taken when using multiprocessing to ensure that shared
resources (such as memory or database connections) are accessed
safely from multiple processes.
Using concurrent.futures
The concurrent.futures module in Python provides a high-level

interface for asynchronously executing functions using threads or
processes. This can be useful for performing I/O-bound tasks or
CPU-bound tasks in parallel. In this subtopic, we will explore how to
use the concurrent.futures module in Python to execute functions
concurrently.
Here is an example that demonstrates how to use the
concurrent.futures module with threads:
import concurrent.futures
import time
def worker(num):
time.sleep(1)
return num
if __name__ == '__main__':
with
concurrent.futures.ThreadPoolExecutor() as
executor:
results = [executor.submit(worker, i) for i
in range(5)]
for f in
concurrent.futures.as_completed(results):
print(f.result())
In this example, we define a function called worker that prints a
message indicating that it is executing and then sleeps for 1 second.
We then create a ThreadPoolExecutor object and use a list
comprehension to create five futures by submitting the worker
function and an argument to identify the worker to the executor.
We then use a for loop and the as_completed function to iterate over
the futures as they complete. The as_completed function returns an
iterator that yields futures as they complete. We print the result of
each future to the console when it completes.
When we run this code, we should see five messages printed to the
console indicating that each worker is executing concurrently in its
own thread. We should also see the results of each worker printed to
the console as they complete.
We can also use the concurrent.futures module with processes by
using the ProcessPoolExecutor class instead of the
ThreadPoolExecutor class. Here is an example that demonstrates
how to use the concurrent.futures module with processes:
import time
def worker(num):
time.sleep(1)
return num
if __name__ == '__main__':
with
concurrent.futures.ProcessPoolExecutor() as
executor:
results = [executor.submit(worker, i) for i
in range(5)]
for f in
concurrent.futures.as_completed(results):
print(f.result())
In this example, we define the same worker function as before. We
then create a ProcessPoolExecutor object instead of a
ThreadPoolExecutor object.
The rest of the code is the same as before. When we run this code,
we should see five messages printed to the console indicating that
each worker is executing concurrently in its own process. We should
also see the results of each worker printed to the console as they
complete.
The concurrent.futures module provides a powerful way to execute
functions asynchronously using threads or processes. By using the
ThreadPoolExecutor class or the ProcessPoolExecutor class, we can
take advantage of multiple CPU cores to perform CPU-bound tasks in
parallel or execute I/O-bound tasks concurrently.
Coroutines and asyncio

Understanding coroutines
Coroutines are a powerful feature in Python that allow for

asynchronous programming. They are functions that can pause their
execution, save their state, and resume execution from where they
left off later. This makes it possible to write code that can handle
multiple tasks in parallel, without the need for multiple threads or
processes.
To use coroutines in Python, we use the asyncio module, which
provides the infrastructure for running coroutines and managing their
execution. Here are the key concepts to understand when working
with coroutines in Python:
async and await keywords: The async keyword is used to
define a coroutine function, while the await keyword is used
to pause execution of a coroutine until some asynchronous
operation is completed.
Event loop: The event loop is the heart of the asyncio

module. It is responsible for scheduling and running
coroutines, and managing the execution of asynchronous
operations.
Coroutines: Coroutines are functions that use the async

keyword to define themselves as asynchronous functions
that can be paused and resumed.
Here is a simple example of a coroutine that uses the async and

await keywords:
import asyncio
async def my_coroutine():
print('Coroutine started')
await asyncio.sleep(1)
print('Coroutine resumed')
return 'Coroutine finished'
asyncio.run(my_coroutine())
In this example, we define a coroutine function called my_coroutine()
using the async keyword. The function prints a message, pauses for
1 second using the await keyword and the asyncio.sleep() function,
prints another message, and returns a value. Finally, we run the
coroutine using the asyncio.run() function.
Here is another example that shows how to use coroutines to run
multiple tasks in parallel:
import asyncio
async def my_coroutine(id):
print(f'Coroutine {id} started')
print(f'Coroutine {id} resumed')
return f'Coroutine {id} finished'
async def main():
tasks =
[asyncio.create_task(my_coroutine(i)) for i in
range(3)]
results = await asyncio.gather(*tasks)
print(results)
asyncio.run(main())
In this example, we define a main() coroutine function that creates
three instances of the my_coroutine() function using the
asyncio.create_task() function. We then use the asyncio.gather()
function to wait for all the tasks to complete and collect their results.
Finally, we print the results.
To summarize, coroutines are a powerful feature in Python that allow
for asynchronous programming. They are functions that can pause
their execution, save their state, and resume execution from where
they left off later. To use coroutines in Python, we use the asyncio
module, which provides the infrastructure for running coroutines and
managing their execution.
Using asyncio for I/O-bound tasks
Asyncio is a library in Python that helps to write asynchronous code,

which can be beneficial for handling I/O-bound tasks. When a
program performs a lot of I/O-bound tasks, it often spends most of
its time waiting for the I/O operation to complete. By using asyncio,
we can write code that efficiently manages I/O-bound tasks and can
improve the overall performance of the program.
Here's how asyncio works: Instead of waiting for the I/O operation to
complete, it switches to another task that can be executed. When the
I/O operation completes, the corresponding task is resumed. This
way, the program can perform other operations while waiting for the
I/O operation to complete, making it more efficient.
To use asyncio, we need to write coroutines. A coroutine is a function
that can be paused and resumed during its execution. In Python, a
coroutine is defined using the async def keyword. Here's an example:
import asyncio
async def my_coroutine():
print("Coroutine started")
print("Coroutine resumed")
In this example, my_coroutine is a coroutine that prints a message,
pauses for one second using the await keyword, and then prints
another message. The await keyword tells asyncio to suspend the
coroutine until the sleep function completes.
To run a coroutine, we need to create an event loop. An event loop is
an object that manages the execution of coroutines. We can create
an event loop using the asyncio.get_event_loop function, like this:
loop = asyncio.get_event_loop()
Once we have an event loop, we can run a coroutine using the
run_until_complete method of the event loop. Here's an example:
loop.run_until_complete(my_coroutine())
This code will run the my_coroutine coroutine until it completes.
Now let's look at an example of using asyncio to perform I/O-bound
tasks. In this example, we'll download a list of URLs and save the
contents to a file. We'll use the aiohttp library to perform the
downloads. Here's the code:
import asyncio
import aiohttp
async def download_coroutine(session, url):
async with session.get(url) as response:
filename = url.split("/")[-1]
with open(filename, "wb") as f:
while True:
chunk = await
response.content.read(1024)
if not chunk:
break
f.write(chunk)
print(f"Downloaded {url}")
async def download_all(urls):
async with aiohttp.ClientSession() as
session:
tasks = []
for url in urls:
task =
asyncio.ensure_future(download_coroutine(
session, url))
tasks.append(task)
await asyncio.gather(*tasks)
urls = [
"https://www.python.org",
"https://www.google.com",
"https://www.bing.com",
"https://www.yahoo.com",
]
loop = asyncio.get_event_loop()
loop.run_until_complete(download_all(urls))
In this code, we define a coroutine called download_coroutine that
downloads a file from a URL using aiohttp and saves it to a file. We
also define a coroutine called download_all that runs multiple
download_coroutine coroutines concurrently using asyncio.gather. We
pass a list of URLs to download_all, and it downloads all the files
concurrently.
Using asyncio for CPU-bound tasks
Asyncio is a Python library that is used to write concurrent code in a

simple and efficient way. Traditionally, asyncio was mainly used for
IO-bound tasks, but it can also be used for CPU-bound tasks. In this
note, we will discuss how to use asyncio for CPU-bound tasks.
When we talk about CPU-bound tasks, we refer to tasks that involve
significant computation and are CPU-intensive. These tasks can block
the event loop and make the program unresponsive. To avoid this, we
can use asyncio to run these tasks in a separate thread or process.
To use asyncio for CPU-bound tasks, we need to create a custom
event loop and run our tasks in a separate executor. Here is an
example of how to do this:
import asyncio
async def cpu_bound_task(num):
"""
A sample CPU-bound task that computes
the sum of the first N natural numbers
"""
return sum(range(num))
async def main():
"""
The main function that creates an executor
and runs the task
"""
loop = asyncio.get_running_loop()
executor =
concurrent.futures.ThreadPoolExecutor()
result = await
loop.run_in_executor(executor,
cpu_bound_task, 1000000)
print(f"The result is {result}")
asyncio.run(main())
In this example, we define a CPU-bound task that computes the sum
of the first N natural numbers. We then define a main function that
creates an executor and runs the task using the run_in_executor
method of the event loop. The ThreadPoolExecutor is used in this
case, but we could also use a ProcessPoolExecutor.
The await keyword is used to suspend the coroutine until the result is
ready. When the task is complete, the result is printed to the console.
One thing to note is that using an executor comes with some
overhead, so it may not always be the best option for small tasks.
However, for large and complex tasks, using an executor can
significantly improve performance.
Asyncio can be used for CPU-bound tasks by creating a custom
event loop and running the tasks in a separate executor. This can
significantly improve performance and prevent the program from
becoming unresponsive.
Using asyncio with third-party libraries
Asyncio is a powerful tool for writing concurrent code in Python. While

it was originally designed for IO-bound tasks, it can also be used with
third-party libraries that support asyncio. In this note, we will discuss
how to use asyncio with third-party libraries.
Many popular Python libraries have added support for asyncio,
making it easier to write concurrent code with these libraries. Some
examples of popular libraries that support asyncio include:
aiohttp: A library for making HTTP requests asynchronously
aioredis: A library for using Redis asynchronously
asyncpg: A library for using PostgreSQL asynchronously
aiomysql: A library for using MySQL asynchronously
Using these libraries with asyncio is usually straightforward. Here is

an example of how to use aiohttp to make an HTTP request
asynchronously:
import asyncio
import aiohttp
async def main():
"""
The main function that makes an HTTP
request asynchronously
"""
async with aiohttp.ClientSession() as
session:
async with
session.get('https://www.google.com') as
response:
print(response.status)
print(await response.text())
asyncio.run(main())
In this example, we define a main function that makes an HTTP
request asynchronously using the aiohttp library. We create a
ClientSession and use it to make a GET request to the Google
homepage. We then print the status code and the text of the
response.
Another example is using aioredis to interact with a Redis database
asynchronously:
import asyncio
import aioredis
async def main():
"""
The main function that interacts with Redis
asynchronously
"""
redis = await
aioredis.create_redis_pool('redis://localhost')
await redis.set('key', 'value')
value = await redis.get('key', encoding='utf-
8')
print(value)
redis.close()
await redis.wait_closed()
asyncio.run(main())
In this example, we define a main function that interacts with a Redis
database asynchronously using the aioredis library. We create a
Redis pool and use it to set a key-value pair and get the value of the
key. We then print the value and close the connection to the Redis
database.
Using asyncio with third-party libraries can be a powerful way to write
concurrent code in Python. Many popular libraries have added
support for asyncio, making it easier to write asynchronous code with
these libraries. By combining the power of asyncio with these
libraries, we can write code that is both efficient and easy to main
Debugging asyncio code
Debugging asyncio code can be challenging, especially when dealing

with complex asynchronous programs. In this note, we will discuss
some techniques and tools that can be used to debug asyncio code.
One common technique for debugging asyncio code is to use print
statements. However, this can be difficult because of the
asynchronous nature of the code. One way to overcome this is to use
the asyncio.gather function to wait for multiple coroutines to complete
before printing the results. Here's an example:
import asyncio
async def coroutine1():
print("Start coroutine 1")
print("End coroutine 1")
async def main():
print("Start main")
await asyncio.gather(coroutine1(),
coroutine2())
print("End main")
asyncio.run(main())
In this example, we define two coroutines that each print a start
message, sleep for a specified amount of time, and then print an end
message. We also define a main coroutine that calls the
asyncio.gather function to run both coroutines concurrently. We then
print a start message, wait for the coroutines to complete, and print
an end message.
Another technique for debugging asyncio code is to use the
asyncio.Task.all_tasks() method to get a list of all pending tasks. This
can be useful for finding tasks that are stuck or taking too long to
complete. Here's an example:
import asyncio
async def main():
print("Start main")
task1 = asyncio.create_task(coroutine1())
task2 = asyncio.create_task(coroutine2())
tasks = asyncio.Task.all_tasks()
print("All tasks:", tasks)
await asyncio.gather(task1, task2)
print("End main")
asyncio.run(main())
In this example, we use the asyncio.create_task function to create
two tasks that run concurrently. We then use the
asyncio.Task.all_tasks() method to get a list of all pending tasks and
print it to the console. Finally, we wait for the tasks to complete and
print an end message.
Additionally, tools like aiodebug and asyncio.run_in_executor can also
be used for debugging asyncio code. aiodebug is a third-party library
that provides a debugger for asyncio applications, while
asyncio.run_in_executor can be used to run synchronous code in an
executor, which can make it easier to debug.
Debugging asyncio code can be challenging, but using techniques
such as print statements, the asyncio.Task.all_tasks() method, and
tools like aiodebug and asyncio.run_in_executor can help to make it
easier. By using these techniques, you can quickly identify and
resolve issues in your asyncio applications.
Chapter 6:
Built-in Modules
Python is a high-level, interpreted programming language that has
gained immense popularity over the years due to its simplicity,
versatility, and ease of use. One of the many reasons for Python's
popularity is its extensive library of built-in modules, which provide a
vast array of functions and tools to developers, without requiring them
to write the code from scratch.
Built-in modules in Python are pre-existing modules that come
bundled with the Python installation and offer a range of functionalities
that can be used in a wide range of applications. These modules are
designed to save developers time and effort by providing ready-to-
use functions that can perform complex tasks quickly and efficiently.
In this chapter, we will explore the various built-in modules available in
Python and discuss how they can be used to simplify development
and increase productivity. We will cover a range of modules, from the
more commonly used ones, such as math, datetime, and os, to some
of the lesser-known ones, such as ctypes, pickle, and hashlib.
We will begin by discussing the math module, which provides a range
of mathematical functions, including trigonometric, logarithmic, and
exponential functions, among others. The module is widely used in
scientific applications and can be used to perform a variety of
calculations, such as finding the square root of a number or
generating random numbers.
Next, we will look at the datetime module, which provides a range of
functions for working with dates and times. The module can be used
to perform various operations, such as calculating the difference
between two dates, formatting dates and times, and converting
between different time zones.
We will also explore the os module, which provides functions for
interacting with the operating system. The module can be used to
perform various tasks, such as creating and deleting files and
directories, navigating file systems, and setting environment variables.
Another module that we will discuss is the pickle module, which is
used for serializing and deserializing Python objects. The module
allows developers to store Python objects in a file or a database and
retrieve them later, making it easier to work with complex data
structures.
We will also cover the hashlib module, which provides functions for
generating secure hashes of data. The module can be used to
generate hashes of passwords, to verify the integrity of data, and to
ensure that data has not been tampered with.
Throughout the chapter, we will provide examples of how these
modules can be used in real-world applications, including web
development, data analysis, and machine learning. We will also
discuss some of the best practices for using built-in modules in
Python, such as importing only the required modules, using aliases for
long module names, and handling exceptions.
Built-in modules in Python are a powerful tool for developers,
providing a range of functionalities that can be used to simplify
development and increase productivity. By exploring the various
modules available in Python, we can gain a better understanding of
how they can be used to solve complex problems and build robust
applications.
Collections
Using namedtuple
Python's collections module provides several useful data structures

that are not included in the standard built-in types. One of these data
structures is namedtuple, which is a subclass of tuple that has named
fields. In this note, we will discuss how to use namedtuple and
provide some sample code.
namedtuple is defined using the collections.namedtuple() factory
function. The first argument to this function is the name of the new
tuple type, and the second argument is a string containing the names
of the fields separated by spaces or commas. Here's an example:

p = Point(1, 2)
print(p)
print(p.x)
print(p.y)
In this example, we define a new namedtuple type called Point with
two fields, x and y. We then create a new instance of this type with
values (1, 2) and print it to the console. We also print the values of
the x and y fields individually.
One advantage of using namedtuple is that it provides a more
readable and self-documenting alternative to defining tuples or using
dictionaries. For example, instead of using a plain tuple or a
dictionary to represent a 2D point, we can use a namedtuple with
fields x and y:
p = (1, 2)
# vs
p = {'x': 1, 'y': 2}
# vs
p = Point(1, 2)
namedtuple also provides some convenient features, such as the
ability to access fields using dot notation instead of indexing:
p = (1, 2)
x = p[0]
y = p[1]
# vs
p = Point(1, 2)
x = p.x
y = p.y
namedtuple instances are immutable, which means that their values
cannot be changed once they are created. This can help to prevent
bugs caused by accidental modifications of tuples.
namedtuple can also be used in some situations where a class would
be overkill, such as when defining simple data structures that only
have a few fields. For example, a namedtuple could be used to
represent a user's login credentials:
User = namedtuple('User', ['username',

'password'])
user = User('john_doe', 'password123')
Namedtuple is a useful data structure provided by Python's
collections module. It can be used to create tuples with named fields,
providing a more readable and self-documenting alternative to plain
tuples or dictionaries. namedtuple instances are immutable, and they
can be used in situations where a full-fledged class would be overkill.
Using deque
In Python, the collections module provides several useful data

structures that are not included in the standard built-in types. One of
these data structures is deque, which is a double-ended queue that
provides O(1) time complexity for adding or removing elements from
either end. In this note, we will discuss how to use deque and provide
some sample code.
To use deque, you first need to import it from the collections module:
from collections import deque
Once you have imported deque, you can create a new instance of the
deque using the following syntax:
my_deque = deque()
This creates an empty deque instance. You can also pass an iterable
to the deque() constructor to initialize it with some values:
my_deque = deque([1, 2, 3])

Now that you have a deque instance, you can add elements to it
using the append() method. This adds an element to the right end of
the deque:
my_deque.append(4)
You can also add elements to the left end of the deque using the
appendleft() method:
my_deque.appendleft(0)
To remove an element from the right end of the deque, you can use
the pop() method:
last_element = my_deque.pop()
This removes the last element from the deque and returns it.
Similarly, to remove an element from the left end of the deque, you
can use the popleft() method:
first_element = my_deque.popleft()
This removes the first element from the deque and returns it.
deque also provides a rotate() method that allows you to rotate the
deque by a specified number of steps. Positive values rotate the
deque to the right, while negative values rotate it to the left:
my_deque.rotate(1) # rotates the deque to

the right by one step
Finally, deque provides a few other methods for querying and
manipulating the deque, such as clear(), extend(), and remove(). You
can read more about these methods in the Python documentation.
Deque is a useful data structure provided by Python's collections
module. It provides O(1) time complexity for adding or removing
elements from either end of the deque, and it also provides a few
other useful methods for querying and manipulating the deque. deque
can be used in situations where you need to efficiently add or remove
elements from both ends of a collection.
Using defaultdict
The defaultdict is a powerful tool in Python's built-in collections

module that provides a convenient way to create dictionaries with
default values for missing keys. By using a defaultdict, you can
simplify your code and avoid having to manually check if a key is
already in the dictionary before accessing or modifying its value.
Here's an example of how to use a defaultdict to count the frequency
of items in a list:
from collections import defaultdict

my_list = ['apple', 'banana', 'apple', 'cherry',
'cherry', 'cherry']
my_dict = defaultdict(int)
my_dict[item] += 1
print(my_dict)
In this code, we import the defaultdict class from the collections
module and create a new instance called my_dict with a default value
of int (which is 0). Then, we iterate over the items in my_list and add
1 to the value associated with each item in my_dict. Finally, we print
out the resulting dictionary, which shows the frequency of each item in
the list:
defaultdict(<class 'int'>, {'apple': 2, 'banana':

1, 'cherry': 3})
Notice that we didn't have to check whether each key was already in
the dictionary before incrementing its value. If a key didn't exist yet,
its default value of 0 was used instead, and then incremented.
Another useful feature of defaultdict is the ability to specify a default
value function that gets called whenever a missing key is accessed.
This can be useful if you want to use a different default value
depending on the context. Here's an example:
from collections import defaultdict

def default_list():
return []
my_dict = defaultdict(default_list)
my_dict['colors'].append('red')
my_dict['colors'].append('blue')
my_dict['fruits'].append('apple')
print(my_dict)
In this code, we define a function called default_list that returns an
empty list. Then, we create a new defaultdict called my_dict that
uses this function as its default value. We add some items to my_dict
using the append method, and then print out the resulting dictionary:
defaultdict(<function default_list at
0x7f7f60eeca60>, {'colors': ['red', 'blue'],
'fruits': ['apple']})
Notice that the default value function was only called when a missing
key was accessed, not when the dictionary was first created.
Overall, defaultdict can be a powerful tool for simplifying your code
and avoiding common errors. By providing a default value for missing
keys, you can avoid having to manually check for their existence and
handle them separately.
Using OrderedDict
The OrderedDict is a class provided by the collections module in

Python that is similar to a regular dictionary, but with the added
feature of preserving the order in which the items were inserted. This
can be particularly useful in cases where the order of items in a
dictionary matters.
To use OrderedDict, first, we need to import it from the collections
module. Here's an example:
from collections import OrderedDict

Now, let's see some of the useful methods that are available with
OrderedDict:
1. Creating an OrderedDict
To create an OrderedDict, we can simply call the OrderedDict()
constructor. Here's an example:
od = OrderedDict()
2. Adding elements to an OrderedDict
To add an element to an OrderedDict, we can use the update()
method. Here's an example:
od.update({'a': 1})
od.update({'b': 2})
od.update({'c': 3})
This will add the key-value pairs ('a', 1), ('b', 2), and ('c', 3) to the
OrderedDict, in that order.
Alternatively, we can also use the od[key] = value syntax to add an
element to the OrderedDict. Here's an example:
od['d'] = 4
3. Removing elements from an OrderedDict
To remove an element from an OrderedDict, we can use the pop()
method. Here's an example:
od.pop('a')
This will remove the key-value pair ('a', 1) from the OrderedDict.
Alternatively, we can also use the del keyword to remove an element
from the OrderedDict. Here's an example:
del od['b']
This will remove the key-value pair ('b', 2) from the OrderedDict.
4. Iterating over an OrderedDict
To iterate over an OrderedDict, we can simply use a for loop. Here's
an example:
for key, value in od.items():
print(key, value)
This will print the key-value pairs in the OrderedDict, in the order in
which they were inserted.
5. Reversing the order of an OrderedDict
To reverse the order of an OrderedDict, we can use the reversed()
function. Here's an example:
for key, value in reversed(od.items()):

print(key, value)
This will print the key-value pairs in the OrderedDict in reverse order.
Example usage of OrderedDict
Here's an example of using OrderedDict to count the number of
occurrences of words in a text:
text = "the quick brown fox jumps over the

lazy dog"
words = text.split()
word_count = OrderedDict()
for word in words:
if word in word_count:
word_count[word] += 1
else:
word_count[word] = 1
for key, value in word_count.items():
print(key, value)
This will output the following:
the 2
quick 1
brown 1
fox 1
jumps 1
over 1
lazy 1
dog 1
As you can see, OrderedDict has preserved the order in which the
words were inserted, allowing us to count the number of occurrences
of each word in the text while still preserving the original order.
Using Counter
Counter is a built-in module in Python that provides a simple and

efficient way to count the frequency of elements in an iterable. It is
particularly useful when working with large datasets where counting
occurrences manually can be tedious and time-consuming. In this
note, we will discuss how to use the Counter module in Python.
To use the Counter module, we first need to import it. This can be
done using the following code:
from collections import Counter

Now, let's say we have a list of fruits and we want to count the
frequency of each fruit in the list. We can use the Counter module to
do this as follows:
fruits = ['apple', 'banana', 'orange', 'apple',

'banana', 'apple']
fruit_count = Counter(fruits)
print(fruit_count)
This will output:
Counter({'apple': 3, 'banana': 2, 'orange': 1})

As we can see, the Counter module has counted the frequency of
each fruit in the list and returned a dictionary-like object with the
counts.
We can also use the most_common() method of the Counter object
to get a list of the n most common elements and their counts. For
example, to get the two most common fruits in our list, we can use
the following code:
print(fruit_count.most_common(2))
This will output:
[('apple', 3), ('banana', 2)]

Another useful method of the Counter object is elements(), which
returns an iterator over the elements in the Counter object, repeating
each element as many times as its count. For example, to get an
iterator of all the fruits in our list, we can use the following code:
print(list(fruit_count.elements()))
This will output:
['apple', 'apple', 'apple', 'banana', 'banana',

'orange']
In addition to lists, Counter can also be used with other iterables such
as tuples, strings, and dictionaries. For example, to count the
frequency of characters in a string, we can use the following code:
sentence = "The quick brown fox jumps over
the lazy dog"
char_count = Counter(sentence)
print(char_count)
This will output:
Counter({' ': 8, 'o': 4, 'e': 3, 'u': 2, 'h': 2, 'r': 2,

'T': 1, 'q': 1, 'i': 1, 'c': 1, 'k': 1, 'b': 1, 'w': 1, 'n':
1, 'f': 1, 'x': 1, 'j': 1, 'm': 1, 'p': 1, 's': 1, 'v': 1, 't':
1, 'l': 1, 'a': 1, 'z': 1, 'y': 1, 'd': 1, 'g': 1})
The Counter module is a very useful tool for counting the frequency of
elements in iterables. Its simple interface and efficient implementation
make it a great choice for dealing with large datasets where counting
occurrences manually would be impractical.
Using ChainMap
ChainMap is a built-in module in Python that provides a way to

combine multiple dictionaries or mappings into a single, unified view. It
acts as a single dictionary that contains all the keys and values from
the input dictionaries, and allows us to perform lookups and
modifications on the entire chain of mappings at once. In this note, we
will discuss how to use the ChainMap module in Python.
To use the ChainMap module, we first need to import it. This can be
from collections import ChainMap

Now, let's say we have two dictionaries, one representing the default
configuration settings and the other representing the user-defined
settings. We want to merge these two dictionaries into a single
dictionary with the user-defined settings taking precedence over the
default settings. We
can use the ChainMap module to do this as follows:
default_settings = {'debug': False, 'log_level':

'INFO', 'timeout': 30}
user_settings = {'log_level': 'DEBUG',
'timeout': 60}
settings = ChainMap(user_settings,
default_settings)
print(settings)
This will output:
ChainMap({'log_level': 'DEBUG', 'timeout':

60}, {'debug': False, 'log_level': 'INFO',
'timeout': 30})
As we can see, the ChainMap object contains all the keys and values
from both dictionaries, with the user-defined settings taking
precedence over the default settings.
We can now perform lookups and modifications on the settings
dictionary, and these will be reflected in both the user_settings and
default_settings dictionaries. For example, to get the value of the
'log_level' key, we can use the following code:
print(settings['log_level'])
This will output:
DEBUG
To modify the value of the 'timeout' key, we can use the following
code:
settings['timeout'] = 90
print(default_settings['timeout'])
print(user_settings['timeout'])
This will output:
30
60
As we can see, the value of the 'timeout' key in the default_settings
dictionary has not changed, while the value in the user_settings
dictionary has also remained unchanged. This is because the
ChainMap object only modifies the first dictionary in the chain that
contains the key.
In addition to dictionaries, ChainMap can also be used with other
mappings such as OrderedDicts and defaultdicts. For example, to
merge two OrderedDicts into a single OrderedDict with the keys and
values in the order they were added, we can use the following code:
from collections import OrderedDict
od1 = OrderedDict([('a', 1), ('b', 2)])
od2 = OrderedDict([('c', 3), ('d', 4)])
od = ChainMap(od2, od1)
print(od)
This will output:
ChainMap(OrderedDict([('c', 3), ('d', 4)]),

OrderedDict([('a', 1), ('b', 2)]))
The ChainMap module is a very useful tool for combining multiple
dictionaries or mappings into a single, unified view. Its simple
interface and efficient implementation make it a great choice for
dealing with complex configuration settings or other scenarios where
multiple mappings need to be combined.
Using UserDict
UserDict is a built-in module in Python that provides a convenient way

to create our own dictionary-like objects. It is a subclass of the built-
in dict class, but with some additional features that make it easier to
customize the behavior of the dictionary. In this note, we will discuss
how to use the UserDict module in Python.
To use the UserDict module, we first need to import it. This can be
from collections import UserDict

Now, let's say we want to create a dictionary-like object that allows
us to access the values using both keys and attribute names. We can
create a new class that inherits from the UserDict class and
implements the getattr method to allow attribute access. Here's an
example:
class MyDict(UserDict):
def __getattr__(self, key):
if key in self.data:
return self.data[key]
elif key in self.__dict__:
return self.__dict__[key]
else:
raise AttributeError(f"'MyDict' object
has no attribute '{key}'")
In this example, we define a new class called MyDict that inherits
from the UserDict class. We override the getattr method to allow
attribute access to the dictionary keys. The method first checks if the
key is in the self.data dictionary, and returns the value if it is. If the
key is not in the self.data dictionary, it checks if it is in the instance's
dict attribute, which contains the instance's attributes. If the key is not
found in either dictionary, it raises an AttributeError.
We can now create an instance of this class and access the values
using both keys and attributes. Here's an example:
d = MyDict({'a': 1, 'b': 2})

print(d.a)
print(d['b'])
This will output:
1
2
As we can see, we can access the values using both attributes and
keys.
We can also override other methods of the UserDict class to
customize the behavior of our dictionary-like object. For example, we
can override the setitem method to allow setting values using
attributes as well as keys. Here's an example:
class MyDict(UserDict):
def __getattr__(self, key):
if key in self.data:
return self.data[key]
elif key in self.__dict__:
return self.__dict__[key]
else:
raise AttributeError(f"'MyDict' object
has no attribute '{key}'")
def __setitem__(self, key, value):
self.data[key] = value
setattr(self, key, value)
In this example, we add a new setitem method that sets the value
using both the key and the attribute name. We use the setattr method
to set the attribute with the same name as the key to the value.
We can now create an instance of this class and set values using
both keys and attributes. Here's an example:
d = MyDict()
d.a = 1
d['b'] = 2
print(d.a)
print(d['b'])
This will output:
1
2
As we can see, we can set values using both attributes and keys.
The UserDict module is a very useful tool for creating custom
dictionary-like objects that can be accessed using both keys and
attributes. Its simple interface and flexible implementation make it a
great choice for scenarios where custom dictionary behavior is
needed.
Using UserList
UserList is a built-in module in Python that provides a way to create
list-like objects with customized behavior. It is a subclass of the built-
in list class, but with some additional features that make it easier to
customize the behavior of the list. In this note, we will discuss how to
use the UserList module in Python.
To use the UserList module, we first need to import it. This can be
from collections import UserList

Now, let's say we want to create a list-like object that always returns
the sum of the values in the list when we call the sum method. We
can create a new class that inherits from the UserList class and
implements the sum method to return the sum of the values in the list.
Here's an example:
class MyList(UserList):
def sum(self):
return sum(self.data)
In this example, we define a new class called MyList that inherits
from the UserList class. We add a new sum method that returns the
sum of the values in the list by calling the built-in sum function on the
self.data attribute, which contains the list of values.
We can now create an instance of this class and call the sum method
to get the sum of the values in the list. Here's an example:
l = MyList([1, 2, 3, 4])
print(l.sum())
This will output:
10
As we can see, the sum method returns the sum of the values in the
list.
We can also override other methods of the UserList class to
customize the behavior of our list-like object. For example, we can
override the getitem method to return the negative index when a
positive index is provided. Here's an example:
class MyList(UserList):
def sum(self):
return sum(self.data)
def __getitem__(self, index):
if index >= 0:
return self.data[index]
else:
return self.data[len(self.data) + index]
In this example, we add a new getitem method that returns the
negative index when a positive index is provided. If the index is
greater than or equal to zero, it returns the value at that index. If the
index is negative, it calculates the corresponding positive index by
adding the length of the list to the index and returns the value at that
index.
We can now create an instance of this class and access the values
using negative indices as well as positive indices. Here's an example:
l = MyList([1, 2, 3, 4])
print(l[0])
print(l[-1])
This will output:
1
4
As we can see, we can access the values using both positive and
negative indices.
The UserList module is a very useful tool for creating custom list-like
objects that can be customized to suit specific needs. Its simple
interface and flexible implementation make it a great choice for
scenarios where custom list behavior is needed.
Using UserString
The UserString module is a built-in Python module that provides a

convenient way to work with strings in a more flexible and
customizable manner. The module provides a wrapper class called
UserString, which allows you to create string-like objects that can be
customized to your specific needs. In this note, we will explore how to
use the UserString module and its various features.
Creating a UserString object:
To create a UserString object, we simply instantiate the UserString
class with the string we want to work with as an argument. Here's an
example:
from collections import UserString

my_string = UserString("Hello, World!")
print(my_string)
In this example, we created a UserString object called my_string that
contains the string "Hello, World!". We then printed the object, which
outputs the string just like any regular string object would.
Customizing a UserString object:
One of the key features of the UserString module is the ability to
customize the behavior of the UserString object. This is done by
subclassing the UserString class and overriding various methods.
For example, let's say we want to create a UserString object that
always outputs its contents in uppercase letters. We can do this by
creating a subclass of UserString and overriding the __str__ method:
class UppercaseString(UserString):
def __str__(self):
return self.data.upper()
my_string = UppercaseString("Hello,
World!")
print(my_string)
In this example, we created a new class called UppercaseString that
inherits from UserString. We then defined the __str__ method to
return the string in all uppercase letters. When we create a new
UppercaseString object and print it, we see that the string is indeed in
all uppercase letters.
Using a UserString object with built-in string methods:
Another advantage of the UserString module is that UserString
objects can be used with most of the built-in string methods. This
includes methods such as strip, replace, split, and many others.
For example, let's say we want to create a UserString object that
strips all whitespace from the beginning and end of the string. We can
do this by creating a subclass of UserString and overriding the
__str__ method:
class StrippedString(UserString):
def __str__(self):
return self.data.strip()
my_string = StrippedString(" Hello,
World! ")
print(my_string)
In this example, we created a new class called StrippedString that
inherits from UserString. We then defined the __str__ method to
return the stripped version of the string. When we create a new
StrippedString object and print it, we see that the whitespace has
been removed from the beginning and end of the string.
The UserString module is a powerful tool for working with strings in
Python. By creating UserString objects and customizing their
behavior, we can create strings that fit our specific needs. And
because UserString objects can be used with most built-in string
methods, they can be seamlessly integrated into existing code.
Itertools
Using count, cycle, and repeat
The itertools module in Python provides a collection of functions for

working with iterable objects efficiently. Among these functions are
count, cycle, and repeat, which are used to generate infinite or finite
sequences of values. In this note, we will explore how to use count,
cycle, and repeat in itertools.
count function:
The count function generates an infinite sequence of numbers starting
from a specified value and incrementing by a specified step. Here's
an example:
from itertools import count
for i in count(start=1, step=2):
if i > 10:
break
print(i)
In this example, we import the count function from itertools. We then
use a for loop to iterate over an infinite sequence of numbers starting
from 1 and incrementing by 2. We use the break statement to
terminate the loop once the value of i exceeds 10.
cycle function:
The cycle function generates an infinite sequence by cycling through
the values of an iterable. Here's an example:
from itertools import cycle
colors = ['red', 'green', 'blue']
color_cycle = cycle(colors)
for i in range(6):
print(next(color_cycle))
In this example, we import the cycle function from itertools. We then
create a list of colors and use the cycle function to create an infinite
sequence that cycles through the values of the colors list. We use the
next function to retrieve the next value in the sequence and print it.
We repeat this process six times, which produces the output:
red
green
blue
red
green
blue
repeat function:
The repeat function generates an infinite sequence by repeating a
specified value a specified number of times. Here's an example:
from itertools import repeat

for i in repeat(10, 5):
print(i)
In this example, we import the repeat function from itertools. We then
use a for loop to iterate over a sequence that repeats the value 10
five times. We print each value in the sequence, which produces the
output:
10
10
10
10
10
The count, cycle, and repeat functions in itertools provide a
convenient way to generate infinite or finite sequences of values. By
using these functions, we can easily create sequences of numbers,
cycle through the values of an iterable, or repeat a value a specified
number of times. These functions are powerful tools that can be used
in a wide range of applications.
Using chain, tee, and zip_longest
The itertools module in Python provides a collection of functions for

working with iterable objects efficiently. Among these functions are
chain, tee, and zip_longest, which are used to manipulate and
combine iterables. In this note, we will explore how to use chain, tee,
and zip_longest in itertools.
chain function:
The chain function is used to combine multiple iterables into a single
iterable. Here's an example:
from itertools import chain

numbers = [1, 2, 3]
combined = chain(colors, numbers)
for item in combined:
print(item)
In this example, we import the chain function from itertools. We then
create two lists: colors and numbers. We use the chain function to
combine the two lists into a single iterable called combined. We use a
for loop to iterate over the combined iterable and print each item in
the sequence. The output of this code is:
red
green
blue
1
2
3
tee function:
The tee function is used to create multiple independent iterators from
a single iterable. Here's an example:
from itertools import tee

iter1, iter2 = tee(colors, 2)
print(list(iter1))
print(list(iter2))
In this example, we import the tee function from itertools. We then
create a list called colors. We use the tee function to create two
independent iterators (iter1 and iter2) from the colors list. We use the
list function to convert the iterators to lists and print them. The output
of this code is:
['red', 'green', 'blue']

['red', 'green', 'blue']
zip_longest function:
The zip_longest function is used to combine two or more iterables
into a single iterable. Unlike the built-in zip function, which stops when
the shortest iterable is exhausted, zip_longest continues until the
longest iterable is exhausted. Here's an example:
from itertools import zip_longest

numbers = [1, 2, 3, 4]
combined = zip_longest(colors, numbers)
for item in combined:
print(item)
In this example, we import the zip_longest function from itertools. We
then create two lists: colors and numbers. We use the zip_longest
function to combine the two lists into a single iterable called
combined. We use a for loop to iterate over the combined iterable
and print each item in the sequence. The output of this code is:
('red', 1)
('green', 2)
('blue', 3)
(None, 4)
The chain, tee, and zip_longest functions in itertools provide a
convenient way to manipulate and combine iterables. By using these
functions, we can easily combine multiple iterables into a single
iterable, create multiple independent iterators from a single iterable,
or combine two or more iterables into a single iterable with different
lengths. These functions are powerful tools that can be used in a
wide range of applications.
Using islice, dropwhile, and takewhile
One useful aspect of Python's itertools module is its ability to

manipulate and iterate over iterable objects in a variety of ways.
Among the most useful functions in this module are islice, dropwhile,
and takewhile, which allow you to perform complex operations on
iterables with minimal code.
islice() Function:
The islice() function allows you to slice an iterable object like a list,
tuple, or string, just like you would with square brackets, but without
actually creating a new list. This means that you can slice very large
iterables without taking up a lot of memory.
The syntax of the islice() function is as follows:
from itertools import islice

islice(iterable, start, stop[, step])
Here, iterable is the iterable object you want to slice, start is the
starting index of the slice, stop is the ending index of the slice, and
step is the step size. You can omit the step argument if you want to
slice the iterable in the default step size of 1.
Here's an example of using islice() to slice a list:
my_list = ['a', 'b', 'c', 'd', 'e']

print(list(islice(my_list, 1, 4)))
Output:
['b', 'c', 'd']

dropwhile() Function:
The dropwhile() function allows you to drop elements from an iterable
until a certain condition is met. Once the condition is met, the function
returns the remaining elements of the iterable.
The syntax of the dropwhile() function is as follows:
from itertools import dropwhile

dropwhile(predicate, iterable)
Here, predicate is a function that takes an element of the iterable as
an argument and returns a Boolean value indicating whether to drop
the element or not. iterable is the iterable object you want to iterate
over.
Here's an example of using dropwhile() to drop elements from a list:
my_list = [1, 3, 5, 7, 2, 4, 6]
print(list(dropwhile(lambda x: x < 5,
my_list)))
Output:
[5, 7, 2, 4, 6]
takewhile() Function:
The takewhile() function allows you to take elements from an iterable
until a certain condition is met. Once the condition is not met, the
function stops taking elements and returns what it has taken so far.
The syntax of the takewhile() function is as follows:
from itertools import takewhile

takewhile(predicate, iterable)
Here, predicate is a function that takes an element of the iterable as
an argument and returns a Boolean value indicating whether to take
the element or not. iterable is the iterable object you want to iterate
over.
Here's an example of using takewhile() to take elements from a list:
my_list = [1, 3, 5, 7, 2, 4, 6]
print(list(takewhile(lambda x: x < 5, my_list)))
Output:
[1, 3]
The itertools module provides useful functions like islice(),
dropwhile(), and takewhile() to perform complex operations on
iterables. These functions are particularly useful for large data sets
where you want to perform operations efficiently and without creating
unnecessary lists.
Using groupby
The groupby() function in Python's itertools module is a powerful tool

for grouping items in an iterable based on a key function. The key
function returns a value for each item in the iterable, and the items
with the same value are grouped together into a single group.
The groupby() function works by creating an iterator that produces
pairs of keys and groups, where the key is the value returned by the
key function, and the group is an iterator that produces the items in
the iterable that have that key.
The syntax of the groupby() function is as follows:
from itertools import groupby

groupby(iterable, key=None)
Here, iterable is the iterable object you want to group, and key is an
optional function that takes an element of the iterable as an argument
and returns a key value for that element. If no key function is
provided, the element itself is used as the key.
Here's an example of using groupby() to group a list of words by their
first letter:
words = ['apple', 'banana', 'cherry', 'date',

'elderberry', 'fig']
groups = groupby(words, key=lambda x:
x[0])
for key, group in groups:
print(key, list(group))
Output:
a ['apple']
b ['banana']
c ['cherry']
d ['date']
e ['elderberry']
f ['fig']
In this example, we created a list of words, and then passed it to the
groupby() function along with a key function that returns the first letter
of each word. The function returned an iterator that produced pairs of
keys and groups, where each group contained the words that started
with the same letter.
We then looped over the iterator using a for loop, printing each key
and the list of words in the corresponding group. Note that the group
object produced by the iterator is itself an iterator, so we had to
convert it to a list to print it.
Here's another example of using groupby() to group a list of numbers
by their parity:
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

groups = groupby(numbers, key=lambda x:
'even' if x % 2 == 0 else 'odd')
for key, group in groups:

print(key, list(group))
Output:
odd [1]
even [2, 4, 6, 8, 10]
odd [3, 5, 7, 9]
In this example, we created a list of numbers, and then passed it to
the groupby() function along with a key function that returns the string
'even' for even numbers and 'odd' for odd numbers. The function
returned an iterator that produced pairs of keys and groups, where
each group contained the numbers with the same parity.
We then looped over the iterator using a for loop, printing each key
and the list of numbers in the corresponding group.
The groupby() function in Python's itertools module is a powerful tool
for grouping items in an iterable based on a key function. It is
particularly useful for tasks such as data analysis and data
manipulation, where grouping by certain criteria is a common
operation.
Using starmap and product
The starmap() and product() functions in Python's itertools module

are useful tools for performing calculations on multiple iterables. They
can be used to generate all possible combinations of elements from
two or more iterables and perform an operation on each combination.
The starmap() function takes a function and an iterable of tuples as
arguments. It applies the function to each tuple in the iterable,
unpacking the elements of the tuple as arguments to the function. The
result is an iterator that produces the results of the function for each
tuple in the iterable.
Here's an example of using starmap() to calculate the squares of the
numbers in a list of tuples:
from itertools import starmap
numbers = [(1, 2), (3, 4), (5, 6)]

squares = starmap(lambda x, y: x**2 + y**2,
numbers)
for square in squares:

print(square)
Output:
5
25
61
In this example, we created a list of tuples, where each tuple contains
two numbers. We then passed this list to the starmap() function along
with a lambda function that calculates the sum of the squares of the
two numbers in each tuple. The function returned an iterator that
produced the result of the lambda function for each tuple in the list.
We then looped over the iterator using a for loop, printing each result.
The product() function takes two or more iterables as arguments and
returns an iterator that produces tuples containing all possible
combinations of elements from the input iterables. The length of each
tuple is equal to the number of input iterables.
Here's an example of using product() to generate all possible
combinations of two lists:
from itertools import product
list1 = ['A', 'B']

list2 = [1, 2, 3]
combinations = product(list1, list2)
for combination in combinations:

print(combination)
Output:
('A', 1)
('A', 2)
('A', 3)
('B', 1)
('B', 2)
('B', 3)
In this example, we created two lists, and then passed them to the
product() function. The function returned an iterator that produced
tuples containing all possible combinations of elements from the two
input lists.
We then looped over the iterator using a for loop, printing each tuple.
The starmap() and product() functions in Python's itertools module
are useful tools for performing calculations on multiple iterables. They
can be used to generate all possible combinations of elements from
two or more iterables and perform an operation on each combination.
These functions are particularly useful for tasks such as combinatorial
optimization, simulations, and statistical analysis.
File and Directory Access

Using os and os.path
The os and os.path modules in Python provide a wide range of

functions for accessing files and directories on the file system. These
modules can be used to perform various operations such as
navigating directories, creating and deleting files, checking file
properties, and more.
Here are some examples of how to use the os and os.path modules
for file and directory access:
Navigating Directories:
The os module provides functions for navigating directories such as
chdir() to change the current working directory, listdir() to list the
contents of a directory, and mkdir() to create a new directory. Here's
an example:
import os
# get current working directory

print(os.getcwd())
# change working directory

os.chdir('/path/to/directory')
# list directory contents

print(os.listdir())
# create new directory

os.mkdir('new_directory')
File Properties:
The os.path module provides functions for working with file properties
such as exists() to check if a file or directory exists, getsize() to get
the size of a file, and isdir() to check if a given path is a directory.
Here's an example:
import os
# check if file exists

if os.path.exists('file.txt'):
print('file exists')
# get file size

print(os.path.getsize('file.txt'))
# check if path is a directory

if os.path.isdir('/path/to/directory'):
print('path is a directory')
File Operations:
The os module provides functions for performing file operations such
as remove() to delete a file, rename() to rename a file, and stat() to
get detailed information about a file. Here's an example:
import os
# delete file
os.remove('file.txt')
# rename file
os.rename('old_file.txt', 'new_file.txt')
# get file stats
stat_info = os.stat('file.txt')
print(stat_info.st_size)
Walking Directories:
The os module provides the walk() function, which can be used to
traverse a directory tree and perform operations on files and
directories. Here's an example:
import os
# traverse directory tree

for dirpath, dirnames, filenames in
os.walk('/path/to/directory'):
print('Current directory: {}'.format(dirpath))
print('Directories: {}'.format(dirnames))
print('Files: {}'.format(filenames))
These are just a few examples of the many functions provided by the
os and os.path modules for file and directory access. By utilizing
these modules, you can perform various operations on files and
directories in a convenient and efficient manner.
Using pathlib
Python's pathlib module provides an object-oriented interface to

access files and directories. It is part of the standard library and
offers a more intuitive way to work with file and directory paths
compared to the os and os.path modules.
Here are some examples of how to use the pathlib module for file and
directory access:
Creating Paths:
The pathlib.Path() function can be used to create a path object
representing a file or directory path. Here's an example:
from pathlib import Path

# create a path object representing a file
file_path = Path('/path/to/file.txt')
# create a path object representing a

directory
dir_path = Path('/path/to/directory')
Checking Path Properties:
The pathlib.Path() object provides a number of useful methods for
checking the properties of a file or directory path such as exists() to
check if a path exists, is_file() to check if a path is a file, and is_dir()
to check if a path is a directory. Here's an example:
# check if path exists

if file_path.exists():
print('file exists')
# check if path is a file

if file_path.is_file():
print('path is a file')
# check if path is a directory

if dir_path.is_dir():
print('path is a directory')
Creating Directories:
The pathlib.Path() object provides a mkdir() method that can be used
to create a new directory. Here's an example:
# create a new directory

dir_path = Path('/path/to/new/directory')
dir_path.mkdir()
Listing Directory Contents:
The pathlib.Path() object provides a iterdir() method that can be used
to iterate over the contents of a directory. Here's an example:
# list directory contents

for item in dir_path.iterdir():
print(item)
Reading and Writing Files:

The pathlib.Path() object provides methods for reading and writing
files such as read_text() to read the contents of a text file,
write_text() to write text to a file, and read_bytes() and write_bytes()
for reading and writing binary files. Here's an example:
# read contents of a file

contents = file_path.read_text()
print(contents)
# write text to a file

file_path.write_text('Hello, World!')
# read contents of a binary file

binary_file_path = Path('/path/to/binary/file')
binary_contents =
binary_file_path.read_bytes()
# write bytes to a binary file
binary_file_path.write_bytes(b'binary data')
These are just a few examples of the many functions and methods
provided by the pathlib module for file and directory access. By
utilizing this module, you can perform various operations on files and
directories in a more intuitive and object-oriented manner.
Using shutil
Python's shutil module provides a high-level interface for file and

directory operations. It is part of the standard library and can be used
to copy, move, or delete files and directories, as well as to archive
files.
Here are some examples of how to use the shutil module for file and
directory access:
Copying Files and Directories:
The shutil.copy() function can be used to copy a file, while the
shutil.copytree() function can be used to copy an entire directory tree.
Here's an example:
import shutil
# copy a file
src_file = '/path/to/source/file.txt'
dest_dir = '/path/to/destination'
shutil.copy(src_file, dest_dir)
# copy a directory tree

src_dir = '/path/to/source'
shutil.copytree(src_dir, dest_dir)
Moving and Renaming Files and Directories:
The shutil.move() function can be used to move or rename a file or
directory. Here's an example:
import shutil
# move a file
src_file = '/path/to/source/file.txt'
shutil.move(src_file, dest_dir)
# rename a file
src_file = '/path/to/source/old_name.txt'
dest_file = '/path/to/source/new_name.txt'
shutil.move(src_file, dest_file)
# move a directory
shutil.move(src_dir, dest_dir)
# rename a directory
src_dir = '/path/to/source/old_name'
dest_dir = '/path/to/source/new_name'
shutil.move(src_dir, dest_dir)
Removing Files and Directories:
The os.remove() function can be used to remove a file, while the
shutil.rmtree() function can be used to remove an entire directory
tree. Here's an example:
import os
import shutil
# remove a file
file_path = '/path/to/file.txt'
os.remove(file_path)
# remove a directory tree

dir_path = '/path/to/directory'
shutil.rmtree(dir_path)
Archiving Files:
The shutil.make_archive() function can be used to create an archive
file of a directory tree. Here's an example:
import shutil
# create a zip archive of a directory tree

archive_name = 'my_archive'
shutil.make_archive(archive_name, 'zip',
src_dir)
These are just a few examples of the many functions provided by the
shutil module for file and directory access. By utilizing this module,
you can perform various file and directory
operations in a simple and convenient way.
Using glob
Python's glob module provides a way to search for files and

directories that match a specified pattern. It uses Unix shell-style
wildcards such as * and ? to match filenames.
Here are some examples of how to use the glob module for file and
directory access:
Finding Files with a Specific Extension:
The glob.glob() function can be used to find all files with a specific
extension in a directory. Here's an example:
import glob
# find all .txt files in a directory

txt_files = glob.glob(f"{dir_path}/*.txt")
print(txt_files)
Finding Files with a Specific Name:
The glob.glob() function can also be used to find all files with a
specific name in a directory. Here's an example:
import glob
# find all files named 'file.txt' in a directory

file_path = f"{dir_path}/file.txt"
matching_files = glob.glob(file_path)
print(matching_files)
Finding Directories:
The glob.glob() function can also be used to find all directories in a
directory. Here's an example:
import glob
# find all directories in a directory

matching_dirs = glob.glob(f"{dir_path}/*/")
print(matching_dirs)
Recursive Search:
The glob.glob() function can also be used to recursively search for
files and directories in a directory and all its subdirectories. Here's an
example:
import glob
# find all .txt files in a directory and its
subdirectories
txt_files = glob.glob(f"{dir_path}/**/*.txt",
recursive=True)
print(txt_files)
These are just a few examples of the many ways in which the glob
module can be used for file and directory access. By utilizing this
module, you can easily search for files and directories that match a
specific pattern, which can be very useful for organizing and
processing large numbers of files.
Dates and Times
Using datetime
The datetime module in Python provides classes to work with dates

and times. It's useful when working with file and directory access
because it allows us to manipulate and format time stamps
associated with file metadata. In this note, we will explore how to use
datetime in file and directory access, including how to retrieve the
creation time, modification time, and access time of a file, as well as
how to convert time stamps to human-readable formats.
Retrieving Time Stamps:
To retrieve time stamps associated with file metadata, we can use
the os.path module, which provides functions for working with file
paths. Specifically, we can use the os.path.getctime(),
os.path.getmtime(), and os.path.getatime() functions to retrieve the
creation time, modification time, and access time of a file,
respectively. These functions return time stamps in seconds since the
epoch (January 1, 1970, 00:00:00 UTC).
import os
import datetime
# Get creation time of file

creation_time = os.path.getctime(file_path)
creation_time =
datetime.datetime.fromtimestamp(creation_ti
me)
print("Creation time:", creation_time)
# Get modification time of file

modification_time =
os.path.getmtime(file_path)
modification_time =
datetime.datetime.fromtimestamp(modificati
on_time)
print("Modification time:",
modification_time)
# Get access time of file

access_time = os.path.getatime(file_path)
access_time =
datetime.datetime.fromtimestamp(access_ti
me)
print("Access time:", access_time)
In the code above, we use the fromtimestamp() method of the
datetime.datetime class to convert the time stamps to datetime
objects, which we can then manipulate and format.
Converting Time Stamps:
To convert time stamps to human-readable formats, we can use the
strftime() method of the datetime.datetime class. This method allows
us to format a datetime object as a string, using a format string that
specifies the desired format.
import os
import datetime

modification_time =
modification_time =
datetime.datetime.fromtimestamp(modificati
on_time)
# Convert modification time to a string in ISO

format
modification_time_str =
modification_time.strftime("%Y-%m-%d
%H:%M:%S")
modification_time_str)
# Convert modification time to a string in a

custom format
modification_time_str =
modification_time.strftime("%b %d, %Y
%I:%M:%S %p")
In the code above, we use the %Y, %m, %d, %H, %M, and %S
format codes to represent the year, month, day, hour, minute, and
second of the datetime object, respectively. We also use the %b
format code to represent the abbreviated month name, %d to
represent the day of the month, %Y to represent the year, %I to
represent the hour in 12-hour format, %M to represent the minute,
%S to represent the second, and %p to represent the AM or PM
designation.
Using the datetime module in file and directory access allows us to
manipulate and format time stamps associated with file metadata. By
retrieving time stamps using the os.path module and converting them
to datetime objects, we can then use the strftime() method to format
them as human-readable strings.
Using time
The time module in Python provides functions to work with time and
date values. It is particularly useful when working with file and
directory access because it allows us to manipulate and format time
stamps associated with file metadata. In this note, we will explore
how to use time in file and directory access, including how to retrieve
the creation time, modification time, and access time of a file, as well
as how to convert time stamps to human-readable formats.
Retrieving Time Stamps:
To retrieve time stamps associated with file metadata, we can use
the os.path module, which provides functions for working with file
paths. Specifically, we can use the os.path.getctime(),
os.path.getmtime(), and os.path.getatime() functions to retrieve the
creation time, modification time, and access time of a file,
respectively. These functions return time stamps in seconds since the
epoch (January 1, 1970, 00:00:00 UTC).
import os
import time
# Get creation time of file

creation_time = os.path.getctime(file_path)
creation_time =
time.localtime(creation_time)
print("Creation time:", time.strftime("%Y-
%m-%d %H:%M:%S", creation_time))

modification_time =
modification_time =
time.localtime(modification_time)
time.strftime("%Y-%m-%d %H:%M:%S",
modification_time))
# Get access time of file

access_time = os.path.getatime(file_path)
access_time = time.localtime(access_time)
print("Access time:", time.strftime("%Y-%m-
%d %H:%M:%S", access_time))
In the code above, we use the localtime() function of the time module
to convert the time stamps to a time struct, which we can then
manipulate and format using the strftime() function.
Converting Time Stamps:
To convert time stamps to human-readable formats, we can use the
strftime() function of the time module. This function allows us to
format a time struct as a string, using a format string that specifies
the desired format.
import os
import time

modification_time =
modification_time =
time.localtime(modification_time)
# Convert modification time to a string in ISO

format
modification_time_str = time.strftime("%Y-
%m-%d %H:%M:%S", modification_time)
# Convert modification time to a string in a

custom format
modification_time_str = time.strftime("%b
%d, %Y %I:%M:%S %p", modification_time)
In the code above, we use the %Y, %m, %d, %H, %M, and %S
format codes to represent the year, month, day, hour, minute, and
second of the time struct, respectively. We also use the %b format
code to represent the abbreviated month name, %d to represent the
day of the month, %Y to represent the year, %I to represent the hour
in 12-hour format, %M to represent the minute, %S to represent the
second, and %p to represent the AM or PM designation.
Using timedelta
The timedelta class in Python's datetime module represents a

duration of time. It can be used to perform arithmetic operations on
date and time values, which makes it useful in file and directory
access when working with time intervals. In this note, we will explore
how to use timedelta in file and directory access, including how to
calculate time intervals, add and subtract time from a date or time,
and format time intervals.
Calculating Time Intervals:
To calculate time intervals, we can create two datetime objects
representing two points in time and subtract one from the other. The
result will be a timedelta object representing the duration of time
between the two points.
from datetime import datetime, timedelta

modification_time =
datetime.fromtimestamp(os.path.getmtime(fi
le_path))
# Get current time

current_time = datetime.now()
# Calculate time interval

time_interval = current_time -
modification_time
print("Time interval:", time_interval)
In the code above, we use the datetime.fromtimestamp() function to
convert the modification time of the file (retrieved using
os.path.getmtime()) to a datetime object. We then create a datetime
object representing the current time using the datetime.now()
function. Finally, we subtract the modification time from the current
time to obtain a timedelta object representing the time interval.
Adding and Subtracting Time:
We can add and subtract time from a datetime object using the
timedelta class. For example, we can add a certain number of days,
hours, minutes, or seconds to a datetime object.
from datetime import datetime, timedelta
# Get current time

current_time = datetime.now()
# Add 1 day to current time

new_time = current_time + timedelta(days=1)
print("New time:", new_time)
# Subtract 1 hour from current time

new_time = current_time -
timedelta(hours=1)
In the code above, we use the timedelta(days=1) and
timedelta(hours=1) functions to add and subtract 1 day and 1 hour,
respectively, to and from the datetime object representing the current
time.
Formatting Time Intervals:
We can format a timedelta object as a string using the str() function.
By default, the string representation of a timedelta object includes the
number of days, hours, minutes, and seconds.
from datetime import timedelta
# Create a timedelta object representing 1

hour, 30 minutes, and 45 seconds
time_interval = timedelta(hours=1,
minutes=30, seconds=45)
# Format timedelta object as a string

time_interval_str = str(time_interval)
print("Time interval:", time_interval_str)
In the code above, we create a timedelta object representing 1 hour,
30 minutes, and 45 seconds. We then use the str() function to format
the timedelta object as a string.
Using timedelta in file and directory access allows us to calculate time
intervals, add and subtract time from a date or time, and format time
intervals. By performing arithmetic operations on date and time
values, we can more easily work with time intervals in our Python
programs.
Using pytz
When working with time zones in file and directory access, it's
important to use a reliable library like pytz to ensure accurate and
consistent time zone conversion. In this note, we will explore how to
use pytz in file and directory access, including how to convert
between time zones and how to handle daylight saving time.
Installing pytz:
Before we can use pytz, we need to install it. We can install pytz
using pip:
pip install pytz

Converting Time Zones
To convert a date or time from one time zone to another, we can use
the pytz library. First, we need to create a datetime object
representing the date and time in the original time zone. We can then
use the pytz.timezone() function to specify the original time zone, and
the astimezone() method to convert the datetime object to the
desired time zone.
from datetime import datetime

import pytz
# Create a datetime object representing the

date and time in the original time zone
original_time = datetime(2023, 3, 17, 15, 30)
# Convert to a different time zone
original_timezone =
pytz.timezone('America/New_York')
new_timezone =
pytz.timezone('Europe/London')
new_time =
original_timezone.localize(original_time).asti
mezone(new_timezone)
print("Original time:", original_time)

In the code above, we create a datetime object representing the date
and time in the original time zone. We then use the pytz.timezone()
function to specify the original and new time zones, and the localize()
and astimezone() methods to convert the datetime object to the new
time zone. The resulting datetime object represents the same date
and time, but in the new time zone.
Handling Daylight Saving Time:
When working with time zones that observe daylight saving time, it's
important to handle transitions between standard time and daylight
saving time correctly. pytz provides functions to handle these
transitions automatically.

import pytz
date and time in the original time zone
original_time = datetime(2023, 3, 12, 2, 30)
# Convert to a different time zone that

observes daylight saving time
original_timezone =
pytz.timezone('America/New_York')
new_timezone =
pytz.timezone('Europe/London')
new_time =
original_timezone.localize(original_time,
is_dst=None).astimezone(new_timezone)
print("Original time:", original_time)

In the code above, we create a datetime object representing the date
and time in the original time zone. We then use the localize() method
with the is_dst=None argument to specify that the date and time
should be interpreted as ambiguous (i.e., it occurs during the
transition from standard time to daylight saving time). When we
convert the datetime object to the new time zone using the
astimezone() method, pytz automatically adjusts the time to account
for the daylight saving time transition.
Using pytz in file and directory access allows us to accurately and
reliably convert between time zones and handle daylight saving time.
By using a library like pytz, we can ensure that our Python programs
handle time zones correctly and consistently, even when dealing with
complex scenarios like daylight saving time transitions.
Using dateutil
When working with dates and times in file and directory access, we
often need to parse and manipulate date and time strings. The
dateutil library provides powerful tools for parsing and manipulating
date and time strings in a flexible and intuitive way. In this note, we
will explore how to use dateutil in file and directory access, including
how to parse date and time strings and how to perform date
arithmetic.
Installing dateutil:
Before we can use dateutil, we need to install it. We can install
dateutil using pip:
pip install python-dateutil

Parsing Date and Time Strings:
To parse a date or time string, we can use the dateutil.parser.parse()
function. This function can parse a wide variety of date and time
formats, including ISO 8601 format, RFC 2822 format, and many
others.
from dateutil.parser import parse
# Parse a date string in ISO 8601 format

date_string = '2023-03-17'
date = parse(date_string)
print("Parsed date:", date)

In the code above, we use the parse() function to parse a date string
in ISO 8601 format. The
resulting date object represents the same date, but as a Python
datetime object.
Performing Date Arithmetic:
Once we have parsed a date or time string, we can perform date
arithmetic using the dateutil.relativedelta.relativedelta() function. This
function allows us to add or subtract a specific amount of time from a
datetime object.

from dateutil.relativedelta import
relativedelta

current date and time
now = datetime.now()
# Add one week to the current date and time

one_week_from_now = now +
relativedelta(weeks=1)
print("Current date and time:", now)

print("One week from now:",
one_week_from_now)
In the code above, we use the datetime.now() function to create a

datetime object representing the current date and time. We then use
the relativedelta() function to add one week to the current date and
time. The resulting one_week_from_now object represents the date
and time one week in the future.
Using dateutil in file and directory access allows us to parse and
manipulate date and time strings in a flexible and intuitive way. By
using a library like dateutil, we can ensure that our Python programs
handle dates and times correctly and consistently, even when dealing
with a wide variety of date and time formats.
Serialization and Persistence

Using json
Serialization is the process of converting data from its native format

into a format that can be stored or transmitted. JSON (JavaScript
Object Notation) is a lightweight data interchange format that is easy
to read and write. In this note, we will explore how to use JSON in
serialization and persistence, including how to serialize Python objects
to JSON and how to store and retrieve JSON data using files.
Serializing Python Objects to JSON:
Python objects can be serialized to JSON using the json module. The
json module provides two methods for serializing Python objects to
JSON: dumps() and dump(). The dumps() method serializes a Python
object to a JSON-formatted string, while the dump() method
serializes a Python object and writes the resulting JSON to a file.
import json
# Create a Python dictionary

person = {"name": "John", "age": 30, "city":
"New York"}
# Serialize the dictionary to JSON

json_string = json.dumps(person)
# Print the JSON string

print(json_string)
# Serialize the dictionary to JSON and write
to a file
with open("person.json", "w") as f:
json.dump(person, f)
In the code above, we create a Python dictionary representing a
person's name, age, and city. We then use the json.dumps() method
to serialize the dictionary to a JSON-formatted string and print it.
Finally, we use the json.dump() method to serialize the dictionary to
JSON and write it to a file named person.json.
Deserializing JSON to Python Objects:
JSON data can be deserialized to Python objects using the json
module. The json module provides two methods for deserializing
JSON data to Python objects: loads() and load(). The loads() method
deserializes a JSON-formatted string to a Python object, while the
load() method deserializes JSON data from a file to a Python object.
import json
# Deserialize a JSON string to a Python
object
json_string = '{"name": "John", "age": 30,
"city": "New York"}'
person = json.loads(json_string)
# Print the Python object
print(person)
# Deserialize JSON data from a file to a

Python object
with open("person.json", "r") as f:
person = json.load(f)

print(person)
In the code above, we use the json.loads() method to deserialize a
JSON-formatted string to a Python object representing a person's
name, age, and city. We then use the json.load() method to
deserialize JSON data from a file named person.json to a Python
object representing the same person.
Using JSON in serialization and persistence allows us to easily store
and transmit data in a lightweight, easy-to-read format. By using the
json module in Python, we can serialize Python objects to JSON and
vice versa, allowing us to easily store and retrieve data in a
consistent, standardized format.
Using pickle

into a format that can be stored or transmitted. Pickle is a module in
Python that enables the serialization and deserialization of Python
objects to and from a binary format. In this note, we will explore how
to use pickle in serialization and persistence, including how to
serialize Python objects to a binary format using pickle and how to
store and retrieve pickled data using files.
Serializing Python Objects to a Binary Format using Pickle:
Python objects can be serialized to a binary format using the pickle
module. The pickle module provides two methods for serializing
Python objects: dumps() and dump(). The dumps() method serializes
a Python object to a binary string, while the dump() method serializes
a Python object and writes the resulting binary data to a file.
import pickle
# Create a Python dictionary
person = {"name": "John", "age": 30, "city":
"New York"}
# Serialize the dictionary to binary format

pickled_data = pickle.dumps(person)
# Print the pickled data

print(pickled_data)
# Serialize the dictionary to binary format

and write to a file
with open("person.pickle", "wb") as f:
pickle.dump(person, f)
In the code above, we create a Python dictionary representing a
person's name, age, and city. We then use the pickle.dumps()
method to serialize the dictionary to a binary string and print it.
Finally, we use the pickle.dump() method to serialize the dictionary to
a binary format and write it to a file named person.pickle.
Deserializing Pickled Data to Python Objects:
Pickled data can be deserialized to Python objects using the pickle
module. The pickle module provides two methods for deserializing
pickled data to Python objects: loads() and load(). The loads()
method deserializes a binary string to a Python object, while the
load() method deserializes pickled data from a file to a Python object.
import pickle
# Deserialize pickled data to a Python object

pickled_data =
b'\x80\x04\x95\x17\x00\x00\x00\x00\x00\x00\x00
}\x94(\x8c\x04name\x94\x8c\x04John\x94\x8c\x
03age\x94K\x1e\x8c\x04city\x94\x8c\tNew
York\x94u.'
person = pickle.loads(pickled_data)
print(person)
# Deserialize pickled data from a file to a
Python object
with open("person.pickle", "rb") as f:
person = pickle.load(f)

print(person)
In the code above, we use the pickle.loads() method to deserialize a
binary string to a Python object representing a person's name, age,
and city. We then use the pickle.load() method to deserialize pickled
data from a file named person.pickle to a Python object representing
the same person.
Using pickle in serialization and persistence allows us to easily store
and transmit data in a binary format. By using the pickle module in
Python, we can serialize Python objects to a binary format and vice
versa, allowing us to easily store and retrieve data in a consistent,
standardized format. However, it's important to note that the pickle
module is not secure, and deserializing untrusted pickled data can
potentially execute arbitrary code on your machine. It's recommended
to only unpickle data that comes from trusted sources.
Using shelve

into a format that can be stored or transmitted. Shelve is a built-in
module in Python that provides a simple way to persist and store
objects in a key-value store. In this note, we will explore how to use
shelve in serialization and persistence, including how to store and
retrieve Python objects using shelve.
Storing and Retrieving Python Objects using Shelve:
Shelve provides a simple way to store and retrieve Python objects
using a key-value store. The shelve module provides two main
classes for working with shelves: Shelf and DbfilenameShelf. Shelf is
a dictionary-like object that stores its data in memory, while
DbfilenameShelf is a subclass of Shelf that stores its data in a
persistent file on disk.
import shelve
# Create a new shelf

with shelve.open("my_shelf") as shelf:
# Store some data in the shelf
shelf["name"] = "John"
shelf["age"] = 30
shelf["city"] = "New York"
# Open the shelf again and retrieve the data

name = shelf["name"]
age = shelf["age"]
city = shelf["city"]
print(name, age, city)
In the code above, we create a new shelf using the shelve.open()

method and store some data in it using dictionary-like syntax. We
then close the shelf and reopen it to retrieve the data using the same
dictionary-like syntax.
Customizing Serialization using Shelve:
Shelve provides a way to customize the serialization and
deserialization of Python objects using the pickle module. The Shelf
and DbfilenameShelf classes both accept an optional protocol
argument that specifies the protocol version to use for pickling and
unpickling Python objects.
import shelve
import pickle
# Define a custom class to store in the shelf

class Person:
def __init__(self, name, age, city):
self.name = name
self.age = age
self.city = city
def __str__(self):
return f"{self.name} ({self.age}) from
{self.city}"
# Create a new shelf and set a custom

protocol for pickling
with shelve.open("my_shelf",
protocol=pickle.HIGHEST_PROTOCOL) as
shelf:
# Store a custom object in the shelf
person = Person("John", 30, "New York")
shelf["person"] = person
# Open the shelf again and retrieve the

custom object
person = shelf["person"]
print(person)
In the code above, we define a custom Person class and create a
new shelf with a custom protocol for pickling. We store a Person
object in the shelf and then retrieve it using the dictionary-like syntax.
The Person object is automatically deserialized to its original form.
Using shelve in serialization and persistence provides a simple way to
store and retrieve Python objects in a key-value store. By using the
shelve module in Python, we can easily store and retrieve data in a
persistent format, allowing us to maintain state across different
program executions. It's important to note that shelve uses pickle for
serialization, which can have security implications if untrusted data is
being stored or retrieved. It's recommended to only use shelve with
trusted data.
Using dbm
dbm (database manager) is a module in Python that provides a

simple way to store and retrieve key-value pairs in a persistent
format. In this note, we will explore how to use dbm in serialization
and persistence, including how to store and retrieve Python objects
using dbm.
Storing and Retrieving Key-Value Pairs using dbm:
dbm provides a simple way to store and retrieve key-value pairs
using a hash-based file format. The dbm module provides four main
classes for working with databases: dumbdbm, gdbm, ndbm, and
dbm.gnu. dumbdbm is the most basic implementation and is available
on all platforms. gdbm and ndbm provide more advanced features,
but are not available on all platforms. dbm.gnu is a more advanced
implementation that is available on most Unix-like systems.
import dbm
# Open a new database
with dbm.open("my_database", "c") as
database:
# Store some data in the database
database[b"name"] = b"John"
database[b"age"] = b"30"
database[b"city"] = b"New York"
# Open the database again and retrieve the

data
with dbm.open("my_database", "r") as
database:
name = database[b"name"]
age = database[b"age"]
city = database[b"city"]
print(name.decode(), age.decode(),
city.decode())
In the code above, we create a new database using the dbm.open()
method and store some data in it using bytes-like syntax. We then
close the database and reopen it to retrieve the data using the same
bytes-like syntax.
Customizing Serialization using dbm:
dbm provides a way to customize the serialization and deserialization
of Python objects using the pickle module. The dbm module provides
a open() method that accepts an optional pickle_protocol argument
that specifies the protocol version to use for pickling and unpickling
Python objects.
import dbm
import pickle
# Define a custom class to store in the

database
class Person:
def __init__(self, name, age, city):
self.name = name
self.age = age
self.city = city
def __str__(self):
return f"{self.name} ({self.age}) from
{self.city}"
# Open a new database and set a custom
protocol for pickling
with dbm.open("my_database", "c",
pickle_protocol=pickle.HIGHEST_PROTOCO
L) as database:
# Store a custom object in the database
person = Person("John", 30, "New York")
database[b"person"] =
pickle.dumps(person)
# Open the database again and retrieve the

custom object
with dbm.open("my_database", "r") as
database:
person =
pickle.loads(database[b"person"])
print(person)
In the code above, we define a custom Person class and create a
new database with a custom protocol for pickling. We store a Person
object in the database and then retrieve it using the pickle.loads()
method. The Person object is automatically deserialized to its original
form.
Using dbm in serialization and persistence provides a simple way to
store and retrieve key-value pairs in a persistent format. By using the
dbm module in Python, we can easily store and retrieve data in a
hash-based file format, allowing us to maintain state across different
program executions.
Using SQLite
SQLite is a lightweight relational database management system that

is included in Python's standard library. It provides a simple and
efficient way to store and retrieve data in a persistent format. In this
note, we will explore how to use SQLite in serialization and
persistence, including how to create and manipulate tables, and how
to store and retrieve data using SQL commands.
Creating a SQLite Database:
The first step in using SQLite is to create a new database. This can
be done using the sqlite3 module, which provides a simple way to
connect to a database and execute SQL commands.
import sqlite3
# Connect to a new database or open an

existing one
conn = sqlite3.connect("my_database.db")
# Create a new table

conn.execute(
"""
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY
AUTOINCREMENT,
name TEXT NOT NULL,
age INTEGER NOT NULL,
city TEXT NOT NULL
)
"""
)
# Commit the changes

conn.commit()
# Close the connection

conn.close()
In the code above, we create a new database using the
sqlite3.connect() method and execute an SQL command to create a
new table called users. We define three columns for the table,
including an id column that is automatically incremented, a name
column that must not be null, an age column that must not be null, and
a city column that must not be null.
Storing and Retrieving Data:
Once we have created a table, we can store and retrieve data using
SQL commands.
import sqlite3
# Connect to the database

conn = sqlite3.connect("my_database.db")
# Insert some data into the table
conn.execute(
"""
INSERT INTO users (name, age, city)
VALUES (?, ?, ?)
""",
("John", 30, "New York")
)
# Commit the changes

conn.commit()
# Retrieve the data from the table

cursor = conn.execute(
"""
SELECT id, name, age, city FROM users
"""
)
for row in cursor:
print(row)
# Close the connection

conn.close()
In the code above, we insert some data into the users table using an
SQL INSERT command. We then retrieve the data from the table
using an SQL SELECT command and print the results.
Using SQLite in serialization and persistence provides a simple way
to store and retrieve data in a persistent format. By using the sqlite3
module in Python, we can easily create and manipulate tables, and
store and retrieve data using SQL commands. It's important to note
that SQLite is not designed for high concurrency or high volume data
storage, but it is a useful tool for many small to medium-sized
applications.
Testing and Debugging

Writing unit tests
Unit testing is an essential part of software development. It involves

testing individual units or components of a software application to
ensure they work as expected. In this note, we will explore how to
write unit tests in Python using the built-in unittest module, including
how to write test cases, test fixtures, and assertions.
Writing Test Cases:
Test cases are the individual units of testing that make up a unit test
suite. They should test a single functionality of the software
component and be independent of other test cases. To write a test
case, we need to define a class that inherits from the
unittest.TestCase class and define one or more test methods.
import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('hello'.upper(), 'HELLO')
def test_isupper(self):
self.assertTrue('HELLO'.isupper())
self.assertFalse('Hello'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello',
'world'])
with self.assertRaises(TypeError):
s.split(2)
In the code above, we define a class called TestStringMethods that
inherits from unittest.TestCase. We define three test methods,
test_upper(), test_isupper(), and test_split(), each of which tests a
specific functionality of the str class.
Writing Test Fixtures:
Test fixtures are used to set up the environment for a test case or
clean up after a test case. We can define a test fixture by using the
setUp() and tearDown() methods of the TestCase class.
import unittest
def setUp(self):
self.test_string = 'hello world'
def tearDown(self):
self.test_string = None
self.assertEqual(self.test_string.split(),
['hello', 'world'])
In the code above, we define a test fixture using the setUp() method.
This method sets up a test_string variable that is used in the
test_split() method. We also define a tearDown() method that cleans
up the test_string variable after the test is complete.
Writing Assertions:
Assertions are used to verify that a test case has the expected result.
We can use various assertion methods provided by the TestCase
class to write assertions.
import unittest
def test_upper(self):
self.assertEqual('hello'.upper(), 'HELLO')
def test_isupper(self):
self.assertTrue('HELLO'.isupper())
self.assertFalse('Hello'.isupper())
s = 'hello world'
self.assertEqual(s.split(), ['hello',
'world'])
with self.assertRaises(TypeError):
s.split(2)
In the code above, we use the assertEqual() method to verify that the
output of hello.upper() is equal to 'HELLO'. We use the assertTrue()
method to verify that 'HELLO' is uppercase and the assertFalse()
method to verify that 'Hello' is not uppercase. We also use the
assertRaises() method to verify that a TypeError is raised when we
call the split() method with an argument.
Using pytest
Pytest is a powerful testing framework for Python that simplifies

writing and running tests. It supports a wide range of testing features
and integrates seamlessly with other testing tools. In this note, we
will explore how to use pytest to write and run tests in Python.
Installing Pytest:
Before we start writing tests, we need to install pytest. We can install
pytest using pip, the Python package installer, by running the
following command in our terminal:
pip install pytest

Writing Test Functions:
Pytest uses standard Python functions to define test cases. A test
function should start with test_ and should contain one or more
assertions to verify that the test has passed or failed. Here's an
example:
def test_addition():
assert 1 + 1 == 2
assert 2 + 2 == 4
In the code above, we define a test function called test_addition()
that contains two assertions. The first assertion verifies that 1 + 1 is
equal to 2, and the second assertion verifies that 2 + 2 is equal to 4.
Running Tests:
To run tests with pytest, we need to create a file with our test
functions and save it with a name starting with test_, for example,
test_example.py. We can then run our tests by simply running the
following command in our terminal:
pytest
Pytest will automatically discover and run all test functions in the file,
and display the results in the terminal.
Assertions:
Pytest provides a wide range of assertion functions that we can use
to verify that our tests have passed or failed. Here are some
examples:
def test_addition():
assert 1 + 1 == 2
assert abs(-1) == 1
assert 1.0 / 3.0 == pytest.approx(0.333,
abs=1e-3)
assert 'hello' in 'hello world'
assert [1, 2, 3] == [3, 2, 1][::-1]
assert {'a': 1, 'b': 2} == {'b': 2, 'a': 1}
In the code above, we use the assert keyword to make assertions
about the output of our code. We use the abs() function to get the
absolute value of a number, the approx() function to compare floating-
point numbers with a tolerance, and the in keyword to check if a
string is a substring of another string. We also use slicing to reverse
a list, and compare dictionaries for equality, ignoring the order of the
keys.
Fixtures:
Pytest also provides a powerful mechanism for setting up and tearing
down test fixtures. Fixtures are functions that provide a set of
preconditions for a test or a set of postconditions. Here's an example:
import pytest
@pytest.fixture
def example_list():
return [1, 2, 3]
def test_example(example_list):
assert sum(example_list) == 6
In the code above, we define a fixture function called example_list()
that returns a list containing the values 1, 2, and 3. We use the
@pytest.fixture decorator to mark this function as a fixture. We then
define a test function called test_example() that takes example_list as
an argument. The example_list argument is automatically injected into
the test function by pytest, and we can use it to make assertions
about the output of our code.
Debugging with pdb
Debugging with pdb (Python Debugger) is a built-in module in Python

that allows developers to inspect the execution of their code line by
line and identify any errors or bugs. In this subtopic, we will discuss
how to use pdb for debugging, and we will provide some sample
code to illustrate its usage.
To use pdb, you need to import the module and insert the
pdb.set_trace() function in your code where you want to start
debugging. This function will create a breakpoint in your code,
allowing you to interact with the debugger and inspect your program's
execution.
Here's an example code that contains a bug that we will debug using
pdb:
def factorial(n):
if n <= 0:
return 1
else:
return n * factorial(n - 1)
print(factorial(5))
print(factorial(-1))
The above code is a recursive implementation of the factorial
function. When we run the code, it will compute the factorial of 5
correctly but will raise an error when trying to compute the factorial of
-1. To debug this code, we will insert the pdb.set_trace() function in
the code and run it in the terminal.
import pdb
def factorial(n):
pdb.set_trace()
if n <= 0:
return 1
else:
print(factorial(5))
When we run the above code in the terminal, we will see that it stops
at the pdb.set_trace() function, and the debugger prompt ((Pdb))
appears. We can now interact with the debugger by entering different
commands to inspect the state of our program.
Some useful commands that we can use in pdb include:
n: execute the next line of code
c: continue execution until the next breakpoint
s: step into a function call
r: continue execution until the current function returns
q: quit debugging
We can also use p (print) command to print the value of any variable,
l (list) command to display the code surrounding the current line of
execution, and h (help) command to display the list of available pdb
commands.
Here's an example of how we can use pdb to debug our factorial
function:
> /path/to/file.py(5)factorial()
-> if n <= 0:
(Pdb) n
> /path/to/file.py(8)factorial()
-> return n * factorial(n - 1)
(Pdb) p n
-1
(Pdb) c
Traceback (most recent call last):
File "/path/to/file.py", line 10, in <module>
File "/path/to/file.py", line 8, in factorial
RuntimeError: maximum recursion depth
exceeded in comparison
In the above example, we executed the n command to move to the
next line of execution, then we executed the p n command to print the
value of n, which was -1. We then continued execution using the c
command, which caused the program to crash with a RuntimeError.
This error occurred because the factorial function called itself
recursively with a negative value, causing an infinite recursion loop.
pdb is a useful tool for debugging Python code. By inserting
breakpoints in your code and interacting with the debugger, you can
easily identify and fix errors in your program.
Debugging with logging
Debugging is an essential part of software development. When

building complex software, it is inevitable that bugs will be
encountered. One way to locate and fix bugs is by using logging.
Python has a built-in logging module that can be used to record
messages from an application.
The logging module is a versatile and powerful tool for debugging
code. It allows developers to log information about what the program
is doing while it's running. This information can then be used to trace
the flow of the program and locate any bugs.
The logging module has a hierarchy of logging levels. These levels
are used to determine the severity of the message being logged.
There are five built-in levels: DEBUG, INFO, WARNING, ERROR,
and CRITICAL. Messages at higher levels are more severe than
messages at lower levels.
To use the logging module, you first need to import it:
import logging
Then, you need to configure the logging system. This can be done
using the basicConfig() function. This function takes several
arguments, including the filename to use for the log file, the level of
logging to use, and the format of the log messages.
logging.basicConfig(filename='example.log',
level=logging.DEBUG, format='%(asctime)s
%(levelname)s %(message)s')
This example configuration sets up a log file called "example.log" and
sets the logging level to DEBUG. The format argument specifies the
format of the log messages. The '%(asctime)s' parameter will be
replaced with the current time, '%(levelname)s' will be replaced with
the logging level, and '%(message)s' will be replaced with the log
message.
To log a message, you simply call the logging function corresponding
to the desired logging level:
logging.debug('This is a debug message')

logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')
This will log a message with the corresponding logging level to the log
file.
Here is an example of using logging to debug a function:
def divide(a, b):

try:
result = a / b
except ZeroDivisionError:
logging.error("Tried to divide by zero")
return None
return result
print(divide(4, 2))
print(divide(4, 0))
This code defines a function that divides two numbers. If the second
number is zero, it logs an error message and returns None.
Otherwise, it returns the result of the division.
When run, this code produces the following output:
2.0
ERROR:root:Tried to divide by zero
None
The first call to divide() produces the expected result of 2.0.
However, the second call produces an error message and returns
None.
By using the logging module to log the error, we can track down the
source of the bug and fix it.
In summary, logging is a powerful tool for debugging Python code. By
using the logging module, you can log messages at various levels of
severity and trace the flow of your program. This can help you locate
and fix bugs in your code more quickly and efficiently.
Using assertions
Assertions are a way of verifying that a condition is true at a specific

point in the code. They can be a useful tool for testing and debugging
code, as they help identify and locate errors in the code. In Python,
assertions can be made using the assert keyword.
When an assertion is made, Python evaluates the expression and
raises an AssertionError exception if the expression is false. If the
expression is true, Python continues executing the code without
interruption.
Here is an example of how to use assertions in Python:
def divide(x, y):
assert y != 0, "Divisor cannot be zero"
return x / y
print(divide(10, 2)) # Output: 5.0

print(divide(10, 0)) # Raises an
AssertionError with message "Divisor cannot
be zero"
In this example, the divide() function takes two arguments x and y.
Before the division is performed, an assertion is made that the value
of y is not zero. If y is zero, an AssertionError is raised with the
message "Divisor cannot be zero". If y is not zero, the division is
performed and the result is returned.
Assertions can also be used in conjunction with unit tests to verify that
the code is working as expected. Here is an example of using
assertions in a unit test:
import unittest
class TestMath(unittest.TestCase):
def test_divide(self):
self.assertEqual(divide(10, 2), 5)
with self.assertRaises(AssertionError):
divide(10, 0)
if __name__ == '__main__':
unittest.main()
In this example, a unit test is defined for the divide() function. The test
checks that the result of dividing 10 by 2 is 5, and that an
AssertionError is raised when dividing by zero. The assertRaises()
method is used to check that the divide() function raises an
AssertionError when called with arguments (10, 0).
Assertions are a powerful tool for testing and debugging code, but
they should be used with caution. Assertions can be disabled globally
in the Python interpreter using the -O option, so they should not be
used to check for conditions that may occur in production code.
Instead, assertions should be used to check for programmer errors
that can be detected during development and testing.
Chapter 7:
Collaboration and Development
Python is a powerful programming language that has rapidly gained
popularity in the software development industry. It is an open-source
and high-level language that can be easily read and understood. The
language offers numerous advantages for developers such as
simplicity, versatility, and ease of use. Due to these reasons, Python
has become the go-to language for many developers across the
world.
Python is not only popular for its syntax and functionality but also for
its large community of developers who contribute to its growth and
development. Collaboration among developers is one of the essential
aspects of Python's success. Collaboration is an integral part of
software development as it allows developers to combine their skills
and knowledge to create better quality software.
Collaboration in Python can take many forms, such as open-source
projects, online communities, and team-based development. In this
chapter, we will discuss the different types of collaboration in Python
and how they contribute to the growth and development of the
language.
The first type of collaboration we will examine is open-source
projects. Open-source projects are software projects that are publicly
available and can be modified and distributed by anyone. Many of the
most popular Python libraries, such as NumPy, Pandas, and
Matplotlib, are open-source projects. These libraries are developed
and maintained by a community of developers who work together to
enhance the functionality and usability of the library. Open-source
projects are an excellent way for developers to collaborate, as they
allow developers from all over the world to contribute to the project
and improve it.
The second type of collaboration we will examine is online
communities. Online communities are forums or chat groups where
developers can come together to discuss Python-related topics, ask
for help, and share their knowledge. These communities are an
excellent way for developers to collaborate and learn from one
another. They provide a platform for developers to connect with like-
minded individuals and to receive support from the community when
they encounter challenges in their development projects.
The third type of collaboration we will examine is team-based
development. Team-based development involves developers working
together in a team to create software. This type of collaboration
requires communication, coordination, and a shared understanding of
the project goals. Team-based development is essential for large-
scale software projects as it allows developers to divide the workload
and work on different aspects of the project simultaneously.
Code Quality
Using linters
Linters are tools that analyze source code to flag programming

errors, bugs, and stylistic errors. They help improve code quality and
readability by enforcing coding standards and best practices. In
Python, one popular linter is pylint.
pylint can be installed using pip, and can be run on a Python module
or package like this:
pip install pylint

pylint mymodule.py
Here is an example of using pylint to check the quality of a Python
module:
# mymodule.py
def add_numbers(a, b):

# This is a comment that pylint will check
return a + b
When we run pylint mymodule.py, pylint will output a report of the
code quality issues it has found:
************* Module mymodule

mymodule.py:1:0: C0103: Module name
"mymodule" doesn't conform to snake_case
naming style (invalid-name)
mymodule.py:3:0: C0116: Missing function
or method docstring (missing-function-
docstring)
mymodule.py:4:4: W0105: String statement
has no effect (pointless-string-statement)
mymodule.py:4:4: C0304: Final newline
missing (missing-final-newline)
-----------------------------------
Your code has been rated at -7.50/10
The output shows that pylint has identified four issues in the code,
including a naming style issue, a missing docstring, a pointless string
statement, and a missing final newline. Each issue is accompanied by
a code violation message and a score, and the total score for the
code is reported at the end.
pylint can also be customized to enforce specific coding standards
and best practices. For example, we can create a .pylintrc file in the
project directory to specify the configuration settings for pylint. Here
is an example of a .pylintrc file that specifies a custom set of rules for
pylint:
[FORMAT]
max-line-length = 120
[BASIC]
indent-string = " "
[MESSAGES CONTROL]
disable = W0611
This .pylintrc file sets the maximum line length to 120 characters, the
indentation string to four spaces, and disables the "unused import"
warning. These settings will be used by pylint when analyzing the
code.
Using linters like pylint can help improve the quality of Python code by
enforcing coding standards and best practices, and can help identify
errors and bugs before they cause problems in production code. By
incorporating linters into the development workflow, developers can
catch errors earlier in the process, reducing the time and effort
required for testing and debugging.
Using type checkers
Type checkers are tools that analyze Python code to detect type-
related errors, and to ensure that the code is type-safe. They help
improve code quality by identifying potential bugs and errors, and by
enforcing strong typing in Python. One popular type checker for
Python is mypy.
mypy can be installed using pip, and can be run on a Python module
or package like this:
pip install mypy

mypy mymodule.py
Here is an example of using mypy to check the types of a Python

module:
# mymodule.py
def add_numbers(a: int, b: int) -> int:

return a + b
x: int = 5
y: str = "hello"
z = add_numbers(x, y)
When we run mypy mymodule.py, mypy will output a report of the
type-related issues it has found:
mymodule.py:6: error: Argument 2 to

"add_numbers" has incompatible type "str";
expected "int"
mymodule.py:6: note: Following overload(s)
are available
mymodule.py:6: note: def
add_numbers(a: int, b: int) -> int
mymodule.py:8: error: Incompatible types in
assignment (expression has type "Union[int,
str]", variable has type "int")
The output shows that mypy has identified two type-related issues in
the code. The first issue is that the argument y passed to
add_numbers has an incompatible type (str instead of int). The
second issue is that the variable z has an incompatible type
(Union[int, str] instead of int).
mypy can also be customized to enforce specific typing rules and
conventions. For example, we can create a mypy.ini file in the project
directory to specify the configuration settings for mypy. Here is an
example of a mypy.ini file that specifies a custom set of rules for
mypy:
[mypy]
python_version = 3.8
ignore_missing_imports = True
[strict_optional]
enabled = True
warn_return_any = True
This mypy.ini file sets the target Python version to 3.8, ignores
missing imports, and enables strict optional typing. These settings will
be used by mypy when analyzing the code.
Using type checkers like mypy can help improve the quality of Python
code by enforcing strong typing and identifying type-related errors
and bugs. By incorporating type checkers into the development
workflow, developers can catch errors earlier in the process,
reducing the time and effort required for testing and debugging.
Using code formatters
Code formatting is an essential aspect of software development.

Consistent code formatting helps in improving the readability of the
code and ensures that it adheres to a consistent style, making it
easier to maintain and debug.
Using code formatters is an effective way to ensure that the code is
formatted correctly. A code formatter is a tool that can automatically
format code according to specific rules and guidelines. This can save
time and effort in manually formatting code and ensure that the
codebase is consistent.
There are several popular code formatters available for Python,
including Black, YAPF, and autopep8. In this note, we'll explore Black
and demonstrate how to use it in a Python project.
Black is a code formatter that enforces a strict style guide for Python
code. It can automatically format code according to the PEP 8 style
guide and applies a set of opinionated rules for code layout and
formatting.
To use Black, first, you need to install it using pip:
pip install black

Once Black is installed, you can run it on your Python code files. For
example, to format a single file named example.py, run the following
command:
black example.py
This will format the code in the example.py file according to Black's
rules and save the changes to the file.
Black can also be used to format an entire project directory. To do
this, navigate to the root directory of the project and run the following
command:
black .
This will format all Python files in the project directory and its
subdirectories.
It's important to note that Black can modify your code files, so it's
recommended to commit your changes to version control before
running Black.
In addition to formatting code files, Black can also be integrated into
code editors and IDEs. For example, the Black extension for Visual
Studio Code automatically formats Python code using Black when you
save a file.
Using code formatters like Black can help ensure that your code
adheres to consistent formatting rules, improving the readability and
maintainability of your codebase.
Using docstring conventions
Documentation is an important aspect of software development, as it

helps developers understand how a piece of code works, its purpose,
and how to use it. Docstrings are a type of documentation that are
used to describe a function, method, or module in Python.
Docstrings should follow a consistent format and provide relevant
information about the code, such as the purpose of the function, the
arguments it accepts, and what it returns. There are several
conventions for writing docstrings in Python, including the Google
Style Guide, the numpydoc format, and the reStructuredText format.
Let's take a look at an example of a function with a docstring using
the Google Style Guide convention:
"""
Adds two numbers together.
Args:
a (int): The first number.
b (int): The second number.
Returns:
int: The sum of the two numbers.
"""
return a + b
In this example, the docstring describes what the function does, the
arguments it accepts, and what it returns. The arguments are listed
with their types and a brief description of what they represent. The
return value is also described with its type and a brief explanation of
what it represents.
Here's another example using the numpydoc format:
def multiply_numbers(a, b):

"""
Multiply two numbers.
Parameters
----------
a : int
The first number.
b : int
The second number.
Returns
-------
int
The product of the two numbers.
"""
return a * b
In this example, the numpydoc format is used, which is commonly
used in scientific computing projects. The arguments are listed using
the Parameters section, and the return value is described using the
Returns section.
Using consistent docstring conventions can help improve the
readability and maintainability of your codebase. It can also make it
easier for other developers to understand and use your code.
In addition to using docstring conventions, it's also important to
ensure that your docstrings are up to date and accurate. Docstrings
should be updated when the code changes or when new features are
added. By keeping your docstrings up to date, you can help ensure
that your code remains understandable and easy to use.
Writing maintainable code
Maintainable code is code that is easy to understand, modify, and

extend over time. Writing maintainable code is important for ensuring
that your codebase remains readable and maintainable as it grows in
size and complexity.
Here are some best practices for writing maintainable code:
Use clear and descriptive variable and function names.
# Bad
x=5
y = 10
z=x+y
# Good
num1 = 5
num2 = 10
sum_of_nums = num1 + num2
Write small, reusable functions that do one thing well.
# Bad
def process_data():
# some code here
if condition:
# some more code here
# some more code here
# Good
def validate_data(data):
# some code here
return valid_data
def process_valid_data(valid_data):
# some code here
return processed_data
def process_data(data):
valid_data = validate_data(data)
processed_data =
process_valid_data(valid_data)
return processed_data
Use comments to explain why code exists, not what it does.
# Bad
# Loop through list and print each item
print(item)
# Good
# Print each item in the list
print(item)
Write tests for your code to ensure that it works as intended.
# Bad
return a + b
# Good
return a + b
def test_add_numbers():
assert add_numbers(2, 3) == 5
assert add_numbers(0, 0) == 0
assert add_numbers(-1, 1) == 0
Follow consistent code formatting conventions to make code more
readable.
# Bad
def some_function():
print('hello')
return None
# Good
def some_function():
print('hello')
return None
By following these best practices, you can write code that is easier to
understand, modify, and extend over time. Writing maintainable code
is an important part of code quality, and can help ensure that your
codebase remains robust and reliable over time.
Code Reviews
Conducting effective code reviews
Code reviews are an essential part of the development process, as

they help identify potential issues and improve code quality.
Conducting effective code reviews involves a systematic and
collaborative process that allows developers to share feedback,
identify bugs and issues, and ensure that code adheres to standards
and best practices. In this note, we will discuss how to conduct
effective code reviews, including best practices and sample code.
Best Practices for Conducting Effective Code Reviews:
Establish clear goals and expectations: Before starting a
code review, it is essential to establish clear goals and
expectations for the process. This includes defining the
scope of the review, outlining the objectives, and providing
guidelines for feedback.
Assign roles and responsibilities: It is crucial to assign roles

and responsibilities for the code review process. This
includes identifying reviewers and providing clear guidelines
for their responsibilities and expectations.
Conduct a thorough review: When conducting a code

review, it is important to conduct a thorough review of the
code. This includes checking for adherence to best
practices, identifying potential issues, and ensuring that the
code meets the specified requirements.
Provide constructive feedback: Providing constructive
feedback is critical to conducting an effective code review.
Feedback should be specific, actionable, and focused on
improving the code quality and adhering to best practices.
Communicate effectively: Effective communication is

essential for conducting an effective code review. This
includes using clear and concise language, being respectful,
and providing feedback in a timely manner.
Use code review tools: There are many code review tools
available that can help facilitate the process. These tools
provide features such as code highlighting, commenting,
and issue tracking, which can help streamline the review
process and ensure that all feedback is captured.
Sample Code:
Let's consider an example of how to conduct a code review for a
Python script. Suppose we have the following script that calculates
the sum of two numbers:
def add_numbers(num1, num2):

sum = num1 + num2
return sum
result = add_numbers(5, 10)

print(result)
To conduct a code review, we can follow these steps:
Define the scope and objectives of the code review. In this case, we
want to ensure that the code adheres to best practices, is free of
potential issues, and meets the specified requirements.
Assign roles and responsibilities. We can assign one or more
reviewers to the code review process, providing clear guidelines for
their responsibilities and expectations.
Conduct a thorough review. The reviewer can check the following

aspects of the code:
Code structure: Ensure that the code is well-structured and
follows best practices for coding style and formatting.
Variable naming: Check that variable names are clear and

descriptive.
Comments: Ensure that code comments are used where

necessary to explain the code and its functionality.
Error handling: Check that appropriate error handling is

used, such as try-except blocks.
Provide constructive feedback. Based on the review, the reviewer can

provide feedback to the developer, such as:
Suggesting more descriptive function and variable names, such as
calculate_sum instead of add_numbers and first_num and
second_num instead of num1 and num2.
Suggesting adding comments to explain the code and its functionality.
Recommending using try-except blocks for error handling.
Communicate effectively. The reviewer can communicate the
feedback to the developer using clear and concise language,
providing examples and suggestions where necessary.
Use code review tools. Code review tools such as Github pull
requests, Bitbucket code reviews, and Review Board can be used to
facilitate the review process, providing a platform for reviewers to
comment, suggest changes and track issues.
Giving and receiving feedback
Giving and receiving feedback is an important aspect of collaboration

and development in software engineering. Providing constructive
feedback helps improve the quality of code and promotes personal
and professional growth.
Here are some tips for giving and receiving feedback:
Be specific: Avoid general comments like "this code is bad".
Instead, point out specific issues and suggest solutions. For
example, "the variable name x is not descriptive. Can you
rename it to something more meaningful like total_sales?"
Be objective: Critique the code, not the person. Avoid using

language that can be perceived as personal attacks.
Be respectful: Choose your words carefully and be mindful

of your tone. Make sure your feedback is delivered in a
respectful and professional manner.
Be actionable: Suggest actionable steps to improve the
code. For example, "Can you add comments to explain the
purpose of this function?" or "Can you reformat the code to
conform to the style guide?".
Receiving feedback can be challenging, but it is essential

for growth and development. Here are some tips for
receiving feedback:
Listen actively: Listen carefully to the feedback and try to

understand the other person's perspective.
Ask questions: If you are not sure about something, ask for
clarification. This shows that you are willing to learn and
improve.
Don't take it personally: Remember that the feedback is
about the code, not you as a person.
Be open-minded: Be receptive to new ideas and

suggestions for improvement. Don't be defensive.
Sample code for giving feedback:
def calculate_sales(data):
"""
This function calculates the total sales
from a list of transactions.
"""
total = 0
for transaction in data:
total += transaction['amount']
return total
Example of feedback:
The function logic looks good, but the docstring could be improved.
Can you add more details on the input and output of the function?
Also, can you format it to follow the Google docstring convention?
Sample code for receiving feedback:
def calculate_sales(data):
"""
This function calculates the total sales
from a list of transactions.
"""
total = 0
for transaction in data:
total += transaction['amount']
return total
Improving code quality through reviews
Code reviews are an important part of the software development

process that help improve code quality, ensure adherence to coding
standards, and identify and fix bugs early on. In this subtopic, we will
discuss how to improve code quality through reviews, and provide
sample codes to illustrate the concepts.
1. Code Review Checklist
To conduct an effective code review, it is important to have a
checklist of items to look for. Here are some common items to include
in your code review checklist:
Code formatting: Is the code consistent and easy to read?
Naming conventions: Are variables, functions, and classes
named descriptively?
Comments and docstrings: Are there enough comments and
are they helpful in understanding the code?
Code functionality: Does the code achieve what it's
supposed to do?
Error handling: Does the code handle errors and edge
cases appropriately?
Security: Is the code secure and resistant to attacks?
Performance: Does the code perform efficiently and not
have any bottlenecks?
Testing: Are there enough tests and are they
comprehensive enough to cover different scenarios?
2. Sample Code Review
Let's consider an example of a Python function that accepts a list of
numbers and returns the sum of all even numbers in the list. Here is
the original implementation:
def sum_even_numbers(numbers):
result = 0
for number in numbers:
if number % 2 == 0:
result += number
return result
Here are some possible suggestions to improve this code based on
the code review checklist:
Code formatting: The code is well-formatted and easy to
read.
Naming conventions: The function name and variable names
are descriptive.
Comments and docstrings: There is no docstring explaining
the purpose of the function, which could be helpful for future
maintenance.
Code functionality: The code correctly sums even numbers.
Error handling: The code assumes that the input is a list of
integers, and will raise an exception if it is not. It may be
useful to add a check to handle this case more gracefully.
Security: There are no security concerns with this code.
Performance: The code is efficient and does not have any
obvious performance issues.
Testing: There are no tests included with this code.
Based on this code review, we can make the following modifications

to the code:
def sum_even_numbers(numbers):
"""
Returns the sum of all even numbers in a
list of integers.
Args:
numbers (list): A list of integers.
Returns:
int: The sum of all even numbers in the
list.
Raises:
TypeError: If the input is not a list of
integers.
"""
if not isinstance(numbers, list) or not
all(isinstance(x, int) for x in numbers):
raise TypeError("Input must be a list of
integers.")
result = 0
for number in numbers:
if number % 2 == 0:
result += number
return result
In this modified code, we have added a docstring explaining the
purpose of the function, and included type checking to ensure that the
input is a list of integers. We have also raised a specific exception to
handle this case more gracefully. Finally, we have added type
annotations to the function signature to make it more clear what the
function expects as input and returns as output.
Code reviews are an important part of the software development
process and can help improve code quality and identify potential
issues early on. By using a code review checklist and providing
constructive feedback, you can ensure that the code being produced
meets the necessary standards and is of high quality.
Collaboration Tools
Using version control with Git
Version control is an essential aspect of software development,

especially in collaboration projects. It enables developers to track
changes to the source code, collaborate with other developers, and
revert to previous versions if necessary. Git is one of the most
popular version control systems and is widely used in collaboration
tools. In this note, we will explore how to use Git in collaboration
tools.
Git is a distributed version control system, which means that each
developer has their copy of the repository. This makes it easy for
developers to work on different parts of the codebase without
affecting others' work. When using Git in collaboration tools, it is
essential to follow a few best practices to ensure that the workflow is
smooth and efficient.
Here are some of the best practices for using Git in collaboration
tools:
Create a Git repository: The first step is to create a Git repository
that will serve as the central repository for the project. You can
create a Git repository on GitHub, Bitbucket, or any other Git hosting
service. Once you have created the repository, you can clone it to
your local machine.
Create branches: In Git, branches are used to isolate changes and
work on specific features or bug fixes. Each developer should create
their branch when working on a new feature or bug fix. This allows
them to work independently without interfering with others' work.
# create a new branch

git checkout -b new-feature
Commit changes: After making changes to the code, developers
should commit their changes to their local repository. It is important to
write a descriptive commit message that explains the changes made.
# stage changes
git add .
# commit changes
git commit -m "added new feature"
Push changes: Once the changes are committed, they can be pushed
to the central repository to make them available to other developers.
# push changes to remote branch

git push origin new-feature
Pull changes: To get the latest changes from the central repository,
developers should pull changes from the remote repository before
making changes to their local repository.
# pull changes from remote branch

git pull origin master
Resolve conflicts: When multiple developers work on the same file,
conflicts can arise when they try to push changes to the central
repository. To resolve conflicts, developers should pull changes from
the remote repository, merge the changes, and resolve any conflicts.
# merge changes from master branch

git merge master
# resolve conflicts
Review changes: Before merging changes to the master branch, they
should be reviewed by other developers. This ensures that the
changes do not break the code and meet the project's requirements.
# create a pull request
# review changes and merge pull request

Git is an essential tool for collaboration in software development
projects. By following these best practices, developers can work
together efficiently and effectively to build high-quality software.
Using GitHub for collaboration
GitHub is one of the most popular collaboration tools for software

development. It provides a platform for version control, project
management, and team collaboration. In this note, we will explore
how to use GitHub for collaboration in collaboration tools.
GitHub provides several features that make collaboration easy and
efficient. Here are some of the features that make GitHub a great
collaboration tool:
Pull requests: GitHub's pull request feature enables developers to
review and merge changes from other developers. Pull requests
provide a platform for discussing code changes, suggesting
improvements, and resolving conflicts before merging changes into
the main codebase.
Issues: GitHub's issue tracking system provides a way for developers
to track and manage bugs, feature requests, and other tasks. Issues
can be assigned to specific team members, labeled, and prioritized to
ensure that they are addressed in a timely and efficient manner.
Wiki: GitHub's wiki feature provides a platform for documenting
project requirements, processes, and best practices. This enables
team members to have a shared understanding of the project,
reducing misunderstandings and improving collaboration.
Milestones: GitHub's milestone feature provides a way to group
issues and pull requests by a specific deadline or release. This
enables teams to track progress and ensure that project milestones
are met on time.
Here are some sample codes for using GitHub for collaboration:
Creating a pull request:
To create a pull request, first, fork the repository, clone it to your
local machine, and create a new branch:
# fork the repository on GitHub

# clone the repository to your local machine
git clone https://github.com/your-
username/repository-name.git
# create a new branch

git checkout -b new-feature
Make changes to the code, commit the changes, and push the
changes to your forked repository:
# stage changes
git add .
# commit changes
git commit -m "added new feature"
# push changes to your forked repository

Create a pull request from your forked repository to the original

repository:
# go to your forked repository on GitHub

# click on "New pull request"
# select the branch you want to merge into
the original repository
# add a description of the changes
# click on "Create pull request"
Creating an issue:
To create an issue, go to the "Issues" tab in the repository and click
on "New issue." Add a title and description of the issue, assign it to a
team member, and add any necessary labels and milestones.
Creating a wiki:
To create a wiki, go to the "Wiki" tab in the repository and click on
"New page." Add a title and content for the page, and save it.
Creating a milestone:
To create a milestone, go to the "Issues" tab in the repository and
click on "Milestones." Click on "New milestone," add a title, due date,
and description, and save it.
GitHub is an excellent collaboration tool for software development
projects. Its features such as pull requests, issues, wiki, and
milestones make collaboration efficient and effective. By following
these best practices, developers can work together to build high-
quality software.
Using continuous integration
Continuous Integration (CI) is a software development practice that

involves continuously integrating code changes into a shared
repository and testing them automatically. CI is essential for
collaborative software development as it enables teams to catch
errors early in the development process and ensure that the
codebase is always in a releasable state. In this note, we will explore
how to use continuous integration in collaboration tools with sample
codes.
There are several collaboration tools that support continuous
integration, such as Jenkins, Travis CI, CircleCI, and GitHub Actions.
Here are some of the benefits of using continuous integration in
collaboration tools:
Early detection of errors: Continuous integration enables teams to
detect errors early in the development process, reducing the time and
cost of fixing them.
Faster release cycles: Continuous integration enables teams to
release software faster and more frequently, reducing time-to-market
and increasing customer satisfaction.
Better code quality: Continuous integration ensures that code
changes are tested automatically, reducing the risk of introducing
bugs and improving overall code quality.
Here are some sample codes for using continuous integration in
Jenkins:
Jenkins is an open-source automation server that supports continuous
integration. Here's an example of a Jenkinsfile for a Java project:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
}
stage('Deploy') {
steps {
sh 'mvn deploy'
}
}
}
}
This Jenkinsfile defines a pipeline that builds, tests, and deploys a
Java project. Each stage corresponds to a step in the software
development lifecycle, and the sh command executes shell
commands.
GitHub Actions:
GitHub Actions is a native continuous integration service built into
GitHub. Here's an example of a GitHub Actions workflow for a
Node.js project:
name: Node.js CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Use Node.js
uses: actions/setup-node@v2
with:
node-version: '14.x'
- run: npm ci
- run: npm run build
- run: npm test
This workflow defines a job that builds, tests, and deploys a Node.js
project. The steps correspond to a sequence of tasks, and the uses
command specifies the dependencies required for each step.
Continuous integration is a critical component of collaborative
software development. Collaboration tools such as Jenkins and
GitHub Actions provide powerful tools for implementing continuous
integration, making it easy for teams to catch errors early, release
software faster, and improve overall code quality. By following these
best practices, developers can work together to build high-quality
software.
Using code coverage tools
Code coverage is a metric that measures how much of your source

code is executed during testing. Code coverage tools help teams
identify areas of the codebase that are not being tested, ensuring
that software is thoroughly tested before release. In this note, we will
explore how to use code coverage tools in collaboration tools with
sample codes.
There are several code coverage tools that support collaborative
software development, such as Jacoco, Istanbul, and Coveralls. Here
are some of the benefits of using code coverage tools in collaboration
tools:
Identify untested code: Code coverage tools enable teams to identify
areas of the codebase that are not being tested, ensuring that
software is thoroughly tested before release.
Improve code quality: Code coverage tools encourage teams to write
more testable code and improve overall code quality.
Increase confidence in software: Code coverage tools provide teams
with confidence that their software is thoroughly tested and ready for
release.
Here are some sample codes for using code coverage tools in
Jacoco:
Jacoco is a Java code coverage tool that supports collaborative
software development. Here's an example of how to configure
Jacoco in a Maven project:
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-
plugin</artifactId>
<version>0.8.7</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
This configuration sets up Jacoco in a Maven project, preparing the
agent for testing and generating a report after the test phase.
Istanbul:
Istanbul is a JavaScript code coverage tool that supports
collaborative software development. Here's an example of how to
configure Istanbul in a Node.js project:
"scripts": {
"test": "istanbul cover
./node_modules/mocha/bin/_mocha --report
lcovonly -- -R spec && cat
./coverage/lcov.info |
./node_modules/coveralls/bin/coveralls.js &&
rm -rf ./coverage"
}
This configuration sets up Istanbul in a Node.js project, generating a
coverage report in the lcov format and sending it to Coveralls, a code
coverage service.
Code coverage tools are a critical component of collaborative
software development. Collaboration tools such as Jacoco and
Istanbul provide powerful tools for implementing code coverage,
making it easy for teams to identify untested code, improve overall
code quality, and increase confidence in software. By following these
best practices, developers can work together to build high-quality
software.
Documentation and Packaging

Writing documentation
Documentation is an essential aspect of software development that

helps ensure that code is maintainable and understandable by others.
Collaborative software development requires documentation that can
be shared across teams to help reduce the knowledge gap between
team members. In this note, we will explore how to write
documentation in collaboration and development with sample codes.
There are different types of documentation, including functional
specifications, technical specifications, user manuals, and API
documentation. Here are some best practices for writing
documentation in collaboration and development:
Start early: Documentation should be written early in the development
process, ideally at the same time as code. This ensures that the
documentation is accurate and up-to-date.
Use plain language: Use simple and concise language to make the
documentation easy to understand. Avoid using technical jargon that
may not be familiar to all team members.
Organize the documentation: Organize the documentation in a logical
and easy-to-follow structure. Use headings and subheadings to break
up the content, making it easier to read and navigate.
Use examples: Use examples to illustrate the concepts and
functionality of the software. This helps to make the documentation
more accessible and understandable.
Here are some sample codes for writing documentation in
collaboration and development:
Functional specifications:
Functional specifications describe the features and functionality of the
software from the user's perspective. Here is an example of a
functional specification for a shopping cart feature:
Feature: Add items to shopping cart
Scenario: User adds an item to the shopping cart
Given the user is on the product page
When the user clicks the 'Add to Cart' button
Then the item is added to the shopping cart
And the total price is updated
This example uses the Gherkin language to describe the shopping
cart feature from a user's perspective.
Technical specifications:
Technical specifications describe how the software works and the
technologies used to build it. Here's an example of a technical
specification for a Node.js application:
Architecture: Node.js and Express.js
Database: MongoDB
Deployment: Heroku
API Endpoints:
GET /products - Get all products
POST /products - Create a new product
GET /products/:id - Get a product by ID
PUT /products/:id - Update a product by ID
DELETE /products/:id - Delete a product by ID
This example describes the technical aspects of a Node.js
application, including the architecture, database, and deployment. It
also includes the API endpoints and their corresponding HTTP
methods.
User manuals:
User manuals provide instructions on how to use the software from a
user's perspective. Here is an example of a user manual for a web
application:
Getting started:
1. Go to the web application URL
2. Click the 'Sign up' button to create a new account
3. Follow the instructions to create your account
4. Log in to your account
Creating a new project:
1. Click the 'New project' button
2. Enter the project name and description
3. Click the 'Create' button
Adding tasks to a project:
1. Click on the project name
2. Click the 'Add task' button
3. Enter the task details
4. Click the 'Save' button
This example provides instructions on how to get started with the web
application, create a new project, and add tasks to a project.
Documentation is an essential aspect of collaborative software
development. By following best practices and using the appropriate
tools, teams can create documentation that is accurate, easy to
understand, and accessible to all team members. Whether it is
functional specifications, technical specifications, or user manuals,
documentation helps to reduce the knowledge gap between team
members and ensures that the software is maintainable and
understandable.
Using Sphinx
Sphinx is a documentation tool that can be used to generate high-

quality documentation for software projects. It is widely used in
collaborative software development to document Python-based
projects, but it can also be used to document projects written in other
programming languages. In this note, we will explore how to use
Sphinx in documentation and packaging in collaboration and
development with sample codes.
Sphinx uses a markup language called reStructuredText (reST) to
write documentation. It is similar to Markdown, but it is more powerful
and flexible. Here are some best practices for using Sphinx in
documentation and packaging:
Use Sphinx to generate HTML and PDF documentation: Sphinx can
be used to generate documentation in different formats, including
HTML and PDF. The HTML format can be hosted on a website or
added to a project's documentation directory, while the PDF format
can be distributed as a standalone document.
Use reStructuredText to write documentation: Sphinx uses
reStructuredText to write documentation, which is a markup language
that is easy to read and write. It supports syntax highlighting, code
blocks, and hyperlinks.
Use Sphinx themes to customize the documentation: Sphinx comes
with several built-in themes that can be used to customize the
appearance of the documentation. The themes can be customized
further by using CSS stylesheets.
Use Sphinx to generate package documentation: Sphinx can be used
to generate documentation for Python packages. The documentation
can be included in the package and installed along with it.
Here are some sample codes for using Sphinx in documentation and
packaging:
Install Sphinx:
To install Sphinx, run the following command:
pip install sphinx

Create a documentation directory:
Create a directory for the documentation. In this example, we will call
it "docs".
mkdir docs
Initialize the documentation:
Change to the "docs" directory and run the following command to
initialize the documentation:
sphinx-quickstart
This will prompt you to enter some information about the project, such
as the project name, author, and version.
Write documentation using reStructuredText:
Create a file called "index.rst" in the "docs" directory and add the
following content:
My Project
==========
This is the documentation for My Project.
Installation
------------
To install My Project, run the following command:
.. code-block:: console
$ pip install myproject
Usage
-----
To use My Project, import the following module:
.. code-block:: python
import myproject
myproject.do_something()
Generate HTML documentation:
To generate HTML documentation, run the following command:
make html
This will create an HTML documentation directory in the
"docs/_build/html" directory.
Generate PDF documentation:
To generate PDF documentation, run the following command:
make latexpdf
This will create a PDF documentation file in the "docs/_build/latex"
directory.
Generate package documentation:

To generate documentation for a Python package, add the following
code to the "setup.py" file:
from setuptools import setup

from sphinx.setup_command import
BuildDoc
setup(
name='myproject',
version='1.0',
cmdclass={
'build_sphinx': BuildDoc,
'install_sphinx': BuildDoc,
},
command_options={
'build_sphinx': {
'project': ('setup.py', 'My Project'),
'version': ('setup.py', '1.0'),
'release': ('setup.py', '1.0.0'),
'source_dir': ('setup.py', 'docs'),
'build_dir': ('setup.py', 'docs/_build'),
},
'install
Packaging Python projects
Packaging Python projects is an essential step in software

development, as it enables easy distribution and installation of the
code. In this note, we will explore how to package Python projects in
documentation and packaging in collaboration and development, with
sample codes.
Python packages can be distributed in two main ways: as source
distributions or as binary distributions. Source distributions contain the
source code and any necessary files, such as configuration files or
documentation. Binary distributions contain compiled code and can be
installed directly on the target machine. In this note, we will focus on
creating source distributions.
Here are the steps to package a Python project:

Create a setup.py file: The setup.py file contains the metadata of the
project, such as the name, version, description, and dependencies. It
also contains the instructions to build and distribute the package.
Here is a sample setup.py file:
from setuptools import setup, find_packages
setup(
name='myproject',
version='0.1.0',
description='My project description',
author='John Doe',
author_email='john.doe@example.com',
packages=find_packages(),
install_requires=[
'numpy>=1.18.1',
'matplotlib>=3.2.0',
],
)
Create a MANIFEST.in file: The MANIFEST.in file specifies the files
that should be included in the source distribution. It can include files
such as README, LICENSE, or data files. Here is a sample
MANIFEST.in file:
include README.md
include LICENSE.txt
recursive-include myproject/data *
Build the source distribution: To build the source distribution, run the
following command in the project directory:
python setup.py sdist
This will create a source distribution file in the "dist" directory.

Install the package: To install the package, run the following
command:
pip install dist/myproject-0.1.0.tar.gz

This will install the package and its dependencies.
Upload the package to PyPI: PyPI is the Python Package Index,
where Python packages are hosted. To upload the package to PyPI,
you need to create an account and use a package manager such as
twine. Here are the steps to upload the package:
Create an account on PyPI (https://pypi.org/account/register/)
Install twine:
pip install twine

Upload the package:
twine upload dist/*

This will upload the package to PyPI and make it available to other
users.
Here are some best practices for packaging Python projects:
Use setuptools: Setuptools is a package that provides extensions to
the Python distutils. It simplifies the packaging process and provides
useful features such as dependency management.
Use version control: Version control systems such as Git or SVN
enable you to track changes to your code and collaborate with other
developers. They also enable you to create tags and releases for
your packages.
Include documentation: Documentation is essential for users to
understand how to use your package. Sphinx is a popular
documentation tool that can be used to generate high-quality
documentation.
Use a virtual environment: Virtual environments enable you to create
isolated Python environments for your project. They prevent conflicts
between different versions of packages and ensure that your project
works consistently across different machines.
Use a consistent naming convention: Use a consistent naming
convention for your package, module, and function names. This
makes it easier for users to understand your code and prevents
naming conflicts with other packages.
Packaging Python projects is an essential step in software
development. By following the best practices and using the tools
mentioned above, you can create high-quality packages that are easy
to distribute and install.
Distributing Python packages
Distributing Python packages is an essential part of Python

development, and it is critical to ensure that your package is easy to
install and use. In this note, we will discuss distributing Python
packages in documentation and packaging in collaboration and
development.
Documentation is an integral part of any package, and it plays a vital
role in the distribution process. The documentation should be concise,
clear, and easy to read, and it should provide all the necessary
information about the package, including installation instructions, API
documentation, and usage examples. One common tool used to
generate documentation is Sphinx.
Packaging is the process of creating a distribution package that can
be installed using standard Python tools like pip. Python packages
can be distributed in two ways: source distributions (sdist) or binary
distributions (bdist). Source distributions contain the package's
source code, whereas binary distributions contain pre-compiled code
that can be installed directly on the target system.
To distribute your package, you need to create a package
configuration file, setup.py, which is used to generate the distribution
package. Here's an example of a setup.py file:
setup(
name='my_package',
version='1.0.0',
author='John Doe',
description='My Python package',
long_description='A longer description of
my Python package',
install_requires=[
'numpy>=1.0.0',
'scipy>=1.0.0',
],
classifiers=[
'Development Status :: 5 -
Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3',
'Programming Language :: Python ::
3.6',
3.7',
3.8',
3.9',
],
)
In this example, we are using setuptools to define our package's
metadata and dependencies. We also specify the required packages
using find_packages(), which automatically discovers all packages in
the project.
Once you have created your setup.py file, you can generate a
distribution package by running the following command:
python setup.py sdist bdist_wheel
This command generates both a source distribution (sdist) and a
binary distribution (bdist_wheel). The resulting files can be uploaded
to a package repository such as PyPI for distribution.
Collaboration is essential in software development, and Python
makes it easy to collaborate with others. One popular tool for
collaboration is Git, which allows multiple developers to work on the
same codebase simultaneously. To collaborate on a Python project,
you can use a Git repository hosting service like GitHub or GitLab.
When collaborating on a Python project, it is essential to maintain a
consistent coding style and follow best practices. Tools like Flake8
and Black can help enforce coding standards and maintain
consistency across the project.
Distributing Python packages is an essential part of Python
development, and it is crucial to ensure that your package is easy to
install and use. Documentation, packaging, and collaboration are
critical components of this process, and Python provides powerful
tools to help with each of these tasks.
Managing dependencies
Managing dependencies is an essential aspect of Python

development, and it involves ensuring that your package can work
correctly with other packages it relies on. In this note, we will discuss
managing dependencies in documentation and packaging in
collaboration and development.
Documentation is an essential part of any package, and it plays a
vital role in the distribution process. In the context of managing
dependencies, documentation should include a clear list of all the
dependencies required to use the package. This includes any third-
party packages that the package depends on, as well as any specific
versions required for compatibility. The documentation should also
provide instructions on how to install these dependencies using
package managers like pip.
To manage dependencies in your package, you can use a tool like
pipenv. Pipenv is a package manager that combines the functionality
of pip and virtualenv into a single tool. It provides an easy way to
manage dependencies and virtual environments for Python projects.
Here is an example of a Pipfile created using pipenv:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
requests = "==2.25.1"
numpy = "==1.19.5"
pandas = "==1.2.1"
[dev-packages]
pytest = "==6.2.2"
flake8 = "==3.9.0"
In this example, we have specified the required packages in the
[packages] section and the development packages in the [dev-
packages] section. We have also specified the specific versions
required using the == operator.
To install the dependencies listed in the Pipfile, you can run the
following command:
pipenv install
This command creates a virtual environment and installs all the
required packages specified in the Pipfile.
Packaging is the process of creating a distribution package that can
be installed using standard Python tools like pip. When creating a
package, it is essential to ensure that all the required dependencies
are included in the package. This can be achieved using tools like
setuptools, which automatically include all required packages in the
distribution package.
Here's an example of a setup.py file that uses setuptools to specify
the required packages:
setup(
name='my_package',
version='1.0.0',
author='John Doe',
description='My Python package',
long_description='A longer description of
my Python package',
install_requires=[
'numpy>=1.0.0',
'scipy>=1.0.0',
],
classifiers=[
'Development Status :: 5 -
Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3',
3.6',
3.7',
3.8',
3.9',
],
)
In this example, we have specified the required packages using the
install_requires argument, which automatically includes these
packages in the distribution package.
Collaboration is essential in software development, and Python
makes it easy to collaborate with others. When collaborating on a
Python project, it is essential to ensure that all team members are
using the same dependencies and versions. This can be achieved
using tools like pipenv or conda, which provide a consistent
environment for all team members.
THE END

Mastering Python 50 Specific Tips For Writing Better Code

Uploaded by

Mastering Python 50 Specific Tips For Writing Better Code

Uploaded by

Mastering Python: 50 Specific Tips

for Writing Better Code

Copyright © 2023 Ziyob Publishers

2. Write expressive code

Choosing good names

3. Take advantage of Python's features

4. Writing idiomatic Python

Writing Pythonic loops

Function arguments and return values

Writing pure functions

3. Function decorators and closures

Writing simple decorators

Writing clean, readable classes

3. Advanced class topics

Understanding the Global Interpreter Lock (GIL)

2. Coroutines and asyncio

3. File and Directory Access

4. Dates and Times

5. Serialization and Persistence

Writing unit tests

Using version control with Git

4. Documentation and Packaging

The Zen of Python

"Explicit is better than implicit": This principle encourages

"Simple is better than complex": This principle encourages

"Readability counts": This principle emphasizes the importance

Let's take a look at some sample code that demonstrates the

# Example 1: Beautiful is better than ugly.

# Using for loop to create a new list

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

# Using built-in function sum() to sum a list

# Using list comprehension to create a list of

# Using try/finally block to manage file

# Using the built-in module datetime to work

Python is a popular high-level programming language known for its

Know your data structures

In Pythonic thinking, it is essential to know the data structures

# Creating a tuple using parentheses

# Accessing tuple elements

# Creating a list using square brackets

# Accessing list elements

# Modifying list elements

# Adding elements to a list

# Removing elements from a list

In Pythonic thinking, it is essential to know the available data

# Creating a dictionary using curly braces

# Accessing dictionary elements

# Modifying dictionary elements

Adding elements to a dictionary:

# Adding elements to a dictionary

# Removing elements from a dictionary

# Checking if a key exists in a dictionary

In Pythonic thinking, it is important to know the available data

# Creating a set using curly braces

# Accessing set elements

Modifying set elements:

# Modifying set elements

# Checking if a set is a subset or superset

Removing duplicate elements:

# Removing duplicate elements from a list

In Pythonic thinking, it is important to know the available data

import array as arr

# Accessing array elements

# Modifying array elements

# Converting arrays to lists

Queues are a fundamental data structure in computer science that

from collections import deque

# Adding elements to a queue

Removing elements from a queue:

# Removing elements from a queue

# Checking the size of a queue

Stacks are a fundamental data structure in computer science that are

# Adding elements to a stack

# Removing elements from a stack

# Checking the size of a stack

Heaps are a fundamental data structure in computer science that are