Python Programming

Course Code : CSE3011

Course Credit : 3
Course Type : LP

Dr. Trapti sharma

Module 4
• File Handling, Writing Data to a File, Reading Data From a File
• Additional File Methods: Using Pipes as Data Streams
• Handling IO Exceptions,
• Working with Directories,
• Metadata,
• File Organization,
• Database Programming - Generic Database
• Connectivity using ODBC,
Database Programming
• Postgres connection in Python,
• MySQL connection in Python.
File Handling
• File handling in Python is a powerful and versatile tool that can be used to perform a wide range of
operations. However, it is important to carefully consider the advantages and disadvantages of file handling
when writing Python programs, to ensure that the code is secure, reliable, and performs well.

• Python supports file handling and allows users to handle files i.e., to read and write files, along with many
other file handling options, to operate on files. The concept of file handling has stretched over various other
languages, but the implementation is either complicated or lengthy, like other concepts of Python, this
concept here is also easy and short.

• Python treats files differently as text or binary and this is important. Each line of code includes a sequence of
characters, and they form a text file. Each line of a file is terminated with a special character, called the EOL or
End of Line characters like comma {,} or newline character. It ends the current line and tells the interpreter a
new one has begun.
Python has several functions for creating, reading, updating, and deleting files.

File Handling
• The key function for working with files in Python is the open() function.

• The open() function takes two parameters; filename, and mode.

There are four different methods (modes) for opening a file:

In addition you can specify if the file should be handled as binary or text mode
Syntax
To open a file for reading it is enough to specify the name of the file:

The code above is the same as:

Because "r" for read, and "t" for text are the default values, you do not need to specify them.

Note: Make sure the file exists, or else you will get an error.
Open a File on the Server
Assume we have the following file, located in the same folder as Python:

[Link]

To open the file, use the built-in open() function.

The open() function returns a file object, which has a read() method for reading the content of the file:

If the file is located in a different location, you will have to specify the file path, like this:
Welcome to this text file!
Example
This file is located in a folder named "myfiles", on the D drive.
Open a file on a different location:
Good Luck!
Read Only Parts of the File
By default the read() method returns the whole text, but you can also specify how many characters you want to return:

Example
Return the 5 first characters of the file:

Hello

Read Lines
You can return one line by using the readline() method:

Example
Read one line of the file:

Hello! Welcome to [Link]

By calling readline() two times, you can read the two first lines:

Example
Read two lines of the file:

Hello! Welcome to [Link]

This file is for testing purposes.

By looping through the lines of the file, you can read the whole file, line by line:

Example
Loop through the file line by line:

Hello! Welcome to [Link]

This file is for testing purposes.
Good Luck!
Close Files
It is a good practice to always close the file when you are done with it.

Example
Close the file when you are finish with it:

Hello! Welcome to [Link]

Note: You should always close your files, in some cases, due to buffering, changes made to a file may not
show until you close the file.
Write to an Existing File
To write to an existing file, you must add a parameter to the open() function:

• "a" - Append - will append to the end of the file

• "w" - Write - will overwrite any existing content

Open the file "[Link]" and append content to the file:

Hello! Welcome to [Link]

This file is for testing purposes.
Good Luck! Now the file has more content!
 Example
Open the file "[Link]" and overwrite the content:

Woops! I have deleted the content!

Note: the "w" method will overwrite the entire file.

Create a New File

To create a new file in Python, use the open() method, with one of the following parameters:

• "x" - Create - will create a file, returns an error if the file exist

• "a" - Append - will create a file if the specified file does not exist

• "w" - Write - will create a file if the specified file does not exist
Example
Create a file called "[Link]":

Result: a new empty file is created!

Example
Create a new file if it does not exist:

Delete a File
To delete a file, you must import the OS module, and run its [Link]() function:
Example
Remove the file "[Link]":
Check if File exist:
To avoid getting an error, you might want to check if the file exists before you try to delete it:

Example
Check if file exists, then delete it:

Delete Folder
To delete an entire folder, use the [Link]() method:

Example
Remove the folder "myfolder":

Note: You can only remove empty folders.

Advantages of File Handling in Python

• Versatility: File handling in Python allows you to perform a wide range of operations, such as creating,

reading, writing, appending, renaming, and deleting files.

• Flexibility: File handling in Python is highly flexible, as it allows you to work with different file types (e.g. text

files, binary files, etc.), and to perform different operations on files (e.g. read, write, append, etc.).

• User–friendly: Python provides a user-friendly interface for file handling, making it easy to create, read, and

manipulate files.

• Cross-platform: Python file-handling functions work across different platforms (e.g. Windows, Mac, Linux),

allowing for seamless integration and compatibility.

Disadvantages of File Handling in Python

• Error-prone: File handling operations in Python can be prone to errors, especially if the code is not carefully

written or if there are issues with the file system (e.g. file permissions, file locks, etc.).

• Security risks: File handling in Python can also pose security risks, especially if the program accepts user

input that can be used to access or modify sensitive files on the system.

• Complexity: File handling in Python can be complex, especially when working with more advanced file

formats or operations. Careful attention must be paid to the code to ensure that files are handled properly

and securely.

• Performance: File handling operations in Python can be slower than other programming languages,

especially when dealing with large files or performing complex operations.

[Link]() Method
The Python [Link]() method of the OS module creates a pipe for inter-process communication. This method
allows set of data to be passed from one process to another. This process is possible because it returns a pair of
file descriptors namely "r" and "w" usable for reading and writing, respectively.

Syntax
Following is the syntax for Python [Link]() method −
[Link]()

Parameters
The Python [Link]() method does not accept any parameter.

Return Value
The Python [Link]() method returns a pair of file descriptors.
When we run the program, it produces following
result −

The child will write text to a pipe and

the parent will read the text written by child...
Parent reading....
Child writing....
Child closing
text = Text written by child...
IOError
• When working with Input and Output Operations in Python, if we encounter an error related to file, the
code will throw the IOError.
• When we attempt to open a file and if it does not exist, the IOError will be encountered.
• In a case where the statement or the line of code is correct, it may result in an error while execution.
• Now the error like these, which are detected during the program execution are known as exceptions.
• Commonly, the IOError is raised when an input output operation like open() file, or a method or a simple
print statement is failed due to IO reasons like “Disk full” or “File not found”.
• The IOError class is inherited from the EnvironmentError.

Syntax:
The syntax mentioned below is a standard format using which the occurrence of the error will be printed.
How IOError work in Python?

• In a Python program, where we have a simple operation of print the content of a file, we pass the file name

and path of the file location.

• But if the file that we passed, does not exist at the passed location or the file name has been changed, then

the operation we intend to execute won’t happen. Which will result in an error related to Input Output,

which is IOError.

• So, basically, IOError is an exception type error that occurs when the file that we passed in as argument, does

not exist or as a different name or the file location path is incorrect. Any of these reason could raise an

IOError.

• There are many other errors that can be encountered and based on the requirement of the code we have

handle these error.

Example
Here we will demonstrate how we can catch a python IOError, or any other error using try except block. Code for the
program with try except block is as follows.

None
[Errno 2] No such file or directory: '[Link]'

• Our program starts with importing system files, as we will be working on Input Output operations. Then we have
our first function defined with name of something().
• Inside function, we start our try keyword, meaning every thing from this point will be looked for an error. Within
out try, we have our first file operation, which is to open a file.
• Our expected out is like: No such file or directory: ‘[Link]’.
How to Avoid IOError in Python?

• In any programming or scripting language, it is completely possible to write program in a way that it can catch

the expected errors.

• Understanding the code prior to execution and including the necessary catch exception clause will help in

avoiding the exception.

• One of the basic and simplest way to work with exceptions in Python is to implement the catch-all except clause.

• The keywords like try and except hold the real value here and used to catch any kind of exceptions.

• It is also recommended not to use many try except blocks in a program.

• The error is caught in the try part while it is handled in except part.

• In addition, we can use finally, along with the try except block.

• The primary use of finally here will be to execute the code no matter what error is caught or handled.
Directory management using Python
• Python contains several modules that has a number of built-in functions to manipulate and process data.
• Python has also provided modules that help us to interact with the operating system and the files.
• These kinds of modules can be used for directory management also.

os and [Link] module

The os module is used to handle files and directories in various ways. It provides provisions to create/rename/delete
directories. This allows even to know the current working directory and change it to another. It also allows one to copy
files from one directory to another. The major methods used for directory management is explained below.

Creating new directory:

• [Link](name) method to create a new directory.
• The desired name for the new directory is passed as the parameter.
• By default it creates the new directory in the current working directory.
• If the new directory has to be created somewhere else then that path has to be specified and the path should
contain forward slashes instead of backward ones.
Getting Current Working Directory (CWD):
• [Link]() can be used.
• It returns a string that represents the path of the current working directory.
• [Link]() can also be used but it returns a byte string that represents the current working directory.
• Both methods do not require any parameters to be passed.

String format : /home/nikhil/Desktop/test

Byte string format : b'/home/nikhil/Desktop/test'

Renaming a directory:
• [Link]() method is used to rename the directory.
• The parameters passed are old_name followed by new_name.
• If a directory already exists with the new_name passed, OSError will be raised in case of both Unix and Windows.
• If a file already exists with the new_name, in Unix no error arises, the directory will be renamed. But in Windows
the renaming won’t happen and error will be raised.
[Link](‘old_name’,’dest_dir:/new_name’) method works similar to [Link]() but it moves the renamed

file to the specified destination directory(dest_dir).

For example, consider there is a file named ‘[Link]’ in current working directory. Now to just rename it :

If renaming and moving the file to some other directory is required, then the code snippet should be:
Changing Current Working Directory (CWD):
• Every process in the computer system will have a directory associated with it, which is known as Current
Working Directory(CWD).
• [Link]() method is used to change it.
• The parameter passed is the path/name of the desired directory to which one wish to shift.
For example, If we need to change the CWD to my_folder in D:/, then the following code snippet is used.

Current directory : /home/nikhil/Desktop/test

Current directory : /home/nikhil/Desktop
Removing a directory
• [Link]() method is used to remove/delete a directory.
• The parameter passed is the path to that directory.
• It deletes the directory if and only if it is empty, otherwise raises an OSError.

For example, Let us consider a directory K:/files. Now to remove it, one has to ensure whether it is empty and
then proceed for deleting.
Problem: Write a Python program to read a text file named [Link],

1. Take the input to the file in string format

2. Count the occurrences of each word in the file

3. Then write the results to a new text file named [Link].

Python Database Tutorial

• Python being a high-level language provides support for various databases. We can connect and run queries

for a particular database using Python and without writing raw queries in the terminal or shell of that

particular database, we just need to have that database installed in our system.

• Most commonly used relational databases such as MySQL, SQLite, NoSQL databases like MongoDB are used in

Python.

Python MySQL
• MySQL is a Relational Database Management System (RDBMS) whereas the structured Query Language (SQL)
is the language used for handling the RDBMS using commands i.e Creating, Inserting, Updating and Deleting
the data from the databases.
• A connector is employed when we have to use MySQL with other programming languages. The work of mysql-
connector is to provide access to MySQL Driver to the required language. Thus, it generates a connection
between the programming language and the MySQL Server.
Python MySQL
• Python MySQL Connector is a Python driver that helps to integrate Python and MySQL. This Python MySQL library
allows the conversion between Python and MySQL data types.

MySQL Database

To be able to experiment with the code examples in this tutorial, you should have MySQL installed on your computer.

You can download a MySQL database at [Link]

Install MySQL Driver

• Python needs a MySQL driver to access the MySQL database.

• You can use PIP to install "MySQL Connector". (mysql-connector, mysql-connector-python, mysql-connector-python-rf)

• PIP is most likely already installed in your Python environment.

>> pip install mysql-connector-python-rf

To connect to a database and retrieve data using the Python programming language, you only need

a few functions:

• connect() function to create a connection to the database;

• cursor() function to create a cursor from the connection;

• execute() function to execute a select statement;

• fetchone() function to retrieve rows from the query.

 Test MySQL Connector
To test if the installation was successful, or if you already have "MySQL Connector" installed, create a Python page
with the following content:
demo_mysql_test.py:

If the above code was executed with no errors, "MySQL Connector" is installed and ready to be used.
Create Connection
Start by creating a connection to the database.
Use the username and password from your MySQL database:
demo_mysql_connection.py:

<[Link] object ar 0x016645F0>

Python MySQL Create Database
Creating a Database
To create a database in MySQL, use the "CREATE DATABASE" statement:

Example
create a database named "mydatabase":

Process finished with exit code 0.

If the above code was executed with no errors, you have successfully created a database.
Check if Database Exists
You can check if a database exist by listing all databases in your system by using the "SHOW DATABASES" statement:
Example
Return a list of your system's databases:

('information_scheme',)
('mydatabase',)
('performance_schema',)
('sys',)
Example
Try connecting to the database "mydatabase":

If the database does not exist, you will get an error.

Creating a Table
To create a table in MySQL, use the "CREATE TABLE" statement.
Make sure you define the name of the database when you create the connection
Example
Create a table named "customers":

If the above code was executed with no errors, you have now successfully created a table.
Check if Table Exists
You can check if a table exist by listing all tables in your
database with the "SHOW TABLES" statement:
Example
Return a list of your system's databases:

('customers',)
Primary Key
When creating a table, you should also create a column with a unique key for each record.
This can be done by defining a PRIMARY KEY.
We use the statement "INT AUTO_INCREMENT PRIMARY KEY" which will insert a unique number for each record.
Starting at 1, and increased by one for each record.
Example
Create primary key when creating the table:
If the table already exists, use the ALTER TABLE keyword:

Example
Create primary key on an existing table:
Insert Into Table
To fill a table in MySQL, use the "INSERT INTO" statement.
Example
Insert a record in the "customers" table:

[Link]() - It is required to make the changes,

otherwise no changes are made to the table.

1 record inserted.
Delete Record
You can delete records from an existing table by using the "DELETE FROM" statement:
Example
Delete any record where the address is "Mountain 21":

1 record(s) deleted
Delete a Table
You can delete an existing table by using the "DROP TABLE" statement:
Example
Delete the table "customers":
Introduction to Psycopg2 module
• Psycopg is the most popular PostgreSQL adapter used in Python.
• Its works on the principle of the whole implementation of Python DB API 2.0 along with the thread safety (the same
connection is shared by multiple threads).
• It is designed to perform heavily multi-threaded applications that usually create and destroy lots of cursors and
make a large number of simultaneous INSERTS or UPDATES.
• Psycopg features client-side and server-side cursors, asynchronous communication, and notification.
• Psycopg 2 is both Unicode and Python 3 friendly.
Installation:
The current psycopg2 implementation supports:
• Python versions from 3.6 to 3.10
• PostgreSQL server versions from 7.4 to 14
• PostgreSQL client library version from 9.1
• pgAdmin 4
>> pip install psycopg2 #install psycopg2
For most of the available Operating Systems, the quickest way to install this package is through the wheel
package available in the PyPI library. We must make sure that we use the latest version of pip, which can be
updated using the following command in the terminal.

This will install the pre-compiled binary version of the module which doesn’t require the built or runtime
prerequisites. Then we can import the psycopg2 package in the usual manner:

Basic module usage:

The basic use of Psycopg is in implementing the DB API 2.0 protocol to all the database adapters. Here is the
basic interactive session of the basic commands.
Example 1: Program to establish a connection between python program and a PostgreSQL database.

Database connected successfully

Example 2: Creating a table using python

Database connected successfully

Table Created successfully
Example 3: Inserting data into the table:

Database connected successfully

Example 4: Deleting data from the database.

Database connected successfully

Data deleted Successfully
Total row affected 1
Connectivity of ODBC
Open Database Connectivity or ODBC is an open standard Application Programming Interface (API) for accessing a

database. It can manage multiple database management system by single application with same source code. Database

applications call functions in the ODBC interface, which are implemented in database-specific modules called drivers.

Drivers are used to isolates applications from database-specific calls.

• ODBC provides extensive data support for data also known as metadata.
• ODBC provide variety of service to obtain data about type and functions.
• ODBC use SQL syntax and whenever statement is transferred by a user to an ODBC driver it resembles and converts
it into the appropriate SQL statement accepted by the basic database.
• ODBC driver can develop a variety of applications that can communicate with a different Databases and switching
application from one database to another can be enabled.
• The architecture of ODBC-based data, Application is connected to ODBC Driver and Driver manager then this further
links to database, which is useful to access information from database.
• ODBC Driver processes ODBC activity calls, sends SQL requests to a specific data source and returns results in
the system. The ODBC driver may also modify the system application to suit the syntax supported by the
corresponding database.
• Driver Manager loads driver for each application.
What is pyodbc connect?
pyodbc is a Python open-source module that simplifies access to ODBC databases such as Oracle, MySQL, PostgreSQL, SQL
Server, etc. A module implements the DB API 2.0 specification created to provide a consistent interface to various databases
and help developers write applications that can work with different databases without significant changes in the code.

What is ODBC in Python?

ODBC (Open Database Connectivity) — is an Application Programming Interface (API) designed to access data storage.
Many databases are supplied with an ODBC drivers, so you can use any of them with the pyodbc interface to access
databases such as SQL Server, Oracle, PostgreSQL as well or cloud applications such as Streak, Zoho CRM, BigCommerce,
etc. from your Python application.

How to Connect Python to SQL Server Using pyodbc?

1. Connect to the SQL Server database by importing the pyodbc module and creating a connection to the database.
2. Execute an insert statement to test the connection to the database. The script inserts a new record to the table.
3. Retrieve data from the SQL Server: use the [Link]() function to retrieve rows from the select query on a
datase and the [Link]() function to iterate over the result set returned by [Link](). To print all
records from the table to the console, use the print() function.
Pros :

• It is well integrated into many different RAD tools.

• It easily gets into various “data-bound” objects in various development sites such as Power builder, Delphi,

Visual Basic and Java etc.

• It also simplifies and speeds up application development.

• It is also helpful to organize many different items at a time, with the help of templates provided by ODBC.

• It also allows full integration which means that a single system can easily access different data management

systems or we can also say that ODBC allows a single system to manage different types of DBMS depending

on user requirement or available DBMS type.

• With the help of built-in function we can create custom applications.

Cons :

• ODBC drivers are not user friendly

• ODBC drivers are not standardized. That’s why clients maintain their own driver, naming tables which

creates a problem for management of large sites

• ODBC drivers have Slow speed

• As ODBC specification specifies only application protocol so it basically inherits features of framework

in which it is used on. Thus we can say that reliability depends on implementation of

request/response protocol of underlying framework that is being used.

Problem: Write a Python program that connects to a MySQL database, creates a table named

Employees (check, if it doesn't already exist), inserts a few records into the table, and then

retrieves and displays all the records from the Employees table. The Employees table should have

the following columns: ID (Primary Key), Name, Position, and Salary.

Records to be added:

1. 'John Doe', 'Software Engineer', 60000

2. 'Jane Smith', 'Data Scientist', 80000

3. 'Mike Johnson', 'Project Manager', 75000

Thank you!!