DSBDAL Lab Manual

STES’s
SINHGAD COLLEGE OF ENGINEERING

Vadgaon(Bk), Pune
Department of Computer Engineering
LABORATORY MANUAL
2023-24
DATA SCIENCE AND BIG DATA

ANALYTICS LABORATORY
TE-COMPUTER ENGINEERING
SEMESTER-II
Subject Code:310256
-: Name of Faculty:-
Prof. A.M.Karanjkar
Prof. S.S.Peerzade
DSBDAL T.E.C.E (Sem- [2023-24]
II)
INDEX
GROUP A: DATA SCIENCE
Sr.No. Title Page
Number
1 Data Wrangling, I
Perform the following operations using Python on any open source dataset (e.g., data.csv)
1. Import all the required Python Libraries.
2. Locate an open source data from the web (e.g. https://www.kaggle.com).
Provide a cleardescription of the data and its source (i.e., URL of the web site).
3. Load the Dataset into pandas data frame.
4. Data Preprocessing: check for missing values in the data using pandas insult(),
describe()function to get some initial statistics. Provide variable descriptions.
Types of variables etc. Check the dimensions of the data frame.
5. Data Formatting and Data Normalization: Summarize the types of variables by
checking the data types (i.e., character, numeric, integer, factor, and logical) of
the variables in the data set. If variables are not in the correct data type, apply
proper type conversions.
6. Turn categorical variables into quantitative variables in Python.
In addition to the codes and outputs, explain every operation that you do in the above steps
and explain everything that you do to import/read/scrape the data set.
2 Data Wrangling II
Create an “Academic performance” dataset of students and perform the following
operations using Python.
1. Scan all variables for missing values and inconsistencies. If there are missing
2 Department of Computer Engineering, SCOE, Pune

DSBDAL T.E.C.E (Sem-II) [2023-24]
values and/or inconsistencies, use any of the suitable techniques to deal with them.
2. Scan all numeric variables for outliers. If there are outliers, use any of the suitable
techniques to deal with them.
3. Apply data transformations on at least one of the variables. The purpose of this
transformation should be one of the following reasons: to change the scale for
better understanding of the variable, to convert a non-linear relation into a linear
one, or to decrease the skewness and convert the distribution into a normal
distribution. Reason and document your approach properly.
3 Descriptive Statistics - Measures of Central Tendency and variability
Perform the following operations on any open source dataset (e.g., data.csv)
1. Provide summary statistics (mean, median, minimum, maximum, standard
deviation) for a dataset (age, income etc.) with numeric variables grouped by one
of the qualitative (categorical) variable. For example, if your categorical variable
is age groups and quantitative variable is income, then provide summary statistics
of income grouped bythe age groups. Create a list that contains a numeric value
for each response to the categorical variable.
2. Write a Python program to display some basic statistical details like percentile,
mean, standard deviation etc. of the species of ‘Iris-setosa’, ‘Iris-versicolor’ and
‘Iris- versicolor’ of iris.csv dataset. Provide the codes with outputs and explain
everything that you do in this step.
4 Data Analytics I
Create a Linear Regression Model using Python/R to predict home prices using Boston
Housing Dataset (https://www.kaggle.com/c/boston-housing). The Boston Housing
dataset contains information about various houses in Boston through different parameters.
There are 506 samplesand 14 feature variables in this dataset.
The objective is to predict the value of prices of the house using the given features.
5 Data Analytics II
1. Implement logistic regression using Python/R to perform
classification onSocial_Network_Ads.csv dataset.
2. Compute Confusion matrix to find TP, FP, TN, FN, Accuracy, Error rate,
Precision, Recall on the given dataset.
6 Data Analytics III
1. Implement Simple Naïve Bayes classification algorithm using Python/R on
iris.csvdataset.
2. Compute Confusion matrix to find TP, FP, TN, FN, Accuracy, Error rate,
Precision, Recall on the given dataset.
7 Text Analytics
1. Extract Sample document and apply following document preprocessing

methods:Tokenization, POS Tagging, stop words removal, Stemming

and Lemmatization.
2. Create representation of document by calculating Term Frequency and Inverse
Document Frequency.
8 Data Visualization I
1. Use the inbuilt dataset 'titanic'. The dataset contains 891 rows and contains
information about the passengers who boarded the unfortunate Titanic ship. Use
the Seaborn libraryto see if we can find any patterns in the data.
2. Write a code to check how the price of the ticket (column name: 'fare') for
each passenger is distributed by plotting a histogram.
9 Data Visualization II
1. Use the inbuilt dataset 'titanic' as used in the above problem. Plot a box plot for
distribution of age with respect to each gender along with the information about
whether they survived or not. (Column names : 'sex' and 'age') Write observations
on the inference from the above statistics.
10 Data Visualization III
Download the Iris flower dataset or any other dataset into Data Frame.(e.g.,
https://archive.ics.uci.edu/ml/datasets/Iris ). Scan the dataset and give the inference as:
1. List down the features and their types (e.g., numeric, nominal) available in
the dataset.
2. Create a histogram for each feature in the dataset to illustrate the
feature distributions.
3. Create a box plot for each feature in the dataset. Compare distributions
and identify outliers.
GROUP B: BIG DATA ANALYTICS

Any 3 assignments are mandatory.
Sr.No. Title Page

Number
1 Write a code in JAVA for a simple Word Count application that counts the number of
occurrences of each word in a given input set using the Hadoop Map-Reduce framework
onlocal-standalone set-up.
2 Design a distributed application using Map-Reduce which processes a log file of a system.
3 Locate dataset (e.g., sample_weather.txt) for working on weather data which reads the text
input files and finds average for temperature, dew point and wind speed.

GROUP C: MINI PROJECT/CASE STUDY PYTHON/R

Any 2 mini project are mandatory.
Sr.No. Title Page
Number
1 Write a case study on Global Innovation Network and Analysis (GINA). Components of
analyticplan are 1. Discovery business problem framed, 2. Data, 3. Model planning analytic
technique and 4. Results and Key findings.
2 Use the following dataset and classify tweets into positive and negative tweets.
https://www.kaggle.com/ruchi798/data-science-tweets
3 Develop a movie recommendation model using the scikit-learn library in python.
Refer dataset https://github.com/rashida048/Some-NLP-
Projects/blob/master/movie_dataset.csv
4 Use the following covid_vaccine_statewise.csv dataset and perform following analytics
on thegiven dataset
https://www.kaggle.com/sudalairajkumar/covid19-
inndia?select=covid_vaccine_statewise.csv
a. Describe the dataset
b. Number of persons state wise vaccinated for first dose in India
c. Number of persons state wise vaccinated for second dose in India
d. Number of Males vaccinated
d. Number of females vaccinated
5 Write a case study to process data driven for Digital Marketing OR Health care
systems withHadoop Ecosystem components as shown. (Mandatory)
● HDFS: Hadoop Distributed File System
● YARN: Yet Another Resource Negotiator
● MapReduce: Programming based Data Processing
● Spark: In-Memory data processing
● PIG, HIVE: Query based processing of data services
● HBase: NoSQL Database (Provides real-time reads and writes)
● Mahout, Spark MLLib: (Provides analytical tools) Machine Learning
algorithmlibraries Solar, Lucene: Searching and Indexing

GROUP A: DATA SCIENCE

Assignment No. 1 (GROUP A)
Title Perform the following operations using Python on any open

source dataset.
Subject Data Science & Big Data Analytics Laboratory
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature
Department of Computer Engineering, SCOE, Pune

Title Create an “Academic performance” dataset of students and

perform the following operations using Python.
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

0
Title Perform the following operations on any open source dataset.
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Analytics I
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Analytics II
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Analytics III
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Text Analytics:
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Visualization I
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Visualization II
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Data Visualization III
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

GROUP B: BIG DATA ANALYTICS

Assignment No. 1 (GROUP B)
Title Write a code in JAVA for a simple Word Count

application that counts the number of occurrences of
each word in a given input set using the Hadoop Map-
Reduce framework onlocal-standalone set-up.
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Design a distributed application using Map-Reduce which

processes a log file of a system.
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title Locate dataset (e.g., sample_weather.txt) for working on weather

data which reads the textinput files and finds average for
temperature, dew point and wind speed.
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

GROUP C: MINI PROJECT/CASE STUDY

PYTHON/R
Any 2 mini project are mandatory

Assignment No. 1 (GROUP C) (Mini Project)
Title
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

Title
Class T.E. (C.E.)
Roll No. 305A016
Date
Signature

DSBDAL Lab Manual

Uploaded by

DSBDAL Lab Manual

Uploaded by

STES’s

SINHGAD COLLEGE OF ENGINEERING

DATA SCIENCE AND BIG DATA

2 Department of Computer Engineering, SCOE, Pune

3 Department of Computer Engineering, SCOE, Pune

methods:Tokenization, POS Tagging, stop words removal, Stemming

GROUP B: BIG DATA ANALYTICS

Sr.No. Title Page

4 Department of Computer Engineering, SCOE, Pune

GROUP C: MINI PROJECT/CASE STUDY PYTHON/R

5 Department of Computer Engineering, SCOE, Pune

GROUP A: DATA SCIENCE

6 Department of Computer Engineering, SCOE, Pune

Assignment No. 1 (GROUP A)

Title Perform the following operations using Python on any open

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 2 (GROUP A)

Title Create an “Academic performance” dataset of students and

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 3 (GROUP A)

Title Perform the following operations on any open source dataset.

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 4 (GROUP A)

Title Data Analytics I

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 5 (GROUP A)

Title Data Analytics II

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 6 (GROUP A)

Title Data Analytics III

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 7 (GROUP A)

Title Text Analytics:

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 8 (GROUP A)

Title Data Visualization I

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 9 (GROUP A)

Title Data Visualization II

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

Assignment No. 10 (GROUP A)

Title Data Visualization III

Subject Data Science & Big Data Analytics Laboratory

Class T.E. (C.E.)

Roll No. 305A016

Department of Computer Engineering, SCOE, Pune

GROUP B: BIG DATA ANALYTICS

Department of Computer Engineering, SCOE, Pune

Assignment No. 1 (GROUP B)

Title Write a code in JAVA for a simple Word Count

Subject Data Science & Big Data Analytics Laboratory