Scribd
Upload
51 views

Lecture Notes 6

The document introduces machine learning concepts including convolutional neural networks (CNNs), pooling, and recurrent neural networks (RNNs). RNNs can process sequential data like time-series and have memory, making them well-suited for natural language, speech recognition, and image captioning. However, RNNs struggle with vanishing gradients, leading to the development of long short-term memory (LSTM) and gated recurrent unit (GRU) networks to improve memory. The document then demonstrates building a sentiment analysis model using Keras and TensorFlow on a text dataset in Databricks.

Uploaded by

fgsfgs
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
51 views

Lecture Notes 6

The document introduces machine learning concepts including convolutional neural networks (CNNs), pooling, and recurrent neural networks (RNNs). RNNs can process sequential data like time-series and have memory, making them well-suited for natural language, speech recognition, and image captioning. However, RNNs struggle with vanishing gradients, leading to the development of long short-term memory (LSTM) and gated recurrent unit (GRU) networks to improve memory. The document then demonstrates building a sentiment analysis model using Keras and TensorFlow on a text dataset in Databricks.

Uploaded by

fgsfgs
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Chapter 1 Introduction to Machine Learning

Figure 1-31.  Pooling

We can repeat the previous steps (convolution and pooling) multiple


times in a network to learn the main features of the image and finally pass
it to the fully connected layer at the end to make the classification.

R
 NN
The general feedforward neural networks and CNNs are not good for
time-series kinds of datasets as these networks don’t have any memory of
their own. Recurrent neural networks bring with them the unique ability
to remember important stuff during the training over a period of time.
This makes them well suited for tasks such as natural language translation,
speech recognition, and image captioning. These networks have states
defined over a timeline and use the output of the previous state in the
current input, as shown in Figure 1-32.

39
Chapter 1 Introduction to Machine Learning

Figure 1-32.  RNN

Although RNNs have proved to be really effective in time-series kinds


of applications, it does run into some serious limitations in terms of
performance because of its architecture. It struggles with what is known as
a vanishing gradient problem that occurs due to no or very little updates in
the weights of the network as the network tries to use data points that are
at the early stages of timeline. Hence, it has a limited memory to put it in
simple terms. To tackle this problem, there are couple of other variants of
RNNs.

• Long short-term memory (LSTM)

• Gradient recurring unit (GRU)

• Attention networks (encoder-decoder model)

40
Chapter 1 Introduction to Machine Learning

Now we are going to use a small dataset and build a deep learning
model to predict the sentiment given the user review. We are going to
make use of TensorFlow and Keras to build this model. There are couple of
steps that we need to do before we train this model in Databricks. We first
need to go to the cluster and click Libraries. On the Libraries tab, we need
to select the Pypi option and mention Keras to get it installed. Similarly, we
need to mention TensorFlow as well once Keras is installed.
Once we upload the reviews dataset, we can create a pandas dataframe
like we did in the earlier case.

[In]: from tensorflow.keras.models import Sequential


[In]: from tensorflow.keras.layers import LSTM,Embedding
[In]: from tensorflow.keras.layers import Dense
[In]: from tensorflow.keras.preprocessing.text import Tokenizer
[In]: from tensorflow.keras.preprocessing.sequence import pad_
sequences
[In]:sparkDF= spark.read.csv('/FileStore/tables/text_summary.
csv', header="true", inferSchema="true")

[In]: df=sparkDF.toPandas()

[In]: df.columns
[Out]: Index(['Sentiment', 'Summary'], dtype='object')

As we can see, there are just two columns in the dataframe.

[In]: df.head(10)

[Out]:

41
Chapter 1 Introduction to Machine Learning

[In]: df.Sentiment.value_counts()

[Out]:
1 1000
0 1000

We can also confirm the class balance by taking a value counts of the
target column. It seems the data is well balanced. Before we go ahead with
building the model, since we are dealing with text data, we need to clean it
a little bit to ensure no unwanted errors are thrown at the time of training.
Hence, we write a small helper function using regular expressions.

[In]:
import re
def clean_reviews(text):
    text=re.sub("[^a-zA-Z]"," ",str(text))
    return re.sub("^\d+\s|\s\d+\s|\s\d+$", " ", text)

[In]: df['Summary']=df.Summary.apply(clean_reviews)

[In]: df.head(10)

42
Chapter 1 Introduction to Machine Learning

[Out]:

The next step is to separate input and output data. Since the data is
already small, we are not going to split it into train and test sets; rather, we
will train the model on all the data.

[In]: X=df.Summary
[In]: y=df.Sentiment

We now create the tokenizer object with 10,000 vocab words, and an
out-of-vocabulary (oov) token is mentioned for the unseen words that the
model gets exposed to that are not part of the training.

[In]: tokenizer=Tokenizer(num_words=10000,oov_token='xxxxxxx')

[In]: tokenizer.fit_on_texts(X)

[In]: X_dict=tokenizer.word_index

[In]: len(X_dict)

[Out]: 2018

43

You might also like