Abstractive Text Summarization Using Transformer Based Approach

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/371314917
Abstractive Text Summarization Using Transformer Based Approach
Research · June 2023

DOI: 10.55041/IJSREM22369
CITATIONS READS
0 1,645
5 authors, including:
Karishma Shukla Dnyaneshwar Bavkar

Terna Engineering College University of Mumbai
1 PUBLICATION 0 CITATIONS 9 PUBLICATIONS 98 CITATIONS
SEE PROFILE SEE PROFILE
All content following this page was uploaded by Karishma Shukla on 06 June 2023.
The user has requested enhancement of the downloaded file.

International Journal of Scientific Research in Engineering and Management (IJSREM)
Volume: 07 Issue: 06 | June - 2023 SJIF Rating: 8.176 ISSN: 2582-3930
Abstractive Text Summarization Using Transformer Based Approach

Karishma Shukla1, Kartik Barange2, Prajakta Shahabade3, Akanksha Pandey4, Bavkar Dnyaneshwar Madhukar5
1, 2, 3, 4
B.E. Student, Department of Computer Engineering, Terna Engineering College, Navi Mumbai, Maharashtra, India
5
Assistant Professor, Department of Computer Engineering, Terna Engineering College, Navi Mumbai, Maharashtra
Abstract—The amount of textual data on the Internet has reflect the content of the source text. The correct application of
risen in an exponential manner over the past decade. such abstraction in deep learning issues can help overcome
Information utilization has become an expensive and time- grammatical errors.
consuming activity since data expands in large quantities at a
time and includes irrelevant content or noise. One such
common technique that helps in dealing with enormous data
is Text summarization. Automatic text summarization is a
renowned approach that is used to reduce a document to
its main ideas. It operates by preserving substantial
information by creating a shortened version of the text.
Automatic Text Summarization helps in creating a short,
coherent, and fluent summary of a longer text document
and involves outlining of the text’s major points using
Natural language processing (NLP). Automatic Text Sum-
marization is the process of creating a condensed form of a Fig. 1. Text Summarization Techniques
text document which maintains significant information and
the general meaning of source text. This project aims to build The goal of summarising in this project is to condense a
a web-based text summarization application using the lengthy document or a news article without losing any infor-
abstractive text summarization technique to get the most mation. Deep Learning has become more and more popular in
precise and useful information from a document and recent years for generating text summaries. Understanding the
eliminate irrelevant data. This is achieved by using T5 (Text- principles of natural language processing and developing a tool
to-Text Transfer Transformer) which is a state-of-the-art
for text summarization are the goals of this research. The need for
model that converts every language problem into a text-to-
text format along with the CNN Daily Mail dataset which automatic summarization is growing significantly, eliminating the
is used as the training dataset. need for manual labor. The project focuses on developing a tool
that summarises the document automatically. This project’s
Index Terms—Text Summarization, T5 (Text-to-Text primary goal is to extract or produce a summary from the text
TransferTransformer), Natural Language Processing. provided as input. Examples of the range of this online
application’s functionality include writing headlines for news
I. INTRODUCTION channels, briefings, note-taking, making highlights for speeches,
The process of utilizing software to condense a text document etc.
into a concise summary that highlights the main concepts from
the original text is known as text summarization the process II. LITERARY SURVEY
of text summarization involves using software to reduce a text The paper [1] This research [1] illustrates how the volume of
document in order to provide an abstract or summary of the online text data is continuously increasing, making text
original material automatic text summariza- tions objective is to summarization critical for modern recommender and text clas-
extract the crucial information from a large body of text however sification systems. Extractive and abstractive summarization are
because the process is time- consuming and more information is two ways for creating summaries. Extractive summarization
being created every day this is becoming increasingly picks relevant information. Sentences from the original content,
challenging. There are two for text summarization namely whereas abstractive summarization analyses the text to construct
Extractive Summarization and Abstractive Summarization. a summary. The research evaluates and compares the summaries
generated by machine learning models using the BBC news
Extractive Summarization: By selecting the key phrases or
dataset with human-written summaries. The authors used pre-
sentences from the original text and piecing together chunksof the
trained transformer-based language models to summarise text and
content to create a condensed version, extractive methods try to
discovered that carefully tailored models provided fluent
summarise articles. The summary is then created using these
summaries. They used ROUGE ratings to compare the models
extracted sentences..
and found that the T5 model performed the best. A model hybrid
Abstractive Summarization: Contrary to extraction, this could improve the summaries’ correctness, fluency, and
method depends on the ability to condense and paraphrase coherence.
portions of a document utilising sophisticated natural language
approaches. Considering that abstractive machine learning
algorithms can produce fresh words and phrases to accurately
© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 1

The authors implemented pre-trained transformer-based lan- were tested using a variety of datasets, including the CNN
guage models for text summarization and found that finely- tuned corpus, DUC2000, and single and multiple text documents. The
models produced fluent summaries. They evaluated the models paper also analyses previous work, achievements, trends, and the
using ROUGE scores and concluded that the T5 model performed future scope of text summarization in many domains. These
the best. A hybrid of the models could improve the accuracy, algorithms can generate several sorts of summaries, and their
fluency, and coherence of the summaries. accuracy can be compared using metrics like as ROGUE and TF-
IDF scores. However, the generated summaries are not always
The paper [2] proposes a model for web document sum- perfect and may be irrelevant to the original content. The field of
marising that makes advantage of social media data such as user text summarization is still evolving, and numerous efforts have
comments and tweets. Unlike previous research that used feature been made to improve it. Because there is no single model that
engineering, this model uses transformers to fully use the data’s produces the best summaries, the discussed models can be
contextual component. To enrich the information, relevant user updated in the future utilising GANs and transfer learning to
postings are matched with sentences, and a convolution neural produce more accurate summaries. This can result in the
network layer is added for categorization. Experiment findings on development of new and improved text summarising ideas.
two English datasets show that this model outperforms other
models that do not use transformers in summarising single These methods can produce different types of summaries, and
documents. their accuracy can be compared using metrics such as ROGUE
and TF-IDF scores. However, the generated summaries are not
The paper [3] describes The explosion of data on the internethas always perfect and can be irrelevant to the original document.
created a demand for solutions that convert raw data into valuable The field of text summarization is ongoing, and various works
data that humans can understand. Text summarization, which have been done to improve it. There is no specific model that
entails condensing a document to its key ideas, is one method for generates the best summaries, so the discussed models can be
dealing with massive amounts of data. There are two types of text modified using GANs and transfer learning for more accurate
summarization: extractive and abstractive. sentences from the summaries in the future. This can lead to the development of new
original text to create a subset of mean- ingful sentences, whereas and enhanced ideas for text summarization.
abstractive approaches construct a shorter version of the text
while retaining significant information. NLP academics are The paper [5] explores the significance of text summarization
particularly interested in extractive approaches, which use in light of the increasing amount of text data available online.
linguistic and statistical properties to calculate the implications of Extractive and abstractive summarization are two ways for
phrases. This research examines both extractive and abstractive producing summaries. The former pulls relevant sentences from
strategies for summarising texts and provides insight into methods the original material, while the latter interprets the text to
that result in less repetition and a more condensed summary. The construct the summary. The research compares transformer-based
future scope of automatic text summarization is to answer these pre-trained models for text summarization using the BBC news
issues and make the technology easier and more feasible to apply. dataset as an example. Human-generated summaries are used to
The search for the optimal model that can generate a summary evaluate and compare machine learning model summaries. This
like a humanis ongoing. study describes the implementation of pre-trained language
models for summarization tasks using the transformer
The paper reviewed various research papers on abstractive, architecture. Finely tuned transformer-based language models
extractive, and hybrid techniques for text summarization, as well achieved great results and delivered fluent summaries for a
as learning methods such as supervised, unsupervised, and particular text material, according to the researchers. They used
reinforcement. Each of these techniques has its own set of ROUGE ratings to compare the performance of several models
challenges that need to be addressed, including evaluating the and discovered that the T5 model outperformed all others. Future
quality of the summaries, obtaining labeled data, dealing with research should focus on developing more robust models that can
anaphora and cataphora problems, and others. The future scope of generate summaries of varied lengths and be applied to multi-
automatic text summarization is to resolve these challenges and document summarising, according to the researchers. They also
make the technology easier and more feasible to implement. propose that a hybrid of the models be employed to increase the
Research on automatic text summarization is ongoing to find the summaries’ accuracy, fluency, and coherence.
perfect model that can generate a summarylike a human.
III. EXISTING SYSTEM
The paper [4] investigates several strategies for producing text
document summaries, including Abstractive (ABS) and Our proposal employs the T5 (Text-to-Text Transfer Trans-
Extractive (EXT) summaries. Query-based summarising ap- former) model, which is a more sophisticated and success-ful
proaches are also presented. The research article focuses on method to text summarization than the present system’s
structured-based and semantic-based approaches to summarising TextRank algorithm. Transformers outperform the TextRank
text documents. The summaries produced by these models

algorithm for text summarization because they can capture the Saloni; Verma, Srishti : Created an project A Review: Abstractive
contextual relationships and dependencies between words in a Text Summarization Techniques using NLP. The disadvantage
sentence or document. Transformers, unlike TextRank, are was They are doing abstractive summarization but with LSTM
trained on large-scale data to learn representations of languagethat and RNN. Transformers are better than LSTM and RNN for text
capture the intricacies of natural language. This allows summarization because they can learn relationships between
transformers to generate more human-like and informative words in a sentence or document more effectively, making them
summaries. Transformers can also be fine-tuned f or certain more accurate and able to generate more informative summaries.
domains or jobs, something TextRank does not allow. Trans-
formers, unlike TextRank, are trained on large-scale data to IV. COMPARATIVE STUDY
learn representations of language that capture the intricacies of
Comparison between T5 Transformer, BART, Pegasus, and
natural language. This allows transformers to generate more
TextRank algorithm for Abstractive Text Summarization based on
human-like and informative summaries. Transformers can also be
their Rouge Score.
fine-tuned F urthermore, t ransformers o utperform LSTM and
RNN for text summarising because they can learn as- T5 Transformer has been shown to outperform both BART and
sociations between words in a sentence or document more ef- Pegasus models for abstractive text summarization tasks. The T5
fectively, resulting in more accurate and insightful summaries. In model has achieved a RougeL score of 41.2 on the CNN/Daily
general, the T5 model is a better option for abstractive text Mail dataset, which is considered to be a good score. This is
summarization. 1.Transformers can also be fine-tuned Subash because the T5 model has the ability to generate summaries with
Voleti, Chaitan Raju, and Teja Rani Mugada Swetha : Text higher semantic coherence and accuracy by using its powerful
summarization project developed utilising natural language pre-training and fine-tuning algorithms.
processing and Google text-to-speech API. It is extractive (no
paraphrasing) and employs the TextRank algorithm, which are T5 Transformer has been proven to outperform both the BART
both negatives. TextRank employs graph-based techniques and and Pegasus models for abstractive text summarization tasks. On
heuristics, whereas T5 is a transformer-based model.T5captures the CNN/Daily Mail dataset, the T5 model earned a RougeL
contextual linkages and dependencies between words in a score of 41.2, which is considered a good score. Because of its
sentence or page better than TextRank.T5 is trained on large- sophisticated pre-training and fine-tuning techniques, the T5
scale data to understand linguistic representations, whereas model can provide summaries with improved semantic coherence
TextRank depends on heuristics. TextRank cannot be fine-tuned f and correctness.
or s pecific do mains or ta sks, wh ereas T5 ca n. T5 is a more However, it is worth noting that the TextRank algorithm, an
advanced and successful text summarising method than unsupervised approach for text summarization, has demon-strated
TextRank. 2. Pooja Batra; Sarika Chaudhary; Kavya Bhatt; good performance on some datasets. The TextRank algorithm
Saloni Varshney; Srishti Verma : I started a project. A Review selects key sentences from the input text using graph- based
of Abstractive Text Summarization Techniques Using Natural ranking methods. While the RougeL score obtained by the
Language Processing. The disadvantage was that they were TextRank algorithm is lower than that of the T5, BART, and
performing abstractive summarization using LSTM and RNN. Pegasus models, it has demonstrated good performance on
Transformers outperform LSTM and RNN for text summarising particular datasets.
because they can more efficiently understand relationships
between words in a sentence or document, making them more In conclusion, T5 Transformer has proven to be the best model
accurate and capable of producing more insightfulsummaries. for abstractive text summary tasks because to its high RougeL
score and ability to generate summaries with higher semantic
1. Subash Voleti, Chaitan Raju, Teja Rani M. Swetha: Developed coherence and accuracy. While BART and Pegasus models have
a project Text summarization using natural language processing also performed well, they fall short of T5. Furthermore, while the
and google text-to-speech API. The disad- vantages where it is TextRank algorithm, an unsupervised approach for text
extractive (no paraphrasing) and it uses the TextRank algorithm. summarization, has demonstrated good performance on some
While T5 is a transformer-based model, while TextRank uses datasets, it falls short of the performance of supervised models
graph-based approaches and heuristics. such as T5, BART, and Pegasus.
T5 captures contextual relationships and dependencies between
words in a sentence or document better than Tex- tRank.T5 is V. PROBLEM STATEMENT
trained on large-scale data to learn representations of language,
Over the last decade, the volume of textual material on the
whereas TextRank relies on heuristics. T5 can be fine-tuned f
Internet has grown at an exponential rate. Because data expands
or s pecific do mains or ta sks, wh ile TextRank cannot. T5 is a
in vast numbers at once and includes useless content or noise,
more sophisticated and effective approach to text summarization
information utilisation has become an expensive and time-
than TextRank.
consuming task. Text summarization is a frequent technique for
dealing with large amounts of data. Automatic
2. Batra, Pooja; Chaudhary, Sarika; Bhatt, Kavya; Varshney,

text summary is a well-known method for distilling a docu- used to create the web application, using which any user can
ment’s primary points. It works by compressing the text and generate summaries from textual data.
preserving important information. summarising can help many
downstream applications by compressing enormous amounts of
information into brief summaries, such as making news digests,
report generation, news summarizing and headlinedevelopment.
VIII. DESIGN
VI. METHODOLOGY
This project employs Natural Language Processing for
automatic text summarization. We use the CNN DailyMail A. Data Flow Chart (Application)
Dataset trained on the T5 transformer model. The CNN Dai-
lyMail Dataset is an English-language dataset including over
300,000 unique news stories authored by CNN and the Daily
Mail journalists. T5, or Text-to-Text Transfer Transformer, is a
Transformer-based architecture that works with text. The dataset
is separated into three sections: training, testing, and validation.
The training set has 287,113 articles, the testing set has 11,490
articles, and the validation set has 13,368 articles.
The dataset’s characteristics are article and highlight, where

article is a string holding the body of the news story and
highlights is a string containing the article author’s highlight
(considered the summary).Before developing the model, the
dataset is pre-processed. In this phase, we tokenize, lowercase,
and remove stop words and quotations.
The training data is used to train the T5 model. We utilise
Rouge Score as a performance metric to evaluate the model’s
performance. The Rouge Score is a set of criteria used to
assess the quality of artificial text summarising systems, notably
abstractive text summarization.
The training phase is subsequently followed by the testing
phase, in which we employ the reserved text data for testing.
Following testing, the model is used to construct a REST API
that is delivered to the web application and allows any user to
generate summaries from textual input.
VII. PROPOSED SYSTEM
The project proposes developing a web-based text sum-

Fig. 2. Data Flow Chart (Application)
marising program that employs the abstractive text summa-
rization technique to extract the most precise and useful infor-
As shown in Fig.3 The purpose of this process is to extract a
mation from a document while removing extraneous material.
summary that focuses only on the most important points from a
This is accomplished by combining the CNN Daily Mail dataset
large amount of data. To begin, the user must either login or
with T5 (Text-to-Text Transfer Transformer), a cutting- edge
proceed as a guest. Input can be provided in one of three forms:
model that turns every language difficulty into text-to- text
text or document. The model will then process the input and
format. The collection contains 3,11,971 new articles. The
provide a summary of the given information as output. If the
dataset is separated into train, test, and validation data. Before
user has not registered, the output can be downloaded. Otherwise,
constructing the model, the dataset is pre-processed. In this
it can be saved in the database upon completion of registration.
phase, we tokenize, lowercase, and remove stop words and
Once the process is complete, the user can create as many text
quotations. The T5 model is trained using training data and
summaries as desired and save them in the database if they are
assessed using the Rouge Score. The training phase is
logged in as a registered user or download them directly in .txt
subsequently followed by the testing phase, in which we employ
format.
the reserved text data for testing. This model is further

B. Data Flow Chart (Model) A use case diagram illustrates the step-by-step process of how a
user can interact with our web application and how the system
will handle various input data scenarios to provide proper output
in the form of summarized text to the user. This diagram
visually represents the different use cases of the application,
which includes a user’s ability to login, register, upload input
from a PDF, save output as a .txt file, or simply download the
generated summary.
The use case diagram is an important tool used in software
development to help identify and define the application’s
functional requirements. It helps developers understand the
different actions a user can take in the system and how the system
should respond to those actions. The diagram provides a clear
overview of the application’s functionality, making it easier for
developers to design, develop, and test the system.
In the context of a web application, a use case diagram can help
identify potential user scenarios and streamline the user
experience. For example, if the user’s ability to login is a key use
case, the diagram can help identify any potential issues with the
login process and allow developers to optimize it for a smoother
user experience.
Overall, a use case diagram is an important tool for under-
standing and designing software systems. It helps ensure that the
application meets user needs and requirements and provides a
clear understanding of how the system will function.
Fig. 3. Data Flow Chart (Model)
As shown in Fig.4 To begin the process, input the data into the
provided box in the form of a document or text. The next step is
to preprocess the input data by performing the following
operations:(i) Tokenization: This involves splitting the entire data
into tokens, meaning splitting paragraphs into sentences and then
those sentences into words. This step is crucial for the
subsequent parsing process of NLP. (ii) Lowercasing: Converting Fig. 4. Use Case Diagram
uppercase letters to lowercase helps to standardize the text and
make it easier to compare and analyze. (iii) Removal of stop
words: Stop words like articles are removed to create a summary
that focuses only on important points. This helps to improve the
efficiency and accuracy of the model. Our model works based on
the Rouge score, Rouge Score is a performance metric used for
evaluating automatic summarization by comparing produced
summary against a set of references (human-produced). After
completing all the steps in the procedure, the desired output is
generated: the most accurate summary, covering all the important
points. Thismarks the end of the process.

IX. RESULT D. Saved Summaries

A. Main Page S
Fig. 8. Saved
Summaries
Fig. 5. Main Page
The user’s inputted data get processed by our model and it
generates a summary. Users can download the generated
Users can login or register using the Main Page.
summary in the form of a .txt file. If the user has registered, he
can save his generated summaries in the database.
B. Generated Summary
E. Imported PDF
Fig. 6. Generated
Summary Fig. 9. Imported PDF
The user’s inputted data get processed by our model and it The user can import the document file which needs to be
generates a summary. processed to generate summaries.
F. Upload PDF
C. Exported File
Fig. 10. Upload PDF
the user can upload the document file as PDF which needs to be
processed to generate summaries. if the user has registered he
can save his generated summaries in the database. Users can
download the generated summary in the form of a .txt file.
Fig. 7. Exported File

The user’s inputted data get processed by our model and it
generates a summary. Users can download the generated
summary in the form of a .txt file.

G. Summary from PDF between the system-generated summary and the reference
summaries, allowing researchers and developers to compare
different systems and to track improvements over time.
ROUGE-1 ROUGE-1 Precision and Recall compare the

similarity of uni-grams between reference and candidate sum-
maries. By uni-grams, we simply mean each token of com-
parison is a single word.
For instance, we can express the uni-grams of ‘John

loves data science’ as a Python list of tokens:
[’John’,’loves’,’data’,’science’]
RECALL By Recall, we simply refer to the proportion of

words in the reference summary captured by the candidate
summary. Recall = TruePositives / (TruePositives + FalseNeg-
Fig. 11. Upload PDF atives)
the user can upload the document file as PDF which needs to PRECISION By Precision, we refer to the proportion of words
be processed to generate summaries. After uploading the PDF, suggested by the candidate summary that actually appear in the
we can generate a summary of that PDF too. reference summary. Precision = TruePositives
/ (TruePositives + FalsePositives)
X. EVALUATION METRICS
Rouge (Recall-Oriented Understudy for Gisting Evaluation) is ROUGE-2 ROUGE-2 Precision and Recall compare the
a set of metrics used to evaluate the quality of automatic text similarity of bi-grams between reference and candidate sum-
summarization systems, particularly in abstractive text maries. By bi-grams, we mean each token of comparison is 2
summarization. Rouge scores measure the similarity between the consecutive words from the reference and candidate summaries.
generated summary and a set of human-generated reference For instance, the bi-grams of ‘John loves data science’ can be
summaries. expressed as the following tokens: [’Johnloves’,’loves data’,’data
science’]
There are several variations of Rouge scores, such as Rouge- N,
Rouge-L, and Rouge-S. Rouge-N measures the n-gram overlap ROUGE-L ROUGE-L Precision and Recall measures the
between the generated summary and the reference summary, Longest Common Subsequence (LCS) words between reference
while Rouge-L computes the longest common subsequence and candidate summaries. By LCS, we refer to word tokens that
between them. Rouge-S is a variant of Rouge that focuses on are in sequence, but not necessarily consecutive. To understand
sentence-level similarity. this, let us look at a convoluted example with reference summary:
John really loves data science very much and studies it a lot. And
In Rouge-S, the system-generated summary is compared to the the candidate summary (italicised words represent LCS): John
reference summaries based on sentence-level skip-bigram co- very much loves data science and enjoys it a lot.
occurrence statistics. The skip-bigram co-occurrence statistics
measure the degree of similarity between sentences based on the Semantic Similarity Semantic Similarity, or Semantic Tex- tual
number of overlapping skip-bigrams (i.e., pairs of non- Similarity, is a task in the area of Natural Language Processing
contiguous words) in the sentences. Rouge-S is particularly (NLP) that scores the relationship between texts or documents
useful for evaluating abstractive summarization systems that using a defined metric. Semantic Similarity has various
generate new sentences rather than just selecting sentences from applications, such as information retrieval, text summarization,
the source text. sentiment analysis, etc.
To compute Rouge-S, the system-generated summary and the There have been a lot of approaches for Semantic Similarity.
reference summaries are first tokenized and then the skip- The most straightforward and effective method now is to use
bigrams are extracted from each sentence. The skip-bigrams are a powerful model (e.g. transformer) to encode sentences to get
then compared between the system-generated summary and their embeddings and then use a similarity metric (e.g. cosine
each of the reference summaries. The Rouge-S score is the similarity) to compute their similarity score. The similarity score
average of the F1 scores for all the reference summaries. indicates whether two texts have similar or more different
meanings.
Rouge scores are widely used in the field of natural language
processing to evaluate the quality of text summarization sys- T5 (Text-to-Text Transfer Transformer) is a state-of-the-art
tems. They provide a quantitative measure of the similarity Transformer-based model that has achieved impressive results in
many NLP tasks, including STS and text summarization. T5

is a text-to-text model, which means that it can be fine-tuned on a unlike previous transformer models, is a unified text-to-text
wide range of NLP tasks by framing them as text-to-text transformer that can be fine-tuned for a variety of purposes,
problems. including summarization. T5 has a large pre-training corpus,
allowing it to master a wide range of language problems, making
To use T5 for STS, the model is fine-tuned on a specific STS it extremely successful at abstractive summarization.
dataset, such as the STS-Benchmark dataset, which consists of Furthermore, T5 features a decoder-only architecture, allowing it
pairs of sentences with human-labelled similarity scores. During to generate summaries in a single pass, which makes it faster and
fine-tuning, the input to the model consists of two concatenated more efficient. T5 has also outperformed other state- of-the-art
sentences separated by a special separator token. The model models on numerous benchmark datasets for abstractive
learns to predict a similarity score between 0 and 5 for each summarization, attaining the highest ROUGE scores.
sentence pair. Furthermore, T5’s capacity to generate summaries based on
T5 has several advantages over other STS models, includingthe prompts enables it to write summaries for certain domains or
ability to handle long input sequences, the ability to generate text tasks, which is a useful feature for real-world applications.
as well as score text, and the ability to generate diverse and Overall, the T5 transformer model’s remarkable performance,
informative summaries. adaptability, and speed make it one of the finest models for
abstractive text summarising tasks.
In text summarization, T5 can be fine-tuned on a sum-
marization the output is the summary. T5 has been shown to XII. CONCLUSION
outperform previous state-of- the-art models in text
Text summarization is a significant challenge since there is a
summarization, achieving high scores on various evaluation
large amount of information that grows every day, and collect-ing
metrics, including ROUGE.
the important data from it is a time-consuming procedure. We
Overall, the use of T5 in STS and text summarization has created a web-based tool for summarising by implementing the pre-
led to significant improvements in the quality of text trained language model T5, which is based on the transformer
summarization systems. By accurately measuring the semantic architecture. Using a Deep Learning model, the output is more
similarity between sentences, STS models can be used to select exact and accurate than other approaches. The abstractive
the most relevant and informative sentences for inclusion in a approach has been demonstrated to be the most effective
summary, resulting in more coherent and informative summaries. summarising strategy. Automatic text summarising assists users
T5’s ability to generate diverse and informative summaries in extracting vital information from large amounts of data.Our
makes it a powerful tool for text summarizationand other NLP research revealed that finely tailored transformers- based pre-
tasks. trained language models produced excellent results, producing a
sound and fluent summary of a given text material. We estimated
XI. EXPERIMENT RESULT AND DISCUSSION ROUGE scores [21] for each model’s predictions for comparison
studies and determined that the T5 model beat all other models
A. Result
based on the transformer architecture for the job of
T5 Transformer, an advanced text-to-text transformer model, summarization.
was trained using CNN Daily Mail dataset for abstractive text
summarization. The model achieved a ROUGEL score of 41.2 REFERENC
after training on 287,113 news items, which is a decent result. ES
ROUGE-L is a popular text summarising evaluation metric that
[1] Anushka Gupta, Diksha Chugh, Anjum, Rahul Katarya,
calculates the longest common subsequence of words between the
“Automated News Summarization Using Transformers”,
generated summary and the reference summary. A greater
Delhi Technological Uni- versity, New Delhi, India 110042,
ROUGE-L score denotes a higher quality of the resulting
2021.
summary. While a ROUGE-L score of 41.2 is commendable, [2] Minh-Tien Nguyen, Van-Chien Nguyen, Huy-The Vu, Van-
there is always space for growth. Various strategies can be used Hau Nguyen, “Transformer-based Summarization by
to improve the score, such as using larger training datasets, Exploiting Social Information”, The 2020 12th International
including more complicated model architectures, and fine-tuning Conference on Knowledge and Systems Engineering (KSE),
the model with domain-specific data. Overall, the T5 Transformer 2021.
model’s high ROUGE-L score on the CNN Daily Mail dataset [3] Ishitva Awasthi, Kuntal Gupta, Sahejpreet Singh Anand,
suggests its ability in creating high-quality abstractive Prabjot Singh Bhogal, Prof. Piyush Kumar Soni, “Natural
summaries. Language Processing (NLP) based Text Summarization - A
Survey”, Proceedings of the Sixth In- ternational Conference
B. Discussion on Inventive Computation Technologies [ICICT 2021].
[4] Rahul, Surabhi Adhikari, Monika, “NLP based Machine
The T5 transformer model has emerged as the cutting- edge Learning Ap- proaches for Text Summarization”,
architecture for abstractive text summarising tasks. T5, Proceedings of the Fourth Interna- tional Conference on
Computing Methodologies and Communication (ICCMC

2020).
[5] Manish Shinde, Disha Mhatre, Gaurav Marwal,
“Techniques and Re- search in Text Summarization – A
Survey”, 2021 International Confer- ence on Advance
Computing and Innovative Technologies in Engineer- ing
(ICACITE 2017).
[6] 5) 6) Subash Voleti, Chaitan Raju, Teja Rani,Mugada
Swetha “Text Summarization Using Natural Language
Processing And Google Text To Speech Api”, 2020
International Research Journal of Engineering and
Technology (IRJET).
View publication stats

Abstractive Text Summarization Using Transformer Based Approach

Uploaded by

Abstractive Text Summarization Using Transformer Based Approach

Uploaded by

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Abstractive Text Summarization Using Transformer Based Approach

Research · June 2023

Karishma Shukla Dnyaneshwar Bavkar

SEE PROFILE SEE PROFILE

The user has requested enhancement of the downloaded file.

Abstractive Text Summarization Using Transformer Based Approach

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 1

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 2

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 3

The dataset’s characteristics are article and highlight, where

VII. PROPOSED SYSTEM

The project proposes developing a web-based text sum-

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 4

Fig. 3. Data Flow Chart (Model)

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 5

IX. RESULT D. Saved Summaries

Fig. 10. Upload PDF

Fig. 7. Exported File

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 6

ROUGE-1 ROUGE-1 Precision and Recall compare the

For instance, we can express the uni-grams of ‘John

RECALL By Recall, we simply refer to the proportion of

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 7

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 8

© 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM22369 | Page 9

View publication stats

You might also like