Ajeenkya DY Patil School of Engineering, Pune

Department of Artificial Intelligence & Data Science Engineering

AY: 2024-25 Class: TE SEM-I

A Presentation
on

AI Assistant For Handicap

Presented by:
Guided by:
Name :Rohan Satpute
Gopika Fatthepurkar
Pranav Gadage
Soham Chattar
1
 Agenda
 Introduction
 Motivation

 Problem Statement

 Literature Survey

 System Architecture/Algorithm

 Conclusion

 Future Scope

 References

2
 Introduction

The project demonstrates the integration of advanced speech

recognition, natural language processing, and text-to-speech
technologies to create a voice-activated assistant. The assistant is
designed to recognize user speech, process it using OpenAI’s GPT-3,
and provide real-time responses through voice. This system leverages
the power of GPT-3 for intelligent, human-like conversations and
incorporates speech-to-text and text-to-speech components to ensure
smooth, hands-free interaction.

3
 Motivation
The primary goal of this project is to develop a voice-activated assistant that
can recognize spoken commands, process them using OpenAI’s GPT-3 model, and
then respond with human-like speech. The motivation behind this project stems from
the desire to make technology more accessible and intuitive through natural language
interaction. In addition, there is a growing trend in the tech industry towards voice
interfaces, with virtual assistants such as Siri, Alexa, and Google Assistant becoming
an integral part of everyday life. However, while these assistants are functional, many
are still limited in their conversational depth and understanding.

Voice interfaces provide a significant advantage in terms of ease of use,

especially for individuals who may face physical challenges that make typing or using
traditional input devices difficult. For example, people with disabilities, the elderly, or
those in hands-free environments can benefit greatly from a system like this. By
integrating speech-to-text and text-to-speech technologies, the system becomes
accessible to a wider audience.
4
 Problem Statement

Voice assistants like Siri, Alexa, and Google Assistant excel at simple tasks but
struggle with complex, context-aware interactions, causing user frustration. The
challenge is developing systems that can handle diverse, spontaneous requests with
relevant, intelligent responses

5
 Literature Survey :
Sr No Details Of Paper Problem Identification Paper approach for the Results/Datasets
problem

1. Veeresh Ambe; As per the World Health Organization This paper proposes an intelligent text is converted to
Prayag Gokhale; (WHO), 285 million people are visually text reader using python. This speech by a Python
Vaishnavi Patil; impaired of whom 39 million are completely product is built on a Raspberry Pi based TTS (text to
blind. Though there exist enough remedies module connected with camera speech) conversion unit
Rajamani M. Kulkar to the problems of assisting individuals who that is used to capture the input
ni embedded in the
; Preetam R. are visually impaired to read, there is a image. The input image is Raspberry Pi. Finally,
requirement for an intelligent text reader enhanced using Image processing the audio output is
Kalburgimath which is economical, accurate and easily techniques. given to the Audio
accessible in order to help them read for day Amplifier for it to be
to day activities. read out.
2. Hasan U. Zaman; The idea of this paper is to build an The whole bodywork is also Conversion of text
Saif Mahmood; automated virtual reader. In this integrated with Optical to speech can also
Sadat Hossain; modern era, there is an urge for an Character Recognition (OCR), be done in Matlab
Iftekharul automated reader which is cost- Text-To-Speech (TTS) and a but that wouldn't be
Islam Shovon effective, accurate and also portable at speaker. portable and user-
the same time. friendly.
Sr Details of paper Problem Paper approach for the Results/Datasets
No Identification problem
.
3. Elan Markowitz1,2 Zheng Chen1 Multi-domain We propose to unify these Our proposed methods have
Ziyan Jiang1 Fan Yang1 Greg Ver recommendation approaches: using information been proven effective in a
Steeg1,3 Xing Fan1 Aram leverages users’ from interactions in other scalable ,real-world Ai
Galstyan1, assistant usecase, bringing
interactions in previous domains as well as external
domains to improve knowledge graphs to make significant benefits to
personalized AI assistants
recommendations in a predictions in a new domain
new one. that would not be possible with
either information source
alone.

4. Naveenkumar T. Rudrappa; Speech processing Research on speech processing The method utilizes the
Mallamma V. Reddy; M. embeds the recording mandates recording, storing, operating system
Hanumanthappa of speech which is a playing and analyzing a wide functionality to activate
huge container of variety of spoken languages the microphone for
private, confidential specifically for Indian context. recording, hard disk for
and business records, storage and activation of a
used for a wide variety speaker for voice output.
of applications like
health care, customer
services and individual
identification.
 Algorithm/Mathematical Model
Algorithm for Voice-Activated Assistant

[Link]-to-Text (STT):
- Capture audio from the microphone.
- Convert audio to text using a speech recognition model (e.g., Google Speech API).

2. Natural Language Understanding (NLU):

- Tokenize the text.
- Recognize entities (e.g., dates, locations).
- Identify user intent (e.g., query, command).
- Analyze context.

3. Contextual Query Processing with GPT-3:

- Format the input as a prompt for GPT-3.
- GPT-3 generates a response based on the input.

8
 Algorithm/Mathematical Model

4. Text-to-Speech (TTS):
- Convert GPT-3’s response into speech using a TTS engine.
- Play the audio response to the user.

5. Continuous Loop:
- Wait for new input.
- Repeat the process until the user ends the session.

9
 Conclusion
The voice-activated assistant system leverages state-of-the-art technologies such as
speech recognition, natural language processing, and text-to-speech synthesis to create a
seamless and interactive user experience. By integrating powerful models like OpenAI’s
GPT-3, the system is capable of understanding and generating human-like responses to a
wide variety of voice commands, enabling dynamic conversations with users. The
continuous loop of capturing audio, processing it for meaning, generating an appropriate
response, and then converting that response back into speech allows for natural and
intuitive interactions. Despite the impressive advancements, challenges remain in
improving context retention, handling ambiguous queries, and ensuring ethical responses.
However, the future of such voice-driven systems is promising, with potential applications
across numerous industries, from customer service and healthcare to smart home
management and education. As research progresses, these systems will become
increasingly adaptive, intelligent, and capable of understanding complex, real-world
interactions.
10
 Future Scope
The future scope of voice-activated assistant systems is vast and continually evolving,
driven by advancements in artificial intelligence, machine learning, and natural language
processing. As speech recognition accuracy improves, these systems will be able to
understand a wider range of accents, dialects, and languages, making them more
accessible to a global audience. Furthermore, with the integration of more advanced
contextual understanding and emotional intelligence, future systems will be able to
engage in more meaningful and empathetic conversations. Enhanced voice assistants
could also have broader applications in various fields such as healthcare, where they can
assist with diagnostics, patient monitoring, and elderly care, or in education, providing
personalized tutoring and learning experiences. With advancements in multi-modal AI,
where voice assistants integrate with visual data (e.g., via cameras or augmented
reality), they could expand their functionalities to handle more complex tasks, such as
object recognition or real-time translation. The future holds great promise for
developing more intuitive, secure, and intelligent voice-driven interfaces, enabling them
to become an integral part of daily life and transforming industries worldwide. 11
 References
[Link] GPT-3 Documentation
OpenAI. (2021). "GPT-3: Language Models are Few-Shot Learners." Retrieved from
[Link]

[Link] Speech-to-Text API Documentation

Google Cloud. (2021). "Speech-to-Text Documentation." Retrieved from
[Link]

3. TechCrunch: "The Future of Voice Assistants: Trends to Watch" (2022). Retrieved from
[Link]

[Link] 2: Generating Human-like Speech from Text

Wang, Y., et al. (2017). "Tacotron: Towards End-to-End Speech Synthesis." Google Research. Retrieved
from [Link]

[Link] Speech Recognition Library

SpeechRecognition Documentation. (2021). "SpeechRecognition: Recognizing Speech from Audio."
Retrieved from [Link]
12
13