I

I
ORAC LE
University

Oracle Cloud Infras truct ure

Retr ieva l Aug men ted Gen era tion

OCI Generative Al
 •·
-

Retrieval Augmented Generation

-
Retrlnal-Augmellltd Generadon for
Kaowledge-lDtemhe NLP 1ub

'r.•:• .. ..

'"-. .. ~
fl-•-
•-AJ-'V-.io,c.a.,..-·----,; • Retrieval Augmented Generation (RAG)
is a method for generating text using
additional information fetched from an
external data source.

.. ~

0 RAG models retrieve documents and

pass them to a seq2seq model.
RAG Framework
-

information
or database
Retriever
Function: Sources
from a large
relevant
corpus
Ranker
Function: Evaluates and prioritizes
the information retrieved by the
retriever
+ Generator
Function: Generates human-like text based
on the retrieved and ranked information,
along with the user's original query it receives

Workinc: Uses retrieval techniques Working: Uses various algorithms to Working: Uses generative models to cratt
evaluate the quality of the retrieved ~ human-like text that is contextually relevant,
Purpose: Provides the generation content coher,nt, and informative
system with contextually relevant,
accurate, and up-to-date Purpose: Ensures that the Purpose: Ensures that the final response is
information that might not be generation system receives the most factually accurate. relevant, and also
present in the model's pre-trained pertinent and high-quality input coherent, fluent, and styled typically like
knowledge human language
RAG Techniques
--
C,
•I
For each input query (like a chapter topic), the model retrieves a set of
I
I
relevant documents or information.
I
I
I
I
I

•
I
It then considers all these documents together to generate a single, cohesive
RAG Sequence response (the entire chapter) that reflects the combined information.

•'
•
I
I
For each part of the response (like each sentence or even each word),
the model retrieves relevant documents.
I
I
I
I
I

•
I
The response is constructed incrementally, with each part reflecting
RAG Token the information from the documents retrieved for that specific part.
RAG Pipeline
-
RAG Application
-
· what are my
-What~ tM policy?" corporate benefits?"

Prompt Enhanced Embedding Augmented

Prompt Model Prompt

+
Chat History
1 r
T
Private content ""'
1
HIGHLY
Similarity Search ~~ ,;
ACCCRATE
RESPONSE

Fetch docs for matching IDs

Vector ID
Matches

Relational Database
 - ·- .

RAG Evaluation
-

Answer Relevance Context Relevance

Is the answer relevant RAG Triad Is the retrieved context
to the query? relevant to the query?

Groundedness
RESPONSE Is the response supported
by the context?
CONTEXT

Source: truera; https://truera.com/ai-quality-educatlon/generatlve-al-r1gs/what-ls-the-rag-triad/

ORACLE
University

Oracle Cloud Infrastructure

Vector Databases
-
OCI Generative Al

Himanshu Raj
SENIOR INSTRUCTOR Al/ML
ORACLE UNIVERSITY
 LLM Versus LLM + RAG
- f ~tMI
0
II
LLMs withou t RAG rely on
interna l knowledge learned
• ···-•••in ino 0 rino tunin9 4Jf.~ 4
QMry
C ,

a
~,,l
I
I R4t•pon••
I during pre-tra ining on a large Gigantic dataset Base LLM Flnetune d LLM User

I corpus of text. It may or may

not use Fine-Tuning.
web, wild, books, etc.
-.-!
... .II Org/dom ain
'a. \. specific dataset
r
\ I
"' ·•. , I
·. t
[ LLM wi} h RAG
.. ...'
' 'j I
~ I
LLMs with RAG use an
external database, which BaseLLM QJASyste m

1QJ ~
is a Vector Database.
0
Query
• Jte•pon••
Pro-trai nino
1'•apen••
..
Gigantic dataset '-
web, wiki, books, etc.
Query•
relevant docs
l User

Vector DB
 Vector
-
Vector
':) A vector is a sequence of numbers,
DATA I
I
I ca11ed dimensions, used to capture
I

~
I
I
I
I
the important "features" of the data.
I
I

•
I
I

Embeddings in LLMs are essentiaJly

• •
I
I
I
I
high-dimensional vectors.
I
I
I

:/ ~II]
I
I

•
I

Vectors are generated using deep

learning embedding models and
. represent the semantic content of data,
,-•
.. not the underlying words or pixe1s.
..
 •

Vector
- •
I
There are three groups of words
! here based on similarity:
!
I
animals, fruit, places.
I
I
elephant
e
I
I

dog tlion "Tiger" is closest to the Animals

f ca .. tiger group, closer to cat family members.
puppykitien

Optimized for multidimensi onal spaces

New York where the relationship is based on
Maine distances and similarities in a high-
Vermont dimensional vector space

Word relatedness in two dimensions

Embedding Distance
MNIST Data 20 fmbeddif'l9
Dot Product 60
t

•
A.B = ,n
Lt=1
Ai Bi
40

20

Cosine Distance
-20
A.B
l - llAII llBII -40

0
-40 -30 -20 -10 0 10 20 lO
t-SNl!FHturitl
 oracle.com - To exit full screen, press ~
Similar Vectors
Data vectors

Efficient and robust approximate nearest

neighbor search using Hierarchical Navigable
Small Wor1d graphs
_____________ .................,.......,....
,..... __________ _
---_______
-·- --
-·-
...
________
,...._.,.. .,..

.-.-------~---------..- 4------~--- Ou t ., , t( ~

--------------------·--
-----~-·--------·
------
........
___ ..__..,_..,.. ___ _,...-.-----
....._----·---·-·-----
___________ -.-----
---
_____.........-_
==-...:.=:-::~........
~------·- ~
:r.=-~·-T.::..--=:.--
------.--------
_
._..
---------._,....,.._.,_,.
...,,_.,.._.,. __
........_,_ _____ ~-----
...,..................... --..-...-..-
...,._ _____
::~:;-~•
_,.._
... ____
-:. _3::
..,_,------·-·----
_ .............. ....-

....- ....,- ....- •- -______

.. .........__.,__....,_,..._.
______
__,_,_._.
..-.
-..---.----..-.-
-
------
......_.....__,_,..
_....., ....
, ____ ,,_ ._
---·--·---
..__ _
~~,.;,:.-.:..•::::--.:-.::
===-===-:::.:::·..:.:.:: queryvedor
I -

_..__
- --------·-•-
:a!'---=-.----===-"!:
·-
c:=-~----=-=-~-=
.................... ....... Oc,,• So..rco
------·
---··-··--·--
---·-··---
-·----·-- -=--=-~....
............................--.__., :.::t=..--. - .
--.-----~
:--=.;;.~_:;::.· -
·------·- .. __
--~...............- --------
---_____
.. ____
~ , . . . . . . . . . . . . . . . . . &-Mia_

:-.c:-.:.::.=::::-:-..=-.:
........... 0 K-Nearest Neighbors algorithm can be used to perform
...- ~------.~ ..., g spoilty/ annoy ,_....

<>0.0 0 - ..
n ,.._ • 0 - -=:.:-=------
_""""
le _...;._______,____
._
i
I
I
a vector or semantic search to obtain nearest vectors
I
I in embedding space to a query vector.
I
I
'
·--p,A- I)
I
I
I
ANN algorithms are designed to find near-optimal
neighbors much faster than exact KNN searches.

·-
I
I

•
!

·-
·-
ANN metho~s such as HNSW, FAISS, Annoy are often
preferred for large-scale similarity search tasks in
embedding spaces due to their efficiency.
Vector Database Workflow
-

Vectors ----.-
• Indexing Vector
Database
- -~--1 Querying .. Post
Processing
Vec tor Databases
-

ORA CLE
A~Vector Search

• Chroma w Weaviate
Vector Databases
-
Accuracy
(!J Pinecone
ORACLE Latency
Al Vector Search

• Chroma w Woavlato ~~ , -·-

• Scalablllty
l
 Role of Vector Databases with LLMs
-
Q Address the hallucination (i.e., inaccuracy) problem inherent in LLM responses.
•
•'•
0' Augment prompt with enterprise-specific content to produce better responses.
•I
•'
•
• Avoid exceeding LLM token limits by using most relevant content.
.. ..-
.". -..
.. .. ORACLE
_.
-
LLM
+
Broad range of data
from the internet
Private enterprise data
~~ af elm"°"'. powtt In tine Curl'lf\t dltl • frequtndy Uoeiated
R o le o f V e c to r Databas
es with LL Ms
-
'? C he ap er th an fin e- tu ni ng
LLMs, w hi ch can be expe
: nsive to update
'•
0 R ea l- tim e up da te d kn ow
l\ le dg e base
~
O" C ac he pr ev io us LL M pr om
pt s/ re sp on se s to improve perform
ance and reduce costs

B et te r bu si ne ss outcom
es

- -
LLM
-- - ~-
+ Ve ct or Database
 ORACL E
University

Oracle Cloud Infrastr ucture

I
Keyword Searc h
I -
OCI Generativ e Al

Himanshu Raj
SENIOR INSTRUCTOR Al/ML
ORACLE UNIVERSITY
Keyword Search G oogle
> Keywords are words used to match with
~--__.,.
'l - •1

0. , . . _ .. _ _ . . .
a. _ _ .. .._..
X G!

0. -
0. - - l- · -
the terms people are searching for, l)N e

0. -
0. _·_-
......
-

when looking for products, services, or <\ - - · -

o. - - • -
general information. 0. - - - - . . . . . . .

> Simplest form of search based on exact

.
Query: "What is the fastest
-
--
Com mon Words
matches of the user-provided keywords animal?"
-- -
in the database or index ' Response1: "Cheetahs are the
fastest of all land animals."
Response2: ~'The earth orbits the
> Evaluates documents based on the
presence and frequency of the query
!
-sun."- - - - -- - - - -
Response3: "Usain Bolt is the
0 ()

, fastest runner." 1 ('fastest')

term. For example, BM25.
 •
-
• •

ORA CLE
University

Oracle Cloud Infras tructu re

Semantic Search
-
OCI Genera tive Al

Himanshu Raj
SENIOR INSTRUCTOR Al/ML
ORACLE UNIVERSITY
 Semantic Search
-
Search by meaning
Google
> Retrieval is done by understanding
0. Wh1I It Se~
intent and context, rather than
matching keywords. ¥met It ~nUCI • I.. man{l)ill,/
o_ lhe brand! ol lt'flNICa and logic c,onc.emed ...at lftNlllnt. ni.rt . . a 111,mber
of lnnc:hM and ....,.IC:Ne ol ..-...nllc:e. ~ b,1111 eemanb, whkil
1111dlet the loglc.al ~ of fflMNIIO, audl u NIIM, ,.._,.,_, lmplcallan, •

.. .. Ways to do this: O. whit II Mrnando fMfflO,Y

- Dense Retrieval: Uses text embeddings 0. whit 1, Mfflln8 • .,,..
- Reranking: Assigns a ~elevance score 0. ¥mil 11 MmanUo encoding
0. whal 11 MmlnUo nol11
0. whit 11 Mmantlo memory In p1ychology
0. whit 11 NrnanUC ...rch
:
Embeddings
-
7

Dog
•I
I
I
I
I
I
Embeddings represent the meaning of
text as a list of numbers.
I
6 • I
I
I
I
• Capture the essence of the data in a lower-
5 dimensional space while maintaining the
semantic relationships and meaning.

-~·~ =-
4 '

Car •
3 • Words
[-17.216797, -14.016798]
2
Rose
•
Fruit
• •
[0.015594482, -
rose 0.0038833618, -0.0635376,
1 -0.07...

6
Oracle offers a free [0.02619934, -0.028915405.
0 1 2 3 4 5
0 pricing tier for most -0.015777588, -0.07...
Al services.
Embeddings in two dimensions b...-..-
Dense Retrieval
-
7

6 ~
•
(Doc6J
5
[Doc9l
~
•
•
• Relies on embeddings of both queries
and documents to identify and rank
• re1evant documents for a given query
4

3
[QodJ [Dcd]
• • Enables the retrieva1 system to
understand and match based on the
contextua1 simi1arlties between queries
• ---------- Query
~
•
2 and documents

1
~
•
-~
•
0
0 1 2 3 4 5 6

*Relevant Docs
Rerank
-
Query text Initia l Searc h Resu lts Relev ance Scor e New Rank ing

~2

Rerank
>

• Assigns a relevance score to (query, resp ons e) pair

s from initi al sea rch resu lts

0I Hig h relevance score pair s are mor e like ly to be corr

I
I
ect
I
I

0 Imp lem ente d thro ugh a trai ned LLM