Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat assistant response slow #2065

Open
IamHarri opened this issue Aug 23, 2024 · 4 comments
Open

Chat assistant response slow #2065

IamHarri opened this issue Aug 23, 2024 · 4 comments
Labels
question Further information is requested

Comments

@IamHarri
Copy link

IamHarri commented Aug 23, 2024

Describe your problem

I deploy ragflow (infiniflow/ragflow:v0.9.0) on aws eks. I have two nodes to run all the dependencies ( redis, mysql, minio, elasticsearch)
Nodes detail:
RAM: 64 GB
CPU: 8
GPU: 0
Disk: 100 GB
Physical Processor: Intel Xeon Platinum 8175:

Locally host ollama which has llama3:latest as chat model and mxbai-embed-large as embeded model. I have one node for it
Nodes detail:
RAM: 64 GB
CPU: 8
GPU: 0
Disk: 100 GB
Physical Processor: Intel Xeon 8375C (Ice Lake)
GHZ: 3.5

The parsing document workwell, It tooks around 5 mins for large document. but the chat assistant is very slow, I only say "hi" and it tooks 1 mins to search and reponse to me. Do you have any idea why It is slow like that, even my compute resource is big ?s

@IamHarri IamHarri added the question Further information is requested label Aug 23, 2024
@IamHarri IamHarri changed the title Chat assistant reponse slow Chat assistant response slow Aug 23, 2024
@KevinHuSh
Copy link
Collaborator

It might be caused by searching from ES whoes performance is highly related to RAM and docs it indexed.
One PDF can generate thousands of docs into ES.

@IamHarri
Copy link
Author

I have check ES resource consumption, It uses only 8GB of RAM. It still has a lots of available resources. Do you have any idea to optimize the performance of the assistant in my case (one simple question takes 2 mins and complex questions do not reponse )

@qiqi0308
Copy link

我检查了 ES 资源消耗,它只使用了 8GB 的 RAM。它仍然有很多可用的资源。在我的情况下,您有什么想法可以优化助手的性能吗(一个简单的问题需要 2 分钟,复杂的问题不需要回答)

metoo,did you solve it? if this ,what do you make it?

@KevinHuSh
Copy link
Collaborator

Click the lamp and check the time elapsed down there.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants