Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add kvcache support to debug_analysis.py and engine.py #1132

Merged
merged 6 commits into from
Jul 21, 2023
Merged

Add kvcache support to debug_analysis.py and engine.py #1132

merged 6 commits into from
Jul 21, 2023

Conversation

alexm-neuralmagic
Copy link
Contributor

Adds the ability to run kvcache via the debug_analysis.py.

SageMoore
SageMoore reviewed Jul 21, 2023
View reviewed changes
Copy link
Contributor

@SageMoore SageMoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diff looks reasonable to me, Alex. I think once we resolve the formatting issues we should be good to merge.

src/deepsparse/debug_analysis.py Outdated Show resolved Hide resolved
@alexm-neuralmagic
Copy link
Contributor Author

Formatting issues are resolved (thanks to Michael)

bnellnm
bnellnm reviewed Jul 21, 2023
View reviewed changes
src/deepsparse/engine.py Outdated Show resolved Hide resolved
bnellnm
bnellnm reviewed Jul 21, 2023
View reviewed changes
src/deepsparse/engine.py Outdated Show resolved Hide resolved
bnellnm
bnellnm reviewed Jul 21, 2023
View reviewed changes
Copy link
Member

@bnellnm bnellnm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Just a few minor comments

bnellnm
bnellnm requested changes Jul 21, 2023
View reviewed changes
Copy link
Member

@bnellnm bnellnm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to hit "request changes" on my last round of comments. I couldn't figure out how to go back and change it.

bnellnm
bnellnm previously approved these changes Jul 21, 2023
View reviewed changes
mgoin
mgoin reviewed Jul 21, 2023
View reviewed changes
src/deepsparse/utils/onnx.py Outdated Show resolved Hide resolved
@mgoin mgoin merged commit 0ae6aa7 into main Jul 21, 2023
@mgoin mgoin deleted the kvcache_debug_analysis branch July 21, 2023 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SageMoore SageMoore SageMoore left review comments

@bnellnm bnellnm bnellnm approved these changes

@mgoin mgoin mgoin approved these changes

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@alexm-neuralmagic @mgoin @SageMoore @bnellnm