Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference UX, accept input data #1285

Merged
merged 24 commits into from
Oct 2, 2023
Merged

Inference UX, accept input data #1285

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
6c8dfb3
[deepsparse.infer] UX improvements, data only mode
Sep 25, 2023
b4b7ec6
fix bug on main
Sep 25, 2023
17168f6
draft, load files line by line, return iter, save up memory
horheynm Sep 26, 2023
84b03f8
add inference
horheynm Sep 26, 2023
8a69c5f
pass passing in files
horheynm Sep 26, 2023
4091503
latest changes'
horheynm Sep 26, 2023
b0f65af
revert
horheynm Sep 26, 2023
8ee765b
make new folder for inderence
horheynm Sep 26, 2023
8a47e01
allow input to pass thru cli
horheynm Sep 26, 2023
6957067
Merge branch 'main' into infer-ux-fixes
horheynm Sep 26, 2023
b429917
Update src/deepsparse/transformers/inference/infer.py
horheynm Sep 27, 2023
1dc2ee3
remove hardcoded
horheynm Sep 27, 2023
86a2daf
better error message
horheynm Sep 27, 2023
939c6bc
clean up
horheynm Sep 27, 2023
274570b
Merge branch 'main' into infer-ux-fixes
horheynm Sep 27, 2023
7b1edfa
clean up, check kwargs
horheynm Sep 27, 2023
064013a
Merge branch 'infer-ux-fixes' of github.com:neuralmagic/deepsparse in…
horheynm Sep 27, 2023
8cab34f
Merge branch 'main' into infer-ux-fixes
horheynm Sep 27, 2023
4699837
get rid of breakpoint()
horheynm Sep 27, 2023
218b584
Merge branch 'infer-ux-fixes' of github.com:neuralmagic/deepsparse in…
horheynm Sep 27, 2023
ff4b48f
return type
horheynm Sep 27, 2023
8243740
Merge branch 'main' into infer-ux-fixes
horheynm Oct 2, 2023
2a4b972
typo
horheynm Oct 2, 2023
ad9e96d
Merge branch 'infer-ux-fixes' of github.com:neuralmagic/deepsparse in…
horheynm Oct 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove hardcoded
  • Loading branch information
@horheynm
horheynm authored Sep 27, 2023
commit 1dc2ee378e81ad3fc164ac2a2453689608987ead
1 change: 0 additions & 1 deletion src/deepsparse/transformers/inference/infer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,6 @@ def main(

if data:
for prompt, prompt_kwargs in _iter_prompt_from_file(data):
prompt_kwargs = {}
_run_inference(
pipeline,
sampling_temperature,
Expand Down