Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c# I need to run the program on NPU (OnnxRuntime + DirectML + NPU), #682

Open
Gusha-nye opened this issue Jan 14, 2025 · 0 comments
Open

Comments

@Gusha-nye
Copy link

Hello, big guys, I checked the Microsoft blog and did the following test(https://blogs.windows.com/windowsdeveloper/2024/02/01/introducing-neural-processor-unit-npu-support-in-directml-developer-preview/)

Following the demo of Microsoft(https://github.com/microsoft/Phi3-Chat-WinUI3-Sample), I can run phi3 model on GPU via Onnxruntime + DirectML. By using C#, I want to run program on NPU (OnnxRuntime + DirectML + NPU), but it failed.
I already install Microsoft.AI.DirectML 1.13.1 and the Microsoft.ML.Onnxruntime.DirectML 1.17.1 on VS 2022 Nuget
And I write C# test code in demo, as follows:
Image
And when '4> Index >= 0' output with [DmlExecutionProvider, CPUExecutionProvider]
Image
When I set 'Index=4', the error is occured as follows:
Image
when 'Index >= 5', the error is occured as follows:
Image

However, in the Microsoft demo, the loading of the phi3 model is also not configured using similar code as below
SessionOptions options = new SessionOptions();
options.AppendExecutionProvider_DML(Index)
Image

Therefore, how can the reasoning of phi3 be executed on the NPU?
Is there currently a demo that can reason about Phi3 via OnnxRuntime + DirectML on NPU? (C#, C++, python are all good)

My PC configuration as follows:
Processor: Intel(R) Core(TM) Ultra 7 165H 1.40GHz
GPU: Intel(R) Arc(TM) Graphics
NPU: Intel(R) AI Boost
Installed RAM: 32.0GB
System Type:64-bit operating system, x64-based processor
Windows Edition: Windows 11 pro
version: 24H2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant