KATANA: A Fast, Low-Power Mapping of Kalman Filters onto Edge NPUs for Real-Time Tracking

Real-time state estimation is the closed-loop heartbeat of modern tracking systems. In defense, ground-based and airborne radars are mounted on vehicles, ships, and aircraft far from fixed infrastructure [1, 2], while counter-UAV and missile-guidance loops run on drones and interceptors that themselves move at high speed. In civilian deployments, the same tracking workloads run on cars (for advanced driver assistance), drones, robots, and handheld devices [3, 4], where the tracker has to live on-device for latency and connectivity reasons. All of these are edge platforms: battery-powered and either fanless or weight-constrained, so every additional watt of compute erodes mission duration or operational range, while the control loop must still close within milliseconds of each new measurement and scaled by the number of tracked targets. Across all of these settings, the LKF and its non-linear counterpart, the EKF, have been the dominant estimators for over six decades [5, 6]: optimal in the minimum-mean-square-error (MMSE) sense, recursive (hence streaming-friendly), and structured around dense linear algebra that any modern compute substrate should be able to execute.

The hardware to do so already exists. Contemporary client SoCs, such as the Intel Core Ultra Series 1 and Series 2, Apple M-series, and Qualcomm Snapdragon X, are now heterogeneous, integrating a CPU, a GPU, and a dedicated NPU optimized for low-power AI inference. Yet, despite this hardware diversity, the KF today runs almost exclusively on the CPU in commodity software stacks, or on custom FPGA/ASIC accelerators in dedicated tracking systems [7]. Three problems follow. First, MOT with $N$ independent filters serializes on the CPU, capping throughput and burning power. Second, designing a dedicated accelerator extends time-to-deployment and adds die area on top of an already-busy SoC. Third, the on-die NPU, a data-parallel matrix engine designed for sustained low-power operation, sits idle whenever no neural workload is feeding it. Mapping the KF to this otherwise-idle NPU yields a double dividend: it avoids a separate accelerator and keeps the CPU and GPU free for their primary general-compute and graphics workloads, so the tracker coexists with rather than monopolizes the SoC, making the end-to-end system more responsive.

This work therefore asks the natural question: can the KF be mapped onto the existing AI-PC NPU and made real-time at low active power, without custom silicon? Fig. 1 captures the overall premise: take a classical signal-processing algorithm such as KF-based tracking, target a heterogeneous AI-PC platform, and offload the recursive estimator to the on-die NPU. The challenge, however, is that NPUs are designed around dense multiply-accumulate (MAC) dataflow on the DPU; any operation that falls outside that pattern (Subtract, Reshape, Transpose, Gather) is routed to a scalar Digital Signal Processor (DSP) and forces costly DPU$\leftrightarrow$DSP context switches that can consume 10–30% of inference time on the small kernels typical of KF tracking.

Building on the above motivation, we present KATANA, a novel NPU-aware optimization framework for mapping traditional state estimators onto AI-PC silicon. Our contributions are:

• •

  We present the first cross-platform (CPU, GPU, NPU) characterization of the LKF and EKF inference on two consecutive AI-PC generations (Series 1 and Series 2).

• •

  We introduce three NPU-aware algebraic graph rewrites: a precomputed negative-projection matrix $\mathbf{H}_{\text{neg}}$ that converts subtract-heavy innovations into DPU-native adds, static-shape tensor fusion that removes runtime Reshape/Transpose nodes, and block-diagonal batched parallelization that packs $N$ independent filters into one inference call. Together, these rewrites move 100% of operations onto the DPU.

• •

  We demonstrate that the optimized NPU pipeline meets real-time latency budgets within a sustained 13–14 W active envelope, sustains $>$200 FPS multi-filter throughput on the Series 2, and reduces dynamic energy by up to 97.9% versus CPU execution.

• •

  We validate end-to-end on a live video stream, where the NPU-resident LKF and EKF consume $<$1% of a 33 ms frame budget at 30 FPS, leaving the CPU and GPU free for detection and downstream analytics.

We briefly review the algorithmic primitive we wish to accelerate (Section II-A) and the NPU substrate on which we will run it (Section II-B). Together, these two pieces frame the optimization problem addressed in Section IV.

Our work sits at the intersection of two research threads: hardware acceleration of state estimators, and algorithm–NPU co-design for non-CNN workloads.

Hardware acceleration of KF tracking. FPGA implementations of multi-dimensional KFs for object tracking achieve deterministic latency but require lengthy RTL design cycles and carry higher static power than current SoC fabrics [7]. GPU implementations exploit batch parallelism in many-target settings but exceed the thermal envelopes of fanless and battery-powered edge devices, and lower-precision strategies such as stochastic computing [10] reduce per-operation energy in principle without addressing the DSP$\leftrightarrow$DPU context-switch cost on heterogeneous AI accelerators.

NPU co-design for non-CNN workloads. Modern NPUs increasingly host workloads beyond convolutional networks: FlexNPU offers a dataflow-flexible substrate for energy-efficient edge inference [9], and recent frameworks have mapped LLMs [11], GNNs [12], SSMs [13], and Hyena/Kolmogorov–Arnold Networks [14] onto resource-constrained NPUs. A consistent finding is that every successful NPU mapping requires algorithm-level restructuring to align the workload with the matrix-engine dataflow.

Gap. Classical signal-processing kernels (KFs in particular) have not, to our knowledge, been mapped to a commercial NPU, and no cross-platform (CPU, GPU, NPU) characterization exists for the LKF and EKF on shipping AI-PC silicon. KATANA fills this gap.

Our design goal follows directly from Section II-B: every operation in the LKF and EKF prediction-update recursion must execute on the DPU, with zero fall-back to the DSP, so that the NPU’s GEMM pipeline is the only critical path. All filters are authored in PyTorch, exported to ONNX, and compiled to each target backend through Intel OpenVINO 2024.5 [8, 4]. Three graph rewrites, applied prior to compilation, transform a naive ONNX export into a pure-DPU graph. Fig. 3 traces the resulting Netron graphs for the LKF (top panel) and EKF (bottom panel) through four stages: Baseline, Optimization-1 (subtract elimination), Optimization-2 (static-shape tensor fusion), and Batched (block-diagonal parallelization).

We measure each filter variant of Section IV on representative AI-PC hardware, profiling latency, power, and dynamic energy across all three compute targets in the SoC.

Hardware platforms. We evaluate KATANA on two AI-PC class platforms: the Intel Core Ultra Series 1 and Series 2 reference systems. The CPU, GPU, and NPU are measured on the same chassis under the default thermal policy, so that platform variation is the only difference across compute targets.

Toolchain. All filters are authored in PyTorch, exported to ONNX, and compiled to each target backend through Intel OpenVINO 2024.5 [8]. We use FP16 precision uniformly across CPU, GPU, and NPU to isolate architectural effects from numerical-format effects.

Profiling. Per-operator latency and the DPU/DSP/DMA breakdown shown in Fig. 4 are captured with the OpenVINO Perfetto trace export. Whole-SoC active power is sampled with the Intel Power Gadget at 100 Hz over the measurement window. Each latency value in Table I is the mean of 1000 inference iterations after a 100-iteration warm-up.

Workloads. The LKF uses a state of dimension $n=6$ (3-D position and velocity); the EKF uses $n=8$ (constant-turn-rate with acceleration). Single-instance configurations process one filter per inference call; batched configurations process $N=200$ filters in parallel through the block-diagonal expansion of Section IV-D.

End-to-end tracking pipeline. To validate KATANA on a realistic workload, we deploy it inside a live tracking pipeline derived from the OpenVINO tracking notebooks [4]. A Haar-cascade detector runs on the CPU and supplies bounding-box centroids as measurements; the NPU-resident LKF and EKF maintain independent state estimates in parallel and feed predicted positions back to the renderer overlay on the next frame. The resulting tracks over a 23-s sequence are shown in Fig. 5.

We characterize KATANA along four axes: the per-stage compute breakdown that confirms DPU saturation (Fig. 4); the latency and throughput scaling with workload density (Table I); the sustained power and dynamic energy budgets; and the end-to-end tracking demonstration on live video (Fig. 5). Each axis verifies one design hypothesis from Section IV in turn.

KATANA delivers the first end-to-end mapping and cross-platform (CPU, GPU, NPU) characterization of the LKF and EKF on commercial AI-PC silicon. Three algebraic rewrites (subtract-to-add reformulation, static-shape fusion, and block-diagonal batching) move 100% of the recursion onto the DPU, so the optimized pipeline sustains $>$200 FPS multi-filter throughput at sub-15 W and cuts dynamic energy by up to 97.9% versus the CPU. Classical signal-processing pipelines can therefore be re-targeted to existing AI-PC NPUs without custom accelerators.