et-backend: F32 vecdot GEMV + matrix-engine GEMM

[RehanQasim-dev]

Overview

Improves the ET backend's F32 MUL_MAT path for both decode (GEMV) and prefill (GEMM):

GEMV (decode, N <= 2): Stripes output elements across every hart of all 32 shires instead of blocking work into 16-element chunks, which only filled 8 shires for a typical decode GEMV. Adds a register-resident f32 row-dot helper with L2 prefetch for the streamed weight row, and stages the reused B activation vector into per-shire L2 SCP. Also fixes the matrix-engine GEMV dispatch check, which compared src1->ne[0] instead of src1->ne[1] and so never actually caught the decode case, sending it to the matrix engine kernel where it stalled on a single/near-single-column matmul.

GEMM (prefill, N > 2): Adds a double-buffered producer/consumer F32 matrix-engine kernel — hart 1 transposes weights into double-buffered L2 SCP while hart 0 runs tensor-engine compute — with a weight-reuse path and software prefetch for both weights and activations.

Dispatch now routes N <= 2 to the vecdot GEMV kernel and N > 2 to this matrix-engine GEMM kernel.

Additional information

Performance (Llama-3.2-1B-Instruct F32, ET-SoC-1):

Prefill t/s

N	et	optimized	speedup
100	210.11	185.72	0.88x
220	299.64	344.39	1.15x
512	345.19	526.19	1.52x
700	320.69	429.39	1.34x
900	327.94	480.64	1.47x

Verified with llama-bench on ET-SoC-1 hardware (Llama-3.2-1B-Instruct F32), comparing this branch ("optimized") against unmodified et ("et") at the same prompt sizes used in the Q4_0/Q8_0 matrix-engine PRs.

Note: N=100 is a regression (0.88x), reproduced twice (4 repetitions each). The new weight-reuse matrix-engine kernel appears to have per-call setup overhead that doesn't amortize until N is large enough — gains are consistent and substantial from N=220 upward, but the smallest prefill size is currently worse than baseline et. Flagging for review before merge; may need a higher matrix-engine dispatch threshold for F32 than the current N > 2, or further tuning of the small-N case in the kernel itself.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - the optimization strategies and design decisions are my own. AI assisted with understanding the hardware reference manual, some pieces of code implementation and guided debugging. I have thoroughly reviewed the code.