Fused rms silu

[reckylurker]

Fixes #4263

This adds a fused RMSNorm+SILU operation that computes the composition in a single pass instead of two Metal passes.

Empiral Benchmarks (M2)

FP16 Benchmarks

Shape	mx.fast.rms_norm + silu	mx.fast.fused_rms_silu	Speedup
1 x 4096	0.215 ms	0.172 ms	1.25x
512 x 4096	0.537 ms	0.413 ms	1.30x
2048 x 2048	0.989 ms	0.531 ms	1.86x
2048 x 4096	1.648 ms	1.156 ms	1.43x
4096 x 4096	6.253 ms	4.091 ms	1.53x

FP32 Benchmarks

Shape	mx.fast.rms_norm + silu	mx.fast.fused_rms_silu	Speedup
1 x 4096	0.302 ms	0.262 ms	1.15x
512 x 4096	1.025 ms	0.488 ms	2.10x
2048 x 2048	1.731 ms	0.766 ms	2.26x
2048 x 4096	3.449 ms	2.663 ms	1.30x
4096 x 4096	9.088 ms	4.194 ms	2.17x

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

[reckylurker]

@zcbenz There are many more operations for which fusion can be performed like:

1. Residual Add + RMSNorm (Would a fused residual-add + rms_norm be welcome in mx.fast? #4259) y = RMSNorm (x +residual,w,ε)
2. Add + RMSNorm + SiLu y = SiLU (RMSNorm (x +residual,w,ε))
3. RMSNorm + GELU y = GELU (RMSNorm (x,w,ε))
4. LayerNorm + Bias + GELU y = GELU (LayerNorm (x,w,b,ε)+ bias)
5. GroupNorm + SiLU y = SiLU (GroupNorm(x,groups,w,b))
6. Conv2D + BatchNorm + ReLU y = ReLU (BatchNorm (Conv2D(x)))
7. Linear + Bias + Activation y = Activation (XWᵀ + b)
8. Fused SwiGLU Gated Projection: y = (XWᵀ · sigmoid(XW_gateᵀ)) ⊙ (XW_upᵀ)

and many more. I would be interested in contributing these. Would PRs adding these into mx.fast useful? Are there other specific operations or architectural priorities you would recommend looking at instead?

[zcbenz]

Answered in #4259 (comment).