Repository navigation
Add opt-in Winnow 12B image decisions - #59
Conversation
|
Clarification on validation coverage: Winnow-12B CUDA image parity was not run because my available GPU is an RTX 4070 with 12 GB VRAM, while the Q8_0 model weights alone are about 12.7 GB, before the projector, KV cache, and runtime buffers. Full GPU offload therefore does not fit on this machine. Partial CPU/GPU offload was not tested either. |
…yout - docs: the 12B rows join the image parity table, with the RTX 5090 run against its own stock llama-server b11146 at the shipped 8,192-token context (21 requests, 65/65 decisions, option logits within 7.6e-6) and the 505 text questions of winnow:12b on the same GPU (505/505, option logits within 1.1e-5). The separate 12B validation and reproduction sections fold into it: the reproduction keeps only what differs for 12B (its files and thought: true), and the packaging notes stay in the pull request. - memory: 12b-vision peaked at 16,263 MiB on the RTX 5090, against 15,877 MiB for winnow:12b, so the family docs and the library page call a 16 GB GPU marginal for it. - library page: no "pending packaging". The maintainers merge right before the release, so the page never names a tag the registry does not have; like e4b-vision, the tag needs Ollaya 0.10.0 or newer. The two parity bullets become one. - site/docs/api.md: winnow:12b-vision next to winnow:e4b-vision.
|
@Txoka, thank you. Our RTX 5090 ran it with the shipped 8,192-token context against its own stock llama-server b11146: 21 requests, 65/65 decisions, option logits within 7.6e-6, right next to your 7.66e-6 on the CPU. The 505 text questions of The change is what we hoped for: no runtime code, the projector pinned to the same revision with its sha256, and a test that keeps As with #52, I pushed one docs commit to your branch (ae45126): the 12B rows join the image parity table with the RTX 5090 numbers, the reproduction notes shrink to what differs for 12B ( What's left is on our side: |
|
Thanks, @Txoka. Merged and released in Ollaya 0.11.0: |
Adds opt-in
winnow:12b-visionusing the existing image runtime merged in #52. The tag retains Winnow-12B's pinned Q8_0 weights, prompt (including its empty thought block), and temperature 1, adding the matching author projector from the same revision. Text tags and thelatestalias retain their existing behavior.The catalog entry uses the normal packaging pipeline; no registry manifest or blob is published by this change. Family documentation contains the CPU validation and reproduction commands, and the library page marks the tag as pending packaging. This branch is rebased onto current main and preserves the newer E4B RTX 5090 validation.
Validation
git diff --checkpasses.CUDA, Metal, Windows and Linux ARM 12B image parity, the full 505-question 12B CPU text suite, image calibration, and visual accuracy have not been measured. Numerical parity is not an accuracy benchmark. No GPU validation was rerun during this rebase.