Skip to content

Add opt-in Winnow 12B image decisions - #59

Merged
cobanov merged 2 commits into
ollaya-dev:mainfrom
Txoka:feat/winnow-12b-image-input
Oct 6, 2026
Merged

cobanov merged 2 commits into
ollaya-dev:mainfrom
Txoka:feat/winnow-12b-image-input

Conversation

@Txoka

@Txoka Txoka commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Adds opt-in winnow:12b-vision using the existing image runtime merged in #52. The tag retains Winnow-12B's pinned Q8_0 weights, prompt (including its empty thought block), and temperature 1, adding the matching author projector from the same revision. Text tags and the latest alias retain their existing behavior.

The catalog entry uses the normal packaging pipeline; no registry manifest or blob is published by this change. Family documentation contains the CPU validation and reproduction commands, and the library page marks the tag as pending packaging. This branch is rebased onto current main and preserves the newer E4B RTX 5090 validation.

Validation

  • Previously measured on Linux x86-64 CPU against stock llama-server b11146, on the same device: 21 image requests, 65/65 decisions, maximum normalized option-logit difference 7.66e-6, below the unchanged 1e-3 gate.
  • Previously measured native API checks at the shipped 8,192-token context: PNG/data URL equivalence, 2 and 16 images, malformed inputs, 17-image rejection, context overflow/recovery, image-free text compatibility, and unloading. Debug-log checks passed.
  • Real packaging output was checked against pinned author metadata, with isolated outputs and no publishing.
  • After rebase: both projector packaging tests pass for E4B and 12B; git diff --check passes.

CUDA, Metal, Windows and Linux ARM 12B image parity, the full 505-question 12B CPU text suite, image calibration, and visual accuracy have not been measured. Numerical parity is not an accuracy benchmark. No GPU validation was rerun during this rebase.

@Txoka

Txoka commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Clarification on validation coverage: Winnow-12B CUDA image parity was not run because my available GPU is an RTX 4070 with 12 GB VRAM, while the Q8_0 model weights alone are about 12.7 GB, before the projector, KV cache, and runtime buffers. Full GPU offload therefore does not fit on this machine. Partial CPU/GPU offload was not tested either.

…yout

- docs: the 12B rows join the image parity table, with the RTX 5090 run against its own stock
  llama-server b11146 at the shipped 8,192-token context (21 requests, 65/65 decisions, option
  logits within 7.6e-6) and the 505 text questions of winnow:12b on the same GPU (505/505, option
  logits within 1.1e-5). The separate 12B validation and reproduction sections fold into it: the
  reproduction keeps only what differs for 12B (its files and thought: true), and the packaging
  notes stay in the pull request.
- memory: 12b-vision peaked at 16,263 MiB on the RTX 5090, against 15,877 MiB for winnow:12b, so
  the family docs and the library page call a 16 GB GPU marginal for it.
- library page: no "pending packaging". The maintainers merge right before the release, so the page
  never names a tag the registry does not have; like e4b-vision, the tag needs Ollaya 0.10.0 or
  newer. The two parity bullets become one.
- site/docs/api.md: winnow:12b-vision next to winnow:e4b-vision.
@cobanov

cobanov commented Oct 6, 2026

Copy link
Copy Markdown
Member

@Txoka, thank you. Our RTX 5090 ran it with the shipped 8,192-token context against its own stock llama-server b11146: 21 requests, 65/65 decisions, option logits within 7.6e-6, right next to your 7.66e-6 on the CPU. The 505 text questions of winnow:12b pass on the same card too (505/505, option logits within 1.1e-5, probabilities within 2.3e-6), so the text tag and latest are untouched. One request takes 271 ms at the median with one image and three questions, measured in the runner, and 16 images still answer correctly. The tag needs about 16 GB on the GPU: 16,263 MiB at the peak against 15,877 MiB for winnow:12b on the same card, so we call a 16 GB card marginal rather than the floor.

The change is what we hoped for: no runtime code, the projector pinned to the same revision with its sha256, and a test that keeps latest on 12b and the projector off the text tags.

As with #52, I pushed one docs commit to your branch (ae45126): the 12B rows join the image parity table with the RTX 5090 numbers, the reproduction notes shrink to what differs for 12B (thought: true from the shipped decision.json, the 12B files), the memory note is in, the library page drops "pending packaging", and site/docs/api.md lists the new tag next to e4b-vision.

What's left is on our side: package.py writes the 12b-vision manifest with its config blob in the release commit, and we merge right before the next release, so the site never names a tag the registry does not have yet.

@cobanov
cobanov merged commit 539a32b into ollaya-dev:main Oct 6, 2026
7 checks passed
@cobanov

cobanov commented Oct 6, 2026

Copy link
Copy Markdown
Member

Thanks, @Txoka. Merged and released in Ollaya 0.11.0: winnow:12b-vision is live on ollaya.dev, and its manifest and blobs are checked against the registry files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants