Live: https://rishabh-anand.com/agent-strudel/
A fine-tuned SmolLM2-135M-Instruct that generates Strudel live-coding patterns from a text description, and runs in-browser via WASM — no server, no GPU, just a 130 MB quantized model loaded by transformers.js.
The full loop lives in this repo: an agent generates + validates training data, a fine-tuner trains the model, a GLM-5.2 evaluator judges quality, and the result is exported to ONNX and served to the browser.
The deployed model (out_m402) was trained on 402 Strudel snippets spanning 45 genres. It generates most reliably in the well-represented ones (≥ 5 samples):
| Genre | Samples |
|---|---|
| deep-house | 87 |
| downtempo | 63 |
| afrobeat | 45 |
| techno | 37 |
| jazz-house | 29 |
| funk | 20 |
| jazz | 15 |
| lofi-hiphop | 14 |
| breakbeat | 12 |
| fusion | 11 |
| house | 9 |
| ambient | 8 |
| synthwave | 6 |
| dub | 5 |
A long tail of 31 niche genres (1–4 samples each) — e.g. tech-house, world-fusion, jazz-fusion, acid-techno, gamelan-ambient, klezmer-fusion, bluegrass-fusion, afro-cuban-jazz, shoegaze, dnb, trance, metal, baroque-fusion, cinematic-ambient — is also in the training data. Try them, but expect more hit-or-miss results since the model saw few examples.
There's no deep science here. The choice was driven by one constraint: I don't have infinite money.
- Training data costs real $ — every sample is generated by a Claude (Sonnet-5) agent that writes a snippet, self-validates it headlessly, and is then judged by a second Claude call. ~705 accepted samples cost ~$67 in Anthropic API spend.
- Evaluation costs real $ — quality is judged by GLM-5.2 (Z.AI API) over a fixed set of prompts.
- Fine-tuning costs time/compute — even on a free Colab T4 a 135M model trains in minutes; on CPU it's tolerable.
So the model had to be the smallest thing that could plausibly learn Strudel syntax and run in a browser. SmolLM2-135M-Instruct fit:
- 135M params → full fine-tune (not LoRA) fits in ~2–4 GB VRAM, trains in minutes.
- Instruct variant → ChatML template +
eos=<|im_end|>already baked in, so it follows instructions out of the box (we're doing task adaptation, not teaching instruction-following from scratch). - Already tagged transformers.js on the Hub → the architecture is proven to run as WASM.
- Quantizes to ~130 MB (int8) → deliverable over HTTP, loads in seconds, generates on WASM CPU.
A bigger model would likely score higher, but the data + eval budget wouldn't scale to justify it. 135M is the cheapest viable point on the curve.
Cumulative-from-base training: at each milestone the base model is retrained on all accumulated data (3 epochs, lr 3e-4, cosine, max_length 4096, completion-only loss). Quality is the average GLM-5.2 score (1–10) over 5 fixed prompts, same seed — so the trend is a real signal, not sampling noise.
| Milestone (samples) | Score | Model |
|---|---|---|
| baseline (4-sample smoke) | 1.0 | — |
| 60 | 3.0 | fine-tuner/out_m60 |
| 120 | 4.8 | fine-tuner/out_m120 |
| 240 | 5.8 | fine-tuner/out_m240 |
| 402 | 6.6 | fine-tuner/out_m402 ← peak |
| 705 | 5.6 ↓ | fine-tuner/out_m705 (decrease → stop) |
Best model: fine-tuner/out_m402 (6.6). At 402 samples every generation compiles as valid Strudel; GLM's critiques are about musical nuance ("kick isn't 4-on-the-floor", "no BPM set", "rolling bass missing"), not syntax. The baseline emitted C++/Python and scored 1.0 — so the fine-tune clearly taught Strudel.
Notable finding: more data hurt. Quality peaked at 402 and regressed at 705 (5.6). Since both were evaluated on identical prompts/seed, this is a genuine signal — likely overfitting at 3 epochs on 705 samples, or lower-quality late samples (the generator was straining for obscure genres) diluting the set. The run auto-stopped on the decrease, per the stop rule. The natural next experiment would be fewer epochs at 705 or a quality-weighted subset — but that's beyond the current budget.
training-data-gen/ Claude agent (Sonnet-5) writes + self-validates Strudel,
judged by a 2nd Claude call → labelled JSONL in data/
↓
fine-tuner/ trl SFTTrainer full-fine-tunes SmolLM2-135M-Instruct
on (description → code) pairs
↓
fine-tuner/evaluator.py LangChain agent: FT model generates + validate.js
checks compile, GLM-5.2 (Z.AI) judges quality
↓
ONNX export optimum → text-generation-with-past, quantized to int8 (~130 MB)
↓
frontend/ transformers.js loads the q8 ONNX in a Web Worker,
generates Strudel from a prompt, feeds the Strudel REPL
uv run python training-data-gen/main.py --limit 30
Appends labelled samples to data/training.jsonl (uses ANTHROPIC_API_KEY in training-data-gen/.env).
uv run --project fine-tuner python fine-tuner/main.py
Smoke test: append --epochs 1 --max-train-samples 4 --batch-size 2.
uv run --project fine-tuner python fine-tuner/evaluator.py --prompt "groovy jazz with indian fusion"
Uses Z_AI_API_KEY in the root .env. Prints FINAL_SCORE: <avg>.
uv run python overnight.py # train+eval only on existing data (no generation)
uv run python overnight.py --cap 1000 # allow generation up to 1000 samples
Resumable; defaults to --max-cost 60 and --max-hours 8. State in overnight_state.json.
make serve # python -m http.server 8000 in frontend/
Open http://localhost:8000 — the model loads (~130 MB, cached), type a description, click Generate, then Play.
training-data-gen/ data generator agent (claude-agent-sdk)
fine-tuner/ trl fine-tuner, evaluator agent, ONNX export
frontend/ Strudel REPL + transformers.js model (the browser app)
data/ training.jsonl (gitignored)
overnight.py resumable generate→train→eval orchestrator
- Spend: ~$67 on data generation (705 samples), slightly over the $60 target. A Windows subprocess-timeout bug let generation run past the 8-hr wall — now fixed with
taskkill /T /Fand a hard cost cap inovernight.py. - Eval is 5-prompt averaged with a fixed seed — low-noise enough to show the trend, but not a full benchmark.
- The 705 model is kept (
out_m705) butout_m402is the one shipped to the browser. - The
@strudel/*packages are AGPL-3.0 — fine for local use, review if you ever host/distribute.
Copyright (C) 2026 Rishabh Anand.
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License v3.0 as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. See LICENSE for the full text.
This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.
Why AGPL-3.0: the frontend uses @strudel/repl (@strudel/* packages), which are AGPL-3.0. Because the frontend and Strudel run together as a single combined program, AGPL-3.0 applies to the whole app — including a Section 13 obligation: if you make this app available over a network, you must offer every user the complete corresponding source code (a "Source" link satisfies this). Commercial use and ads are not restricted — only source disclosure is.
The fine-tuned model weights and training pipeline are part of this same program; treat them as covered by the AGPL-3.0 source-disclosure obligation if you serve the app over a network.