Feature Discussion: (smart) grinder integration and assisted dial-in

[MaxRink]

I have a Mahlkönig E64 WS at home and have been exploring how a connected grinder, a Decent and a scale can work together. I would like to keep the device integration, assisted dial-in and any future automatic adjustments as separate pieces, so the useful parts also work with manual grinders.

What is already delivered

Decaid now has the connected Grinder domain and generic driver runtime from #889 / #896, plus validated control metadata and plugin-owned surfaces from #949 / #962. A driver can maintain multiple independent instances, while the host selects one connected Grinder.

The external E64 plugin is implemented as a read-only type: grinder driver. It publishes generic state and exposes vendor diagnostics; it does not change settings or run the motor. Its hardware-free CI exercises actual host QuickJS/API behavior, two mock grinders, errors, lifecycle changes and credential redaction. Physical firmware compatibility remains a separate check. Platform-trusted WSS is supported; custom self-signed trust remains proposed in #758.

Recommendation UX belongs in a skin or community application, following the maintainer direction on #866. I have implemented decaid-assisted-dialin-plugin, an independent application plugin using the existing shot APIs, events, plugin storage and HTTP endpoints. The first implementation and richer follow-up suggestions are tracked in its own repository; the first reference-shot/feedback/UI slice is implemented. The plugin owns its HTTP UI and does not register a fictitious device to obtain a settings row.

The first slice is published at cbda7af9887f3b6980b8df9bc7562965c11f5941; all four CI jobs pass. This includes 29 behavioral and controlled HTML-script checks, actual Decaid package parsing, and a real QuickJS/API/storage fixture for useful recommendations, mark/unmark, concurrent writes and reload persistence. The current bean batch, persistent grinder identity and profile execution contents are required. Optional comparison fields missing on both sides are shown as warnings; current setup entries do not backfill historical shots. Changes to profile presentation metadata are ignored, while changed execution content prevents a match.

The initial UI is opened through the plugin endpoint; native/browser rendering, custom API ports and physical firmware validation remain separate. Installation, bounded history refresh and safe error/recovery behavior are documented in the repository. It makes no grinder or workflow changes.

First version: make successful shots reusable

The first version should be useful without automatic grinder control or an AI service:

• Let the user mark a shot as a good reference and add taste notes.
• Find comparable history by bean/batch, persistent grinder identity, burr/calibration configuration, profile content, dose/yield and basket where known.
• Show the last comparable successful shot, its recorded grind setting and the evidence used to select it.
• Distinguish missing information from a known match. Explain when there is insufficient evidence rather than guessing a setting.
• Keep suggestions separate from recorded observations. The user accepts, edits or ignores a suggestion; the plugin does not automatically change the grinder, workflow or profile.

Taste categories and more advanced finer/coarser suggestions can follow after the first history/feedback flow is useful. A fast shot is not automatically a bad shot, and a setting number has no universal direction or step size across grinders. The plugin should not infer these from elapsed time alone.

Feedback and reference-shot markings can initially live in namespaced plugin storage keyed by shot ID. If some of that data needs to become portable shared shot metadata, that should be an explicit later contract rather than silently rewriting existing annotations.

What shot history currently records

At main af4e99524ad7731dbdb7db4540cd272bc91da038, the workflow may contain grinderId, grinderModel, grinderBurrs and a grinderSetting string. Those are optional. Workflow JSON is saved with the shot; burr metadata is refreshed from the equipment record at persistence time.

This is useful recorded workflow data, but it is not an immutable observation of the configuration used for the dose. The shot does not automatically contain observed RPM, the vendor configuration or calibration provenance, and the E64 diagnostics are not copied into shots. A current setting read at espresso start/save may belong to a later adjustment rather than the grind that produced the dose.

I have filed two separate core proposals:

• #973: read-only Grinder setting/RPM observations, without advertising writable capabilities.
• #974: recording the grinder configuration used with a shot, with intended-versus-observed values, capture timing, identity and provenance.

Both await maintainer review. They are not prerequisites for the first successful-shot lookup, which can label its inputs as recorded/user-confirmed workflow data.

#964 is already accepted for missing shot events after History edits/imports. The plugin can use bounded history reconciliation until that is delivered. An updated-since history filter would be a separate API proposal; it is not part of #964's accepted event fix.

Questions for the next iteration

1. Is a user-marked good shot plus optional notes the right first feedback model? Which taste categories would be useful next?
2. Which matching fields should be required, and how should the UI handle missing basket or calibration information?
3. Which changes should start a new grinder calibration period: zero adjustment, burr replacement, servicing, or explicit user reset?
4. Should the next version suggest only grind changes, or also explain dose/yield/profile alternatives? I would start with history lookup and keep those decisions separate.
5. Which skin entry point would be most useful for the plugin UI? Existing Streamline plugin navigation work should be reused where possible.

Future device writes, profile/bean-driven setup and automatic grind-by-weight remain separate work requiring explicit user consent, safe command handling and real-device validation. Beanconqueror/roaster interchange remains Discussion #721; it does not need to block the first dial-in plugin.

[tadelv]

hey @MaxRink , we're definitely interested. I have been thinking about how to integrate that for some time now, especially since there are reverse-engineered implementations popping up.

I am not familiar with E64WS control method, can you explain a bit more? At the moment I know about a BLE controlled grinder and a Websocket controlled grinder.

My initial idea was to use plugins to control grinders - and your idea about extending the Grinder data model could fit into that too perhaps. In the sense, that Grinder object gains the interface for controlling parameters as well as an optional transport object that would relay that info.

Please check if #146 would help with this idea. There are other interesting things hanging on it too, so it would be nice to confirm we can solve a lot with one task.

I'll also move this to discussions, where we'll be more free to discuss before formalizing a plan.

[MaxRink]

I had AI summarize the current state of my reverse engineering in my Repo

The E64 WS is a recipe-driven grinder with three dosing modes:

    Manual: start/stop grinding manually.
    Grind-by-Time: grind until the recipe's time target is reached.
    Grind-by-Weight: grind until the internal load cell reaches the target dose. The same load cell can identify a previously registered portafilter and select its associated recipe.

Recipes also store a reference disc distance (DD, in microns), dose settings, and an optional Grind-by-Sync (GbS) flag. The HMI can select/edit recipes with its buttons and rotary encoder. It also provides quick manual DD/dose editing and purge; these are controls rather than additional dosing modes. The local web UI is available in either the grinder's access-point mode or station mode.

GbS is the closed-loop mode: a compatible espresso machine or Sync Scale provides a brew event, principally the observed brew time. The grinder compares that with the recipe target and its AGSA motor adjusts the electronic burr distance finer when the shot is too fast, or coarser when it is too slow. It is intended as small ongoing corrections after an initial dial-in, not as a single-shot guarantee. The built-in Assisted Dial-In wizard is a separate workflow that asks for coffee/shot parameters and brew time, suggests a DD, and can refine/assign the resulting recipe.

For the E64 WS image I inspected, the local control surface is:

HTTPS REST: POST /rest_api/<route>
TLS WebSocket: wss://<grinder-ip>:9997/?token=<short-lived-local-token>
Messages: JSON request/response envelopes

The recovered WebSocket registry includes state/config/log reads and named actions such as StartGrindManual, StartGrindGbw, StartGrindGbt, Stop, SetAgsaTargetDdValue, AddBrewEvent, and GbsCreateReport. The normal action envelope is {"type":"GrinderAction","data":{"actionType":"..."}}. This is separate from the vendor cloud API; the local WSS client does not need to call the cloud, although the officially documented Sync Scale → grinder GbS path is cloud/Machine-to-Machine based.

The E64 firmware contains a brew-event parser and an AddBrewEvent action, but we have not yet proven that a locally injected event is accepted by every firmware revision or causes a physical GbS adjustment. That makes two useful Decaid paths possible: a safe host-side recommendation mode, and an explicit, owner-enabled E64 event/actuation adapter after live validation. A Decent- attached scale or OpenScale can be normalized as an independent mass source; flow-only data must not be treated as weight.
API map

The API split is roughly:

local device                         Mahlkönig Sync cloud / mobile app
------------                         -----------------------------
HTTPS REST /rest_api/*               HTTPS https://sync.mahlkoenig.com/api/*
JSON WSS :9997                       scale WSS scale.sync.mahlkoenig.com/socket/device
HMI/driver state + actuation         accounts, pairing, recipes, events, analytics
local onboarding/Wi-Fi               cloud device binding and firmware presentation

The local REST surface is PHP behind lighttpd. The recovered routes include session/login (ping, check_session, login, logout, update_session), device information (get_machine_info), Wi-Fi/onboarding (scan_wifi, get_wifi_info, request_onboarding), and operational or maintenance functions (reboot, get_fw_update_state, download_database, and download_machine_logs). Firmware and ESP/splash uploads are separate local PHP pages rather than normal REST routes. They should be treated as maintenance surfaces, not as part of a generic grinder driver.

The local WSS registry has 52 records: six state/config reads, provisioning and Wi-Fi messages, and 39 GrinderAction variants. The read families cover driver state/config, logs, ADC/config-manager settings, and machine info. The important control variants cover manual/time/weight grinding, stop, tare and calibration, AGSA/DD adjustment, brew-event insertion, and GBS report creation. The remaining variants expose fans, lights, language, setup state, simulated buttons/rotary input, endurance tests, ESP maintenance, and configuration updates. Exact names, payload keys, development flags, callbacks.

Local WSS requests use msgId; ordinary replies use the corresponding refId, while action replies normally contain data.success. The local web login creates a short-lived bearer token used in the WSS query string. The examined binary contains no per-message HMAC/signature scheme. These are static findings for one firmware image, not a promise that every command is enabled or safe on a production unit.

The cloud/mobile side is a different API. The Android APK contains the cloud root https://sync.mahlkoenig.com/api/, auth/token-provider code, and these route families:

    authentication and account lifecycle: security-service/auth/token, security-service/auth/refresh, password/reset and registration routes;
    device/pairing: mobile-service/device/query, device-union queries, offboarding, profile/store/region and invitation routes;
    grinder and brew history: mobile-service/grind-event/find and count, device-event/brew-event/find and count, error events, and grind totals;
    recipes: recipe presets, cloud-recipe query/slot/create/status/merge, and recipe-threshold services;
    scale/live data: ScaleApiClient plus wss://scale.sync.mahlkoenig.com/socket/device;
    dashboards, chat/help, Firebase messaging/analytics/crash/performance, remote configuration, and firmware/app-version status models.

The APK also contains local provisioning candidates such as /api/v2/network/list, /api/v2/network/connect, and /api/v2/network/forcescan, alongside /rest_api/* onboarding routes. A route literal proves only that the string is present: the exact HTTP method, headers, body, response schema, retry policy, and authorization are not all recoverable from the stripped Flutter AOT snapshot. In particular, the APK does not contain a literal E64 RequestDriverState or GrinderAction string, so stock-app-to-local-WSS compatibility remains unproven.

Looking at #146: its useful for the asynchronous transport and workflow-event plumbing, but its generic outbound transport should cover TLS text frames and request/response streams, not only binary frames.

[MaxRink]

It also looks to me like someone mostly just copied over the software feom their commercial grinders, as i also found references for the E80 in the GBS code

[tadelv]

Another potential pathway to the grinder control might be through the generic sensor device. Maybe worth investigating

[MaxRink]

Yes, but id say grinders are important enough in the workflow to warrant their own modeling.

[MaxRink]

Following up with a more concrete assisted-dial-in shape, including how it overlaps the existing bean, grinder, workflow, and tasting discussions. This remains exploratory.

Build on the existing models

Decaid's grinder model already distinguishes numeric settings from named values and supports small/big setting and RPM increments (current model). WorkflowContext already links the selected grinder, grinder setting, bean batch, target dose, and target yield (current workflow context). Shot annotations already carry actual dose/yield, enjoyment, and espresso notes (current annotations).

The existing discussions remain the right homes for their underlying schemas:

• #62 Grinders API
• #64 Wiring beans/grinders/equipment into Workflow
• #66 Shot record metadata and tasting fields

Grinder comparison model

For recommendations, the model additionally needs:

• setting direction: whether a larger numeric value means finer or coarser;
• the existing increments/presets expressed consistently;
• calibration/maintenance epochs, so burr replacement, zero-point changes, or recalibration starts a new comparison period;
• capabilities separated from transport, so manual and connected grinders use the same evidence model.

A raw setting must never be compared across different grinders or calibration epochs. Smart-grinder communication is a separate capability; issue #146 is already discussing permission-gated plugin transports and workflow-change events that could support it.

Taste and outcome input

Keep the existing enjoyment score and free-form notes, while optionally adding a small structured vocabulary useful for suggestions. A first iteration could capture balance/direction and defects such as sour, bitter, astringent, weak, or harsh, plus optional flavor observations.

The vocabulary should be tested with real users before promoting a large tasting schema into typed core fields. #66 already contains a much broader tasting-field proposal and should remain the schema discussion.

Recommendation behavior

Assisted dial-in should be suggestion-only:

• never change a grinder, workflow, or machine profile automatically;
• show the prior shots used as evidence and the reason for the suggestion;
• state confidence and abstain when there is not enough comparable history;
• let the user explicitly accept, edit, or ignore a suggestion.

Comparable history should require, at minimum:

• the same bean or batch relationship;
• the same grinder and calibration epoch;
• the same machine-profile identity/version or a deliberately compatible group;
• compatible dose and target-yield context.

The first useful implementation can be deterministic rather than AI-based: find the last-known-good comparable shot, compare extraction outcome and taste feedback, then suggest one bounded change. Advice should avoid changing grind and machine recipe simultaneously unless the evidence can explain why.

Roaster-provided settings or profiles can be a starting prior, but must be visibly attributed and treated as recommendations for the named hardware rather than universal truth. The separate interoperability discussion covers that source data: #721 Beanconqueror and roaster interoperability

Questions

1. What is the smallest useful structured taste vocabulary?
2. How many comparable observations are required before suggesting a change?
3. Should the first iteration only recommend grind changes, or may it also recommend an explicit profile/recipe adjustment?
4. How should users mark a shot as a trusted "good" reference?
5. Which grinder maintenance events should start a new calibration epoch?

[loganfuller]

I'm very interested in this. Would be cool to be able to sync the DE1 and commercial GbS grinders in a cafe environment.

[tadelv]

Hey @loganfuller, check #890 for progress. We're targeting 0.9.0 to support both the /grinder endpoints as well as dynamic drivers required for people to add support for any device without requiring changes in Decaid

[allofmeng]

Streamline.js: plugins can now add to the dashboard (relevant to question 5)

Re the skin entry point question: the latest commits on decentespresso/streamline-js main (most recently #138, plus #133 to #137) let any plugin author add and edit small elements on the Streamline dashboard. This works through a fixed set of surface kinds, not arbitrary UI. The plugin answers a skin HTTP endpoint with JSON, Streamline renders it in its own style, and only actions from a fixed list run. No plugin JavaScript runs in the skin, and with the plugin off the dashboard is unchanged.

Reference: docs/plugin-surfaces.md (kinds, limits, actions, tile modes and a worked example).

What a plugin can add (Kinds):

• header-button: a button in the header that opens the plugin's page, as a page or an overlay.
• tile-hint and tile-line: a hint under a tile's value and a line under its label, on dose, drink, grind or temperature.
• tile-mode: an extra option in a tile's toggle with its own number, presets and -/+ (Tile modes).
• ratio-action: makes the "(1:x)" label under Drink tappable (new in fix: de1app folder import on Android 11+ via SAF #138).
• screensaver-button: a button on the screensaver that opens a plugin page.

Actions are open, set, step, stepsize and call (Actions). #138 also lets stepsize target grinderSetting, so grind gets the same step options as dose and drink.

Grinder example

A grinder plugin such as the E64 driver, or the assisted dial-in plugin, could answer skin like this:

{
  "version": 1,
  "surfaces": [
    { "id": "gear", "kind": "tile-line", "tile": "grind",
      "text": "E64 · 1200 rpm" },
    { "id": "last", "kind": "tile-hint", "tile": "grind",
      "text": "Last 6.4", "title": "Last good shot of this coffee",
      "action": { "type": "set", "field": "grinderSetting", "value": 6.4 } },
    { "id": "dialin", "kind": "header-button", "label": "Dial-in",
      "action": { "type": "open", "view": "main", "presentation": "overlay" } }
  ]
}

How that maps to this discussion:

• The tile-line shows the connected grinder's state under the Grind label without moving the tile.
• The "Last 6.4" hint is the "last comparable successful shot's setting" from the first version. It is suggestion-only: nothing changes until the user taps it. Tapping runs set, which goes through the same setter as a tile edit, so the active profile remembers the value until it is changed again.
• The header-button opens the plugin's own UI as an overlay, which is the entry point question 5 asks about.
• One tile-hint and one tile-line per tile are drawn; if two plugins ask for the same slot the first wins. A stepsize pill ({ "type": "stepsize", "field": "grinderSetting", "sizes": [1, 0.5, 0.1] }) would share the grind hint slot with "Last 6.4".

The manifest needs permissions: ["api"] and an http endpoint named skin (and ui for open). The Writing a plugin section and the checklist cover the rest, including installing a local copy with PUT /api/v1/plugins/<id>/source and inspecting it with GET /api/v1/plugins/<id>/skin.