Skip to content

[6.x] Improve performance of data reference updates - #15089

Open
daun wants to merge 1 commit into
statamic:6.xfrom
daun:feat/reference-update-performance
Open

[6.x] Improve performance of data reference updates#15089
daun wants to merge 1 commit into
statamic:6.xfrom
daun:feat/reference-update-performance

Conversation

@daun

@daun daun commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

TLDR

Speed up asset & term reference updates with a cheap string check on each item's raw data, performed before resolving the blueprint and iterating over any fields.

3x faster in benchmarks + memory stays flat.

The problem

On larger sites (20k+ entries) the UpdateAssetReferences / UpdateTermReferences jobs regularly exceed queue worker timeouts and memory limits. Once a job exceeds the timeout it gets discarded and the references silently never get updated :(

The listener already iterates lazily over items, so that part is efficient. The expensive part is that for every item it resolves the blueprint and instantiates all fields to figure out which fields could realistically hold a reference. On an asset rename, 99% of items don't reference the asset at all, but they all pay the full cost of resolving the blueprint.

The fix

An early exit in DataReferenceUpdater: json-encode the item's raw data, check if the resulting string contains the value being replaced, and bail before touching the blueprint if it's not found.

Is it safe?

This sounds too dumb to be correct, I thought, but it works out because the updater itself never augments anything: it reads and writes raw stored data, i.e. the blueprint is only used to decide which keys to look at. Every replacement path requires the old value to appear verbatim in the data: assets compare using ===, including in Bard, markdown searches exact statamic:// urls. So if the old value isn't a substring of the encoded data, there is nothing the updater could ever rewrite.

Numbers

The more complex the blueprints, the bigger the speedup, since the skipped work grows while the string check stays constant. Tge production site in question went from exceeding a 128MB memory limit to finishing within it.

Entries Before After
2,000 701 ms 244 ms
5,000 1,704 ms 568 ms

On Eloquent

On Eloquent sites there's an even better early exit available: a where on the data column before hydrating anything. We run that in production with a subclassed listener and it's really effective but tied directly to the Eloquent driver using the data column setup. The check in this PR is driver agnostic. It runs on hydrated data, so flat files, a data column or hoisted columns all behave the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant