Skip to content

Bound the repository query cache with :query_cache_max_size - #4809

Open
bmalum wants to merge 1 commit into
elixir-ecto:masterfrom
bmalum:fix/bound-query-cache
Open

bmalum wants to merge 1 commit into
elixir-ecto:masterfrom
bmalum:fix/bound-query-cache

Conversation

@bmalum

@bmalum bmalum commented Oct 25, 2026

Copy link
Copy Markdown

The repository query cache is an ETS :set with no size limit and no eviction. Its key is the query shape, and several constructs put runtime values into the shape rather than into the parameter list: field(x, ^"name") with a string, json_extract_path(f, ^path) / f[^key], identifier/1, constant/1, the :prefix option, and literal leaves of an interpolated select map. Each distinct value adds one entry holding the planned query and the adapter's prepared statement, and the entry is never released. Queries that fail at execution (an unknown column) are cached too, since planning succeeded. The json_extract_path/2 docs encourage dynamic paths and the field/2 docs recommend string names "to avoid creating a large number of atoms", neither mentions that every value is kept forever.

Measured through Ecto.CachingTestAdapter:

baseline:                       1 entry (the first query)
5000 distinct json paths:       5001 entries, ~8 MB
5000 distinct string fields:   10001 entries, ~13 MB

With a real adapter each entry also creates a named server-side prepared statement on every pooled connection.

This adds a :query_cache_max_size repo option (default 10_000, :infinity to disable). The limit is read from the cache table itself and checked only on inserts, so hits cost nothing extra; once the table is past the limit it is cleared before the new query is inserted, and queries are re-prepared as they are executed. field/2, json_extract_path/2 and the Ecto.Repo option docs now say that each distinct value is a separately cached query and point at query_cache: false for user-controlled values.

One limitation worth knowing: clearing the ETS table does not close the server-side prepared statements ecto_sql created for the evicted entries. Those live for the connection lifetime regardless. Bounding them needs a hook in ecto_sql (DEALLOCATE on eviction, or treating those query shapes as :nocache), which I left for a follow-up there.

Checks run:

  • mix test test/ecto/query/planner_test.exs test/ecto/repo_test.exs: 5 new tests (default, invalid values, clear at limit, :infinity, repo option wiring); all pass (331 in the two files); the clear-at-limit test fails on master.
  • mix test: 1617 passed (97 doctests, 1520 tests)
  • mix compile --force --warnings-as-errors: clean
  • mix format --check-formatted: clean. Note: mix format also reformatted a few unrelated lines in planner_test.exs and repo_test.exs that were not formatted on master; happy to drop those hunks if you prefer the diff minimal.

Not run: the integration suites in ecto_sql (no database). Elixir 1.20.2 / OTP 29 only.

Prepared with AI assistance (Claude); I reproduced the growth against Ecto.Query.Planner directly and verified the red-green tests.

The query cache is an ETS set with no eviction policy. Its key is the query shape, and several constructs embed runtime values in the shape: string field names in field/2, interpolated JSON paths in json_extract_path/2, identifier/1, constant/1 and the :prefix option. Each distinct value therefore adds a new entry holding the planned query and the adapter prepared statement that is never released, so applications that let request data reach those constructs grow their memory without bound.

Add a :query_cache_max_size repository option (default 10_000, :infinity to disable). The limit is only checked on cache misses and, once reached, the cache is cleared before inserting the new query. Document the per-value caching behaviour in field/2 and json_extract_path/2 and point to query_cache: false for user-controlled values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant