Mark HTTP request functions VOLATILE to prevent double execution on constant arguments - #37
Merged
rustyconover merged 1 commit intoSep 8, 2026
Conversation
…onstant arguments http_get/http_post/http_post_form/http_head register with the default FunctionStability::CONSISTENT, which lets the optimizer evaluate an all-constant call at plan time (constant folding) in addition to execution time. Concretely, nesting a constant-arg call in a larger expression -- SELECT http_post(url, ...)->>'access_token' -- sends the request twice per query, while still returning a single correct row, so the duplicate is only visible server-side. For non-idempotent endpoints (OAuth code exchange, webhooks, row-creating POSTs) that is a correctness bug. Network requests are side effects, so these functions must be VOLATILE (same class as random()/nextval()). Verified against a request-counting server: the nested constant-arg call sends 2 requests on a stock build and exactly 1 after this change; bare calls and column/bind arguments already sent once and are unaffected.
lmangani
approved these changes
Sep 8, 2026
Contributor
|
I'll get the release out shortly. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an HTTP function call with constant (literal) arguments is nested inside a larger expression, the request is sent twice per query:
The query result is still a single, correct row — the duplicate is only visible on the server side. For idempotent GETs that is silently doubled load; for side-effecting requests it is a correctness bug with no client-side symptom. The sharpest example is an OAuth authorization-code exchange:
http_post(token_url, ...)->>'access_token'is exactly the idiom this fires on, and the second send replays a one-time code.Two things narrow when it manifests:
SELECT http_post(...)sends once;http_post(...)->>'status'sends twice.Root cause
On current
v1.5(f0fedb2),http_get/http_post/http_post_form/http_headare registered with the defaultFunctionStability::CONSISTENT. That tells DuckDB's optimizer the function is pure, so an all-constant sub-expression is evaluated during constant folding at plan time — and evaluated again at execution time. Two evaluations → two network requests.HTTP requests are side-effecting, so these functions should be
VOLATILE(the same classification DuckDB uses forrandom(),nextval(),uuid()), which excludes them from constant folding and pins one evaluation per logical call.main/v1.5.5already do this viaHttpFn(function.SetVolatile()), landed with PR #36. This PR is the same flag on the four functions for thev1.5line only — not a reopen of #34 (that branch mixed telemetry, submodules, and a merge frommain).Fix
SetVolatile()on each registeredScalarFunction(http_head, bothhttp_getoverloads,http_post,http_post_form). No telemetry, submodule, or Vector-API changes.No behavior change for column/bind arguments (already one send); nested constant-argument calls now send once instead of twice.
Verification
A local HTTP target that logs every inbound request, fired with the nested constant-arg call above (same query, same target, only the extension binary differs):