Skip to content

[ty] Avoid recursive protocol intersection stack overflows - #29059

Open
charliermarsh wants to merge 6 commits into
mainfrom
charlie/ty-4099-protocol-intersection
Open

[ty] Avoid recursive protocol intersection stack overflows#29059
charliermarsh wants to merge 6 commits into
mainfrom
charlie/ty-4099-protocol-intersection

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

We now avoid a stack overflow when checking recursive protocol methods with intersection types:

from __future__ import annotations
from typing import Protocol, reveal_type
from ty_extensions import Intersection, Not

class A[T](Protocol):
    def make_invariant(self, value: T) -> T: ...
    def cause_problems(self) -> A[Intersection[Not[A[T]], A[str]]]: ...

def foo[T](x: A[T]):
    reveal_type(x.cause_problems().cause_problems())

Specifically, we now specialize protocol members only when they are accessed or compared, avoiding eager expansion of recursive return types.

Closes astral-sh/ty#4099.

@charliermarsh charliermarsh added the ty The ty type checker label Oct 1, 2026
@astral-sh-bot

astral-sh-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 98.24%. The percentage of expected errors that received a diagnostic held steady at 98.24%. The number of fully passing files held steady at 134/146.

@astral-sh-bot

astral-sh-bot Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
flake8 38.28MB 38.18MB -0.24% (95.98kB) ⬇️
trio 92.60MB 92.31MB -0.31% (289.55kB) ⬇️
sphinx 172.51MB 172.03MB -0.28% (496.49kB) ⬇️
prefect 475.17MB 473.96MB -0.25% (1.20MB) ⬇️

Significant changes

Click to expand detailed breakdown

flake8

Name Old New Diff Outcome
ProtocolInterface 55.13kB 9.45kB -82.87% (45.69kB) ⬇️
CallableType 339.59kB 314.17kB -7.48% (25.41kB) ⬇️
cached_protocol_member 0.00B 23.53kB +23.53kB (new) ⏫
FunctionType 298.06kB 280.68kB -5.83% (17.38kB) ⬇️
cached_protocol_interface 19.13kB 3.34kB -82.56% (15.80kB) ⬇️
cached_protocol_member::interned_arguments 0.00B 12.80kB +12.80kB (new) ⏫
GenericAlias<'db>::try_mro_ 229.25kB 216.46kB -5.58% (12.79kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 146.88kB 138.05kB -6.01% (8.83kB) ⬇️
Type<'db>::apply_specialization_inner_ 96.23kB 90.05kB -6.42% (6.18kB) ⬇️
when_constraint_set_assignable_to_owned_impl 80.56kB 83.03kB +3.06% (2.47kB) ⏫
Specialization 247.25kB 245.91kB -0.54% (1.34kB) ⬇️
TypePair 266.72kB 265.41kB -0.49% (1.31kB) ⬇️
is_equivalent_to_object_inner 10.84kB 12.04kB +11.02% (1.20kB) ⏫
infer_expression_types_impl 786.21kB 787.23kB +0.13% (1.02kB) ⏫
infer_definition_types 1.17MB 1.17MB +0.08% (1016.00B) ⏫
... 39 more

trio

Name Old New Diff Outcome
ProtocolInterface 200.94kB 42.58kB -78.81% (158.36kB) ⬇️
cached_protocol_member 0.00B 85.20kB +85.20kB (new) ⏫
CallableType 1.63MB 1.56MB -4.47% (74.84kB) ⬇️
cached_protocol_interface 73.48kB 19.88kB -72.95% (53.61kB) ⬇️
FunctionType 1.20MB 1.16MB -3.88% (47.74kB) ⬇️
cached_protocol_member::interned_arguments 0.00B 46.15kB +46.15kB (new) ⏫
GenericAlias<'db>::try_mro_ 735.45kB 691.81kB -5.93% (43.64kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 932.27kB 892.19kB -4.30% (40.08kB) ⬇️
Type<'db>::apply_specialization_inner_ 654.41kB 626.20kB -4.31% (28.21kB) ⬇️
infer_definition_types 4.68MB 4.69MB +0.24% (11.36kB) ⏫
infer_expression_types_impl 5.97MB 5.97MB +0.16% (9.79kB) ⏫
Specialization 1.05MB 1.05MB -0.53% (5.72kB) ⬇️
when_constraint_set_assignable_to_owned_impl 510.98kB 515.77kB +0.94% (4.80kB) ⏫
is_equivalent_to_object_inner 36.12kB 39.98kB +10.68% (3.86kB) ⏫
GenericAlias 373.22kB 370.05kB -0.85% (3.16kB) ⬇️
... 44 more

sphinx

Name Old New Diff Outcome
ProtocolInterface 322.68kB 29.92kB -90.73% (292.77kB) ⬇️
cached_protocol_member 0.00B 155.95kB +155.95kB (new) ⏫
CallableType 2.05MB 1.90MB -7.27% (152.48kB) ⬇️
cached_protocol_interface 113.33kB 12.59kB -88.89% (100.73kB) ⬇️
FunctionType 1.78MB 1.69MB -5.24% (95.57kB) ⬇️
GenericAlias<'db>::try_mro_ 1.41MB 1.33MB -6.07% (87.91kB) ⬇️
cached_protocol_member::interned_arguments 0.00B 84.39kB +84.39kB (new) ⏫
Type<'db>::apply_specialization_inner_::interned_arguments 1.16MB 1.11MB -4.15% (49.22kB) ⬇️
Type<'db>::apply_specialization_inner_ 775.89kB 741.40kB -4.45% (34.49kB) ⬇️
when_constraint_set_assignable_to_owned_impl 923.67kB 951.22kB +2.98% (27.55kB) ⏫
infer_expression_types_impl 11.66MB 11.68MB +0.22% (25.87kB) ⏫
infer_definition_types 11.71MB 11.73MB +0.12% (14.00kB) ⏫
Specialization 1.59MB 1.58MB -0.58% (9.44kB) ⬇️
infer_scope_types_impl 6.20MB 6.21MB +0.12% (7.46kB) ⏫
is_equivalent_to_object_inner 59.04kB 65.74kB +11.35% (6.70kB) ⏫
... 43 more

prefect

Name Old New Diff Outcome
ProtocolInterface 910.42kB 113.59kB -87.52% (796.83kB) ⬇️
cached_protocol_member 0.00B 405.67kB +405.67kB (new) ⏫
CallableType 8.94MB 8.65MB -3.27% (299.47kB) ⬇️
cached_protocol_interface 304.23kB 40.35kB -86.74% (263.88kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 4.10MB 3.88MB -5.33% (223.98kB) ⬇️
cached_protocol_member::interned_arguments 0.00B 217.08kB +217.08kB (new) ⏫
GenericAlias<'db>::try_mro_ 4.56MB 4.36MB -4.34% (202.79kB) ⬇️
FunctionType 5.73MB 5.53MB -3.38% (198.32kB) ⬇️
Type<'db>::apply_specialization_inner_ 2.96MB 2.80MB -5.21% (157.93kB) ⬇️
infer_definition_types 37.83MB 37.93MB +0.29% (110.41kB) ⏫
when_constraint_set_assignable_to_owned_impl 3.58MB 3.65MB +2.01% (73.66kB) ⏫
infer_expression_types_impl 24.05MB 24.10MB +0.19% (46.50kB) ⏫
Specialization 4.87MB 4.83MB -0.72% (35.69kB) ⬇️
GenericAlias 1.80MB 1.79MB -1.07% (19.76kB) ⬇️
is_redundant_with_impl 3.10MB 3.12MB +0.62% (19.59kB) ⏫
... 66 more

@astral-sh-bot

astral-sh-bot Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.

Full report with detailed diff (timing results)

@codspeed

codspeed Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will improve performance by 5.25%

⚡ 1 improved benchmark
✅ 175 untouched benchmarks
⏩ 60 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation ty_micro[pandas_tdd] 904.9 ms 859.8 ms +5.25%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

Comparing charlie/ty-4099-protocol-intersection (206a543) with main (c33cdaa)

Open in CodSpeed

Footnotes

  1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

astral-automations-bot[bot]
astral-automations-bot Bot reviewed Oct 1, 2026
View reviewed changes
Comment thread crates/ty_python_semantic/src/types/protocol_class.rs Outdated
@charliermarsh
charliermarsh force-pushed the charlie/ty-4099-protocol-intersection branch from 179bd63 to 0c82815 Compare October 1, 2026 22:02
@charliermarsh charliermarsh added the bug An issue describing something that isn't working, or a PR that fixes a bug label Oct 1, 2026
@charliermarsh
charliermarsh changed the base branch from main to charlie/ty-call-inference-generic-context October 1, 2026 22:17
@charliermarsh
charliermarsh force-pushed the charlie/ty-4099-protocol-intersection branch from aa625b8 to 7503019 Compare October 1, 2026 22:18
Base automatically changed from charlie/ty-call-inference-generic-context to main October 1, 2026 23:17
@charliermarsh
charliermarsh force-pushed the charlie/ty-4099-protocol-intersection branch from 7503019 to 9a5050f Compare October 1, 2026 23:41
@charliermarsh
charliermarsh marked this pull request as ready for review October 2, 2026 00:10
@charliermarsh
charliermarsh requested a review from a team as a code owner October 2, 2026 00:10
@astral-sh-bot
astral-sh-bot Bot requested a review from dcreager October 2, 2026 00:10
@charliermarsh
charliermarsh force-pushed the charlie/ty-4099-protocol-intersection branch from 6cbb3d6 to c88701f Compare October 5, 2026 14:43
@charliermarsh
charliermarsh force-pushed the charlie/ty-4099-protocol-intersection branch from c88701f to 206a543 Compare October 7, 2026 02:00

This branch was successfully deployed

1 active deployment
automations — 206a5435 Deployed Oct 7, 2026 by charliermarsh via security-review / security review #90912
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@astral-automations-bot astral-automations-bot[bot] astral-automations-bot[bot] left review comments
@dcreager dcreager Awaiting requested review from dcreager

Assignees

No one assigned

Labels

bug An issue describing something that isn't working, or a PR that fixes a bug ty The ty type checker

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Stack overflow on invariant protocol nested intersection not self and unrelated specialization twice

1 participant

@charliermarsh