Skip to content

[ty] Preserve stdlib identity when search paths overlap - #29064

Open
charliermarsh wants to merge 1 commit into
mainfrom
charlie/fix-ty-523
Open

charliermarsh wants to merge 1 commit into
mainfrom
charlie/fix-ty-523

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Summary

We now preserve the identity of standard-library stubs when a custom typeshed overlaps a first-party or extra search path. This fixes the module-identification error behind the dependency-cycle panic in astral-sh/ty#523.

The two configuration settings establish different import roots:

  • typeshed selects the standard-library stubs to use instead of our bundled stubs. With typeshed = "./typings", we look under ./typings/stdlib/.
  • extra-paths adds high-priority import roots for Python modules and stubs. With extra-paths = ["./typings"], we search ./typings/ directly.

For example, a custom stub distribution can contain both additional modules and standard-library replacements:

project/
├── main.py
├── ty.toml
└── typings/
    ├── device.pyi
    └── stdlib/
        ├── VERSIONS
        ├── builtins.pyi
        ├── typing.pyi
        └── pathlib.pyi
# ty.toml
[environment]
root = ["."]
typeshed = "./typings"
extra-paths = ["./typings"]

Here, root identifies our first-party import roots. An import device finds typings/device.pyi through the extra path, while import pathlib finds typings/stdlib/pathlib.pyi through typeshed. This overlapping layout, as used by the MicroPython stubs in #2819, remains supported.

The ambiguity arises when we identify a file we are already analyzing. Depending on the search root, the same typings/stdlib/typing.pyi file can have three names:

Search root Module name
typings/stdlib/ typing
typings/ stdlib.typing
The project directory typings.stdlib.typing

Previously, an earlier extra or first-party path could determine that identity, even when we originally found the file through a standard-library import. We then failed to recognize its builtin or typing definitions as having special behavior. In the reported case, inference followed paths that formed an unhandled dependency cycle and panicked. Losing that special handling can also produce incorrect types even when inference does not panic.

We now derive the identity of standard-library stub files from the configured stdlib root first: builtins.pyi is builtins, and typing.pyi is typing. This preserves the semantics needed to check ordinary Python code with the configuration above:

from typing import Literal, reveal_type

def add(a: int, b: int) -> int:
    return a + b

reveal_type(add(1, 2))  # int
add(1, "2")  # invalid-argument-type

value: Literal[1] = 1
reveal_type(value)  # Literal[1]

We resolve the canonical name using normal precedence and Python-version availability, and verify that it selects the same file. If it is shadowed or unavailable, we do not fall back to an alias under an enclosing search path.

We also change how we handle an exact duplicate of the stdlib import root:

[environment]
root = ["."]
typeshed = "./typings"
extra-paths = ["./typings/stdlib", "./overrides"]

We now search ./typings/stdlib only at the stdlib position, after the other extra paths and project roots. In this example, overrides/pathlib.pyi wins. The same applies when the stdlib directory appears in root.

Listing the stdlib directory in extra-paths or root no longer lets users:

  • Give it priority over other extra paths or project roots.
  • Import .py files from it through that entry.
  • Import stubs that its VERSIONS file excludes for the configured Python version.

The configuration is still accepted, but code relying on those behaviors can break. The earlier extra-paths = ["./typings"] layout still supports additional stubs such as device.pyi. Runtime lookup is unchanged.

Closes astral-sh/ty#523.

@charliermarsh charliermarsh added the ty The ty type checker label Oct 1, 2026
@astral-sh-bot

astral-sh-bot Bot commented Oct 2, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 98.24%. The percentage of expected errors that received a diagnostic held steady at 98.24%. The number of fully passing files held steady at 134/146.

@astral-sh-bot

astral-sh-bot Bot commented Oct 2, 2026

Copy link
Copy Markdown

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot

astral-sh-bot Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Large timing changes:

Project Old Time New Time Change
git-revise 0.12s 0.05s -57%

Full report with detailed diff (timing results)

@charliermarsh
charliermarsh marked this pull request as ready for review October 2, 2026 01:43
@charliermarsh
charliermarsh requested review from a team as code owners October 2, 2026 01:43
@astral-sh-bot
astral-sh-bot Bot requested a review from ibraheemdev October 2, 2026 01:43
@ibraheemdev
ibraheemdev removed request for a team and ibraheemdev October 2, 2026 17:59
@ibraheemdev ibraheemdev closed this Oct 2, 2026
@ibraheemdev ibraheemdev reopened this Oct 2, 2026
@astral-sh-bot
astral-sh-bot Bot requested a review from dhruvmanila October 2, 2026 18:00
@ibraheemdev
ibraheemdev deployed to automations October 2, 2026 18:00 — with GitHub Actions Active
@charliermarsh
charliermarsh requested review from AlexWaygood and removed request for dhruvmanila October 7, 2026 01:40

This branch was successfully deployed

1 active deployment
automations — f59dcb6f Deployed Oct 7, 2026 by charliermarsh via security-review / security review #90911
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty The ty type checker

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panic if --typeshed is set to a subdirectory of (or the same path as) a first-party or extra search path

2 participants