Skip to content

Release 2.27.0 — Connectors Directory review fixes - #269

Merged
arkinjo merged 6 commits into
mainfrom
dev
Oct 17, 2026
Merged

arkinjo merged 6 commits into
mainfrom
dev

Conversation

@arkinjo

@arkinjo arkinjo commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator

Release 2.27.0 (MINOR): fixes from the Connectors Directory listing review of 2.26.2, except the privacy policy, which is handled separately.

What changes for clients

  • Every tool has a human-readable title in both Tool.title and annotations.title ("Get database schema (MIE file)" instead of the name-derived "Get Mie File"). New tools get it through **read_only_tool("<title>"); a test enforces it.
  • Tool descriptions state facts instead of giving orders. "CALL THIS TOOL FIRST every turn", "CALL get_MIE_file FIRST", ALWAYS/MANDATORY and "must not be reported as one" are rewritten as what each tool returns and what its results depend on; the measured figures (×45,848 row inflation, etc.) stay. A test rejects the old phrasings.
  • run_sparql names where queries go, generated from endpoints.csv: RDF Portal for most databases, plus the seven databases with their own endpoints (glycosmos, togovar, wikipathways, idsm, lipidmaps, swisslipids, marpolbase).
  • Descriptions are more even in length: 471–2,732 characters, down from 79–3,493. Short ones now name their API and return shape; long ones lost implementation detail, not usage facts.
  • The Usage Guide no longer tells the agent how to write its answer (OUTPUT QUALITY section, "Repetitive answer" row, KEGG "do not report" notes). Query rules are unchanged.

Security

  • The endpoint_url exemption for registered endpoints now requires an exact scheme/host/port/path match. Through 2.26.2 any path on a registered host skipped the public-address check.

Evidence

  • Canary (10 questions × 1 replicate, local, Sonnet 5.5, Opus 4.8 judge): TogoMCP 15.80 before and after the wording change; the guide was called in 10/10 cells both times.
  • Q013, the one question that moved, re-run ×3 on each version: 20/20/20 on both, same SPARQL route.
  • Transcript audits clean in every run (MEMORY 0, VIOLATION 0).
  • uv run pytest: 1,860 passed.

Also includes a benchmark-runner fix: an out-of-repo --canary-file no longer crashes manifest writing.

🤖 Generated with Claude Code

arkinjo and others added 6 commits October 17, 2026 14:11
…ns, R2

- Every tool now carries a human title in both Tool.title and
  annotations.title via **read_only_tool("<title>"); tests enforce
  presence, equality, non-derived form and uniqueness.
- run_sparql names where queries go, built from endpoints.csv: RDF
  Portal for most databases plus the seven external endpoints by URL.
  README, intro pages, CLAUDE.md and sibling tools no longer imply every
  endpoint is on RDF Portal.
- Short descriptions (ncbi_efetch/esummary/list_databases, MeSH, both
  PubChem tools, togoid_getDescription) now state what they return and
  which API they call, checked against the live responses.
- endpoint_url exemption for registered endpoints now requires an exact
  scheme/host/port/path match, so other paths on a registered host (e.g.
  a Virtuoso admin UI) get the public-address check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tyle

- Tool descriptions no longer issue directives ("CALL THIS TOOL FIRST every
  turn", "CALL get_MIE_file FIRST", ALWAYS/MANDATORY, "must not be reported
  as one", CHECK FOR): the same information is stated as what a tool returns
  and what its results depend on; measured figures kept. A test rejects the
  old phrasings in any served description.
- Long descriptions trimmed to <= 2,800 characters (togoid_identifyId,
  search_chembl_*), cutting implementation detail, not usage facts.
- Usage Guide (Option A): OUTPUT QUALITY section, the "Repetitive answer"
  row and STEP 4's style rules removed; the stale-tool-list row and the
  generated KEGG notes state facts instead of telling the agent what to
  say. Query rules unchanged.

Canary (10 Q x 1) vs dev baseline: TogoMCP 15.80 vs 15.80 (Opus judge),
guide called 10/10 in both; Q013 re-run x3 on each: 20/20/20 both, same
SPARQL route. Transcript audits clean (MEMORY 0, VIOLATION 0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… --canary-file

Path.relative_to(REPO) raised after answering and judging were paid for,
leaving the run without manifest.json/report.md. Record the absolute path
instead when the file is outside the repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng, registered-path SSRF fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@arkinjo
arkinjo merged commit 6670e06 into main Oct 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant