Skip to content

fix: 修复 llms.txt 目录重复并排除跳转页 - #11

Open
qqqqqf-q wants to merge 1 commit into
felinics:mainfrom
qqqqqf-q:fix/memoh-207-llms-txt-dedupe
Open

qqqqqf-q wants to merge 1 commit into
felinics:mainfrom
qqqqqf-q:fix/memoh-207-llms-txt-dedupe

Conversation

@qqqqqf-q

Copy link
Copy Markdown
Member

这个 PR 做什么

修复 https://docs.memoh.ai/llms.txt 的两个问题:目录重复,以及混入旧路径的跳转页。agent 通过 memoh-help skill 读取这个目录来找文档页面,这两个问题会让它多读大量无效内容。

Linear:MEMOH-207

原因

  • 目录重复 3 遍:每种语言的 sidebar(docs/.vitepress/{en,zh,ja,ru}.ts)都把 /guides/、/integrations/、/self-hosted/ 三个 key 指向同一个 references 对象。vitepress-plugin-llms@1.14.0 生成目录时会展开全部 key(Object.values(sidebar).flat()),同一个对象就输出了 3 次。网站本身只按当前路径取一个 sidebar,所以页面上看不出问题;llms-full.txt 对文件做了去重,也不受影响。
  • 152 条 Redirecting:旧 URL 以 frontmatter 为 title: Redirecting 的空页面保留,页面里只有一段客户端跳转脚本。它们不在 sidebar 里,被插件归到了目录末尾的 ### Other。

改动

只改 docs/.vitepress/config.ts 中传给插件的配置,网站 sidebar 和页面内容都不变。

  • 给插件单独传一份去重后的 sidebar:[...new Set(Object.values(sidebar))].flat()。
  • 构建时扫描 frontmatter 为 title: Redirecting 的页面,通过 ignoreFilesPerOutput 从 llms.txt 和 llms-full.txt 中排除。插件按输出路径匹配目录下的 index.md(channels/index.md 对应 channels.md),所以列表也做了同样的换算。以后新增跳转页会自动排除,不用手动维护名单。
  • 跳转页仍然生成单页 .md,旧链接的行为不变。

验证

本地执行 pnpm build,对比修改前后在 main 上的构建产物:

产物 修改前 修改后
llms.txt 大小 34,789 B 9,907 B
llms.txt 中 Redirecting 条目 152 0
llms.txt 各语言目录 每种语言 3 份 en/zh/ja/ru 各 1 份
llms-full.txt 页面数 319 167(只去掉了跳转页)
单页 .md 文件列表 319 个 相同
  • HTML 页面:抽查 guides/container.html,除资源文件名的 hash 外与修改前一致。main 自身连续构建两次 hash 也会变化,这部分差异与本次改动无关。
  • 没有在浏览器里检查网站页面;已发布环境的效果要等合并部署后才能确认。

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T16:45:29.428623Z bbafea9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bbafea932b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/.vitepress/config.ts
return entry.name.startsWith('.') || entry.name === 'public' ? [] : redirectPages(path)
}
if (!entry.name.endsWith('.md')) return []
return /^---\n(?:[^\n]*\n)*?title: Redirecting\n/.test(readFileSync(path, 'utf8'))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Accept CRLF when identifying redirect stubs

On Git-for-Windows checkouts using the common core.autocrlf=true, Markdown lines end in \r\n, so this LF-only regex fails at the opening ---\n and redirectPages() returns no redirect files. A pnpm build in that environment therefore puts all 152 redirect stubs back into llms.txt and llms-full.txt; parse the frontmatter or accept \r?\n so filtering is checkout-independent.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant