Skip to content
View roli-lpci's full-sized avatar

Organizations

@hermes-labs-ai

Block or report roli-lpci

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
roli-lpci/README.md

Roli Bosch

Founder of Hermes Labs, an AI reliability engineering lab building tools, research, and autonomous infrastructure for agents and LLM systems.

I work on the operational language layer of AI systems: instructions, tool interfaces, retrieval, memory, prompt injection, runtime policy, evaluation, and the evidence needed to reconstruct what an agent actually did.

Start here: Hermes Labs · GitHub org · LintLang · Research · Contribution ledger

LinkedIn X Substack Site Hermes Labs on GitHub


Open-source systems

  • LintLang: static linter for agent configs, tool descriptions, and system prompts; no LLM call. lintlang.ai · playground
  • Little Canary: prompt-injection detection through sacrificial canary-model probes. littlecanary.ai
  • Fidelis Memory: agent memory that returns original passages verbatim instead of a paraphrase.
  • Hermes Rubric: evidence-first LLM-as-judge scoring; every dimension ties to a quoted file:line, hedged on thin evidence.
  • Hermes Blind: recovers the original goal of a long agent session from its first turn, for multi-turn drift recovery.
  • zer0dex: local dual-layer memory pattern for agents, pairing a markdown index with local semantic retrieval.

Full catalog with evidence boundaries: hermes-labs.ai/open-source


Evidence in the ecosystem

Hermes Labs has 50+ merged external contributions and ecosystem PRs across AI frameworks, agent infrastructure, developer tooling, documentation, integrations, and public technical systems. The public contribution ledger keeps the categories separate; the underlying record is also available as machine-readable JSON.

  • LintLang runs in Character.AI's Larch CI as an operational lint dependency, documented in Larch's own linting docs (scoped to Larch's public repo).
  • LintLang is a catalog plugin in MegaLinter, merged upstream as oxsecurity/megalinter#8899 (2026-09-11); a catalog listing, not an install-count claim.
  • Independent Gentoo ebuilds for LintLang are maintained across releases in the Haven overlay, packaged without our involvement.
  • zer0dex changed an external team's roadmap: AllSource documented how their architecture moved from zer0dex's pattern to their own system (roadmap influence, not a dependency), and Mnemosyne independently implemented a related approach.
  • Little Canary's technique was adapted in Preflight with attribution, and named as the closest prior work in ProxyCanary's manuscript (a repository-hosted manuscript citation, not peer review).
  • Hermes Rubric is listed in Inspect AI's extensions gallery (accepted upstream docs PR, UKGovernmentBEIS/inspect_ai#5265, merged 2026-09-09).
  • A CISA Vulnrichment score correction: CVE-2026-14216 was stored at CVSS 5.3 against a 6.5 vector; a collaborator confirmed the correction and republication in cisagov/vulnrichment#333 (case study). A scoring correction, not a vulnerability discovery, partnership, or endorsement.

Upstream engineering

Merged fixes in AI frameworks and agent infrastructure, each with a regression test: Microsoft Semantic Kernel #13610 (a chat-history truncation reducer was silently deleting the system prompt), LangChain #35544 (dropped a forced tool_choice that crashed Anthropic extended-thinking requests), DSPy #9978 (an empty eval devset failed with a bare ZeroDivisionError instead of a descriptive one), crewAI #7369 (memory access times were updated even when a read was marked read_only; merged 2026-09-15), and Hindsight #4285 (added task-local retain suspension to the Python client; merged 2026-09-15). A related Mem0 patch fixing a Redis cosine-distance-to-similarity conversion closed unmerged after a maintainer acknowledged the conversion in a broader sweep. It is not a merged contribution.


Autonomous engineering

Many contributions from this account are independently discovered and executed by Hermes Labs' autonomous engineering infrastructure, rather than beginning with me selecting or prompting the specific task. I set objectives, operating constraints, and authorization boundaries; steer or review where needed; and remain the responsible human for work published from this account. The machinery is part of the experiment: can an AI-native engineering institution notice useful work, investigate it, make bounded changes, preserve evidence, and stop appropriately without requiring a human to originate every individual action?


Research

Hermes Labs publishes research and technical notes on AI reliability, epistemic failure, measurement validity, agent-tool semantics, prompt injection, and the limits of model self-report. Current work includes Tool Differentia, Behavioral Canarying for Prompt Injection, The Generative Horizon, Precise Records, Unstable Meanings, A Taxonomy of Epistemic Failure Modes in Large Language Models, and The Asymmetric Burden of Proof.

hermes-labs.ai/research · hermes-labs-ai/hermes-publications


Where this came from

Hermes Labs emerged from an exploration of philosophy of language, phenomenology, and hermeneutics applied to AI systems. The practical consequence became an engineering thesis: in agent systems, language is part of the runtime. System prompts, tool descriptions, retrieved passages, memory, summaries, policies, and evaluation criteria do not merely describe a system. They participate in what it notices, chooses, remembers, and does. Hermes Labs treats that layer as an engineering surface: something that can be inspected before deployment, tested under adversarial conditions, controlled at runtime, and verified afterward.


hermes-labs.ai · github.com/hermes-labs-ai

Pinned Loading

  1. hermes-labs-ai/little-canary hermes-labs-ai/little-canary Public

    Detects prompt injection by its effect on a sacrificial canary model, not just pattern matching: untrusted input hits a powerless model first, a behavioral check reads the residue, and it returns b…

    Python 30 4

  2. hermes-labs-ai/lintlang hermes-labs-ai/lintlang Public

    Static analysis for AI agent configs, tool descriptions, and system prompts — catches vague tool descriptions, missing stop conditions, and schema gaps before they reach runtime. Zero-LLM, determin…

    Python 72 3

  3. hermes-labs-ai/zer0dex hermes-labs-ai/zer0dex Public

    A local dual-layer memory pattern for AI agents: a compact, human-readable markdown index paired with semantic retrieval from a local vector store, queried before each message. For cross-project re…

    Python 59 5

  4. hermes-labs-ai/fidelis hermes-labs-ai/fidelis Public

    Zero-LLM agent memory for Claude Code and AI agents: local-first BM25, dense-vector, and reciprocal-rank-fusion retrieval. Returns original passages verbatim by default. Available on PyPI as fideli…

    Python 23 2

  5. hermes-labs-ai/hermes-blind hermes-labs-ai/hermes-blind Public

    Recovers the original goal of a long Claude Code or Codex session from its first user turn, so you can restate it before continuing — plus a prompt wrapper that adds evidence and hedging constraint…

    Python 8

  6. hermes-labs-ai/hermes-rubric hermes-labs-ai/hermes-rubric Public

    Evidence-first LLM-as-judge scoring for AI artifacts — papers, PRs, prompts, cold emails: synthesizes a rubric, collects quoted-evidence citations, scores only against that evidence, and hedges on …

    Python 1