Skip to content

Document the columnar output formats; fix the mkdocs nav - #335

Merged
amc-corey-cox merged 1 commit into
mainfrom
docs-output-formats
Aug 25, 2026
Merged

amc-corey-cox merged 1 commit into
mainfrom
docs-output-formats

Conversation

@amc-corey-cox

Copy link
Copy Markdown
Contributor

Light docs pass before the release, focused on where the docs are wrong rather than merely thin.

Three passages contradicted the shipped CLI

Parquet and DuckDB (#333) landed undocumented, which left index.md actively misleading in three places:

  1. The -f/--output-format list omitted parquet and duckdb.
  2. The -O extension list omitted .parquet and .duckdb.
  3. Worst of the three — the architecture section said output was "intentionally limited to text-based representations" and told readers to "use the appropriate downstream tool" for loading into DuckDB or Parquet. That now steers people away from the feature this release exists for.

Added a Columnar Output section covering the nested-STRUCT behaviour, --table-name, and accumulating several runs into one .duckdb file.

Everything here was verified, not transcribed

I ran each documented command against tests/input/examples/tabular rather than paraphrasing #333's description:

  • Parquet output — produced a real artifact
  • Two runs at one .duckdb path with different --table-name values → SHOW TABLES returns ['person', 'person_csv'], confirming accumulation
  • The STRUCT claim is asserted by existing tests (STRUCT(value_decimal DOUBLE, unit VARCHAR)), so I cited behaviour that's already pinned
  • "Adds no dependency" — duckdb was already required by the join engine

SQL backend caveat sharpened

It was described as "experimental" with "a limited subset". The word that was missing is silently: expr, value, case(), unit conversion and value mappings are dropped without raising, so a spec using any of them compiles to SQL that quietly omits those slots. That's a correctness trap, not a feature gap, and it now says so.

mkdocs nav

The Examples and Specification sections used YAML mappings where mkdocs wants lists, so mkdocs build --strict aborted on two config warnings before reaching any content checks. Pre-existing on main; fixed here since it's cheap and it makes strict builds usable.

Not fixed here

With the nav corrected, strict mode surfaces 21 broken image links in docs/schema/overview.md — references to img/Mapping Between LinkML Schemas*.png that have never existed in the repo. That file is generated by make gendoc, so the fix belongs in the generator or the source schema, not a docs edit. Filing separately. CI runs mkdocs gh-deploy without --strict, so this isn't newly broken — just newly visible.

Normal mkdocs build succeeds, as before.

The Parquet and DuckDB formats shipped undocumented, and three passages in
index.md were left saying the opposite of what the CLI now does:

- the -f format list omitted parquet and duckdb
- the -O extension list omitted .parquet and .duckdb
- the architecture section told readers that output was "intentionally limited
  to text-based representations" and to reach for a downstream tool for exactly
  the thing map-data now does natively

Adds a Columnar Output section covering nested STRUCT columns, --table-name,
and accumulating several runs into one .duckdb file. Every command and claim
here was run against tests/input/examples/tabular first.

Also states plainly that the SQL backend drops expr, value, case(), unit
conversion and value mappings *silently*, since a spec using them compiles to
SQL that quietly omits those slots.

The nav used mappings where mkdocs wants lists, which made `mkdocs build
--strict` abort on two config warnings before reaching any content checks.
Copilot AI lite review requested due to automatic review settings August 25, 2026 17:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the project documentation to accurately reflect the currently shipped CLI capabilities for columnar outputs (Parquet and DuckDB) and fixes the MkDocs navigation so strict builds don’t fail due to nav schema warnings.

Changes:

  • Corrects map-data output format documentation to include parquet and duckdb, including extension-based inference and -O additional outputs.
  • Adds a “Columnar Output” section documenting nested STRUCT preservation and DuckDB --table-name / multi-run accumulation behavior.
  • Fixes mkdocs.yml nav structure by converting Examples and Specification entries from mappings to lists (MkDocs-compatible).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
mkdocs.yml Fixes nav structure to satisfy MkDocs’ expected list syntax and avoid strict build warnings.
docs/index.md Updates CLI docs to include Parquet/DuckDB output formats and documents columnar output behavior and SQL backend caveats.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@amc-corey-cox
amc-corey-cox merged commit f18e3f5 into main Aug 25, 2026
10 checks passed
@amc-corey-cox
amc-corey-cox deleted the docs-output-formats branch August 25, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants