Support local reftable git scans

[VatsalGoel3]

Description:

Fixes #5019.

This PR adds local Git repository support for repositories using Git's extensions.refstorage = reftable format.

The failure came from TruffleHog opening local repositories with go-git before scanning. go-git rejects the refstorage repository extension, even though TruffleHog already uses the system git CLI for log and diff parsing. The new path detects local reftable repositories from Git config and scans them through the existing CLI-backed parser, avoiding the unsupported go-git open path.

What changed:

• detect extensions.refstorage = reftable for working tree, bare, and worktree/common-dir layouts
• route local reftable directories through a system-Git-backed scan path
• resolve --branch, --since-commit, merge-base, and remote URL metadata through system Git on that path
• report reftable system-Git scan failures through the existing ChunkErr path so one bad unit does not abort a multi-unit scan
• keep successful Git CLI stderr separate from stdout when resolving hashes and metadata
• add focused tests for detection, real reftable scans, worktree common-dir detection, invalid reftable chunk errors, and successful-stderr handling

Checklist:

• Tests passing (make test-community)?
• Lint passing (make lint this requires golangci-lint)?

Additional validation run locally:

• CGO_ENABLED=0 go test ./pkg/sources/git -run 'TestRunGitOutputIgnoresSuccessfulStderr|TestChunkUnit_ReftableRepoInvalid|TestIsReftableRepo|TestChunkUnit_ReftableRepo' -count=1
• CGO_ENABLED=0 go test ./pkg/sources/git -run 'TestGitChunk_LongLine|TestPrepareRepoWithNormalization$|TestPrepareRepoWithNormalizationBare$' -count=1
• CGO_ENABLED=0 make test-community
• CGO_ENABLED=0 make lint
• built a local binary and verified trufflehog git --trust-local-git-config --detector-timeout=120s file://. from inside a real git init --ref-format=reftable repo scans successfully

---

Note

Medium Risk
Changes core local git scan routing and adds a parallel metadata/ref-resolution path, but scope is limited to reftable detection and existing CLI parsing is reused; failures are isolated per unit.

Overview
Local directory scans now detect extensions.refstorage = reftable (working tree, bare, and worktree commondir layouts) and skip go-git PlainOpen, which cannot open that format.

Those repos are scanned through a new ScanRepoPath path that still uses the existing CLI-backed log/diff parser. Base/head refs, merge-base, and remote metadata are resolved with git -C instead of go-git. runGitOutput reads stdout only so warnings on stderr do not break successful commands.

scanDir reports detection and scan failures via ChunkErr so other units can continue. Non-reftable local repos keep the previous go-git flow; commit/staged scanning is refactored to share scanRepoPath with an injected remote URL.

^{Reviewed by Cursor Bugbot for commit 3aee2ac. Bugbot is set up for automated code reviews on this repo. Configure here.}

[CLAassistant]

All committers have signed the CLA.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 634ea476ad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Report reftable scan failures via ChunkErr

When this path is used through ChunkUnit, returning the ScanRepoPath error here makes SourceManager record it as Fatal (source_manager.go wraps returned chunking errors that way), unlike the non-reftable path below which sends scan failures to reporter.ChunkErr. For a reftable unit with an invalid base/head ref or object error in a multi-unit job, this aborts the scan instead of recording a per-unit chunk error; route this error through reporter.ChunkErr as the normal ScanRepo path does.

Useful? React with 👍 / 👎.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit d91308c. Configure here.}