Repository navigation
Fix CodeQL scanning on GitHub and Azure DevOps - #401
Conversation
Default setup failed for five languages: the JavaScript extractor overflows its stack on a deep parser test, C/C++ runs out of memory on vendored code, and Swift, Ruby and Java/Kotlin have nothing it can build. Scan Actions, C/C++, JavaScript/TypeScript and Python without a build, and exclude external/, tests and benchmarks from analysis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Maintainer validation is needed because enabled default setup blocks SARIF processing, while the still-running C/C++ analysis leaves successful completion, coverage, and memory usage unverified.
No open issues 🎉
What changed in this PR
Moves Hermes Windows code scanning to CodeQL advanced setup to address extraction failures and restore C/C++ scanning while excluding vendored code and test fixtures.
Changes:
- Adds no-build scans for Actions, C/C++, JavaScript/TypeScript, and Python on pushes, pull requests, and a weekly schedule.
- Configures path exclusions to avoid problematic fixtures and reduce analysis memory usage.
- Documents the requirement to keep default setup disabled so workflow uploads are accepted.
| File | Description |
|---|---|
| AGENTS.md | Documents the CodeQL configuration locations and default-setup restriction. |
| .github/workflows/codeql.yml | Adds the four-language scanning workflow and preserves existing alert categories. |
| .github/codeql/codeql-config.yml | Excludes vendored code, tests, benchmarks, and Yarn-managed files from scanning. |
Review effort: Balanced
The no-build C/C++ analysis takes about 30 minutes. Run it on pushes to main, weekly and on demand; PRs scan only the fast languages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Benchmark ResultsTotal benchmarks: 204 Inputs: baseline/baseline.json, bench_result.json v8 (6176ms, 5005ms) -19.0%
test-suites (317886ms, 234219ms) -26.3%
micros (60876ms, 55501ms) -8.8%
jit-benches (8563ms, 5650ms) -34.0%
many-subclasses (73437ms, 54542ms) -25.7%
map-objects (3627ms, 2754ms) -24.1%
map-strings (4335ms, 3542ms) -18.3%
nbody (3448ms, 2133ms) -38.2%
string-switch (6503ms, 5242ms) -19.4%
raytracer (5608ms, 4379ms) -21.9%
MiniReact (30332ms, 24180ms) -20.3%
widgets (12934ms, 9829ms) -24.0%
|
Run all four CodeQL languages on pull requests again; the C/C++ scan takes about as long as the Azure PR build. Add CodeQL.yml so the Azure DevOps CodeQL 3000 scan classifies vendored code as library and tests and benchmarks as test, keeping their findings out of SDL bug filing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The JavaScript extractor overflows its stack on the deeply nested expressions in test/Parser/nested-expressions.js, which fails the whole interpreted-language database (JavaScript, Python, PowerShell, Actions) in the 1ES SDL Sources job. CodeQL 3000 offers no documented setting that changes what is extracted, so set LGTM_INDEX_FILTERS, which the interpreted-language extractors read from the environment, in a shared variables template used by the CI, release and PR pipelines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert before merge. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CodeQL 3000 does not run on pull branches, so check instead that the multi-line pipeline variable reaches job tasks intact. Revert before merge. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The PR build showed all six filter lines reach job tasks intact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
The fix relies on an undocumented extraction filter and injected Azure DevOps tasks unavailable here, while main-branch scanning and GitHub uploads still require end-to-end validation after disabling default setup.
No open issues 🎉
Review effort: Balanced
bef618e
into
microsoft:main
Summary
The Security overview reports a code scanning configuration error. CodeQL
default setup fails for five of its seven languages:
StackOverflowErroron thedeliberately deep
test/Parser/nested-expressions.jsparser test.vendored code in
external/. Default setup then dropped C/C++, so theengine is not being scanned on GitHub.
The same parser test also breaks the interpreted-language scan in the Azure
DevOps pipelines: CodeQL 3000 builds one database for JavaScript, Python,
PowerShell and Actions, and the crash fails all of them. CodeQL 3000 has no
documented setting that changes what is extracted, but the extractors read
LGTM_INDEX_FILTERSfrom the environment. The C++ scan in Azure DevOps isunaffected.
.github/workflows/codeql.ymlscans Actions, C/C++,JavaScript/TypeScript and Python without a build, on pushes and PRs to
main, weekly and on demand. It keeps the/language:<lang>categoriesso existing alerts carry over.
.github/codeql/codeql-config.ymlexcludesexternal,test,unittests,benchmarks,.yarnand**/__tests__. This avoidsthe extractor crash and keeps C/C++ within runner memory.
.ado/codeql-variables.ymlsetsLGTM_INDEX_FILTERSto exclude the same paths as the GitHub config. TheCI, release and PR pipelines include it; traced C++ builds ignore it.
CodeQL.ymlmarks vendored code aslibraryand tests and benchmarks astest, so CodeQL 3000 does not filetheir findings as SDL bugs. It does not change what is extracted.
setup must stay off.
After merge, GitHub default setup must be switched off in Settings → Advanced
Security, and the stale Swift, Ruby, Java/Kotlin and 2022
codeql-analysis.ymlconfigurations deleted from the CodeQL status page.Test Plan
actionlint .github/workflows/codeql.yml(v1.7.12): passed.All three YAML files parse with PyYAML; every
CodeQL.ymlpath exists.git diff --check: passed.The first PR run analyzed all four languages; C/C++ finished in 28 minutes
on a standard runner without running out of memory. Every job failed only
at upload, because GitHub rejects advanced-setup results while default
setup is enabled. They should pass once default setup is switched off.
Local CodeQL CLI 2.27.0 on a clean checkout, using the CodeQL 3000
commands from the CI log (
database init --build-mode none --db-cluster,database trace-command --use-build-mode,database finalize):StackOverflowErrorontest/Parser/nested-expressions.js, exit code 2 (same as CI);LGTM_INDEX_FILTERS: JavaScript, Python and Actions databasesfinalize; no files from the excluded folders;
lib/InternalJavaScript,API/hermes/extensions,.ado/scriptsand workflows are included; noquery-pack downloads.
A temporary step in this PR's Azure build printed all six
LGTM_INDEX_FILTERSlines intact; it has been removed.CodeQL 3000 does not run on pull branches, so the CI scan of
mainis thefirst real run; its cadence allows one after 2026-11-02 22:15 UTC.
No engine build or tests were run: only CI configuration and docs changed.