Skip to content

Commit 87a52eb

Browse files
craigthackerxcraigthackerx
committed
Add corporate proxy support, ldo network test and AI.md
- Every HTTPS call, and every az ldo runs, follow one set of proxy rules: loopback and the metadata endpoint direct, then no_proxy and NO_PROXY, then LDO_PROXY_ADDRESS, the config file's proxy, HTTPS_PROXY and the rest, and the operating system's setting; host:port means http://, as for cntlm and Px - The OS certificate store is trusted by default, with the public roots and the config's ca_bundle, in one bundle az is handed as REQUESTS_CA_BUNDLE; LDO_CA_BUNDLE, REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE names one to use exactly instead - ca_bundle now adds to the public roots rather than replacing them, and reaches the Azure CLI - ldo network test asks each service for an unsigned answer and says which proxy each call used, what is trusted, and what to try when one fails - AI.md holds the instructions for AI coding assistants; AGENTS.md is written from it by just ai, CLAUDE.md imports it, and Copilot is pointed at it
1 parent 57449b4 commit 87a52eb

34 files changed

Lines changed: 2130 additions & 19 deletions

‎.github/copilot-instructions.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Instructions for GitHub Copilot
2+
3+
This repository's instructions for AI coding assistants are in [`AI.md`](../AI.md) at the
4+
root (and, word for word, in `AGENTS.md`, which the Copilot coding agent also reads). Read
5+
`AI.md` before suggesting or making any change here.
6+
7+
The rules that matter most:
8+
9+
- Every command reads; nothing changes a tenant or an instance unless asked.
10+
- Runtime dependencies are `requests` and `typer` only; the rest is the standard library.
11+
- Tests never touch the network: use the fakes in `tests/fakes`. Run `just check`.
12+
- Never hard-code `ldo` or `LDO_`: take names from `core/brand.py`.
13+
- UK English, and no em or en dashes anywhere.
14+
- No AI attribution in commits, and nothing is committed until the person asks.

‎AGENTS.md‎

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
<!-- Generated from AI.md by 'just ai'. Edit AI.md, not this file. -->
2+
3+
# Instructions for AI coding assistants
4+
5+
This file is for Claude Code, GitHub Copilot, OpenAI Codex and any other assistant working
6+
in this repository. It is the one to edit: `AGENTS.md` is generated from it (`just ai`),
7+
`CLAUDE.md` imports it, and `.github/copilot-instructions.md` points here.
8+
9+
## What this is
10+
11+
`ldo` (distribution `libre-devops-helpers`, import `libre_devops_helpers`): a fast, read-only
12+
CLI and library for Microsoft (Entra ID, Defender XDR, Intune, Azure, Graph, PIM, Logic Apps,
13+
Automation) and ServiceNow. Its users are people signing in as themselves, usually through
14+
the Azure CLI; automation is second. Everything reads, apart from `ldo az use`, which switches
15+
the Azure CLI's account. Never add a command that changes a tenant or an instance without
16+
being asked.
17+
18+
## Working here
19+
20+
- Run everything through `just` (or `uv run`): `just check` before calling anything done,
21+
`just ci` for all of CI's checks, `just fmt` to format, `just run <args>` to try the CLI.
22+
- Python 3.11 or later: nothing newer than 3.11 in the code (`just test-311` proves it).
23+
- ruff, line length 100. Match the surrounding code's naming, comment density and idiom.
24+
- Runtime dependencies are `requests` and `typer` only. Everything else is the standard
25+
library. Do not add a runtime dependency without asking; test-only ones go in the `dev`
26+
group.
27+
28+
## Layout and layering
29+
30+
```text
31+
src/libre_devops_helpers/
32+
core/ vendor-neutral: errors, config, brand, HTTP client, token store,
33+
polling, inputs (CSV and Excel), logging, YAML writer
34+
microsoft/ the shared Microsoft layer: clouds, credentials, tokens, profiles
35+
microsoft/<feature>/ one package per API: entra, xdr, graph, azure, pim, automation ...
36+
microsoft/devices/ the one composite, using entra, xdr and intune
37+
servicenow/ the ServiceNow vendor, same shape
38+
cli/ the ldo command: parses, calls a client, renders. No logic here.
39+
```
40+
41+
`tests/project/test_package_surface.py` enforces the layering: `core` imports nothing of
42+
ours, a feature imports only `core` and its vendor layer, `cli` sits on top. Cross-cutting
43+
code (auth, HTTP, input parsing) goes in `core` or the vendor layer, never in a feature. A new
44+
feature package needs its entry in that test, a `REQUIREMENTS` tuple, and a test folder.
45+
46+
## Tests
47+
48+
- Tests mirror `src` (`tests/core`, `tests/microsoft/<feature>`, `tests/cli/commands`), one
49+
test module per module, and `tests/project/test_layout.py` checks it.
50+
- Nothing touches the network, a real `az` or a real clock. HTTP goes through
51+
`tests/fakes/http.py` (`routes`, `fake_session`), `az` through `fakes/azcli.py`, time
52+
through `fakes/clock.py`. Shared fakes live in `tests/fakes`, one module per concern.
53+
- Coverage is gated at 93% (`fail_under` in `pyproject.toml`). Keep new code covered.
54+
- CI runs with `GITHUB_ACTIONS` set, which makes Typer colour and wrap errors: compare error
55+
text with `usage_error(result)`, never a raw substring of `result.output`.
56+
- Windows caps an environment variable at 32,767 characters and pytest puts each test id in
57+
one: give big parametrised values short `ids`.
58+
- `tests/project/test_rebrand.py` renames a copy of the repository and runs its whole suite.
59+
60+
## Names and branding
61+
62+
The project can be renamed (`just rebrand`). Never hard-code `ldo`, `LDO_` or the package
63+
name in code: use `core/brand.py` (`brand.COMMAND`, `brand.env_var("X")`,
64+
`brand.command("config init")`, `brand.docs("page")` for a link to a docs page).
65+
66+
## Writing code
67+
68+
- Library code raises `LdoError` subclasses (`InputError`, `NotFoundError`,
69+
`AmbiguousError`, `ApiError`, ...) with a `hint` saying what to do, and never exits. Only
70+
the CLI turns errors into messages and exit codes (0 fine, 1 error, 2 usage, 3 needs
71+
attention, 130 interrupted).
72+
- Data goes to stdout; notes, warnings and progress to stderr, through `cli/render.py`
73+
(`note`, `warn`, `error`, `emit`). In the `json` and `otlp` log formats those become log
74+
records, so never write to stderr directly.
75+
- Every data command takes `-o table|json|csv` and `-p PROFILE`; lists of names take
76+
arguments, `-` for stdin and `-f FILE` with `--column` and `--sheet`.
77+
- All HTTP goes through `core.http.ApiClient`, which applies `core/network.py` (the proxy
78+
and `no_proxy`) and `core/trust.py` (the public roots, the OS store and `ca_bundle`), and
79+
`az` runs with the same through `network.subprocess_env()`. Never call `requests` directly.
80+
- Anything a person types that goes into a URL path or a query is validated first (names,
81+
ids, KQL values). A token or secret is never printed (only with an explicit `--raw`), never
82+
logged, and never accepted on the command line.
83+
- Help text is rendered as markdown: no `<`, no `*`, and paragraphs reflow. A test checks.
84+
- Do not hide findings from a scanner or linter by renaming things; fix the code, or say it
85+
is a false positive and why.
86+
87+
## Docs
88+
89+
- `README.md` is a short front page: the command table, install, quickstart and links.
90+
Detail goes in `docs/`, one page per area. `CHANGELOG.md` gets an `Unreleased` entry for
91+
every change a user would notice.
92+
- `tests/project/test_docs.py` runs every `ldo` example in the README, `docs/` and this file
93+
with `--help`, checks every `just` example is a recipe, and follows every link. Keep
94+
examples real.
95+
- UK English (colour, organisation, licence as a noun). Never use em or en dashes, in any
96+
file: use commas, colons, brackets or a plain hyphen.
97+
98+
## Commits and releases
99+
100+
- Do not commit, push, tag or release until the person asks. Use their own git identity.
101+
- No AI attribution in commits or pull requests: no `Co-Authored-By` or "Generated with"
102+
lines.
103+
- A release is `just release` after the version is set in `pyproject.toml` and
104+
`src/libre_devops_helpers/__init__.py` and `CHANGELOG.md` has its section; see
105+
`docs/development.md`. Never move or reuse a released tag, and remember a PyPI version can
106+
never be replaced.
107+
108+
## Pitfalls already met
109+
110+
- `.gitignore` rules match anywhere unless anchored with `/`; ruff and hatchling honour
111+
`.gitignore`, so an ignored folder is silently left out of linting and the wheel.
112+
- The arm64 image is built under emulation: keep heavy work (such as compiling bytecode) in
113+
a `FROM --platform=$BUILDPLATFORM` stage.
114+
- The Azure CLI pins some dependencies exactly. A fix it holds back is forced in
115+
`container/azure-cli/pyproject.toml` (see the comment there), then signed in with.
116+
- Microsoft APIs return stale duplicates (devices share names), paginate with `nextLink`
117+
or `@odata.nextLink`, and answer 403 for a missing scope and a suspended tenant alike: say
118+
which in the hint.

‎AI.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Instructions for AI coding assistants
2+
3+
This file is for Claude Code, GitHub Copilot, OpenAI Codex and any other assistant working
4+
in this repository. It is the one to edit: `AGENTS.md` is generated from it (`just ai`),
5+
`CLAUDE.md` imports it, and `.github/copilot-instructions.md` points here.
6+
7+
## What this is
8+
9+
`ldo` (distribution `libre-devops-helpers`, import `libre_devops_helpers`): a fast, read-only
10+
CLI and library for Microsoft (Entra ID, Defender XDR, Intune, Azure, Graph, PIM, Logic Apps,
11+
Automation) and ServiceNow. Its users are people signing in as themselves, usually through
12+
the Azure CLI; automation is second. Everything reads, apart from `ldo az use`, which switches
13+
the Azure CLI's account. Never add a command that changes a tenant or an instance without
14+
being asked.
15+
16+
## Working here
17+
18+
- Run everything through `just` (or `uv run`): `just check` before calling anything done,
19+
`just ci` for all of CI's checks, `just fmt` to format, `just run <args>` to try the CLI.
20+
- Python 3.11 or later: nothing newer than 3.11 in the code (`just test-311` proves it).
21+
- ruff, line length 100. Match the surrounding code's naming, comment density and idiom.
22+
- Runtime dependencies are `requests` and `typer` only. Everything else is the standard
23+
library. Do not add a runtime dependency without asking; test-only ones go in the `dev`
24+
group.
25+
26+
## Layout and layering
27+
28+
```text
29+
src/libre_devops_helpers/
30+
core/ vendor-neutral: errors, config, brand, HTTP client, token store,
31+
polling, inputs (CSV and Excel), logging, YAML writer
32+
microsoft/ the shared Microsoft layer: clouds, credentials, tokens, profiles
33+
microsoft/<feature>/ one package per API: entra, xdr, graph, azure, pim, automation ...
34+
microsoft/devices/ the one composite, using entra, xdr and intune
35+
servicenow/ the ServiceNow vendor, same shape
36+
cli/ the ldo command: parses, calls a client, renders. No logic here.
37+
```
38+
39+
`tests/project/test_package_surface.py` enforces the layering: `core` imports nothing of
40+
ours, a feature imports only `core` and its vendor layer, `cli` sits on top. Cross-cutting
41+
code (auth, HTTP, input parsing) goes in `core` or the vendor layer, never in a feature. A new
42+
feature package needs its entry in that test, a `REQUIREMENTS` tuple, and a test folder.
43+
44+
## Tests
45+
46+
- Tests mirror `src` (`tests/core`, `tests/microsoft/<feature>`, `tests/cli/commands`), one
47+
test module per module, and `tests/project/test_layout.py` checks it.
48+
- Nothing touches the network, a real `az` or a real clock. HTTP goes through
49+
`tests/fakes/http.py` (`routes`, `fake_session`), `az` through `fakes/azcli.py`, time
50+
through `fakes/clock.py`. Shared fakes live in `tests/fakes`, one module per concern.
51+
- Coverage is gated at 93% (`fail_under` in `pyproject.toml`). Keep new code covered.
52+
- CI runs with `GITHUB_ACTIONS` set, which makes Typer colour and wrap errors: compare error
53+
text with `usage_error(result)`, never a raw substring of `result.output`.
54+
- Windows caps an environment variable at 32,767 characters and pytest puts each test id in
55+
one: give big parametrised values short `ids`.
56+
- `tests/project/test_rebrand.py` renames a copy of the repository and runs its whole suite.
57+
58+
## Names and branding
59+
60+
The project can be renamed (`just rebrand`). Never hard-code `ldo`, `LDO_` or the package
61+
name in code: use `core/brand.py` (`brand.COMMAND`, `brand.env_var("X")`,
62+
`brand.command("config init")`, `brand.docs("page")` for a link to a docs page).
63+
64+
## Writing code
65+
66+
- Library code raises `LdoError` subclasses (`InputError`, `NotFoundError`,
67+
`AmbiguousError`, `ApiError`, ...) with a `hint` saying what to do, and never exits. Only
68+
the CLI turns errors into messages and exit codes (0 fine, 1 error, 2 usage, 3 needs
69+
attention, 130 interrupted).
70+
- Data goes to stdout; notes, warnings and progress to stderr, through `cli/render.py`
71+
(`note`, `warn`, `error`, `emit`). In the `json` and `otlp` log formats those become log
72+
records, so never write to stderr directly.
73+
- Every data command takes `-o table|json|csv` and `-p PROFILE`; lists of names take
74+
arguments, `-` for stdin and `-f FILE` with `--column` and `--sheet`.
75+
- All HTTP goes through `core.http.ApiClient`, which applies `core/network.py` (the proxy
76+
and `no_proxy`) and `core/trust.py` (the public roots, the OS store and `ca_bundle`), and
77+
`az` runs with the same through `network.subprocess_env()`. Never call `requests` directly.
78+
- Anything a person types that goes into a URL path or a query is validated first (names,
79+
ids, KQL values). A token or secret is never printed (only with an explicit `--raw`), never
80+
logged, and never accepted on the command line.
81+
- Help text is rendered as markdown: no `<`, no `*`, and paragraphs reflow. A test checks.
82+
- Do not hide findings from a scanner or linter by renaming things; fix the code, or say it
83+
is a false positive and why.
84+
85+
## Docs
86+
87+
- `README.md` is a short front page: the command table, install, quickstart and links.
88+
Detail goes in `docs/`, one page per area. `CHANGELOG.md` gets an `Unreleased` entry for
89+
every change a user would notice.
90+
- `tests/project/test_docs.py` runs every `ldo` example in the README, `docs/` and this file
91+
with `--help`, checks every `just` example is a recipe, and follows every link. Keep
92+
examples real.
93+
- UK English (colour, organisation, licence as a noun). Never use em or en dashes, in any
94+
file: use commas, colons, brackets or a plain hyphen.
95+
96+
## Commits and releases
97+
98+
- Do not commit, push, tag or release until the person asks. Use their own git identity.
99+
- No AI attribution in commits or pull requests: no `Co-Authored-By` or "Generated with"
100+
lines.
101+
- A release is `just release` after the version is set in `pyproject.toml` and
102+
`src/libre_devops_helpers/__init__.py` and `CHANGELOG.md` has its section; see
103+
`docs/development.md`. Never move or reuse a released tag, and remember a PyPI version can
104+
never be replaced.
105+
106+
## Pitfalls already met
107+
108+
- `.gitignore` rules match anywhere unless anchored with `/`; ruff and hatchling honour
109+
`.gitignore`, so an ignored folder is silently left out of linting and the wheel.
110+
- The arm64 image is built under emulation: keep heavy work (such as compiling bytecode) in
111+
a `FROM --platform=$BUILDPLATFORM` stage.
112+
- The Azure CLI pins some dependencies exactly. A fix it holds back is forced in
113+
`container/azure-cli/pyproject.toml` (see the comment there), then signed in with.
114+
- Microsoft APIs return stale duplicates (devices share names), paginate with `nextLink`
115+
or `@odata.nextLink`, and answer 403 for a missing scope and a suspended tenant alike: say
116+
which in the hint.

‎CHANGELOG.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,40 @@ All notable changes to libre-devops-helpers are recorded here. The project follo
55

66
## Unreleased
77

8+
### Added
9+
10+
- Corporate proxy support. Every HTTPS call, and every `az` `ldo` runs, follow one set of
11+
rules: loopback and the metadata endpoint always direct, then `no_proxy` and `NO_PROXY`,
12+
then `LDO_PROXY_ADDRESS`, the config file's `proxy`, `HTTPS_PROXY` / `HTTP_PROXY` /
13+
`ALL_PROXY`, and the operating system's setting. An address without a scheme means
14+
`http://`, as for cntlm and Px, which sign in to an NTLM or Kerberos proxy for you.
15+
- The operating system's certificate store is trusted by default, with the public roots
16+
and the config file's `ca_bundle`, in one bundle the Azure CLI is handed as
17+
`REQUESTS_CA_BUNDLE` too, so both work behind a TLS-inspecting proxy whose root IT has
18+
installed. `LDO_CA_BUNDLE`, `REQUESTS_CA_BUNDLE` or `CURL_CA_BUNDLE` names a bundle to use
19+
exactly instead. No new dependency.
20+
- `ldo network test`: asks Entra ID, Graph, Azure Resource Manager, Defender and each
21+
ServiceNow instance (and any `--url`) for an unsigned answer, and says which proxy each
22+
call used, which certificates are trusted, and what to try when one fails: a proxy's
23+
NTLM sign-in (cntlm or Px), a proxy that is not running, a TLS-inspecting proxy's
24+
certificate (with its issuer named), or a local proxy listening on 3128 or 3129.
25+
- `AI.md`: instructions for AI coding assistants, read by Claude Code (through `CLAUDE.md`),
26+
GitHub Copilot (`.github/copilot-instructions.md`, and `AGENTS.md` for its coding agent)
27+
and Codex (`AGENTS.md`). `AGENTS.md` is written from it by `just ai`, and a test fails
28+
when it falls behind.
29+
830
### Changed
931

1032
- The README installs from PyPI (`uv tool install`, `pipx`, `uv pip` or `pip`), with a
1133
PyPI badge; installing a tag from GitHub is still shown.
1234

35+
### Fixed
36+
37+
- `ca_bundle` replaced the public roots rather than adding to them, so a host a proxy does
38+
not inspect (sign-in often is not) failed to verify, and it never reached the Azure CLI.
39+
It now adds to the public roots and the OS store, for both.
40+
41+
1342
## 0.4.1
1443

1544
### Added

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
@AI.md

‎CONTRIBUTING.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@ command or improving the documentation.
55

66
## Workflow
77

8+
Working with an AI coding assistant? Its instructions are in [AI.md](AI.md), which Claude
9+
Code, GitHub Copilot and Codex all read (through `CLAUDE.md`, `.github/copilot-instructions.md`
10+
and `AGENTS.md`). Edit `AI.md` and run `just ai`; a test fails when `AGENTS.md` lags behind.
11+
812
1. Fork the repository and branch from `main`.
913
2. Run `just sync` (or `uv sync`) to create the environment.
1014
3. Make your change, with tests.

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ PowerShell module, and importable as a library too.
5252
| `ldo logicapp` | offline checks, export and validation for Consumption Logic Apps and Sentinel playbooks | [logic apps](docs/logic-apps.md) |
5353
| `ldo snow` | ServiceNow: sign in, whoami, the instance, applications, a token | [servicenow](docs/servicenow.md) |
5454
| `ldo az` | switch the Azure CLI between profiles | [signing in](docs/authentication.md) |
55+
| `ldo network test` | test the way out through a corporate proxy: the proxy, the certificates, each service | [network](docs/network.md) |
5556
| `ldo json` | pretty-print any JSON (`az rest ... \| ldo json`) in colour, or as YAML | [configuration](docs/configuration.md#json-yaml-and-logs) |
5657
| `ldo profiles`, `ldo config` | your profiles, and the config file | [configuration](docs/configuration.md) |
5758

@@ -114,10 +115,12 @@ for those. [Permissions](docs/permissions.md) lists what each command needs.
114115
## Documentation
115116

116117
- [Configuration](docs/configuration.md): profiles, common options, environment variables, exit codes
118+
- [Proxies and certificates](docs/network.md): corporate proxies, cntlm, TLS inspection
117119
- [Signing in](docs/authentication.md) and [Permissions](docs/permissions.md)
118120
- [Container images](docs/containers.md)
119121
- [Using it as a library](docs/library.md) and [Rebranding](docs/rebranding.md) for your organisation
120122
- [Development](docs/development.md): `just` recipes, tests, CI and releasing
123+
- [AI.md](AI.md): the instructions for AI coding assistants (Claude Code, Copilot, Codex)
121124

122125
Contributions are welcome: see [CONTRIBUTING.md](CONTRIBUTING.md), and
123126
[SECURITY.md](SECURITY.md) to report a vulnerability. Licensed under [MIT](LICENSE).

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
| [Configuration](configuration.md) | the config file, profiles, options every command takes, environment variables, exit codes |
1010
| [Signing in](authentication.md) | the ways to sign in, automation, when a sign-in lapses, keeping a sign-in, your own app registration |
1111
| [Permissions](permissions.md) | what each command needs, and what the Azure CLI's token covers |
12+
| [Proxies and certificates](network.md) | corporate proxies, cntlm and Px, TLS inspection, `network test` |
1213
| [Container images](containers.md) | the images, their tags, running and building them |
1314

1415
## Commands

0 commit comments

Comments
 (0)