feat: add agentcore-observability-setup skill

[vipulg1993]

Summary

Adds the agentcore-observability-setup skill: validates and bootstraps Amazon Bedrock AgentCore observability so customers can trace agent reasoning, detect silent failures, and measure performance before an outage. It operates on a single principle — verify where reachable, prescribe everywhere else:

• VERIFY (read-only APIs): CloudWatch Transaction Search state, runtime/agent log groups and data arrival, spans flowing, session metrics, Memory/Gateway log & trace deliveries, X-Ray log-group resource policy, and Lambda/ECS host config.
• PRESCRIBE (cannot be read directly): code-level OTEL instrumentation (ADOT distro, opentelemetry-instrument, framework tracing) — validated by effect (are spans arriving?) and emitted as exact steps.

Covers AgentCore Runtime agents (primary), Memory and Gateway resources, built-in tools, and agents hosted outside the runtime (Lambda, ECS, EKS, on-prem, multi-cloud). Three read-only IAM tiers with graceful degradation; the skill reports the exact scoped policy needed to unlock further checks and never mutates anything.

Testing

• Manual testing in DevOps Agent (Isengard): skill activated on natural-language prompts (not named), reached Tier 2, and produced correct findings on both a healthy account and an induced-fault scenario (Transaction Search disabled), where it identified the disabled Transaction Search as the CRITICAL root cause and prescribed remediation.
• Agent Skill Eval (report): overall Grade A, "passed": true. Audit 100/A (0 critical/warning), Trigger 1.00 (6/6, 100% precision both ways), Functional passed (quality gain with lower token cost).

IAM / CloudFormation

Tier 1 (CloudWatch Logs/Metrics telemetry-arrival reads) is covered by AIDevOpsAgentAccessPolicy. Tier 2/3 add read-only bedrock-agentcore (Get/List runtime), xray:GetTraceSegmentDestination, logs:DescribeDeliveries/DeliverySources/DeliveryDestinations/ResourcePolicies, and lambda:GetFunctionConfiguration / ecs:DescribeTaskDefinition,DescribeServices,ListTasks / eks:DescribeCluster. Added as a gated inline policy (EnableAgentCoreObservabilitySetup, default true) to cloudformation/devops-agent-skill-policies.yaml, with the SkillPolicySummary output updated. All permissions are read-only.

Taskei

V2323594559 — accepted by the Project Nebula team (reviewer: pbraz).

Checklist

• SKILL.md has version and author in metadata
• name in frontmatter matches directory name
• description clearly states what/when to activate
• README.md includes non-production disclaimer
• CHANGELOG.md present
• Tested in DevOps Agent (skill activates consistently)
• Agent Skill Eval "passed": true (Grade A)
• Registered in llms.txt
• .skilleval.yaml present (canonical STR-016 ignore)
• No internal Amazon tool names in submitted content
• No customer data, account IDs, or credentials
• Working against latest main

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache License 2.0.

[pbraz-aws]

Two items to address:

1. The Tier 2 table in SKILL.md lists bedrock-agentcore-control:List*.
   That prefix doesn't exist in IAM — the Service Authorization Reference
   shows the service prefix is bedrock-agentcore (the -control suffix is only the SDK client name for
   the control plane). Your CloudFormation policy and references/iam-tiers.md already get this right
   with bedrock-agentcore:ListAgentRuntimes, so it's just the SKILL.md line that's out of sync and
   would mislead anyone hand-writing a policy. Can you change that row to
   bedrock-agentcore:ListAgentRuntimes?

2. Unified span destination is now the default. Per the release notes, agents created
   after July 20, 2026 in supported Regions default to their own log group for spans. The skill still
   treats the X-Ray log-group resource policy (R7) as the opt-in exception and the span-destination
   mode (R6) as INFO/LOW — which is backwards for any new agent, where R7 now applies unless the
   customer opts out. Can you note the default-on date and flip R7 to "applies by default for
   agents created after 2026-07-20"?

[vipulg1993]

1. SKILL.md Tier 2 table: bedrock-agentcore-control:List* is not a real IAM prefix (that's only the SDK client name). Change to bedrock-agentcore:ListAgentRuntimes — CFN and iam-tiers.md already have it right.

2. R6/R7 framing is backwards. Per the AgentCore release notes, agents created after 2026-07-20 default to the unified span destination. Flip R7 from "opt-in exception" to "applies by default for agents created after 2026-07-20" and note the date.

Both fixed in d548a10

[paulbrazell]

Looks good. Approved

[ams-thakkar]

Scanned for repo-convention compliance and IAM safety (leaving AgentCore/Bedrock domain correctness to the SME). One should-fix before merge, otherwise this is in great shape.

1. (should fix — breaks the docs build) Relative .md link in README. README.md links to ](references/iam-tiers.md). The docs pipeline copies each skill README into docs/skills/<id>.md, and from there that relative path doesn't resolve — I ran mkdocs build --strict with this skill staged and it aborts:

WARNING - Doc file 'skills/agentcore-observability-setup.md' contains a link 'references/iam-tiers.md',
          but the target 'skills/references/iam-tiers.md' is not found
Aborted with 1 warnings in strict mode!

Since the docs-deploy workflow runs --strict, this would fail on merge. Fix: make it an absolute GitHub URL (https://github.com/aws/tools-for-devops-agent/blob/main/skills/agentcore-observability-setup/references/iam-tiers.md), the way other skills' READMEs link their own reference files. (SKILL.md and references/ themselves don't enter the docs build, so backtick mentions / relative links are only a problem in README.md.)

What's verified good:

• ✅ llms.txt entry present, well-formed, accurate.
• ✅ CHANGELOG ## 1.0.0, version-synced with SKILL frontmatter.
• ✅ README has the non-production disclaimer; no stale aws-samples org references.
• ✅ Structure: required files present, eval JSON valid (incl. evals/files/agent-context.json), .skilleval.yaml STR-016 ignore, frontmatter complete, description 913/1024.
• ✅ CFN IAM grant is read-only. The added EnableAgentCoreObservabilitySetup inline policy is all Get/List/Describe (bedrock-agentcore:Get*/ListAgentRuntimes, ecs:Describe*/ListTasks, eks:DescribeCluster, lambda:GetFunctionConfiguration, logs:Describe*, xray:GetTraceSegmentDestination) — no mutations, correctly gated (parameter + Condition, default true), SkillPolicySummary updated.

Holding approval only on the README link (#1) since it breaks the strict docs build. Fix that and I'll approve — the verify-where-reachable / prescribe-everywhere-else design is clean.

[vipulg1993]

Added missing relative link in read me with full absolute Github link for md file here

[ams-thakkar]

Thanks for the fix — re-verified and approving.

• ✅ README relative-link fixed: references/iam-tiers.md is now an absolute GitHub URL, and no relative .md links remain. Re-ran mkdocs build --strict with the skill staged against the current tip (incl. your upstream-main merge) — passes, 0 warnings, so the docs deploy won't break.
• ✅ Still-good from the first pass: llms.txt entry, CHANGELOG ## 1.0.0 (version-synced), non-production disclaimer, eval JSON valid, and the CFN grant is all read-only Get/List/Describe, correctly gated.

Scope note: reviewed convention + IAM safety; leaning on the Bedrock SME for AgentCore domain correctness. The verify-where-reachable / prescribe-everywhere-else design is clean. Nice work.