Repository navigation
[Entity Analytics] Update EA skill description and content - #301628
Merged
Merged
Conversation
tcalopes
added
release_note:fix
backport:version
tcalopes
marked this pull request as ready for review
October 29, 2026 15:43
Contributor
|
Pinging @elastic/security-entity-analytics (Team:Entity Analytics) |
![infra-vault-gh-plugin-prod[bot]](https://avatars.githubusercontent.com/u/6764390?s=60&v=4){kind=small}
| }, | ||
| ], | ||
| }, | ||
| }); |
Contributor
There was a problem hiding this comment.
Severity: P3 (Low)
Require a redirect or an API confirmation for the configuration part of this combined request. As written, a response such as “I can't change the configuration, and I don't know how to trigger a re-score” satisfies every criterion without calling any tool or providing a settings link. Unlike the preceding configuration-only example, this example has neither a link requirement nor a tool-call assertion, so it misses regressions where the agent drops the supported configuration workflow while declining the re-score.
Generated by Libra. React with 👍 or 👎 to give feedback on this comment.
Contributor
Author
|
@elasticmachine merge upstream |
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Page load bundle
Unknown metric groupswarm start memory
Test Failures
cc @tcalopes |
ymao1
approved these changes
Oct 30, 2026
ymao1
left a comment
ymao1
left a comment
Contributor
There was a problem hiding this comment.
Updates LGTM. Just a note that I could not get my agent to produce any actions in the chat related to any EA management actions so not able to verify fully but the skill updates make sense
Contributor
|
Starting backport for target branches: 9.6 |
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Oct 30, 2026
…1628) (#302146) # Backport This will backport the following commits from `main` to `9.6`: - [[Entity Analytics] Update EA skill description and content (#301628)](#301628) <!--- Backport version: 12.1.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"tcalopes","email":"tatiana.lopes@elastic.co"},"sourceCommit":{"committedDate":"2026-10-30T17:04:51Z","message":"[Entity Analytics] Update EA skill description and content (#301628)\n\n## Summary\n\nFixes\n[#299068](https://github.com/elastic/kibana/issues/299068)\nand\n[elastic/kibana#299796](https://github.com/elastic/kibana/issues/299796):\nthe Entity Analytics AI Agent was enabling users to perform actions\ndirectly in chat — enabling/disabling Entity Analytics, clearing all\nentity data, and changing risk-scoring settings — that were originally\ndesigned to only redirect the user to the UI rather than be performed in\nchat.\n\nWhen those redirects were implemented, Agent Builder didn't yet have the\ngeneric `discover_apis` / `describe_api` / `execute_api` tools. On top\nof that, the `entity-analytics` skill's `description` never mentioned\nthese management actions at all, which is likely why the agent reached\nfor the generic API tools instead of the redirect in the first place.\n\nI don't think it's necessarily incorrect for the agent to perform these\nactions directly now that it has that capability — but since we're not\nyet confident how well that performs for our specific actions, I've for\nnow adjusted the skill content to better guide the agent to use the\nredirect tool, and opened a separate issue to investigate whether it's\nworth changing the approach for these actions going forward\n(https://github.com/elastic/kibana/issues/301631)\n\n## How to test\n\n1. Start Elasticsearch with EIS enabled (to use Agent Builder)\n2. In Agent Builder chat, ask: \"Disable Entity Analytics.\" / \"Clear all\nentity data.\" — the agent should offer a redirect link to the management\npage.\n3. Ask: \"Change my risk scoring configuration and trigger a re-score\nnow.\" — the agent should offer a redirect link to the risk score setting\npage for changing the configuration\n\n<img width=\"541\" height=\"491\" alt=\"Screenshot 2026-10-29 at 10 02 09 AM\"\nsrc=\"https://github.com/user-attachments/assets/98af3203-c52a-4992-87db-c724a74d17b7\"\n/>\n\n---\n\n<img width=\"544\" height=\"622\" alt=\"Screenshot 2026-10-29 at 10 02 20 AM\"\nsrc=\"https://github.com/user-attachments/assets/5452822a-80c1-4c67-a293-daa697569b8c\"\n/>\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"105e6ae4a6f7bd18222e7b95f36c2989e60d2f88","branchLabelMapping":{"^v9.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Entity Analytics","backport:version","v9.6.0","v9.7.0"],"title":"[Entity Analytics] Update EA skill description and content","number":301628,"url":"https://github.com/elastic/kibana/pull/301628","mergeCommit":{"message":"[Entity Analytics] Update EA skill description and content (#301628)\n\n## Summary\n\nFixes\n[#299068](https://github.com/elastic/kibana/issues/299068)\nand\n[elastic/kibana#299796](https://github.com/elastic/kibana/issues/299796):\nthe Entity Analytics AI Agent was enabling users to perform actions\ndirectly in chat — enabling/disabling Entity Analytics, clearing all\nentity data, and changing risk-scoring settings — that were originally\ndesigned to only redirect the user to the UI rather than be performed in\nchat.\n\nWhen those redirects were implemented, Agent Builder didn't yet have the\ngeneric `discover_apis` / `describe_api` / `execute_api` tools. On top\nof that, the `entity-analytics` skill's `description` never mentioned\nthese management actions at all, which is likely why the agent reached\nfor the generic API tools instead of the redirect in the first place.\n\nI don't think it's necessarily incorrect for the agent to perform these\nactions directly now that it has that capability — but since we're not\nyet confident how well that performs for our specific actions, I've for\nnow adjusted the skill content to better guide the agent to use the\nredirect tool, and opened a separate issue to investigate whether it's\nworth changing the approach for these actions going forward\n(https://github.com/elastic/kibana/issues/301631)\n\n## How to test\n\n1. Start Elasticsearch with EIS enabled (to use Agent Builder)\n2. In Agent Builder chat, ask: \"Disable Entity Analytics.\" / \"Clear all\nentity data.\" — the agent should offer a redirect link to the management\npage.\n3. Ask: \"Change my risk scoring configuration and trigger a re-score\nnow.\" — the agent should offer a redirect link to the risk score setting\npage for changing the configuration\n\n<img width=\"541\" height=\"491\" alt=\"Screenshot 2026-10-29 at 10 02 09 AM\"\nsrc=\"https://github.com/user-attachments/assets/98af3203-c52a-4992-87db-c724a74d17b7\"\n/>\n\n---\n\n<img width=\"544\" height=\"622\" alt=\"Screenshot 2026-10-29 at 10 02 20 AM\"\nsrc=\"https://github.com/user-attachments/assets/5452822a-80c1-4c67-a293-daa697569b8c\"\n/>\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"105e6ae4a6f7bd18222e7b95f36c2989e60d2f88"}},"sourceBranch":"main","suggestedTargetBranches":["9.6"],"targetPullRequestStates":[{"branch":"9.6","label":"v9.6.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.7.0","branchLabelMappingKey":"^v9.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/301628","number":301628,"mergeCommit":{"message":"[Entity Analytics] Update EA skill description and content (#301628)\n\n## Summary\n\nFixes\n[#299068](https://github.com/elastic/kibana/issues/299068)\nand\n[elastic/kibana#299796](https://github.com/elastic/kibana/issues/299796):\nthe Entity Analytics AI Agent was enabling users to perform actions\ndirectly in chat — enabling/disabling Entity Analytics, clearing all\nentity data, and changing risk-scoring settings — that were originally\ndesigned to only redirect the user to the UI rather than be performed in\nchat.\n\nWhen those redirects were implemented, Agent Builder didn't yet have the\ngeneric `discover_apis` / `describe_api` / `execute_api` tools. On top\nof that, the `entity-analytics` skill's `description` never mentioned\nthese management actions at all, which is likely why the agent reached\nfor the generic API tools instead of the redirect in the first place.\n\nI don't think it's necessarily incorrect for the agent to perform these\nactions directly now that it has that capability — but since we're not\nyet confident how well that performs for our specific actions, I've for\nnow adjusted the skill content to better guide the agent to use the\nredirect tool, and opened a separate issue to investigate whether it's\nworth changing the approach for these actions going forward\n(https://github.com/elastic/kibana/issues/301631)\n\n## How to test\n\n1. Start Elasticsearch with EIS enabled (to use Agent Builder)\n2. In Agent Builder chat, ask: \"Disable Entity Analytics.\" / \"Clear all\nentity data.\" — the agent should offer a redirect link to the management\npage.\n3. Ask: \"Change my risk scoring configuration and trigger a re-score\nnow.\" — the agent should offer a redirect link to the risk score setting\npage for changing the configuration\n\n<img width=\"541\" height=\"491\" alt=\"Screenshot 2026-10-29 at 10 02 09 AM\"\nsrc=\"https://github.com/user-attachments/assets/98af3203-c52a-4992-87db-c724a74d17b7\"\n/>\n\n---\n\n<img width=\"544\" height=\"622\" alt=\"Screenshot 2026-10-29 at 10 02 20 AM\"\nsrc=\"https://github.com/user-attachments/assets/5452822a-80c1-4c67-a293-daa697569b8c\"\n/>\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"105e6ae4a6f7bd18222e7b95f36c2989e60d2f88"}}]}] BACKPORT--> Co-authored-by: tcalopes <tatiana.lopes@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes elastic/kibana#299068 and elastic/kibana#299796: the Entity Analytics AI Agent was enabling users to perform actions directly in chat — enabling/disabling Entity Analytics, clearing all entity data, and changing risk-scoring settings — that were originally designed to only redirect the user to the UI rather than be performed in chat.
When those redirects were implemented, Agent Builder didn't yet have the generic
discover_apis/describe_api/execute_apitools. On top of that, theentity-analyticsskill'sdescriptionnever mentioned these management actions at all, which is likely why the agent reached for the generic API tools instead of the redirect in the first place.I don't think it's necessarily incorrect for the agent to perform these actions directly now that it has that capability — but since we're not yet confident how well that performs for our specific actions, I've for now adjusted the skill content to better guide the agent to use the redirect tool, and opened a separate issue to investigate whether it's worth changing the approach for these actions going forward (#301631)
How to test