Description of Problem
As part of looking at an Eval framework for agentic processes there is a requirement to be able to gather the data from an agentic process. This data can then be used to evaluate the performance and accuracy of the agentic process facilitating EDD.
Potential Solutions
Proposal to add a dedicated history layer to the agentic subprocess plugin, enabling users to retrieve a full
history of every agent subprocess execution, including LLM calls, tool invocations, iteration counts, token usage, and final output. This can then be collated as a single json response through a dedicated REST endpoint. The implementation is entirely self-contained within the plugin; no changes to the main fluxnova-bpm-platform repository are required.
Suggested new Events:
agent-subprocess entity
| Constant |
Entity Type |
Event Name |
Fired When |
AGENT_SUBPROCESS_START |
agent-subprocess |
start |
The agent subprocess execution is entered and the first orchestration job is scheduled |
AGENT_SUBPROCESS_END |
agent-subprocess |
end |
The subprocess terminates (LLM returned a text-only response or the tool catalogue was empty) |
agent-loop entity
| Constant |
Entity Type |
Event Name |
Fired When |
AGENT_LOOP_START |
agent-loop |
start |
Each orchestration job begins executing (i.e. each LLM round-trip cycle starts) |
AGENT_LOOP_END |
agent-loop |
end |
The LLM response for that cycle has been processed and the next action determined |
agent-llm entity
| Constant |
Entity Type |
Event Name |
Fired When |
AGENT_LLM_REQUEST |
agent-llm |
request |
Immediately before calling LlmService.call(...) — captures the outbound prompt token count and message count |
AGENT_LLM_RESPONSE |
agent-llm |
response |
Immediately after the LlmResponse is received — captures token usage and whether the response was TEXT or TOOL_CALLS |
agent-tool-call entity
| Constant |
Entity Type |
Event Name |
Fired When |
AGENT_TOOL_CALL_REQUESTED |
agent-tool-call |
requested |
The LLM has requested a tool call and it has been validated against the catalogue, before triggerAdHocActivities fires |
AGENT_TOOL_CALL_COMPLETED |
agent-tool-call |
completed |
SubprocessToolCompletionListener confirms the tool activity ended successfully |
AGENT_TOOL_CALL_FAILED |
agent-tool-call |
failed |
The tool activity ended in error (the result is still fed back to the LLM as an error message) |
Suggested endpoint response would look like:
{
"subprocessExecutionId": "8f3a1c92-...",
"processInstanceId": "d4b72e11-...",
"processDefinitionKey": "riskReportProcess",
"elementId": "agentScope",
"provider": "ollama",
"model": "llama3",
"goal": "Generate risk report",
"finalOutput": "The risk report for Q3 has been generated and stored.",
"iterations": 4,
"totalPromptTokens": 3820,
"totalCompletionTokens": 614,
"executionTime": 4200,
"startTime": "2026-08-05T13:00:00.000Z",
"endTime": "2026-08-05T13:00:04.200Z",
"toolCalls": [
{
"toolCallId": "call_abc123",
"toolName": "Fetch market data",
"toolElementId": "fetchMarketData",
"loopIndex": 1,
"requestedAt": "2026-08-05T13:00:00.950Z",
"completedAt": "2026-08-05T13:00:01.184Z",
"durationMs": 234,
"status": "COMPLETED",
"errorMessage": null
}
"..."
],
"step-history": [
{
"type": "agent-subprocess:start",
"timestamp": "2026-08-05T13:00:00.000Z",
"elementId": "agentScope",
"provider": "ollama",
"model": "llama3"
},
{
"type": "agent-loop:start",
"timestamp": "2026-08-05T13:00:00.050Z",
"loopIndex": 1
},
"..."
]
}
Description of Problem
As part of looking at an Eval framework for agentic processes there is a requirement to be able to gather the data from an agentic process. This data can then be used to evaluate the performance and accuracy of the agentic process facilitating EDD.
Potential Solutions
Proposal to add a dedicated history layer to the agentic subprocess plugin, enabling users to retrieve a full
history of every agent subprocess execution, including LLM calls, tool invocations, iteration counts, token usage, and final output. This can then be collated as a single json response through a dedicated REST endpoint. The implementation is entirely self-contained within the plugin; no changes to the main
fluxnova-bpm-platformrepository are required.Suggested new Events:
agent-subprocessentityAGENT_SUBPROCESS_STARTagent-subprocessstartAGENT_SUBPROCESS_ENDagent-subprocessendagent-loopentityAGENT_LOOP_STARTagent-loopstartAGENT_LOOP_ENDagent-loopendagent-llmentityAGENT_LLM_REQUESTagent-llmrequestLlmService.call(...)— captures the outbound prompt token count and message countAGENT_LLM_RESPONSEagent-llmresponseLlmResponseis received — captures token usage and whether the response wasTEXTorTOOL_CALLSagent-tool-callentityAGENT_TOOL_CALL_REQUESTEDagent-tool-callrequestedtriggerAdHocActivitiesfiresAGENT_TOOL_CALL_COMPLETEDagent-tool-callcompletedSubprocessToolCompletionListenerconfirms the tool activity ended successfullyAGENT_TOOL_CALL_FAILEDagent-tool-callfailedSuggested endpoint response would look like:
{ "subprocessExecutionId": "8f3a1c92-...", "processInstanceId": "d4b72e11-...", "processDefinitionKey": "riskReportProcess", "elementId": "agentScope", "provider": "ollama", "model": "llama3", "goal": "Generate risk report", "finalOutput": "The risk report for Q3 has been generated and stored.", "iterations": 4, "totalPromptTokens": 3820, "totalCompletionTokens": 614, "executionTime": 4200, "startTime": "2026-08-05T13:00:00.000Z", "endTime": "2026-08-05T13:00:04.200Z", "toolCalls": [ { "toolCallId": "call_abc123", "toolName": "Fetch market data", "toolElementId": "fetchMarketData", "loopIndex": 1, "requestedAt": "2026-08-05T13:00:00.950Z", "completedAt": "2026-08-05T13:00:01.184Z", "durationMs": 234, "status": "COMPLETED", "errorMessage": null } "..." ], "step-history": [ { "type": "agent-subprocess:start", "timestamp": "2026-08-05T13:00:00.000Z", "elementId": "agentScope", "provider": "ollama", "model": "llama3" }, { "type": "agent-loop:start", "timestamp": "2026-08-05T13:00:00.050Z", "loopIndex": 1 }, "..." ] }