This is a SaaS-only release.
Inference Red-Team
Observability for agentic attacks
Agents shouldn’t be a black box—especially agents that have the ability to reason, make decisions, and take actions in your environment.
To address this problem, Inference Red-Team now comes with a powerful and interactive observability tool for agents: Agentic fingerprints. Still in beta, this feature gives deep insight into how CalypsoAI’s Red Agent plans, executes, and evaluates Agentic Warfare attacks. Users can drill into attacks by custom intent, connection target, and attack vector (Crescendo, FRAME, or Trolley) and understand:
-
Whether the attack was successful or not — a green path indicates a successful attack.
-
The agent’s thought process at every step.
-
How the agent evolved the attack based on the response from the target connection.
-
Decision points: where the agent decided to continue the attack, and where it terminated an failed line of attack and backtracked to an earlier node.
Users can interact with the tool by:
-
Selecting any custom intent, target connection, and attack vector.
-
Panning and zooming the graph.
-
Scrolling through the key path to see how the successful attack unfolded.
-
Clicking on any individual nodes to see the agent’s thoughts, actions, and decisions.
▶️ See a demo of Agentic fingerprints in action.
To start using Agentic fingerprints, navigate to Reports and click on any report that includes an Agentic Warfare attack. Click the View agentic fingerprints button in the report, then use the dropdown selectors to find the attack you’re interested in.
Note: Agentic fingerprints is not backward compatible. You may need to run a new Agentic Warfare attack to see it.
Bug fixes
-
In the Playground, after the user saved a new version of a scanner, when they went to test the new version the Discard changes modal appeared even though the changes were saved. Resolution: Fixed
-
In the Logs, pressing next row at the end of the list in the logs requested the next page, but would immediately show the first from the list of the current page until the next page loaded. Resolution: Fixed
-
On the Reports page, the canceling status in the filter did not appear correctly. Resolution: Fixed
-
In the Playground, the Load more function in the custom scanners testing panel didn’t behave as expected. Resolution: Fixed
Known issues
On the Red-Team Reports, an error banner appears on the report even if there is only one error in the whole campaign run, and all other attacks ran successfully.