Release notes: Sept 25, 2025 (v9.22.3)

Inference Red-Team

October attack pack: Playing the Word game

At CalypsoAI, we continue to expand the frontier of AI security. Our October Signature attack pack once again pushes adversarial coverage further, built and deployed end-to-end by AI agents to deliver stronger and broader results.

This month’s release introduces Word Game, a new adversarial vector that conceals malicious requests inside playful word puzzles. By scrambling words, letters, or entire phrases into riddles and then asking the model to solve the puzzle before acting on the hidden instruction, attackers can bypass traditional safety filters. Once the puzzle is “solved,” the harmful request is revealed and executed.

Alongside this new vector, the pack includes 1,482 unique plaintext prompts spanning 58 distinct intents and seven converters. Word Game joins a lineup of proven attack families including math prompt, DAN, payload splitting, refusal suppression, flip, and more — collectively delivering 11,500+ adversarial prompts across the spectrum.

Attack pack date alignment

With this release, attack pack names have been updated to align with the CASI leaderboard timeline.

• All attack pack dates are now incremented by one month to match leaderboard reporting.

• This change is applied retroactively across all previous packs. For example:

  • “July Attack Pack” → “August Attack Pack”

  • “August Attack Pack” → “September Attack Pack”

  • And so on.

This adjustment ensures consistency between attack pack labels and CASI leaderboard periods for clearer tracking and reporting.

Report page update

We’ve made major usability improvements to Reports, focusing on a cleaner design and more powerful filtering and sorting.

Redesign

• Columns are realigned for better readability, with Status and Report Name placed up front.

• Statuses are more prominent, making it easy to see which reports are complete, scheduled, or in progress at a glance.

Filter and sort

We’ve added a new filter and sort UI to help you quickly find and organize reports:

• Sort by: Report name, run date, CASI score

• Filter by: Connection, run date, CASI score rating (critical, warning, good, etc.), and created by

This gives you more control over how you view and work with your reports, whether you’re searching for recent runs, high-severity results, or reports from a specific connection.

Campaigns UI enhancements

We’ve improved Campaigns to provide more visibility and control over your attack campaigns.

New metadata in Campaigns table

Users can now see additional details for each campaign:

• Created by

• Created date

• Last run date

• Total number of runs (how frequently each campaign is executed)

Filters and sort

• Sort by: Name, created date, last run date, number of runs.

• Filter by: Campaign type, created date, last run date

UI improvements

• The All attacks campaign name is changed to All standard signature attacks, which is a more accurate description of what it covers.

• Since this campaign is always pinned to the top, it now has a distinct row styling for easier recognition.

Inference Defend

Bulk-add keywords to custom scanners

We've made it easier and faster to create custom keyword scanners. You can now add large lists of keywords by uploading a CSV file or pasting them directly into the scanner configuration. This helps you quickly build scanners from existing keyword lists without having to enter each term manually.

For example, if your security team maintains a list of sensitive project codenames in a spreadsheet, you can now save it as a CSV file and upload it directly to create a scanner that detects those terms.

To add keywords to a scanner, go to the scanner page and click Build a custom scanner. From the keyword scanner form, you can:

• Paste a list: Copy a list of keywords from an application like Excel and paste them into the form.

• Upload a file: Drag and drop a CSV file into the UI. You can then select which column from your file contains the keywords.

• Manage keywords: A new Clear button has been added to remove all keywords at once. You can also remove individual keywords by clicking their X icon.

The keyword scanner accepts up to 1,000 keywords. When bulk-adding keywords, the system will import the first 1,000 terms and ignore anything past that.

Navigate project details faster with tabs

We've updated the layout of the project detail page to make finding information easier and faster. Previously, this page was a single view that required you to scroll to find different sections. Now, the content is organized into a tabbed format.

Simply click the tabs at the top of the page to jump directly to the section you need, without scrolling.

Platform

Red-Team entitlements

Running Red-Team reports should be predictable and transparent. With this release, Red-Team now includes report entitlements that allow organizations to track, manage, and stay within their allocated number of reports based on their subscription.

What this means

• Each organization has a fixed number of reports per year.

• Entitlements are tied to your subscription tier (contact sales team if you need more reports or an upgrade).

• Your remaining quota is always visible in the product.

How reports are counted?

When a report starts running, it decrements from your balance.

• If the report finishes successfully (Completed), the decrement stays.

• If the report errors out or is canceled before it finishes, the report is credited back.

This ensures your quota is only used for reports that complete successfully.

Where to see your balance?

You can view your usage and remaining reports in three places:

1. Settings → Usage → Track Account Usage: Shows your plan, the total reports available, the number used, and what’s included in your tier.

2. Side navigation: At the bottom of the navigation bar you’ll see your usage summary, including reports remaining.

3. Reports → Run attack: In the bottom-right corner of the Run attack panel, usage and available reports are displayed while creating a report.

Scheduling reports

If you schedule more reports than your remaining balance allows, you’ll see a message: “This schedule will end when the usage limit is reached.”

This ensures recurring or bulk schedules stop automatically when your entitlement runs out.

Bug fixes

• Changing the tab during role creation deleted the role without discarding changes. Resolution: Fixed.

• The "Application not found" image had a different background colour. Resolution: Fixed.

• On-Prem: The Help Center link led to a "Page not found" error. Resolution: Fixed.

• The Send tooltip in the chat box on the chat page had an incorrect style. Resolution: Fixed.

• Operational attacks were missing from the report top-level summary when a campaign only contained operational attacks. Resolution: Fixed.