Release date: Oct 14, 2024
Back end version: 7.77.1
Front end version: 2.89.1
Scanner version: 1.0.9
Keycloak 25.0.2
CalypsoAI SDK 2.7.0 > 2.9.0
Admin Export/Import Version 6
helm app: v1.0.28
helm scanner version: v1.0.2
Features
Redesigned prompt history and audit logs UI
The Prompt History and Audit Logs tables have undergone a major redesign to improve usability and the depth of information available. Users can now drill into individual prompts to see which scanners ran, and the outcome of each individual scanner—blocked, passed, or flagged (when scanners are set to audit). Users can also see the outcomes of prompts sent via the scans API.
Multi-factor authentication for on-prem
Customers with an on-premises deployment can now set up multi-factor authentication for their Calypso organization by setting an environment variable: CAI_MODERATOR_AUTH_REQUIRE_MFA=1. When enabled, users will set up and enter a one-time-password (OTP) via Google Authenticator, Microsoft Authenticator or FreeOTP when logging into the CalypsoAI app. We are working to add this capability to the SaaS product and make it configurable via the product UI.
Model configuration error messaging
If an error occurs when configuring a model, we now show the full error message beneath the configuration form. This provides more context about what went wrong and allows users to keep the error in view until it has been fixed.
“Updating” status while user role is being changed
In earlier versions of the product, there was no feedback when a user role was changed. To address that, we’ve added an Updating badge in the Users table while the update is happening. This badge disappears when the page is refreshed.
Admin export feature
Organizations can now export settings from one system to another, using the API. Below is a script example using the SDK:
from calypsoai import CalypsoAI
cai = CalypsoAI(url="url1", token="token1")
export = cai.admin.exportSettings()
cai.admin.importSettings(export)
Scanner improvements
We split the Blocked Terms scanner into two—Blocked Terms and Audit Terms—so it’s now possible to have a list of terms to block and a list of terms to audit, instead of having to choose one modality.
Other UI updates
We added more detail in the UI to the instructions for integrating CalypsoAI into Slack and Teams, which should make chatbot deployments much smoother.
Some users noticed inconsistent results in the Dashboard, which was due to a timezone bug. We fixed the bug and adjusted how data is grouped for display. If the date range selected is 3 months or less, results are grouped by day. If more than 3 months, data is grouped by week. If more than 6 months, data is grouped by month.
We fixed the behavior of the “Go Back” button on the 404 errors page, and improved the wording on this page.
The date pickers on the Dashboard and API Token screens has been modified to be consistent with the new date picker on the redesigned Logs screen.
When adding a secret to a custom scanner, the secret is now represented with asterisks in the UI to show it exists.
We’ve improved the readability of the audit log summaries with proper grammar and capitalization, instead of JSON syntax.
Other API updates
API users can now request a CSV download of the audit logs and prompt history. This capability has been accessible in the UI for several versions, and is now included in the API as well.
API users can now attach arbitrary JSON to a prompt or scan message, making it easier to track conversations based on timestamps or categories, for example. The JSON field is limited to 10 items and a max depth of 1. Keys can be a maximum of 50 characters long and values can be 255 characters.
In earlier releases, when a user posted a prompt or scan request, they would get back the results of scans but would have to do a separate API call to get the details of the scanners that run those scans. For example, the user would have the IDs of the scans that failed, but would want the names. Now if a user adds
verbose=Truefor these calls, they get back the details on the scanners along with the results.
Sunset feature: Validate & rate responses
We’ve removed the Validate and Rate Responses feature from the Chat UI due to low user adoption.
Bugs
We fixed over 50 bugs since the last versioned release, including:
Chat users encountered an issue where changes to scanner settings (e.g., turning scanners on/off or changing from blocked to audit) were inconsistently applied to different groups. This has been fixed.
Two scanners were missing descriptions in the last release. They have been added back in.
Some Audit Logs users were encountering an “Unexpected Error Occurred” message for no apparent reason. This is fixed.
Users were able to select a date range starting as far back as 1900, which was amusing but not very useful. The earliest year is now 2023.
We fixed some links in our user documentation that were missing parameters for CSS formatting in the product.
In the default response for the Malware: Source Code Policy scanner, we corrected the word “message” to “response,” since that scanner only blocks or audits responses.
Users will now see an error message when attempting to upload a file larger than 20MB to a scanner. Previously there was no error message and it was possible for the Save button to get stuck in a loading state.
API keys can now be saved with an expiry date of today.
A styling issue in the Playground resulted in screen flickering when errors and required messages popped up. That’s fixed.
Expired user invitations were still showing in the UI as “Expired” even after another invitation had been sent. Now they will show as “Invited” while the invitation is pending.
We removed a restriction in the Chat UI that prevented users from sending a second prompt before a response to the first prompt was generated.
Some scanners that are not configurable in the UI had a 3-dot icon that made it look like they should be configurable. Those icons have been removed.
We fixed an issue that caused user role updates to not register in the UI right away.
We fixed a few bugs in the Search UI that were giving errors and incorrect results.