Mastering Connections in CalypsoAI Red Team: A Practical Guide

Why Connections Matter
Every Red Team test run needs to talk to an LLM or another AI-powered backend. A connection is your platform’s map: it tells Red Team where to send prompts, how to authenticate, and how to read the response. Nail this early, and you’ll save yourself hours of debugging mysterious errors down the line.

Key Terms—No Jargon Left Behind

Term	What It Means
Provider	The umbrella for a group of models or a backend (think OpenAI, Claude, or your in-house chatbot).
Provider Name	The unique slug (‎`my-chatbot`) you’ll use in YAML and API calls.
Provider Display Name	The friendly label you see in the UI (“My Internal Chatbot”).
Model Display Name	The specific model or tier (“gpt-4o-preview”, “internal-basic”).
Secrets	Encrypted key/value pairs (API keys, tokens, endpoint roots)—stored securely, never in YAML.
Configuration (YAML)	The workflow recipe: builds the request, sends it, and maps the response for Red Team.

Prerequisites Checklist

Before you start, make sure you have:

Endpoint documentation (URL, method, headers, and payload)
Authentication details (API key or bearer token)
Error-handling rules (status codes that trigger retries: 429, 502, 503, 504)

Step-by-Step Tutorial

1. Gather Your Connection Details

Info Needed	Where to Find It
Base URL	Vendor docs or service discovery
Auth Scheme	Docs or your security team
Required Headers	Sample cURL/Postman calls
Payload Template	API swagger or example requests
Success & Retry Codes	Vendor docs

2. Create Secrets in Red Team

Go to Settings → Secrets.
Click New Secret and add keys like:

Secret Key	Example Value
‎`baseUrl`	‎`https://api.vendor.com/v1`
‎`apiKey`	‎`sk-live-123…`
Optional	‎`orgId`, ‎`defaultModel`, etc.

Pro Tip: Rotate tokens via CI/CD or your secret manager. Red Team’s API lets you update secrets without editing YAML.

3. Build Your YAML Configuration

Here’s a minimal template for token-based authentication. Replace the ALL-CAPS placeholders:# Example: JSON chat completion

outputs:
content: ""
statusCode: ""

type: workflow
stages:
- type: retry
attempts: 3
backoff: "0" # 2s, 4s, 8s
block:
type: request
method: POST
timeout: 300
url: "/chat/completions"
headers:
Accept: application/json
Content-Type: application/json
Authorization: "Bearer "
json: >

Customization Tips:

Add ‎`queryParams:` if needed.
Extend the JSON block for vendor-specific fields.
Edit ‎`outputs.content` to map a different response path.
Adjust ‎`timeout` and ‎`attempts` as required.

4. Save & Test

Go to Settings → Connections → Add Provider.
Fill in Provider Name, Provider Display Name, Model Display Name.
Paste your YAML, select your secrets, and hit Test Connection.

Success? You’ll see status 200 and the assistant’s reply.
Failure? The status code and raw body show you exactly what the backend didn’t like.

Best Practices Playbook

Area	Pro Move
Naming conventions	Use ‎`providerName-env` (like ‎`openai-prod`) to clarify environments.
Secrets vs YAML	Keep all sensitive info in Secrets, not YAML.
Retry window	3 attempts + exponential back-off (2, 4, 8s) handles most transient errors.
Error surfacing	Echo the full response body during setup; tighten it later.
Token refresh	Use short-lived tokens and automate nightly secret refresh.
Template library	Maintain a YAML template per auth pattern. Clone and tweak as needed.
Version control	Store YAML in Git and peer-review changes. Keep secrets in the vault.
Least privilege	Scope tokens to read-only inference if possible.
Audit	Enable request/response metadata logging for visibility (no full prompts stored).

Troubleshooting Cheat Sheet

Symptom	Likely Cause	Quick Fix
YAML won’t save	Formatting error	Run it through ‎`yamllint` or an online validator.
401/403 errors	Token missing or expired	Refresh the token in Secrets.
Works in curl, fails here	Missing header or wrong payload key	Compare your curl and YAML configs.
5xx after a wait	Provider idle timeout	Increase ‎`timeout` or reduce batch size.

Template Library—Copy & Paste Ready

OpenAI-Compatible Chat Completionurl: /v1/chat/completions

headers:
Authorization: "Bearer "
json: >

outputs:
content: ""

Generic REST JSON Model method: POST

url: "/generate"
headers:
X-API-KEY: ""
json: >
{input=null, params=null}
outputs:
content: ""

You’re Ready to Connect!

With this guide, your team can spin up new model connections in minutes, troubleshoot with confidence, and keep credentials locked down—no cookies, no streaming, no headaches.