# Roalla AI Gateway — App integration guide

> **Live URL:** https://ai.roalla.com/downloads/AI-GATEWAY-INTEGRATION.md

**Audience:** Developers building Roalla customer apps, agents, or internal tools that need AI features.

**Production URLs**

| Service | URL |
|---------|-----|
| AI Gateway | https://ai.roalla.com |
| Auth Hub (login / JWT) | https://sso.roalla.com |

Use this document to understand **how the gateway fits your app**, **what to configure**, and **how to call it from server-side code**. You do not need access to the gateway repo to integrate.

---

## Quick answers

| Question | Answer |
|----------|--------|
| Do I put `OPENAI_API_KEY` in my app? | **No.** Keys live only on ai.roalla.com. |
| Does the browser call the gateway? | **No.** Call from your server (API routes, BFF, workers). |
| What identifies my app? | `AUTH_CLIENT_ID` — same App ID as on Auth Hub (`client_id`). |
| How does the gateway know who the user is? | Your server sends the user's Auth Hub **access token** as `Authorization: Bearer …`. |
| What if there is no signed-in user? | Use **`AUTH_MAIL_SECRET`** (`ams_…`) from Auth Hub — server-side only. |
| How do I know AI is enabled for my app? | `GET {AI_GATEWAY_URL}/api/config?client_id={AUTH_CLIENT_ID}` |
| Who registers my app on the gateway? | An admin at https://ai.roalla.com/onboard.html (or your ops team). |

---

## Architecture

Roalla splits **identity** (Auth Hub) from **AI** (AI Gateway). Your app talks to both, but only stores public config — never provider API keys.

```text
┌────────────────────────────────────────────────────────────────────┐
│  Your app (Next.js, Express, worker, CLI, etc.)                     │
│  - Users sign in via Auth Hub (sso.roalla.com)                      │
│  - AI calls go to AI Gateway (ai.roalla.com) from SERVER code only  │
│  - Env: AUTH_URL, AUTH_CLIENT_ID, AI_GATEWAY_URL                    │
└───────────────┬──────────────────────────────┬─────────────────────┘
                │                              │
                ▼                              ▼
     ┌────────────────────┐        ┌────────────────────┐
     │  sso.roalla.com    │        │  ai.roalla.com     │
     │  Auth Hub          │        │  AI Gateway        │
     │  - login / MFA     │        │  - provider keys   │
     │  - JWT issuer      │◄─JWKS──│  - monthly quotas  │
     │  - service tokens  │        │  - usage analytics │
     │    (ams_…)         │        │  - OpenAI /v1 proxy│
     └────────────────────┘        └──────────┬─────────┘
                                              ▼
                                    OpenAI · Anthropic · Groq · Gemini
                                    (keys only on gateway)
```

### Request flow (typical signed-in user)

```text
1. User signs in → your app holds Auth Hub access token (session / BFF)
2. User triggers AI feature → your API route runs on the server
3. Server resolves access token (e.g. resolveHubAccessToken())
4. Server POSTs to ai.roalla.com/v1/… with:
     Authorization: Bearer {access_token}
     X-Client-Id: {AUTH_CLIENT_ID}
5. Gateway verifies JWT, checks monthly quota, calls provider
6. Gateway returns provider-shaped JSON; your app stores result in its DB
```

### What lives where

| Concern | Auth Hub | AI Gateway | Your app |
|---------|----------|------------|----------|
| User login | ✓ | — | BFF / session |
| JWT issuance | ✓ | verifies | passes token |
| OpenAI / Whisper / etc. keys | — | ✓ | **never** |
| Monthly AI request cap | — | ✓ per `client_id` | optional app-level limits |
| Prompts, scores, chat history | — | — | your database |
| Service token (`ams_…`) | issues | validates | server env only |

---

## Prerequisites

1. **App registered on Auth Hub** with a stable `client_id` (e.g. `myapp`, `tova`, `pitchhotshot`).
2. **App registered on AI Gateway** with the **same** `client_id`.
   - New app: https://ai.roalla.com/onboard.html (login + AI in one step when SSO sync is configured).
   - Already on SSO: onboard with **Already on SSO** checked and select your app from the dropdown.
3. **At least one provider** connected on the gateway (Admin → AI providers).
   - OpenAI required for Whisper (`/v1/audio/transcriptions`) and OpenAI SDK proxy paths.
   - Groq / Gemini optional for cost-free chat via model name routing.

Your admin should send you an env snippet after onboarding:

```env
AUTH_URL=https://sso.roalla.com
AUTH_CLIENT_ID=myapp
AI_GATEWAY_URL=https://ai.roalla.com
APP_URL=https://myapp.example.com
```

---

## Environment variables (your app)

| Variable | Required | Example | Notes |
|----------|----------|---------|-------|
| `AI_GATEWAY_URL` | Yes (for AI) | `https://ai.roalla.com` | No trailing slash required |
| `AUTH_URL` | Yes | `https://sso.roalla.com` | Same as login integration |
| `AUTH_CLIENT_ID` | Yes | `myapp` | Must match both hubs |
| `AUTH_MAIL_SECRET` | If background AI | `ams_…` | Service Bearer when no user session |
| `OPENAI_API_KEY` | **Never in prod** | — | Remove after migration |

Optional app-side model overrides (your naming):

```env
OPENAI_EVAL_MODEL=gpt-4o-mini
OPENAI_WHISPER_MODEL=whisper-1
```

The gateway **forwards** the `model` field from each request; these env vars are conventions in your app code.

---

## Choose an integration pattern

| Pattern | Best for | Gateway paths |
|---------|----------|---------------|
| **A. `@roalla/gateway` SDK** | Simple text chat from Node | `POST /v1/chat/completions` |
| **B. OpenAI Node SDK** | Whisper, JSON mode, vision, full OpenAI API surface | `/v1/chat/completions`, `/v1/audio/transcriptions`, `/v1/responses` |
| **C. Raw `fetch` / `curl`** | Non-Node stacks, scripts, agents | Same `/v1/*` paths |

All patterns require **server-side** calls with a Bearer token and `X-Client-Id`.

---

## Pattern A — `@roalla/gateway` SDK (simple chat)

Install:

```bash
npm install @roalla/gateway
```

```javascript
const { chatCompletion } = require("@roalla/gateway");

const text = await chatCompletion(
  [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Summarize this ticket." },
  ],
  {
    gatewayUrl: process.env.AI_GATEWAY_URL,
    authToken: accessToken, // Auth Hub JWT from your session
    clientId: process.env.AUTH_CLIENT_ID,
    model: "gpt-4o-mini", // or llama-3.3-70b-versatile, gemini-2.0-flash, etc.
    temperature: 0.3,
    max_tokens: 1024,
    raw: false, // true to get full OpenAI response object
  }
);
```

**Model routing:** For chat completions proxied to OpenAI, the request goes to OpenAI. To use **Groq** or **Gemini** free tiers from the SDK path, pass a model name that your gateway admin has configured (e.g. `llama-3.3-70b-versatile`) — routing applies on gateway-internal paths; the `/v1/chat/completions` proxy forwards to **OpenAI** when using OpenAI-style model names (`gpt-*`).

For multi-provider chat without OpenAI SDK, prefer explicit model names documented by your ops team.

---

## Pattern B — OpenAI Node SDK (full compatibility)

Use when you need **Whisper**, **`response_format: json_object`**, or the **Responses API** (vision). Point the official SDK at the gateway:

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "roalla-gateway", // placeholder — gateway uses Authorization Bearer
  baseURL: `${process.env.AI_GATEWAY_URL}/v1`,
  defaultHeaders: {
    Authorization: `Bearer ${accessToken}`,
    "X-Client-Id": process.env.AUTH_CLIENT_ID!,
  },
});

// Chat + JSON mode
await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "…" }],
  response_format: { type: "json_object" },
});

// Whisper
await client.audio.transcriptions.create({
  model: "whisper-1",
  file: audioFile,
});

// Vision (Responses API)
await client.responses.create({
  model: "gpt-4o-mini",
  input: [/* text + input_image blocks */],
});
```

**Reference app:** PitchHotshot (`client_id: pitchhotshot`) — see Roalla-AI-Gateway repo `ROALLA-HUB_Instructions.md` Part 11.

---

## Pattern C — Raw HTTP

### Chat completions

```http
POST {AI_GATEWAY_URL}/v1/chat/completions
Authorization: Bearer {access_token_or_ams_secret}
X-Client-Id: myapp
Content-Type: application/json

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Hello" }],
  "temperature": 0.3,
  "max_tokens": 500,
  "response_format": { "type": "json_object" }
}
```

Response shape matches [OpenAI Chat Completions](https://platform.openai.com/docs/api-reference/chat).

### Audio transcriptions (multipart)

```http
POST {AI_GATEWAY_URL}/v1/audio/transcriptions
Authorization: Bearer {access_token}
X-Client-Id: myapp
Content-Type: multipart/form-data

model=whisper-1
file=@recording.webm
```

Timeout: gateway allows up to **120 s** for large uploads (≤ 25 MB typical).

### Responses API (vision)

```http
POST {AI_GATEWAY_URL}/v1/responses
Authorization: Bearer {access_token}
X-Client-Id: myapp
Content-Type: application/json

{ "model": "gpt-4o-mini", "input": [ … ] }
```

Body limit **32 MB**; timeout **120 s**.

---

## Authentication

Every `/v1/*` request needs a Bearer token. The gateway accepts **two** types:

| Token | Prefix | When to use | Extra headers |
|-------|--------|-------------|---------------|
| **User access token** | (JWT) | User is signed in | `X-Client-Id` recommended |
| **Service token** | `ams_` | Cron, webhooks, guest flows, no session | **`X-Client-Id` required** |

### User access token

- Issued by Auth Hub after login (same token used for storage SAS, etc.).
- Verified via JWKS: `{AUTH_URL}/.well-known/jwks.json`
- `aud` claim should match your `AUTH_CLIENT_ID` when present.

Resolve in your BFF:

```javascript
// Pseudocode — match your app's session helper
const accessToken = await resolveHubAccessToken(req);
```

### Service token (`AUTH_MAIL_SECRET`)

- Issued per app from Auth Hub (portal / admin).
- Same secret used for `POST /api/mail/send` and `POST /api/storage/sas` on Auth Hub.
- Use as `Authorization: Bearer ams_…` when no user is logged in.
- **Always** send `X-Client-Id: {AUTH_CLIENT_ID}` — there is no user `aud` to infer app.

```javascript
const token = process.env.AUTH_MAIL_SECRET;
const headers = {
  Authorization: `Bearer ${token}`,
  "X-Client-Id": process.env.AUTH_CLIENT_ID,
};
```

### Client identification header

| Header | Accepted aliases |
|--------|------------------|
| `X-Client-Id` | `X-Gateway-Client`, or `client_id` in JSON body |

Usage and quotas are attributed to this `client_id`.

### Optional org context

| Header | Purpose |
|--------|---------|
| `X-Organization-Id` | Recorded in usage analytics when your app is multi-tenant |

---

## Public API reference (app-facing)

No admin key required for these endpoints.

### `GET /api/health`

Liveness check.

```json
{ "ok": true, "service": "roalla-ai-gateway" }
```

### `GET /api/config?client_id={id}`

**Preferred probe** for your app's `/api/config` or feature flags.

```json
{
  "gatewayUrl": "https://ai.roalla.com",
  "authUrl": "https://sso.roalla.com",
  "defaultModel": "gpt-4o-mini",
  "providers": ["groq", "gemini", "openai", "anthropic"],
  "endpoints": {
    "chat_completions": true,
    "audio_transcriptions": true,
    "responses": true
  },
  "client_id": "myapp",
  "registered": true,
  "aiEnabled": true,
  "enabled": true
}
```

| Field | Meaning |
|-------|---------|
| `registered` | App exists and is active on the gateway |
| `aiEnabled` / `enabled` | Registered **and** OpenAI provider key is configured |
| `endpoints` | Which `/v1/*` routes the gateway exposes |

Example in your app:

```javascript
const res = await fetch(
  `${process.env.AI_GATEWAY_URL}/api/config?client_id=${process.env.AUTH_CLIENT_ID}`
);
const { aiEnabled, endpoints } = await res.json();
```

### `GET /api/status`

Operator-oriented health (database, providers). Useful for diagnostics; apps usually use `/api/config`.

### `POST /v1/chat/completions`

OpenAI-compatible chat. **90 s** timeout. Counts as **1** request toward monthly allowance.

### `POST /v1/audio/transcriptions`

OpenAI Whisper-compatible multipart upload. **120 s** timeout. Counts as **1** request.

### `POST /v1/responses`

OpenAI Responses API (vision / multi-modal). **120 s** timeout, **32 MB** JSON. Counts as **1** request.

---

## Expose AI status in your app

Recommended pattern for frontends:

```javascript
// GET /api/config (your app)
export async function GET() {
  const gatewayUrl = process.env.AI_GATEWAY_URL;
  const clientId = process.env.AUTH_CLIENT_ID;
  let aiEnabled = false;

  if (gatewayUrl && clientId) {
    try {
      const r = await fetch(`${gatewayUrl}/api/config?client_id=${clientId}`, {
        next: { revalidate: 60 },
      });
      const data = await r.json();
      aiEnabled = !!data.aiEnabled;
    } catch {
      aiEnabled = false;
    }
  }

  return Response.json({
    aiEnabled,
    aiGatewayUrl: gatewayUrl || "",
    authUrl: process.env.AUTH_URL || "",
  });
}
```

Disable or grey out AI UI when `aiEnabled` is false; do not expose tokens or gateway admin keys to the browser.

---

## Quotas, billing, and rate limits

| Layer | Controlled by | Behavior |
|-------|---------------|----------|
| Monthly request cap | AI Gateway admin (per `client_id`) | `429` + `QUOTA_EXCEEDED` |
| Per-minute burst | Gateway (`DEFAULT_RATE_LIMIT_RPM`) | `429` + `RATE_LIMIT` |
| Usage logging | Gateway Postgres | model, tokens, user_id, org_id — not full prompts |
| Warning email | Gateway (Resend) | Ops notified at threshold |

Each proxied API call (including one Whisper upload) counts as **one** request unless your ops team configures otherwise.

Your app may add its own rate limits (per user, per org) on top of the gateway cap.

---

## HTTP errors (troubleshooting)

| Status | Code / symptom | Likely cause | What to check |
|--------|----------------|--------------|---------------|
| 401 | `Authorization required` / invalid token | Missing or expired JWT | User re-login; `AUTH_URL` correct |
| 400 | `X-Client-Id required` | Service token without client header | Add `X-Client-Id` |
| 403 | `APP_NOT_FOUND` / `APP_PAUSED` | App not onboarded or paused | ai.roalla.com admin |
| 403 | Audience mismatch | `X-Client-Id` ≠ JWT `aud` | Align IDs with Auth Hub |
| 429 | `QUOTA_EXCEEDED` | Monthly cap hit | Admin raises limit or wait for reset |
| 429 | `RATE_LIMIT` | Too many requests/minute | Back off; check `Retry-After` |
| 503 | `PROVIDER_NOT_CONFIGURED` | OpenAI (or other) key missing on gateway | Admin → Providers |
| 502 | `gateway_error` | Upstream OpenAI/provider failure | Gateway logs; provider status |

**Login works, AI fails** → issue is almost always on **ai.roalla.com** (registration, provider key, quota).  
**AI works for one app, not another** → compare `client_id` registration and limits.

---

## Security rules

1. **Never** commit or ship `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or gateway `ADMIN_API_KEY` in product apps.
2. **Never** call `/v1/*` from the browser — users could extract tokens or bypass quotas.
3. **Always** use the same `client_id` on Auth Hub and AI Gateway.
4. Store **`AUTH_MAIL_SECRET`** only in server env (Railway, secrets manager).
5. Provider keys rotate on the gateway only — apps do not redeploy when keys change.

---

## Model and provider notes

| Model prefix / name | Provider (gateway) |
|---------------------|-------------------|
| `gpt-*`, `o1`, `o3`, `o4` | OpenAI |
| `claude*` | Anthropic |
| `llama*`, `mixtral*`, `gemma*` | Groq |
| `gemini*` | Google Gemini (AI Studio) |

The **`/v1/*` OpenAI proxy** (Patterns B & C) forwards to **OpenAI** using the gateway's OpenAI key. Use OpenAI model names on those paths.

Groq/Gemini are primarily for gateway-managed routing and admin model testing; confirm with ops which models are enabled for your app.

---

## FAQ

**Can I use the gateway without Auth Hub?**  
No. JWT verification depends on Auth Hub JWKS. Service tokens are also issued by Auth Hub.

**Can I use different App IDs on SSO vs AI?**  
No. Use one `client_id` everywhere.

**Does the gateway store my prompts?**  
Usage events store metadata (model, tokens, user id). Full prompts are not retained for analytics.

**Can I stream chat completions?**  
The proxy forwards OpenAI responses; streaming support depends on request body and provider. Confirm with your gateway version.

**How do I test locally?**  
Point `AI_GATEWAY_URL` at production or a staging gateway. Use a staging `client_id` registered on both hubs. Still call from server code with a real user JWT or service token.

**Who do I ask for higher limits or new providers?**  
Roalla ops / gateway admin via ai.roalla.com admin console.

**Where is PitchHotshot documented?**  
Roalla-AI-Gateway repo `ROALLA-HUB_Instructions.md` Part 11 (Whisper, vision, coach, guest feedback).

**I'm building a Cursor agent / CLI tool — what do I need?**  
`AI_GATEWAY_URL`, `AUTH_CLIENT_ID`, and either a user JWT (OAuth flow via Auth Hub) or `AUTH_MAIL_SECRET` for unattended jobs. Call `/v1/chat/completions` with `curl` or `fetch` from your tool's server component — not from an untrusted client.

---

## Checklist — new app integration

### Admin (ops)

- [ ] Register app on Auth Hub (or use unified onboard)
- [ ] Register same `client_id` on https://ai.roalla.com/onboard.html
- [ ] Set monthly allowance
- [ ] Connect required providers (OpenAI for Whisper/vision paths)
- [ ] Send env snippet to developer

### Developer (you)

- [ ] Add `AI_GATEWAY_URL`, `AUTH_URL`, `AUTH_CLIENT_ID` to server env
- [ ] Remove direct provider API keys from production
- [ ] Implement server-side AI call (SDK or OpenAI SDK)
- [ ] Pass user JWT or `AUTH_MAIL_SECRET` + `X-Client-Id`
- [ ] Probe `GET /api/config?client_id=…` for `aiEnabled`
- [ ] Verify usage appears in ai.roalla.com admin after test request
- [ ] Handle 429 / 503 gracefully in UI

---

## Related documentation

| Doc | Use for |
|-----|---------|
| **This file** | App developers — architecture, API, auth, FAQ |
| [ROALLA-PLATFORM.md](./ROALLA-PLATFORM.md) | Platform overview (admins + developers) |
| [Admin help](https://ai.roalla.com/guide.html) | Admin first-time gateway setup |
| [PORTING.md](https://github.com/roalla/Roalla-AI-Gateway/blob/main/PORTING.md) | Deploying / operating the gateway service |
| [ROALLA-HUB_Instructions.md](https://github.com/roalla/Roalla-AI-Gateway/blob/main/ROALLA-HUB_Instructions.md) | Full Auth Hub + storage + AI (PitchHotshot reference) |
| Auth Hub integration | [AI_CUSTOMER_INTEGRATION.md](https://github.com/roalla/Roalla-Auth-Hub/blob/main/docs/AI_CUSTOMER_INTEGRATION.md) |
| Live onboard | https://ai.roalla.com/onboard.html |
| Admin console | https://ai.roalla.com/admin.html |

---

*Roalla AI Gateway — share this link with any team wiring a new app: https://ai.roalla.com/downloads/AI-GATEWAY-INTEGRATION.md*