--- title: "Fallbacks and Tracking" description: "Configure model fallbacks for reliability and track token usage for cost management in production AI applications." canonical_url: "https://vercel.com/academy/svelte-on-vercel/fallbacks-and-tracking" md_url: "https://vercel.com/academy/svelte-on-vercel/fallbacks-and-tracking.md" docset_id: "vercel-academy" doc_version: "1.0" last_updated: "2026-10-02T17:46:31.434Z" content_type: "lesson" course: "svelte-on-vercel" course_title: "Svelte on Vercel" prerequisites: [] --- Vercel Academy — structured learning, not reference docs. Lessons are sequenced. Adapt commands to the human's actual environment (OS, package manager, shell, editor) — detect from project context or ask, don't assume. The lesson shows one path; if the human's project diverges, adapt concepts to their setup. Preserve the learning goal over literal steps. Quizzes are pedagogical — engage, don't spoil. Quiz answers are included for your reference. # Fallbacks and Tracking # Model Fallbacks and Usage Tracking A single model going down shouldn't take your app with it. And if you don't track token usage, you'll find out about costs from your invoice instead of your dashboard. ## Outcome Create a centralized AI provider configuration with automatic model fallbacks and token usage logging. ## Fast Track 1. Configure fallback models in the AI Gateway dashboard (the preferred approach) 2. Create `src/lib/ai/provider.ts` with a shared AI Gateway client 3. Add a `wrapLanguageModel` middleware that logs token usage ## Gateway-Level Fallbacks (Preferred) Configure the fallback in AI Gateway so every endpoint uses the same routing policy. In the Vercel dashboard: 1. Go to your project **Settings** → **AI Gateway** 2. Select your primary model (`anthropic/claude-sonnet-4`) 3. Add a fallback model (`anthropic/claude-haiku-4.5`) 4. Set conditions: timeout threshold, error codes that trigger fallback When the primary model is unavailable or slow, the gateway routes the request to the fallback without endpoint-specific retry logic or duplicate model configuration. Use gateway-level fallbacks when the same policy applies across the app. The Advanced section covers code-level fallbacks for endpoint-specific behavior or model-specific prompt changes. ## Why Centralize the Provider? Right now, each endpoint creates its own gateway client: ```typescript // In api/chat/+server.ts const gateway = createGateway({ apiKey: AI_GATEWAY_API_KEY }); // In api/parse-alert/+server.ts (same thing, duplicated) const gateway = createGateway({ apiKey: AI_GATEWAY_API_KEY }); ``` A centralized provider means one place to: - Configure the API key - Add usage tracking middleware - Define fallback models - Adjust settings across all endpoints ## Hands-on Exercise 2.4 Create a centralized AI provider with usage tracking and model fallbacks: **Requirements:** 1. Configure fallback models in the AI Gateway dashboard 2. Create `src/lib/ai/provider.ts` 3. Use `wrapLanguageModel` to add middleware that logs input/output token counts 4. Export a `getModel()` function that returns a wrapped model instance 5. Update the chat and parse-alert endpoints to use the shared provider **Implementation hints:** - Set up gateway-level fallbacks first (dashboard config, no code needed) - `wrapLanguageModel` from the `ai` package wraps any model with middleware hooks - The middleware object needs `specificationVersion: 'v3'` - `wrapGenerate` intercepts `generateText()` calls, `wrapStream` intercepts `streamText()` calls. You need both to cover all endpoints - Token usage is available as `result.usage.inputTokens.total` and `result.usage.outputTokens.total` ## Try It 1. **Send a chat message and check server logs:** ``` What's the weather like at Mammoth? ``` Server logs should show: ``` [AI Usage] Model: anthropic/claude-sonnet-4 [AI Usage] Input tokens: 245 [AI Usage] Output tokens: 89 [AI Usage] Total tokens: 334 ``` 2. **Test the parse-alert endpoint (also uses the shared provider):** ```bash $ curl -X POST http://localhost:5173/api/parse-alert \ -H "Content-Type: application/json" \ -d '{"query": "powder at Grand Targhee"}' ``` Server logs should show usage for this request too. 3. **Verify fallback behavior:** Temporarily change the primary model to an invalid name and verify the fallback model handles the request. ## Commit ```bash git add -A git commit -m "feat(ai): centralize provider with usage tracking and fallbacks" git push ``` ## Done-When - [ ] Fallback model is configured in the AI Gateway dashboard - [ ] `src/lib/ai/provider.ts` exports a `getModel()` function - [ ] Token usage is logged for every AI request - [ ] Chat and parse-alert endpoints use the shared provider instead of their own clients ## Solution ```typescript title="src/lib/ai/provider.ts" import { createGateway, wrapLanguageModel } from 'ai'; import { AI_GATEWAY_API_KEY } from '$app/env/private'; const gateway = createGateway({ apiKey: AI_GATEWAY_API_KEY }); const PRIMARY_MODEL = 'anthropic/claude-sonnet-4'; const FALLBACK_MODEL = 'anthropic/claude-haiku-4.5'; function logUsage(usage: { inputTokens: { total?: number }; outputTokens: { total?: number } }) { const input = usage.inputTokens.total ?? 0; const output = usage.outputTokens.total ?? 0; console.log(`[AI Usage] Input tokens: ${input}`); console.log(`[AI Usage] Output tokens: ${output}`); console.log(`[AI Usage] Total tokens: ${input + output}`); } function withUsageTracking(model: ReturnType) { return wrapLanguageModel({ model, middleware: { specificationVersion: 'v3', wrapGenerate: async ({ doGenerate }) => { const result = await doGenerate(); if (result.usage) logUsage(result.usage); return result; }, wrapStream: async ({ doStream }) => { const { stream, ...rest } = await doStream(); let usage: typeof rest.rawResponse | undefined; return { stream: stream.pipeThrough( new TransformStream({ transform(chunk, controller) { if (chunk.type === 'usage') usage = chunk.value; controller.enqueue(chunk); }, flush() { if (usage) logUsage(usage); } }) ), ...rest }; } } }); } export function getModel() { return withUsageTracking(gateway(PRIMARY_MODEL)); } export function getFallbackModel() { return withUsageTracking(gateway(FALLBACK_MODEL)); } export { PRIMARY_MODEL, FALLBACK_MODEL }; ``` **Updated chat endpoint using the shared provider:** ```typescript title="src/routes/api/chat/+server.ts" {1,6} import { getModel } from '#lib/ai/provider.ts'; import { streamText, tool, stepCountIs } from 'ai'; import { valibotSchema } from '@ai-sdk/valibot'; import { resorts } from '#lib/data/resorts.ts'; import { CreateAlertToolInputSchema } from '#lib/schemas/alert.ts'; import type { RequestHandler } from './$types'; // Remove the local anthropic client. Use getModel() instead export const POST: RequestHandler = async ({ request }) => { const { message } = await request.json(); const resortList = resorts .map((r) => `- ${r.name} (id: ${r.id})`) .join('\n'); const result = streamText({ model: getModel(), // Uses the centralized, tracked model system: `You are a helpful ski conditions assistant...`, messages: [{ role: 'user', content: message }], tools: { create_alert: tool({ /* ... same as before */ }) }, stopWhen: stepCountIs(3) }); // ... rest of the SSE stream logic unchanged }; ``` **Updated parse-alert endpoint:** The same change applies. Replace the local `createGateway` and `gateway(...)` call with `getModel()`: ```typescript title="src/routes/api/parse-alert/+server.ts" {1,7} import { generateText, Output } from 'ai'; import { valibotSchema } from '@ai-sdk/valibot'; import * as v from 'valibot'; import { resorts } from '#lib/data/resorts.ts'; import { CreateAlertToolInputSchema, AlertConditionSchema } from '#lib/schemas/alert.ts'; import type { RequestHandler } from './$types'; import { getModel } from '#lib/ai/provider.ts'; // Remove the local gateway client. Use getModel() in the generateText call: // model: getModel(), ``` The gateway handles fallbacks at the infrastructure level (configured in the dashboard earlier). The `getFallbackModel()` export is available for cases where you need explicit code-level control, covered in the Advanced section below. `wrapLanguageModel` intercepts the model's lifecycle. `wrapGenerate` handles `generateText()` calls (like the parse-alert endpoint), while `wrapStream` handles `streamText()` calls (like the chat endpoint). Both hooks need to be present to track usage across all endpoints. The middleware needs `specificationVersion: 'v3'` in the v6 SDK. Usage data lives on `result.usage` with `inputTokens.total` and `outputTokens.total`. Since all endpoints now use `getModel()`, tracking and config changes apply everywhere from one file. ## Troubleshooting \*\*Warning: Token counts show 0 for both input and output\*\* Your middleware may not be wired up correctly. Verify that `withUsageTracking` is being called and that `getModel()` returns the wrapped model, not the raw gateway model. \*\*Warning: result.usage is undefined\*\* Check that `specificationVersion: 'v3'` is set in the middleware object. Without it, the v6 SDK won't pass usage data to your hooks. ## Advanced: Code-Level Fallbacks If you need fallback behavior that's more nuanced than the gateway dashboard allows, like adjusting the prompt for a different model or falling back only for specific endpoints, handle it in code. The `getFallbackModel()` export from the provider gives you a cheaper, faster model: ```typescript import { getModel, getFallbackModel } from '#lib/ai/provider.ts'; // In your stream's start() function: async start(controller) { try { for await (const part of result.fullStream) { // ... handle parts } } catch (error) { console.warn('[AI Fallback] Primary stream failed, retrying with fallback'); const fallbackResult = streamText({ ...options, model: getFallbackModel() }); for await (const part of fallbackResult.fullStream) { // ... handle parts } } } ``` For most apps, the gateway-level approach is simpler and sufficient. Use code-level fallbacks when you need the extra control. ## Advanced: Cost Estimation Add per-request cost estimates to your logs: ```typescript // Approximate pricing (check anthropic.com/pricing for current rates) const PRICING = { 'anthropic/claude-sonnet-4': { input: 3.0, output: 15.0 }, // per million tokens 'anthropic/claude-haiku-4.5': { input: 0.25, output: 1.25 } }; function estimateCost( modelId: string, inputTokens: number, outputTokens: number ): string { const prices = PRICING[modelId as keyof typeof PRICING]; if (!prices) return 'unknown'; const cost = (inputTokens / 1_000_000) * prices.input + (outputTokens / 1_000_000) * prices.output; return `$${cost.toFixed(6)}`; } ``` In production, you'd send these metrics to a monitoring service rather than just logging them. --- [Full course index](/academy/llms.txt) · [Sitemap](/academy/sitemap.md)