Simple Chat Completion:
├── User question: 50 tokens
├── System prompt: 200 tokens  
├── Context window: 1,000 tokens
└── Total per interaction: ~1,250 tokens

Monthly consumption: 50,000 interactions × 1,250 = 62.5M tokens
Cost at $0.03/1K tokens: ~$1,875/month

Agentic Workflow with Tools:
├── Initial query: 100 tokens
├── System prompt + instructions: 800 tokens
├── Available tools context: 2,000 tokens
├── Conversation history: 5,000 tokens
├── Tool call iterations (3×): 3,000 tokens
├── RAG retrieval context: 8,000 tokens
├── Reasoning steps: 2,000 tokens
└── Total per interaction: ~20,900 tokens

Monthly consumption: 10,000 interactions × 20,900 = 209M tokens
Cost at $0.03/1K tokens: ~$6,270/month

// Each tool definition adds 50-200 tokens to every request
const tools = [
  {
    name: "search_database",
    description: "Searches the customer database for records...",
    parameters: { /* ... */ }
  },
  {
    name: "send_email", 
    description: "Sends an email to specified recipients...",
    parameters: { /* ... */ }
  }
  // 20+ tools = 2,000-4,000 tokens per request
];

Conversation Growth Pattern:
├── Turn 1: 1,000 tokens
├── Turn 2: 2,500 tokens (includes Turn 1)
├── Turn 3: 5,000 tokens (includes Turns 1-2)
├── Turn 10: 25,000 tokens
└── Turn 50: 120,000 tokens

Cost per turn increases linearly with conversation length

Typical Retry Pattern:
├── Initial attempt: 5,000 tokens
├── Tool timeout retry: 5,000 tokens
├── Invalid response retry: 5,000 tokens
├── Clarification request: 3,000 tokens
├── Final successful attempt: 5,000 tokens
└── Total for single operation: 23,000 tokens

┌─────────────────────────────────────────────────────────────────────────┐
│              COST-AWARE AI WORKFLOW ARCHITECTURE                        │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   Input                                                                 │
│     │                                                                   │
│     ▼                                                                   │
│   ┌──────────────────┐                                                 │
│   │  Cost Guardrail  │  ◄── Check budget limits before processing     │
│   │  (Redis/Cache)   │                                                 │
│   └────────┬─────────┘                                                 │
│            │                                                           │
│     Budget │ OK?                                                       │
│            │                                                           │
│     ▼      ▼                                                           │
│   ┌──────────────────┐         ┌──────────────────┐                   │
│   │  Request         │         │  Reject/Fallback │                   │
│   │  Deduplication   │         │  Flow            │                   │
│   │  (Cache Check)   │         │                  │                   │
│   └────────┬─────────┘         └──────────────────┘                   │
│            │                                                            │
│     Cache  │ Hit?                                                       │
│            │                                                            │
│     ▼      ▼                                                            │
│   ┌──────────────────┐         ┌──────────────────┐                    │
│   │  Return Cached   │         │  Context         │                    │
│   │  Response        │         │  Optimization    │                    │
│   │  $0 Cost         │         │  (Token         │                    │
│   └──────────────────┘         │   Reduction)   │                    │
│                                └────────┬─────────┘                    │
│                                         │                              │
│                                         ▼                              │
│                                ┌──────────────────┐                    │
│                                │  Smart Routing   │                    │
│                                │  (Model         │                    │
│                                │   Selection)   │                    │
│                                └────────┬─────────┘                    │
│                                         │                              │
│     ┌───────────────────────────────────┼───────────────────┐          │
│     │                                   │                   │          │
│     ▼                                   ▼                   ▼          │
│   ┌──────────┐                   ┌──────────┐       ┌──────────┐     │
│   │  Local   │                   │  Claude  │       │  GPT-5.5 │     │
│   │  Ollama  │                   │  Sonnet  │       │  Direct  │     │
│   │  (Free)  │                   │  4.8     │       │  API     │     │
│   └────┬─────┘                   └────┬─────┘       └────┬─────┘     │
│        │                              │                    │          │
│        └──────────────────────────────┼────────────────────┘          │
│                                       │                               │
│                                       ▼                               │
│                              ┌──────────────────┐                    │
│                              │  Response        │                    │
│                              │  Caching         │                    │
│                              │  & Storage       │                    │
│                              └────────┬─────────┘                    │
│                                       │                               │
│                                       ▼                               │
│                              ┌──────────────────┐                    │
│                              │  Cost Tracking   │                    │
│                              │  & Analytics     │                    │
│                              │  (Database)      │                    │
│                              └──────────────────┘                    │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

// CostGuardrail.js - Execute node in n8n
const workflowId = $workflow.id;
const userId = $execution.customData?.userId || 'anonymous';

// Define budget limits
const BUDGET_CONFIG = {
  dailyLimit: 50,      // $50 USD per day
  hourlyLimit: 10,     // $10 USD per hour
  perRequestLimit: 0.50 // $0.50 per request max
};

// Query current spending from PostgreSQL
const spendingQuery = await $executeRawQuery({
  query: `
    SELECT 
      COALESCE(SUM(cost_usd), 0) as daily_spend,
      COALESCE(SUM(CASE WHEN created_at > NOW() - INTERVAL '1 hour' THEN cost_usd END), 0) as hourly_spend
    FROM ai_token_usage
    WHERE workflow_id = $1
    AND DATE(created_at) = CURRENT_DATE
  `,
  parameters: [workflowId]
});

const { daily_spend, hourly_spend } = spendingQuery[0];

// Check budget violations
const budgetStatus = {
  withinDaily: daily_spend < BUDGET_CONFIG.dailyLimit,
  withinHourly: hourly_spend < BUDGET_CONFIG.hourlyLimit,
  currentDailySpend: daily_spend,
  currentHourlySpend: hourly_spend,
  remainingDaily: BUDGET_CONFIG.dailyLimit - daily_spend,
  remainingHourly: BUDGET_CONFIG.hourlyLimit - hourly_spend
};

// Return control decision
if (!budgetStatus.withinDaily) {
  return {
    json: {
      allowed: false,
      reason: 'DAILY_BUDGET_EXCEEDED',
      currentSpend: daily_spend,
      limit: BUDGET_CONFIG.dailyLimit,
      suggestion: 'Request will be queued for tomorrow or escalate to premium tier'
    }
  };
}

if (!budgetStatus.withinHourly) {
  return {
    json: {
      allowed: false,
      reason: 'HOURLY_RATE_LIMIT',
      currentSpend: hourly_spend,
      limit: BUDGET_CONFIG.hourlyLimit,
      retryAfter: new Date(Date.now() + 60 * 60 * 1000).toISOString()
    }
  };
}

return {
  json: {
    allowed: true,
    budgetStatus,
    maxRequestCost: BUDGET_CONFIG.perRequestLimit
  }
};

# Cost Guardrail Sub-Workflow
trigger:
  type: webhook
  path: cost-check

nodes:
  - name: Get Budget Config
    type: postgres
    operation: select
    query: |
      SELECT * FROM workflow_budgets 
      WHERE workflow_id = {{ $workflow.id }}

  - name: Check Current Spending
    type: postgres
    operation: select
    query: |
      SELECT 
        COALESCE(SUM(cost_usd), 0) as daily_spend,
        COALESCE(SUM(CASE WHEN created_at > NOW() - INTERVAL '1 hour' 
                     THEN cost_usd END), 0) as hourly_spend
      FROM ai_token_usage
      WHERE workflow_id = {{ $workflow.id }}
      AND DATE(created_at) = CURRENT_DATE

  - name: Evaluate Budget
    type: code
    js: |
      const config = $('Get Budget Config').item.json;
      const spending = $('Check Current Spending').item.json;
      
      const dailyRemaining = config.daily_limit - spending.daily_spend;
      const hourlyRemaining = config.hourly_limit - spending.hourly_spend;
      
      return [{
        json: {
          allowed: dailyRemaining > 0 && hourlyRemaining > 0,
          dailyRemaining,
          hourlyRemaining,
          maxRequestCost: Math.min(config.per_request_limit, dailyRemaining),
          budgetStatus: {
            dailyUsed: spending.daily_spend,
            dailyLimit: config.daily_limit,
            hourlyUsed: spending.hourly_spend,
            hourlyLimit: config.hourly_limit
          }
        }
      }];

  - name: Conditional Flow
    type: if
    conditions:
      - name: Budget OK
        value: '{{ $json.allowed }}'
        operator: equal
        conditionValue: true

  - name: Log Denied Request
    type: postgres
    operation: insert
    query: |
      INSERT INTO denied_requests 
      (workflow_id, reason, requested_at, retry_after)
      VALUES ({{ $workflow.id }}, 'BUDGET_EXCEEDED', NOW(), 
              NOW() + INTERVAL '1 hour')
    condition: '{{ !$json.allowed }}'

Traditional Approach (No Caching):
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Query     │────▶│   LLM API    │────▶│   $0.05     │
│ "How do I   │     │  Processing  │     │   Cost      │
│  reset my   │     │   10,000     │     │             │
│  password?" │     │   tokens     │     │             │
└─────────────┘     └──────────────┘     └─────────────┘

   [User asks same question 100 times]
   
   Total Cost: $5.00

Semantic Caching Approach:
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Query     │────▶│   Vector     │────▶│   Redis     │
│ "How do I   │     │   Similarity │────▶│   Cache     │
│  reset my   │     │   Check      │     │   Hit!      │
│  password?" │     │   100 tokens │     │   $0.0005   │
└─────────────┘     └──────────────┘     └─────────────┘

   [User asks variations 100 times]
   "Reset password"
   "Forgot my password"
   "Password reset help"
   "How to change password"
   
   Total Cost: ~$0.10 (98% reduction)

// SemanticCache.js - Function node
const OpenAI = require('openai');
const { createClient } = require('redis');

const openai = new OpenAI({ apiKey: $env.OPENAI_API_KEY });
const redis = createClient({ url: $env.REDIS_URL });
await redis.connect();

const CACHE_THRESHOLD = 0.92; // Cosine similarity threshold
const CACHE_TTL = 60 * 60 * 24 * 7; // 7 days

async function getEmbedding(text) {
  const response = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
    dimensions: 1536
  });
  return response.data[0].embedding;
}

async function findSimilar(queryEmbedding) {
  // Search Redis Vector DB for similar queries
  const results = await redis.ft.search('semantic_cache', '*', {
    PARAMS: {
      vec: Buffer.from(new Float32Array(queryEmbedding).buffer),
      radius: 0.15
    },
    SORTBY: { BY: '__v_score', DIRECTION: 'ASC' },
    LIMIT: { from: 0, to: 1 }
  });
  
  if (results.total > 0) {
    const match = results.documents[0];
    const similarity = 1 - parseFloat(match.value.__v_score);
    
    if (similarity >= CACHE_THRESHOLD) {
      return {
        hit: true,
        similarity,
        response: JSON.parse(match.value.response),
        cachedAt: match.value.timestamp
      };
    }
  }
  
  return { hit: false };
}

async function cacheResponse(query, embedding, response) {
  const cacheKey = `cache:${Date.now()}:${Math.random().toString(36).substr(2, 9)}`;
  
  await redis.hSet(cacheKey, {
    query: query,
    embedding: Buffer.from(new Float32Array(embedding).buffer),
    response: JSON.stringify(response),
    timestamp: Date.now(),
    usage_count: 0
  });
  
  await redis.expire(cacheKey, CACHE_TTL);
}

// Main execution
const userQuery = $input.first().json.query;
const queryEmbedding = await getEmbedding(userQuery);
const cacheResult = await findSimilar(queryEmbedding);

if (cacheResult.hit) {
  // Cache hit - return cached response
  await redis.hIncrBy(`cache:${cacheResult.cachedAt}`, 'usage_count', 1);
  
  return [{
    json: {
      source: 'cache',
      response: cacheResult.response,
      similarity: cacheResult.similarity,
      cost_saved: 0.05, // Estimated cost avoided
      cached_at: new Date(parseInt(cacheResult.cachedAt)).toISOString()
    }
  }];
} else {
  // Cache miss - need to call LLM
  return [{
    json: {
      source: 'llm_required',
      query: userQuery,
      embedding: queryEmbedding,
      estimated_cost: 0.05
    }
  }];
}

# Create semantic cache index in Redis
redis-cli FT.CREATE semantic_cache ON HASH PREFIX 1 cache: SCHEMA 
  query TEXT 
  embedding VECTOR FLAT 6 DIM 1536 DISTANCE_METRIC COSINE 
  TYPE FLOAT32 
  response TEXT 
  timestamp NUMERIC SORTABLE 
  usage_count NUMERIC

// ToolSelector.js
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: $env.OPENAI_API_KEY });

const ALL_TOOLS = [
  { name: 'search_customers', description: 'Search customer database', category: 'crm' },
  { name: 'create_invoice', description: 'Generate new invoice', category: 'billing' },
  { name: 'send_email', description: 'Send email notification', category: 'communication' },
  { name: 'slack_notify', description: 'Send Slack message', category: 'communication' },
  { name: 'zendesk_ticket', description: 'Create support ticket', category: 'support' },
  { name: 'refund_order', description: 'Process refund', category: 'billing' },
  // ... 20+ more tools
];

async function selectRelevantTools(userQuery) {
  // Use a small, fast model for tool selection
  const response = await openai.chat.completions.create({
    model: 'gpt-4.1-nano', // Cheapest model for routing
    messages: [
      {
        role: 'system',
        content: `You are a tool selector. Given a user query and available tools, 
        return ONLY the names of tools that might be relevant (JSON array).
        Available tools: ${JSON.stringify(ALL_TOOLS.map(t => ({name: t.name, desc: t.description})))}`
      },
      { role: 'user', content: userQuery }
    ],
    response_format: { type: 'json_object' }
  });
  
  const selected = JSON.parse(response.choices[0].message.content).tools;
  const relevantTools = ALL_TOOLS.filter(t => selected.includes(t.name));
  
  return {
    tools: relevantTools,
    tokensSaved: (ALL_TOOLS.length - relevantTools.length) * 150, // ~150 tokens per tool
    selectionCost: response.usage.total_tokens * 0.000002 // Cost of selection call
  };
}

const query = $input.first().json.query;
const result = await selectRelevantTools(query);

return [{
  json: {
    selectedTools: result.tools,
    optimization: {
      tokensSaved: result.tokensSaved,
      estimatedSavings: (result.tokensSaved / 1000) * 0.015, // At $0.015/1K tokens
      selectionCost: result.selectionCost,
      netSavings: (result.tokensSaved / 1000) * 0.015 - result.selectionCost
    }
  }
}];

// ConversationSummarizer.js
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: $env.OPENAI_API_KEY });

const MAX_CONTEXT_TOKENS = 4000;
const SUMMARIZATION_THRESHOLD = 3000;

async function optimizeConversation(messages) {
  const estimatedTokens = estimateTokens(messages);
  
  if (estimatedTokens < SUMMARIZATION_THRESHOLD) {
    return { messages, tokens: estimatedTokens, action: 'no_change' };
  }
  
  if (estimatedTokens > MAX_CONTEXT_TOKENS) {
    // Summarize older messages
    const messagesToSummarize = messages.slice(0, -4); // Keep last 4 exchanges
    const recentMessages = messages.slice(-4);
    
    const summary = await openai.chat.completions.create({
      model: 'gpt-4.1-nano',
      messages: [
        {
          role: 'system',
          content: `Summarize this conversation concisely. Include key facts, 
          decisions, and context needed for future responses. Be brief.`
        },
        {
          role: 'user',
          content: messagesToSummarize.map(m => `${m.role}: ${m.content}`).join('\n')
        }
      ]
    });
    
    const optimizedMessages = [
      { role: 'system', content: `Previous conversation summary: ${summary.choices[0].message.content}` },
      ...recentMessages
    ];
    
    const newTokenCount = estimateTokens(optimizedMessages);
    
    return {
      messages: optimizedMessages,
      tokens: newTokenCount,
      action: 'summarized',
      reduction: estimatedTokens - newTokenCount,
      savings: ((estimatedTokens - newTokenCount) / 1000) * 0.03
    };
  }
  
  return { messages, tokens: estimatedTokens, action: 'trimmed' };
}

function estimateTokens(messages) {
  // Rough estimate: 4 characters ≈ 1 token
  const text = messages.map(m => m.content).join('');
  return Math.ceil(text.length / 4);
}

const conversation = $input.first().json.conversationHistory;
const result = await optimizeConversation(conversation);

return [{
  json: result
}];

// ResponseLimiter.js - Code node in n8n
const MAX_OUTPUT_TOKENS = {
  'gpt-4.1': 2000,
  'gpt-4.1-mini': 1500,
  'claude-sonnet-4.8': 4000,
  'claude-haiku-3': 2000
};

const model = $input.first().json.model || 'gpt-4.1';
const maxTokens = MAX_OUTPUT_TOKENS[model] || 2000;

// Add to your OpenAI/Anthropic call
const completion = await openai.chat.completions.create({
  model: model,
  messages: messages,
  max_tokens: maxTokens,  // Hard limit on output
  temperature: 0.7
});

return [{
  json: {
    response: completion.choices[0].message.content,
    usage: completion.usage,
    cost: calculateCost(completion.usage, model),
    maxTokensEnforced: maxTokens
  }
}];

function calculateCost(usage, model) {
  const pricing = {
    'gpt-4.1': { input: 0.000002, output: 0.000008 },
    'gpt-4.1-mini': { input: 0.0000004, output: 0.0000016 },
    'claude-sonnet-4.8': { input: 0.000003, output: 0.000015 }
  };
  
  const rates = pricing[model] || pricing['gpt-4.1'];
  return (usage.prompt_tokens * rates.input) + (usage.completion_tokens * rates.output);
}

┌─────────────────────────────────────────────────────────────────────────┐
│                    SMART MODEL ROUTING LAYER                            │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   Request ──▶ Intent Classification ──▶ Model Selection              │
│                                                                         │
│   ┌─────────────────────────────────────────────────────────────────┐  │
│   │ Classification Categories:                                       │  │
│   │                                                                  │  │
│   │ • SIMPLE_QA ──────────────▶ GPT-4.1-Nano (cheapest)           │  │
│   │   "What hours are you open?"                                    │  │
│   │   Cost: ~$0.0001 per request                                    │  │
│   │                                                                  │  │
│   │ • MODERATE_TASK ────────────▶ GPT-4.1-Mini                       │  │
│   │   "Summarize this article"                                      │  │
│   │   Cost: ~$0.002 per request                                      │  │
│   │                                                                  │  │
│   │ • COMPLEX_ANALYSIS ─────────▶ GPT-4.1 or Claude Sonnet 4.8     │  │
│   │   "Analyze this contract for risks"                             │  │
│   │   Cost: ~$0.05 per request                                       │  │
│   │                                                                  │  │
│   │ • CODING_TASK ──────────────▶ Claude Opus 4.8 (best coding)    │  │
│   │   "Debug this Python function"                                    │  │
│   │   Cost: ~$0.10 per request                                       │  │
│   │                                                                  │  │
│   │ • CREATIVE_WRITING ─────────▶ GPT-5.5 (best creativity)          │  │
│   │   "Write a compelling marketing campaign"                         │  │
│   │   Cost: ~$0.15 per request                                       │  │
│   │                                                                  │  │
│   └─────────────────────────────────────────────────────────────────┘  │
│                                                                         │
│   Typical Savings: 60-80% vs. using most expensive model for all        │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

// ModelRouter.js
const { OpenAI } = require('openai');
const axios = require('axios');

const openai = new OpenAI({ apiKey: $env.OPENAI_API_KEY });

// Model configurations with cost per 1K tokens
const MODELS = {
  'gpt-4.1-nano': {
    provider: 'openai',
    cost: { input: 0.0001, output: 0.0004 },
    capabilities: ['simple_qa', 'classification'],
    maxTokens: 16000
  },
  'gpt-4.1-mini': {
    provider: 'openai',
    cost: { input: 0.0004, output: 0.0016 },
    capabilities: ['simple_qa', 'classification', 'summarization', 'moderate_analysis'],
    maxTokens: 128000
  },
  'gpt-4.1': {
    provider: 'openai',
    cost: { input: 0.002, output: 0.008 },
    capabilities: ['all_except_specialized'],
    maxTokens: 128000
  },
  'claude-sonnet-4.8': {
    provider: 'anthropic',
    cost: { input: 0.003, output: 0.015 },
    capabilities: ['complex_reasoning', 'analysis', 'long_context'],
    maxTokens: 200000
  },
  'claude-opus-4.8': {
    provider: 'anthropic',
    cost: { input: 0.015, output: 0.075 },
    capabilities: ['coding', 'advanced_reasoning', 'creative', 'all_tasks'],
    maxTokens: 200000
  }
};

async function classifyIntent(query, context = {}) {
  const classificationPrompt = `
    Classify this query into exactly one category:
    - SIMPLE_QA: Simple questions, facts, greetings
    - MODERATE_TASK: Summaries, basic analysis, formatting
    - COMPLEX_ANALYSIS: Deep analysis, comparisons, recommendations
    - CODING_TASK: Code generation, debugging, technical help
    - CREATIVE_WRITING: Marketing copy, creative content, storytelling
    
    Query: ${query}
    ${context.previousQueries ? `Previous context: ${context.previousQueries.join(', ')}` : ''}
    
    Respond with JSON: {"category": "CATEGORY", "confidence": 0.0-1.0, "reason": "brief explanation"}
  `;
  
  const response = await openai.chat.completions.create({
    model: 'gpt-4.1-nano', // Use cheapest model for classification
    messages: [{ role: 'user', content: classificationPrompt }],
    response_format: { type: 'json_object' },
    temperature: 0.1
  });
  
  return JSON.parse(response.choices[0].message.content);
}

function selectModel(classification, requiredCapabilities = []) {
  const categoryToModel = {
    'SIMPLE_QA': 'gpt-4.1-nano',
    'MODERATE_TASK': 'gpt-4.1-mini',
    'COMPLEX_ANALYSIS': 'claude-sonnet-4.8',
    'CODING_TASK': 'claude-opus-4.8',
    'CREATIVE_WRITING': 'gpt-4.1'
  };
  
  // Override if specific capabilities are required
  if (requiredCapabilities.includes('coding')) return 'claude-opus-4.8';
  if (requiredCapabilities.includes('long_context')) return 'claude-sonnet-4.8';
  
  const selectedModel = categoryToModel[classification.category];
  const costEstimate = MODELS[selectedModel].cost;
  
  return {
    model: selectedModel,
    provider: MODELS[selectedModel].provider,
    estimatedCost: costEstimate,
    reasoning: classification.reason,
    confidence: classification.confidence
  };
}

// Main execution
const input = $input.first().json;
const intent = await classifyIntent(input.query, input.context);
const selection = selectModel(intent, input.requiredCapabilities);

return [{
  json: {
    selectedModel: selection.model,
    provider: selection.provider,
    estimatedCostPer1k: selection.estimatedCost,
    classification: intent,
    optimization: {
      strategy: 'intent_based_routing',
      estimatedSavingsVsDefault: '60-80%'
    }
  }
}];

// LocalModelRouter.js
const axios = require('axios');

const OLLAMA_URL = $env.OLLAMA_URL || 'http://localhost:11434';

const LOCAL_MODELS = {
  'llama3.2': {
    description: 'Good for simple Q&A and classification',
    cost: 0, // Self-hosted
    latency: 'medium',
    quality: 'acceptable_for_simple'
  },
  'phi4': {
    description: 'Microsoft model, good balance',
    cost: 0,
    latency: 'medium',
    quality: 'good'
  },
  'qwen2.5': {
    description: 'Strong performance on many tasks',
    cost: 0,
    latency: 'medium',
    quality: 'very_good'
  }
};

async function routeToLocalOrCloud(query, complexity) {
  // Use local model for simple queries
  if (complexity === 'low' && query.length < 500) {
    try {
      const response = await axios.post(`${OLLAMA_URL}/api/generate`, {
        model: 'llama3.2',
        prompt: query,
        stream: false,
        options: {
          temperature: 0.7,
          num_predict: 500
        }
      });
      
      return {
        source: 'local',
        model: 'llama3.2',
        response: response.data.response,
        cost: 0,
        latency_ms: response.data.total_duration / 1000000
      };
    } catch (error) {
      // Fall back to cloud if local fails
      return { source: 'cloud_fallback', error: error.message };
    }
  }
  
  // Route to cloud for complex queries
  return { source: 'cloud', reason: 'high_complexity' };
}

const input = $input.first().json;
const result = await routeToLocalOrCloud(input.query, input.complexity);

return [{
  json: result
}];

┌─────────────────────────────────────────────────────────────────────────┐
│                  COST MONITORING ARCHITECTURE                            │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   n8n Workflows                                                         │
│     │                                                                   │
│     │  Each LLM call logs:                                              │
│     │  { workflow_id, model, tokens_in, tokens_out, cost, timestamp }   │
│     │                                                                   │
│     ▼                                                                   │
│   ┌──────────────────┐                                                 │
│   │  PostgreSQL      │                                                 │
│   │  ai_usage_logs   │                                                 │
│   └────────┬─────────┘                                                 │
│            │                                                           │
│            ▼                                                           │
│   ┌──────────────────┐                                                 │
│   │  Aggregated      │                                                 │
│   │  Metrics         │                                                 │
│   │  (Hourly/Daily)  │                                                 │
│   └────────┬─────────┘                                                 │
│            │                                                           │
│     ┌──────┴──────┬───────────────┐                                    │
│     │             │               │                                    │
│     ▼             ▼               ▼                                    │
│   ┌──────────┐ ┌──────────┐ ┌──────────┐                          │
│   │ Grafana  │ │ Alerting │ │  Budget  │                          │
│   │ Dashboard│ │  (Slack) │ │  Checks  │                          │
│   └──────────┘ └──────────┘ └──────────┘                          │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

-- Token usage tracking table
CREATE TABLE ai_token_usage (
    id SERIAL PRIMARY KEY,
    workflow_id VARCHAR(100) NOT NULL,
    execution_id VARCHAR(100),
    node_id VARCHAR(100),
    model VARCHAR(50) NOT NULL,
    provider VARCHAR(50) NOT NULL,
    tokens_input INTEGER NOT NULL,
    tokens_output INTEGER NOT NULL,
    cost_usd DECIMAL(10, 6) NOT NULL,
    cached BOOLEAN DEFAULT FALSE,
    cache_hit BOOLEAN DEFAULT FALSE,
    user_id VARCHAR(100),
    session_id VARCHAR(100),
    metadata JSONB,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Indexes for efficient querying
CREATE INDEX idx_token_usage_workflow_date ON ai_token_usage(workflow_id, created_at);
CREATE INDEX idx_token_usage_model ON ai_token_usage(model);
CREATE INDEX idx_token_usage_user ON ai_token_usage(user_id);
CREATE INDEX idx_token_usage_session ON ai_token_usage(session_id);

-- Aggregated daily stats
CREATE TABLE ai_usage_daily (
    id SERIAL PRIMARY KEY,
    workflow_id VARCHAR(100) NOT NULL,
    usage_date DATE NOT NULL,
    total_requests INTEGER DEFAULT 0,
    total_tokens_input BIGINT DEFAULT 0,
    total_tokens_output BIGINT DEFAULT 0,
    total_cost_usd DECIMAL(12, 4) DEFAULT 0,
    cache_hits INTEGER DEFAULT 0,
    cache_misses INTEGER DEFAULT 0,
    avg_response_time_ms INTEGER,
    UNIQUE(workflow_id, usage_date)
);

-- Budget thresholds
CREATE TABLE workflow_budgets (
    id SERIAL PRIMARY KEY,
    workflow_id VARCHAR(100) UNIQUE NOT NULL,
    daily_limit_usd DECIMAL(10, 2) DEFAULT 50.00,
    hourly_limit_usd DECIMAL(10, 2) DEFAULT 10.00,
    per_request_limit_usd DECIMAL(10, 4) DEFAULT 0.50,
    alert_threshold_pct INTEGER DEFAULT 80,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

// LogTokenUsage.js - Execute after each LLM call
const { Pool } = require('pg');

const pool = new Pool({
  connectionString: $env.DATABASE_URL
});

async function logUsage(usage) {
  const query = `
    INSERT INTO ai_token_usage 
    (workflow_id, execution_id, node_id, model, provider, 
     tokens_input, tokens_output, cost_usd, cached, cache_hit, 
     user_id, session_id, metadata)
    VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
  `;
  
  const values = [
    usage.workflowId,
    usage.executionId,
    usage.nodeId,
    usage.model,
    usage.provider,
    usage.tokensInput,
    usage.tokensOutput,
    usage.costUsd,
    usage.cached || false,
    usage.cacheHit || false,
    usage.userId,
    usage.sessionId,
    JSON.stringify(usage.metadata || {})
  ];
  
  await pool.query(query, values);
}

// Get usage from previous node
const llmResponse = $input.first().json;

await logUsage({
  workflowId: $workflow.id,
  executionId: $execution.id,
  nodeId: 'ai_agent_node',
  model: llmResponse.model,
  provider: llmResponse.provider,
  tokensInput: llmResponse.usage.prompt_tokens,
  tokensOutput: llmResponse.usage.completion_tokens,
  costUsd: llmResponse.calculatedCost,
  cached: llmResponse.fromCache || false,
  cacheHit: llmResponse.cacheHit || false,
  userId: $execution.customData?.userId,
  sessionId: $execution.customData?.sessionId,
  metadata: {
    intent: llmResponse.classification,
    optimizationApplied: llmResponse.optimizationStrategy
  }
});

return [{
  json: { logged: true, timestamp: new Date().toISOString() }
}];

// CostAlerting.js
const { Pool } = require('pg');
const axios = require('axios');

const pool = new Pool({ connectionString: $env.DATABASE_URL });
const SLACK_WEBHOOK = $env.SLACK_WEBHOOK_URL;

async function checkBudgetsAndAlert() {
  const alerts = [];
  
  // Check workflows approaching limits
  const budgetChecks = await pool.query(`
    SELECT 
      wb.workflow_id,
      wb.daily_limit_usd,
      wb.alert_threshold_pct,
      COALESCE(ud.total_cost_usd, 0) as current_spend,
      ROUND((COALESCE(ud.total_cost_usd, 0) / wb.daily_limit_usd) * 100, 2) as pct_used
    FROM workflow_budgets wb
    LEFT JOIN ai_usage_daily ud ON wb.workflow_id = ud.workflow_id 
      AND ud.usage_date = CURRENT_DATE
    WHERE wb.daily_limit_usd > 0
  `);
  
  for (const row of budgetChecks.rows) {
    if (row.pct_used >= row.alert_threshold_pct) {
      alerts.push({
        workflowId: row.workflow_id,
        severity: row.pct_used >= 100 ? 'CRITICAL' : 'WARNING',
        currentSpend: row.current_spend,
        limit: row.daily_limit_usd,
        percentUsed: row.pct_used,
        message: row.pct_used >= 100 
          ? `Budget EXCEEDED for ${row.workflow_id}: $${row.current_spend} / $${row.daily_limit_usd}`
          : `Budget threshold reached for ${row.workflow_id}: ${row.pct_used}% of daily limit`
      });
    }
  }
  
  // Send alerts
  for (const alert of alerts) {
    await axios.post(SLACK_WEBHOOK, {
      text: `🚨 AI Cost Alert: ${alert.severity}`,
      attachments: [{
        color: alert.severity === 'CRITICAL' ? 'danger' : 'warning',
        fields: [
          { title: 'Workflow', value: alert.workflowId, short: true },
          { title: 'Spend', value: `$${alert.currentSpend}`, short: true },
          { title: 'Limit', value: `$${alert.limit}`, short: true },
          { title: 'Used', value: `${alert.percentUsed}%`, short: true }
        ]
      }]
    });
  }
  
  return { alertsSent: alerts.length, alerts };
}

const result = await checkBudgetsAndAlert();

return [{
  json: result
}];

// BatchProcessor.js
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: $env.OPENAI_API_KEY });

async function processBatch(requests) {
  // Single API call for up to 20 requests
  const batchPrompt = `
    Process the following ${requests.length} requests. 
    Return a JSON array with responses in the same order.
    
    Requests:
    ${requests.map((r, i) => `[${i}] ${r.text}`).join('\n')}
    
    Format: {"responses": [{"index": 0, "response": "..."}, ...]}
  `;
  
  const response = await openai.chat.completions.create({
    model: 'gpt-4.1',
    messages: [{ role: 'user', content: batchPrompt }],
    response_format: { type: 'json_object' },
    max_tokens: 4000
  });
  
  const results = JSON.parse(response.choices[0].message.content).responses;
  
  // Cost per request is approximately:
  // Single calls: 20 × $0.05 = $1.00
  // Batch: ~$0.15 (70% savings)
  
  return {
    results,
    totalTokens: response.usage.total_tokens,
    cost: calculateCost(response.usage),
    savingsVsIndividual: '70%'
  };
}

// Queue requests for batching
const BATCH_SIZE = 20;
const BATCH_INTERVAL = 5000; // 5 seconds

const requestQueue = [];

// Add requests to queue
requestQueue.push(...$input.all().map(item => item.json));

// Process when queue fills or time expires
if (requestQueue.length >= BATCH_SIZE) {
  const batch = requestQueue.splice(0, BATCH_SIZE);
  const result = await processBatch(batch);
  
  return [{
    json: {
      batched: true,
      count: batch.length,
      ...result
    }
  }];
}

return [{
  json: {
    queued: true,
    queueLength: requestQueue.length,
    willProcessWhen: requestQueue.length >= BATCH_SIZE ? 'now' : `in ${BATCH_INTERVAL}ms`
  }
}];

// KVCacheOptimizer.js
const { Redis } = require('ioredis');
const redis = new Redis($env.REDIS_URL);

const KV_CACHE_TTL = 60 * 60 * 2; // 2 hours

async function getCachedKV(contextHash) {
  const cached = await redis.get(`kv_cache:${contextHash}`);
  if (cached) {
    return {
      hit: true,
      kvCache: JSON.parse(cached),
      tokensSaved: '~30% of context tokens'
    };
  }
  return { hit: false };
}

async function storeKV(contextHash, kvData, tokenCount) {
  await redis.setex(
    `kv_cache:${contextHash}`,
    KV_CACHE_TTL,
    JSON.stringify({ kvData, tokenCount, storedAt: Date.now() })
  );
}

// Hash the system prompt + available tools context
const contextHash = require('crypto')
  .createHash('md5')
  .update($input.first().json.systemPrompt + $input.first().json.tools)
  .digest('hex');

const kvResult = await getCachedKV(contextHash);

if (kvResult.hit) {
  // Use cached KV, only send new user message
  return [{
    json: {
      optimization: 'kv_cache_hit',
      cachedContext: true,
      tokensToSend: $input.first().json.userMessage.length / 4, // ~tokens
      estimatedSavings: kvResult.tokensSaved
    }
  }];
}

return [{
  json: {
    optimization: 'kv_cache_miss',
    contextHash,
    cacheForFuture: true
  }
}];

// PriorityQueue.js
const { Queue } = require('bullmq');
const { Redis } = require('ioredis');

const redis = new Redis($env.REDIS_URL);

const PRIORITY_LEVELS = {
  CRITICAL: 1,    // Revenue-impacting, real-time needs
  HIGH: 2,        // Customer-facing, time-sensitive
  NORMAL: 3,      // Standard operations
  LOW: 4,         // Background tasks, analytics
  BATCH: 5        // Deferred processing, reports
};

const COST_THRESHOLDS = {
  CRITICAL: 1.00,  // Up to $1 per request
  HIGH: 0.50,
  NORMAL: 0.10,
  LOW: 0.05,
  BATCH: 0.02
};

const MODEL_BY_PRIORITY = {
  CRITICAL: 'claude-opus-4.8',
  HIGH: 'claude-sonnet-4.8',
  NORMAL: 'gpt-4.1',
  LOW: 'gpt-4.1-mini',
  BATCH: 'gpt-4.1-nano'
};

async function enqueueRequest(request) {
  const priority = determinePriority(request);
  const costLimit = COST_THRESHOLDS[priority];
  const model = MODEL_BY_PRIORITY[priority];
  
  // Check current spending for this priority level
  const currentSpend = await redis.get(`spend:${priority}:${new Date().toISOString().split('T')[0]}`) || 0;
  
  if (parseFloat(currentSpend) > costLimit * 100) { // Budget exceeded
    // Degrade to lower priority/model
    return {
      queued: true,
      originalPriority: priority,
      actualPriority: 'LOW',
      model: MODEL_BY_PRIORITY.LOW,
      reason: 'daily_budget_exceeded'
    };
  }
  
  return {
    queued: true,
    priority,
    model,
    costLimit,
    estimatedWait: priority <= 2 ? '< 5 seconds' : '< 2 minutes'
  };
}

function determinePriority(request) {
  if (request.urgent || request.revenue_impact) return 'CRITICAL';
  if (request.customer_facing) return 'HIGH';
  if (request.background) return 'LOW';
  if (request.batch) return 'BATCH';
  return 'NORMAL';
}

const request = $input.first().json;
const result = await enqueueRequest(request);

return [{
  json: result
}];

Priority: CRITICAL
Effort: Low
Impact: High

Tasks:
□ Set up ai_token_usage table
□ Add logging to all existing AI nodes
□ Create basic Grafana dashboard
□ Configure daily cost reports via email

Expected Outcome:
- 100% visibility into token consumption
- Baseline cost metrics established
- Team awareness of spending patterns

Priority: HIGH
Effort: Medium
Impact: High

Tasks:
□ Implement budget checks on all AI workflows
□ Set up alerting thresholds (80%, 100%)
□ Create graceful degradation paths
□ Configure fallback to cheaper models

Expected Outcome:
- No more runaway costs
- Automatic protection against budget overruns
- Clear escalation paths

Priority: MEDIUM
Effort: Medium
Impact: Very High

Tasks:
□ Implement semantic caching
□ Deploy context optimization (summarization)
□ Set up smart model routing
□ Configure response truncation
□ Implement tool selection optimization

Expected Outcome:
- 40-60% cost reduction
- Improved response times
- Better user experience

Priority: LOW
Effort: High
Impact: Medium-High

Tasks:
□ Deploy local models for simple queries
□ Implement batch processing
□ Set up KV caching
□ Configure priority queuing
□ Build automated cost optimization recommendations

Expected Outcome:
- 60-75% total cost reduction
- Enterprise-grade cost governance
- Competitive advantage through efficient AI usage

-- Daily cost report
SELECT 
  DATE(created_at) as date,
  workflow_id,
  COUNT(*) as requests,
  SUM(tokens_input) as input_tokens,
  SUM(tokens_output) as output_tokens,
  SUM(cost_usd) as total_cost,
  AVG(cost_usd) as avg_cost_per_request,
  SUM(CASE WHEN cache_hit THEN 1 ELSE 0 END) as cache_hits,
  ROUND(100.0 * SUM(CASE WHEN cache_hit THEN 1 ELSE 0 END) / COUNT(*), 2) as cache_hit_rate
FROM ai_token_usage
WHERE created_at >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY DATE(created_at), workflow_id
ORDER BY date DESC, total_cost DESC;