Console Timing Metrics & Telemetry Plan

Details

Raw Data

{
  "id": "367edd14-213f-4b5d-bc2f-cedfd2143ee3",
  "type": "general",
  "status": "completed",
  "title": "Console Timing Metrics & Telemetry Plan",
  "description": "Implementation plan for measuring prompt-to-kontask turn times",
  "context": {
    "output": "<div style=\"background:#1e1e1e;color:#e0e0e0;padding:20px;border-radius:8px;font-family:'Segoe UI','Helvetica Neue',sans-serif;\">\n<h2 style=\"color:#fff;margin:0 0 16px 0;\">⏱️ Console Timing Metrics & Telemetry Plan</h2>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\">\n<h3 style=\"color:#e06c75;margin:0 0 12px 0;\">The Problem</h3>\n<p style=\"margin:0;line-height:1.6;\">Currently we have <strong style=\"color:#fff;\">zero visibility</strong> into where time is spent between typing a prompt and seeing a kontask. The experience <em>feels</em> slow but we don't know if it's:</p>\n<ul style=\"margin:8px 0 0 0;padding-left:20px;line-height:1.8;\">\n<li>Network latency (browser ↔ konui)</li>\n<li>Session lookup/image processing (konui)</li>\n<li>Claude spawn time (konsole)</li>\n<li>Claude thinking time (anthropic API)</li>\n<li>Kontask creation overhead</li>\n</ul>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\">\n<h3 style=\"color:#56b6c2;margin:0 0 12px 0;\">Request Lifecycle (Current)</h3>\n<pre style=\"background:#1e1e1e;padding:12px;border-radius:4px;overflow-x:auto;font-size:0.85em;line-height:1.6;\">\n<span style=\"color:#98c379;\">Browser</span>                    <span style=\"color:#61afef;\">Konui</span>                     <span style=\"color:#c678dd;\">Konsole</span>                   <span style=\"color:#e5c07b;\">Claude</span>\n   │                          │                          │                          │\n   │─── POST /stream ────────▶│                          │                          │\n   │                          │─── lookup session ──────▶│                          │\n   │                          │◀── session ID ───────────│                          │\n   │                          │─── POST /prompt ─────────▶│                          │\n   │                          │                          │─── spawn claude ────────▶│\n   │                          │                          │◀── first event ──────────│\n   │◀─────── SSE events ──────│◀── SSE events ───────────│◀── stream output ────────│\n   │                          │                          │                          │\n   │◀─── kontext event ───────│◀── kontext ──────────────│◀── done ─────────────────│\n   │                          │                          │                          │\n   ▼ kontask visible          ▼                          ▼                          ▼\n</pre>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\">\n<h3 style=\"color:#98c379;margin:0 0 12px 0;\">📊 Proposed Metrics</h3>\n\n<table style=\"width:100%;border-collapse:collapse;margin-top:8px;\">\n<tr style=\"border-bottom:1px solid #444;\">\n<th style=\"text-align:left;padding:8px;color:#fff;\">Metric</th>\n<th style=\"text-align:left;padding:8px;color:#fff;\">From → To</th>\n<th style=\"text-align:left;padding:8px;color:#fff;\">Target</th>\n</tr>\n<tr style=\"border-bottom:1px solid #333;\">\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_network</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">Browser send → Server receive</td>\n<td style=\"padding:8px;color:#98c379;\">&lt;100ms</td>\n</tr>\n<tr style=\"border-bottom:1px solid #333;\">\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_prep</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">Server receive → Konsole POST</td>\n<td style=\"padding:8px;color:#98c379;\">&lt;200ms</td>\n</tr>\n<tr style=\"border-bottom:1px solid #333;\">\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_spawn</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">Konsole receive → First Claude event</td>\n<td style=\"padding:8px;color:#98c379;\">&lt;500ms</td>\n</tr>\n<tr style=\"border-bottom:1px solid #333;\">\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_work</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">First event → Done event</td>\n<td style=\"padding:8px;color:#9ca3af;\">varies</td>\n</tr>\n<tr style=\"border-bottom:1px solid #333;\">\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_total</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">Browser send → Kontask visible</td>\n<td style=\"padding:8px;color:#9ca3af;\">varies</td>\n</tr>\n<tr>\n<td style=\"padding:8px;\"><code style=\"color:#e5c07b;\">t_ttfb</code></td>\n<td style=\"padding:8px;color:#9ca3af;\">Browser send → First SSE event</td>\n<td style=\"padding:8px;color:#98c379;\">&lt;1s</td>\n</tr>\n</table>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\">\n<h3 style=\"color:#61afef;margin:0 0 12px 0;\">🏗️ Implementation Plan</h3>\n\n<div style=\"background:#1e1e1e;padding:12px;border-radius:4px;margin:12px 0;border-left:3px solid #98c379;\">\n<h4 style=\"color:#98c379;margin:0 0 8px 0;\">Phase 1: Instrument Existing Events</h4>\n<p style=\"margin:0 0 8px 0;color:#9ca3af;\">Add timestamps to existing SSE events without new infrastructure.</p>\n<ul style=\"margin:0;padding-left:20px;line-height:1.6;font-size:0.9em;\">\n<li><code style=\"color:#e5c07b;\">router.ts</code>: Add <code>serverReceivedAt</code> to stream response</li>\n<li><code style=\"color:#e5c07b;\">stream.ts</code>: Add <code>promptReceivedAt</code>, <code>firstEventAt</code>, <code>completedAt</code> to kontext</li>\n<li><code style=\"color:#e5c07b;\">console-streaming.ts</code>: Calculate deltas client-side, display in status bar</li>\n</ul>\n</div>\n\n<div style=\"background:#1e1e1e;padding:12px;border-radius:4px;margin:12px 0;border-left:3px solid #61afef;\">\n<h4 style=\"color:#61afef;margin:0 0 8px 0;\">Phase 2: Metrics Storage</h4>\n<p style=\"margin:0 0 8px 0;color:#9ca3af;\">Persist metrics for analysis and trending.</p>\n<ul style=\"margin:0;padding-left:20px;line-height:1.6;font-size:0.9em;\">\n<li>New file: <code style=\"color:#e5c07b;\">/konnectvol/konui/data/metrics.jsonl</code></li>\n<li>Append timing record per request (requestId, all t_* values)</li>\n<li>Include prompt length, has images, output size</li>\n</ul>\n</div>\n\n<div style=\"background:#1e1e1e;padding:12px;border-radius:4px;margin:12px 0;border-left:3px solid #c678dd;\">\n<h4 style=\"color:#c678dd;margin:0 0 8px 0;\">Phase 3: Real-time Display</h4>\n<p style=\"margin:0 0 8px 0;color:#9ca3af;\">Show timing in the console UI.</p>\n<ul style=\"margin:0;padding-left:20px;line-height:1.6;font-size:0.9em;\">\n<li>Status bar: <code>Working (3.2s) • TTFB: 0.8s • Claude: 2.4s</code></li>\n<li>On completion: <code>Done in 4.1s (prep: 0.2s, work: 3.9s)</code></li>\n<li>Hover tooltip with full breakdown</li>\n</ul>\n</div>\n\n<div style=\"background:#1e1e1e;padding:12px;border-radius:4px;margin:12px 0;border-left:3px solid #e5c07b;\">\n<h4 style=\"color:#e5c07b;margin:0 0 8px 0;\">Phase 4: Dashboard View</h4>\n<p style=\"margin:0 0 8px 0;color:#9ca3af;\">Add a metrics page for historical analysis.</p>\n<ul style=\"margin:0;padding-left:20px;line-height:1.6;font-size:0.9em;\">\n<li>Route: <code>/metrics</code> or tab on Reports page</li>\n<li>Charts: p50/p90/p99 over time</li>\n<li>Breakdowns by: prompt length, hour of day, with/without images</li>\n<li>Identify slow requests (click to see full kontext)</li>\n</ul>\n</div>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\">\n<h3 style=\"color:#e5c07b;margin:0 0 12px 0;\">🎯 Quick Win (Can Do Now)</h3>\n<p style=\"margin:0;line-height:1.6;\">Add timing to the <strong>kontext</strong> object that already gets sent to browser:</p>\n<pre style=\"background:#1e1e1e;padding:12px;border-radius:4px;overflow-x:auto;font-size:0.85em;margin-top:8px;\">\n<span style=\"color:#7f848e;\">// In kontext (already exists):</span>\n<span style=\"color:#e5c07b;\">durationMs</span>: <span style=\"color:#d19a66;\">4100</span>,  <span style=\"color:#7f848e;\">// Total Claude work time</span>\n\n<span style=\"color:#7f848e;\">// Add:</span>\n<span style=\"color:#e5c07b;\">timing</span>: {\n  <span style=\"color:#98c379;\">\"promptReceivedAt\"</span>: <span style=\"color:#98c379;\">\"2026-01-03T12:00:00.000Z\"</span>,\n  <span style=\"color:#98c379;\">\"firstEventAt\"</span>:     <span style=\"color:#98c379;\">\"2026-01-03T12:00:00.850Z\"</span>,\n  <span style=\"color:#98c379;\">\"completedAt\"</span>:      <span style=\"color:#98c379;\">\"2026-01-03T12:00:04.100Z\"</span>,\n  <span style=\"color:#98c379;\">\"spawnMs\"</span>: <span style=\"color:#d19a66;\">850</span>,\n  <span style=\"color:#98c379;\">\"workMs\"</span>: <span style=\"color:#d19a66;\">3250</span>\n}\n</pre>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;\">\n<h4 style=\"margin:0 0 8px 0;color:#fff;\">What's Next?</h4>\n<p style=\"color:#9ca3af;margin:0;\">Choose a direction:</p>\n</div>\n</div>",
    "choices": [
      {
        "label": "Start Phase 1",
        "value": "Implement Phase 1: Add timing to kontext object in konsole and display in browser status bar",
        "primary": true
      },
      {
        "label": "Quick win only",
        "value": "Just add timing to kontext object in konsole/src/stream.ts - minimal change to prove the concept"
      },
      {
        "label": "Full implementation",
        "value": "Implement all 4 phases as a complete telemetry system with storage and dashboard"
      },
      {
        "label": "Investigate first",
        "value": "Before adding metrics, let me do some manual timing tests to identify the actual bottleneck"
      }
    ],
    "requestedAt": "2026-01-03T12:05:00Z",
    "requestId": "4c7b4ec0-6b18-424a-9556-b4336cd53849"
  },
  "createdBy": "claude",
  "createdAt": "2026-01-03T07:15:09.388Z",
  "updatedAt": "2026-01-03T07:15:09.544Z",
  "requestId": "4c7b4ec0-6b18-424a-9556-b4336cd53849",
  "scope": "vibetools",
  "tags": [
    "console",
    "performance",
    "telemetry"
  ],
  "targetUser": "claude"
}