AgentScope — Browser DevTools Panel That Visualizes Every LLM API Call Your App Makes in Real Time
Your AI agent is burning $200/day in tokens and you have no idea why. AgentScope intercepts every OpenAI, Anthropic, and Gemini call from your web app, visualizes token flow, context size, and cost per call in a DevTools panel, and tells you exactly which prompt is the budget vampire.
Difficulty
advanced
Category
Developer Tools
Market Demand
Very High
Revenue Score
8/10
Platform
Browser Extension
Vibe Code Friendly
No
Hackathon Score
🏆 8/10
Validated by Real Pain
— sourced from real community discussions
Developers building LLM apps and AI agents cannot observe how their code consumes tokens and context in real time, leading to surprise API bills and wasted hours debugging inefficient prompt chains through manual logging.
What is it?
Developers building LLM-powered apps have zero observability into how their agents actually consume context and tokens until the invoice arrives. AgentScope is a Chrome DevTools extension that intercepts XHR and fetch calls to OpenAI, Anthropic, and Gemini APIs, parses request and response payloads, and renders a real-time panel showing: tokens in/out per call, cumulative session cost, context window fill percentage, and a call tree visualizing agent decision chains. It requires zero code changes in the target app — just open DevTools. Buildable now using Chrome DevTools Protocol, chrome.devtools API, and the tiktoken library for client-side token counting.
Why now?
OpenAI, Anthropic, and Google have all raised API prices in early 2026 while vibe-coding tools like Cursor and Lovable have pushed thousands of non-expert developers into building LLM apps with zero cost visibility.
- ▸Zero-config XHR/fetch interception of OpenAI, Anthropic, and Gemini API calls from any web app (Implementation note: chrome.devtools.network.onRequestFinished listener)
- ▸Real-time token counter per call using tiktoken-js with cumulative session cost in dollars
- ▸Context window fill percentage bar showing how close each call is to the model limit
- ▸Call tree visualization linking chained agent requests so multi-step loops are visible as a graph
Target Audience
Developers building LLM apps and AI agents with Cursor, Vercel AI SDK, LangChain JS — roughly 500k developers actively building AI-powered web apps.
Example Use Case
Chen is debugging why his LangChain agent costs $40 per user session. AgentScope shows him that a retrieval step is injecting 18,000 tokens of irrelevant context on every loop iteration. He fixes the retrieval filter and cuts costs by 70% in 20 minutes.
User Stories
- ▸As an AI app developer, I want to see token usage per API call in real time so that I can identify which prompts are causing runaway costs.
- ▸As a solo founder building an AI agent, I want a call tree showing how my agent chains requests so that I can debug infinite loops without adding logging code.
- ▸As an engineering lead, I want to set a cost alert threshold so that I get notified before a runaway agent burns through my monthly budget.
Done When
- ✓Interception: done when opening AgentScope panel on any app making OpenAI calls shows a live call entry within 1 second of the request completing.
- ✓Token count: done when the token count shown for a call matches the usage field returned in the OpenAI API response within 5% margin.
- ✓Call tree: done when two chained agent calls appear as parent-child nodes in the D3 visualization.
- ✓Cost display: done when cumulative session cost updates in real time and matches the sum of individual call cost estimates.
Is it worth building?
$29/month x 100 dev teams = $2,900 MRR at month 3. $29/month x 400 teams = $11,600 MRR at month 8. Math: 2% conversion from 5,000 installs via Hacker News and dev Twitter.
Unit Economics
CAC: $15 via Hacker News and dev Twitter organic. LTV: $348 (12 months at $29/month). Payback: 1 month. Gross margin: 93%.
Business Model
SaaS subscription
Monetization Path
Free: 7-day history, single session. Pro: $29/month for 30-day history, team sharing, and cost alerts.
Revenue Timeline
First dollar: week 3 via beta Pro upgrade. $1k MRR: month 2. $5k MRR: month 5. $10k MRR: month 9.
Estimated Monthly Cost
Vercel for Pro dashboard API: $20, Supabase for history storage: $25, Stripe: $15, no AI API costs. Total: ~$60/month.
Profit Potential
Full-time viable at $8k–$20k MRR targeting AI-first dev teams.
Scalability
High — add Vercel AI SDK auto-detection, team workspaces, cost budget alerts, export to CSV for finance reviews.
Success Metrics
1,000 installs week 1, 80 Pro upgrades month 1, 4.7+ Chrome Web Store rating.
Launch & Validation Plan
Post a Loom of the extension catching a $40 agent session on Hacker News Show HN before building the Pro tier.
Customer Acquisition Strategy
First customer: DM 15 developers who have posted about LLM cost issues on Twitter/X offering lifetime Pro access for a testimonial. Then: Hacker News Show HN launch, r/LocalLLaMA, r/ChatGPTCoding, ProductHunt, write a dev.to post titled 'I found out my agent was wasting $200/day in 10 minutes'.
What's the competition?
Competition Level
Low
Similar Products
LangSmith (LangChain-only, requires SDK wrapping), Helicone (proxy-based, requires code changes), OpenAI usage dashboard (no real-time call-level visibility). Gap: zero-config, any framework, in-browser DevTools panel.
Competitive Advantage
LangSmith requires SDK instrumentation and is LangChain-only. Helicone requires a proxy. AgentScope works on any app with zero code changes — just open DevTools.
Regulatory Risks
Low regulatory risk. Must clearly disclose in Web Store listing that the extension reads API request/response payloads which may contain user data. GDPR note: session history stored server-side for Pro requires data deletion endpoint.
What's the roadmap?
Feature Roadmap
V1 (launch): intercept all 3 providers, token count, cost estimate, call tree. V2 (month 2-3): cost alerts, 30-day history, team session sharing. V3 (month 4+): Node.js proxy mode for server-side calls, export to CSV.
Milestone Plan
Phase 1 (Week 1-2): interceptor, token counter, cost estimator, basic panel UI ship. Phase 2 (Week 3-4): D3 call tree, LemonSqueezy billing, Chrome Web Store submission. Phase 3 (Month 2): 1k installs, 80 Pro upgrades, Anthropic and Gemini parsers refined.
How do you build it?
Tech Stack
Chrome Extension MV3, Chrome DevTools API, TypeScript, React for DevTools panel UI, tiktoken-js for token counting, D3.js for call tree visualization — build with Cursor for intercept logic, v0 for panel UI
Suggested Frameworks
Chrome DevTools Protocol, tiktoken-js, D3.js
Time to Ship
3 weeks
Required Skills
Chrome DevTools API, TypeScript, XHR/fetch interception, React, D3.js basics.
Resources
Chrome DevTools Extensions docs, developer.chrome.com/docs/extensions/mv3, tiktoken GitHub, Anthropic and OpenAI API response schemas.
MVP Scope
src/devtools/panel/App.tsx (main DevTools panel UI), src/devtools/panel/CallTree.tsx (D3 call tree), src/devtools/background/interceptor.ts (network request listener), src/devtools/background/tokenCounter.ts (tiktoken-js wrapper), src/devtools/background/costEstimator.ts (cost per model lookup table), src/devtools/index.html (panel entry), src/devtools/devtools.html (DevTools page registration), manifest.json (MV3 with devtools permission), src/lib/apiSchemas.ts (OpenAI/Anthropic/Gemini response parsers), README.md
Core User Journey
Install extension -> open DevTools on LLM app -> AgentScope panel shows live call list -> click a call to see token breakdown -> identify the wasteful call -> fix and verify cost drops.
Architecture Pattern
DevTools panel opens -> background script registers chrome.devtools.network.onRequestFinished -> filters for LLM API hostnames -> parses request/response JSON -> tokenCounter counts tokens -> costEstimator looks up model pricing -> events streamed to React panel via chrome.runtime.sendMessage -> call tree rendered in D3.
Data Model
Session has many ApiCalls. ApiCall has: model, tokensIn, tokensOut, estimatedCost, requestPayload hash, responsePayload hash, timestamp, parentCallId. User has many Sessions.
Integration Points
Chrome DevTools API for network interception, tiktoken-js for token counting, Supabase for Pro history storage, LemonSqueezy for billing, OpenAI and Anthropic pricing tables embedded as static JSON.
V1 Scope Boundaries
V1 excludes: mobile debugging, non-browser apps (Node.js server-side calls), custom model endpoints, team workspaces, export to external observability platforms.
Success Definition
A paying developer discovers and fixes a token waste bug in their production AI app using only AgentScope, without any founder help or documentation.
Challenges
HTTPS interception via DevTools only exposes request headers and bodies if the app is running on localhost or the user accepts the DevTools connection — production apps behind HTTPS with CORS will not expose full payloads without a proxy. Must be honest about this limitation in onboarding. Distribution: dev tool Chrome extensions live or die by a viral Hacker News post.
Avoid These Pitfalls
Do not promise full production HTTPS payload inspection — DevTools network API has CORS limits that block response bodies from cross-origin requests in some configurations. Do not store raw request payloads server-side — only hashes and metadata. Getting first 1,000 installs takes longer than building — budget 3x dev time on Hacker News and Twitter seeding.
Security Requirements
No raw request payloads stored server-side — only metadata hashes. All Pro data in Supabase with RLS per user. Rate limit history API to 60 req/min. GDPR: data deletion endpoint required for Pro accounts.
Infrastructure Plan
Chrome Web Store for distribution, Vercel for Pro history API, Supabase for session storage, LemonSqueezy for billing, GitHub Actions for build and publish CI.
Performance Targets
Panel renders new call entry within 200ms of request completion. Token count computed in under 50ms client-side. D3 tree renders up to 50 nodes without lag.
Go-Live Checklist
- ☐Security audit complete.
- ☐Payment flow tested end-to-end.
- ☐Error tracking (Sentry) live.
- ☐Monitoring dashboard configured.
- ☐Custom domain set up with SSL.
- ☐Privacy policy and terms published.
- ☐10+ beta developers signed off.
- ☐Rollback plan documented.
- ☐Launch post drafted for Hacker News Show HN.
First Run Experience
On first run: panel shows a demo session with 5 pre-recorded API calls from a sample LangChain agent including token counts and a call tree. User can immediately: click any demo call to see the full token breakdown and cost estimate. No manual config required: demo data loads automatically, no API keys or accounts needed.
How to build it, step by step
1. Define ApiCall schema in src/lib/types.ts with all fields: model, tokensIn, tokensOut, cost, parentCallId, timestamp. 2. Scaffold Chrome MV3 extension with CRXJS Vite and TypeScript, register devtools.html. 3. Implement network interceptor in background using chrome.devtools.network.onRequestFinished filtering for api.openai.com, api.anthropic.com, generativelanguage.googleapis.com. 4. Build apiSchemas.ts parsers for OpenAI chat completion, Anthropic messages, and Gemini generateContent response shapes. 5. Integrate tiktoken-js in tokenCounter.ts to count tokens from request messages array. 6. Build costEstimator.ts with static pricing table per model updated to current month pricing. 7. Build React DevTools panel with call list, selected call detail view, and cumulative session cost header. 8. Build D3 call tree in CallTree.tsx linking calls by parentCallId field injected via a request header the extension adds. 9. Wire LemonSqueezy license check gating 30-day history and cost alert features behind Pro. 10. Verify: run a LangChain JS app in localhost, open AgentScope panel, trigger 3 agent calls, and confirm token counts and costs appear correctly in real time.
Generated
May 14, 2026
Model
claude-sonnet-4-6
Disclaimer: Ideas on this site are AI-generated and may contain inaccuracies. Revenue estimates, market demand figures, and financial projections are illustrative assumptions only — not financial advice. Do your own research before making any business or investment decisions. Technology availability, pricing, and market conditions change rapidly; always verify details independently.