LLM Cost and Latency Observability
Know what every feature costs per user before the invoice tells you.
Beginner6–12 hours estimatedEvaluation
The problem
LLM spend is invisible until the monthly bill, and by then you cannot tell which feature caused it. This is unglamorous, takes a day, and is the project most likely to save a real company real money.
Who uses it: Any team running LLM features in production without per-feature cost attribution.
Before you start
- Python or TypeScript
- An application making model calls
Stack
Any languageA wrapper around your model clientPostgres or ClickHouseGrafana or a simple dashboard