Hacker News[forum]
Everyone is building LLM routers, we deprecated ours
Why it matters
A gateway vendor killed its own LLM router after real-world use, undercutting the cost-saving promise driving the current model-routing hype.
The brief
AI summary · 5 points
- Manifest launched its LLM router in March, deprecated it in June, and shut it down for good on September 1st.
- The router sorted each request into four complexity tiers — simple, standard, complex, and reasoning — to reduce inference costs.
- After four months across 7,000 cloud users, the company reports mixed results and a wave of GitHub issues and discussions.
- The company argues task complexity cannot be deduced from the prompt alone, since the prompt is only the trigger.
- Manifest now contends that sticking to a single battle-tested model is best for most use cases.
“Everyone is building LLM routers, we deprecated ours Product LLM Gateway Auto-fix New Observability Solutions For AI builders For teams For platforms…”Read the original