Cutting LLM Cost and Latency on Joye with Redis
What actually helped once Claude was running in production for Teams.
Shipping Claude for Joye's coaching flows was straightforward at first — send a prompt, get a response, show it in Teams. Production was different. Users have long, multi-turn conversations. Each message needs context from earlier in the session, and if you resend the full history every time, latency goes up and the API bill grows fast.
We started storing session state in Redis — summarized history, user context, things the model already knows. I track token usage on the server and trim prompts before they go out. We cache structured context, not final answers, so responses stay accurate. When something gets slow or expensive, we can see which flow caused it instead of guessing.
None of this is exciting to demo, but it's what keeps the feature running day to day. The LLM is just another dependency — it needs session boundaries, caching where it makes sense, and visibility into cost. Redis did more for us than any prompt tweak ever did.