Adding AI to an Existing App Without Rewriting It

Most teams who want to add AI to an existing app start from the wrong end. They pick a model, then look for something to do with it. The projects that ship do the opposite: they find one annoying, well-defined job and ask whether a model can do it well enough.

Pick a job with a boring success criterion

Good first candidates share a trait — you can tell immediately whether the output is right. Summarising a support thread, tagging a product, drafting a reply, extracting fields from a document. Bad first candidates are the ones where nobody agrees what "good" means.

If you cannot describe success in one sentence, pick something else first.

Put the model behind your own interface

Call the provider from one service class, not from controllers scattered across the app. Everything else should ask for a summary, not for a chat completion. This matters more than it sounds: providers change pricing, deprecate models and have outages. When that happens you want to edit one file.

It also makes the feature testable. Your test suite should never make a real API call.

Treat it as an unreliable network call, because it is

Model APIs are slow, occasionally fail and sometimes return something you did not expect. So:

• Queue it. Do not block a web request on a call that can take thirty seconds.
• Set a timeout and decide what the product does when it fires.
• Validate the output before you store it. If you asked for JSON, parse it and check the shape.
• Always have a fallback. A missing summary should degrade the page, not break it.

Keep humans in the loop at first

Ship the first version as a suggestion rather than an action. Draft the reply but let a person send it. Propose the tags but let someone confirm them. You get the value immediately, and you get a stream of corrections that tells you exactly how good the feature actually is.

Watch the cost from day one

Log the tokens and the spend per feature, not just in total. It is very easy to ship something that works beautifully and quietly costs more per user than you charge. Cache aggressively — identical inputs should not be paid for twice.

Measure whether it helped

Before you build, write down the number you expect to move: time to first response, support tickets closed per day, items tagged per hour. If the number does not move after launch, the honest conclusion is that the feature was interesting rather than useful. Remove it and try the next job.

We cover this full workflow — service layer, queues, validation, cost control — in the AI automation course, and several of the products in our store ship with a provider-agnostic AI service already wired up.