llm

Four options with genuinely different costs, and the requirement is usually narrower than it first sounds. Establish whether it is about the network path, the data residency, or the provider's retention terms, because those need different answers.

You have added a dependency with a latency distribution an order of magnitude wider than anything else in your stack, whose failure mode is a plausible wrong answer rather than an error.

Without one, every team implements its own retries, its own key handling, its own logging policy, and nobody can answer what the company spends or what a model change would break.

Token spend has no natural ceiling. Every retry, every retrieved document, every chain step multiplies, and nothing in the system tells anyone until the invoice arrives.

A loop that calls a model, gets asked to run a tool, runs it, and feeds the result back. Every operational difference comes from the fact that the number of iterations is not known in advance.

Tokens are the unit and input and output are priced differently. Get the arithmetic on the table in the first minute, because the number usually decides the design before any other consideration does.

Prompts contain whatever the user typed, which in a support product is names, account numbers and medical detail. Full-text logging is a data-protection decision, not a debugging preference.