Models and APIs

Model choice is only valuable when connected to task shape, risk tolerance, and operating cost. This section frames APIs and models as components in a production system, not as abstract benchmark winners.

Core paths

Model routing Decide when routing, fallback, or tiered models make more sense than a single default model.

GPT-5.5 agentic workflows A current-to-evergreen rollout page for deciding where GPT-5.5 belongs in agentic workflows, cost budgets, and eval gates.

AI compute capacity planning Translate compute headlines into hosted API, batch, flex, rented GPU, utilization, and margin decisions.

How much does an AI agent cost in production? Use this page when the real buying question is not token price alone but total workflow cost per successful outcome.

LLM cost allocation and showback Use this page when raw provider invoices need to become workflow, tenant, feature, and budget-owner accountability.

How do you calculate AI agent ROI? Use this page when the budgeting debate is turning into a business-case debate around real workflow return.

Do you need RAG for an AI agent or AI product? Use this page when retrieval is on the roadmap but the team still needs to prove the knowledge boundary is real.

Structured outputs vs JSON mode Use this page when the output contract matters more than simple valid JSON formatting.

Prompt caching vs retrieval vs fine-tuning A durable systems-design page for teams deciding which optimization lever actually fits the problem.

Responses API vs Chat Completions A current and durable API decision page for teams building tool-connected products, agents, or stateful workflows.

OpenAI background processing systems Use this page when the search question is how to build background processing AI systems with OpenAI background mode.

Build a background processing AI system Use this page when the implementation problem is job records, status tracking, cancellation, review gates, and failure handling around OpenAI background mode.

Background mode, ZDR, and retention Use this page when enterprise data-control rules decide whether background mode is allowed for a workload.

Reasoning models vs fast models Use this page when the team needs a concrete rule for which steps deserve premium reasoning and which belong on cheaper high-throughput lanes.

OpenAI Batch API vs background mode Use this page when the real problem is async design: bulk deferred processing versus one long-running tracked job.

OpenAI background mode job lifecycle Use this page when the implementation problem is polling, webhooks, status transitions, retries, and review-aware completion.

OpenAI Batch pricing and fit Use this page when the budget question is whether Batch actually saves money for the workload shape you have.

Batch limits, expiration, and output files Use this page when the implementation question is batch size, queued tokens, expired requests, output files, error files, and retry handling.

Built-in search economics Use this page when the team is deciding whether built-in search is solving a real workflow problem or just adding latency and spend.

AI browser search readiness Use this page when AI browsers, ChatGPT search, and conversational product discovery require stronger page structure, evidence, comparison logic, and measurement.

Web search vs RAG Use this page when the team needs a clear boundary between live external discovery and retrieval from owned knowledge.

File search vs external vector databases Use this page when managed retrieval looks attractive but the team is unsure whether product needs now justify an owned vector layer.

When is OpenAI file search enough? Use this page when the team is deciding whether managed retrieval still fits or whether custom retrieval ownership is now justified.

What drives vector database spend? Use this page when retrieval cost is rising and the team needs to separate storage from freshness, fan-out, and ownership burden.

Conversation state vs RAG vs long context Use this page when teams keep calling all three of these patterns memory and need a cleaner architecture boundary.

Code interpreter vs external Python sandboxes Use this page when execution moves from convenient tool use into runtime ownership, dependency control, and sandbox policy.

Tool-use latency and cost budgets Use this page when the workflow keeps adding search, retrieval, or execution and the team needs a real budget instead of tool sprawl.

OpenAI Batch vs Flex vs Priority Use this page when the apparent Batch vs Flex question also includes priority lanes, latency, reliability, and workload class.

Cost per success and tool economics Use this page when the product needs workflow-level economics instead of call-level spend snapshots.

Hosted tools vs self-managed tooling Use this page when the team needs a cleaner boundary between hosted convenience and infrastructure it genuinely needs to own.

GPU cloud vs hosted model APIs Use this page when the team is trying to decide whether infrastructure ownership now beats hosted API economics.

A100 vs H100 economics Use this page when rented compute is already justified and the real decision is which GPU class earns its premium.

When batch and flex are cheaper than rented GPUs Use this page before renting GPUs if lower-cost hosted service tiers may still solve the economics problem.

Use cases Start with the task and operating risk before narrowing the model or API layer.

Evaluation Tie model choice to measurable quality, not only anecdotal prompt demos.

Current durable pressure point

GPT-5.5 rollout boundary A high-current-intent page that should age into a durable model-routing, cost-per-success, and eval-gate page for frontier releases.

AI compute capacity planning A high-value infrastructure economics page for teams deciding whether demand justifies async lanes, rented compute, or continued hosted APIs.

Model routing for support operations Use the current multi-provider pricing and capability spread to design cleaner queue boundaries instead of paying premium-model economics for every support task.

Responses API adoption boundary A stronger product-platform decision page for teams adding tool use, agent behavior, state, and orchestration to the roadmap.

Structured outputs implementation boundary A high-intent implementation page for teams deciding whether valid JSON is enough or a schema contract is now mandatory.

Caching, retrieval, and tuning economics A durable systems-economics page around one of the easiest ways AI teams waste money and engineering effort.

OpenAI background processing systems A strong current implementation page for teams asking how to build background processing AI systems with OpenAI background mode.

Background processing system design A direct answer page for builders searching how to build an OpenAI background processing AI system with durable jobs, status, review, and recovery.

Background mode data-control boundary A high-intent enterprise page around background mode, Zero Data Retention, store=true, polling state, and workload policy.

Reasoning vs fast-model economics A durable high-intent page for teams deciding whether premium reasoning is solving an expensive problem or simply inflating runtime cost.

Background job lifecycle A stronger implementation page for teams designing polling, webhook, retry, and approval behavior around long-running jobs.

OpenAI Batch vs background async design A cleaner decision page for one of the easiest ways product teams confuse backlog processing with tracked long-running jobs.

OpenAI Batch pricing and discount fit A stronger economics page for teams deciding whether lower-cost deferred processing is actually the right lane.

OpenAI Batch limits and expiration A practical implementation page for teams designing Batch jobs around request limits, 24-hour windows, output files, error files, and partial replay.

Built-in search economics A high-intent implementation page for teams trying to price the real value of search instead of enabling it everywhere by default.

AI browser and product discovery readiness A current-to-evergreen page for teams preparing content, product data, and comparison pages for AI-assisted browsing and conversational discovery.

Web search versus RAG boundary A durable implementation page for teams trying to stop mixing current-web search problems with owned-knowledge retrieval problems.

File search versus vector database boundary A high-intent retrieval architecture page for teams deciding when managed file search is enough and when owned indexing is now justified.

When OpenAI file search is enough A stronger managed-retrieval economics page for teams deciding whether the hosted layer still fits the product's control and cost needs.

Conversation state versus RAG versus long context A current architecture page for teams deciding what should live in thread state, what should be retrieved, and when long context is enough.

Code execution boundary A strong implementation page for teams deciding whether built-in execution is still enough or runtime ownership has become unavoidable.

Tool-use budget discipline A stronger economics page for teams trying to cap latency and cost before stacked tool calls quietly damage the product.

OpenAI Batch versus Flex economics A current high-intent economics page for teams deciding which traffic deserves guaranteed speed, which can wait, and which can tolerate softer service tiers.

Cost per success A stronger economics page for teams that need to judge search, retrieval, and execution by completed outcomes instead of line-item API cost.

LLM cost allocation and showback A high-value FinOps page for teams turning AI usage into feature, workflow, tenant, and budget-owner accountability.

Hosted tools versus self-managed tooling A higher-intent economics and control page for teams deciding when hosted tools are still the right product choice and when internal ownership is justified.

GPU cloud versus hosted model APIs A high-value infrastructure economics page for teams deciding when API spend has matured into a real compute-ownership decision.

A100 versus H100 economics A high-value hardware economics page for teams that have already crossed into rented compute and now need to choose the right GPU class.

Batch and flex before rented GPUs A current cost-control page for teams that may still have cheaper hosted service tiers available before taking on GPU ownership.

How much does an AI agent cost in production? A question-led economics page for teams that need an honest production budget before expanding agent scope.

Do you need RAG for an AI agent or AI product? A question-led architecture page for teams deciding whether retrieval belongs in the design or is only adding overhead.

How do you calculate AI agent ROI? A question-led economics page for teams that need a defensible business case instead of generic automation optimism.

Routing questions

What minimum capability is required for the task to clear the bar?
Which requests justify slower or more expensive reasoning?
What should happen when a provider degrades, rate limits, or changes behavior?
Which tasks need deterministic formatting, policy enforcement, or tool calling support?