How ChatGPT Works

OpenAI operates three distinct web crawlers, each with a different function. Understanding which one matters for visibility — and how to configure your robots.txt for each — is a technical prerequisite before any content strategy.

GPTBot — Crawls publicly accessible websites to collect training data for OpenAI's language models. This is for the model's static knowledge, not for real-time search results. (robots.txt tag: GPTBot. Impact: long-term, indirect.)

OAI-SearchBot — Indexes content specifically for ChatGPT Search results — the real-time, cited answers users see when ChatGPT searches the web. (robots.txt tag: OAI-SearchBot. Impact: direct — this determines if you appear in cited search responses.)

ChatGPT-User — Accesses content on demand when a user initiates browsing, uses plugins, or triggers GPT Actions. (robots.txt tag: ChatGPT-User. Impact: situational.)

OpenAI has confirmed that OAI-SearchBot and GPTBot share crawl information with each other to avoid duplicate crawling — meaning allowing both maximizes efficiency.

The practical implication: if OAI-SearchBot is blocked in your robots.txt, you cannot appear in ChatGPT's cited search responses — regardless of how well your content is written. This is a binary technical gate.

ChatGPT doesn't send your exact question to a search provider. It rewrites the query — often into multiple targeted sub-queries — to maximize retrieval quality.

This multi-query behavior — called query fan-out — means your content needs to match the sub-queries ChatGPT generates, not just the original user question.

ChatGPT Search retrieves content primarily through Microsoft Bing's index, supplemented by partner search providers and direct publisher agreements.

Publisher partnerships announced at ChatGPT Search launch include: Axel Springer, News Corp, Wall Street Journal, The Atlantic, Reuters, Le Monde, Associated Press, Financial Times, Conde Nast, Hearst, and Time. Content from these partners carries elevated trust signals in ChatGPT's retrieval system.

ChatGPT synthesizes retrieved content into a response with inline citations. Users can hover over citations to preview the source and click through to the original page.

Standard ChatGPT Search and Deep Research are not the same product. They operate differently, cite differently, and require different strategies. Deep Research is increasingly the mode used by decision-makers doing serious vendor evaluation.

In OpenAI's own words: "Deep research is OpenAI's next agent that can do work for you independently — you give it a prompt, and ChatGPT will find, analyze, and synthesize hundreds of online sources to create a comprehensive report at the level of a research analyst."

It is powered by the o3-deep-research model — a specialized version of OpenAI's o3, optimized for web browsing and data analysis. A lightweight version using o4-mini-deep-research is also available.

The pipeline works in three steps: (1) Clarification — an intermediate model (gpt-4.1) asks follow-up questions to understand intent, preferences, and constraints. (2) Prompt Rewriting — the same model rewrites the query into a detailed, expanded prompt. (3) Agentic Research — the o3-deep-research model autonomously decomposes the expanded prompt into sub-questions, browses hundreds of sources, and synthesizes a citation-rich report.

Key difference from standard Search Mode: standard ChatGPT Search performs one query and synthesizes immediately. Deep Research performs multiple iterative searches, reads deeply, pivots based on what it finds, and takes 5-30 minutes to complete.

Why this changes everything for B2B brands: when someone runs a Deep Research query like "compare the top AI visibility platforms for enterprise brands in France", the agent will decompose it into sub-questions, browse your website, your reviews on G2 and Capterra, your press mentions, your case studies — and do the same for competitors. Depth of source coverage matters more here than in standard search. Third-party review platforms are read and cited. Your content can be restricted or prioritized by the user — getting your domain onto trusted lists is a long-term strategic asset.