Back to articles

OpenAI Uses Codex Agents to Investigate Performance Regressions

3 min read

OpenAI has outlined an internal performance-engineering system called Performance Factory, designed to move Codex beyond answering the question “What got slower?” The system can investigate an alert, document its findings, propose a code change for an engineer to review and check performance again after deployment. The company presents it as a way to manage regressions in services such as ChatGPT—not as permission for an agent to make unchecked changes in production.

From coding assistant to workflow

OpenAI describes a progression from engineers manually inspecting metrics, logs, traces and code to using Codex as an investigative partner. The team then captured its approach in a shared skill called perf-forensics. It guides the agent through steps such as finding when a slowdown began, identifying affected requests and checking whether deployments or configuration changes could explain it. OpenAI says the skill was tested and refined against past regressions.

Running investigations in the cloud automated the initial diagnosis, but left engineers to sort through results, write fixes and verify deployments. Performance Factory was built to coordinate those handoffs as persistent tasks. Work can move between agents and people, pause while awaiting a review or deployment, and resume when a relevant event arrives.

Four building blocks

  • A task service defines workflow states, permitted transitions and steps that require an engineer. Configuration and Markdown prompts describe what agents should do at each stage.
  • A dispatch loop checks which tasks are ready and starts a Codex turn with the instructions for that step.
  • Cloud execution gives agents access, within OpenAI’s network, to repositories and tools such as metrics, logs, traces, CPU profiles and memory snapshots. A service identity helps scope permissions and attribute actions.
  • Subscriptions wake a task when something changes, such as a review comment, a CI result or a completed deployment, rather than keeping an agent running while it waits.

In the regression workflow, an alert starts an investigation. An agent records its findings and proposes a fix for an engineer to review. After the change is deployed, another agent picks up the task and checks whether performance has returned to baseline. OpenAI says the workflow tracked more than 1,700 investigations and found root causes in over 600 cases in just over three months—about 17 automated investigations a day. Root-causing a case should not be confused with automatically fixing every issue.

What changes—and what does not

The main shift is from automating an isolated analysis to organizing a repeatable engineering process. Performance investigations often draw on code, deployment history and operational signals; a persistent task can reduce repetitive work and make handoffs easier to follow. OpenAI also says recurring optimizations can be turned into workflows that run across services.

The account still places engineers in the approval path for fixes. The system’s permissions are scoped through its service identity, and its reported figures describe OpenAI’s internal use rather than a general benchmark. Other teams would need to assess diagnostic accuracy, manage access to production systems and decide how to handle incorrect suggestions or misleading signals. The broader lesson is that agent automation depends not only on model capability, but also on workflow design, permissions, evaluation and human review.

Web Search

Comments

Checking sign-in status...

Loading comments...

Related articles