Apodex moves AI benchmarking beyond answer keys with TRACES real-world discovery environments

Apodex has launched TRACES, an artificial intelligence benchmark designed to test whether complete AI agent systems can conduct sustained, verifiable investigations on real-world scientific problems rather than merely reproduce answers already contained in datasets. The August 19 release evaluates the full trajectory of an AI investigation across six capabilities covering tool use, error repair, alternative hypotheses, long-horizon coherence, evidence and appropriate limits on conclusions. The underlying Apodex Discovery research surveyed 561 industries across 16 sectors, assembled 423 potentially high-value problems and selected 20 for its initial problem registry, while the accompanying research paper describes 17 executable environments spanning 218 evaluation episodes. The central issue is whether TRACES can become a credible yardstick for increasingly ambitious “AI scientist” claims, particularly because Apodex is simultaneously the benchmark developer and the creator of its own heavy-duty AI solver.

The benchmark arrives as scientific AI is moving beyond question answering and code generation toward systems that can search literature, manipulate data, operate scientific tools, formulate hypotheses and revise their approach after experiments fail. That transition makes conventional benchmark scores increasingly incomplete. A model might produce the correct final answer through weak evidence, accidental reasoning or contamination from material encountered during training, while another system could conduct a strong investigation without immediately reaching an answer that becomes verifiable only months later. TRACES is Apodex’s attempt to measure that difference by treating the investigation itself as an object of evaluation.

Why does Apodex think conventional AI benchmarks are becoming inadequate for scientific discovery?

Most familiar artificial intelligence benchmarks begin with a known solution. The model receives a problem, generates an answer and receives a score based largely on whether that answer matches the benchmark’s hidden ground truth. That architecture works reasonably well for mathematics problems, factual questions, code tests and other tasks where success can be determined immediately.

Scientific discovery is structurally different. Researchers may begin without knowing which hypothesis is correct, whether the available evidence is sufficient or even whether a proposed mechanism exists. Progress can involve repeated searches, simulations, failed approaches, contradictory observations and revisions before a defensible conclusion emerges.

TRACES therefore evaluates an AI system inside an executable environment rather than handing it a static prompt. Depending on the problem, an environment can expose scientific literature, structured datasets, code execution, simulation tools, protein-folding systems, experimental feedback and other specialized interfaces. The solver chooses actions, observes their consequences, updates its working state and continues until it submits an outcome.

The change sounds technical, but it has a significant consequence for how AI capability is measured. Apodex is evaluating the complete solver, including the foundation model, agent harness, tools, memory and control policies, rather than treating the underlying large language model as the only source of performance.

That distinction is increasingly important because two systems using the same frontier model can behave very differently once their tool environments, memory architectures, verification mechanisms and agent loops change. Apodex’s research found precisely that effect in several of its experiments.

What do the six TRACES capabilities reveal that a single benchmark score cannot show?

TRACES is an acronym built around six dimensions: Tools, Repair, Alternatives, Coherence, Evidence and Scope.

Tools measures whether an AI chooses appropriate external tools, calls them correctly and interprets their outputs accurately. Repair evaluates whether it can identify the real source of an error after receiving feedback, correct the mistake and confirm that the correction worked.

Alternatives tests whether the system maintains competing explanations instead of prematurely locking onto one hypothesis. Coherence asks whether constraints, evidence and previous decisions remain logically consistent over a long investigation.

Evidence evaluates whether conclusions can be traced back to actual observations, calculations, tool outputs, experiments or sources. Scope measures whether the AI understands the boundaries of its findings, including the conditions under which a conclusion holds and circumstances where it should not be generalized.

Collectively, those dimensions target failure modes that become especially dangerous in scientific applications. An AI can produce an eloquent research narrative while quietly using the wrong dataset, forgetting a condition established several steps earlier, ignoring evidence inconsistent with its preferred theory or extrapolating a narrow experimental observation far beyond the population that was tested.

A final-answer score may miss those weaknesses entirely.

TRACES instead records an episode trajectory containing the solver’s actions, tool calls, observations, resource consumption, errors, workspace state and submitted results. Outcome verification and process verification are then kept conceptually separate. The outcome verifier measures whether the system ultimately succeeded, while the process evaluator examines whether the investigation was conducted in a way that justifies its conclusion.

That makes TRACES less like an academic exam and more like an audit of an AI research process.

How large is TRACES, and are its problems genuinely different from ordinary benchmark questions?

Apodex says a ten-person team of researchers with STEM doctorates spent two months scouting problems across scientific research, healthcare, engineering, finance and industrial applications. The team reviewed 561 industries across 16 sectors and assembled 423 candidate real-world problems before selecting 20 for the initial registry. Problems were assessed for reasoning depth, technical feasibility, verifiability, data and tool availability, verification latency, potential real-world impact and whether a resulting solution could plausibly enter an operating workflow.

The accompanying research paper describes TRACES as containing 17 executable environments and 218 episodes across areas including biomedicine, clinical translation and frontier-model engineering. Apodex’s public discovery site currently highlights environments covering adeno-associated virus capsid engineering, drug repurposing and reformulation, clinical trials and large language model development.

That scale remains modest compared with benchmarks containing thousands of conventional questions, but raw task count is not necessarily the correct comparison. Building an executable scientific environment requires data, tools, constraints, hidden verification, resource budgets and a reproducible interface through which competing AI systems can operate.

Each episode is consequently much heavier than answering a multiple-choice science question.

The economic implication is also interesting. If scientific-agent developers increasingly need realistic environments rather than enormous static question banks, benchmark development itself becomes a significant infrastructure exercise involving domain specialists, compute and verification systems. That could make high-quality scientific evaluation substantially more expensive to construct than conventional model testing.

What did Apodex actually demonstrate in AAV capsid design rather than merely proposing a new scoring framework?

Apodex has paired the benchmark launch with experimental results intended to show that environment design can materially change scientific AI performance.

One of the clearest examples involves adeno-associated virus capsids used in gene therapy. The benchmark breaks the problem into four stages: determining whether an engineered capsid remains viable, predicting its tissue tropism, predicting three-dimensional structure and generating novel sequences for further testing.

Using the same Claude Opus 4.8 model, the Apodex domain-specific environment produced an average score of 0.741 across the first three AAV tasks compared with 0.716 when the model operated through the more generic Claude Code harness. The largest difference appeared in structure prediction, where the domain-specific environment scored 0.657 compared with 0.595.

Apodex then reported that its Apodex 1.1 system exceeded the published comparison methods used in its study across all four AAV stages. It achieved an out-of-distribution AUROC of 0.904 on viability prediction compared with 0.878 for the CAP-PLM reference, while tropism performance reached 0.635 compared with 0.622 for Fit4Function. Its structure score averaged 0.649 compared with 0.605 for the AlphaFold 3-based reference implementation used in the evaluation, while its generative-design score reached 0.180 compared with 0.109 to 0.116 for three specialist methods included in the paper.

These are Apodex-authored benchmark results and should not be interpreted as broad proof that the company’s system is generally superior across scientific discovery. They do, however, provide evidence for a narrower and commercially important point: the environment surrounding a frontier model can materially affect what that model accomplishes.

That shifts competitive attention away from model weights alone.

Why could the environment around an AI model become as important as the frontier model itself?

The drug repurposing work reinforces that interpretation.

Apodex constructed an environment in which a solver evaluates drug-disease combinations using information available before a defined historical cutoff. Later clinical and regulatory outcomes form the hidden evaluation evidence, allowing researchers to test whether the system would have identified promising candidates without seeing what eventually happened.

The benchmark includes a 100-pair public development set and a 200-pair hidden test set. When GPT-5.5 and GPT-5.6-sol were given access to Apodex’s structured biomedical environment rather than operating closed-book, their mean normalized prediction scores improved by 2.5 and 7.6 points respectively in the reported experiments. The paper cautions that the public-set comparison involved three runs and should be interpreted descriptively rather than as a definitive test-set estimate.

The broader implication reaches beyond medicine.

AI companies have spent enormous sums improving foundation models, but agentic systems increasingly derive performance from the combination of model, search, proprietary data, domain tools, memory, verification and workflow design. A sufficiently good foundation model placed inside a superior scientific environment may outperform a nominally stronger model operating with poor tools or weak feedback.

If that pattern continues, enterprise and scientific AI competition could become less about which company owns the single smartest model and more about which system surrounds a capable model with the most useful environment.

TRACES is effectively designed to make those architecture differences measurable.

Can process verification actually improve an AI scientist rather than simply diagnose what it did wrong?

Apodex’s most commercially interesting experiment may be its verification-driven repair loop.

When TRACES detects weaknesses in a trajectory, the process verifier can create a structured repair note identifying which parts of the investigation were deficient and what stronger behavior would require. That feedback can then be returned to the solver for another attempt without exposing the hidden answer or outcome score.

Across 434 trajectories that the process verifier classified as deficient, Apodex reported an average outcome-score improvement of 0.155 after a repaired rerun. Outcomes improved in 204 cases, were unchanged in 144 and declined in 86. Nine of the ten evaluated environments produced a positive average change.

The result requires caution. Apodex explicitly notes that the experiment did not include a matched control group in which the same deficient trajectories were simply rerun without the repair note. Some of the improvement could therefore reflect ordinary run-to-run variation rather than the verification intervention itself.

Even with that limitation, the experiment points toward an important shift in AI evaluation.

A benchmark traditionally sits outside the model and hands down a score. TRACES is designed so evaluation can feed back into the system and potentially improve the next attempt. The benchmark therefore starts becoming development infrastructure rather than merely a leaderboard.

For commercial AI laboratories, that distinction could be valuable. A diagnostic system capable of identifying whether failures come from tool selection, evidence quality, memory, alternative-hypothesis management or the foundation model itself could make research-agent development substantially more systematic.

How does TRACES compare with the growing wave of scientific AI benchmarks appearing in 2026?

Apodex is entering a rapidly developing evaluation field rather than an empty market.

ResearchClawBench, released earlier in 2026, evaluates end-to-end autonomous scientific research across 40 tasks spanning 10 domains. Its authors reported that current systems remained far from reliably reproducing research outcomes, with the strongest autonomous agent in their evaluation achieving an average score of 21.5.

SDABench takes a different approach, evaluating 527 real-data and 6,000 synthetic instances across biology, chemistry, environment, geography and physics. It separates scientific analysis into descriptive, exploratory, inferential, predictive, causal and mechanistic capabilities and found that stronger models still struggled particularly with analytical procedure selection, variable relationships and valid conclusions.

TRACES differentiates itself by emphasizing reality-based executable environments, complete solver systems and process verification across open-ended discovery episodes.

That does not automatically make it a superior benchmark. Different benchmarks are measuring different pieces of a problem that remains poorly understood. ResearchClawBench emphasizes end-to-end research reconstruction, SDABench focuses heavily on scientific data-analysis capabilities, while TRACES places unusual emphasis on how an AI system behaves throughout a stateful investigation.

The proliferation itself is informative. The AI industry is beginning to recognize that performance on static reasoning tests provides increasingly limited evidence about whether an agent can autonomously conduct trustworthy scientific work.

Does Apodex face a credibility challenge because it builds both TRACES and its own AI solver?

Yes, and the company’s decision to open TRACES to outside systems will therefore matter.

Apodex develops the benchmark while also developing the heavy-duty solver whose performance features prominently in the accompanying research. That structure does not invalidate the results, but it makes reproducibility, transparent methodology and independent participation especially important.

Apodex says outside teams can submit models, harnesses, agent loops or complete solver systems for evaluation. It is also inviting researchers and companies to submit additional real-world problems for future benchmark environments. Founder and chief executive officer Tianqiao Chen is identified by Apodex as the project lead behind Apodex Discovery.

External participation could turn TRACES into a broader evaluation platform rather than an internal benchmark primarily demonstrating Apodex technology.

Independent results will be particularly important if competing AI laboratories begin submitting their strongest scientific agents. A benchmark gains strategic importance not simply because its creator publishes it, but because researchers trust its scoring rules enough to optimize systems against it and because performance correlates with useful outcomes outside the benchmark.

The risk is the familiar benchmark problem in a new form. Once a test becomes prestigious, developers can begin designing systems specifically to perform well on its known structure. TRACES attempts to reduce that problem through hidden verifiers, protected ground truth, contamination checks and real-world environments, but no benchmark architecture can permanently eliminate adaptation to the test itself.

Its credibility will ultimately depend on whether the benchmark continues adding unfamiliar problems and whether high scores predict genuine scientific performance.

Why is Apodex offering researchers up to $100,000 a month in compute credits alongside the TRACES strategy?

The benchmark also fits a broader user-acquisition strategy.

In July, Apodex launched its Frontier Program for universities, research institutions and deep-tech startups, offering selected participants up to $100,000 per month in compute credits alongside access to its Deep Research, Deep Solve and Deep Discover systems and technical support.

That program can provide Apodex with something particularly valuable for a scientific AI company: difficult real-world problems.

Frontier users can expose the solver to research questions that do not resemble standardized benchmarks, while failures can reveal weaknesses in tool orchestration, verification and domain knowledge. TRACES provides a framework for turning at least some of those difficult problems into structured environments that can subsequently be evaluated.

This creates a potential flywheel. Apodex attracts researchers with subsidized compute, encounters harder scientific problems, converts suitable problems into verifiable environments, uses those environments to evaluate and improve solver architectures, and potentially attracts additional scientific users because the system becomes stronger.

Whether that produces a defensible commercial platform remains to be demonstrated. Apodex is privately held and does not disclose the type of public quarterly revenue and customer metrics available from listed software companies, so adoption cannot yet be evaluated through recurring-revenue growth or public market response.

The meaningful indicators will instead be participation from credible laboratories, independently reproduced results, growth in the problem registry and evidence that discoveries generated inside these environments survive external experimental validation.

What evidence would show that TRACES is becoming a genuine standard for AI scientific discovery?

The first proof point will be external solver participation. A benchmark dominated by results from the organization that created it offers useful research evidence, but a benchmark used by multiple frontier laboratories becomes industry infrastructure.

The second will be whether benchmark performance predicts real scientific outcomes. A system that scores well on Tools, Repair, Alternatives, Coherence, Evidence and Scope should ideally be more reliable when deployed against unfamiliar research problems, not merely more fluent at producing trajectories that satisfy the evaluator.

The third will be prospective discovery. Retrospective problems with hidden historical outcomes are valuable because they provide rigorous ground truth, but the strongest evidence for “discoverative AI” will come when a system proposes something genuinely unknown and later experiments confirm it.

That is a much higher bar than beating a static AI benchmark.

Apodex has nevertheless identified the correct underlying problem. As artificial intelligence systems become capable of performing longer research tasks, asking whether the final answer is correct becomes increasingly insufficient. Scientists, companies and regulators will need to know whether the evidence was real, whether the appropriate tools were used, whether contradictory explanations were considered, whether mistakes were repaired and whether conclusions remained within the boundaries justified by the data.

TRACES is an attempt to turn those questions into measurable properties of an AI system. Its next challenge is proving that the measurement itself survives the standard it seeks to impose on AI: independent evidence, reproducibility and verification against the real world.

What are the key takeaways from Apodex launching the TRACES scientific discovery benchmark?

• Apodex launched TRACES on August 19, 2026 to evaluate AI systems working on open-ended, real-world scientific problems.
• TRACES evaluates six capabilities: Tools, Repair, Alternatives, Coherence, Evidence and Scope.
• The benchmark evaluates complete solver systems, including foundation models, agent harnesses, tools and control policies, rather than model performance alone.
• Apodex’s research team surveyed 561 industries across 16 sectors and assembled 423 candidate high-value problems before selecting 20 for its initial registry.
• The accompanying research describes 17 executable environments spanning 218 evaluation episodes.
• Apodex reported that its domain-specific environment improved performance over a generic harness in AAV capsid-engineering experiments.
• The company also reported that Apodex 1.1 exceeded selected published reference methods across four AAV capsid tasks, although those results come from Apodex’s own benchmark research.
• Verification-driven repair improved the average outcome score across 434 deficient trajectories in Apodex’s experiments, but the company notes that no matched rerun control was included.
• TRACES enters a growing scientific-agent benchmark field that also includes ResearchClawBench and SDABench.
• Independent participation, prospective discoveries and real-world experimental validation will determine whether TRACES becomes an industry benchmark rather than primarily an Apodex development framework.