AI-Enabled Optimization of Early-Phase Clinical Trials

Aurelyn recommends anchoring the pilot in the highest-uncertainty, smallest-N contexts where model-informed methods deliver the greatest marginal value: first-in-human oncology dose escalation and rare-disease trials, with adaptive Phase 1b/2a designs as a secondary focus. These settings have the clearest decision points (dose, expansion, go/no-go) and the strongest existing precedent for quantitative methods.

A tiered approach: anchor in oncology and rare disease for interpretable early signal, but require platform-agnostic architecture so methods and governance generalize. This protects learning velocity without over-fitting the pilot to one indication.

Yes — prioritize the three with the clearest measurable endpoints and regulatory touchpoints: (1) dose optimization (aligned with Project Optimus), (2) safety-signal detection, and (3) recruitment & biomarker stratification. These produce the cleanest evidence for the Category B metrics.

Select on: a well-defined context-of-use, an assigned model-risk tier, demonstrated data readiness/ALCOA+ maturity, a pre-specified credibility-assessment plan, and evidence of a managed AI lifecycle (Good Machine Learning Practice). This mirrors the FDA's own draft-guidance framework and keeps selection objective.

Use stratified selection quotas across sponsor size (small/emerging biotech through large pharma), AI maturity, and therapeutic area. The chief barrier for smaller sponsors is infrastructure — so a low-infrastructure delivery model is essential to genuine representation.

A four-party sponsor–technology vendor–academic–FDA consortium, with an independent technology/assurance layer that no single sponsor owns. This separates the party that builds the model from the party that governs and validates it.

Stand up a shared validation harness and benchmark datasets in secure enclaves, with federated evaluation so participants contribute to common metrics without exposing proprietary data or models.

Embed both directly in the Govern function of the RMF: patient advisors weigh in on context-of-use, acceptable risk, and fairness; investigators provide the clinical-workflow reality check and serve as the human-in-the-loop for every consequential recommendation.

Early regulatory engagement (a pre-pilot context-of-use agreement), technical guidance on credibility assessment and model-risk tiering, and a standing review cadence so participants aren't guessing at expectations mid-pilot.

Secure, validated data environments (21 CFR Part 11 compliant), shared tooling for validation and monitoring, and immutable audit trails common across participants.

Adopt a tiered maturity model — from advisory/shadow-mode for low-maturity participants to integrated decision support for the most mature — so every sponsor contributes evidence at a level matched to its readiness.

18–24 months — long enough to carry at least one cohort from first-in-human dosing through a Phase 2 initiation decision, while remaining short enough to inform a summer-2026-style expansion cycle.

Recommended gates: (1) onboarding & context-of-use lock; (2) data-readiness gate; (3) mid-pilot safety & model-performance review; (4) model-drift checkpoint; (5) Phase 1→2 decision capture. Each gate has pre-registered pass/fail criteria.

Use a learn-and-confirm staging with pre-registered metrics: continuous telemetry provides rapid operational insight, while confirmatory conclusions are gated on pre-specified, locked endpoints to preserve rigor.

Maintain a structured pilot registry with standardized context-of-use and credibility-assessment templates, culminating in a public summary report so the broader ecosystem inherits the learning.

Adopt tiered disclosure: public model cards and system cards describe intended use, performance, and limitations at the context-of-use level; deeper artifacts are shared confidentially with the regulator. This satisfies transparency without exposing trade secrets.

Track cycle-time metrics against a pre-defined baseline: time-to-trial-initiation, time-to-first-patient-in, enrollment rate, and time-to-last-patient-last-visit — each compared to historical or concurrent non-AI benchmarks.

Measure the interval between Phase 1 completion and Phase 2 initiation, decomposed into data-lock, analysis, decision, and start-up sub-intervals so the source of any acceleration is attributable.

Use screen-fail rate, screen-to-enroll ratio, time-to-enroll, and retention/dropout rate, stratified by site and subgroup to surface where AI enrichment actually helps.

Combine decision latency, decision-reversal rate, and calibration (predicted vs. observed outcomes) across both FDA regulatory and sponsor-internal decision points.

Run blinded parallel decisioning: AI-supported and traditional decisions are made independently, then compared for concordance and, where ground truth emerges, for accuracy (AUROC, Brier) against adjudicated outcomes.

This requires longitudinal registry linkage: track downstream Phase 3 success conditional on AI-supported early decisions, acknowledging the long horizon and using survival/competing-risk framing rather than a simple rate.

Time-to-signal-detection, time-to-response, and the sensitivity/specificity and false-alarm rate of detection, measured against adjudicated safety events.

Compare AE/SAE rate deltas and protocol-deviation frequency between AI-supported and comparator arms, controlling for population and exposure.

ALCOA+ scorecards, query rates, and source-data-verification discrepancy rates provide an objective, auditable view of data integrity.

Report discrimination (AUROC), calibration, subgroup performance, and external validation on held-out and independent data — generalizability is demonstrated, not assumed.

Monitor input drift (e.g., population stability index), performance-over-time, and a defined retraining cadence under change control, with alerting when drift breaches thresholds.

Maintain stratified performance dashboards with fairness slices by demographic and clinical subgroup, by site, and by therapeutic area — surfacing heterogeneity rather than hiding it in an aggregate.

A context-of-use-scoped credibility assessment per the FDA draft guidance — analytical validation plus clinical validation, with credibility evidence proportional to model risk.

Through model-risk tiering, human-in-the-loop controls, fail-safe defaults, and override logging — with the residual-risk profile documented per context-of-use.

Use model and system cards, explanation-fidelity measures, and a behavioral (black-box) testing harness that evaluates input/output behavior without requiring source access — making the same metrics applicable to proprietary third-party systems.

Assess de-identification, access controls, data lineage, and Part 11 compliance, with a documented data-governance plan per context-of-use.

Subgroup performance-parity analysis and bias audits across demographic and clinical strata, with parity thresholds agreed at context-of-use registration.

A blend: historical controls, concurrent non-AI arms, and in-silico simulation / digital-twin benchmarks — triangulating rather than relying on any single comparator.

Through covariate adjustment, stratification, matched comparisons, and simulation-based benchmarking so that observed differences are attributable to AI rather than to design heterogeneity.

Deploy validated trust and acceptance instruments (e.g., adapted technology-acceptance and trust-in-automation scales) for investigators, participants, and regulators at defined checkpoints.

System Usability Scale scores, task-completion rates, and time-and-motion analysis of how recommendations enter the clinical workflow.

Adoption metrics, net-promoter-style value scores, cost-per-decision, and scalability stress tests across sites and indications.