The problem with PTRS

Probability of Technical and Regulatory Success (PTRS) is a widely used metric in biotech and pharma, employed to estimate the likelihood that a drug will successfully advance through clinical trials and regulatory approval. By quantifying the probability of success, PTRS aims to provide investors, companies, and stakeholders with a risk-adjusted view of potential outcomes. While PTRS appears to offer a rigorous framework for decision-making, its application often oversimplifies the complexities of drug development, leading to significant shortcomings. This is especially true for innovative or first-in-class therapies, where the historical data underpinning PTRS calculations are often inadequate for predicting success.

The Definition and Formula

PTRS is typically expressed as the product of success probabilities across each development phase:

$$
PTRS = P_{preclinical} \cdot P_{phase\_I} \cdot P_{phase\_II} \cdot P_{phase\_III} \cdot P_{approval}
$$

Where:

• Ppreclinical​: Probability of advancing from preclinical to Phase I
• Pphase I: Probability of advancing from Phase I to Phase II
• Pphase II​: Probability of advancing from Phase II to Phase III
• Pphase III: Probability of advancing from Phase III to regulatory submission
• Papproval: Probability of gaining regulatory approval

PTRS combines these probabilities to provide a single measure of a drug’s technical and regulatory likelihood of success.

The Flaws in PTRS

1. Overreliance on Historical Averages

PTRS often relies on historical averages to estimate probabilities for each phase of drug development. For instance, historical data might suggest that 10% of drugs successfully transition from Phase I to Phase III. These averages are then applied universally, even for innovative treatments or novel mechanisms of action.

Why This Is Problematic:

• Innovative Therapies and Black Swan Events:
  Black swan events—highly impactful but rare occurrences—are by definition not well-represented in historical data. For example:
  • First-in-class therapies like mRNA vaccines for COVID-19 or CRISPR-based gene editing treatments defied conventional expectations.
  • Historical success rates would have significantly underestimated their potential, deterring investment or undervaluing their transformative impact.
• Bias Toward Incremental Innovation:
  Historical data are dominated by conventional drugs or therapies within well-understood mechanisms of action. PTRS derived from these averages penalizes breakthrough treatments, which often involve higher uncertainty but also greater potential rewards.
• Dynamic Regulatory Landscape:
  Regulatory agencies have increasingly adopted adaptive and expedited approval pathways (e.g., Breakthrough Therapy Designation, Fast Track). Historical data may fail to reflect these evolving standards, leading to outdated or overly pessimistic probability estimates.

2. Lack of Integration of External Factors

PTRS typically assumes that success probabilities are intrinsic to the drug and its development process, ignoring critical external factors:

• Regulatory Changes: New regulatory guidelines or fast-track designations can significantly alter a drug’s likelihood of success. PTRS often fails to incorporate these dynamics.
• Competition: The presence of competing therapies can erode a drug’s market position or increase the stringency of regulatory scrutiny, but these factors are rarely factored into PTRS calculations.
• Market Access: Even technically successful drugs may fail to achieve approval or reimbursement in key markets due to pricing constraints, cost-effectiveness evaluations, or payer resistance.

3. Overemphasis on Technical Success

PTRS focuses narrowly on technical and regulatory milestones without considering broader commercial viability:

• Limited Market Potential: A drug with a high PTRS may still face limited market potential due to small target populations, competitive pressures, or suboptimal pricing.
• Commercial Risks: PTRS ignores risks such as manufacturing scale-up challenges, post-approval monitoring requirements, or market entry delays.

4. Multiplicative Nature of PTRS

The formula for PTRS assumes that probabilities are independent and multiplicative. However, this rarely holds true in practice:

• Correlated Risks: Success in one phase may depend heavily on factors present in subsequent phases. For example, a drug targeting a novel mechanism of action may face consistent skepticism across all phases.
• Cascade Failures: If a trial design or endpoint is flawed, it can affect multiple phases simultaneously, amplifying the likelihood of failure beyond what PTRS suggests.

Illustrative Example

Consider two hypothetical drugs:

1. Drug A: A reformulated generic with well-established success probabilities.
   • Probabilities:
     $$
     P_{preclinical} = 0.90, \quad
     P_{phase_I} = 0.85, \quad
     P_{phase_II} = 0.75, \quad
     P_{phase_III} = 0.70, \quad
     P_{approval} = 0.95
     $$
• Calculating PTRS$$
  PTRS_A = 0.90 \cdot 0.85 \cdot 0.75 \cdot 0.70 \cdot 0.95
  $$
  $$
  PTRS_A = 0.42 \, (42\%)
  $$
  
2. Drug B: A first-in-class therapy targeting a novel mechanism.
   • Probabilities:
     $$
     P_{preclinical} = 0.90, \quad P_{phase_I} = 0.85, \quad P_{phase_II} = 0.75, \quad P_{phase_III} = 0.70, \quad P_{approval} = 0.95
     $$
   • Calculating PTRS:
     $$
     PTRS_B = 0.70 \cdot 0.60 \cdot 0.50 \cdot 0.40 \cdot 0.60
     $$
     $$
     PTRS_B = 0.05 \, (5\%)
     $$

While Drug A appears far more likely to succeed based on PTRS, this calculation ignores the transformative potential of Drug B. If Drug B succeeds, it may command a significantly larger market share, justifying the higher risk despite its lower technical probability of success.

Further Critique: Black Swans and Transformative Potential

Overreliance on historical averages in PTRS calculations systematically undervalues innovative therapies that challenge the status quo. Black swan events, such as the approval of groundbreaking treatments, often occur in the context of:

• Paradigm Shifts: Therapies that redefine standards of care, such as CAR-T cell therapy or RNA-based drugs, initially faced skepticism due to limited precedent.
• Unpredictable Accelerators: Unexpected factors, such as global health crises or philanthropic funding, can rapidly advance novel technologies beyond what PTRS would predict.

Ignoring these possibilities leads to:

• Underinvestment in High-Risk, High-Reward Therapies: Investors may shy away from opportunities that appear risky under traditional PTRS calculations but hold transformative potential.
• Misallocation of Resources: Firms may overinvest in incremental improvements that align with historical norms rather than pursuing breakthroughs with broader long-term impacts.

Conclusion: The Limitations of PTRS

Probability of Technical and Regulatory Success (PTRS) provides a structured framework for assessing risk in biotech and pharma, but its simplicity often masks its inadequacy. By relying on historical averages, ignoring external factors, and overemphasizing technical milestones, PTRS fails to capture the nuanced risks and rewards of drug development. This is particularly problematic for innovative therapies and black swan events, where past data are less applicable, and success depends on factors beyond technical progress. While PTRS can be a useful starting point, it must be complemented by a broader analysis of market dynamics, competitive positioning, and commercial potential to provide a more accurate picture of a drug’s true value.