The problem with POPT

The Problem with Probability of Phase Transition (POPT) in Biotech and Pharma

Probability of Phase Transition (POPT) is a commonly used metric in drug development that measures the likelihood of a drug candidate advancing from one specific phase of development to the next. Unlike broader metrics such as Probability of Success (POS) or Likelihood of Approval (LOA), POPT focuses narrowly on individual steps in the drug development process, such as moving from Phase II to Phase III. While POPT provides a granular view of progress, its narrow scope and reliance on historical data limit its utility, particularly for innovative therapies and dynamic development environments.

Defining POPT

POPT is calculated as the probability of a drug transitioning successfully from one development phase to the next:

$$
POPT_{n \to n+1} = \frac{\text{Number of drugs progressing from Phase } n \text{ to Phase } n+1}{\text{Total number of drugs in Phase } n}
$$

Where:

• POPTn→n+1​: Probability of transition from phase nnn to phase n+1.
• n: Current phase (e.g., Phase I, Phase II, or Phase III).

The metric is influenced by the specific criteria for each phase:

• Phase I: Focuses on safety and tolerability.
• Phase II: Emphasizes efficacy and dose optimization.
• Phase III: Confirms efficacy in larger populations and collects data for regulatory submission.

A Comparative Example: A Classic Oncology Pathway vs. a Rare Disease Program

Case 1: Oncology Therapy Targeting EGFR

• Context: An oncology therapy targets Epidermal Growth Factor Receptor (EGFR), a well-established pathway in cancer. Numerous drugs targeting EGFR have been approved, creating a wealth of data to inform POPT.
• Probabilities:
  • POPT I→II = 0.85: High safety predictability in Phase I.
  • POPT II→III = 0.65: Moderate efficacy predictability in Phase II.
• POPT III→Approval = 0.75: Established endpoints streamline Phase III progression.

Calculation

$$
Overall\_POPT_{EGFR} = 0.85 \cdot 0.65 \cdot 0.75 = 0.414 \, (41.4\%)
$$

Case 2: Rare Disease Program for Spinal Muscular Atrophy (SMA)

• Context: A therapy for Spinal Muscular Atrophy (SMA) targets a novel mechanism involving SMN2 gene splicing. Limited historical data and small patient populations make phase transitions challenging.
• Probabilities:
  • POPT I→II = 0.60: Safety challenges due to the novelty of the approach.
  • POPT II→III = 0.50: Uncertainty in efficacy given small patient cohorts.
  • POPT III→Approval = 0.40: Regulatory hurdles due to limited precedent.

Calculation

$$
Overall\_POPT_{SMA} = 0.60 \cdot 0.50 \cdot 0.40 = 0.12 \, (12\%)
$$

Interpreting the Results

1. Oncology Therapy (EGFR):
   • A higher overall POPT reflects the well-understood nature of the EGFR pathway and significant historical data supporting safety and efficacy.
   • Established trial designs and biomarkers contribute to smoother phase transitions.
2. Rare Disease Therapy (SMA):
   • A lower POPT highlights the challenges associated with novel mechanisms and rare disease indications.
   • Limited patient populations and regulatory uncertainties increase the risks at each transition.

While POPT provides insights into specific development phases, it fails to capture broader factors that influence the likelihood of success, such as long-term commercial viability or shifts in the regulatory landscape.

Critique of POPT

1. Narrow Scope

POPT focuses exclusively on individual phase transitions, overlooking the interdependencies and cumulative risks across the entire development process:

• Phase-Specific Silos: Success in one phase often influences subsequent phases. For example, robust safety data in Phase I can bolster confidence in Phase II, while weak efficacy in Phase II may increase scrutiny in Phase III.
• Cumulative Impact: By isolating individual transitions, POPT does not account for the cascading effects of failures or setbacks.

2. Overreliance on Historical Data

As with other metrics, POPT heavily depends on historical averages:

• Bias Toward Established Pathways: Therapies targeting well-known mechanisms, such as EGFR, benefit from high POPT values based on extensive precedent.
• Challenges for Innovation: Novel therapies, such as SMA treatments, face artificially low POPT values due to a lack of historical benchmarks, despite their potential to transform treatment paradigms.
• Evolving Standards: Historical POPT values may not reflect recent regulatory changes, such as accelerated approvals or adaptive trial designs.

3. Exclusion of Broader Contextual Factors

POPT isolates technical success probabilities but often neglects external factors that influence phase transitions:

• Trial Design and Patient Recruitment: Rare disease programs or therapies requiring complex trial designs may face challenges unrelated to safety or efficacy, lowering POPT II→III​ or POPT III→Approval
• Regulatory Shifts: Agencies may alter trial requirements or grant expedited pathways that fundamentally change phase transition probabilities.

4. Penalizing Rare or High-Risk Indications

POPT tends to penalize therapies for rare diseases or challenging indications:

• Small Patient Populations: Recruitment challenges and limited data availability reduce confidence in efficacy, lowering POP TII→III​ or POPT III→Approval​.
• Novel Mechanisms: High scientific uncertainty further diminishes phase transition probabilities for first-in-class approaches, even when they address unmet needs.

Illustrative Shortcomings of POPT

1. Overlooking Real-World Dynamics

A rare disease therapy with low POPT II→III might benefit from advocacy groups or philanthropic funding that facilitate trial success. Conversely, an oncology drug with high POPT III→Approval may face unexpected regulatory hurdles if efficacy is marginal.

2. Bias Toward Incremental Innovation

POPT’s reliance on historical averages favors therapies that align with established mechanisms, such as EGFR-targeting agents. This bias discourages investment in transformative science and perpetuates incremental innovation.

Conclusion: The Limitations of POPT

Probability of Phase Transition (POPT) offers a granular lens for evaluating specific steps in the drug development process. While useful for identifying phase-specific risks, its narrow scope, reliance on historical data, and exclusion of broader contextual factors limit its effectiveness, particularly for novel or high-risk therapies. By focusing exclusively on phase transitions, POPT fails to capture the cumulative, interdependent nature of drug development.