The problem with PRS

Probability of Regulatory Success (PRS) is a metric used to estimate the likelihood that a drug candidate will successfully navigate the regulatory review process and achieve market approval. PRS focuses exclusively on the final stages of the drug development pipeline, where regulatory agencies assess the efficacy, safety, and quality of a therapy. While PRS provides a targeted lens for evaluating regulatory outcomes, its narrow scope and reliance on historical benchmarks limit its utility, particularly in dynamic regulatory environments or for innovative therapies that challenge conventional paradigms.

Defining PRS

PRS is defined as the likelihood of achieving regulatory approval after submission of a New Drug Application (NDA) or Marketing Authorization Application (MAA):

$$
PRS = \frac{\text{Number of approved applications}}{\text{Total applications submitted}}
$$

Where

• Approved applications: NDAs or MAAs granted market authorization.
• Total applications submitted: The total number of regulatory submissions for review.

The metric reflects the proportion of drugs that successfully meet regulatory standards, focusing on the quality of data submitted, adherence to guidelines, and the therapeutic profile of the drug.

A Comparative Example: Oncology Therapy vs. Gene Editing for Rare Disease

Case 1: Oncology Therapy with Established Precedent

• Context: An oncology drug targeting PD-L1 inhibitors is submitted for regulatory approval in a well-defined cancer indication. Regulatory agencies are familiar with this class of therapies.
• Probabilities:
  • Historical PRS for oncology drugs: 0.90 (90%).
  • Adjusted PRS due to high-quality trial data and alignment with guidelines: 0.95 (95%)
• Interpretation:
  The high PRS reflects regulatory familiarity with PD-L1 inhibitors, well-characterized biomarkers, and adherence to established clinical endpoints.

Case 2: First-in-Class Gene Editing Therapy for Rare Disease

• Context: A gene editing therapy targeting a rare genetic disorder is submitted for approval. While it addresses a high unmet need, it involves novel technology with limited precedent.
• Probabilities:
  • Historical PRS for novel modalities: 0.60 (60%).
  • Adjusted PRS due to unproven technology and variability in endpoints: 0.50 (50%).
• Interpretation:
  The lower PRS reflects regulatory uncertainty surrounding gene editing, including concerns about long-term safety, variability in trial endpoints, and manufacturing complexity.

Critique of PRS

1. Overreliance on Historical Averages

PRS calculations heavily rely on historical benchmarks, which may not reflect evolving regulatory environments:

• Established Therapies: Oncology drugs with precedent benefit from high PRS values due to regulatory familiarity and well-defined review pathways.
• Innovative Therapies: Gene editing therapies or mRNA vaccines often receive lower PRS values due to limited precedent, despite their potential to redefine treatment paradigms.

2. Narrow Focus on Regulatory Hurdles

While PRS provides insights into regulatory review, it ignores broader contextual factors influencing approval outcomes:

• Efficacy and Safety Profiles: PRS often assumes that a strong clinical profile ensures approval, overlooking external pressures such as public scrutiny or political influences.
• Therapeutic Area Dynamics: Therapies for rare diseases may face unique challenges, such as variability in endpoints or limited patient populations, which PRS models inadequately address.

3. Static Nature of PRS

PRS is typically treated as a fixed probability, failing to account for real-time developments:

• Adaptive Pathways: Accelerated approval programs or conditional pathways can significantly improve the likelihood of approval for innovative therapies. PRS often lags in capturing these opportunities.
• Evolving Guidelines: Regulatory agencies frequently update guidelines to reflect emerging science, but PRS models may not adjust to these shifts.

Illustrative Shortcomings of PRS

1. Penalizing Innovation

PRS systematically penalizes therapies with limited regulatory precedent:

• Gene Editing Therapies: Despite addressing high unmet needs, these therapies face low PRS values due to concerns about safety and manufacturing scalability, discouraging investment in transformative science.
• First-in-Class Drugs: Therapies targeting novel mechanisms often struggle with endpoint variability, further lowering PRS.

2. Rewarding Incremental Innovation

Conversely, PRS rewards therapies that make incremental improvements within established modalities:

• PD-L1 Inhibitors: Regulatory agencies are familiar with these therapies, resulting in high PRS values for new entrants, even when differentiation from existing products is minimal.
• Generic Approvals: Simplified regulatory pathways for generics further skew PRS in favor of well-established classes.

3. Ignoring Broader Influences

PRS fails to incorporate external factors that can significantly influence regulatory decisions:

• Public and Political Pressure: High-profile diseases or therapies addressing global health crises may receive expedited reviews or lenient scrutiny.
• Industry Trends: Therapies aligned with regulatory priorities, such as precision medicine or sustainability, may benefit from informal advantages not captured by PRS models.

Black Swan Considerations

While PRS does not explicitly account for outlier events, such occurrences often highlight its limitations:

• Unforeseen Approvals: The rapid approval of mRNA vaccines during the COVID-19 pandemic defied historical benchmarks, emphasizing the need for flexibility in PRS models.
• Transformative Therapies: Breakthroughs in gene editing or cell therapy may face initial skepticism but ultimately redefine regulatory standards, rendering static PRS estimates obsolete.

Conclusion: The Limitations of PRS

Probability of Regulatory Success (PRS) provides a focused assessment of the likelihood of achieving market approval, but its reliance on historical data, narrow scope, and static nature limit its effectiveness. By favoring incremental innovations and penalizing transformative therapies, PRS risks perpetuating conservative approaches to drug development. To maximize its utility, PRS must be supplemented with dynamic analyses that incorporate real-time regulatory shifts, external pressures, and the broader impact of successful approval.