Qualifying a Novel Clinical Trial Endpoint (iBOX) Predictive of Long-Term Kidney Transplant Outcomes

Introduction

Graft failure following kidney transplantation has significant negative implications, including return to dialysis, lower life expectancy, decreased quality of life, and need for retransplantation. Additionally, graft survival is the most important outcome for people living with a kidney transplant [1]. Currently, immunosuppressive therapies (ISTs) have improved short-term outcomes in kidney transplantation, with 1 year graft survival rates of over 90% [2–5]. Despite the relatively low rate of efficacy failure at 1 year posttransplant, long-term graft survival remains suboptimal. The 5 and 10 years graft survival rates are 77% and 49% for deceased donor and 86% and 64% for living donor transplants [4]. Therefore, there remains a significant unmet need for ISTs that improve long-term outcomes. One of the challenges for biopharmaceutical sponsors is executing registration trials of a feasible size and duration (1–2 years) to support superiority claims using the historically accepted primary efficacy failure composite endpoint consisting of death, graft failure, biopsy-proven acute rejection, and lost to follow-up. These current endpoints, while acceptable to regulators, are not optimized for short-term superiority of ISTs that are predictive of longer-term graft survival. Such studies would require extended duration (e.g., 5 years or more), which may be impractical and unfeasible.

Transplant Therapeutics Consortium (TTC)—A Regulatory-Focused Neutral Convener for Transplant

In 2014, the 2 major US transplantation societies, the American Society of Transplantation and the American Society of Transplant Surgeons, recognized the need for a pathway to develop new ISTs for transplant recipients [6]. In 2017, these societies partnered with Critical Path Institute and other transplant community members to create TTC (https://c-path.org/programs/ttc/). By facilitating a public–private partnership among scientists from the biopharmaceutical industry, diagnostics companies, academic institutions, professional societies, and government and regulatory agencies, TTC fosters consensus and data-driven research to increase speed in developing new therapies. TTC’s primary focus is obtaining regulatory endorsement of an early novel endpoint capable of predicting long-term graft survival in pivotal clinical trials designed to support regulatory approval of new ISTs for kidney transplantation.

To develop a novel trial endpoint, it is important to understand the multifactorial causes of late kidney graft failure; predicting failure accurately with a single marker may not be optimal [3]. Several composite scores have been proposed as surrogates, but iBOX is based on the largest dataset and the only specifically designed multivariate model that predicts long-term death-censored graft failure [7, 8]. iBOX is a risk prediction tool that utilizes multiple clinically relevant features demonstrated to be mechanistically associated with an increased risk of late graft functional decline and failure. These features are estimated glomerular filtration rate (eGFR), proteinuria, anti-human leukocyte antigen donor-specific antibody, and kidney graft biopsy histopathology measured cross-sectionally at any time point posttransplantation. iBOX then integrates these parameters to generate individualized predictions of outcomes at 3, 5, and 7 years posttransplant. iBOX was originally designed to be used at the patient level to inform clinical care and management of kidney transplant patients. In close collaboration with the Paris Transplant Group, TTC translated this work into a clinical trial endpoint acceptable to European Medicines Agency (EMA), intending to streamline drug development by predicting long-term outcomes using short-term data, summarized in Supplementary Table S1. Additionally, the qualification of iBOX as a reasonably likely surrogate endpoint (RLSE) is proceeding with the US Food and Drug Administration (FDA). The regulatory process and timeline associated with FDA and EMA interactions are shown in Figure 1.

FIGURE 1. Regulatory timeline with FDA and EMA. CPIM, Critical Path Innovation Meeting; EMA, European Medicines Agency; FDA, Food and Drug Administration; IR, information request; LOI, letter of intent; QO, qualification opinion; QP, qualification plan; RLSE, reasonably likely surrogate endpoint.

iBOX Scoring System–First Qualified Endpoint in Transplantation

In December of 2022, EMA issued a qualification opinion for iBOX as a secondary endpoint prognostic for death-censored graft loss in kidney transplant recipients intended to be used in clinical trials to support the evaluation of novel IST applications [9, 10]. EMA qualified both a full iBOX (including biopsy), and an abbreviated iBOX (excluding biopsy), allowing flexibility in using this endpoint in studies with and without protocol/surveillance biopsies. Importantly, the component measures in iBOX are modifiable by IST interventions and are further described in Table 1. The iBOX is the first qualified endpoint in transplantation and the fifth qualified endpoint with EMA [10].

TABLE 1. Component measures of the full and abbreviated iBOX.

An important outcome of this qualification is that iBOX can be used as a key secondary endpoint to demonstrate superiority of a new IST compared with the standard of care (SOC) from 6 months to 2 years posttransplant in exploratory or pivotal drug therapeutic studies for regulatory purposes. The datasets supporting this regulatory endorsement represent adult kidney-only transplant recipients with varying underlying diagnoses, multiple donor types, various induction therapies, and either calcineurin inhibitor (CNI)-based or CNI-free therapeutic regimens. As a result, iBOX can be used in registration-driven trials representative of a broad population of kidney transplant recipients. The context-of-use (COU) for iBOX is summarized in Table 2.

TABLE 2. Context-of-use for the qualification opinion of the iBOX Scoring System.

Additionally, in Europe, sponsors and investigators will be able to assess and promote the potential superiority of novel ISTs when measured using iBOX. Further, iBOX will be included in the summary of product characteristics, claims, and other product labeling. Although conditional marketing authorization (CMA) is a separate consideration outside the purview of the Qualification of Novel Methodologies for Drug Development process, superiority to current SOC, thereby addressing an unmet need in kidney transplant, is one of the key criteria for CMA in the European Union [8, 10].

A Community-Based Approach to Endpoint Development

Datasets

A fundamental component of developing an evidentiary package that meets the requirements of regulatory endorsement for iBOX was the success of TTC’s extensive global patient-level data-sharing initiative [9–11]. Datasets from relevant clinical trials, including those used by [7] in their 2019 publication and real-world data from international clinical transplant centers, were prioritized for acquisition. A flow diagram of the dataset selection process is shown in Figure 2 with additional rationale provided in the Supplementary Material. The original iBOX [7] development included time posttransplant to account for varying iBOX assessments of individual patients and to assist in patient care and prognosis estimation. Figure 3 from [7] shows the density of iBOX risk evaluation time points after transplantation. The derivation dataset included in the EMA qualification submission represents all 4,000 subjects described in [7]. For application as a 1-year endpoint in a typical phase 3 clinical trial, we examined the number of subjects in the derivation dataset with iBOX assessments fixed at 1 year posttransplant and had outcome data of at least 5 years.

FIGURE 2. Flow diagram of the data selection process. COU, context-of-use; RCT, randomized controlled trial; TTC, Transplant Therapeutics Consortium.

FIGURE 3. Density of time points where iBOX assessments were made (y-axis), compared with the time posttransplant (x-axis) out to 8 years, as shown in [7].

Five datasets supporting the regulatory endorsement of iBOX included data from clinical transplant centers (i.e., Loupy et al., 2019 derivation [7]. Mayo Clinic Rochester, and Helsinki University Hospital) and clinical trials (i.e., BENEFIT randomized controlled trial [RCT] [12] and BENEFIT-EXT RCT [13]) representing over 2,500 de novo kidney transplant recipients with 1-year iBOX assessments (Table 3). Participant consent was obtained from the transplant centers and clinical trials for primary uses. These datasets contained all elements necessary to assess the performance of iBOX as a pivotal trial endpoint including IST information, iBOX variables at 1 year posttransplant, and 5 years follow-up for death and graft loss of at least 5 years. Additionally, these datasets were accompanied by assay information for each component iBOX measure and laboratory certification documentation ensuring that the analytical methods were robust, reliable, and fit-for-purpose.

TABLE 3. Five-y posttransplant c-statistics values (SE) for the iBOX at 1 year posttransplant in the derivation and validation datasets.

Clinical transplant center data are inherently heterogeneous and reflect the diversity of the kidney transplant recipient population in the United States and European Union. Datasets were curated, standardized, and aligned to conduct internal and external validation analyses to support the iBOX COU with EMA. Two clinical trial datasets have the most extensive CNI-free (belatacept [BELA]) patient-level data with the 4 core iBOX variables and sufficient follow-up period available. This represents, as stated by EMA, “extensive global effort to collect clinical trials and real-world data” [10].

Critical Path Institute explored the number of transplant recipients with full and abbreviated iBOX assessments at varying times posttransplant in the already curated and aligned validation datasets. Supplementary Figure S1 shows the distribution of assessment time points for donor-specific antibody (DSA) measurements up to 2 years postkidney transplant. DSA was selected for illustration because it is collected less frequently than eGFR and/or proteinuria and therefore acts as the key limiting factor for the availability of abbreviated iBOX measurements. The data distribution for iBOX assessments ranges from 6 months up to 2 years posttransplant. Helsinki University Hospital only assessed proteinuria and DSA data at 1 year posttransplant and therefore was excluded from the additional time points exploration. The number of transplant recipients with iBOX assessments at 6 months and 2 years posttransplant in the external validation datasets is shown in Supplementary Table S2. There were significantly more abbreviated iBOX assessments at the varying time points because biopsies were more typically “for-cause” and not taken “per protocol” at 6 months or 2 years posttransplant. Although the full iBOX measurements at 2 years were limited due to lack of biopsy information, because the abbreviated iBOX performed well at this time point, the addition of biopsy information should only further improve the performance, and therefore, the full iBOX is expected to also perform well at 2 years.

Analyses

Validation analyses were performed to support the COU for iBOX with EMA for predicting death-censored graft loss. Both internal validations, evaluating iBOX on the data it was trained on (i.e., the derivation dataset), and external validation, evaluating iBOX on data it was not trained on, were performed. The abbreviated iBOX was treated as a modification of the full iBOX and not validated internally, save for checking the overall c-statistic. Both the full and abbreviated iBOX models were validated on 4 external datasets (i.e., validation datasets) (previously described above).

To avoid survivor bias, patients who did not reach their scheduled evaluation (i.e., those who lost their graft, died, or were lost to follow-up beforehand) were given an imputed worst-case iBOX score (Supplementary Tables S3, S4).

iBOX was validated by assessing its discrimination, the ability to rank individuals from a lower to a higher risk of graft loss, and its calibration, the ability to accurately predict absolute risk level [14]. Discrimination was assessed using Harrell’s c-statistic [15], which gives the probability that, for any 2 randomly selected individuals, the individual with the higher iBOX score, i.e., the higher model-predicted hazard of graft loss, has a shorter death-censored graft survival time. A c-statistic value of 0.7 or greater indicates good discriminatory ability [16]. Secondly, calibration was evaluated by checking whether observed events (graft losses) matched predicted using a Poisson calibration method (see Supplementary Material for a summary of the method) [14].

The full iBOX discrimination in the derivation dataset, when restricted to transplant recipients with an iBOX score at 1-year posttransplant and follow-up to 5 years, had a c-statistic of 0.85, demonstrating iBOX discriminates appropriately among subjects for use in a phase 3 study (Table 3). In the validation datasets, c-statistics ranged from 0.70 to 0.93 (Table 3), and the predicted versus observed graft losses were not significantly different for iBOX assessments at 1 year posttransplant (Table 4).

TABLE 4. Poisson calibration for the full and abbreviated iBOX at 1 year posttransplant in the validation datasets.

Given that the iBOX models are trained primarily on subjects receiving CNI-based maintenance ISTs, it was unclear if iBOX would perform similarly in kidney transplant recipients not on CNI-based therapies. Internally, the iBOX was found to discriminate appropriately between higher- and lower-risk patients receiving mTOR inhibitor-based therapies (c-statistic >0.8) (Table 5). Externally, 5 years iBOX c-statistic values for CNI-free subjects, consisting primarily of patients on BELA-based regimens, at 1 year posttransplant in the validation datasets were evaluated; full and abbreviated iBOX c-statistics were 0.75 and 0.73, respectively (Table 6). These analyses demonstrate that iBOX can discriminate between subjects at higher and lower risk of death-censored graft loss in diverse datasets, including CNI and CNI-free populations, in clinical transplant centers and RCTs. Likewise, the results also showed that iBOX has good prediction accuracy based on calibration analyses (Table 6).

TABLE 5. Five-y posttransplant c-statistics values for the full iBOX for subset of subjects in the derivation dataset.

TABLE 6. Five-y posttransplant c-statistic values for the full and abbreviated iBOX for CNI and CNI-free subjects at 1 year posttransplant in the validation datasets.

The performance of the full and abbreviated iBOX were also tested in the validation datasets at 6 months and 2 years posttransplant. The 5 years posttransplant discrimination (Supplementary Table S5) and calibration analyses (Supplementary Table S6) support the inclusion of time posttransplant in the iBOX model at 6 months and 2 years posttransplant.

Based on the iBOX formulas shown in Table 7, iBOX is not just the sum of the parts (i.e., the addition of components) but includes continuous and dichotomous variables weighted differently based on the beta coefficients. The c-statistic for eGFR alone and eGFR with proteinuria in comparison with the full and abbreviated iBOX is shown in Table 8, with calibration results in Supplementary Tables S7, S8, indicating that the iBOX score is influenced most by eGFR, and the other 3 components, proteinuria, anti-human leukocyte antigen DSA, and biopsy, all increase the predictive power.

TABLE 7. Formulas to calculate full and abbreviated iBOX scores.

TABLE 8. C-statistics for each validation dataset as parameters are removed in the iBOX with all parameters (“full”), without biopsy (“abbreviated”), without biopsy and DSA (“only eGFR and proteinuria”), and without biopsy, DSA, and proteinuria (“only eGFR”).

In addition to validation, an analysis of the BENEFIT and BENEFIT-EXT RCTs included imputation of the worst-case iBOX scores at 1 year posttransplant for recipients who died or lost their graft in the first year (Table 9). This sensitivity analysis was performed to replicate the clinical trial setting where avoidance of survivor bias at 1 year would be necessary, and all randomized subjects would have an iBOX score at 1 year even if there were death or graft loss before that time. In both studies, the full and abbreviated iBOX score at 1 year was significantly lower in the BELA group than in cyclosporine, indicating a lower predicted risk of long-term graft failure. This corresponded to a statistically significant improvement in 5 years graft survival in the BENEFIT study. The BENEFIT-EXT study showed directionally higher 5 years death-censored graft survival. However, the difference was not statistically significant. The larger treatment difference in iBOX score at 1 year in the BENEFIT study compared with BENEFIT-EXT also corresponded to a larger treatment difference in graft survival. The lack of statistical significance on some of the 5 years graft survival analyses is related to limitations in the power to detect differences based on sample size.

TABLE 9. Treatment effect for 5 year graft survival with imputation (i.e., all-cause and death-censored) is the log HR, while the 1 year full and abbreviated iBOX scores are the difference in medians.

Additional analyses were performed testing the performance of the full iBOX at 1 years posttransplant on all-cause 5 years graft loss (Supplementary Tables S9, S10). The discriminatory ability of iBOX for all-cause graft loss underperforms, with the full iBOX having reduced c-statistics, many of which are below 0.7, and poor all-cause calibration. This is expected given that iBOX was originally developed using variables more likely to impact risk of graft loss. Based on this evidence, iBOX was qualified with EMA with death-censored graft loss as the outcome measure.

Sample Size Calculator Using iBOX Scores Using a Public-Facing Graphical User Interface

Separate from this EMA qualification submission, TTC developed a sample size calculator to assist sponsors in designing prospective clinical trials using iBOX as an endpoint. Sponsors can apply various inclusion/exclusion criteria and other specifications, consistent with the qualified COU, to calculate a sample size and project death-censored graft survival. This sample size calculator is publicly available at https://cpath.shinyapps.io/ibox_v3 to benefit the community and improve future clinical trial efficiency.

Conclusion and Future Directions

The successful qualification opinion of iBOX by EMA is the first step in the process of providing an endpoint to allow the demonstration of superiority of new therapies and to stimulate the development of innovative therapies in kidney transplant. Validation analyses show that iBOX is suitable for predictions of graft loss events, with good performance based on c-statistics and the ability to predict numbers of graft loss events with reasonable margins of error, supporting the qualified COU with EMA. Although the original iBOX by [7] focused on the prognostic value for individual patient decision making, the tool was able to be adapted for regulatory purposes as a qualified clinical trial endpoint (Supplementary Table S1). iBOX as a secondary endpoint was put forward by EMA to further stimulate robust assessment of iBOX and may lend future opportunities to advance iBOX for other COUs, such as treatment of T cell-mediated or antibody-mediated rejection trials. Although this is an important step forward, it will not automatically lead to new innovative therapeutic development but must be applied strategically as an important tool in global development programs to demonstrate advantages over current SOC, which has good short-term results and is available as lower-cost generics.

Importantly, EMA has a higher evidentiary standard for qualifying a surrogate endpoint compared with the FDA. Unlike the FDA, EMA does not have a category of “reasonably likely” surrogate endpoints (RLSE), nor is CMA linked to surrogacy [17, 18] whereas the FDA has both an RLSE and an accelerated approval pathway that is based on surrogate endpoints. To facilitate the harmonization of multinational trials, TTC submitted the iBOX as an RLSE to the FDA Biomarker Qualification Program, and it is currently under review by the FDA [19]. Recent TTC interactions with the FDA have focused on the needs of transplant recipients for new innovative therapeutics that have demonstrated superiority to the current SOC and the inadequacy of relying solely or primarily on the historical efficacy failure endpoint, which is driven by acute rejection. Ideally, we envision designing one phase 3 de novo trial with iBOX as a primary endpoint in the United States for Accelerated Approval (i.e., RLSE) and a secondary endpoint in the European Union after establishing noninferiority for efficacy failure, alongside pursuing CMA. The ability to conduct trials with sites in the United States and the European Union is critical to advancing the field and bringing new and improved therapies to kidney transplant recipients. As stated by the EMA in the qualification opinion, “The Committee for Medicinal Products for Human Use encourages the use of the iBOX scoring system as a secondary endpoint in future trials of kidney transplantation and further development of the scoring system targeting a potential future qualification as a surrogate endpoint” [10].

Data Availability Statement

The aggregated dataset that was the basis for the work discussed in this publication is not publicly available as per requirements in the data contribution agreements. Requests to access these datasets should be directed to corresponding author.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

Critical Path Institute is supported by the FDA of the US Department of Health and Human Services (HHS) and is 54.2% funded by the FDA/HHS, totaling $13,239,950, and 45.8% funded by nongovernment sources, totaling $11,196,634. TTC is supported by funds from the transplant community, including the biopharmaceutical and diagnostic industries, professional societies, and regulatory agencies, combined with support from academic institutions, ensuring that people living with a transplant are at the core of all we do.

Author Disclaimer

The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement by, FDA/HHS or the US Government.

Conflict of Interest

AK, LK, EF, and VA are employed by C-Path. AL holds shares in Cibiltech. IH holds research funds from MSD and Hansa Biopharma, receives consultancy honoraria from Hansa Biopharma, MSD, Novartis, and Takeda, is an associate editor for American Journal of Transplantation, and is a Coordinating Committee member of TTC. H-UM-K is employed by Veloxis Pharmaceuticals. KN serves as a scientific advisory board member for Angion, Care Dx, CSL Behring, Hansa Biopharma, Immucor, Novartis, Sangamo Therapeutics, Sanofi, Takeda, Talaris, and Viela Bio. RM is employed by University of Nebraska Medical Center; serves as a consultant for Chinook Therapeutics and Olaris Inc; holds research funding from Transplant Genomics, Inc and Verici DX; has received honoraria from CSL Behring; holds patents or royalties with Eurofins; has an advisory or leadership role with Vitaerris VKTX01 IMAGINE Trial and Verici Dx; and is Chair, ASN Policy and Advocacy Committee; Immediate Past-Chair, Women in Transplantation; Member, ASN Grants Committee; Chair, SRTR Review Committee; Member, DSMB, NIDDK/NIH; and Deputy Editor, American Journal of Transplantation. AG receives research funding from Hansa Biopharma, Veloxis Pharmaceuticals, Novartis, and Medeor Therapeutics. WF serves on the Board of Directors of CTI Clinical Trial Services; is Adjunct Professor at University of Illinois at Chicago; is founder of Tutela Pharmaceuticals; and serves as a consultant to Azoth Immune Medicines, Tract Therapeutics, and Veloxis Pharmaceuticals.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

C-Path and TTC gratefully acknowledge the support and contributions of all the individuals who participate in research and our partners who have dedicated the time and energy required to generate innovations that advance medical product development and improve care for patients.

TTC is supported by funds from the transplant community, including the biopharmaceutical and diagnostic industries, professional societies, and regulatory agencies, combined with support from academic institutions, ensuring that people living with a transplant are at the core of all we do.

The authors would like to thank the TTC members for their guidance and support. The BENEFIT and BENEFIT-EXT RCTs were conducted by the BENEFIT and BENEFIT-EXT investigators and supported by Bristol Myers Squibb. The data from BENEFIT and BENEFIT-EXT reported here were supplied by Bristol Myers Squibb. This manuscript was not prepared in collaboration with investigators of the BENEFIT and BENEFIT-EXT RCTs and does not necessarily reflect the opinions or views of these 2 studies or Bristol Myers Squibb.

TTC recognizes the Paris Transplant Group for their development of iBOX, data sharing, and support. C-Path compensated OrganX to provide biostatistical support to C-Path for iBOX qualification with the EMA.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontierspartnerships.org/articles/10.3389/ti.2023.11951/full#supplementary-material

Abbreviations

BELA, belatacept; CMA, conditional marketing authorization; CNI, calcineurin inhibitor; COU, context-of-use; DSA, donor-specific antibody; eGFR, estimated glomerular filtration rate; EMA, European Medicines Agency; FDA, Food and Drug Administration; IST, immunosuppressive therapy; RCT, randomized controlled trial; RLSE, reasonably likely surrogate endpoint; SOC, standard of care; TTC, Transplant Therapeutics Consortium.

References