Search results
The search of the bibliographic databases resulted in the inclusion of 14,586 records (Fig. 1). The last search was performed on 9 September 2021.
PRISMA flow-chart showing the study selection process.
After duplicate removal, title/abstract screening was performed, resulting in 823 included records which underwent full-text screening. After the full-text screening, 33 results were included in the present systematic review. Fifteen were reports of trials’ results [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24], four were trials’ protocols [25,26,27,28] and fourteen were trials’ registrations.
Among the included records, there was a match between three trials’ protocols [26,27,28] and the corresponding trials’ registrations, and between two reports of trials’ results [16, 22] and the corresponding trials’ registrations.
Among trials’ results reports and protocols, nine were published in oncology journals [11, 12, 14, 15, 18, 19, 22,23,24], three in experimental and research medicine journals [25,26,27], two in internal and general medicine journals [10, 16]. The other five records were published in specialised journals in other areas of medicine, including clinical neurology [17], respiratory system [20] and peripheral vascular diseases [21]. One study was published in a nursing journal [28], and only one study was published in a statistics & probability journal [13].
All studies were found to present with some concerns at risk of bias assessment (Table S2, Supplementary Material), except for that of Marshall et al. [21].
Trials’ results
Fifteen studies presenting trials’ results were included in the present work. Table S3, Supplementary material, presents the detailed characteristics of the studies included.
Eight trials were located in the EU and 5 in North America. Thirteen out of fifteen were multicenter, and half (8 out of 15) received public or private funding. The first study was published in 1992. Six trials were published between 2010–2021, three between 2000–2009 and another six in the period 1990–1999.
Two studies [13, 22] presented a reanalysis of previously published data; for what concerns that of Wang et al. [13], the study presenting the first analysis of the data was included in this review [12], whereas for the one by Petracci et al. [22], it was included the one reporting the reanalysis.
Furthermore, two studies presented the same trial’s short- and long-term results [10, 11].
All the trials tested chemo/radio/hormone therapy for cancer treatment, including lung cancer, neuroblastoma, glioblastoma, pancreatic cancer, breast cancer, prostate cancer, colorectal cancer and recurrent venous thrombosis in solid tumours.
Interestingly, only one study out of the fifteen included was reported to have a SMART design [13]. All the trials were characterised by a two-stage design (Table 1 for detailed study design). The decision rule was most frequently based on the response to first-stage treatment. The objectives reported by most of the studies identified were to compare first and second-stage treatments separately or only first or second-stage treatments, except for Petracci et al. [22], Thall et al. [12] and Wang et al. [13]. The authors of these studies explicitly declared in the manuscript that the study’s objective was to identify the best treatment regimen resulting from the multiple assignments.
Eight studies did not report sample size calculation. Those reporting sample size calculations did not take into account the multiple assignments in the sample size estimation. Generally, the sample size was provided for each stage, or the powering of the study was made on one of the two stages and inflating according to the expected proportion of subjects entering the second randomisation. Of notice, Tummarello et al. [24] declared that the number of people entering the second randomisation was too small to allow groups’ comparison. Marshall et al. [21] and Bianchi et al. [14] underwent premature closure because of the low recruitment rate.
Regarding the data analysis (Table 2 for study details), the approaches most frequently used were the Kaplan–Meier method and the Cox Proportional Hazard model since most trials considered a time-to-event endpoint (overall or progression-free survival). Matthay et al. [10, 11] used such analysis approaches to compare the treatment regimens resulting from the two-stage randomisation among subjects entering the second randomisation. In all other trials, separate analyses of first and second-stage treatments were carried out, except for Petracci et al. [22], Thall et al. [12] and Wang et al. [13], which were interested in identifying the best treatment regimen. These authors adopted three different strategies of analysis to estimate the treatment effect taking into account patients’ baseline characteristics and outcome history throughout the trial. The analysis of Petracci et al. [22] involved the estimation of Inverse Probability of Censoring Weighting (IPCW) to account for selection bias resulting from patients’ selection in the second stage. Thall et al. [12] used a conditional logistic regression approach. Wang et al. [13] proposed the estimation of Inverse Probability Treatment Weighting (IPTW) for the reanalysis of Thall et al. [12].
Trials’ protocols
The review included four trial protocols [25,26,27,28]. They were all located in the USA and published after 2009. Three [26,27,28] out of four corresponded to trials’ registrations included in the present review. All the study’s protocols received funding, and two were multicenter (Table S3 for trials’ protocols characteristics).
No trials were aimed at testing chemo/radiotherapy for cancer treatment. Two tested interventions to reduce cancer symptoms in patients with different types of solid tumours [28] and breast cancer [27]. One tested pharmacological treatment for depression in melanoma patients undergoing IFN-alpha therapy [25]. Finally, that of Fu et al. [26] was aimed at lung cancer prevention through a smoking cessation programme.
In all four protocols included, the design was defined to be SMART. They were all based on two stages, and the decision rule was based on the response to the first-stage treatment (Table 1). All the protocols were declared to be aimed at identifying the optimal treatment strategy. However, it is worth pointing out that only one protocol presented the identification of the optimal treatment strategy as the study’s primary objective [25].
All protocols reported the sample size calculation, but the power analysis was based only on one of the two stages in three out of four records. Only Auyeung et al. [25] proposed an approach accounting for the two-stage design. Not least, Fu et al. [26], in the last trial’s update published within the trial registration, declared that a sample size reassessment was done to account for the low enrolment rate.
For data analysis (Table 2), all protocols planned to use traditional statistical tests and regression-based analyses to compare first and second-stage treatment separately. Furthermore, Sikorsii et al. [28], Kelleher et al. [27] and Auyeung et al. [25] proposed three different analysis approaches to identify the optimal treatment strategy. Auyeung et al. [25] proposed using marginal mean models to estimate the mean outcome for each regimen. Sikorsii et al. [28] declared that the optimal intervention sequence will be identified through Q-learning algorithm, including two Q functions considering patients and their caregivers’ baseline characteristics and history through the two stages. Also, Kelleher et al. [27] planned the use of the Q-learning algorithm and value search estimation. No technical details about models’ estimations were provided.
Trials’ registrations
Fourteen trials’ registrations were included in the review. Five referred to already included trials’ results [16, 22] and protocols [26,27,28]. All registrations were made after 2008, twelve were retrieved on clinicaltrials.gov, one from australianclinicaltrials.gov and one from the Clinical Trials Peruvian Registry. Half of the studies were located in North America.
Only five out of 14 trials (36%) were aimed at testing cancer chemo/radio treatments on overall survival or disease-free survival of patients with pancreatic cancer (3 registrations), colorectal cancer (1 registration) and neuroblastoma (1 registration). Six trials tested treatments for cancer and cancer treatment symptoms, such as fatigue, pain, sensory symptoms, depression, anxiety and quality of life, in patients with breast cancer or solid tumours and their caregivers. One trial’s registration was aimed at improving the management of cardiovascular comorbidities in cancer patients, and another one at testing interventions for COVID-19 prevention and treatment in cancer patients. Finally, one trial was aimed at cancer prevention (lung cancer), through a programme for smoking cessation, corresponding to the registration of the trial protocol published by Fu et al. [26].
Interestingly, all but five registrations referred to the study design as a SMART one. All designs were two-stage based, except for two studies. One included three stages, but only one decision-rule-based randomisation was specified (from the second to the third stage), while in the other trial, the number of stages depended on the patients’ COVID-19 status (no exposure, exposure to COVID-19, moderate or severe COVID-19 infection).
No information is reported regarding sample size calculation and data analysis because the statistical analysis plan was not available in almost all trials’ registrations.
Detailed characteristics of each one of the trials’ registrations included in the review are reported in Table S4, Supplementary Material.