Importantly, many neuroscience endpoints using brain imaging require 400 subjects, a sample size that is not achievable for any single clinical center. This limitation can be overcome through multi-site recruitment, allowing researchers to examine imaging outcomes in larger study populations.
An MRI scan may look a little like a photograph, but in a clinical trial it acts as a series of measurements. By assigning a number to every voxel, it enables researchers to compute outcomes instead of just visualizing images. Analysing multi-site trials alters the context in which we sample.
What all this means, of course, is that each center has its own scanner/coil/console/technologists and operating practices which are capable of affecting the results independently from biology. So the challenge is more about grouping scans, but also finding out where cross-site variability starts.
Where Cross-Site Variability in Imaging Data Begins
Unlike measurement of a purely visual material (say, an image of the brain), you are not trained on qualitative data but rather quantitative data. Anatomy is characterized by structural MRI, and changes like the BOLD contrast can be measured using fMRI or functional MRI. Not surprisingly, outputs from image-derived data also rely on repeatable acquisition and analysis, as evidenced by high-plex imaging insights.
Scanner Hardware, Vendors and Field Strength
In particular, structural MRI data acquired at 1.5T may differ from that obtained at 3T (Holmes et al., in press), and other factors such as vendors, coil channel counts and gradient performance can cause differences in image quality, signal-to-noise ratios and derivative measures independent of the participant.
Cortical thickness derived from FreeSurfer segmentation or BOLD contrast from an fMRI run can be biased as any instrument reading. Multi-vendor research that has been published has highlighted scanner-related differences that are substantial compared to the effect sizes that trials are designed to identify. Hardware variances without proper controls can mimic biological or treatment-related changes.
Sequence Drift, Operators and Participant Factors
Inter-site variability can occur during a study as well. Software upgrades, replacement/new coils and shimming changes may segregate scans from a single site into separate populations even if the acquisition remains nominally unchanged as per study protocol.
Then operator and participant factors account for more variation. Results can be influenced by slice positioning, phrasing of instructions, coffee intake, time needed to scan an MRI and movement during the survey. While head-motion correction alleviates one source of noise, it cannot reconcile all inconsistencies.
Such differences can appear in the final dataset as site effects rather than procedural effects since practices tend to cluster within sites.
Harmonizing Protocols Before the First Scan
Although providing the trials can limit some scanner variability during acquisition, it is impossible to fully compensate for this afterward. This distinction is further supported by multi-vendor, multi-center reproducibility studies demonstrating that MRI measurements cannot be simply considered as freely exchangeable between sites.
This indicates that protocol harmonization should be a design feature of measurement rather than an administrative necessity. Stricter controls are especially relevant when changes in structural measures are gradual, as the biological effect may be small compared to variability between scanners.
This includes research in Alzheimer’s disease and Parkinson’s disease, but also more recently Huntington’s Disease neuroimaging, whereby the modifications related to equipment must be independent from small longitudinal changes.
Phantoms and Traveling Subjects
Stability: Geometric and contrast phantoms provide reference objects to monitor changes in distortion, intensity, or uniformity of the signal. They are scanned at regular intervals to quantify drift and enable analysts to correlate a drop in data quality with a coil replacement, software update or similar.
Traveling-subject designs provide another comparison. Adding structural MRI within different sites on the same volunteers makes it possible to estimate how much of the variation in region-of-interest measurements derives from scanner differences when participants are held constant. That then directly informs correction methods while testing reproducibility and validity.
Locking the Acquisition Protocol Across Sites
A shared imaging manual prior to enrollment should outline sequence parameters, voxel size and orientation, anatomical coverage, repeat-scan processes, and acceptable equivalent vendors.
Decisions delegated to local teams can introduce protocol discrepancies which later manifest as biological noise. Documentation alone is insufficient. Site-qualification scans determine whether or not each scanner is capable of reproducing the acquisition which is necessary.
The standardization of positioning, sequence selection, and instructional wording is what technologist training does. This qualification must also confirm that the entire anatomy is covered, free of wrapping or any avoidable artifacts.
After enrollment commences, records of formal deviations then assist analysts in discriminating between one-off errors and changes sustained to site performance.
Single Pipeline, Standard Set of Quality Checks
Aligned acquisitions themselves may also become desynchronous in processing. As a result, a centrally managed workflow would extract measurements from the same underlying raw scans using the same software and templates (with equivalent decision rules) at all sites in the trial.
Hidden technical differences can enter the endpoint if sites process some data locally prior to pooling outputs.
A Common Template with Shared Preprocessing
FSL (FMRIB Software Library) and FreeSurfer Metrics: Measurements from FSL or FreeSurfer can only be comparable if all scans use the same pinned software version and configuration.
Impact of release updates on segmentation and volume calculations for cortical or subcortical regions can confound measurements even when no biological change has occurred. This requires a standard, such as the MNI ICBM 2009c template for spatial normalization.
By mapping all participants into the same space, we can make corresponding voxels in different scans comparable; template mixing would destroy that comparability. Therefore, the pipeline needs to use the same pre-processing procedures and head-motion correction thresholds, as well as subject-exclusion criteria on every scan.
Alternatively, a center with more movement may seem to cater to another type of patient.
Central Reading, Quality Control and Data Sharing
Shifting the reliance for verification away from data production: central quality control. The presence of incomplete coverage, wrapping, coil faults, malpositioning and failed segmentation can be recognized by blinded reviewers irrespective of the treatment allocation.
By maintaining the availability of participants, one can also rerun a poorly executed scan. Automatic checks and AI-assisted image analysis can identify peaks in intensity or processing failures, but borderline cases still need to be routinely reviewed by a human.
The same acceptance criteria should be used across all sites. Add raw data, pipeline versions, processing logs and unthresholded statistical maps to the records that are shared.
Our proposed revisions could be inspired by large-scale datasets, such as the UK Biobank — with common conventions in naming and metadata/provenance details.
Model Site Effects Rather Than Ignoring It
The only difference from a single-site power calculation is that adding sites increases sample size, but multi-site variance differs. In frequentist statistics, adding more participants will serve to improve statistical inference; yet as the effect of scanner and site remains strong, each additional participant provides less information.
This is not to say that assuming scale alone increases effect size or statistical power, although benefit comes from controlling variance. In accordance with the approach, site needs to be represented explicitly in analysis.
General linear model (GLM): By estimating differences between known centers, each site can be treated as a fixed covariate. Given sufficient sites, a random intercept in a mixed-effects model can thus provide an estimate of the distribution of site-level variation rather than an individual site correlation.
Phantom and traveling-subject measurements can inform post-hoc harmonization of derived measures. It aims to correct for effects of the scanner without artificially bringing genuine group differences into equality. At each site, randomization also ensures that treatment assignment is not confounded with specific scanners.
Longitudinal imaging allows for group-level conclusions and predictive modeling, but does not yield individualized diagnoses a priori. The former, for instance, cannot be diagnosed from a single scan alone.
MRI may also detect incidental structural findings that have no bearing on the trial’s endpoints. Such observations do not change the analysis after viewing data, as they are made out in prespecified endpoints and review procedures.
Consistency Is a Design Decision, Not an Accident
Only when harmonized acquisition, central processing, standardized quality-control operations, as well as site-effects statistical strategies are determined prior to enrollment do multi-site trials yield higher-quality brain imaging data.
These controls work to reduce variability that can be seen during analysis, but retrofitting cannot fully recover information lost through incompatible scans or undocumented changes in the specimen.
Scale makes comparative patterns more difficult, but allows for greater reproducibility and validity in combination with control. The final imaging endpoint better characterizes the biological variation across the wider study sample as opposed to being determined by equipment, practices or habits of specific sites.
