↓ Skip to main content

Cochrane Database of Systematic Reviews

Individual participant data meta‐analyses compared with meta‐analyses based on aggregate data

Overview of attention for article published in Cochrane database of systematic reviews, September 2016
Altmetric Badge

About this Attention Score

  • In the top 5% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (95th percentile)
  • High Attention Score compared to outputs of the same age and source (80th percentile)

Mentioned by

news
1 news outlet
blogs
2 blogs
policy
1 policy source
twitter
47 X users
facebook
3 Facebook pages

Citations

dimensions_citation
118 Dimensions

Readers on

mendeley
137 Mendeley
Title
Individual participant data meta‐analyses compared with meta‐analyses based on aggregate data
Published in
Cochrane database of systematic reviews, September 2016
DOI 10.1002/14651858.mr000007.pub3
Pubmed ID
Authors

Catrin Tudur Smith, Maura Marcucci, Sarah J Nolan, Alfonso Iorio, Maria Sudell, Richard Riley, Maroeska M Rovers, Paula R Williamson

Abstract

Meta-analyses based on individual participant data (IPD-MAs) allow more powerful and uniformly consistent analyses as well as better characterisation of subgroups and outcomes, compared to those which are based on aggregate data (AD-MAs) extracted from published trial reports. However, IPD-MAs are a larger undertaking requiring greater resources than AD-MAs. Researchers have compared results from IPD-MA against results obtained from AD-MA and reported conflicting findings. We present a methodology review to summarise this empirical evidence . To review systematically empirical comparisons of meta-analyses of randomised trials based on IPD with those based on AD extracted from published reports, to evaluate the level of agreement between IPD-MA and AD-MA and whether agreement is affected by differences in type of effect measure, trials and participants included within the IPD-MA and AD-MA, and whether analyses were undertaken to explore the main effect of treatment or a treatment effect modifier. An electronic search of the Cochrane Library (includes Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, CENTRAL, Cochrane Methodology Register, HTA database, NHS Economic Evaluations Database), MEDLINE, and Embase was undertaken up to 7 January 2016. Potentially relevant articles that were known to any of the review authors and reference lists of retrieved articles were also checked. Studies reporting an empirical comparison of the results of meta-analyses of randomised trials using IPD with those using AD. Studies were included if sufficient numerical data, comparing IPD-MA and AD-MA, were available in their reports. Two review authors screened the title and abstract of identified studies with full-text publications retrieved for those identified as eligible or potentially eligible. A 'quality' assessment was done and data were extracted independently by two review authors with disagreements resolved by involving a third author. Data were summarised descriptively for comparisons where an estimate of effect measure and corresponding precision have been provided both for IPD-MA and for AD-MA in the study report. Comparisons have been classified according to whether identical effect measures, identical trials and patients had been used in the IPD-MA and the AD-MA, and whether the analyses were undertaken to explore the main effect of treatment, or to explore a potential treatment effect modifier.Effect measures were transformed to a standardised scale (z scores) and scatter plots generated to allow visual comparisons. For each comparison, we compared the statistical significance (at the 5% two-sided level) of an IPD-MA compared to the corresponding AD-MA and calculated the number of discrepancies. We examined discrepancies by type of analysis (main effect or modifier) and according to whether identical trials, patients and effect measures had been used by the IPD-MA and AD-MA. We calculated the average of differences between IPD-MA and AD-MA (z scores, ratio effect estimates and standard errors (of ratio effects)) and 95% limits of agreement. From the 9330 reports found by our searches, 39 studies were eligible for this review with effect estimate and measure of precision extracted for 190 comparisons of IPD-MA and AD-MA. We classified the quality of studies as 'no important flaws' (29 (74%) studies) or 'possibly important flaws' (10 (26%) studies).A median of 4 (interquartile range (IQR): 2 to 6) comparisons were made per study, with 6 (IQR 4 to 11) trials and 1225 (542 to 2641) participants in IPD-MAs and 7 (4 to 11) and 1225 (705 to 2541) for the AD-MAs. One hundred and forty-four (76%) comparisons were made on the main treatment effect meta-analysis and 46 (24%) made using results from analyses to explore treatment effect modifiers.There is agreement in statistical significance between the IPD-MA and AD-MA for 152 (80%) comparisons, 23 of which disagreed in direction of effect. There is disagreement in statistical significance for 38 (20%) comparisons with an excess proportion of IPD-MA detecting a statistically significant result that was not confirmed with AD-MA (28 (15%)), compared with 10 (5%) comparisons with a statistically significant AD-MA that was not confirmed by IPD-MA. This pattern of disagreement is consistent for the 144 main effect analyses but not for the 46 comparisons of treatment effect modifier analyses. Conclusions from some IPD-MA and AD-MA differed even when based on identical trials, participants (but not necessarily identical follow-up) and treatment effect measures. The average difference between IPD-MA and AD-MA in z scores, ratio effect estimates and standard errors is small but limits of agreement are wide and include important differences in both directions. Discrepancies between IPD-MA and AD-MA do not appear to increase as the differences between trials and participants increase. IPD offers the potential to explore additional, more thorough, and potentially more appropriate analyses compared to those possible with AD. But in many cases, similar results and conclusions can be drawn from IPD-MA and AD-MA. Therefore, before embarking on a resource-intensive IPD-MA, an AD-MA should initially be explored and researchers should carefully consider the potential added benefits of IPD.

X Demographics

X Demographics

The data shown below were collected from the profiles of 47 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 137 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
United States 1 <1%
Canada 1 <1%
Unknown 135 99%

Demographic breakdown

Readers by professional status Count As %
Student > Master 20 15%
Student > Ph. D. Student 18 13%
Student > Bachelor 15 11%
Researcher 13 9%
Professor > Associate Professor 5 4%
Other 19 14%
Unknown 47 34%
Readers by discipline Count As %
Medicine and Dentistry 34 25%
Nursing and Health Professions 10 7%
Pharmacology, Toxicology and Pharmaceutical Science 10 7%
Psychology 6 4%
Biochemistry, Genetics and Molecular Biology 5 4%
Other 16 12%
Unknown 56 41%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 50. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 05 March 2023.
All research outputs
#849,668
of 25,457,297 outputs
Outputs from Cochrane database of systematic reviews
#1,641
of 11,499 outputs
Outputs of similar age
#15,978
of 345,055 outputs
Outputs of similar age from Cochrane database of systematic reviews
#44
of 222 outputs
Altmetric has tracked 25,457,297 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 96th percentile: it's in the top 5% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 11,499 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 40.0. This one has done well, scoring higher than 86% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 345,055 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 95% of its contemporaries.
We're also able to compare this research output to 222 others from the same source and published within six weeks on either side of this one. This one has done well, scoring higher than 80% of its contemporaries.