C Class

Types of selective reporting
Selective Outcome
Reporting
Kerry Dwan1, Jamie Kirkham1, Carrol Gamble1, Paula R
Williamson1, Doug G Altman2
SMG Training Course 2010
1University

of Liverpool, UK, 2University of Oxford, UK

outcomes
 subgroups
 adjusted versus unadjusted results
 prognostic or risk factors
 first period results in crossover trials
 PP rather than ITT
 complete case versus LOCF versus other
methods


kerry.dwan@liverpool.ac.uk

Types of selective outcome reporting

Outcome Reporting Bias


Definition: Selection of a subset of the original recorded
outcomes, on the basis of the results, for inclusion in
publication



Selective reporting of the set of study outcomes




Not all analysed outcomes are reported

Selective reporting of a specific outcome
Hutton and Williamson (2000)
Selection from multiple time points
 Subscales
 Endpoint score versus change from baseline
 Continuous versus binary (choice of cut-offs)
 Different measures of same outcome, e.g. pain




Statistically significant outcomes more likely to be fully
reported: OR 2.2 to 4.7 (Dwan et al, 2008)



Potential threat to validity of systematic review / metaanalysis. Potentially a missing data problem if measured
and analysed but not reported – similar impact to
publication bias i.e. non-publication of whole studies





Incomplete reporting of a specific outcome


e.g. “Not significant” or “p>0.05”

Approved
application

Impact of ORB
Completed

Stopped
early

(not necessarily
meeting target
recruitment)

Interim
analysis

Never
started

Other e.g. poor
recruitment

Not
submitted

Submitted

Published

Not
published

Not
accepted

Abstract
only

Missing
outcome
data

Some
outcomes

Full
publication

All
outcomes

Assessment within review

OR 1.41 (1.04,1.91)

Trial ID
(author,
date of publication)

OR 1.55 (1.13,2.14)

Review primary
outcome
Overall survival

Review Outcomes

Other Trial Outcomes

Event-free
survival

Overall
remission rate

Relapse
rate

Toxicity...