close

Вход

Забыли?

вход по аккаунту

?

Diagnosis and prognosis unclear for scientific manuscripts.

код для вставкиСкачать
MESSAGE FROM THE EDITOR
Diagnosis and Prognosis Unclear for
Scientific Manuscripts
D
iagnostic and prognostic tools are important,
whether they be created from standard components
of clinical history and exam or derived from imaging,
cerebrospinal fluid, or gene expression arrays. As neurologists know too well, clinical diagnosis is imperfect, sometimes with surprisingly poor reliability and validity of
well established tests1 and poor agreement on final diagnoses even for common conditions.2 Inefficient
approaches to diagnosis drive up healthcare costs and
missed diagnoses are even more expensive and with no
gain. Accurate diagnosis can reduce cost by getting
patients appropriately managed sooner, which may also
increase the impact of therapy on health by averting early
and irreversible manifestations of disease. Prognosis is
also important to patients and their families as they prepare for consequences, and it may alter treatment decisions to avoid unnecessary or inconsequential care.
Finally, improvements in diagnosis and prognosis make
development of treatments easier by increasing power
since those who are unlikely to benefit from treatment
can be more efficiently excluded from study.
The Annals of Neurology is committed to publishing
studies that convincingly demonstrate the accuracy of important diagnostic and prognostic tools. In fact, 28% of
articles published in 2010 broadly covered topics of diagnosis and 22% were related to prognosis (Figure). Some
of our most highly cited studies are also in this category,
including the report on cerebrospinal fluid biomarkers
for Alzheimer’s disease from the Alzheimer’s Disease
Neuroimaging Initiative.3 We also reject many more than
we accept in these categories. The bar for convincing
demonstration is a high one, and for good reason.
The primary goal of the Annals is to advance
knowledge. As we have commented in these pages previously, publishing irreproducible results is a constant concern. Although we might generate more traffic by publishing articles with dramatic claims based on limited
data, our brand and our mission is based on being authoritative and accurate.
Unfortunately, forces are lined up against us. Simulation studies suggest that most published research is
incorrect.4 Investigators are, by nature, biased toward
producing stunning findings that break common conven-
tion. Journals also compete for attention; they are
attracted to paradigm-changing discoveries and may
reject negative studies even when these fail to confirm
previously published claims. The culture of research permits investigator discretion that can influence p-values.
Such discretion spans the research process, from whether
to exclude a patient or animal that just did not fit the
pattern, whether to adjust for a specific group of potential confounders, or whether to use a continuous or dichotomous variable (with an ‘‘arbitrary’’ cut point). Given
investigators’ biases and the freedom to make decisions
that shrink p-values, it is not surprising that results are
positive much more frequently than they should be. In
addition, investigators may test a large number of associations, only a fraction of which are positive (hopefully
more than 5%) and make their way to manuscript submission. Many studies cannot be reproduced and, over
time, previously-demonstrated associations often become
weaker and weaker, as recently illustrated in the declining
efficacy in trials of atypical antipsychotics.5
The problem with irreproducibility is well recognized for epidemiological and genetic association studies.
As recently commented in these pages, epidemiology has
become progressively more susceptible to irreproducibility as the size and detail of available databases continues
to increase, a problem that is likely to grow dramatically
with utilization of electronic health records for
research.6,7 Even more dramatic, examples of failure to
confirm findings from genetic studies have become so
widespread that extremely stringent, conservative criteria
are now generally required to support claims of true association, especially for genome-wide explorations. Letters
and brief communications that fail to validate findings
published in the Annals in earlier years continue to be
submitted. In response, the Annals has developed policies
that require independent validation of any association
demonstrated and mandate authors to report all associations evaluated.8 Although this doesn’t eliminate the
problem of publishing irreproducible genetic associations,
it should help.
Over the past several years, we have been informally
applying these same criteria to non-genetic diagnostic
and prognostic studies, though with a little more discretion for hypothesis generation. We now want to
C 2011 American Neurological Association
V
A9
ANNALS
of Neurology
Requirements for epidemiological studies in the
Annals of Neurology.
1. Authors must report the number of associations
studied. How many different exam findings, protein markers, imaging regions, or other independent associations were evaluated to find the
reported association?
2. Authors must have an explicit approach to multiple testing. A Bonferroni test may not be
required for all the associations tested but some
correction and discussion should always be present. If the study is judged not to require correction for multiple testing, claims of clinical utility
probably are not justified.
3. Authors must validate the sensitivity and specificity of the test in an independent cohort. It
should be clear that the tool was not altered
between the derivation testing and its validation
(i.e., no editing of the tool after viewing data on
its validation). Internal validation by bootstrap or
jackknife methods is no substitution for an external validation, but cohort splitting is acceptable
as long as the split was defined a priori.
FIGURE 1: Articles published in the Annals of Neurology
that broadly relate to diagnosis (blue) and prognosis (red)
during the last 6 years, as a percentage of all articles
published. Classification was based on criteria available
through Pubmed’s Clinical Queries filters.
formalize our approach to these studies (Box). Thus, for
all studies in which a new diagnostic or prognostic tool
is being proposed, we request the following information
be included in the manuscript. First, authors should
report the number of associations studied. How many
different exam findings, protein markers, imaging
regions, or other independent associations were evaluated
to find the reported association? Second, authors should
have an explicit approach to multiple testing. A Bonferroni test may not be required for all the associations
tested but some correction and discussion should always
be present. If the study is judged not to require correction for multiple testing, claims of clinical utility probably are not justified. Third, authors should validate the
sensitivity and specificity of the test in an independent
cohort. It should be clear that the tool was not altered
between the derivation testing and its validation (ie, no
editing of the tool after viewing data on its validation).
Internal validation by bootstrap or jackknife methods is
no substitution for an external validation but cohort
splitting is acceptable as long as the split was defined a
priori. Hypothesis generating studies that reveal important aspects of the underlying biology may still be considered but it should be clear throughout the manuscript
that the findings are not useful for clinical purposes.
We realize this sets a high bar but we believe such a
barrier is appropriate for any tool that is being suggested
for clinical use. Changing clinical practice is an essential
A10
function of the Annals, but change is slow and costly, justifying a particularly high level of evidence.
S. Claiborne Johnston, MD, PhD
Stephen L. Hauser, MD
Editors
References
1.
Miller TM, Johnston SC. Should the Babinski sign be part of the
routine neurologic examination? Neurology 2005;65:1165–1168.
2.
Kraaijeveld CL, van Gijn J, Schouten HJ, Staal A. Interobserver
agreement for the diagnosis of transient ischemic attacks. Stroke
1984;15:723–725.
3.
Shaw LM, Vanderstichele H, Knapik-Czajka M et al. Cerebrospinal
fluid biomarker signature in Alzheimer’s disease neuroimaging initiative subjects. Ann Neurol 2009;65:403–413.
4.
Ioannidis JP. Why most published research findings are false.
PLoS Med 2005;2:e124.
5.
Lehrer J. The truth wears off. The New Yorker. New York, 2010.
6.
Johnston SC, Hauser SL. The challenge of publishing newsworthy
epidemiology. Ann Neurol 2010;68:A8–10.
7.
Johnston SC, Hauser SL. Epidemiology in the Annals: Part of the
problem or the solution? Ann Neurol 2007;62:A8–A9.
8.
Oksenberg JR, Hauser SL. Neurogenetics in the Annals: Dealing
with complexity. Ann Neurol 2008;63:A11–A14.
DOI: 10.1002/ana.22374
Volume 69, No. 2
Документ
Категория
Без категории
Просмотров
1
Размер файла
92 Кб
Теги
scientific, unclear, manuscripts, prognosis, diagnosis
1/--страниц
Пожаловаться на содержимое документа