вход по аккаунту


The airheum knowledge-based computer consultant system in rheumatology. performance in the diagnosis of 59 connective tissue disease patients from japan

код для вставкиСкачать
Performance in the Diagnosis of 59 Connective Tissue
Disease Patients from Japan
AI/RHEUM is a knowledge-based computer consultant system for the diagnosis of rheumatic diseases.
Its diagnostic accuracy was evaluated using information
that was supplied by Japanese rheumatologists on 59
patients with connective tissue diseases. The diagnoses
of the AURHEUM model were in full or partial agreement with those of the Japanese rheumatologists in 54 of
59 cases (92%). Preliminary evaluation of the criteria
used by the model to diagnose mixed connective tissue
disease showed a sensitivity of 90% and a specificity of
AURHEUM is a computer consultant system
that uses artificial intelligence techniques to model the
consultative behavior of the expert rheumatologist in
From the University of Missouri-Columbia Multipurpose
Arthritis Center, and the Department of Internal Medicine, Keio
University, Tokyo, Japan.
Supported by NIH-NIAMS Multipurpose Arthritis Center
grant 2P60, AR-20658.
James F. Porter, MD: University of Missouri-Columbia
Multipurpose Arthritis Center; Lawrence C. Kingsland, 111, PhD:
University of Missouri-Columbia Multipurpose Arthritis Center (current address: National Library of Medicine, Bethesda, MD); Donald
A. B. Lindberg, MD: University of Missouri-Columbia Multipurpose
Arthritis Center (current address: National Library of Medicine,
Bethesda, MD); lndravadan Shah, MD: University of MissouriColumbia Multipurpose Arthritis Center; James M. Benge, MD:
University of Missouri-Columbia Multipurpose Arthritis Center;
Susan E. Hazelwood, BS: University of Missouri-Colurnbia Multipurpose Arthritis Center; Donald R. Kay, MD: University of Missouri-Columbia Multipurpose Arthritis Center; Mitsuo Homma, MD:
Keio University; Masashi Akizuki, MD: Keio University; Makoto
Takano, MD: Keio University; Gordon C. Sharp, MD: University of
Missouri-Columbia Multipurpose Arthritis Center.
Address reprint requests to Gordon C. Sharp, MD, Division
of Immunology and Rheumatology, MA427 Medical School Addition, I Hospital Drive, Columbia, MO 65212.
Submitted for publication March 3. 1986; accepted in revised form July 9. 1987.
Arthritis and Rheumatism, Vol. 31, No. 2 (February 1988)
the diagnosis of rheumatic diseases (1,2). The system’s
current knowledge base includes diagnostic criteria
tables for the 26 rheumatic diseases shown in Table 1.
The criteria tables, which represent the knowledge
that is the basis of the AI/RHEUM reasoning process, were developed at the University of MissouriColumbia and were reviewed externally by American
rheumatology consultants (3). The tables are structured as combinations of major and minor decision
elements, required elements, and exclusions which
lead the system to diagnostic conclusions categorized
as definite, probable, or possible.
The AURHEUM system is a consultative tool
for use by physicians who do not have specialty training
in rheumatology. To address the problem of potentially
inaccurate input that might mislead the system, AI/
RHEUM makes available on line more than 180 definitions of items from the patient data checklist. Many of
these definitions oEer information in 4 categories:
WHAT (is the observation), WHY (is it being requested), HOW (is the observation performed), and
REFS (specific citations from the literature).
The physician user enters information into the
system in terms of positive, negative, unknown, and
numeric findings. The AI/RHEUM reasoning process
leads the system to conclusions that are categorized as
definite, probable, or possible and are presented in a
differential diagnosis. For each component of the
differential, the system presents a statement of the
findings for the patient that support the conclusion, a
statement of currently unknown findings which, if
known and positive, would tend to strengthen the
conclusion, and a statement of findings true for this
patient that are not normally expected with this dis-
Table 1.
List of diseases known to the AURHEUM diagnostic
Chronic inflammatory arthritis
Rheumatoid arthritis
Juvenile rheumatoid arthritis, pauciarticular onset
Juvenile rheumatoid arthritis, polyarticular onset
Juvenile rheumatoid arthritis, systemic onset
Systemic rheumatic diseases
Systemic lupus erythematosus
Scleroderma (progressive systemic sclerosis)
Primary Raynaud’shndifferentiated connective tissue disease
Pol y m yositis/dermatomyositis
Mixed connective tissue disease
Sjogren’s syndrome
Giant cellitemporal arteritis
Polymyalgia rheumatica
Polyarteritis nodosa
Spond y larthropat hies
Ankylosing spondylitis
Psoriatic arthritis
Reiter’s syndrome
Enteropathic arthritis
Crystal-induced arthritides
Calcium pyrophosphate deposition disease
Microbial-associated arthropathies
Bacterial, nongonococcal arthritis
Gonococcal arthritis
Tuberculous arthritis
Rheumatic fever
Nonarticular rheumatism
Fibromy algia
Carpal tunnel syndrome
ease. The statement of findings that are present and
are not components of the disease is an attempt by the
program to flag for the user conditions which should
lead him or her to consider the presence of other
disease processes.
The system can understand, store, and reason
from 877 observations on each case, Its reasoning
process uses whatever information is given, and can
reach correct diagnostic conclusions from a handful of
observations, if those observations are important indicators. If the information offered is not sufficient to
trigger any diagnostic conclusions, the system will
indicate so. The data-entry phase of the interaction
takes 6-8 minutes as the user, beginning with a
filled-out patient data checklist, responds to menuoriented question frames. The actual reasoning time in
which the system’s logic is executed varies from 3 to 6
seconds on a VAX-111780 supermini computer, and
from 9 to 16 seconds on an ISM PC AT microcomputer.
AIiRHEUM was developed using systembuilding software called EXPERT, which was pro-
duced at Rutgers University (New Brunswick. NJ).
EXPERT is a general-purpose system for building
artificial intelligence models in various domains (4).
The AURHEUM system has been evaluated
using more than 500 clinical cases in several series. It
was 94% correct, agreeing with a consensus diagnosis
of rheumatologist clinicians in 360 of 384 cases, when
tested at the University of Missouri-Columbia on a
series of cases selected retrospectively because they
had discharge diagnoses that were in the system (5).
The system was 85% correct, agreeing with
rheumatologist clinicians in 63 of 74 cases, in an
unselected series that consisted of all patients admitted to the rheumatology service at the University of
Missouri-Columbia during 2 60-day periods. In this
series, every patient who had a disease that was listed
in the AI/RHEUM knowledge base (63 of 74) received
a correct diagnosis. Of the 1 1 patients with diseases
not in the system, diagnosis was correctly refused on
5. This brought the number of cases on which the
system had made an appropriate statement to 68 of 74
(92%). It was wrong on 6 patients (6,7). It should be
noted that although the system’s knowledge base
includes information on only 26 rheumatic diseases, its
coverage in this unselected series of cases was 85%: 63
of the 74 patients had diseases already defined in the
knowledge base.
We report on a third series of patients and
address issues of how the consistency with which
trained rheumatologists make observations half a
world apart will affect the performance of knowledgebased consultant systems. The AVRHEUM model was
challenged with data from 59 connective tissue disease
cases sent by colleagues at Keio University in Tokyo,
Japan, an institution not involved in the previous model
development or testing. This report also provides a
preliminary evaluation of the sensitivity and specificity
of the proposed criteria for mixed connective tissue
disease (MCTD) used by the model.
The reasoning of the AI/RHEUM system is embodied in a series of more than 1,000 production rules of the
form “IF (premise) . . . THEN (conclusion).” The production rules are derived directly from the diagnostic criteria
tables specified for each disease in the system’s knowledge
base. Criteria tables are formulated in terms of major and
minor decision elements, required (must have) elements, and
exclusionary (must not have) elements. Any of these decision elements may be individual findings from a patient data
checklist (e.g., clinical signs, symptoms, laboratory test
results, radiologic observations, tissue biopsy results), or
may be intermediate hypotheses.
Intermediate hypotheses are derived from combinations of findings, and usually represent specific pathophysiologic states. There are 467 such intermediate hypotheses in
the knowledge base. In its reasoning process, AIiRHEUM
begins with patient findings, determines the status of intermediate hypotheses such as “serious renal involvement,”
and combines intermediate hypotheses and other findings to
reach conclusions regarding disease.
The criteria form of knowledge representation developed for AI/RHEUM is new to medical artificial intelligence.
In the system building process, it allows the physician
subject-matter expert to define definite, probable, and possible categories of disease; major and minor decision
elements let the expert define very specific clinical presentations for the knowledge base when appropriate and let him
or her be more general when presentations are less well
characterized. The criteria are readily understood by consultants, who can begin useful critiquing almost immediately
upon reading them.
In some circumstances, a critical diagnostic feature
may be constituted by a pattern of events, rather than by a
simple event. Examples include the various patterns of
arthritis. AURHEUM asks the user physician to answer if
the patient’s pattern of arthritis is “onset with maximum
inflammation within 24 hours,” “chronic with acute exacerbations,” etc. Thus, multiple choices with respect to the
arthritis distribution, duration, onset, course, and response
to therapy are presented; the physician who knows the
patient selects the correct response. In this way, he or she
codes a conclusion that is relatively easy for a human being
and relatively difficult for a computer system. Other investigators are interested in constructing computer programs
that can draw such distinctions when presented with computer records of multiple visits by a patient, over time. AI/
RHEUM circumvents this serious problem.
Though we examined the published literature for
other criteria as we added new diseases to the knowledge
base (8,9), in each case we found it necessary to derive our
own. One reason for this was that published criteria often
focus on patients with advanced disease in order to assure
consistency in retrospective or collaborative studies. To
make AURHEUM effective in a general clinical setting, we
needed criteria appropriate to early and intermediate stages
of disease, as well as to full-blown presentations. The
criteria we postulate for mixed connective tissue disease
(MCTD) are presented in Table 2.
In the test series reported here, 59 patients under
continuing treatment at Keio University were studied (Table
3). Medical records were reviewed by the 3 Japanese rheumatologists (MH, MA, MT). Patients were included if they
met the following 3 criteria: 1) serum was definitely positive
for antinuclear antibodies (ANA) by indirect immunofluorescence; 2) sufficient clinical and laboratory findings were present to satisfy criteria for 1 of the following systemic rheumatic diseases: systemic lupus erythematosus (SLE) (lo),
progressive systemic sclerosis (PSS; scleroderma) (1 I),
polymyositis (PM) (12), rheumatoid arthritis RA (13), MCTD
(14), Sjogren’s syndrome (SS) (15), or overlap syndrome
(criteria fulfilled for 2 or more of these diseases); and 3) a
22 1
Table 2. Diagnostic criteria for mixed connective tissue disease*
Maior criteria
Minor criteria
1. Severe myositis
2. Pulmonary involvement,
with I or more of:
DLco 170% normal
Pulmonary hypertension
Proliferative vascular
lesions on lung biopsy
3. Raynaud’s phenomenon,
OR esophageal
4. Swollen hands observed,
OR sclerodactyly
5. Highest observed
anti-ENA rl:10,000,
AND anti-RNP+ ,
AND anti-Sm -
1 . Alopecia
2. Leukopenia (<4,000
3. Anemia
(510 gm/dl, women,
512.0 gm/dl, men)
4. Pleuritis
5. Pericarditis
6 . Arthritis
7. Trigeminal neuropathy
8. Malar rash
9. Thrombocytopenia
10. Mild myositis
1 1 . History of swollen hands
* DLco
carbon monoxide diffusing capacity; ENA = extractable
nuclear antigen; WBC = white blood cells. For a diagnosis to be
considered definite, the patient had to have 4 major criteria with
serologic requirements of anti-ENA 2 1:4,000, positive anti-RNP,
and negative anti-Sm. For a diagnosis to be considered probable, the
patient had to have present either 3 major criteria or 2 major criteria
(including 1 or more criteria from 1 , 2 , or 3) and 2 minor criteria and
serologic requirements of anti-ENA 2 1 :1,000 and positive antiRNP. For a diagnosis to be considered possible, the patient had to
have 3 major criteria and no serologic requirements, or 2 major
criteria, or I major criterion and 3 minor criteria with anti-ENA
~1:lOOand positive anti-RNP.
sufficiently extensive database was collected to definitely
support 1 or more diagnoses and to exclude other differential
These selection requirements explain the relative
paucity of cases of RA and the absence of cases of polyarteritis and early undifferentiated connective tissue disease in
this study. The large number of complicated cases with
multiple diagnoses or overlap syndromes reflects the special
interest of the Keio University group, and posed a special
challenge for the AI/RHEUM system. The patient information and the clinicians’ diagnoses were obtained from
Japanese rheumatologists who completed the AI/RHEUM
patient data checklist. The patient information was entered
into an AIIRHEUM model running on a DEC system-20
mainframe computer, and the AI/RHEUM diagnostic statement was obtained.
The results of this testing were tabulated as follows:
For cases assigned a single diagnosis by the Japanese
clinicians, the AI/RHEUM program was considered correct
if it listed this diagnosis at the highest level in its differential.
AI/RHEUM was considered partially correct if it listed this
diagnosis in its differential, but listed another diagnosis at a
higher level (e.g., probable rather than possible). The model
was judged incorrect when it failed to include the diagnosis
assigned by the Japanese rheumatologists in its differential
diagnosis. The model was also judged incorrect when a
diagnosis the physicians had not mentioned was listed as
In the diagnosis of overlap syndromes, the situation
was slightly different. When the Japanese rheumatologists
Table 3. Diagnoses by Japanese physicians and AI/RHEUM in 59 Japanese patients with connective tissue disease*
AIiRHEUM diagnosis no.
I t Overlap
Def PSS, prob MCTD, 21
poss SLE
2 PSS, ss
Def SS, poss PSS,
poss MCTD
Prob MCTD, poss
3 t SLE
SLE, poss PSS
Def MCTD, prob PSS,
4t Overlap
poss SLE
Def PM
Prob SLE
Prob MCTD, poss
SLE, poss PSS
8 PM
Def PM
Poss PSS, poss
MCTD, poss SLE
Def PSS, prob SLE,
10 PSS
poss ss
Def PM, poss SS
Prob PM
Def SS, prob MCTD,
prob PSS, poss SLE
Def PSS, prob SLE,
prob RA, poss PM,
poss PAN
Poss PSS, poss
MCTD, poss SLE
Def SLE, def PSS,
poss PM, poss PAN
Prob SLE, poss PAN
Def PSS, poss ss
Def PSS, poss SLE
PSS, ss
Overlap (PSS,
Overlap (PSS,
PSS, ss
AIfliHEUM diagnosis
Def ss,poss PSS,
poss RA, poss SLE
Def PSS, def SS, poss
PM, poss SLE
Poss PSS, poss MCTD
Def MCTD, def SS,
poss PSS, poss SLE
Def PSS, prob SS,
poss PM, poss SLE,
poss MCTD
Def MCTD, prob PSS,
poss SLE
Def MCTD, def SS,
poss PSS, poss SLE
Def PSS, prob SS,
poss SLE
Def PSS, def SS, poss
MCTD, poss PM,
poss RA, poss SLE
Prob MCTD, prob
PSS, poss SLE
Def PSS, prob SLE,
poss RA, poss PM
Def PSS, poss ss
Def PSS, poss SLE
Prob SLE, poss SS
Poss SLE, poss SS
Def MCTD, def SS,
prob PSS, poss SLE
Poss SLE, poss RA
Poss PM, poss PSS
Prob SLE
Def SS, poss SLE
Poss PM, poss
Def SLE, poss
Prob PM, poss
Prob SLE, poss
Poss SLE
MCTD,SLE, Prob MCTD, poss
Def SS, poss SLE
Def SS, poss SLE
Prob SLE
Prob SLE
Prob SS, poss PSS,
poss PM, poss
Poss SLE
RA, ss
Def ss,poss RA,
poss PSS, poss
Prob SLE
Prob SLE, poss
Def PSS, poss RA,
poss SLE, poss
Poss PSS
Prob SLE
* SLE = systemic lupus erythematosus; PSS = progressive systemic sclerosis; def = definite; prob = probable; MCTD = mixed connective
tissue disease; poss = possible; SS = Sjogren’s syndrome; RA = rheumatoid arthritis; PM = polymyositis; PAN = polyarteritis nodosa.
t Patients whose diagnoses by AURHEUM were partially correct. See Results for explanation.
f Patients whose diagnoses by AI/RHEUM were incorrect. See Results for explanation.
listed multiple connective tissue disease diagnoses for a single
patient, the computer model was judged as correct only when
it listed each of these diagnoses in its differential at a higher
level of probability than any diagnosis not listed by the
For example, if a patient was diagnosed by the
Japanese clinicians as having an overlap syndrome with
features of SLE, RA, and PM, the computer model’s response was scored correct if it listed a sequence of diagnoses
such as definite SLE, probable RA, possible PM, possible
SS. However, if AURHEUM listed definite SLE, probable
RA, probable SS, and possible PM, it would no longer be
considered fully correct because it had listed the diagnosis of
SS (not mentioned by the Japanese clinicians) at a level
higher than PM. In this situation, the model was considered
partially correct.
The computer model was also considered partially
correct when, for the 3- and 4-diagnosis overlap syndrome
cases, it listed in its differential diagnosis all but 1 of the
diagnoses assigned by the clinicians.
The patient was a 47-year-old man (patient 27,
Table 3) whose onset of symptoms occurred at age 38.
Over the subsequent 9 years, the patient’s clinical
findings included Raynaud’s phenomenon, temperatures as high as 40°C, hypertension (blood pressure
>140/90 mm Hg basal), sclerodactyly , telangiectasias,
ecchymoses, keratoconjunctivitis sicca, xerostomia,
parotid enlargement, symmetric polyarticular arthritis,
mild proximal muscle weakness, lymphadenopathy,
pleuritis, evidence of pulmonary interstitial disease,
and hepatornegaly. Diffuse sclerosis, digital ulcers,
calcinosis cutis, alopecia, photosensitivity, rash, subcutaneous nodules, pericarditis, splenomegaly, and
neurologic abnormalities were not present.
Laboratory findings were as follows: hemoglo-
bin 11.6 gm/dl, white blood cell count 4,900/mm3,
eosinophil count 45/mm3, platelet count 170,000/
mm3, erythrocyte sedimentation rate (Westergren) 82
mm/hour, normal urinalysis results, blood urea nitrogen 16.9 mg/dl, serum creatinine 0.91 mg/dl, creatinine clearance 96 ml/minute, serum uric acid 7.1
mg/dl (normal 5 5 mg/dl), serum gamma globulin 4.3
gm/dl, creatine kinase 42 (normal 512), serum glutamic oxaloacetic transaminase 57 (normal 540). Carbon monoxide diffusing capacity (DLco) was 9.4% of
normal. Electrocardiogram showed a prolonged PR
interval. There was esophageal hypomotility. Electromygram showed myopathy, and muscle biopsy
showed mild myositis. Lip-biopsy showed SS, and
there was a positive result on the Schirmer’s test.
Serologic results included positive rheumatoid factor
(RF) at a titer of 1:640,C3 50 mg/dl (normal r70),
positive fluorescent ANA with a speckled pattern and
a titer of 1: 1,024, anti-extractable nuclear antigen
(ENA) antibody (hemagglutination method) positive at
a titer of 1 :20,480,anti-DNA antibody (hemagglutination method) positive at a titer of 1:14.By immunodiffusion, RNP, SS-A, and SS-B antibodies were all
positive and Sm, PM-I, and Scl-70 antibodies were all
This patient had 4 diagnoses given by the Japanese clinicians: MCTD, PSS, PM, and SS. The
differential diagnosis given this case by AI/RHEUM
was definite MCTD, definite SS, possible PSS, and
possible SLE. The diagnosis of PM would also have
been triggered, but was excluded by the presence of
the anti-RNP antibody in conjunction with an ENA
titer > 1 :1,000.We scored AI/RHEUM partially correct, since the system indicated 3 of the 4 clinician’s
The AURHEUM system explained its reasoning by stating that the diagnosis of definite MCTD was
concluded on the basis of the findings of Raynaud’s
phenomenon, sclerodactyly , pleuritis, esophageal hypomotility, DLco <70% of normal, anemia (an intermediate hypothesis), anti-RNP antibody present with
an ENA titer > I : 10,000and anti-Sm negative (another
intermediate hypothesis).
The AURHEUM diagnosis of definite SS was
supported by the findings of keratoconjunctivitis sicca,
xerostomia, parotid enlargement, a positive Schirmer’s test result, a lip biopsy specimen positive for SS,
positive RF, and positive fluorescent ANA at a titer
> 1 :40 (an intermediate hypothesis).
The system’s diagnosis of possible PSS was
triggered by the findings of sclerodactyly, pulmonary
interstitial disease or DLco <70% of normal (an
intermediate hypothesis about pulmonary involvement; actually, both findings were present), telangiectasias, and Raynaud’s phenomenon, esophageal hypomotility, or digital ulcers (another intermediate
hypothesis; the first 2 findings were present).
The final diagnosis in the AI/RHEUM differential was possible SLE. The system stated that this
diagnosis was supported by the findings of serositis (an
intermediate hypothesis), temperature >38”C (an intermediate hypothesis), hypergammaglobulinemia (an
intermediate hypothesis triggered at levels > 1.8
gm/dl), hypocomplementemia (an intermediate hypothesis triggered by comparison with normal ranges),
the presence of Raynaud’s phenomenon, positive fluorescent ANA at a titer >1:40,and positive anti-DNA
Other portions of the AI/RHEUM output statement list, for each disease in the differential diagnosis,
the findings present which are not explained by the
disease, and the findings not yet known, which, if
known and positive, would strengthen the conclusion.
The cases presented by the Japanese clinicians
at Keio University were truly complex (Table 3). Most
frequent single diagnoses were of SLE (17 patients)
and PSS (16 patients). Seven patients had 2 diagnoses,
and 5 patients were labeled as having overlap syndrome, with features of 2-4 connective tissue diseases
specified. Four additional cases were not specified as
overlap syndrome, but carried 3 and 4 separate diagnoses for an individual patient.
The AI/RHEUM computer consultant system
presented a differential diagnosis for each of the 59
cases in the series, listing from 1 to 6 connective tissue
diseases. Each diagnosis in the differential was categorized as definite, probable, or possible.
The model was in complete agreement with the
Japanese rheumatologists in 45 of 59 cases (76%), and
was judged partially correct in 9 additional cases.
These 9 partially correct cases included 5 in which
AURHEUM listed the correct diagnosis in its differential but listed another diagnosis at a higher confidence level. Two of these 5 cases involved the diagnosis of SS. In the remaining 3 partially correct cases,
AURHEUM included in its differential all but 1 of the
3 or 4 diagnoses listed by the Japanese clinicians in
overlap syndromes. For these 3 cases, the missing
diagnosis was PM. In each case, the features of the
patient’s clinical disease could be adequately explained by the alternative diagnoses AI/RHEUM did
make. For example, in a patient with MCTD with an
inflammatory myositis, MCTD adequately explains
the myositis; therefore, PM would not be separately
diagnosed by AURHEUM. The computer model was
considered to have made a correct or partially correct
diagnosis in 54 of 59 cases (92%).
F~~~of the 5
judged incorrect occurred
when AI/RHEUM diagnosed definite SS, but the Japanese clinicians did not mention this diagnosis. In each
of these cases, the findings of keratoconjunctivitis
sicca, a positive result on Schirmer’s test, and xerostomia listed on the patient data checklist would seem
to offer justification for the diagnosis. In the fifth
incorrect case AI/RHEUM missed a diagnosis of
MCTD, in part because although anti-ENA antibody
was present, the titration value was unknown.
This study also provided an opportunity for a
preliminary evaluation of the sensitivity and specificity
of the criteria used for MCTD (Table 4). The Japanese
clinicians diagnosed 10 patients as having MCTD,
including those with multiple diagnoses and overlap
syndromes. The computer model correctly identified 9
of these patients by listing the diagnosis of MCTD at
the highest level in its differential. For example,
MCTD might be listed at the probable level, with no
other diagnosis listed as definite. In the last of these
cases, the model incorrectly listed SS as definite and
MCTD as probable.
AI/RHEUM correctly diagnosed as not having
MCTD 47 of the 49 cases the Japanese rheumatologists
diagnosed as not having MCTD. In only 2 of these 49
cases was MCTD diagnosed at a probability higher
than the correct diagnosis. Both cases were listed as
probable MCTD by the model, whereas the correct
diagnosis of SLE was listed only at the possible level.
The opportunity to test this model with cases
from Keio University was appealing because the Japanese patient data collection and diagnostic processes
would be totally independent of those which formed
the background of AURHEUM. At the same time, we
recognized that expert rheumatologists abroad would
probably make highly reliable patient observations.
Thus, the testing of the model would be fair and
independent, but not so severe as the ultimate (future)
test with nonrheumatologist physicians.
A special difficulty in testing the model in the
Table 4. Sensitivitykpecificity evaluation of the proposed criteria
for mixed connective tissue disease (MCTD) used in AIIRHEUM”
Diagnosis by
Diagnosis by AI/RHEUM
computer consultant program
Japanese clinicians
Other than MCTD
Other than MCTD
* These data result in 90% sensitivity and 96% specificity.
context of Japanese rheumatology practice was the
use of the diagnostic term overlap syndrome. The term
in Japan refers to the simultaneous occurrence of
findings associated with multiple connective tissue
diseases, usually rather strongly manifested. It should
be noted that AURHEUM has the ability to make
multiple diagnoses in the same patient when findings
are present that trigger multiple conclusions.
This capability, while obviously a traditional
part of the clinical approach, is not at all a common
part of most computer models of diagnosis. Computer
models that use the Bayesian approach typically rankorder individual diagnoses (16). Models based upon
“branching tree” or flow chart logic are inherently
incapable of dealing explicitly with the overwhelming
combinations of the many individual findings that map
to multiple simultaneous diagnoses in the complex
clinical case. Even in the case of INTERNIST-1
(17,18), arguably the most extensive and substantial of
computer diagnostic models, the system would likely
not be well suited to dealing with overlap syndromes.
INTERNIST-1 is limited by the use of a formalism
which allocates each finding to a single diagnosis. This
effectively excludes the capability of dealing with
highly interrelated disease diagnoses such as we see in
rheumatic conditions like the overlap syndrome as it is
defined in Japan. AURHEUM specifically is capable of
dealing with an overlap syndrome as a combination of
individual diseases.
Thus, AI/RHEUM has proven successful in our
own academic setting (1,2,6,7) and in association with
specialists elsewhere. Problems in diagnosing these 59
connective tissue disease cases centered on the system’s not listing diagnoses mentioned by the Japanese
clinicians or on the system’s identifying a diagnosis as
definite, when it was not listed in the clinician’s
In all 4 of the cases in which a diagnosis was not
listed by the model, the patient had an overlap syndrome; AI/RHEUM justifiably incorporated the pa-
tients’ clinical findings into its conclusion of another
appropriate connective tissue disease. An example of
this situation would be the incorporation of clinical
findings of P M into a diagnosis of MCTD.
In 4 of the 5 cases in which the computer model
identified as definite a diagnosis not listed by the
Japanese clinicians, the disease in dispute was SS.
Review of the patient data checklists for these cases
showed strong evidence for the presence of SS; it is
uncertain why the Japanese physicians did not diagnose SS in these cases. Possibly, their criteria for
diagnosis of definite SS (strongly positive staining with
rose bengal dye on biomicroscopy , histopathologic
confirmation of destructive lymphocytic infiltration, or
abnormal findings on sialography [ 151) were more
demanding than those of AI/RHEUM. In general,
however, there appears to be a high level of agreement
between the Japanese and American physicians in
their conceptualization of rheumatic disease conditions, which may be because of their use of the same
(American Rheumatism Association) o r similar criteria
for systemic rheumatic diseases.
The AIiRHEUM knowledge base incorporates
criteria tables for each of the 26 diseases it is capable
of diagnosing. Preliminary results based on the 59
patients in this study suggest that the criteria for
MCTD shown in Table 2 are helpful in diagnosing this
syndrome. The model correctly identified 90% of the
patients diagnosed by the Japanese clinicians as having MCTD. The model correctly did not diagnose
MCTD in 47 of the 49 patients the Japanese clinicians
diagnosed as not having MCTD. For both of the cases
in which it incorrectly diagnosed MCTD, AI/RHEUM
listed MCTD at the probable level and listed the
correct diagnosis of SLE at the possible level. It is
anticipated that testing of these preliminary criteria by
other groups may provide additional new insights
regarding the relationship between MCTD and other
connective tissue diseases.
Our plans for future investigations are influenced by these encouraging findings. One limitation to
testing in a typical clinical environment is the fact that
the current AURHEUM system deals only with the
diagnostic process. Knowledge concerning the management of patients with rheumatic diseases will add
an important component in making AURHEUM a
useful tool for the primary care physician. This management phase is currently under development.
Finally, field testing by nonrheumatologist physicians will be required before AI/RHEUM can be
considered appropriate for general use. This validation
and evaluation process may take 2-3 years, after
which we anticipate the system’s release in a configuration for the IBM PC A T microcomputer. We have
criteria tables for 6 additional diseases in draft form,
which when tested and incorporated into the knowledge base, will bring the system’s coverage to 32
diseases. Since a system such as AURHEUM can be
only as accurate and reliable as the observations and
laboratory test results on which it bases its diagnostic
conclusions and therapy recommendations, we are
now developing a n adjunct program called AI/
LEARN, which is intended to help improve the accuracy of clinical observations of rheumatic disease made
by physicians (19).
We thank Shem Potts for expert secretarial assistance.
1. Lindberg DAB, Sharp GC, Kingsland LC Ill, Weiss
SM, Hayes SD, Ueno H, Hazelwood SE: Computer
Based Rheumatology Consultant, Proceedings of
MEDINFO 1980. Edited by DAB Lindberg, S Kaihara.
Amsterdam, North-Holland, 1980, pp 131 1-1315
Kingsland LC 111, Lindberg DAB: Research methods in
A1 model building: the history of a project, Proceedings
of the American Association of Medical Systems and
Informatics Congress 1983. Edited by DAB Lindberg, E
van Brunt, MA Jenkin. Bethesda, AAMSI, 1983, pp
Lindberg DAB, Kingsland LC 111, Waugh W, Benge JM,
Sharp GC: Criteria tables as a possibly general knowledge representation, Proceedings of the American Association of Medical Systems and Informatics Congress
1984. Edited by DAB Lindberg, MF Collen. Bethesda,
AAMSI, 1 9 8 4 , : ~187-192
Weiss SM, Kulikowski CA: EXPERT: a system for
developing consultation models, Proceedings of the
Sixth International Joint Conference on Artificial Intelligence. Tokyo, 1979, pp 942-947
Kingsland LC 111, Lindberg DAB, Sharp GC: AI/
RHEUM: a consultant system for rheumatology. J Med
Syst 7~221-227, 1983
Benge JM, Sharp GC, Kay DR, Capps RJ, Hazelwood
SE, Kingsland LC 111, Reese GR, Lindberg DAB:
Evaluation of an A1 consultant in rheumatology, Proceedings of the American Association of Medical Systems and Informatics Congress 1983. Edited by DAB
Lindberg, EE van Brunt, MA Jenkin. Bethesda.
AAMSI, 1983, pp 83-86
Kingsland LC 111, Sharp GC, Capps RJ, Benge JM, Kay
DR, Reese GR, Hazelwood SE, Lindberg DAB: Testing
of a criteria-based consultant system in rheumatology,
Proceedings of MEDINFO 1983. Edited by JH van
I 1.
Bemmel, MJ Ball. 0 Wigertz. Amsterdam, North-Holland, 1983, pp 514-517
Cohen AS, Reynolds WE, Franklin EC, Kulka JP,
Ropes MW, Shulman LE, Wallace SL: Preliminary
criteria for the classification of systemic lupus erythematosus. Bull Rheum Dis 21:643-648. 1971
Rodnan GP, Schumacher HR, Zvaifler NJ, editors:
Primer on the Rheumatic Diseases. Eighth edition. Atlanta, Arthritis Foundation, 1983, pp 207-208
Tan EM, Cohen AS, Fries JF, Masi AT, McShane DJ,
Rothfield NF, Schaller JG, Tala1 N , Winchester RJ: The
1982 revised criteria for the classification of systemic
lupus erythematosus. Arthritis Rheum 25: 1271-1277,
Subcommittee for Scleroderma Criteria of the American
Rheumatism Association Diagnostic and Therapeutic
Criteria Committee: Preliminary criteria for the classification of systemic sclerosis (scleroderma). Arthritis
Rheum 23581-590, I980
Bohan A, Peter JB, Bowman RS, Pearson CM: A
computer-assisted analysis of I53 patients with polymyositis and dermatomyositis. Medicine (Baltimore)
56:255-283, 1977
Ropes MW, Bennett GA, Cobb S, Jacox R , Jessar RA:
1958 revision of diagnostic criteria for rheumatoid arthritis. Bull Rheum Dis 9:175-176. 1958
Kasukawa R , Tojo T, Miyawaki S, Yoshida H, Tani-
moto K, Nobunaga M, Suzuki T, Takasaki Y . Tamura
T: Preliminary diagnostic criteria for classification of
mixed connective tissue disease, Mixed Connective
Tissue Disease and Anti-Nuclear Antibodies. Edited by
R Kasukawa, GC Sharp. Amsterdam, Elsevier, 1987, pp
15. Diagnostic criteria of the Japanese Ministry of Health
and Welfare for Sjogren’s syndrome. Annual Report,
Japanese Intractable Diseases Research Foundation,
16. Lusted LB: Some studies of medical diagnoses using
Bayes’ theorem and computers, Introduction to Medical
Decision Making. Springfield, IL, Charles C Thomas,
1968, pp 24-69
17. Miller RA, Pople HE, Myers JD: INTERNIST-I, an
experimental computer-based diagnostic consultant for
general internal medicine. N Engl J Med 307:468-476,
18. Miller RA: INTERNIST-I/CADUCEUS: problems facing expert consultant programs. Methods Inf Med 23:914, 1984
19. Kochtanek T, Kay DR, TenBrink T: AI/LEARN: an
education innovation in rheumatology, Proceedings of
the American Association of Medical Systems and Informatics Congress 1985. Edited by AH Levy, BT
Williams. Bethesda, AAMSI, 1985, pp 395-398
Без категории
Размер файла
784 Кб
knowledge, patients, japan, performance, disease, system, tissue, airheum, base, connection, consultants, rheumatology, computer, diagnosis
Пожаловаться на содержимое документа