Main Report W Hauptreferat W Rapport principal Folia Phoniatr Logop 1998;50:165–182 Tanya M. Gallagher McGill University, Montreal, Que., Canada Treatment Research in Speech, Language and Swallowing: Lessons from Child Language Disorders OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO Key Words Treatment research W Child language intervention W Clinical trials W Outcomes OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO Abstract Three major types of treatment research methodologies are described. Studies on child language intervention are reviewed as examples of trends and methodological issues characterizing treatment research in speech, language, and swallowing within the last 2–3 decades. Principles are drawn from that literature and suggestions for future directions are discussed with particular attention to recent efforts to support clinical trials and treatment outcomes research. OOOOOOOOOOOOOOOOO Behandlungsforschung bei Sprach-, Sprech- und Schluckproblemen: Lehren aus kindlichen Sprachstörungen Drei besondere Forschungsmethoden betreffend Behandlungen werden beschrieben. Untersuchungen über kindliche Sprachprobleme werden als Beispiele für Bestrebungen und methodische Versuche in der Behandlungsforschung von Sprech-, Sprach- und Schluckstörungen der letzten 20–30 Jahre vorgestellt. Aus dieser Literatur werden Prinzipien herausgearbeitet und als Vorschläge für künftige Behandlungen mit besonderer Unterstützung der neuesten Forschungsresultate ausgearbeitet. OOOOOOOOOOOOOOOOO Recherche en thérapie du langage et de la déglutition: enseignement tiré des troubles langagiers de l’enfant Trois principales méthodologies de recherche thérapeutique sont décrites. Une revue des études portant sur l’intervention dans le langage des enfants exemplifie les tendances et les points méthodologiques caractéristiques de la recherche thérapeutique dans les domaines du langage, de la parole, et de la déglutition durant les 2–3 dernières décennies. Des principes découlant de la littérature et des suggestions concernant les directions futures sont discutés avec une attention particulière aux efforts récents appuyant les essais cliniques et la recherche aux fins thérapeutiques. OOOOOOOOOOOOOOOOO © 1998 S. Karger AG, Basel 1021–7762/98/0503–0165$15.00/0 Fax + 41 61 306 12 34 E-Mail email@example.com www.karger.com This article is also accessible online at: http://BioMedNet.com/karger Tanya M. Gallagher, PhD, Professor and Associate Dean University Affairs, Planning and Resources, McGill University Office of the Dean, McIntyre Medical Sciences Building Montreal, Quebec H3G 1A8 (Canada), Tel. +1 (514) 398 3499 Fax +1 (514) 398 3595, E-Mail firstname.lastname@example.org Downloaded by: Florida State University 22.214.171.124 - 10/28/2017 6:57:16 PM ABC 166 Folia Phoniatr Logop 1998;50:165–182 Table 1. Major treatment research designs Treatment efficacy research Randomized or fully specifiable assignment Well-controlled conditions Causal interpretations Treatment effectiveness research Nonrandomized assignment General specification of conditions Comparative interpretations Treatment outcomes research No comparison groups Typical conditions Trends and associations ment can be interpreted as directly attributable to that treatment. Randomized controlled clinical trials are an example of this type of research. Treatment effectiveness research is a research design that involves nonrandomized assignment to intervention and control groups and general specification of treatment methodologies. Within this design, the potential influence of confounding variables is not as well controlled and, therefore, causal interpretations and generalizability of the results obtained are weakened. Comparative treatment studies are an example of this type of research. Treatment outcomes research is a research design with no clearly specified comparison groups or treatment methodologies. Functional outcomes research describes functional changes in the lives of patients following treatments provided by typical providers, in typical treatment settings, with typical clients, under typical treatment conditions. Within this design, only trends and associations can be reported. Patient databases are examples of this type of research. Each of these research methodologies contributes complementary information to our Gallagher Downloaded by: Florida State University 126.96.36.199 - 10/28/2017 6:57:16 PM The demand for information on the efficacy, cost-effectiveness, and functional impact of speech-language pathology treatment on the quality of life of people served has escalated dramatically throughout this decade. Although the need for information on treatments has long been recognized, several factors have contributed to the accelerated interest and sense of urgency that has characterized the 1990s. Aggressive time frames for the incorporation of treatment information into decision-making processes have placed considerable pressure on speech, language, and swallowing service providers to collect data to manage programs, advocate for services, and enrich the scientific knowledge base underlying the development of practice guidelines and critical care paths [1–4]. Among the factors that have contributed to this situation in many countries are efforts at health care cost containment and health care reform in the context of scarce and competing resources; increased attention to treatment evaluation and outcomes by national agencies responsible for health care standard setting, standard monitoring, financing, and health care policy and research and, the growing acceptance of outcomes measurement as a comprehensive and distinctive research methodology combining elements of clinical epidemiology and information technology as well as more traditional research designs [5–7]. As a comparatively recent research methodology, a great deal of variability exists in the terminology used to refer to different types of treatment research. Within this paper, three basic treatment research methodologies will be defined as follows [8–10] (table 1): Treatment efficacy research is an experimental research design that involves randomized or fully specifiable assignment to subject groups and studies treatment methodologies under well-controlled conditions. Within this design, any change measured following treat- knowledge of treatments. Each is characterized by different strengths and weaknesses. Each serves different purposes and entails different interpretative cautions. Treatment efficacy research, traditional in experimental design and often referred to as the ‘gold standard’ of clinical research, is the only design that permits strong causal statements to be made. These types of studies are, therefore, critical to the evaluation and development of clinical procedures. They are expensive, however, and the use of no-treatment conditions often raises ethical concerns. A further difficulty is the very experimental control that supports the robustness of data interpretations of these types of studies. Treatment efficacy studies can assert that changes were the direct result of treatment procedures but only within the conditions tested. The generalizability of clinical trials results to typical treatment settings cannot simply be assumed since the very controls that are the hallmark of the design may create subject inclusion and exclusion conditions, and treatment delivery conditions that are not representative of most treatment settings serving typical clients. Treatment effectiveness research, although only permitting comparative statements and analyses of trends and associations, is costeffective and addresses some of the ethical and methodological concerns raised about efficacy studies. Outcomes research can only make descriptive statements and cannot ascribe changes in behavior directly to the treatments delivered, but those statements describe patients following typical interventions within typical settings for most people being served. The interpretive stength of outcomes research depends upon the size and robustness of the database. Representativeness is achieved when the outcomes database is so large that it includes, or does so as nearly as possible, the population the database is describing. Treatment efficacy research contributes precision and definitiveness, treatment effectiveness research contributes a cost-effective means of exploring the promise of treatment directions, and outcomes research contributes an understanding of real world practice trends and the functional impacts of treatment programs on patients’ lives. Each type of research is complementary and synergistic and, therefore, each is necessary to provide a balanced and comprehensive view of speech, language and swallowing therapy. A review of most treatment studies to date, however, indicates that researchers in speech, language, and swallowing have primarily utilized treatment effectiveness research designs. There have been few treatment efficacy or treatment outcomes studies in speech, language, and swallowing. Within this paper, studies on child language intervention will be reviewed as examples. Principles will be drawn from what that literature illustrates about the conduct of treatment research in general. Finally, suggestions for future directions will be discussed with particular attention to the American Speech-Language-Hearing Association’s (ASHA) efforts to support clinical trials and treatment outcomes research. Treatment Research in Speech, Language and Swallowing Folia Phoniatr Logop 1998;50:165–182 Review of Child Language Intervention Research Child language intervention is a comparatively recent area of practice. Prior to the 1960s intervention with children focused primarily on speech production, and language therapy was infrequent and restricted in scope. Since that time, child language intervention has become a major area of practice, reflecting successive theoretical influences: behaviorism in the 1960s [11, 12], generative syntax in the 1970s, and pragmatics in the Downloaded by: Florida State University 188.8.131.52 - 10/28/2017 6:57:16 PM 167 168 Folia Phoniatr Logop 1998;50:165–182 mance. Most research has compared the effects of different intervention procedures and different intervention agents. Intervention Research: Procedures Language intervention techniques have been described as a continuum that is anchored at each extreme by procedures derived from widely different language theories [18, 21]. At one end are procedures reflecting behavioral theories such as stimulus presentation, response imitation, and external reinforcement. Behaviorally oriented language therapy programs use highly structured approaches that isolate language targets from natural contexts in order to increase their saliency, facilitate the elicitation of direct imitations and performance monitoring, and increase the reinforcement value for correct performance. At the other end of the continuum are procedures derived from generative and pragmatic theories such as focused stimulation, modeling, and expansion/recasting. Generative and pragmatic language therapy programs attempt to approximate the normal language learning situation and increase the frequencies of natural opportunities for learning, provide nondemanding models and expansions/recasts, follow the children’s lead, and increase the children’s access to the naturally occurring rewards for communication. Along the continuum are hybrid models of intervention that combine features of both extremes in varying proportions. Many studies have explored whether one intervention procedure or program is superior to another in facilitating syntactic development. Recent studies have examined imitation, and more interactive, naturalistic approaches such as modeling and expansions/ recasts, and hybrid approaches such as milieu training. Imitation has most often been com- Gallagher Downloaded by: Florida State University 184.108.40.206 - 10/28/2017 6:57:16 PM 1980s and 1990s [see ref. 13–16 for reviews]. Despite the element of truth in a recent description of language acquisition theories as being a ‘confusing morass of theories, subtheories, minitheories, ideas, perspectives, concepts and critiques’ , child language intervention has continuously evolved its content, procedures and indices of success over the last 30 years. Over 200 studies of child language treatment have been conducted. Most of these have been single-subject or comparative treatment effectiveness studies of syntactic performance. They have included small numbers of children, interventions taking place over brief periods of time, circumscribed measures of effectiveness, and dependent measures sampled within treatment setting contexts. The results of these studies have been very encouraging. The overwhelming majority have reported positive overall effects of intervention on syntactic performance [14, 18, 19]. The meta-analysis of 43 studies of language intervention with language/learning-disabled children by Nye et al.  reported an effect size that clearly supported this conclusion. Although a great deal of progress has been made, fundamental questions remain. Among these are: how much of the improvement that has been reported is due to overall maturation and how much is due to the treatments provided; have performance gains in therapy generalized to nontreatment settings and been incorporated into the children’s daily lives; what effects do performance gains have on other areas of the children’s functioning; what intervention procedures and programs result in the greatest performance changes and, what intervention agents should be involved in effective and efficient service program delivery. Most of the experimental treatment studies in the 1980s and 1990s have addressed the last two questions relative to syntactic perfor- pared to each of these more interactive approaches but procedural definitions have varied, making comparisons across studies difficult. For example, Connell  investigated the syntactic rule induction learning patterns of 40 preschool specifically language-impaired (SLI) children and 40 age-matched controls using two intervention procedures, imitation and modeling. Imitation was defined as repeating clinician models following direct prompts. Corrective feedback was provided and praise was used as a reinforcer. Modeling was defined as demonstration. Children were cautioned to listen only and not attempt to repeat and their attention to the task was praised. The learning context in both conditions was a highly structured picture presentation task. Children were taught an invented morphological suffix rule for nouns, ‘a’ meaning ‘part of’ (e.g. ‘cow-a’). No performance differences between imitation and modeling were found for the age-matched control subjects, but the SLI subjects performed better when the instructional technique was imitation rather than modeling. Since rule mastery within this study was assessed only by means of production trials, Connell and Stone  extended the study using comprehension and production performance trials. Within the Connell and Stone  study, SLI children’s performance was also compared to two groups of children who were developing language normally, chronologically age-matched controls and younger language-matched controls. The children were taught four invented morphemes using the same procedures as in the earlier study. The results on production performance did not change. No differences between procedures were found for the normal language control groups, and production performance differences favoring imitation were noted for SLI subjects. The comprehension data, however, clarified that the SLI chil- dren’s preferential learning responses to imitation were due to production practice effects and not true rule learning differences. There were no differences between procedures for all three subject groups within the comprehension data. Both imitation and modeling were equally effective for all subjects. These studies suggest that for SLI subjects, imitation facilitates grammatical production but both imitation and modeling are equally effective for comprehension. Camarata and Nelson  and Camarata et al.  compared imitation with a more naturalistic intervention technique, conversational recasts. Imitation was defined as it was within the study by Connell and Stone  and the learning context was the same, a picture presentation task. External reinforcements as well as verbal praise, however, were used with this procedure. Conversational recasts were defined as clinician restatements of children’s utterances that preserve their meaning and incorporate target structures as syntactic expansions. The context for the conversational procedure was less structured. Play activities were organized so that they maximized opportunities for learning the targets. Open-ended prompts were used and no explicit reinforcers were included. All children received both techniques, each for different intervention targets. There were 4 SLI preschoolers in the 1992 study and 21 SLI preschoolers in the 1994 study. The intervention programs were 16 and 12 weeks, respectively. For most children and most targets, when imitation procedures were used, the children’s elicited productions of the syntactic targets, correct repetitions immediately following target productions, occurred more rapidly. When conversational intervention procedures were used, spontaneous productions of targets occurred more rapidly. Individual differences among targets and children were noted in both studies. The results as a whole suggest Treatment Research in Speech, Language and Swallowing Folia Phoniatr Logop 1998;50:165–182 Downloaded by: Florida State University 220.127.116.11 - 10/28/2017 6:57:16 PM 169 170 Folia Phoniatr Logop 1998;50:165–182 instructional procedures. First, the original assumption that one procedure would be found to be superior to all others will have to be revised. The results as a whole suggest that all of these procedures are useful in particular situations depending on the purpose [24, 25], the modality [22, 23] and the characteristics of the children . At this point, it may be more appropriate to ask a more complicated question than the one originally posed. For example, the question might be what procedure along the continuum is most effective for this type of child with these handicapping and complicating conditions, for this language target, at this stage of mastery, with this socialbehavioral profile, and this type of home and school environment. Our understanding of some of these variables is in its infancy. Second, research methodologies must continue to evolve relative to their precision and degree of experimental control in order for the research and clinical communities to derive maximal benefits from these efforts. The following methodological issues warrant particular attention: (1) Subject Description. Subject characteristics including receptive, expressive and pragmatic language characteristics, chronological age, cognitive level, handicapping and complicating medical conditions, and school placement should be specified. Further, subject inclusion characteristics should be clear and consistently applied. Although considerable progress has been and continues to be made, sustained attention to subject description is needed. (2) Procedural Description. Intervention procedures should be clearly and fully specified including program characteristics such as size of intervention groups, frequency and length of program, and number of trials per procedure. Variability among these dimensions has made it difficult to interpret the results of studies relative to each other. As Gallagher Downloaded by: Florida State University 18.104.22.168 - 10/28/2017 6:57:16 PM that the most effective procedure may be determined by the treatment goal. Comparison between behaviorally oriented approaches and hybrid approaches have also been made. For example, Yoder et al.  compared Milieu Teaching and the Communication Training Program . Milieu Teaching is a hybrid intervention approach that is naturalistic in the sense that it recommends following the children’s lead and interests and uses naturally occurring events and consequences to increase the frequencies of language form use. It is behavioral in the sense that new forms are taught within naturalistic settings using explicit prompts and reinforcement [27, 28]. The Communication Training Program is a highly structured program that specifies number of trials and the sequence of instructional targets. The program uses behavioral procedures, stimulus presentation, imitation, practice and explicit reinforcements. Both programs contained 60 instructional sessions. Forty children served as subjects and ranged in age between 2 and 7 years. The children were described as having language and cognitive delays ranging from near-normal to severely delayed but further information was not provided. The results indicated that there were no differences between the treatment groups regardless of teaching method. Analyses of possible interactions between treatment methods and children’s developmental levels within this study and others [22–25, 29–31] suggest that there may be interactions between treatment method and subject characteristics such as language and cognitive level. Given the definitional differences across studies and the incompleteness of some of the subject information provided it is difficult to determine the nature of those interactions. The studies that have been reviewed are illustrative of several things that we have learned about identifying optimal language Snow et al.  have pointed out relative to the Camarata et al.  study, the terms ‘imitative’ and ‘modeling’ have been variously defined and usually include elements of both within each procedural presentation as we have seen in the studies reviewed. The term ‘naturalistic’ is also difficult to interpret since it is often incompletely and ambiguously defined. Hepting and Goldstein  recently examined 34 studies of naturalistic language intervention to determine the common characteristics of the approach. They found components with each study that justified use of the term, but were unable to identify a common set of characteristics across the 34 studies. Complicating matters further, several elements of intervention programs often vary in addition to the procedures being compared, for example, feedback, reinforcers, context, number of learning trials, and overt practice opportunities. It is, therefore, difficult to know which of these factors or combinations of factors accounted for the results obtained. (3) Treatment Fidelity. Consistency and comparability of treatment delivery across children and clinicians should be documented in order to strengthen data interpretations. For example, clinician procedural consistency was rated within three sessions by Yoder et al. . True precision in this regard may be difficult to achieve for naturalistic language intervention programs, such as those used in the studies by Camarata and Nelson  and Camarata et al. , given their highly learner-responsive character. These programs by definition do not provide a list of steps to follow or specific content to be used and indeed caution against clinician-directed learning. Therefore, the number of learning opportunities provided within treatment sessions may be determined as much by individual children’s interests and responsiveness as by the clinicians’ behaviors. (4) Documentation. Measures such as mean length of utterance and developmental sentence scores should be used with caution to document child progress. Improvement in these measures at the higher levels is unreliable [see ref. 34 for review]. Just as gains in mean length of utterance are not linear with age for children developing language normally , there is no reason to assume that they will be linear for children with language disorders. Incorporating a variety of dependent measures may be most informative. (5) Control Groups. Control groups need to be included within comparative treatment studies to address issues regarding maturational effects. Subjects should also be randomly assigned to treatment conditions as was the case in the Connell , Yoder et al.  and Camarata and Nelson  studies. There has been considerable improvement within treatment studies over the last 10 years in this regard [see ref. 18, 19, 34 for reviews] but appropriate matching variables between children who are developing language normally and those who are not present challenges that still need to be addressed [36, 37]. (6) Generalization. The effects of language intervention must be determined beyond the immediate language learning context and beyond the short term. Most of the studies to date have sampled generalization shortly after intervention in contexts that were similar if not identical to those in which the intervention took place, with the clinician who presented the program, and using techniques similar to those used in training. (7) Content. The focus of language intervention studies needs to be expanded to encompass aspects of language in addition to syntax, such as conversational language use [38, 39], and the interrelationships of language proficiency and other areas of the children’s functioning such as their social behavior/social emotional well-being , the quali- Treatment Research in Speech, Language and Swallowing Folia Phoniatr Logop 1998;50:165–182 Downloaded by: Florida State University 22.214.171.124 - 10/28/2017 6:57:16 PM 171 Intervention Research: Intervention Agents Another major type of child language treatment research has explored the treatment effectiveness of programs delivered by parents , teachers , and peers . Among these, programs designed for parents have had the longest history and have reflected the theoretical influences noted above. Early programs taught parents behavioral operant techniques while more recent programs have incorporated naturalistic interactive techniques . Examples of current programs include the Transactional Intervention Program , the Language Interaction Intervention Program , and the Hanen Program for Parents . Most parent-administered programs have tended to focus on the young child or children in the early stages of language development, have used naturalistic techniques including general stimulation and have involved limited numbers of subjects. Recently, studies by Fey et al. [53, 54] compared the treatment effectiveness of parent-administered programs and clinician-administered programs on the language development of preschool children. Control groups that did not receive treatment were included. The programs used focused stimulation procedures, which were similar to the definition of conversational recasts used by Camarata and Nelson  and Camarata et al. . Opportunities to learn the target were increased in semantically and pragmatically appropriate contexts, targets were modeled, recasts were provided, and children were not 172 Folia Phoniatr Logop 1998;50:165–182 directly prompted to use models. Adult communicative behaviors in these studies, however, included false assertions, contingent queries, and forced-alternative questions to stimulate child target productions. In addition, the clinician-administered programs included 10 min of imitation practice weekly. Three visits were made to the homes of parents participating in the program to monitor treatment fidelity. The results of both studies indicated that children made gains within both clinician-administered and parent-administered programs compared to the notreatment control groups. Gains were greatest for those childen who received clinician-administered intervention, however, in both phase I, the first 5-month period , and phase II, the second 5-month period . Although gains were not as great during phase II compared to phase I, they were again evident for both the parent-administered and clinician-administered programs compared to the no-treatment groups. Parents demonstrated that they could learn and use the techniques presented but they varied in their frequencies of use within posttest parent-child language samples. Ongoing monitoring was not possible during the parent-administered programs so it is unclear how often parents used the techniques during the two intervention periods. The results of the no-treatment control groups were particularly interesting. The notreatment group in phase I were children on the waiting list for services. The no-treatment group in phase II were children who had a hiatus in services due to summer vacation. During phase I, half of these children had been in the parent-administered group and half had been in the clinician-administered treatment group. The no-treatment groups within both studies did not demonstrate gains. The differences between the delayed treatment group (waiting list group) and both treatment groups Gallagher Downloaded by: Florida State University 126.96.36.199 - 10/28/2017 6:57:16 PM ty of their relationships , and their functioning in other modalities [42–44]. The age range that has been typically studied also needs to be extended beyond preschool to include early language intervention and the older child and adolescent . in phase I are understandable and an indication of the effectiveness of the programs. In phase II, the differences between the treatment groups and the members of the dismissal group (summer vacation group) who received clinician-administered programs in phase I are also understandable due to the inaccessibility of further services and also indicate the effectiveness of the programs. However, the lack of progress of children in the dismissal group who had received the parentadministered program in phase I suggests that the techniques the parents learned in phase I were not maintained in phase II in the absence of clinician support. Although the number of children involved was small, 10 per group, and should be interpreted with caution, it suggests that there may be a role for continued clinician involvement within parent-administered programs and the need for further investigation of combined parent- and clinician-administered programs. Girolametto et al.  also examined the effectiveness of a parent-administered focused stimulation program but the study used a pretest-posttest control group design with random assignment to an immediate parentadministered treatment group and delayedtreatment control group. There was no clinician-administered treatment group in this study. The definition of ‘focused stimulation’ was similar to the one used by Fey et al. [53, 54], but it was not clear whether the adult communicative behaviors used to stimulate child target productions that they had included were also included in this study. The subjects were 25 toddlers at the one-word stage of language development. Treatment fidelity was determined using checklists during three home visits. The results of this study also indicated that parents were able to learn and use the techniques and that children in the parent-administered treatment group made significant gains compared to those in the no-treatment group. Again individual differences were noted. Studies of teacher-implemented intervention programs are very limited at this point but suggest that teachers can also be effective in facilitating children’s language development and conversational behavior [56, 57]. This type of research will be important given the increasing use of consultative intervention models with school-aged language-disordered children [58, 59]. The role of peers as potential intervention agents has also been studied. There is a growing literature indicating that peers can be taught to initiate positive interactions more frequently with language-disordered children, to respond more positively to them, and to serve as peer models, monitors and recorders [16, 41, 48, 60, 61]. The impact of these interventions on language-disordered children’s language and communicative behavior as well as on their frequent peer isolation and rejection warrants further study. Although the literature on the effectiveness of intervention agents other than clinicians is more limited than that on intervention procedures delivered by clinicians, it highlights many of the same methodological issues (e.g. small N’s, etc.). The studies suggest that training others, particularly parents, can change their behavior and possibly enhance children’s language skills. Individual differences are consistently reported suggesting the potential influence of the same factors as were discussed earlier (e.g. subject characteristics, etc.). Furthermore, it is not clear whether parent-administered programs are more effective in increasing the use of language structures children have already acquired than in helping them learn new structures [see 49, 62, 63 for reviews]. The studies that have been reviewed, however, represent some of the most sophisticated research designs that have been used to date Treatment Research in Speech, Language and Swallowing Folia Phoniatr Logop 1998;50:165–182 Downloaded by: Florida State University 188.8.131.52 - 10/28/2017 6:57:16 PM 173 Implications for the Future As these exemplary studies of child language intervention illustrate, treatment research has become increasingly sophisticated 174 Folia Phoniatr Logop 1998;50:165–182 over the last two decades. A basic clinical intervention technology has been developed and tested, and numerous studies have indicated the positive effects of language intervention. As Bricker [14, p. 12] concluded in a recent review, the progress that has been made is a major achievement ‘considering the complexity of human learning, the complexity of language acquisition and usage, and the short time that investigators and practitioners have been attempting to develop and implement programs of language intervention’. Although this statement is specific to child language intervention, essentially the same conclusion could be reached across areas of speech, language and swallowing treatment research. We have learned a great deal about how to do treatment research during the last two decades and the proportion of studies addressing treatment issues is steadily increasing. There are two major challenges for the future. One is to address the types of questions that we have been researching with increasing precision so that the results can be interpreted more definitively. The other is to address questions that have been insufficiently researched, one of the most significant of these being the functional impact of treatments on patients’ lives and on the health care and educational service delivery systems of which they are a part. Both require that mechanisms be developed to facilitate the establishment and support the maintenance of large, collaborative networks among researchers and clinicians. Although it is certainly the case that progress on many of the methodological issues discussed will continue to be made at individual sites, for example, subject description, procedural description, treatment fidelity, and documentation, progress on others, such as interactions among specific treatment variables and particular patient characteristics, Gallagher Downloaded by: Florida State University 184.108.40.206 - 10/28/2017 6:57:16 PM to address questions about child language treatment and illustrate several important methodological issues in addition to those discussed in the earlier section. (1) Subject Selection Bias. The literature on parent-administered programs more than any other highlights the potential influence of subject selection bias on the results obtained. For example, parents who have participated in intervention studies have overwhelmingly been self-selected, highly motivated, middleclass mothers as was the case in the studies discussed above. Generalization of these findings to the population at large, therefore, cannot simply be assumed. (2) Treatment Fidelity. Treatment fidelity and frequency of technique use is very difficult to monitor in parent-administered programs. Periodic parent group meetings and language sampling techniques have been used [53–55], but parents’ behaviors in their homes are still difficult to obtain or reliably estimate. (3) No-Treatment Control Groups. Each of the studies reviewed highlighted the interpretive value of including no-treatment control groups within research designs. The means for doing so may introduce bias since they tended to take advantage of events that were beyond their control (e.g. summer vacation, waiting lists for services, etc.) and were not truly randomized designs. Despite this potential limitation, they provide a means for reducing some of the ethical concerns associated with withholding treatment and in so doing provide mechanisms by which important information can be added to the treatment literature. Clinical Trials Research In 1997 ASHA established the Communication Sciences and Disorders Clinical Trials Research Group (CSDRG) to provide a mechanism for preparing an application to the National Institutes of Health: National Institute on Deafness and Communication Disorders (NIDCD). The application was to support multi-center randomized clinical trials research in communication disorders. The application was successful and Dr. Jeri Logemann serves as the Principal Investivator of the project. Clinicians from various practice environments (e.g. acute care hospitals, schools, nursing homes, etc.) and researchers from various backgrounds (e.g. clinical and basic science researchers in communication disorders, statisticians, and epidemiologists) were brought together to design and execute the trials. The CSDRG and the clinical trials that are conducted will be funded by NIDCD and other institutes and foundations. The project is organized with ASHA serving as the fiscal intermediary. The CSDRG has two components. One component provides mechanisms for generating suggestions for clinical trials, provides technical support for designing proposals, and Treatment Research in Speech, Language and Swallowing presents these proposals to NIDCD for consideration. The other component, the protocol management group, supports the implementation of the clinical trial once it has been approved and funded. An Executive Policy Board formulates all policies relative to the conduct of the CSDRG and is assisted by three subcommittees, a Protocol Writing Subcommittee, a Publications Subcommittee, and a Quality Assurance Subcommittee. The Protocol Writing Subcommittee develops ideas and helps to prepare protocols for potential funding to NIDCD. The Publications Subcommitte is responsible for all policies regarding publications that emanate from the work of the group, and the Quality Assurance Subcommittee has the responsibility to develop and implement a program to ensure that standard practices are being uniformly followed across all centers participating in the trials. The CSDRG also established five clinician-researcher liaison groups, one group in each of the following areas: speech disorders, adult language disorders, child language disorders, voice disorders, and hearing impairment/balance. Clinician-researcher groups reflecting various practice settings (e.g. inpatient, outpatient, and schools) were also established to consider issues that may be setting specific factors that might introduce settingrelated considerations in the conduct of a trial. Currently one trial has been initiated. It will examine two treatments for aspiration of liquids in elderly patients with dementia or parkinsonism. Patient accrual began in early 1998 and will continue for 4 years. It is anticipated that other trials will begin in time and eventually several trials will be conducted through the CSDRG simultaneously. These efforts represent a significant advance in the development of treatment research technologies in speech, language and swallowing, and Folia Phoniatr Logop 1998;50:165–182 175 Downloaded by: Florida State University 220.127.116.11 - 10/28/2017 6:57:16 PM will be unlikely without the large patient numbers multi-site-collaboration permits. This is even more true for the two types of treatment research that have been severely underrepresented within the literature to date, treatment efficacy research and outcomes research. Both essentially require large collaborative networks among clinicians and researchers. Since 1993, ASHA has helped to build and support collaborative networks of researchers and professionals in order to conduct clinical trials and functional outcomes research. Those efforts will be summarized below. Treatment Outcomes Research ASHA began its efforts to provide treatment outcomes information on speech-language pathology and audiology services by forming a task force, the Task Force on Treatment Outcomes and Cost Effectiveness (TOCE) in 1993. Nancy Swigert and Tanya Gallagher were Co-project Officers of the Task Force and Herbert Baum (Director of ASHA’s Science and Research Department), Robert Augustine, Audrey Holland, Raymond Kent, Roberta Kreb, Susan Russell, and Kenneth Wolf served as members. Among its early activities, the Task Force evaluated the treatment outcomes data collection systems that were being used nationally. Three major instruments for adults were being used, the Functional Independence Measure (FIM) , the LORS-III , and RESTORE . The first two instruments were designed for outpatient rehabilitation settings and the third was designed for inpatient settings. The UDSmr encompasses the FIM and is one of the largest nationally aggregated rehabilitation databases available within the United States. Patient FIM data became available to participating facilities in 1990 . As of June, 1993, over 300,000 patient records from 470 inpatient medical facilities in 48 states had been entered into the database. The FIM uses a seven-point rating scale with 1 indicating total dependence and 7 indicating complete independence. Communication and swallowing items include eating, auditory comprehension, visual comprehension, vocal expression, nonvocal expression, social 176 Folia Phoniatr Logop 1998;50:165–182 interaction, problem solving, and memory. After studying the FIM the Task Force concluded several important factors limited its ability to adequately represent treatment outcomes in speech, language, and swallowing. These included: items that were not specific enough to adequately characterize the communication processes being assessed; unaccounted for differences among the health care professionals’ ratings of communication-relevant items; ratings were entered for all scales regardless of their appropriateness for certain patient groups (e.g. communication items for hip replacement patients); ratings were entered whether or not intervention was provided or if provided, how often, and the measure lacked sufficient sensitivity to reflect changes in communication. The other adult instruments and the child instruments that were later examined, the Pediatric Evaluation of Disability Inventory  and the Wee FIM , had many of the same limitations. The Task Force concluded that despite the availability and convenience of using these already established instruments and databases, ASHA needed to develop instruments and create a national database for speech-language pathology and audiology services. Only a national outcomes measurement system would be comprehensive enough and rigorous enough to allow national benchmarking, comparisons of treatment services to a national average, which was an important feature national and federal agencies were beginning to demand. The TOCE developed the basic features of the National Outcomes Measurement System in Speech-Language Pathology and Audiology (NOMS), formulated a strategic plan for implementation of the multi-phase project, and completed its work in 1996. Since that time the National Center for Treatment in Communication Disorders has been established within ASHA’s National Office and the NOMS project has been proceeding with the help of an Gallagher Downloaded by: Florida State University 18.104.22.168 - 10/28/2017 6:57:16 PM ultimately in the knowledge base of the professions and the discipline . Another important ASHA initiative has been treatment outcomes research. National Outcomes Measurement System for Speech-Language Pathology and Audiology Four speech-language pathology components of the NOMS have been developed : NOMS: Adult , NOMS: Kindergarten – Grade 12 (Health) , NOMS: Kindergarten – Grade 12 (Education) , and NOMS: Pre-Kindergarten . The NOMS: Adult has been field-tested, and undergone reliability and validity testing. Field-testing for the other components is ongoing, and reliability and validity testing will follow. All NOMS components contain four forms: Admission/Entrance, Discharge/Evaluation of Services, Consumer Satisfaction (optional), and Financial Information (optional). The K-12 (Education) component also contains a Functional Status Measures form. Teachers are asked to use this form to rate children’s communication within classrooms and its impact on their educational participation. The functional communication status of patients/clients is measured using 7-point scales at both admission or entrance into treatment and at discharge or other evaluation points, such as the end of the school year. Functional communication measure (FCM) ratings are entered only for those communication disorders being treated. Expert panels were consulted to write the FCMs, and each point on the scales represents successive stages of communicative improvement for the communicative disorder being treated. In addition, the following information is obtained on the Admission/Entrance and Discharge/Evaluation of Services forms: (1) treatment setting; the type of facility on the Treatment Research in Speech, Language and Swallowing NOMS: Adult forms (e.g. acute care hospital, skilled nursing unit, rehabilitation hospital, etc.), or instructional context on the NOMS: K-12 or Pre-K forms (e.g. day care, special education classroom, preschool, ect.); (2) medical diagnosis and associated factors (e.g. seizures, bronchopulmonary dysplasia, etc.); (3) primary communication disorder; (4) date services began and ended; (5) number of treatment units provided, coded in 15- to 20-min units, and if support personnel were involved, the number of treatment units by each type of service provider; (6) model of SLP services used (e.g. direct treatment, collaborative/consultative, etc.) and if more than one model was used, the percentage of treatment units provided using each type; (7) whether treatment goals were met and if not the primary reason (e.g. funding issues, medical complications, discharged to another program, etc.), and (8) where discharged and whether continued treatment was recommended. The Consumer Satisfaction Form asks adult patients/clients or their spouses/caregivers if they are unable to complete the form, and parents of children to estimate their agreement with the statements provided using a 5-point scale that ranges from ‘strongly agree’ to ‘strongly disagree’. Statements include that their communication/swallowing abilities improved as a result of the treatment provided, that those abilities would not have improved without treatment, etc. The Financial Form indicates total charges, funding sources, and reimbursements by funding source. In order to ensure the comparability of FCM ratings across clinicians and, therefore, the integrity of the national database, a rater registration program was developed. All NOMS users are provided with training manuals and must pass a test at the 85% level or above in order to be included in the registry. Folia Phoniatr Logop 1998;50:165–182 177 Downloaded by: Florida State University 22.214.171.124 - 10/28/2017 6:57:16 PM administrative advisory team, Rob Mullen, Director of the Center, Herbert Baum, Director of the Science and Research Department, Nancy Swigert, and Tanya Gallagher. Primary communication disorder Admission Change Discharge Swallowing Speech production Spoken language comprehension Spoken language production Voice 3.3 4.0 3.9 3.6 3.9 4.8 5.0 4.8 4.6 5.1 Only data from registered users is entered into the national database. The National Center maintains the national database, the registration and subscriber programs, and provides each subscriber program with quarterly reports summarizing their specific data and national reports summarizing the aggregate data from all participating programs. All national data is kept strictly confidential NOMS data can be used by facilities to describe functional changes that typically follow provision of speech-language pathology services, to negotiate with reimbursement agencies and policy makers on a more informed basis, to meet accreditation standards, to benchmark against national and regional trends, to facilitate program planning and evaluation, and to project potential impacts of proposed program changes on resource and utilization patterns. Having these kinds of data helps professionals deliver, plan, and advocate for their services more effectively. The database is also an important means of identifying clinically focused research questions that can guide the growth of the knowledge base of the professions. For example, identifying unexplained variations in outcome can lead us to explore the differences that might exist at the level of procedures and direct us to frame critical studies of those differences. The data can also direct us to investigate areas of intervention where outcomes are not as positive as we might have thought 178 Folia Phoniatr Logop 1998;50:165–182 1.4 0.9 0.9 1.0 1.1 and focus research energies on the development of new procedures. Preliminary data from 1,638 patients included in the field testing of the NOMS: Adult component was published in June, 1996 . The data presented in table 2 is a portion of that data set. It summarizes the mean FCM scores at admission, the mean change in FCM scores following treatment, and the mean FCM scores at discharge for five of the primary communication disorders of adults treated within acute inpatient facilities. The data indicates that patients with swallowing disorders are admitted to treatment within acute care inpatient settings at approximately FCM level 3, on average. The description at that level read: ‘Swallowing disorder prevents eating for a portion of nutritional needs and oneto-one supervision is required for eating.’ The mean change score was 1.4 levels and the mean discharge score was 4.8. Patients are therefore discharged on average at approximately FCM level 5. The description at that level read: ‘Swallowing is function to meet nutritional needs, although self-monitoring and compensatory techniques are used.’ The data in table 3 summarizes the same types of data for the same five primary communication disorders. These data indicate that patients who were treated for spoken language comprehension problems within rehabilitation inpatient facilities were admitted, on average, at approximately FCM level 4. Gallagher Downloaded by: Florida State University 126.96.36.199 - 10/28/2017 6:57:16 PM Table 2. Mean admission, change, and discharge FCM scores from preliminary testing of NOMS:Adult within acute inpatient settings Primary communication disorder Admission Change Discharge Swallowing Sprech production Spoken language comprehension Spoken language production Voice 3.7 3.7 4.4 3.8 – 5.2 4.9 5.4 5.0 – 1.5 1.2 1.0 0.8 – The description at that level read: ‘Comprehension of spoken language is limited to the primary activities of daily living needs and sample ideas and frequently requires repetition and/or rephrasing.’ The mean change score was 1.0 levels and the mean discharge score was 5.4. The description at that level read: ‘Comprehension of spoken language is normal for activities of daily living, but limited in complexity of form, content, or use; self-monitoring is inconsistent.’ As these two examples illustrate the overall data trends, across all primary communication disorders and service delivery settings, indicated that speech, language, and swallowing treatment led to greater functional independence and a reduction in the amount of patient care needed. Such reductions can be realized as economies throughout the health care service delivery (e.g. reduced levels of nursing care required, shortened length of stays, etc.). All the data collected from field testing of the components to date has been incorporated into sample ‘National Report Cards’ and is available from ASHA upon request. As the database grows, it will be possible to analyze critical data subsets. For example, the database will permit interactive analyses of outcomes by number of treatment units, intensity of treatment, type of setting, model of service delivery, type of provider, etc. This information will make the database even more valuable. The features highlighted were those that the project team found to be most important in designing a treatment outcomes system. There is always a tension between the specificity and comprehensiveness of the information collected and the amount of time it will take clinicians to complete the necessary forms. Clinicians found it difficult to accommodate data input demands to ongoing time pressures of practice, and forms were continually refined to include only the minimal critical data set. Further accommodations involved the development of automated input programs. It is clear that we must all continue to learn how to most efficiently and effectively collect outcomes data within the context of standard service delivery. Two further challenges facing treatment outcomes research are the difficulties encountered in tracking continuity of care and in adequately accounting for patient risk adjustment and co-morbidities. Although we were able to sample patient ouctomes within different types of settings, it was not possible to track individual patients as they moved from one type of treatment setting to another. It would provide important data to be able to record admission and discharge outcomes from each successive facility in which patients were treated. For those health care systems providing different levels of care within the same administrative organization, a common NOMS patient number could be maintained Treatment Research in Speech, Language and Swallowing Folia Phoniatr Logop 1998;50:165–182 179 Downloaded by: Florida State University 188.8.131.52 - 10/28/2017 6:57:16 PM Table 3. Mean admission, change, and discharge FCM scores from preliminary testing of NOMS:Adult within rehabilitation inpatient facilities across all facilities. Similarly within school systems a common student number could be used as long as children maintained their enrollment within the same school district. NOMS project will be attempting to implement these procedures. Another challenge is adequately adjusting FCM scores for medical risks and co-morbidities. Although these are identified on the NOMS forms, it is difficult to effectively incorporate such information into outcomes data analyses. These and other challenges, including the long-term impacts of the functional changes noted, will need to be addressed as we continually refine the methodologies used in treatment outcomes research. Summary and Conclusions Intervention research is methodologically complex, logistically difficult, and resourceintensive. In light of these features, the amount of progress that has been made in the last 2–3 decades in speech, language, and swallowing treatment research is impressive. In order to realize its full potential the next generation of work will require increased cooperation and collaboration among researchers and clinicians. Large data collection networks will be necessary to explore the types of interactive variable studies suggested by the child language intervention literature and to pursue treatment efficacy and treatment outcomes research on the scale needed. In order to meet these needs, efforts will have to be expended by organizations that can provide sufficient network infrastructure and support such as those discussed that are being pursued by ASHA. It will also be necessary for researchers and clinicians to be willing to work together within these networks in order for their potential to be realized. If it is possible for us to do so, the next decade promises to provide answers that are critically important to the efficient and effective delivery of clinical practice and to the evolution of the discipline underlying that practice. OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO References 180 5 Johnston M, Granger C: Outcomes research in medical rehabilitation. Am J Phys Med Rehabil 1994;73: 296–302. 6 Piccirillo J: Outcomes research in clinical practice. Insights Otolaryngol 1993;8:2–8. 7 Piccirillo JF: Outcomes research and otolaryngology. Otolaryngol Head Neck Surg 1994;111:764– 769. 8 Frattali CM: Outcomes measurement: Definitions, dimensions, and perspectives; in Fratalli C (ed): Measuring Outcomes in SpeechLanguage Pathology. New York, Thieme, 1997, pp 55–88. 9 Kreb R, Wolf K: Successful Operations in the Treatment-OutcomesDriven World of Managed Care. NSSLHA Clinical Series 13. Rockville, 1997. Folia Phoniatr Logop 1998;50:165–182 10 Fuhrer M: Rehabilitation Outcomes: Analysis and Measurement. Baltimore, Brookes, 1987. 11 Bereiter C, Engelmann S: Teaching Disadvantaged Children in the Preschool. Englewood Cliffs, PrenticeHall, 1966. 12 Guess D, Sailor W, Rutherford G, Baer D: An experimental analysis of linguistic development: The productive use of the plural morpheme. J Appl Behav Anal 1968;1:297–306. 13 Bricker D: The changing nature of communication and language intervention; in Warren S, Reichle J (eds): Causes and Effects in Communication and Language Intervention. Baltimore, Brookes, 1992, pp 361–375. 14 Bricker D: Then, now, and the path between: A brief history of language Gallagher Downloaded by: Florida State University 184.108.40.206 - 10/28/2017 6:57:16 PM 1 Olswang L: Treatment efficacy research: A paradism for investigating clinical practice and the theory. Fluency Disord 1993;18:125–131. 2 Olswang L: Treatment efficacy research; in Frattali C (ed): Measuring Outcomes in Speech-Language Pathology. New York, Thieme, 1997, pp 134–150. 3 Gallagher TM: National initiatives in outcomes measurement; in Frattali C (ed): Measuring Outcomes in Speech-Language Pathology. New York, Thieme, 1997, pp 527–557. 4 Byng S, Van der Gaag A, Parr S: International initiatives in outcomes measurement: A perspective from the United Kingdom; in Frattali C (ed): Measuring Outcomes in Speech-Language Pathology. New York, Thieme, 1997, pp 558–578. 16 17 18 19 20 21 22 23 24 Treatment Research in Speech, Language and Swallowing 25 Camarata S, Nelson K, Camarata M: Comparison of conversationalrecasting and imitative procedures for training grammatical structures in children with specific language impairment. J Speech Hear Res 1994;37:1414–1423. 26 Waryas C, Stremel-Campbell K: Communication Training Program. New York, Teaching Resources, 1983. 27 Hart B, Risley T: Establishing the use of descriptive adjectives in the spontaneous speech of disadvantaged preschool children. J Appl Behav Anal 1968;1:109–120. 28 Hart B, Risley T: In vivo language intervention: Unanticipated general effects. J Appl Behav Anal 1980;7: 243–256. 29 Friedman P, Friedman K: Accounting for individual differences when comparing the effectiveness of remedial language teaching methods. Appl Psycholinguist 1980;1:151– 170. 30 Mirenda P, Donnellan A: Effects of adult interaction style on conversational behavior in students with severe communication problems. Lang Speech Hear Services Schools 1986;17:126–141. 31 Yoder P, Kaiser A, Alpert C, Fischer R: Following the child’s lead when teaching nouns to preschoolers with mental retardation. J Speech Hear Res 1993;36:158–167. 32 Snow D, Swisher L, McNamara M, Kiernan B: A potential limitation of treatment efficacy research: A comment on Camarata, Nelson, and Camarata (1994). J Speech Hear Res 1996;39:221–222. 33 Hepting N, Goldstein H: What’s natural about naturalistic language intervention? J Early Intervent 1996;20:249–265. 34 Fey M, Cleave P: Early language intervention. Semin Speech Lang 1990;2:165–181. 35 Scarborough H, Wyckoff J, Davidson R: A reconsideration of the relation between age and mean utterance length. J Speech Hear Res 1986;29:394–399. 36 McTear M, Conti-Ramsden G: Pragmatic Disability in Children. San Diego, Singular Publishing Group, 1992. 37 Plante E, Swisher L, Kiernan B, Restrepo M: Language matches, illuminating or confounding? J Speech Hear Res 1993;36:772–776. 38 Brinton B, Fujiki M: Conversational intervention with children with specific language impairment; in Fey M, Windsor J, Warren S (eds): Language Intervention: Preschool through the Elementary Years. Baltimore, Brookes, 1995, pp 183–213. 39 Craig H, Washington J: Access behaviors of children with specific language impairment. J Speech Hear Res 1993;36:322–337. 40 Durand V, Carr E: Functional communication training to reduce challenging behavior: Maintenance and application in new settings. J Appl Behav Anal 1991;24:251–264. 41 Gallagher T: Social-interactional approaches to child language intervention; in Beitchman J, Konstantareas M (eds): Language, Learning, and Behavior Disorders: Emerging Perspectives. Cambridge, Cambridge University Press, 1996, pp 418– 435. 42 Perera K: Language acquisition and writing; in Fletcher P, Garman M (eds): Language Acquisition, ed 2. New York, Cambridge University Press, 1986, pp 494–533. 43 Perera K: Reading and writing skills in the National Curriculum; in Fletcher P, Hall D (eds): Specific Speech and Language Disorders in Children: Correlates, Characteristics and Outcomes. San Diego, Singular Publishing Group, 1992, pp 183–193. 44 Gillam R, Johnston JR: Spoken and written language relationship in language/learning-impaired and normally achieving school-age children. J Speech Hear Res 1992;35:1303– 1315. 45 Scott C: A discourse continuum for school-age students: Impact of modality and genre; in Wallach G, Butler K (eds): Language Learning Disabilities in School-Age Children and Adolescents: Some Underlying Principles and Applications. Columbus, Merrill-Macmillan, 1994, pp 219– 252. 46 Klein D, Briggs M: Facilitating mother-infant communicative interaction in mothers of high-risk infants. J Child Commun Disord 1987;10:95–106. Folia Phoniatr Logop 1998;50:165–182 181 Downloaded by: Florida State University 220.127.116.11 - 10/28/2017 6:57:16 PM 15 intervention; in Kaiser A, Gray D (eds): Enhancing Children’s Communication: Research Foundations for Intervention. Baltimore, Brookes, 1993, pp 11–31. Craig H: Pragmatic characteristics of the child with specific language impairment: An interactionist perspective; in Gallagher T (ed): Pragmatics of Language: Clinical Practice Issues. San Diego, Singular Publishing Group, 1991, pp 163–198. Gallagher T: Language and social skills: Implications for clinical assessment and intervention with school-age children; in Gallagher T (ed): Pragmatics of Language: Clinical Practice Issues. San Diego, Singular Publishing Group, 1991, pp 11–41. Warren S, Reichle J: The emerging field of communication and language intervention; in Warren S, Reichle J (eds): Causes and Effects in Communication and Language Intervention. Baltimore, Brookes, 1992, pp 1–8. Fey M: Language Intervention with Young Children. San Diego, College Hill Press, 1986. Goldstein H, Hockenberger E: Significant progress in child language intervention: An 11-year retrospective. Res Dev Disabil 1991;12:401– 424. Nye C, Foster S, Seaman D: Effectiveness of language intervention with the language/learning disabled. J Speech Hear Disord 1987;52:348– 357. Yoder P, Kaiser A, Alpert C: An exploratory study of the interaction between language teaching methods and child characteristics. J Speech Hear Res 1991;34:155–167. Connell P: An effect of modeling and imitation teaching procedures on children with and without specific language impairment. J Speech Hear Res 1987;30:105–113. Connell P, Stone C: Morpheme learning of children with specific language impairment under controlled instructional conditions. J Speech Hear Res 1992;35:844–852. Camarata S, Nelson K: Treatment efficiency as a function of target selection in the remediation of child language disorders. Clin Linguist Phonet 1992;6:167–178. 182 56 Schuele C, Rice M: Redirects: A strategy to increase peer initiations. J Speech Hear Res 1995;38:1319– 1333. 57 Dyer K, Williams L, Luce S: Training teachers to use naturalistic communication strategies in classrooms for students with autism and other severe handicaps. Lang Speech Hear Services Schools 1991;22:313–321. 58 Damico J: Addressing language concerns in the schools. The SLP as consultant. J Child Commun Disord 1987;11:17–40. 59 Secord W: Best practices in school speech-language pathology; in Secord W (ed): Collaborative Programs in the Schools: Concepts, Models and Procedures. San Antonio, The Psychological Corporation, 1990. 60 Goldstein H, Kaczmarek L: Promoting communicative interaction among children in integrated intervention settings; in Warren S, Reichle J (eds): Causes and Effects in Communication and Language Intervention. Baltimore, Brookes, 1992, pp 81–111. 61 Goldstein H, Wickstrom S: Peer intervention effects on communicative interaction among handicapped and nonhandicapped preschoolers. J Appl Behav Anal 1986;19:209–214. 62 Tannock R, Girolametto L: Reassessing parent-focused language intervention programs; in Warren S, Reichle J (eds): Causes and Effects in Communication and Language Intervention. Baltimore, Brookes, 1992, pp 40–79. 63 Ostrosky M, Kaiser A, Odom S: Facilitating children’s social-communicative interactions through the use of peer-mediated interventions; in Kaiser A, Gray D (eds): Enhancing Children’s Communication: Research Foundations for Intervention. Baltimore, Brookes, 1993, pp 159–185. Folia Phoniatr Logop 1998;50:165–182 64 Baum H, Logemann J, Lilienfeld D: Clinical trials and its application to communication sciences and disorders. J Med Speech-Lang Pathol, in press. 65 State University of New York at Buffalo: Guide for Use of the Uniform Data Set for Medical Rehabilitation: Functional Independence Measure. Buffalo, Research Foundation, 1993. 66 Formations in Health Care, Inc: LORS-III. Chicago, 1991. 67 Formations in Health Care, Inc: RESTORE. Chicago, 1993. 68 Granger C, Hamilton B: The uniform data system for medical rehabilitation report of first admissions for 1990. Am J Phys Med 1992;71: 106–113. 69 Haley S, Coster W, Ludlow L: Pediatric Evaluation of Disability Inventory (PEDI): Development, Standardization and Administration Manual. Boston, New England Medical Center Hospital, 1992. 70 Uniform Data System: Functional Independence Measure for Children (WeeFIM). Buffalo, State University of New York, 1990. 71 ASHA: NOMS: Adult. Rockville, 1998. 72 ASHA: NOMS: Kindergarten – Grade 12 (Health). Rockville, in press. 73 ASHA: NOMS: Kindergarten – Grade 12 (Education). Rockville, in press. 74 ASHA: NOMS: Pre-Kindergarten. Rockville, in press. 75 TOCE – Task Force on Treatment Outcome and Cost Effectiveness: ASHA’s Treatment Outcomes Data Collection in Adult Health Care Settings: National Report Cards 1–10. Rockville, ASHA, 1996. Gallagher Downloaded by: Florida State University 18.104.22.168 - 10/28/2017 6:57:16 PM 47 Haring T, Neetz J, Lovinger L, Peck C, Semmel M: Effects of four modified incidental teaching procedures to create opportunities for communication. J Assoc Persons Severe Handicaps 1987;12:218–226. 48 Goldstein H, Ferrell D: Augmenting communicative interaction between handicapped and nonhandicapped preschool children. J Speech Hear Disord 1987;52:200–211. 49 Kaiser A: Parent-implemented language intervention: An environmental system perspective; in Kaiser A, Gray D (eds): Enhancing Children’s Communication: Research Foundations for Intervention. Baltimore, Brookes, 1993, pp 63–84. 50 Mahoney G, Powell A: Transactional Intervention Program: Teacher’s Guide. Farmington, Pediatric Research and Training Center, University of Connecticut Health Center, 1986. 51 Weistuch L, Lewis M, Sullivan M: Use of a language interaction intervention in the preschools. J Early Intervent 1991;15:278–287. 52 Girolametto L, Greenberg J, Manolson A: Developing dialogue skills: The Hanen early language parent program. Semin Speech Lang 1986; 7:367–382. 53 Fey M, Cleave P, Long S, Hughes D: Two approaches to the facilitation of grammar in language-impaired children: An experimental evaluation. J Speech Hear Res 1993;36: 141–157. 54 Fey M, Cleave P, Long S: Two models of grammar facilitation in children with language impairments: Phase 2. J Speech Lang Hear Res 1997;40:5–19. 55 Girolametto L, Pearce P, Weitzman E: Interactive focused stimulation for toddlers with expressive vocabulary delays. J Speech Hear Res 1996; 39:1274–1283.