close

Вход

Забыли?

вход по аккаунту

?

The effect of relationship type on reasoning strategies for systems understanding

код для вставкиСкачать
The Effect of Relationship Type on Reasoning Strategies
for Systems Understanding
Julie H. Youm
Submitted in partial fulfillment of the requirements for
the degree of Doctor of Philosophy
under the Executive Committee of the Graduate School of
Arts and Sciences
COLUMBIA UNIVERSITY
2010
UMI Number: 3420732
All rights reserved
INFORMATION TO ALL USERS
The quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
UMT
Dissertation Publishing
UMI 3420732
Copyright 2010 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, Ml 48106-1346
©2010
Julie H. Youm
All Rights Reserved
ABSTRACT
THE EFFECT OF RELATIONSHIP TYPE ON REASONING STRATEGIES
FOR SYSTEMS UNDERSTANDING
Julie H. Youm
This study investigates whether explicit training and attention to functional
relationships can improve the reasoning and understanding about complex physical
systems better than training and attention to causal relationships alone. Functional
relationships describe the nature or function of change between two objects and are
considered dynamic, complex counterparts to simple, static causal relationships.
The causal models of physical systems that are developed in one's mind are
critical in allowing for the generation of predictions and explanations of a system's
behavior. However, learners often do not account for the complex nature of causality in
their causal models, likely due to their limited perceptions of causality itself (Chi, 2000;
Grotzer, 2003; Wilensky & Resnick, 1999). To address these limitations, a focus on
functional relationships is proposed as an instructional approach to engage learners in an
active perspective and evoke fluid views of system interactions to better enable mental
simulation and mental model development (Tsuei, 2004; Chan & Black, 2006).
In this study, participants were randomly assigned to either the Functional group,
who received training on functional relationships, the Causal group, who received
training on causal relationships, or to a non-treatment control group. They then learned
about a system selected for the study by reading an article and constructing relationships,
or taking notes in the control group. Participants were asked in written posttests and
structured interviews about their understanding of the systems and the strategies used in
constructing relationships or taking notes. A list of representative strategies including
mental simulation, looking for definitions, and considering the ability to manipulate
objects was identified after reviewing the interview transcripts.
Findings suggest that functional relationships engage different reasoning
strategies than causal relationships when learning about complex systems, specifically
those that are considered to elicit dynamic thinking. Though causal models of systems
may be more salient for learners, an emphasis on functional relationships appears to be a
more effective means to promote robust reasoning about a system. The positive findings
from this study provide educators a revised perspective from which to present
information to students, specifically, to stimulate functional reasoning when designing
materials on complex topics.
TABLE OF CONTENTS
LIST OF TABLES
v
LIST OF FIGURES
vii
ACKNOWLEDGEMENTS
viii
CHAPTER 1
1
INTRODUCTION
1
What is a System?
2
Relationships in Systems
3
Research Hypotheses
5
Research Questions
6
Hypotheses
6
Overview of Dissertation
7
CHAPTER II
8
LITERATURE REVIEW
8
Complex Systems
8
The Properties of a Complex System
8
Issues in Understanding Complex Systems
10
Mental Models of Complex Systems
12
Relationships in Complex Systems
16
Causal Reasoning
17
The Development of Causal Reasoning
19
Explaining and Representing Causal Reasoning
20
Causal Relationships for Systems Understanding
25
i
Limited Perceptions of Causality
27
Addressing Limitations of Causal Reasoning
29
Functional Relationships
33
Functional Relationships and Mental Simulation
Overview of Study
35
38
Distinguishing between Causal and Functional Relationships
CHAPTER III
39
42
METHOD
42
Design
42
Participants
42
Procedure
44
Nomenclature
46
Materials
47
Measures, Coding and Inter-rater Reliability
52
CHAPTER IV
59
RESULTS
59
Independent Variables
59
Dependent Measures
59
Group Differences
59
Research Questions and Hypotheses
61
Analysis for the First Hypothesis
62
Analysis for the Second Hypothesis
69
Investigating Transfer Effects
79
CHAPTER V
86
DISCUSSION
86
ii
Overview of Study Results
86
The Effect of Relationship Type on Reasoning Strategies
86
The Effect of Functional Relationships on System Understanding
91
Transfer Task Results
95
Limitations
98
Implications
102
Why Study Functional Relationships?
103
Educational Implications
105
REFERENCES
107
APPENDICES
107
Appendix A: System Pretest and Posttest
117
Appendix B: Causal Relationships Training
118
Appendix B: Functional Relationships Training
119
Appendix C: Causal Relationship Intervention Worksheet for the Greenhouse
Effect
120
Appendix C: Functional Relationship Intervention Worksheet for the Space
Elevator
121
Appendix C: Control Group Intervention Worksheet for the Space Elevator.... 122
Appendix D: Greenhouse Effect Causal Group Posttest Interview Protocol
123
Appendix D: Space Elevator Functional Group Posttest Interview Protocol
124
Appendix D: Greenhouse Effect Control Group Posttest Interview Protocol.... 125
Appendix E: Demographic Questionnaire
126
Appendix F: Greenhouse Effect Article
128
Appendix G: Space Elevator Article (Explicit Functional Relationships in Bold)
129
Appendix H: Training Examples Worksheet
iii
130
Appendix I: Training Relationship Examples
131
Appendix J: Transfer Task
132
Appendix K: Target Elements, Roles and Mechanisms
133
Appendix L: Training Procedure for Systems Understanding Coding
140
Appendix M: Training Procedure for Reasoning Strategies Coding
142
Appendix N: Training Procedure for What-If Coding
146
Appendix O: Training Procedure for Transfer Task Coding
149
IV
LIST OF TABLES
Table 1. Examples of Complex Systems (Bar-Yam, 2003, p. 8)
9
Table 2. Complex Systems Mental Models Framework (Jacobson, 2000)
14
Table 3. Participant Group Sizes by Treatment and System Assignment
44
Table 4. Systems Article Comparison by Word Count, Elements, Roles and Mechanisms
48
Table 5. Percent Agreement for Written Pretest and Posttest Coding
53
Table 6. Percent Agreement and Intraclass Correlation Coefficients for Reasoning
Strategy Coding
56
Table 7. Greenhouse Effect: Means and Standard Deviations of Percentage Recall
Pretest Scores by Group
60
Table 8. Space Elevator: Means and Standard Deviations of Percentage Recall Pretest
Scores by Group
60
Table 9. Definitions and Examples for the Target List of Reasoning Strategies
62
Table 10. Participant Report of Reasoning Strategies by Group
65
Table 11. Contingency Tables for the Cause and Effect (CE) and Functional Relations
(FR) Strategies with Dynamic Strategies (DS) and Acknowledging Difficulty (DF)
68
Table 12. Greenhouse Effect: Means and Standard Deviations of Percentage Recall
Difference Scores by Group
70
Table 13. Space Elevator: Means and Standard Deviations of Percentage Recall
Difference Scores by Group
71
v
Table 14. Target Items from the Explicit Functional Relationships in the Space Elevator
Article
74
Table 15. Means and Standard Deviations of Percentage Recall Derived from Functional
Relationships in the Structured Interview
76
Table 16. Means and Standard Deviations of Percentage Recall Derived from Functional
Relationships in the Written Posttest
78
Table 17. Greenhouse Effect: Frequency of What-If Score by Group
80
Table 18. Space Elevator: Frequency of What-If Score by Group
80
Table 19. Greenhouse Effect: Frequency of Mars City Transfer Task Complexity by
Group
82
Table 20. Space Elevator: Frequency of Mars City Transfer Task Complexity by Group
82
VI
LIST OF FIGURES
Figure 1. Causal Bayes Net Example (Gopnik& Schulz, 2004)
22
Figure 2. Simple Causal Bayes Net Example (Glymour & Danks, 2008)
23
Figure 3. Progression of Conceptual Models for Simple Electrical Circuits (Grotzer &
Sudbury, 2000)
26
Figure 4. An Example of a STELLA Model Representing the Depletion of the Ozone
Layer
34
Figure 5. An Example of a Simulation Diagram for the Rock Cycle (Tsuei, 2004)
36
Figure 6. Causal to Functional Relationship Continuum
39
Figure 7. Distribution of Participant Majors
43
Figure 8. SCST Training Picture Triad
51
Figure 9. Greenhouse Effect: Performance Level by Strategy Use on Mars City Transfer
Task
85
Figure 10. Percentage Recall from the Structured Interview for the Space Elevator
System
94
vii
ACKNOWLEDGEMENTS
The world is complex and what I learned from studying complexity is that it
really takes a village to get a lot of things accomplished, and this dissertation was no
exception. Throughout the years, an interaction with so many helping hands have helped
me get to the day where writing this page is possible.
First, I thank my committee: my advisor, Professor John Black, for working with
me from near and far, through many revisions of my interests and goals, and providing a
greatly appreciated fellowship to jumpstart my academic career; Professors Herb
Ginsburg, O. Roger Anderson and Matthew Johnson, for providing the extra time,
professionalism and critical feedback during the various stages of the dissertation;
Professor Barbara Tversky, who allowed me into her classroom to recruit priceless
participants; Professor Lisa Son, for the refreshing perspectives provided at the defense
for future work; and to Gary Ardan, who felt like a part of my committee at every step of
the Ph.D. process.
Next, without the help of my colleagues, all of whom I consider good friends,
Chris Peterson, Lisa Tsuei, Janet Eisenband Sorkin, Jamie Krenn, Robyn Min and Insook
Han, I know it would have been impossible to code and tackle any of the data for this
study. And for years of statistically significant (yes, I had to throw that in because now I
know what it means) support in so many ways (papers, lunches, showers, drinks,
marriages, children, and lots of coffee) during my years at Teachers College, I am
indebted to Shuli Gilutz, Janet Eisenband Sorkin, David Shaenfield, Michael Preston,
Jared Katz, Seokmin Kang, Chaille Maddox, Genevieve Hartman, Melissa Morgenlander,
viii
Martin Pusic and especially, Susan Jang, who has been my carbon composite ribbon (bad
space elevator joke!) to it all during my times away from campus.
Finally, the family often seems to come last in these acknowledgements, but it is
obvious that this happens because they are the most hard to thank. Words will never
seem to be enough to express the gratitude for the patience and understanding that comes
with the extra cooked meals, the rides to the airports, the unexpected transcribing of
interviews, the frantic phone calls, hugs, and on and on. Daddy, Mommy, Ann, Robyn,
Jae, Henry, Jadyn, Avery, Ava, Gomo, Java, Madison (who sat by my feet patiently
through hours of writing) and Somi Lee (who is like family), I hope each of you know
how thankful I am for your being in my life and supporting me always. And to Paul and
my new extended family, who came into my life at the hardest time, thank you for
wholeheartedly encouraging me through the end.
ix
1
CHAPTER I
INTRODUCTION
Let's do a thought experiment. It's a beautiful weekend day and you don't mind
the drive to your favorite brunch spot on the other side of town. You get in the car and
onto the freeway. Traffic. What do you think caused it? There was an accident. There
was road construction. There was an accident caused by the road construction.
Or, do you consider that this traffic was the result of numerous interactions
between many cars of different sizes, moving at varying speeds, operated by drivers with
individual driving behaviors (e.g., driving fast, driving slow, changing lanes often, never
changing lanes) that reached a sort of equilibrium, i.e., a standstill on the freeway?
If you thought the former, you are in good company. The explanation is simple
and it makes sense. However, as characterized by the latter, a traffic jam is a classic
example of a complex system (Jacobson, 2000; Wilensky & Resnick, 1999), and what
appears as having emerged from a central cause setting off a simple chain of events is in
actuality a far more involved phenomena than most may take time to ponder while stuck
behind the wheel on the way to brunch.
Because systems such as a traffic jam, the stock market, the government and our
climate system, are prevalent in our daily lives, the ability to understand how they work
is requisite for learning about our world, as seen in national science education content
standards where the understanding of physical, living and designed systems such as
geochemical systems, cellular systems and ecosystems are emphasized (National
2
Research Council, 1996). The current focus on science, technology, engineering and
mathematics (STEM) education and government initiatives such as President Obama's
"Educate to Innovate" campaign (The White House, 2010) to increase STEM literacy and
inspire students to excel in science and math reiterate the importance of being able to
understand systems since they are prevalent in these domains. This research proposes
one way in which we can teach for improved systems understanding.
What is a System?
A system can simply be viewed as the interaction of many parts that work
together as a whole. Consider the greenhouse effect as an illustrative example of a
system. The parts of the system represent its independent elements or entities. These
elements are often physical structures (e.g., sun, Earth), but can also be abstract or
passive structures (e.g., gas, temperature). The interactions between the parts are
represented by relationships between them. The relationships exist in different forms
such as causal (e.g., If the sun is in the sky, then the Earth will get warm), spatial (e.g.,
The atmosphere surrounds the Earth) and temporal (e.g., The Earth revolves around the
sun once a year). A system is essentially established by the relationships that it
embodies.
To illustrate this characterization of a system, consider the question "Is it a Heap
or a System?" posed by Sweeney (2001, p. 22). Both a heap and a system are composed
of many parts. However, a heap does not change when you take away or add parts to it.
For example, if you have a bowl of mixed nuts and take away the cashews or add
3
almonds to it, ultimately, you still just have a bowl of mixed nuts (Sweeney, 2001).
Thus, in the case of a heap, the relationships to support the determination of a system do
not exist.
Relationships in Systems
A relationship is the way in which two or more elements/entities are connected or
associated together. Relationships provide context and meaning beyond an element's
individual properties and behavior. A system is defined when the relationships allow
interactions between a set of elements that contribute to a greater purpose. For example,
consider the sun independently. The sun is a very large star comprised of hot gases with
an average temperature of 5,780° K and the surface of the sun is composed primarily of
hydrogen and helium. Now consider the sun and its relationships between the planets,
moons and celestial bodies that travel around it. In this context, the solar system
emerges. Similarly, consider the sun and its relationship to the Earth, the atmosphere and
greenhouse gases. In this context, the greenhouse effect emerges.
From a systems standpoint, the importance of understanding these relationships
has already been recognized. For example, Wilensky & Resnick (1999) argue for
bringing students' focus to the concept of "levels" in reference to the
interactions/relationships between elements at a hierarchy of levels. Levels highlight the
hierarchical and emergence properties of a system by examining them from both
microscopic and macroscopic views as well as deemphasizing a learners' centralized
bias, i.e., a learner's tendency to think towards centralized solutions (Resnick, 1996). A
4
classic example of the centralized bias is the notion that a flock of birds fly in a "V"
formation with control by a "leader" bird at the front, when in fact, theories find that
birds fly in a self-organizing, decentralized manner (Heppner & Grenander, 1990).
As stated earlier, in addition to the hierarchic (or "levels") type of relationship, there are
many different ways in which elements can relate to each other: causally, spatially,
temporally, and functionally, to name a few.
Yet, it is likely that not all types of relationships equally contribute to a better
understanding of the system. For example, could propositional relationships offer
conceptual knowledge that is as vital as those relationships that engage a holistic
perspective? To explore this theory, two types of relationships are examined in this
dissertation, specifically, causal and functional relationships. Causal and functional
relationships were selected because both have proven valuable in improving systems
understanding. Comparing and contrasting the effects of these relationships can reveal
valuable information about their potential contributions to systems understanding.
5
Research Hypotheses
The goal of the present study was to understand the effect of relationship type in
reasoning about and understanding a system. To be able to compare and contrast these
two types of relationships, a set of formal definitions are provided followed by the
overarching research questions:
Causal:
Causal relationships exist between two objects, X and Y, where X is the factor
that precedes Y and uniquely produces the changes observed in Y. They establish a
model of actions and consequences, cause and effect, between objects that allow for
theory formation, explanation and prediction of events. They are further characterized by
linear, unidirectional relationships implying a temporal sequence/order (Grotzer &
Perkins, 2000) of an "if/then" type and a static or discrete mental interpretation.
Examples include: "If the power button is pressed, then the computer will turn on" and
"Greenhouse gases absorb infrared radiation."
Functional:
Functional relationships describe the nature of change between two objects, X and
Y, beyond the existing causal relationship. They help establish a model of the dynamic
interactions between objects that allow for understanding of system activity such as
spatial, temporal and state changes (Black, 2007). They are further characterized by
dynamically causal, fluid and often nonlinear, state changes between two agents with a
continuous mental interpretation. Examples include: "The more greenhouse gases are in
6
the atmosphere, the more heat will be trapped by the atmosphere" and "Igneous rock
builds up as more lava flows and cools."
Research Questions
To highlight the effects of causal and functional relationships, the focus of the
present study is limited to physical systems with strong causal and functional models, and
explores the questions:
(1) When relationships in a system of a particular type (causal or functional) are made
explicit, how do they affect understanding of physical systems?
(2) Does reasoning about a physical system improve more with functional
relationships than with causal relationships?
Hypotheses
Based on the review of the literature as presented in the next chapter, the
following hypotheses are made with respect to the research questions posed for this
study:
HI: Functional relationships engage different reasoning strategies than causal
relationships when trying to understand a physical system.
H2: Functional relationships improve the reasoning and understanding of complex
physical systems better than causal relationships alone.
7
Overview of Dissertation
This dissertation is organized into four chapters. Chapter II provides a review of
the relevant literature from the domains of complex systems, causal reasoning and
functional relationships to outline a theoretical framework for the present study. The
chapter begins with a definition of complex systems and identifies the issues in learning
about them, specifically, the development of mental models to support reasoning and
mental simulation. Focusing attention on system interactions is proposed as one method
to promote mental simulation for mental model development. This follows with a
discussion about causality and the significance of causal reasoning in approaching
systems understanding. Finally, functional relationships are presented as an approach to
address the limitations of causal thinking that have been identified through prior research.
Chapter III describes the research methodology for the present study. The
selection of participants, the design of the study, the materials and the measures are
explained. In addition, the methods by which data was coded and analyzed are described.
Chapter IV reports the results of the study in the context of the hypotheses set
forth in Chapter I.
Chapter V offers an interpretation and general discussion of the study's findings.
The limitations of the study and opportunities for future research are also described. The
chapter concludes with instructional and educational implications on the use of functional
relationships for improving reasoning about complex systems.
CHAPTER II
LITERATURE REVIEW
The purpose of this chapter is to review the literature of complex systems, causal
reasoning and functional relationships, the intersection of which provides the theoretical
framework for the present study. The chapter begins by defining complex systems and
presenting the challenges that they create for learners and educators in studying them.
Systems that have strong causal structures such as physical systems are more likely to be
understood if an appreciation of their causal models is attained. Accordingly, a review of
the causal reasoning domain is included. While a causal approach to decomposing a
system is natural for learners, the causal models that are held are often too simple and
linear to impact systems understanding. This leads to the consideration and review of
functional relationships, or dynamic causal relationships, to serve as an alternative
approach to thinking about causal models in a system.
Complex Systems
The Properties of a Complex System
Complex systems have been widely studied, but as an abstract domain, conclusive
definitions for them are still left up to interpretation. For a basis of discussion, it is
helpful to have a definition and nomenclature: What is meant when a system is
determined to be complex and how can it be described? Though complex systems differ
9
in nature and context, i.e., physical, biological, social, they can all roughly be defined as
comprised of a large number of parts having many interactions (Simon, 1996).
Table 1
Examples of Complex Systems (Bar-Yam, 2003, p. 8)
System
Element
Interaction
Formation
Activity
Proteins
Amino Acids
Bones
Protein folding
Enzymatic
activity
Nervous system
Neural networks
Neurons
Synapses
Learning
Behavior
Thought
Physiology
Cells
Chemical
messengers
Physical support
Developmental
biology
Movement
Physiological
functions
Life
Organisms
Reproduction
Competition
Predation
Communication
Evolution
Survival
Reproduction
Consumption
Excretion
Human
economies
and societies
Human Beings
Technology
Communication
Confrontation
Cooperation
Social evolution
Same as Life?
Exploration?
Further, the following properties are considered common characteristics across
complex systems (Bar-Yam, 2003; Simon, 1996; Jacobson & Wilensky, 2006):
(1) elements, the individual parts of a complex system;
(2) interactions, the relationships/effects of elements influencing other elements;
(3) hierarchy, being "composed of interrelated subsystems" (Simon, 1996);
(4) dynamism, systems are changing as elements interact, and mostly in a nonlinear
fashion; and
10
(5) emergence, "how local interactions of elements in a complex system at a
microlevel can contribute to higher order macrolevel patterns that may have
qualitatively different characteristics than the individual elements at the
microlevel" (Jacobson & Wilensky, 2006, p. 16).
Examples of systems described by these properties can be found in Table 1.
Issues in Understanding Complex Systems
Though "the technique of mastering complexity has been known since ancient
times: divide et impera (divide and rule)" (Dijkstra, 1979), it is an intuitive and wellstudied fact that learning about complex systems is difficult (Feltovich et al., 1992).
Towards this end, those in the field of complex systems and cognitive studies have been
investigating ways in which learners can improve their understanding of the complex
world. They have found that the key to understanding complexity is to decompose it into
smaller, more manageable chunks. However, the method by which this decomposition
occurs has an effect on the overall understanding of the complex system being studied
(Hmelo, Holton & Kolodner, 2000; Perkins, 1986; Wilensky & Resnick, 1999).
Traditional approaches for teaching complex systems have included the
introduction of its structural elements through a series of related definitions presented for
memorization (Hmelo, Holton & Kolodner, 2000, Perkins, 1986). This approach
succeeds in the teaching about a system's structures as facts, but often leaves knowledge
"disconnected from the contexts of application and justification that make it meaningful"
(Perkins, 1986, p. 4).
11
In an effort to engage students beyond mere fact learning, Kolodner et al.
(1998) developed the Learning By Design (LBD) project-based inquiry approach for
undertaking complex problems while learning science to actively engaging students in
design and modeling activities. For example, Hmelo, Holton and Kolodner (2000) asked
students in sixth grade life science classes to design and physically construct an artificial
lung as part of an LBD curriculum to teach the respiratory system. They found that the
affordances of the physical construction and revision process did indeed help students
reach a better understanding of how structures were related to each other during
respiration. However, simply building these models was not enough for them to gain a
deep mastery of the respiratory system. Hmelo et al. suggest that by also making the
conceptual framework and vocabulary of LBD explicit to students, they become equipped
with better tools for communicating about and understanding the systems.
Thus, to make the framework more explicit, the LBD approach incorporates the
Structure-behavior-function (SBF) vocabulary and framework that originates in the
artificial intelligence literature on Functional Representation (FR), a language for
describing the function, structure and causal processes of a physical device. (For a
review, see Chandrasekaran, 1993). SBF models extend FR by using well-defined
taxonomies that enable explicit representation of behavioral states, state transitions and
causal behaviors of a device (Goel et al., 1996). An SBF model for a system results in an
expression of its structure, the physical elements of a system (the "what"), behavior, the
dynamic mechanisms that allow the structures to achieve their purpose (the "how"), and
12
function, the role of a structure in a system (the "why") (Hmelo et al., 2000; HmeloSilver & Pfeffer, 2004).
Hmelo-Silver & Pfeffer (2004) used the SBF model as a framework for analysis
in a study comparing expert and novice understanding of a complex system. In this
study, participants' representations (verbal, pictoral) of an aquarium ecosystem were
analyzed using an SBF coding scheme. Results suggested that while novices focused
primarily on structures, experts had an integrated understanding of the system that
included behaviors and functions as well at significantly different levels. The
implications of this work suggest that getting to an advanced understanding of a system
should involve greater knowledge of behaviors and functions.
Mental Models of Complex Systems
To facilitate the understanding of behaviors and functions in a complex system, it
is prudent to consider the types of mental models with which learners reason about these
systems. Mental models are dynamic, psychological representations of real world
phenomena. A seminal book by Johnson-Laird (1983) proposes mental models as
internal representations that support deductive reasoning through syllogisms while
another formative book by Gentner & Stevens (1983) proposes that they support
reasoning about dynamic physical systems as a model of causally interacting entities.
There is consensus that mental models can be thought of as the basic structure of
cognition (Johnson-Laird, 1983) and the highest form of knowledge representation
(Black, 1992). "It is now plausible to suppose that mental models play a central and
13
unifying role in representing objects, states of affairs, sequences of events, the way the
world is, and the social and psychological actions of daily life" (Johnson-Laird, 1983, p.
397).
Mental models serve a significant role in understanding, especially with respect to
complex systems, because they allow one to make inferences about system behavior and
produce explanations or justifications for them (Williams, Hollan & Stevens, 1983).
However, the substance of a learner's mental model may not always reflect the
complexity of the system being learned. For example, in a study subsequent to HmeloSilver, Pfeffer (2004), Hmelo-Silver, Marathe & Liu (2007) used the SBF measure to
assess participants' understanding for two complex systems (an aquarium ecosystem and
the human respiratory system) in another novice-expert study. This time, they also added
a measure of mental models to learn about the relationship between the fine-grained SBF
representations and their holistic understanding of these systems. Consistent with their
previous research, the SBF measure showed that there were minimal differences between
novices and experts on knowledge of the structures of a system, but significant
differences in reporting causal behaviors and functions. The complementary mental
model analysis had converging results revealing that novices held simple, structurallybased models while experts focused significantly more on underlying functions and
behaviors in their models for both types of systems.
The benefit of mental models with functions and behaviors over those with just
structures is that they enable an active perspective of a system and the ability for mental
simulation. Functions support the inference process of "envisioning," constructing a
14
qualitative simulation in one's mind of how a physical system works (deKleer &
Brown, 1983). "Running" this constructed simulation allows reasoning about the
system's behaviors based on the interactions between elements and the causal models that
are assumed. Without this capacity for the simulation of mental models, the dynamism of
a complex system is likely missed and an expert-like understanding difficult to achieve.
Jacobson (2000) further elaborated on the mental model differences between
novices and experts for complex systems. He proposed the Complex Systems Mental
Models Framework (CSMM) by identifying a set of ontological and epistemological
beliefs (referred to as component beliefs) presumed to be related to complex systems
after a review of the literature (Table 2).
Table 2
Complex Systems Mental Models Framework (Jacobson, 2000)
Complex Systems Mental Models Framework
Categories of Component
Beliefs
Set of Component Beliefs Associated
with Clockwork Mental Model
Set of Component Beliefs Associated
with Complex Systems Mental Model
1. Understanding phenomena Reductive (e.g., step-wise sequences,
isolated parts)
2. Comrot
Centralized (within system)
External agent (external to system)
3. Causes
Single
Multiple
4. Action effects
Small actions -> small effects
Small actjon -> big effect
5. Agent actions
Completely predictable
6. Complex actions
From complex rules
Not completely predictable 1 stochastic
/ random
From simple rules
7. Final causes or
purposefiilness of natural
phenomena
8. Ontology
Teleological
Non-teleological or stochastic
Static structures
Events
Equilibration processes
Nan-reductive: whole-iS'greater-thanthe-parts
De-centralized {system interactions)
15
The component beliefs in eight categories were identified as belonging to one
of two types of mental models. The first exemplifies a clockwork mindset attributed to
novice systems thinking. The term "clockwork" comes from the metaphor used to
describe the regularity and predictability of Newtonian mechanics as compared to a wellmade clock (Peat, 2002). As follows, a clockwork mental model is characterized by
reductive tendencies and beliefs such as the centralized control of systems, that actions
and effects are linear, and that actions may be precisely determined (Jacobson, Allison,
Ropella, 2000).
The second type of mental model follows an ecological mindset that is attributed
to experts and complex systems scientists. The term "ecological" comes from the
biologically inspired perspective of complexity that accounts for its self-organization,
adaptation and emergent qualities. The complex systems mental model represents beliefs
such as order being achieved through decentralized interactions, that actions and effects
can be nonlinear, and that predictions about actions and effects are possible only in
probabilistic terms (Jacobson, Allison, Ropella, 2000).
In a study looking at novice (undergraduate students) and expert (complex
systems scientists) differences during problem solving, Jacobson (2000) empirically
explored the hypothesis that the component beliefs of the CSMM framework were
correlated with the two higher order mental models, the clockwork mental model and the
complex systems mental model. A verbal protocol was used to pose problem questions
dealing with complex systems and to probe for responses as participants thought aloud
their ideas while they solved the problems. This exploratory study did not have a large
16
enough sample to allow analysis on all component beliefs, but the subset of these
beliefs with the highest Item-Total Correlations (reductive, centralized, small-actionssmall effects and completely predictable for the Clockwork mental model; non-reductive,
de-centralized, multiple causes, not completely probably, equilibration processes for the
Complex Systems mental model) allowed for the identification of Clockwork and
Complex Systems Belief scales against which significant differences were found between
the two groups. While it was expected that experts would use complex systems concepts
because of their familiarity with them, the novices who were university level students
were also expected to have had exposure to concepts such as equilibration processes and
evolution through standard high school or college curricula, yet made little reference to
these concepts. Experts reported significantly more complex systems concepts than
novices in their problem solving. They also applied these concepts across a wide range
of problem domains (e.g., weather, traffic, design of a city, robotics) demonstrating the
ability for the difficult task of far transfer. This research suggests that promoting
complex systems literacy similar to that demonstrated by systems experts will provide a
useful approach to learning about scientific phenomena that can be applied in a crossdisciplinary manner.
Relationships in Complex Systems
It is proposed that one approach to establishing complex systems literacy and
fostering the development of complex systems mental models is to focus on the
interactions between system elements. System interactions are identified with the
17
component belief about system control (Table 2), but can be associated with most of
the other component beliefs as well, for example, part-to-whole relationships for
understanding system phenomena, and cause and effect interactions when considering
beliefs about system causes.
The different types of relationships and interactions in a system include:
•
spatial, relationships of how elements are related in space;
•
temporal, relationships describing how elements change over time;
•
hierarchical, relationships describing elements at micro- and macro-levels;
•
causal, relationships describing cause and effect behavior between elements; and
• functional, relationships describing the dynamic changes that elements effect on
each other.
Causal and functional relationships have been selected for comparison as the
theoretical framework for the present study based on their core relevance to complex
systems thinking as reviewed in the remainder of this chapter.
Causal Reasoning
From the time we are young children, we are faced with the consequences of our
actions: if we touch something hot, then we will burn our fingers; if we pile blocks too
high, then they will topple over. Causal reasoning is the reasoning about such
consequences, the relationship between cause and effect. More formally, a causal
relationship between two elements, X and Y, exists when any intervening changes to X
18
also change Y. For example, the burning of fossil fuels (X) causes the concentration of
greenhouse gases (Y) in the atmosphere to change. Another way to look at causal
relationships is in terms of necessary (must be present for effect to occur) and sufficient
(if present, guarantees that effect occurs) conditions: "The effect is attributed to that
condition which is present when the effect is present and which is absent when the effect
is absent" (Kelley, 1967, p. 194).
Causality has long been a formal topic of study with dialogue dating back to
Aristotle. A significant juncture in the study of causality came when the philosopher and
historian David Hume posed the problem of causal induction (Hume, 1739/1987). This
problem centers on the inherent assumption of regularity, i.e., that the future will be like
the past. A classic example to demonstrate this problem is asking one how it can be
known that the sun will rise tomorrow. In such an example, the direct causes cannot be
sensed or deduced, rather, they are inferred from events that are perceived as spatially
and temporally contiguous, where the causes precede the effect and the regularity of these
events in nature is assumed. As follows, one would induce that if the sun rose yesterday
and today, then it should also rise tomorrow. There has been much challenge to Hume's
argument, and while a philosophical discussion of causality is not the intention here, the
established significance of this type of thinking in our history and in everyday life is
emphasized.
19
The Development of Causal Reasoning
While there is some debate as to whether the origins of causal perception and
reasoning are innate (Leslie, 1984) or develop over time, there is no doubt that thinking
in causal terms begins early. Though ambiguous results about perceptions of causality
were found in infants at 6-1/2-months (Leslie, 1984), Cohen & Oake (1993) conducted a
series of four experiments to show that infants as young as 10-months-old do have
notions of causality. These studies presented the infants with videotaped and live stimuli
involving the movement of attractive, multicolored toy vehicles to create causal events.
In the causal condition, contact by the first vehicle causes the movement of a second
vehicle. In the noncausal condition, the first vehicle does not make contact with the
second vehicle, but the second vehicle still moves. Using habituation as a measure,
Cohen and Oake's found that infants were able to perceive the differences in the causal
and noncausal events, and that the objects (i.e., cars) acting as agents (i.e., the first
vehicle) were more central to their understanding of causality than the objects acting as
recipients (i.e., the second vehicle). More importantly, the combined results of these four
experiments provide evidence that 10-month-old infants are able to perceive simple
causal events involving an agent, an action (i.e., collision) and a recipient beyond just the
physical changes in an event (i.e., that the cars moved).
Decades of research have shown that young preschool-aged children demonstrate
knowledge of causal relationships in their nai've/folk physics (Bullock, Gelman, and
Baillargeon, 1982; Shultz, 1982), biology (Gelman & Wellman, 1991; Inagaki & Hatano,
1993), and psychology (Wellman, Hickling & Schult, 1997). For example, in a study of
20
physical causality by Bullock, Gelman, and Baillargeon (1982), 3- and 4-year olds
watched a "domino" sequence of events: a rod pushing through a post knocking over five
wooden blocks making a stuffed rabbit fall into a bed. After observing the sequence of
events, the children were asked to predict whether modifications to the sequence such as
a change in the length or color of the initial rod or the removal of wooden blocks would
cause the final event to still occur. Bullock et al. found that the majority of the children
were able to predict correctly the outcomes of the modifications (e.g., the change in
length of the initial rod may affect the final event while the color of the initial rod will
not) demonstrating the understanding of more complex causal events by this age group.
Results such as these and the overall empirical evidence in the literature suggest that by
age 5, children are able to appropriately apply different causalities to the core domains of
physics, biology and psychology (Inagaki & Hatano, 2010). Causal reasoning skills
continue to develop to a level where adults are able to induce accurate causal conclusions
from information in the observable and non-observable events they encounter (Cheng,
1997).
Explaining and Representing Causal Reasoning
Covariation and Causal Powers
The explanation of causal induction has been a topic of research for many years
with the two basic approaches being covariation and causal powers. Covariation is the
extent to which a candidate cause and the effect vary together. In a covariation approach
to modeling causality, a conditional probability is determined between a cause and effect,
21
i.e., the difference between the probability of the effect in the presence of the cause and
the probability of the effect in the absence of the cause. A positive difference indicates a
generative cause while a negative difference indicates a preventative one.
The problem with this approach is that "covariation does not imply causation" and
the way in which people determine a cause cannot be sufficiently described by this
approach. Cheng (1997) presents the examples "sunrise might occur every day after a
rooster on a farm crows" and "one class for a student might routinely follow another" as
cases where, despite evidence of covariation, few would actually infer causal relations
between sunrise and a rooster or the meeting of one class and the meeting of another.
So, how do people make such judgments about causality? The causal powers
approach is one answer to this question. Causal power "is the intuitive notion that one
thing causes another by virtue of the power or energy that it exerts over the other"
(Cheng, 1997, p. 368). Causal powers are assumed to generate or produce their effects.
Accordingly, this approach suggests that one's general belief in causal powers is how an
assessment that a sunrise occurring after a rooster's crow is not in fact a causal relation
because a rooster's crow does not have the causal power to produce a sunrise.
One drawback of the causal powers approach is that it does not explain the
process by which the assessments of power are determined. In 1997, Cheng presented
her power PC theory (an abbreviation for "causal power theory of the probabilistic
contrast model") as an integration of the covariation and causal power approaches to
causal induction. Covariation accounts based on observable events, e.g., "the probability
of lung cancer given yellowed fingers and without yellowed fingers" (in the days of
22
unfiltered cigarettes) are interpreted by unobservable causal powers, e.g., "smoking has
the power to produce lung cancer" (Glymour & Cheng, 1998). The information that is
used to assess covariation from a set of events is used to estimate the magnitude of the
causal power as the probability with which a cause yields an effect (under a set of
assumptions).
Novick & Cheng (2004) extended this theory to account for the more common
situation where a combination of causes, i.e., not just a single cause, produce an effect
using binary variables to represent them. The power PC theory is considered a precursor
to the use of causal Bayes nets (Pearl, 1988) as an implicit model for representing causal
structures and reasoning in both adults and children (Gopnik & Glymour, 2002; Gopnik
et al, 2004; Tenanbaum, Griffiths & Kemp, 2006).
Figure 1
Causal Bayes Net Example (Gopnik & Schulz, 2004)
z
-~
X
—*- w
IHhtoOS in Cagnvte Saenoss
Figure 1. A causal Bayes net. R, S. W„ X. Y, Z represent variables and the arrows
represent causal relations between those variables.
23
Causal Bayes Nets
Causal Bayes nets originate in the statistics, computer science and philosophy
literatures as computational models of events, and the pattern of conditional probabilities
among the events from which inductive inferences can be made. They are directed
graphs where nodes of the graph represent properties of the system with variable values,
and directed edges represent causal relationships (Gopnik & Nazzi, 2003) (Figure 1).
They are almost always acyclic, i.e., they do not have a sequence of directed edges that
lead back to a variable as in a cycle (Glymour & Danks, 2008).
Figure 2
Simple Causal Bayes Net Example (Glymour & Danks, 2008)
Timer i n i T a n i
"A....
SUUCIHPIVo:;i
r
:*__.
- '
P<met <ort"i»tt;
*L
l..imp n>l-'o!ii
"The state of Lamp is determined uniquely by its two inputs: Lamp is on if Power
is on and Switch is on. If the value of Power is ignored or unknown and varies
from case to case, then the state of Lamp will appear to be an indeterministic
function of Switch. The causal content is captured by the supposition that a direct
intervention that changes the state of a variable changes the state of variables
downstream from it, but leaves the state of other variables unchanged. So, for
example, an intervention that breaks the bulb of the Lamp leaves the state of the
Lamp fixed at off, regardless of any variation in Power, Switch or Timer, and
leaves the probabilities of Power, Switch and Timer unaltered, as if the edges
24
from Switch and Power to Lamp were broken by the intervention, though the
arrow from Timer to Switch is left unaltered." (Glymour & Danks, 2008, p. 465)
Causal Bayes nets have been adopted as an informative representation of how
people use their prior knowledge to process events they observe in the world to make
inferences about causal relationships. The network in Figure 2 described by Glymour &
Danks (2008) provides a simple example. Using this causal Bayes net model, the
expectations from these events and interventions allow one to develop beliefs about the
possible causes for observed events such as the Lamp powering on by itself at 7 pm or
why turning the Switch on does not turn the Lamp on.
The ability to represent causal learning with causal Bayes net allows researchers
to computationally infer how this learning occurs. Research has shown that even young
children are able to make normative judgments about causality consistent with these
representations (Gopnik & Schulz, 2004). Children can detect patterns of conditional
probabilities and understand how interventions can be used to infer causal structures.
Adults appear to make causal predictions according to representations like causal Bayes
nets as well. However, because they have an extensive prior causal knowledge on which
they rely heavily when making judgments, there is little motivation to revise earlier
causal knowledge and construct new knowledge (Gopnik & Glymour, 2002). This prior
knowledge also makes it difficult to study how adults learn new causal relations in most
domains. Science, a domain of constructing new causal knowledge through observation
and inquiry, is an exception, and consequently, the domain chosen for the present study.
25
Causal Relationships for Systems Understanding
In systems with a strong casual structure such as physical, electrical and
mechanical systems like the rock cycle, the water cycle and a car's braking system,
understanding the causal relations within the system is undeniably central to
understanding the overall system. Grotzer and Sudbury (2000) offer one example of how
the progression of causal understanding might unfold in the study of electrical circuits.
Upon a review of the research on students' ideas of how simple electrical circuits work,
they proposed a sequence of conceptual models moving from simple to complex causal
reasoning patterns to represent the development of causal understanding about these
systems (Figure 3). They then investigated the effect of focusing fourth grade students
on these causal patterns while teaching a unit on electrical circuits by comparing the
performance of three groups: a causal models group that had activities which infused
attention to causal patterns paired with explicit discussion of causality, an activities group
that had activities which infused attention to causal patterns, but without explicit
discussion of causality, and a control group which had neither the activities nor
discussions on causality.
Grotzer and Sudbury found that as hypothesized, most students held (simple or
double) linear causal models of electrical circuits prior to the intervention, and upon
conclusion of the unit, the students in the causal models group held models of
significantly higher levels of complex causality than the students in the activities or
control group.
26
Figure 3
Progression of Conceptual Models for Simple Electrical Circuits (Grotzer & Sudbury,
2000)
Conceptual Model
Simple Linear Causal Models
«tB^li^. » » H ' j
^.^
'^fc*ifl*i 141 ^fi
Causal Characteristics
A single wire running from the battery to the
bulb "gives" electricity to the bulb in a
consumer source relationship. Note that current
is not conserved, there is nothing to account for
the "flow."
Double Linear Causal Models
Something goes from the battery to the bulb in
a linear, unidirectional pattern.
That agent or substance does something to or
within the bulb to make it light (feeds it,
attracts, clashes, cancels out.)
Cyclic Sequential Causal Models
A substance-like matter leaves the battery and
causes the bulb to light in a linear sequential
pattern. It goes around and into the battery and
is recycled in a cyclic pattern.
There is a beginning and an ending of sorts at
the battery. Once the electrons get back to the
battery, they begin again. Viewed ahistorically,
once the circuit is flowing, it resembles the
cyclic simultaneous model.
There is no real beginning or ending, at least
not once it gets started going.
Something can be a cause and an effect.
Cause does not precede effect temporally.
Cause is distributed around the circle.
Cyclic Simultaneous Causal Models
Relational or Interactive Causal Models
The outcome is caused by the differential
relationship between elements of the system.
Neither "status" (more or less concentration of
charge) is the cause by itself.
27
The causal models group was also able to reason with these more complex models
about circuitry beyond what they had been taught as demonstrated by significant
differences in a posttest inventory. These findings suggest that while students do
maintain causal models of systems such as electrical circuits, they are often simple and
guided by basic misconceptions. The explicit teaching about the nature of causality is an
important component in helping students progress past their naive models and take the
conceptual causal leaps that might otherwise not become apparent to them.
Grotzer & Baska (2003) found similar results in a study with third graders studying
ecosystem concepts. Without instruction on the causal structural knowledge of
ecosystems, students were more likely to distort new information to fit the simple linear
causal models they initially held.
Limited Perceptions of Causality
Even as the developmental literature presents causal reasoning as an instinctive
skill displayed since infancy, it is evident that learners' everyday approach to causality
even as adults does not account for the complexity found in systems. This is likely due to
a learner's limited perceptions of causality itself (Chi, 2000; Grotzer, 2003; Wilensky &
Resnick, 1999). As presented earlier, Bullock, German, and Baillargeon (1982) found
that young children expect causality to occur in a chained or "domino" sequence of
events. A chain of events necessitates a temporal order where causes precede or occur
with effects and that the relationship between events is linear. Perkins & Grotzer (2005)
found that students of all ages assumed simple and linear, "serial" causal models that
28
proved insufficient to support the understanding of complex science concepts. Recall
that linear models are also prevalent in causal Bayes net representations. Other limiting
perceptions of causality include the expectation of obvious causes and obvious effects
(Grotzer & Bell, 1999) resulting in the failure to notice nonobvious or passive agents in a
system and the attribution of causes to a centralized source (Wilensky & Resnick, 1999)
that lead to misconceptions about how systems emerge.
Jacobson (2001) explored the effect of these limited perceptions of causality on
systems understanding in a comparison of novices (undergraduate students) and complex
systems experts. In this comparison, participants were asked a series of questions about
complex systems such as "How would you design a large city to provide food, housing,
goods, services, and so on to your citizens so that there would be minimal shortages and
surpluses?" Jacobson found that novices maintained simple causal models in their city
designs suggesting isolated parts and centralized control, for example, "We're not going
to have any cars for one thing. Have everything in one central location. No driving
around, no big thick pollution..." (Jacobson, 2001, p. 44) while experts had complex
causal models demonstrating decentralized control and not completely predictable
actions, for example, "[have] very detailed demographic information about who lives
where, how many cars they drive, where they need to go, when they need to be there,
etc., and then [simulate] their driving patterns car by car on exact models of city streets
and freeways" (Jacobson, 2001, p. 45). The disparity between the causal models of
novices and experts in these specific examples stemmed from the attribution of causes to
a centralized vs. decentralized source and confirm how the limited perceptions of
29
causality can present a barrier to advanced systems understanding for novices.
Though research on causality in young children and representations using causal
Bayes net has shown that causal thinking is both natural and normative, previous research
also emphasizes that ordinary views of causality are simple and limited by
misconceptions. It follows then that addressing these limitations from an educational
standpoint is important, especially for the goal of improving understanding for systems.
Addressing Limitations of Causal Reasoning
Similar to the causal model progression offered by Grotzer and Sudbury (2000) in
the domain of electrical circuits, Halbwachs (1971)1 identified three types of explanation
representing a progression of understanding in the domain of physics. The three types
are (quoted from Besson, 2004, pp. 114-115):
(1) causal, or heterogeneous, where the change in the system is due to
agents outside the system, with delayed actions expressing real
connections between things;
(2) formal, or homogeneous, consisting of simultaneous functional
relations between quantities describing the system (for example,
the ideal gas equation or energy conservation in a system);
(3) 'bathygeneous' explanations, made on a deeper level or an
underlying structure, considering a smaller scale or a more general,
deeper theory.
1
This research is reported from Besson (2004) due to the inability to obtain an English translation of the
original article that was presented in French.
30
Causal explanations are further divided into three steps: simple, linear and
circular explanations of causality representing another progression, in this case, towards a
leap to homogenous explanations involving functional relations. The first step, simple
explanations, provides a basic interpretation of phenomena by constructing causal
relations between a particular cause and subsequent effect. Linear explanations, the
second step, develop from a causal chain of simple explanations providing deeper, but
unilateral representations of phenomena. Finally, circular explanations introduce the
concept of reversibility through a chain of reversible causal relations, enabling the true
progress in the understanding of physical processes. Halbwachs suggests that causal or
heterogeneous explanations "are the most satisfying and appealing to the human mind"
(Barbas & Psillos, 1997, p. 455), making the move to formal or homogeneous
explanations less enticing for most learners.
Barbas & Psillos (1997) investigated bridging the gap between causal and formal
explanations by using Halbwach's steps of causal explanations as a theoretical
framework to design a one-semester teaching sequence in electrostatics for student
teachers enrolled in a university course. The teaching sequence was comprised of three
parts, each part reflecting a level of modeling of the transient states, as opposed to steady
states, in the field of electricity. The intention of these models was to facilitate the
transition of students' causal reasoning patterns as they accounted for the changes that
occur during the transient states.
Students worked first from simple explanations, but moved to linear or circular
explanations when they found the simple ones no longer satisfactory after prompting
31
from teacher questions. As a result, their reasoning patterns also evolved. As elements
of their reasoning patterns contradicted each other or their explanations contradicted the
facts they encountered throughout the teaching sequence, they changed entities (e.g.,
batteries and electrons) or rules (e.g., batteries cause electron flow) to fit their current
form of reasoning, i.e., simple causal links. When they reached a point where they could
not resolve contradictions in light of new knowledge, e.g., becoming aware of
phenomena like Couloumb's law, they devised new reasoning patterns, i.e., iterative
causal chains.
Barbas & Psillos (2003) followed up with a study that used more explicit
modeling of simple, linear and circular explanations by developing a teaching sequence
using a series of short interactive computer simulations to model the interactions of atoms
for primary school students. They also identified an evolution of cognitive states that
characterized students' understanding of electrostatics and found that students' mental
models and reasoning patterns often constrain their views of material or physical
situations, such as the charging of atoms, affecting transitions to advanced cognitive
states, similar to the findings in Barbas & Psillos (1997).
Further, though students transitions between cognitive states were facilitated by
the interactive computer simulations, Barbas & Psillos detected a strong "causalityobservation" interrelation where knowing about the causality of an interaction allowed
the simulated events to be more readily observed, while the reverse, observing simulated
events with poor mental representations did not make finding the underlying causality
easier to see. Based on this research, Barbas & Psillos recommend that "instructional
32
design should take into account the transformation process of learners' mental
representations of material situations and use tools for influencing this process on the
levels of entities, rules of interaction and reasoning patterns" (Barbas & Psillos, 2003, p.
253).
In sum, the causal reasoning literature as presented suggests that though thinking
in causal terms is evident from a young age and is a normative way of viewing the world
(as evidenced by causal Bayes net representations), an explicit educational attention to
causality is necessary, especially when learning about science or complex systems,
because the predominant views held by learners are laden with perceptions that limit the
development of causal models. Learners need to be challenged to move from their more
appealing simple and linear causal reasoning patterns (Barbas & Psillos, 1997) so they
will not conform new knowledge to fit these typically insufficient models.
Prior approaches for motivating learners to advance their causal reasoning have
focused on the progression of explanations at the heterogeneous level of causal
understanding (Barbas & Psillos, 1997; Barbas & Psillos, 2003; Grotzer & Sudbury,
2000, Grotzer & Perkins, 2000). It is still unclear how well this progression helps to
bridge the gap to the homogeneous or formal level characterized by functional relations.
The present study proposes an alternative approach, specifically, to make the leap straight
to the homogeneous level to determine if the gap can be bridged in this manner.
33
Functional Relationships
What are functional relationships? Functional relationships represent a dynamic
causal interaction between elements indicating movement, changes in state and effects on
each other (Hachey, Tsuei & Black, 2001; Tsuei, 2004). This type of relationship is best
defined when the dynamic interaction is viewed as a function, analogous to a
mathematical function as alluded to by Halbwachs (1971) in the context of physics, e.g.,
ideal gas equation, to describe the relationship between two elements, X and Y. For
example, consider the function "increases" in the relationship "the burning of fossil fuels
(X) increases the concentration of greenhouse gases (Y) in the atmosphere." By focusing
on a function between elements, the relationship takes on an active perspective and
evokes more fluid ways in which elements in a system relate with each other.
While the term "functional relationships" is not common in the literature of causal
reasoning, its concept is frequently discussed. For example, Grotzer and Sudbury (2000)
describe a "differential relationship between elements in a system" (see Figure 3) to
explain a causal characteristic of relational or interactive causal models, the highest level
in their progression of conceptual models of simple electrical circuits. Similarly, the
notions of feedback, cyclical and two-way relationships used in describing subcategories
of causality convey qualities of functional relationships.
Functional relationships are best exemplified in a dynamic and interactive
medium such as the systems thinking software tool, STELLA (isee systems, Inc., 2010),
where changes in the system caused by the functional relationships can be simulated.
STELLA (Structural Thinking Experimental Learning Laboratory with Animation)
34
provides an icon-based graphical interface to build models that can be simulated over
hypothetical periods of time (for an example, see Figure 4).
Figure 4
An Example of a STELLA Model Representing the Depletion of the Ozone Layer
Omi&Mateou'cs
*—TK—m
O Cs in Uy«ef Almcci
6>
cfc productKir \
CFCs n Jppsr Alias
oi
cfcs la bppe- a-.itas
**DtiJcfe bf«-iteiow*i
laaciivcivit
J
ysar
The icons represent the stocks, the level of accumulation for things in the simulation
(rectangles), the rates at which the level of the stocks increase or decrease (circles with
spigots), variables, functions or constants that affect the flows such as external inputs or
algebraic equations (circles with curved arrows) and connectors that both link and
provide direction for the effects in the system (arrows). Students fit combinations of
icons together to create diagrams representing their view of the causal relationships in the
35
system. STELLA can automatically generate equations for the user-designed models
and the output of these equations can be readily examined through graphing features
available within the tool. Functional relationships arise after a diagram of causal
relationships is created in STELLA and the relationships are quantified while running
simulations of these models. In other words, the dynamic incarnation of the causal
relationships are the functional relationships.
An early review of the literature on STELLA (Doerr, 1996) found that there is
some evidence that STELLA models improve the understanding and reasoning about
systems, especially in physical science (Mandinach, 1989). The software helps extract
students' conceptions about complex topics, allowing them to become "active builders of
their own intellectual structures" (Webb & Haskell, 1988, p. 271) as they convert mental
models into external ones. Unfortunately, despite the potential benefits of this systems
thinking tool, creating STELLA diagrams requires extensive training (Mandinach &
Cline, 1994) and may seem complicated or difficult to understand if students have limited
mathematical knowledge. Setting aside STELLA'S limitations, the present study seeks to
investigate the affordances of functional relationships as conceptualized in STELLA, but
from the perspective of supporting the mental simulation of systems rather than a
physical simulation as in this computer-based tool.
Functional Relationships and Mental Simulation
One approach that has been proposed to support mental model development and
simulation through functional relationships is the use of simulation diagrams (Tsuei,
36
2004) to teach about systems. Simulation diagrams are static, graphical
representations comprised of nodes and labeled links explicitly depicting the types of
relations among them. Figure 5 is an example of a simulation diagram for the rock cycle.
Figure 5
An Example of a Simulation Diagram for the Rock Cycle (Tsuei, 2004)
Propositional relationships such as the "mantle is beneath the Earth's crust" and "mantle
has magma" are represented by solid lines. Contrast these to functional relationships such
as "magma/2Ws out of the Earth's crust" and "lava cools (into) igneous rock" that are
represented by the dotted and curved links. "Flows out" and "cools" describe functions
that indicate movement and state changes. Functional relationships are further defined in
37
the diagrams by a positive sign "+" indicating an increase or a negative sign "-"
indicating a decrease.
Tsuei (2004) trained middle-school students on the use of simulation diagrams
and compared their learning of how hurricanes are created to a control group trained in
writing summaries about the same system. The students who created simulation
diagrams significantly improved in their thinking about mechanisms and systems over the
control group. While traditional concept maps also helped students improve their
systems thinking in a related study, students benefited more from simulation diagrams
because of their explicit attention to functional relationships.
An advantage of simulation diagrams is that because they are static, they are less
complicated and easier to create and understand than STELLA models. Yet, they still
force students to define the necessary information about system entities and the changes
in relations between entities to support the running of mental models that enable students
to better reason and mentally simulate system behaviors. However, when addressing
more complex systems, a disadvantage of simulation diagrams is that their static nature
makes it difficult to present realistic depictions of change between entities, for example,
the degree of change beyond a general notation of increase or decrease.
The limitation of simulation diagrams was addressed in a study of Newtonian
physics by Chan & Black (2006) who used a special form of animations called directmanipulation animations with middle-school students. Direct-manipulation animations
are animations that incorporate the use of the haptic channel through hand controls like
sliders. These hand controls allow users to interact with the parameters of the system
38
represented in the animation, in this case, a roller coaster to demonstrate energy
transfer between kinetic and potential energy. Chan and Black found that as systems
become more complex, the addition of direct-manipulation animations that focused
learners on functional relationships as a complement to text-based instructional materials
helped students perform significantly better on comprehension tasks than those students
who received only text and static visuals or text alone. Direct-manipulation animations
helped students actively experience the effects in the system through functional
relationships, for example, the change in potential and kinetic energy that transpires
during a loop in the roller coaster, so that students have better support in reasoning and
mentally simulating them.
Overview of Study
"Systems are dynamic when their components are related to changes in other
system components, that is, components of a system affect other components which, in
turn, affect the original or other components." (Jonassen & Ionas, 2008, p. 296). The
research with simulation diagrams and direct-manipulation animations suggests that
functional relationships embody the changes and effects between system components that
constitute their dynamism. The present study follows this line of research to further
investigate the effects of functional relationships when learning about complex systems.
The focus, however, will be to compare the impact of functional relationships on the
reasoning and understanding for systems with their counterpart causal relationships,
especially the differences in their ability to support mental models and mental simulation.
39
Distinguishing between Causal and Functional Relationships
On the surface, it can be conceptually difficult to distinguish between causal and
functional relationships because both relationships share a causal component. One
arguable reason for this is that from a standpoint of interpretation, these two types of
relationships can actually be viewed on a continuum (Figure 6) where at the root of a
functional relationship is a simple and static causal one. As the interpretation of this
causal relationship transforms to a more complex and dynamic mental model of the
relationship, the functional relationship is revealed.
Figure 6
Causal to Functional Relationship Continuum
Causal
Functional
W^'
(Simple,
Static)
(Complex, Dynamic)
Consider the following two examples:
•
Causal Relationship: "The burning of fossil fuels (X) causes the concentration
of greenhouse gases (Y) in the atmosphere to change."
•
Functional Relationship: "The burning of fossil fuels (X) increases the
concentration of greenhouse gases (Y) in the atmosphere."
40
Though the same causal information is conveyed in the two relationships (cause =
burning fossil fuels, effect = change in concentration of greenhouse gases), the functional
relationship offers an opportunity for a more dynamic interpretation and mental
visualization (the burning of fossil fuels producing a higher concentration of greenhouse
gases) over the binary-type thinking, i.e., presence or absence of a cause, that simple
causal relationships might encourage (no burning of fossil fuels, no change in
concentration of greenhouse gases).
This type of mental reasoning is analogous to the shift between the discreteness of
depictive models to the flow of rule-based reasoning demonstrated in the Schwartz &
Black (1996) and Hachey (2005) studies on participants' reasoning with sets of gears.
Schwartz & Black (1996) conducted a series of experiments involving problems using
open and/or closed-chain configurations of a variable number of gears. Participants in
these experiments were asked questions like "[Five] gears are arranged in a horizontal
line, if you try to turn the gear on the far left clock-wise, what will the gear on the far
right do?" (Schwartz & Black, 1996, p. 466). During the course of these experiments, the
cognitive processing engaged to answer these questions evolved from relying on mental
(depictive) models when encountering a novel problem to determining local causal rules
as patterns of gear behavior were recognized and finally the surfacing of global parity
rules when generalizations about behavior could be deduced. A key finding from these
studies was that participants shuttled between the depictive and rule-based processing as
problems either extended or contradicted their models or rules. For example, moving
from open, linear problems to closed-chain, circular problems challenges local causal
41
rules prompting a shift back to mental model processing to reason about the novel
situation.
The proposal in the present study is that like the shuttling between depictive and
number-based rules for reasoning about gear problems, as one's understanding of a
relationship moves from the causal spectrum to the functional, there is a shift in
reasoning strategies, from simple and static rules to more complex and dynamic mental
simulations with a shuttling between the static and the dynamic processing as the
understanding along the causal spectrum develops. By separating causal and functional
relationships as distinct constructs at the extremes of the continuum for the purposes of
investigation, better insight into the influence on the types of reasoning these
relationships engage can be obtained.
42
CHAPTER III
METHOD
Design
This study was conducted as an experimental design with three groups:
functional, causal, and control. Each group differed only in the instruction about the
construct "relationships in a system". Participants in the, functional group learned about
functional relationships while participants in the causal group learned about causal
relationships. A third group served as a control group and did not receive any instruction
on relationships. Further, all participants studied one of two systems, the greenhouse
effect (a climate system) or the space elevator (a transportation system).
Participants
The participants in this study were graduate students from Teachers College,
Columbia University in New York, New York. Of the 101 participants who completed
the study, 49 were recruited from flyers placed on bulletin boards around campus or from
announcements made in classes by instructors. These students were given a Starbucks
gift card (worth either $10 or $20) as compensation. The remaining 52 participants were
recruited from the Fall 2008 and Spring 2009 Cognition & Learning classes
(HUDK4029). Students enrolled in this course are required to complete 3 hours of study
participation during the semester in which they are enrolled and were given 1 hour of
43
study credit as compensation for completing this study. Three students had to be
dropped from the study due to audio and instrument-related issues for a final N=98.
Participants ranged in age from 21 to 58 with an average age of 28.6 years. There
were 30 males and 68 females with an ethnicity distribution of 43.9% White, 23.5%
Asian, 8.2% Hispanic or Latino, 6.1% Mixed, 5.1% Asian American, 4.1% Black or
African American and 9.2% Other.
Figure 7
Distribution of Participant Majors
Major
N/A
Anthropology
App Physiology &
Nutrition
Applied Linguistics
A r t Ed
CCTE
Chemistry
Child Devpt
Clinical Psych
Cognitive Studies
Computer Science
Counseling Psych
Curriculum & Teaching
Devpt Psych
Early Childhood/Special
Ed
Exercise
Higher Ed
Intl Ed Devpt
Jewish Ed
Leadership
Math Ed
Measurement
Negotiation/Conflict Res
Org Psych
Philosophy
Politics & Ed
Psychology Ed
Social Studies
Teaching of English
TESOL
44
The academic majors of participants were recorded and the distribution by
department is presented in Figure 7. The largest groups of students were comprised of 14
students from a Program in Cognitive Studies, 12 students from a Program in
Developmental Psychology and 12 students from a Program in Communication,
Computing and Technology in Education. The remaining 60 students were diversely
distributed across academic majors at Teachers College. All participants were enrolled in
education-based programs, most in their first year, and were not expected to have
particular experience with the physical systems selected for the present study.
Participants were randomly assigned to a group and a system resulting in the
blocks found in Table 3.
Table 3
Participant Group Sizes by Treatment and System Assignment
FUNCTIONAL
CONTROL
CAUSAL
Greenhouse Effect
17
17
15
Space Elevator
17
17
15
Procedure
The study was conducted in a laboratory setting (either an empty office or a study
room) where participants were tested individually by a researcher. Each study session
followed the same procedure:
45
(1) Consent was obtained for their study participation. Participants were randomly
assigned to one of the three instruction groups (functional, causal or control) and
one of two systems (greenhouse effect or space elevator).
(2) Participants were asked about their prior knowledge of the system to which they
were assigned in a pretest (Appendix A).
(3) Participants were provided written instruction about a type of relationship as
determined by their group assignment followed by brief training with the
researcher (Appendix B). The non-treatment control group wrote summaries on
the readings about the systems. Then, all participants received the article on the
system to which they were assigned and asked to complete two intervention
worksheets (Appendix C).
(4) Participants were administered the 17-item Sigel Conceptual Style Sorting Task
(SCST), Form F (Sigel, 1970), as a distractor task. This task took an average of
five minutes to complete.
(5) Participants were given a posttest on their assigned system (Appendix A) to
determine if there was a change in their understanding of the system. The posttest
was followed by a structured interview to elicit the participant's understanding of
the relationship type to which they were assigned (functional or causal) and their
knowledge of the system to which they were assigned (greenhouse effect or space
elevator) (Appendix D).
46
(6) Participants were administered a transfer task that asked them to design a large
city on Mars. They were given a 10-minute time limit to complete this activity2.
(7) Participants completed a brief demographic questionnaire (Appendix E).
Participants took an average of 75 minutes to complete the study.
Nomenclature
The operationalization of systems understanding for the present study is based on
performance on SBF measures adapted from Hmelo-Silver, Pfeffer (2004) and HmeloSilver, Marathe & Liu (2007). The terms "structures", "behaviors" and "functions" have
been assigned a new nomenclature to avoid confusion with the use of the term "function"
as used in the context of functional relationships and to better reflect the constructs they
represent. Specifically, the term "structure" is referred to as "element," the term
"behavior" is referred to as "mechanism" and the term "function" is referred to as "role."
The following definitions describe these new terms:
•
Elements describe the parts of the system (what is in the system?);
•
Roles describe the role or purpose of an element in the system (why is it in the
system?); and
•
Mechanisms describe the dynamic mechanisms that allow elements to achieve
their roles (how does it work in the system?).
2
The transfer task was the only part of the study that was timed. It was determined during pilot testing that
a time-limit set a better expectation for the quality of participant responses.
47
Materials
The Systems
The first system that was used in this study is the greenhouse effect. The
greenhouse effect was chosen because of its potential familiarity to participants thus
allowing the opportunity to direct attention to relationships when reading about them.
Further, the specific description of the system as presented by MSN Encarta (MSN,
2008) was chosen because of its level of factual detail, for example, the percentages of
sunlight absorbed and emitted through the atmosphere, expected to be unfamiliar to
participants so new knowledge could also be obtained during the intervention. (Appendix
F).
The second system used in this study was the space elevator. This system was
chosen because of its novelty. Space elevators are currently scientific visions proposed
as a cheaper alternative to space shuttles and other existing spacecraft for carrying cargo
and humans into space. The specific description of the space elevator system as adapted
from Black (2008) was selected because of its appropriate level of factual detail and the
intentional incorporation of functional relationships into its content (Appendix G). In
addition, this transportation system offers a contrast to a physical (climate) system and
could provide further information on the utility of causal and functional relationships
across systems.
The readings for these two systems are equivalent based on word count and the
number of elements, roles and mechanisms included in the readings (Table 4).
48
Table 4
Systems Article Comparison by Word Count, Elements, Roles and Mechanisms
ARTICLE
WORD
TARGET
TARGET
TARGET
COUNT
ELEMENTS
ROLES
MECHANISMS
Greenhouse Effect
576
33
14
7
Space Elevator
583
33
19
7
Intervention
The intervention for this study occurred in three parts. The first part entailed a
brief written instruction on one type of relationship as determined by a participant's
group assignment. The instruction included a definition of the relationship type and
examples. Participants were also asked to exercise their understanding of the definition
by writing their own examples of relationships. The control group did not receive any
instruction.
The second part of the intervention was a one-on-one training that was conducted
as a manipulation check to ensure that a working knowledge of causal or functional
relationships was achieved before moving on in the study session (Appendix H). During
the training, the researcher asked the participant to provide a verbal definition of either a
causal or functional relationship based on group assignment. If the definition was not
correct, the researcher provided the correct definition. The participant was then asked to
apply this definition to create a relationship for a pair of words presented by the
researcher, for example, "exercise and calories". If the participant did not respond with a
representative causal or functional relationship, then the researcher provided a scripted
49
example, for example, "If you exercise, then you will burn calories" (scripted causal
example) or "The more exercise you do, the more calories you will burn" (scripted
functional example). A set of seven word pairs was prepared for this training (Appendix
I). Each participant was required to respond with relationships for at least five word pairs
that were presented one at a time. This training period averaged about 3-5 minutes. It
was determined that all participants were able to demonstrate a working understanding of
either functional or causal relationships as defined for this study. The control group did
not receive any training.
The third part of the intervention occurred after the training was completed. The
participant was given the reading about either the greenhouse effect or space elevators
and asked to write relationships (of the type for which they had just received training) for
eight keywords that had been selected by the researcher for each article. All participants
saw the same sets of eight keywords, but the causal and functional groups were presented
these words in pairs to make relationships and the control group was presented these
words in a list and asked to simply take notes.
The overall goal of this three-part intervention was to bias participants when
thinking about a system by a specific relationship type, causal or functional. The control
group was used to compare the everyday understanding of these systems without such a
bias.
Structured Interview
A structured interview was conducted after completion of the posttest. The
protocols for this interview can be found in Appendix D. The purpose of this interview
50
was to further understand how well the participants understood the assigned system
after completing the intervention.
The interview was also used to determine a participant's understanding of causal
or functional relationships by asking them to describe their understanding of these
relationships and then to describe the strategies for how they came up with specific
relationships in the third part of the intervention. A participant was given back the
worksheet with the relationships they wrote and asked to describe a relationship, for
example, between "sunlight and the Earth's atmosphere," and then asked to think aloud
the strategies they used to come up with their relationship.
The final question in the interview was a "what-if' problem-solving task. The
what-if task involved an open-ended question in which the participant is asked one of the
following questions: "What do you think would happen to the greenhouse effect if the
Earth's reflective surface area were decreased?" or "What do you think would happen to
the space elevator if it was anchored somewhere other than the equator?"
Transfer Task
A transfer task was included in the study to see if the effects of causal or
functional training influenced participants' reasoning about a novel system (Appendix J).
The transfer task was to design a large city on Mars by providing a written description
and drawing (adapted from Jacobson, 2001).
SCST
The Sigel Conceptual Style Sorting Task (SCST) is an instrument used to measure
one's conceptual style, "a term that refers to stable individual preferences in mode of
51
perceptual organization and conceptual categorization of the external environment"
(Kagan, Moss, Sigel, 1973). SCST-Form F contains 17 black and white pictorial triads
similar to those found in Figure 8. For each triad, a participant is asked to select two
pictures and provide a reason for why the selected belong together.
Figure 8
SCST Training Picture Triad
The SCST was chosen for use in this study because of the link between the
categorization and inductive skills used in causal reasoning (Gopnik & Sobol, 2000;
Tenenbaum, Griffiths & Kemp, 2006). The inclusion of a categorization task could
provide further insight into the nature of a one's inductive inferences.
The SCST was originally designed for administration with groups of elementaryage children. Children are provided with picture triad cards (or they are projected on an
overhead) and an answer booklet to record their own answers. The triad in Figure 8 is the
one used for training purposes. Children are instructed to pick any of two of the three
52
pictures that go together, belong together or are related in any way and a reason for
each choice. They are also given directions on how to record their responses on the
answer sheet. When children have finished the training triad, they have 90 seconds to
complete the two subsequent triads. Children are given 75 seconds to complete the triads
that follow.
For this study, the SCST was administered in a modified format for adults. The
modifications are as follows: (1) a participant completed the task individually; (2) the
participant recorded his/her own picture selections and reasons provided on a researcher
developed answer sheet; and (3) the participant was not given a strict time limit, but was
encouraged to move through the triads at a quick and steady pace if delays occurred.
Measures, Coding and Inter-rater Reliability
EPM Scores
The pretest and the posttest were identical tasks designed to measure knowledge
about the systems of the study, the greenhouse effect and space elevator (Appendix A).
This task included a free recall of participants' systems understanding. Responses from
this free recall were coded for elements, roles and mechanisms.
A target list of elements, roles and mechanisms for the greenhouse effect and
space elevator systems were identified from the selected readings and then revised
through a process of analysis and informal reliability testing on a subset of the participant
responses (Appendix K). Participant responses' were analyzed for the presence or
absence of the items in the target lists.
53
As presented in Hmelo-Silver, Marathe & Liu (2007), credit was given for
elements if there was a mention of a part in the system (i.e., contained in the target
elements list), whether alone or in describing a role or mechanism. An element was only
given credit once per response regardless of the number of times it was mentioned.
Credit was given for a role or mechanism if one from the target list was identified
in the free recall response. If both the role and the mechanism were described, then a
participant was given credit for each. A role or mechanism was only given credit once
per response regardless of the number of times it was mentioned.
In addition to the written system descriptions from the posttest, the verbal system
descriptions provided during the structured interviews were coded for elements, roles and
mechanisms to provide an additional measure of systems understanding. Visual system
descriptions were not coded at this time. All audiotaped interviews were transcribed and
the coders used the resulting transcripts for analysis.
Table 5
Percent Agreement for Written Pretest and Posttest Coding
Researcher and
Researcher and
First Coder
Second Coder
Elements
98
94
Roles
96
89
Mechanisms
99
87
54
The pretests and posttests were coded by the researcher and the two
independent coders. The coders were trained using a subset of the data through which
they gained experience working with the target lists. The subset of data was chosen by
selecting six consecutive participant files from the beginning, middle and end of the data
set. The six consecutive files were chosen so that an equal sample of the groups
(represented in Table 3) would be represented in the training3. The training procedure
can be found in Appendix L. Both coders trained on non-overlapping subsets of the data
so percent agreement between the researcher and each coder was calculated separately
(Table 5). The percent agreement between the researcher and the first coder was
calculated on approximately 25% of the data that was randomly selected for training.
The percent agreement between the researcher and the second coder was calculated on
approximately 12% of the data that was randomly selected for training. Though an
acceptable level of reliability was achieved in the training process, all pretest and posttest
data was double-coded for good measure due to the qualitative nature of these openended responses. In the double-coding process, the researcher and one of the coders
independently coded each pretest and posttest. Coders were blind to participants' group
assignments. The researcher then reviewed the codes and disagreements were identified.
These disagreements were resolved by discussion between the coders.
The verbal system descriptions from the interviews were coded by the researcher
and one of the coders previously trained for the pretest and posttest. Approximately 25%
of the data was randomly selected to calculate a percent agreement between the two
3
Similarly, all training and reliability samples were created in this manner. Training and reliability
samples did not overlap.
55
coders resulting in 95% agreement for elements, 87% agreement for roles and 91%
agreement for mechanisms. Upon achieving a satisfactory level of agreement, the
remaining verbal descriptions were coded by the researcher.
Reasoning Strategies
The responses from the questions in the structured interview about participants'
reasoning strategies were analyzed for the absence or presence of the strategies presented
in Appendix M (the training procedure is also included in this Appendix). This list of
strategies was developed after reviewing all the interview transcripts and identifying the
strategies most frequently used. Note that the term "strategies" is used to mean any
metacognitive thought or processing employed by the participant and not strictly in
reference to a plan of action. Examples of strategies recorded in the "Other" category
included paraphrasing, use of prior knowledge, simplifying information and underlining.
The frequencies for each strategy were calculated to determine a portrait of the
metacognitive reasoning used between groups.
Independent coders for the relationship strategy responses were trained on a
subset of the data through which they gained experience working with reasoning strategy
codes. Note that they were not blind to participants' group assignment because the
structured interviews differed by group. Approximately 25% of the data was randomly
selected and used to calculate inter-rater reliability as a percent agreement between the
researcher and each coder by strategy (Table 6). Intraclass correlations coefficients
(average measures) were also calculated as a measure of inter-rater reliability between the
three coders. Upon achieving a satisfactory level of agreement for all strategy codes, the
56
remaining responses were coded by the researcher.
Table 6
Percent Agreement and Intraclass Correlation Coefficients for Reasoning Strategy
Coding
Researcher and
First Coder
Researcher and
Intraclass
Second Coder
Correlation
Reading Article (AR)
75
75
.71
Cause and effect (CE)
75
88
.68
Functional relations (FR)
92
92
.93
Mental Simulation (MS)
92
100
.88
Intermediary Relationships (IN)
92
96
.60
Difficulty (DF)
88
92
.84
Easy (EZ)
83
88
.73
Object Manipulability (OM)
88
92
.73
Systems (SY)
83
92
.71
Other (OT)
100
83
.84
What-If Problem Solving
The responses from the what-if question in the structured interview were coded on
a scale of 0 to 3 where 0 was an incomplete response or was not complete enough to be
evaluated, 1 was a low complexity response, 2 was a medium complexity response and 3
57
was a high complexity response. The what-if coding training procedure can be found in
Appendix N.
The what-if responses were coded by the researcher and an independent coder.
The independent coder was trained using a subset of the data. Note that coders were not
blind to participants' group assignment because the structured interview protocols
differed by group. An inter-rater reliability analysis using the Cohen's Kappa statistic
was performed on 18% of the data to determine consistency between the coders. The
inter-rater agreement was 78% and the inter-reliability for the coders was found to be
Cohen's Kappa = .66. Upon achieving a satisfactory level of agreement for the what-if
coding, the remaining responses were coded by the researcher.
Mars City Transfer Task
The responses from the Mars city transfer task were coded on a scale of 0 to 3
where 0 was an incomplete response or was not complete enough to be evaluated, 1 was a
low complexity response, 2 was a medium complexity response and 3 was a high
complexity response. The Mars city transfer task training procedure can be found in
Appendix O.
The transfer task responses were coded by the researcher and an independent
coder. The independent coder was trained using a subset of the data. An inter-rater
reliability analysis using the Kappa statistic was performed on 18% of the data to
determine consistency between the coders. The inter-rater agreement was 82% and the
inter-reliability for the coders was found to be Cohen's Kappa = .72. Upon achieving a
58
satisfactory level of agreement for the Mars city transfer task coding, the remaining
responses were coded by the researcher.
CHAPTER IV
RESULTS
Independent Variables
The independent variables in this study are relationship instruction with three
levels (Functional, Causal, Control) referred to as the variable "Group" and systems type
with two levels (Greenhouse Effect, Space Elevator) referred to as the variable "System".
Dependent Measures
There are five dependent measures in this study: (1) two measures of systems
understanding using a percentage recall of Elements, Roles and Mechanisms scores from
written descriptions of the systems in the posttest and a percentage recall of Elements,
Roles and Mechanisms scores from the verbal descriptions of the system obtained during
a structured interview; (2) a measure of the frequency of participant's report of reasoning
strategies use during the intervention; (3) an ordinal measure of problem-solving
performance from the what-if problem-solving interview question; and (4) an ordinal
measure of complexity from the Mars city design transfer task.
Group Differences
An analysis of the scores for the recall of Elements, Roles and Mechanisms on the
written pretest was conducted to determine if there were any preexisting differences
systems knowledge in the randomly assigned groups for each system. The raw scores for
60
Elements, Roles and Mechanisms were converted to percentage recall scores so that
they could be compared to each other. The means and standard deviations of the pretest
percentage recall scores for the Greenhouse Effect system are presented in Table 7 and
for the Space Elevator system in Table 8.
Table 7
Greenhouse Effect
Means and Standard Deviations of Percentage Recall Pretest Scores by Group
Functional
Causal
Control
Mean (SD)
("=17)
("=17)
(" = 15)
Elements
9.45 (9.27)
6.24 (8.67)
8.69 (8.09)
Roles
3.25 (5.65)
3.78 (6.25)
4.76 (5.83)
Mechanisms
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
Table 8
Space Elevator
Means and Standard Deviations of Percentage Recall Pretest Scores by Group
Functional
Causal
Control
Mean (SD)
("=17)
("=17)
(n = 15)
Elements
8.56(8.11)
10.16(11.10)
6.87 (7.20)
Roles
3.87 (5.27)
4.49 (7.96)
1.43(4.00)
Mechanisms
0.84 (3.47)
0.84 (3.47)
0.95 (3.69)
61
A multivariate analysis of variance (MANOVA) was conducted to determine
the effect of the independent variable Group on two dependent variables, the percentage
recall for Elements and Roles on the pretest for the Greenhouse Effect system. The
scores for Mechanisms were not included in the analysis because the means were zero for
each group. The MANOVA indicated no significant differences between Groups for
these dependent variables, F(4, 90) = .90, p = .48.
For the Space Elevator system, the data on all three measures was not normally
distributed so the percentage recall for Elements, Roles and Mechanisms on the pretest
was transformed to square root values (the untransformed means and standard deviations
are presented in Table 8 for ease of interpretation). A MANOVA was conducted to
determine the effect of the independent variable Group on the three transformed
dependent variables and the MANOVA indicated no significant differences between
Groups, F(6, 88) = .57, p = .75.
Research Questions and Hypotheses
An analysis of the data was conducted to address the two main research questions
and hypotheses made with respect to these questions:
(1) When relationships in a system of a particular type (causal or functional) are
made explicit, how do they affect understanding of physical systems?
Hypothesis 1: Functional relationships engage different reasoning strategies than
causal relationships when trying to understand a physical system.
62
(2) Does reasoning about a physical system improve more with functional
relationships than with causal relationships?
Hypothesis 2: Functional relationships improve reasoning about and
understanding for a physical system over causal relationships alone.
Analysis for the First Hypothesis: Functional relationships engage different reasoning
strategies than causal relationships when trying to understand a physical system.
For the analyses of reasoning strategies, each participant's response to the
questions in the structured interview about their strategy use during the intervention was
coded for the presence or absence of the strategies identified in the target list shown in
Table 9. Examples of actual participant responses that were coded for each strategy are
also included in this table.
Table 9
Definitions and Examples for the Target List of Reasoning Strategies
STRATEGY (CODE)
Reading Article (AR):
Reading and searching the text
of an article for information
such as definitions, facts
(including descriptive
information like percentages
and other numerical measures),
key words/main deas
PARTICIPANT EXAMPLES
• "I went back and reread and sort of broke it down
sentence by sentence and made sure I was
understanding the steps that they were taking to
get to their explanation..." (ID#68)
• "... either I look for a definition or something
that, um, discussed that topic" (ID#71)
• ".. .1 was mostly, um, looking for like numbers
and terms, something that I wouldn't been able to
remember like right away." (ID#65)
63
Cause and Effect (CE):
Thinking about one object
causing changes in another
object in a linear, one-way or
sequential order, i.e. order
matters, like when following an
"if-then" format
Functional Relations (FR):
Thinking about dynamic,
functional (or the degree to
which one object changes when
another changes) and/or
mathematical relations such as
inverse, reciprocal or
proportional relationships
Mental Simulation (MS):
Using a mental simulation or
picture, visualization or
imagining
Intermediary Relationships
(IN): _
Thinking about intermediary or
indirect relationships
•
•
•
•
•
•
•
•
Acknowledging Difficulty (DF):
Having difficulty in deriving or
finding the relationship or
information and involving lots
of effort and thought possibly
because relationship does not
exist or information is hard to
understand
•
•
"I try to identify which is the thing that would be
affected first and then I described how the other
one affects it." (ID# 10)
"I chose the first item and I thought how does this
item, like item, X, affect the second item that was
listed in bold..." (ID#45)
".. .one thing increased the other or, or decreased
it or was the same and tried to fit that to each
one..."(ID#19)
"... I looked... how the atmosphere can be
changed by sunlight or the opposite, it's actually,
I was thinking it doesn't have to be that sunlight
is to cause the atmosphere it can be both sides, so
I was trying to think if sunlight, if there is more
sunlight what would happen to atmosphere or if
atmosphere gets thinner, what would happen to
sunlight..." (ID#57)
"I just imagined that process just while, while I
was reading the article" (ID#25)
"I was trying to play out the scenarios in my head
of what a change would look like..." (ID#14)
"... sometimes it didn't appear that the
relationships were direct. It seemed like there
was something else sort of in the middle, or, or,
that, that was in the middle between the two
objects that I was supposed to evaluate the
relationship for" (ID#86)
".. .1 did have to go back to the article sometimes
and reread and see what the different, um, like
mini-relationships are in between there." (ID#32)
".. .then I had to look at like a relationship
between the two that I could create a cause and
effect from so it, it was a little bit more actual
thought process in it as opposed to just rewriting
basically was on, in the article." (ID#94)
".. .1 put a question mark by it because I wasn't
sure if there really was a relationship because I
don't see one exactly impacting the other."
(ID#56)
64
Easy/Low Effort (EZ):
Involving little strategy because
relationship or information was
direct, easy to find, required
little thought or taken straight
from the article
Object Manipulability (OM):
Exploring the ability to
manipulate or change objects
(of/in the relationship)
•
".. .1 mean it was actually the article that blatantly
just said that...." (ID#31)
• ".. .the relationship between these two things is
very clearly shown, shown in the, uh, the last
sentence of that, uh, article" (ID#26)
•
•
Systems Thinking (SY):
Thinking about objects in
relation to a system or parts to a
whole
•
•
".. .1 was kind of like, well the sunlight doesn't
change, right, the sun is not a, is not a, it's a
constant and the Earth's atmosphere, the Earth's
surface, they do not affect the sun in any way, so
how can you even begin to describe this as a
functional relationship, um, because you're
basically just asking like, like, it, how does a
constant thing affect something else that can not
possibly affect it, in turn..." (ID#13)
"I can tell like there's more sunlight, it's like, uh,
the atmosphere, um, the Earth's atmosphere is
going to absorb more sunlight, but actually
there's 20%, uh, in terms of the amount, might
absorb more, but it doesn't really, uh, absorb
more than 20% ..." (ID#25)
"... when I was prompted by a word, I thought
about the word in relation to the whole system
and then I described how that feature of the
system functioned within in the system." (ID#53)
".. .it was supposed to be about systems so I
mean, uh, when I think systems, I think
definitions and, uh, their interactions with each
other." (ID#54)
The mental simulation and object manipulability strategies were pooled to create
the dynamic strategies category for analysis because they were the strategies of most
interest. The dynamic strategies represent the mental and physical aspect of dynamism
that was reported when thinking about how relationships were constructed. The presence
of the dynamic strategies for each participant was counted as a report of either mental
simulation or object manipulability.
65
Participant report of these strategies were then calculated by Group and
presented in Table 10 with the more notable strategies highlighted in grey. Participants
in the Functional group showed overall greater use of reasoning strategies than those in
the Causal group. Participants in the Control group reported very little strategy use.
Table 10
Participant Report of Reasoning Strategies by Group
Functional
Causal
Control
Total
(« = 34)
(« = 34)
(w = 30)
(N=98)
Reading Article (AR)
22
18
24
64
Cause and Effect (CE)**
13
29
0
42
Functional Relations (FR)**
29
11
0
40
Dynamic Strategies (DS)*
15
8
1
24
Intermediary Relationships (IN)*
8
5
0
13
Acknowledging Difficulty (DF)**
20
9
1
30
Easy/Low Effort (EZ)*
9
6
0
15
Systems Thinking (SY)
5
3
5
13
Other (OT)*
7
3
11
21
* Chi-Square statistic, p < .05, ** p < .01
66
A two-way Pearson Chi-square analysis was conducted to evaluate the
relationship between each reasoning strategy and Group. Group was found to be
significantly related to the cause and effect strategy, Pearson x (2, 98) = 47.80, p < .01,
the functional relations strategy, Pearson x2 (2, 98) = 49.54, p < .01, the dynamic
strategies, Pearson x 2 (2, 98) = 14.36, p < .01, the intermediary relationships strategy,
Pearson x (2, 98) = 7.76, p = .02, the acknowledging difficulty strategy, Pearson x (2,
98) = 23.52, p < .01, the easy/low effort strategy, Pearson x2 (2, 98) = 8.84, p = .01, and
the use of Other strategies, Pearson x2 (2, 98) = 7.36, p = .03.
Again, based on the frequencies, it is evident that the Control group reported very
little strategy use during the intervention aside from reading and searching the articles for
facts and definitions when compared to the Functional and Causal groups which likely
contributes to the overall significant Group differences. An analysis on the participant
report of reasoning strategies for just the two treatment groups was conducted. Group
with just two levels, Functional and Causal, is significantly related to the cause and effect
strategy, Fisher's exact test4, p < .01, the functional relations strategy, Fisher's exact test,
p < .01, and the acknowledging difficulty strategy, Fisher's exact test, p = .01. The
Causal group used the cause and effect strategy significantly more than the Functional
group, while the Functional group used the functional relations and acknowledging
difficulty strategies significantly more than the Causal group.
The Fisher's exact test is a stricter measure for smaller sample sizes and is being reported here
specifically for the 2x2 tables.
67
A Posteriori Analysis: Relationships between Reasoning Strategies
To further explore the profile of strategy use from a functional and causal bias
beyond Group differences, additional analyses were conducted to determine if any
relationships existed between the functional relations and cause and effect strategies with
the other strategies, i.e., is there a difference in the report of strategy use when the
functional relations and not the cause and effect strategy is reported and vice versa? This
question was answered by conducting a series of two-way contingency table analyses to
look at the relationship between functional relations and the cause effect strategies vs. the
dynamic strategies, intermediary relationships and acknowledging difficulty. The cases
included in these analyses were comprised of only those participants who reported
mutually exclusive use of either the functional relations strategy (n = 18) or the cause and
effect strategy (n = 20). The strategies of reading articles and systems thinking were not
included here because they did not show significant differences in any prior analyses,
easy/low effort was not of interest and Other strategies were not considered because they
represented a collection of infrequently used strategies.
71% of participants who reported use of the functional relations strategy and 29%
of participants who reported use of the cause and effect strategy reported use of the
dynamic strategies, a difference that was statistically significant, Fisher's exact test, p =
04. Similarly, 71% of participants who reported use of the functional relations strategy
and 29%o of participants who reported use of the cause and effect strategy acknowledged
difficulty in the intervention task, a difference that was statistically significant, Fisher's
exact test, p = 02. The difference in report for the intermediary relationships strategy was
68
not significant, Fisher's exact test, p = .71. Contingency tables for the dynamic
strategies and the acknowledging difficulty strategy can be found in Table 11.
Table 11
Contingency Tables for the Cause and Effect (CE) and Functional Relations (FR)
Strategies with Dynamic Strategies (DS) and Acknowledging Difficulty (DF)
Strategy
CE
DS
No
Yes
Total
Count
% within DS
Count
% within DS
Count
% within DS
Total
FR
16
8
24
66.7%
4
28.6%
20
52.6%
33.3%
10
71.4%
18
47.4%
100.0%
14
100.0%
38
100.0%
Strategy
CE
DF
Total
No
Count
% within DF
15
71.4%
Yes
Count
% within DF
Count
% within DF
5
29.4%
20
52.6%
FR
Total
6
28.6%
12
21
100.0%
17
70.6%
18
47.4%
100.0%
38
100.0%
69
Analysis for the Second Hypothesis: Functional relationships improve the reasoning
and understanding of complex physical systems better than causal relationships alone.
Participants' free recall of system knowledge was used as a measure of systems
understanding for analysis of the second hypothesis. Participants' responses were coded
for the absence or presence of Elements, Roles and Mechanisms that were identified in a
target list from the articles presented during the intervention (Appendix K) and then
summed to get a recall score for Elements, Roles and Mechanisms. Elements are the
parts of the system, e.g., "carbon dioxide" and "counterweight". Roles are the roles of
Elements in the system, e.g., "the atmosphere contains gases such as C02 that would
absorb part of the sun radiation..." and ".. .the carbon ribbon anchored to an offshore
location and attached to a satellite used as a means to attach an elevator..." Mechanisms
explain how Elements work, e.g., "This exchange of light from sun, to Earth, backout to
space regulates temperature - w./out the atmosphere & this g.h. [greenhouse] effect our
world could be a lot colder or warmer" and "since less energy is required to keep the
counterweight in rotation by the equator this is the ideal spot. The more you go toward
the poles the more energy required resulting in a higher cost."
Raw scores for Elements, Roles and Mechanisms were converted to percentage
recall scores for system descriptions from the written posttest and the verbal recall for
system descriptions during the structured interview. All recall scores reported from here
on are percentage recall scores.
The means and standard deviations for the percentage recall scores by Group for
the Greenhouse Effect system in Table 12 and for Space Elevator system in Table 13. No
70
significant interaction effects were found between Group and System in the planned
analyses. Subsequently, all results are reported for the Greenhouse Effect and the Space
Elevator systems separately.
Table 12
Greenhouse Effect
Means and Standard Deviations of Percentage Recall Difference Scores by Group
Functional
Causal
Control
Mean (SD)
(«=17)
(n=17)
(«=15)
Interview Elements
31.91 (9.94)
28.34 (12.72)
31.31 (14.29)
Interview Roles
20.12(7.26)
16.72(10.89)
15.79 (9.54)
Interview Mechanisms
15.13 (14.70)
13.45 (17.84)
13.33(16.61)
Posttest Elements
25.85 (6.35)
24.24(11.19)
25.05 (12.09)
Posttest Roles
21.43(7.58)
21.43(11.85)
18.10(11.73)
Posttest Mechanisms
15.97(14.18)
12.61 (16.66)
11.43(16.38)
71
Table 13
Space Elevator
Means and Standard Deviations of Percentage Recall Difference Scores by Group
Functional
Causal
Control
Mean (SD)
(n = 17)
(n = 17)
(n = 15)
Interview Elements*
28.70 (8.03)
21.75(7.89)
30.10(9.95)
Interview Roles*
22.29(13.10)
11.46(8.37)
16.84(8.49)
Interview Mechanisms
9.24(13.31)
5.04(7.04)
8.57(10.53)
Posttest Elements
30.12(12.00)
25.13 (9.57)
35.15(13.35)
Posttest Roles
21.67(13.66)
15.17(9.28)
19.65(16.10)
Posttest Mechanisms
12.61(18.13)
12.61(15.88)
14.29(21.60)
* p < .05
Analysis of Verbal System Recall from the Structured Interviews
A MANOVA was conducted to determine the effect of Group on the three
dependent variables, the percentage verbal recall for Elements, Roles and Mechanisms
(which will be referred to as simply as Element, Role and Mechanism scores going
forward) from the Greenhouse Effect descriptions provided during the structured
interviews. The MANOVA showed no significant difference for the main effect of
Group, F(6, 88) = .67, p = .67, partial r\2 = .04, observed power = .25.
A MANOVA was conducted to determine the effect of Group on the three
dependent variables, verbal recall for Elements, Roles and Mechanisms from the Space
72
Elevator descriptions provided during the structured interviews. Box's Test of
Equality of Covariance Matrices5 was significant, p = .04, indicating a violation of
homoscedasity. However, the separate Levene's Test of Equality of Error Variances for
each of the dependent variables were not significant so the amount of variance was
presumed to be comparable between the groups. The MANOVA showed a significant
difference for the main effect of Group, F(6, 88) = 2.89, p = .01, partial r|2 = .17,
observed power = .87.
Given the significant main effect at the multivariate level, the univariate betweensubjects tests were examined to further explain the effect of Groups for the Space
Elevator system in the verbal recall from the structured interviews. There was a
significant difference in the verbal recall of Elements by Group, F(2, 49) = 4.44, p = .02,
partial r\2 = .16. Pairwise comparisons using the Tukey method revealed that the Control
group (M = 30.10, SD = 9.95) verbally recalled significantly more Elements than the
Causal group (M = 21.75, SD = 7.89). The Functional group (M - 28.70, SD = 8.03)
also recalled more Elements than the Causal group but the difference was not significant
at the adjusted alpha level.
There was a significant difference in the verbal recall from the structured
interviews of Roles by Group, F(2, 49) = 4.71, p = .01, partial r|2 = .17. Pairwise
comparisons using the Tukey method revealed that the Functional group (M = 22.29, SD
= 13.10) verbally recalled significantly more Roles than the Causal group (M = 11.46, SD
Box's M is very sensitive to nonnormality and should be interpreted cautiously (Stevens, 2002). The
MANOVA is relatively robust in light of a significant Box's M when there are equal sample sizes in each
cell.
73
= 8.37). The Functional group also verbally recalled more Roles than the Control
group (M = 16.84, SD = 8.49) but the difference was not significant at the adjusted alpha
level.
There were no significant differences in verbal recall of Mechanisms by Group
for the Space Elevator, F(2, 49) = .76, p = .47, partial r]2 = .03.
Analysis of System Recall from Written Posttests
A MANOVA was conducted to determine the effect of Group on the three
dependent variables for recall of Elements, Roles and Mechanisms from the Greenhouse
Effect descriptions provided in the written posttests. The MANOVA showed no
significant difference for the main effect of Group, F(6, 86) = .38, p = .89, partial r\ =
.03, observed power = .15.
A MANOVA was conducted to determine the effect of Group on the three
dependent variables for recall of Elements, Roles and Mechanisms from the Space
Elevator descriptions provided in the written posttests. The MANOVA showed a
significant difference for the main effect of Group, F(6, 88) = 2.56, p = .03, partial r\ =
.15, observed power = .82.
Given the significant main effect at the multivariate level, the univariate betweensubjects tests were examined to further explain the effect of Groups for the Space
Elevator system in the written posttests. However, there were no significant differences
in written recall for the three dependent variables by Group.
74
A Posteriori Analysis: Explicit Use of Functional Relationships
Although the readings about the two systems presented during the intervention
were considered equivalent based on word count and target number of Elements, Roles
and Mechanisms, they differed in their use of functional relationships to describe the
systems in the articles, specifically, the space elevator text contained explicit functional
descriptions by design (e.g., "The lower the orbit of an Earth satellite the faster it has to
go" and "The more space elevators there are, the lower the cost of putting cargo into
space and the more the economy will be stimulated."). The use of explicit functional
relationships appears to have been enough to affect recall of Elements, but not Roles or
Mechanisms.
Table 14
Target Items from the Explicit Functional Relationships in the Space Elevator Article
Elements
(8 of 33 total)
Roles
(3 of 19 total)
Mechanisms
(5 of 7 total)
•
•
•
•
•
•
•
•
*
*
*
•
'
•
'
'
cost
developers
Earth's (surface) rotation
economy
energy (to launch a satellite)
equator
orbit (of satellite)
satellites
developers
equator
space elevator
cost/economy
Earth's (surface) rotation
multiple space elevators and danger
multiple space elevators and cost
orbit (of satellite)
75
This evidence that explicit functional relationships in the Space Elevator text
(shown in bold in Appendix G) could contribute to systems understanding was further
explored by identifying the subset of scores for the specific Elements, Roles and
Mechanisms in the target list that were derived from the functional relationships (Table
14).
The means and standard deviations for the percentage recall for the Elements
(number of functional Elements recalled/8), Roles (number of functional Roles
recalled/3) and Mechanisms (number of functional Mechanisms recalled/5) derived from
the functional relationships and the percentage recall for the rest of the Elements (number
of non-functional Elements recalled 125), Roles (number of non-functional Roles recalled
/16) and Mechanisms (number of non-functional Mechanisms recalled 12), i.e., those not
derived from the functional relationships, can be found in Table 15 for the structured
interview and Table 16 for the written posttest.
A paired-samples t-test analysis was conducted to compare the percentage recall
for the Elements, Roles and Mechanisms derived from the functional relationships and
the percentage recall for the Elements, Roles and Mechanisms not derived from the
functional relationships for each group.
76
Table 15
Means and Standard Deviations of Percentage Recall Derivedfrom Functional
Relationships in the Structured Interview
Mean (SD)
From Functional
Not from Functional
Relationships
Relationships
Correlation
FUNCTIONAL GROUP (n = 17)
Elements*
23.53 (13.89)
30.35 (7.36)
.59,p = .01
Roles*
37.25 (30.92)
19.49(11.88)
.49, p = .05
Mechanisms
11.76(18.79)
2.94(12.13)
•.16,p=.54
Elements
21.32(14.50)
21.88(8.85)
. l l , p = .69
Roles
19.61 (23.74)
9.93 (9.90)
.22, p = .41
Mechanisms
4.71 (8.75)
5.88(16.61)
.20, p = .44
Elements**
20.00(16.90)
33.33 (10.22)
.35, p = .20
Roles
24.44 (23.46)
15.42 (10.26)
.26, p = .36
Mechanisms
10.67(14.86)
3.33(12.91)
-.20, p = .48
CAUSAL GROUP (n = 17)
CONTROL GROUP (n = 15)
*p<.05, **p<.01
Structured Interview Results
For the Functional group, there was a significant difference for the percentage
recall of Elements, t{\6) = -2.49, p = .02, with participants recalling a greater percentage
of the Elements not derived from the functional relationships (M = 30.35, SD = 7.36)
77
than those derived from functional relationships (M = 23.53, SD = 13.89). There was a
significant difference for the percentage recall of Roles, /(16) = 2.70, p = .02, with
participants recalling a greater percentage of the Roles derived from the functional
relationships (M = 37.25, SD = 30.92) than those not derived from the functional
relationships (M = 19.49, SD = 11.88). There was no significant difference for the
percentage recall of Mechanisms, ^(16) = 1.52, p = .15.
For the Causal group, there were no significant differences found on recall for
Elements, Roles or Mechanisms.
For the Control group, there was a significant difference for the percentage recall
of Elements, t(l6) = -3.15, p < .01, with participants recalling a greater percentage of the
Elements not derived from the functional relationships (M = 33.33, SD = 10.22) than
those derived from the functional relationships (M = 20.00, SD = 16.90). There was no
significant difference for the percentage recall of Roles, t(l6) = 1.25, p = .23, and
Mechanisms, t(\6) = 1.32, p = .21.
Written Posttest Results
For the Functional group, there was a significant difference for the percentage
recall of Roles, t(l6) = 4.01, p < .01, with participants recalling a greater percentage of
the Roles derived from the functional relationships (M = 33.33, SD = 26.35) than those
not derived from the functional relationships (M = 12.50, SD = 15.31). There was no
significant difference for the percentage recall of Elements, t(\6) = -.66, p = .52, and
Mechanisms, ?(16) = -1,16, p = .26.
78
Table 16
Means and Standard Deviations of Percentage Recall Derivedfrom Functional
Relationships in the Written Posttest
Mean (SD)
From Functional
Not from Functional
Relationships
Relationships
Correlation
FUNCTIONAL GROUP (n = 17)
Elements
24.26(18.99)
27.53 (17.14)
.36,p = .15
Roles**
33.33 (26.35)
12.50(15.31)
.58,p = .01
Mechanisms
11.76 (21.28)
17.65 (30.32)
.72, p = .00
Elements
31.62(14.06)
33.41 (14.90)
.64,p = .01
Roles**
45.10(23.40)
16.91 (14.45)
.26,p = .31
Mechanisms
15.29(19.40)
11.76(21.86)
.43,p = .08
Elements
24.17(12.91)
26.67 (10.76)
.04, p = .88
Roles**
37.78 (24.77)
14.17(12.60)
-.07, p = .80
Mechanisms
8.00(18.21)
13.33 (22.89)
-.10,p = .72
CAUSAL GROUP (n = 17)
CONTROL GROUP (n = 15)
*p<.05, **p<.01
For the Causal group, there was a significant difference for the percentage recall
of Roles, t{\6) = 4.83, p < .01, with participants recalling a greater percentage of the
Roles derived from the functional relationships (M = 45.10, SD = 23.40) than those not
derived from the functional relationships (M = 16.91, SD = 14.45). There was no
79
significant difference for the percentage recall of Elements, t(\6) = -.60, p = .56, and
Mechanisms, ^(16) = .66, p = .52.
For the Control group, there was a significant difference for the percentage recall
of Roles, t(l6) = 3.20, p < .01, with participants recalling a greater percentage of the
Roles derived from the functional relationships (M = 37.78, SD = 24.77) than those not
derived from the functional relationships (M = 14.17, SD = 12.60). There was no
significant difference for the percentage recall of Elements, ^(16) = -.59, p = .57, and
Mechanisms, t(\6) = -.67, p = .51.
Investigating Transfer Effects
What-If Problem Solving
The what-if question in the structured interview was intended to measure the near
transfer of the relationship training from the intervention The what-if problem solving
responses were analyzed using an ordinal regression with Group as a predictor and four
ordinal levels representing complexity (0 = Incomplete, 1 = Low, 2 - Medium, 3 = High).
The frequencies of the what-if scores by Group for the participants who studied the
Greenhouse Effect are presented in Table 17 and for the participants who studied the the
Space Elevator in Table 18. The Logit link function was used in this ordinal regression
assuming cases were to be evenly distributed among the levels.
80
Table 17
Greenhouse Effect
Frequency of What-If Score by Group
Functional
Causal
Control
Total
(/i =17)
(»=17)
(« = 15)
(N=49)
Incomplete
1
0
1
2
Low
5
2
6
13
Medium
6
8
5
19
High
5
7
3
15
Table 18
Space Elevator
Frequency of What-If Score by Group
Functional
Causal
Control
Total
(w=17)
(w =17)
(« = 15)
(N=49)
Incomplete
0
2
Low
5
7
3
15
Medium
10
3
7
20
High
2
5
4
11
1
3
81
The Chi-Square statistic, x2
=
4.37, p = .11, shows that the ordinal regression
model with Group as a factor for the participants who studied the Greenhouse Effect did
not have good predictive value over the baseline model without Group suggesting
potential differences on level of complexity in the responses for the what-if question by
Group.
The Chi-Square statistic, % = 1.06, p = .59, shows that the ordinal regression
model with Group as a factor for the participants who studied the Space Elevator did not
have good predictive value over the baseline model without Group and there also appears
to be no difference on level of complexity in the responses to the what-if question by
Group of the Space Elevator system.
Mars City Transfer Task
The Mars city design task was intended to measure the far transfer of the
relationship training from the intervention to a novel domain. The Mars city transfer task
was analyzed using an ordinal regression with Group as a predictor and four ordinal
levels representing complexity (0 = Incomplete, 1 = Low, 2 = Medium, 3 = High). The
frequencies of the transfer task scores by Group for the participants who studied the
Greenhouse Effect are presented in Table 19 and for the participants who studied the
Space Elevator in Table 20. The Logit link function was used in this ordinal regression
assuming cases were to be evenly distributed among the levels.
82
Table 19
Greenhouse Effect
Frequency of Mars City Transfer Task Complexity by Group
Functional
Causal
Control
Total
0=17)
(n == 17)
(n == 15)
(N=49)
Incomplete
1
0
1
2
Low
6
10
10
26
Medium
4
2
1
7
High
6
5
3
14
Table 20
Space Elevator
Frequency of Mars City Transfer Task Complexity by Group
Functional
Causal
Control
Total
("=17)
("=17)
(rc = 15)
(N=49)
Incomplete
2
Low
21
Medium
20
High
6
The Chi-Square statistic, %2 = 2.52, p = .28, shows that the ordinal regression
model with Group as a factor for the participants who studied the Greenhouse Effect did
83
not have good predictive value over the baseline model without Group and there
appears to be no difference on level of complexity in the responses for the transfer task
by Group, though performance levels were in expected directions.
The Chi-Square statistic, y? = 3.51, p = .17, shows that the ordinal regression
model with Group as a factor for the participants who studied the Space Elevator did not
have good predictive value over the baseline model without Group and there appears to
be no difference on level of complexity in the responses for the transfer task by Group,
though performance levels were in expected directions.
A Posteriori Analysis: Transfer Task Performance by Reasoning Strategies
Additional exploratory analyses were conducted for the transfer tasks based on
participant use of the functional relations or cause and effect strategies to determine if
there were effects on transfer task performance that were not restricted to group
assignment. Specifically, the cases included in these analyses were comprised of only
those participants who reported mutually exclusive use of either the functional relations
strategy (n = 18) or the cause and effect strategy (n = 20). In prior analyses, these cases
revealed a significant difference in use of the dynamic strategies and metacognitive
acknowledgement of difficulty between participants who reported use of the functional
relations strategy and participants who reported use of the cause and effect strategy.
The what-if transfer task was analyzed using an ordinal regression with strategy
use as a predictor and four ordinal levels representing complexity (0 = Incomplete, 1 =
Low, 2 = Medium, 3 = High). For participants who studied both the Greenhouse Effect
and the Space Elevator systems, there was no difference in performance by strategy use.
84
The Mars city transfer task was also analyzed using an ordinal regression with
strategy use as a predictor and four ordinal levels representing complexity (0 =
Incomplete, 1 = Low, 2 = Medium, 3 = High). For the participants who studied the
Greenhouse Effect system, the Chi-Square statistic, x2
=
4.44, p = .04, shows that the
ordinal regression model with strategy use as a factor did have good predictive value over
the baseline model without strategy use and there appears to be a difference on level of
complexity in the responses for the transfer task by strategy use. Further, the Pearson's
fitted model, % = .86, p = .34, and the deviance model, % = .87, p = .35, are both not
significant indicating that the data and the model predictions are similar and that this
regression model fits the data well, the effect size, Nagelkerke's R-Square = .23.
The Wald statistic = 4.07, p = .04, for the functional relations strategy use group
is significant suggesting that there is a difference in performance on the Mars city transfer
task between the functional relations strategy use group and the cause and effect strategy
use group. Figure 9 is a graph of the performance levels by strategy use.
The corresponding ordinal regression on the Mars city transfer task for
participants who studied the Space Elevator did not reveal any difference in Mars city
transfer task performance by strategy use.
85
Figure 9
Greenhouse Effect
Performance Level by Strategy Use on Mars City Transfer Task
Strategy Use
Cause and Effect
Functional Relations
o
u
Low
Medium
Mars City Design Complexity
High
CHAPTER V
DISCUSSION
The purpose of this study was to investigate the effect of reasoning with
functional relationships over causal relationships when studying about complex physical
systems. The key finding is that functional relationships engage different reasoning
strategies than those engaged by causal relationships contributing to an improvement for
an understanding of the roles (of elements) in physical systems. This chapter presents a
discussion of the study results in the context of the two main hypotheses regarding the
profile of reasoning strategies engaged by these two types of relationships and the effect
on systems understanding that results from the difference in reasoning strategy use. This
discussion is followed by the limitations of the study and the potential for future research.
Finally, instructional implications and education applications are considered.
Overview of Study Results
The Effect of Relationship Type on Reasoning Strategies
It was hypothesized that functional relationships would engage different
reasoning strategies from causal relationships when trying to understand a physical
system. As expected, there were significant differences in the frequencies of strategy use
between the Functional, Causal and Control groups for most of the strategies that were
targeted in the present study. In contrast to findings from previous research that contend
students hold simple linear models of causality (Bullock, Gelman, and Baillargeon, 1982;
87
Grotzer & Perkins, 2003; Grotzer & Sudbury, 2000;), the non-treatment Control group
did not report any use of the cause and effect strategy. Left to their own devices, they
relied primarily on looking for definitions, key words and main ideas while studying the
system articles akin to a traditional approach to studying systems (Perkins, 1986).
One possible explanation for the Control group's traditional approach is the
format of the Control group intervention worksheet where the eight key elements were
presented in a list and participants were asked to take notes about these elements from the
article. The Functional and Causal group worksheets presented the same eight elements
as four pairs for which they created relationships. It would have been interesting to see if
simply presenting the words as pairs without relationship training would have evoked
reasoning strategies that involved thinking about relationships of any type.
However, it is proposed that a strategy of looking for definitions is easier to
recognize metacognitively than the process of finding causes and effects, so the Control
group results are not unexpected. Participants, who were inclined to look at the list and
take notes beyond definitions, though few in number, did so despite the worksheet format
and without prompting. In other words, the task was considered open-ended enough to
induce a range of strategies. For example, ParticipantID#53 of the Control group
responded to the question about his strategy use by considering the elements as part of a
system, an example of higher level strategy use: "I used my understanding of the system
as a whole, um, to guide what things to write about any one of these constituent parts so
when I was prompted by a word, I thought about the word in relation to the whole system
and then I described how that feature of the system functioned within in the system."
88
Comparison of the Treatment Groups
Comparison of the two treatment groups, the Functional and Causal groups, found
significant differences in the use of the cause and effect, functional relations and the
acknowledging difficulty strategies. The significant difference in the use of the cause and
effect strategy by the Causal group and the functional relations strategy by the Functional
group would be considered as a manipulation check for the intervention because
participants received training biased to these respective strategy types during the
intervention.
The significant difference in the acknowledgment of difficulty strategy for
creating relationships of a functional type over a causal one further supports the first
hypotheses. Functional relationships are conceptually difficult and when one undertakes
the effort of constructing them, it is expected that this would be a harder task than
constructing causal relationships. The metacognitive awareness associated with the
acknowledgement of difficulty in working with functional relationships is interpreted as
an appreciation for the complexity involved in functional thinking that goes beyond the
simple, static causal views that are more prominent. At a higher level, the metacognitive
awareness in assessing difficulty would seem to engage a different level of thinking about
how relationships are constructed. Chi et al. (1989) found that successful students are
better able to monitor their comprehension failures than unsuccessful students. A
detection of incomprehension initiates a search for additional explanations with specific
questions about what is not understood resulting in a better understanding of a problem
89
space. In the same way, the detection of difficulty in creating functional relationships
should initiate a search for additional explanations about system behaviors as well.
Finally, though not statistically significant, another difference between the
Functional and Causal groups is in the reported use of the dynamic strategies of object
manipulability and mental simulation. These strategies captured participants'
examination of how/if objects can change to take part in a dynamic relationship or if they
mentally simulated relationships. For example, Participant ID#14 states in response to
the interview question about strategy use during the intervention: "I was trying to think,
thinking, I was trying to play out the scenarios in my head of what a change would look
like if the carbon, if there were to be a change in carbon composite ribbon, I don't feel
that would really change the counterweight much. The counterweight would still have its
same properties even if we were to change something about the carbon ribbon. But, if we
changed the counterweight, that would obviously change the carbon ribbon. Uh, without
the counterweight, the, the absence of the counterweight would also not benefit the
carbon ribbon at all." This response indicates both the mental simulation, "I was trying
to play out the scenarios in my head of what a change would look like..." and object
manipulability, i.e., considering the transformative property of a counterweight. Object
manipulability is a feature of functional relationships and can be represented in tools such
as simulation diagrams (Tsuei, 2004) in relationships like "lava cools (into) igneous
rock." However, because the definition of functional relationships used in the training
for the present study emphasized proportional and inverse proportional functions, the
reasoning about the state and manipulability of an object was interesting to find.
90
Overall, what is evident is that explicit instruction and training on the types of
interactions in the system lead learners to think more about them as expected from
previous research (Barbos & Psillos, 2003; Grotzer & Baska, 2003; Grotzer & Sudbury,
2000; Hmelo-Silver et al., 2007). In addition, training on functional relationships
engages more strategies when reasoning about physical systems than training on causal
relationships, or in the absence of relationship training as with the Control group. The
Control group who completed the study without any explicit instruction primarily
resorted to traditional methods (such as identifying definitions, facts/numbers) of
addressing new knowledge.
Conceptually, functional relationships are proposed to be on a continuum with
causal relationships so the underlying foundations are the same, i.e., the causal
relationship. In the functional spectrum of the continuum, it appears that there are
different types of reasoning strategies that are engaged, specifically, thinking about
functional relations, the dynamic strategies and the metacognitive awareness of difficulty.
In the causal spectrum of the continuum, there are consideration of causes and effects
which can invoke thinking about functional relations, possibly in a manner of shuttling
(Schwartz & Black, 1996), but does not fully engage the other strategies that are
demonstrated in the functional spectrum. Most importantly, the functional spectrum
appears to better stimulate reasoning that supports mental simulation and "envisionment"
(deKleer & Brown, 1983) needed for developing robust mental models of physical
systems.
91
Relationships between Reasoning Strategies
The additional exploratory analysis that was conducted to investigate the
relationship between the reported use of the functional relations and cause and effect
strategy with other strategies further supports these findings. The results imply that the
use of the functional relations strategy increases the likelihood for the use of the dynamic
strategies and acknowledging difficulty as demonstrated by the greater percentage of
participants who reported using these strategies when also reporting use of the functional
relations strategy. This represents a significant difference from the use of these two
strategies, i.e., a much lower likelihood of their use, when the cause and effect strategy
was employed. Thus, it is promising that getting learners to think with a functional
relations strategy is a more productive approach for engaging deeper dynamic thinking
than encouraging a cause and effect strategy alone.
The Effect of Functional Relationships on System Understanding
It was hypothesized that functional relationships improved the reasoning and
understanding of complex physical systems better than causal relationships alone.
Systems understanding was operationalized as the percentage recall of elements, roles
and mechanisms in the written posttest system descriptions and the verbal system
descriptions from the structured interviews. The multivariate analysis showed little
difference in systems understanding in the written posttests and the verbal interviews for
the Greenhouse Effect. The Functional group did not perform better than the Causal and
Control groups as expected.
92
For the Space Elevator system, there were significant differences in the verbal
recall of elements and roles during the structured interviews. The findings are as follows:
•
The Control group verbally recalled a significantly greater percentage of elements
than the Causal group.
•
The Functional group also verbally recalled a greater percentage of elements than
the Causal group though the difference was not statistically significant.
•
The Functional group verbally recalled a significantly greater percentage of roles
than the Causal group.
•
The Functional group also verbally recalled a greater percentage of roles than the
Control group though the difference was not statistically significant.
There was a significant multivariate effect between the three groups for written
recall in the posttest for the Space Elevator, but insignificant univariate tests made it hard
to further explain this effect. One possible explanation for this result is that the task of
writing allowed participants more time to thoughtfully explain their systems
understanding reducing the variability of their responses when compared to the verbal
responses from the structured interviews which required spontaneous thought.
These findings are to some degree as expected from previous research (HmeloSilver, Marathe & Liu, 2007; Hmelo-Silver, Pfeffer, 2004) that found when comparing
expert and novice understanding of complex systems, the differences in their
understanding and mental models of systems were at the level of behaviors and functions,
i.e., analogous to the present study's measures of mechanisms and roles, respectively.
Novice and expert understanding of structures, i.e., elements, was the same. The results
93
of the present study indicate that exposure to functional relationships in a single testing
session is not enough to affect performance on mechanisms that reflect expert-like
systems understanding. Therefore, the effect on performance in recalling roles achieved
during the short intervention is viewed as significant.
The Effect of Explicit Functional Relationships
The Space Elevator reading was prepared with explicit functional relationships
integrated into the article; only implicit functional relationships that needed to be inferred
were included in the Greenhouse Effect reading. Some additional insights into the effect
of explicit functional relationships on systems understanding were gained by doing a
comparison of the means of percentage recall for the Space Elevator system (Table 12
and Figure 10).
The Causal group performed the lowest on recall in both the written posttests and
the structured interviews for the Space Elevator system. This was not expected and an
interesting result. A possible explanation for the Causal group performance is that the
processing of the explicit functional relationships compete or interfere with the static,
causal relationships that were emphasized for this group. As the Causal group
participants were trying to fit the functional information into the causal models (Grotzer
& Baska,2003) to which they were biased, there may not have been enough time to
resolve potential contradictions incurring a negative "interference-type" effect on recall
performance in the short term.
94
Figure 10
Percentage Recall from the Structured Interview for the Space Elevator System
Space Elevator Structured Interview
25
10
1
•
1 Functional
• Causal
• Control
H
Elements
EHM
Roles
Mechanisms
Systems Understanding
The Functional group performed better overall on verbal recall from the
structured interviews for the Space Elevator. A possible explanation for this is that
explicit functional relationships provided better support for recognizing functional
relationships beyond the training and intervention activities providing a scaffolding
effect. This would be similar to the "causality-observation" effects that Barbas & Psillos
(2003) found in the use of their computer simulations to teach electrostatics where
knowing about functional relationships made them easier to identify in the reading and
process more deeply.
95
The means of written posttest recall were greater than the means of verbal
recall from the structured interviews. Functional relationships were associated with
difficulty and for this study, though not expected, this was more apparent when verbally
explaining a system then when having the time to think and write about them.
The additional exploratory analysis conducted to determine how the explicit
functional relationships affected recall performance offered some support for the benefit
of making these relationships explicit. The performance on the elements, roles and
mechanisms derived from these functional relationships was calculated and compared to
the performance on the remaining elements, roles and mechanisms in the Space Elevator
system. This analysis revealed that there was an overall greater recall for the roles related
to the explicit functional relationships included in the Space Elevator text in the written
posttest. Roles contribute to a deeper systems understanding than elements, therefore,
benefits are presumed and conceivably played a part in the significant overall recall of
elements for the Space Elevator found during the structured interviews. Only 3 of the 19
target roles were identified as being derived from the explicit functional relationships, so
future research to see if including more explicit functional relationships would yield a
greater percentage recall of roles (and subsequently, more mechanisms) would be
compelling.
Transfer Task Results
Groups did not perform as expected on the near and far transfer tasks, the what-if
problem solving and Mars city design tasks, respectively. Far transfer is a challenging
96
academic goal and achieving results are difficult (Jacobson, 2000), but some near
transfer differences between the groups were expected, even in the short instruction
periods of this study.
What-If Results
Two possible explanations are offered for the lack of findings in the what-if
problem solving performance. The first is the inconsistency in using the "Describe how
you figured this out" prompt with participants when posing the what-if question during
the interview. This was the last question in the protocol and participants may have been
fatigued. The additional prompt would have been necessary to draw out more involved
responses than the ones offered. When the prompt was used, this was found to be true as
in the case of Participant ID#81:
Researcher: Ok, so what do you think would happen to the greenhouse
effect if the Earth's reflective surface area were decreased?
Participant: Reflective surface area was decreased, um, then the
greenhouse effect would be more, um, um, it's it's more of a problem
basically.
Researcher: So could you describe how you figured this out?
Participant: Ok, um, well, I mean because reflecting the heat would
bounce, um, uh, I mean would basically decrease the temperature on the
ground, but if you, um, if there's actually a smaller area that can reflect the
heat, then much of it would come onto the Earth's ground so that would
basically raise the temperature.
After the "describe how" prompt, a deeper response was provided by the participant.
The second explanation for the what-if performance is that the specific questions
may have limited the problem-solving nature that they were intended to produce, i.e., the
what-if questions did not generate the responses as anticipated. The use of an open-
97
ended scenario would have allowed more expression of problem-solving processes to
better highlight group differences.
Mars City Design Results
One major concern about the Mars city design task was that the domain was
actually not far enough from the two domains studied during the intervention. This
became evident during the study as participants overwhelmingly included elements from
the greenhouse effect that shared the idea of temperature regulation, and elements from
the space elevator that shared the ideas of space transportation to transport goods and
services to Mars. When designing the task, the location on Mars was chosen to provide a
context for this activity similar to Tsuei (2004) so that the problem space was less
abstract, but perhaps a city design that was not intended for a particular location would
have had less influence on the outcomes that were sought from this task.
In spite of this limitation, the exploratory analysis that looked at the Mars city
transfer task performance based on the mutually exclusive use of either the functional
relations or the cause and effect strategy yielded some insight into the transfer effects of
these strategies. For the participants that studied the Greenhouse Effect system, those
that used the functional relations strategy produced Mars city designs that were of
significantly higher levels of complexities than those participants that used the cause and
effect strategy. However, there was no difference in the Mars city designs for the
participants that studied the Space Elevator. One theory to explain this result is that the
explicit functional relationships in the Space Elevator provided an exposure to all
participants assigned to this system the notion of dynamic relationships that had some
98
influence in their processing regardless of group. When this exposure was not offered
as in the Greenhouse Effect article, the specific strategy use (functional relations or cause
and effect) appears to have led to some differences in the amount of transfer to a new
task.
Limitations
The present study compared causal relationships with functional relationships,
which can also be considered complex causal relationships. Little research has been done
to compare these types of relationships to each other and the current findings contribute
in this sense. There are some limitations to this study beyond those already cited in this
chapter that would further clarify the nature and benefits of these types of relationships.
The first limitation is a matter of time. The length of the study was variable and took
participants an average of 75 minutes to complete. It is hard to determine if those that
took longer times to complete all the parts were able to study the materials longer, or if
the time led to fatigue and less attention while completing the activities, or if the systems
selected for the study were too complex to understand in a single session. This issue is
especially important to resolve as the transfer task activities took place at the end of the
study and the results for them were not as expected. By keeping the length of the study
sessions under an hour and the materials at an approachable level, the effects of time may
be minimized for future research.
The second limitation is also a matter of time. Functional relationships as
presented in the training materials were often a novel concept for participants. Because
they are also difficult to appreciate and put into practice, the opportunity to exercise
thinking functionally beyond a single study session may allow for a better assessment of
the potential effects they have on systems understanding. Following a teaching sequence
as Barbas & Psillos (2003) did in their study of electrostatics or a researcher-designed
unit as Tsuei (2004) did in her study of simulation diagrams would offer the opportunity
to see a more realistic characterization of the effects of thinking with functional and
causal relationships. The design of the present study appears to have improved the
understanding of roles in a system, but not enough support for affecting the
understanding of mechanisms.
Limitations from Materials and Measures
Another set of limitations come from the materials and measures designed for the
present study. Though pilot studies were conducted to inform the design of materials and
measures, testing on a larger sample highlighted areas were improvements could be
made:
Explicit Functional Relationships
Because the explicit use of functional relationships in the Space Elevator article
was of interest in this study, the intentional integration of more explicit functional
relationships into the article may have highlighted greater effects on system
understandings performance. In addition, an article with implicit functional relationships
for the Space Elevator would have provided a more direct comparison on the recall of
elements, roles and mechanisms than the present comparison with the Greenhouse Effect.
100
Future research on the explicit presentation of functional relationships in this manner
is warranted based on the findings from the exploratory analyses in this area.
Control Group Considerations
The Control group was included as a non-treatment comparison group. If the
activities of the Control group had been designed with those that mirrored the treatment
groups, for example, designing the intervention worksheet with elements from the article
grouped as pairs instead of a list and prompting them about additional strategy use during
the structured interview, additional comparisons from this group could have been
assessed. For example, while the Control group strategy use was different from the
treatment groups as expected, this may have also been due to the limits of the interview
questions seeking their strategy use, i.e., the Control group was asked only one main
question to provide this response while the Functional and Control groups were given
additional opportunities to respond due to the nature of their intervention task.6
In addition, it was expected that the Control group, though not prompted to do so,
would engage in basic causal reasoning and did not report doing so. Causality is an
important part of textual analysis and comprehension and has a central role in recall for
textual information (Black & Bern, 1981). Thus, causally related events are better
remembered than non-causally related events and events that occur on a causal chain are
considered by readers to be the most important part of a story and subsequently recalled
6
It was found during a pilot study that those in the Functional and Causal groups sometimes had difficulty
understanding the question regarding their strategy use and metacognitive thoughts. It was found that
being prompted by a general question followed by two specific questions was more effective in eliciting the
desired responses. This issue did not appear with the Control group and further prompts were not required
to elicit the desired responses so the Control group protocols were not modified.
101
better (Black & Bower, 1980). Analysis of the Control group activities such as the
summaries written during the intervention may have provided further insight into the
actual causal nature of their thinking and future studies should include other ways to elicit
and address this.
Holistic Measures of Systems Understanding
The use of the Structures-Behavior-Function coding scheme (interpreted in the
present study as Elements-Mechanisms-Roles) was successful in highlighting novice and
expert differences in previous studies (Hmelo-Silver, Marathe & Liu, 2007; HmeloSilver, Pfeffer, 2004). However, in the present study that had only novice participants,
this fine-grained coding scheme did not capture well the qualitative differences in
participants' systems understanding. For example, it was hard to quantify the difference
between participant recall of a single mechanism with few elements and roles against the
recall of many elements, roles and no mechanisms. A big limitation in analyzing the
results for the second hypothesis about systems understanding was the absence of a
holistic measure for system recall and should have been included as Hmelo-Silver,
Marathe & Liu (2007) did in their later work.
Another interesting holistic measure for future research would be a qualitative
assessment of the systems understanding responses. Were the causal explanations valid
and reflective of strong causal models? Similarly, were the functional relationships
indicative of dynamic thinking and deeper processing?
SCST
The SCST task was chosen because of the potential correlation between the
inductive processing involved in categorization and those used to create relationships
between objects of either a causal or functional type. Analysis of the SCST data will be
conducted in the future and discussion of the results will answer whether the SCST
showed such correlations or if, on the other hand, it posed a limitation by suppressing the
effects of the relationship training before the posttest because the categorization task was
too close in nature to the constructs being studied.
Implications
This study aims to inform educators about the potential of making the causal and
functional types of relationships in a system explicit for improving understanding of
physical systems. Because many domains of knowledge that one encounters are of a
complex nature, even though they may appear simple on the surface, the skills to
understand these systems are critical as learners develop and navigate in the real world.
Although systems are difficult to appreciate, researchers have shown that providing an
effective and explicit means of decomposing them does indeed help students improve
their understanding of them. Methods that help students discover the relationships of a
system appear particularly useful because relationships engage learners in thinking
beyond a system's elements allowing them to take on the holistic perspectives that are
necessary for their interpretation.
However, it is proposed that not all relationships are created equally. While
causal relationships may be the more natural and salient type of relationship for learners
to experience, an emphasis on functional relationships seems to be the more effective
means to promote a richer and dynamic view of a system. The representation of change
between elements embodied in functional relationships engage dynamic reasoning
strategies to better facilitate mental simulation. Functional relationships have received
limited attention in empirical studies and have not been compared to causal relationships
in earnest. The positive findings from the present study contribute in this sense and could
provide educators with a revised approach from which to present information,
specifically, to explicitly stimulate functional reasoning in addition to causal reasoning,
especially when designing materials on complex topics.
Why Study Functional Relationships?
In one answer to this question, the salience of causality is explored. Causality is
an intrinsic principle of our everyday reasoning. It is routine to seek out and think about
causal relationships in common occurrences, for example, to explain why a dog goes
through the trash, why the lack of coffee leads to irritability or why a baby cries in the
middle of the night. The notion of causality is also taught formally in schools through
science and math phenomena as well as in essay composition. The arguable issue with
causality is that it is too easy to perceive, or rather, the belief that it is easily perceivable.
As a result, when novice learners encounter new systems, they could be hasty or
restrictive in their exploration of these systems because causal relationships are sought
104
and when found functional relationships are probably overlooked.
In addition, findings that causal knowledge may be represented with a Bayesian
approach provide a glimpse into how people make inferences and form theories from new
and existing knowledge. For simple systems, learners are able to make choices between
variables to determine the causal relationships among them. Yet, the causal Bayes nets
break down when moving beyond simple systems to real-world phenomena (Chater,
Tenanbaum & Yuillie, 2006) because they are more difficult to represent by structured,
probabilistic models due to their dynamism and are presumed missing from typical
Bayesian accounts. This likely parallels the break down of systems understanding that
occurs with increasing complexity.
Functional relationships offer the complementary pieces of knowledge that are
critical for a complete portrayal of a real-world, complex system. One characteristic of a
functional relationship that is particularly difficult for learners to grasp is its nonlinear
nature. A well-known example of this can be found in the Butterfly Effect (Lorenz,
1972) where, in theory, the flap of a butterfly's wings in Brazil could set off a tornado in
Texas. An MIT meteorologist, Edward Lorenz coined this term to describe how
seemingly small, insignificant changes in a complex system can lead to unpredictable
results. Changes of this nature, i.e., nonlinear, are what allow the complexity to emerge.
To draw more attention to nonlinearity, it seems to follow that functional relationships
need to be acknowledged above and beyond the salient causal relationships towards
which novice learners may gravitate and perceive as linear.
105
Educational Implications
This study focused on the extremes of the causal continuum for the purposes of
investigation. However, in understanding the implications of this research it is important
to take these distinct constructs and place them back into the context of the continuum.
Causal relationships are fundamental to systems understanding and an arguably necessary
precursor to functional thinking. How can teachers be encouraged to engage students
more deeply in discussions where causal explanations are sought and scaffold them along
the continuum to more functional thinking? What is the progression of functional
explanations and how can the understanding gained from causal thinking be used to
benefit students through this progression?
With the renewed interest and attention to the STEM disciplines, not only is the
learning about complex systems important, an investment in the development and use of
technology to support STEM learning follows suit. Though this study relied on text to
convey system information, functional relationships represent dynamic phenomena and
taking advantage of the advanced state of current technology is likely the best
presentation vehicle for them. Technology enables the creation of simulations and
interactive media to support dynamic instruction about them. For example, "Flow
Blocks" (Zuckerman, Grotzer & Leahy, 2006) designed at the MIT Media Labs allow
young children to connect physical blocks together to experience different causal
structures, e.g., linear, circular. These physical blocks have embedded computation and
magnetic connectors to enable light signals to create causal chains of "moving light."
Consequently, one educational implication of this research is to promote the use of
technology such as flow blocks, simulation diagrams and direct-manipulation
animations to create instructional materials (Chan & Black, 2006; Tsuei, 2004) that make
functional relationships explicit and interactive providing more comprehensive support
for the development of complex systems mental models (Jacobson, 2000). For example,
the recent introduction of the iPad by Apple, Inc. (Apple, Inc., 2010), provides exciting
new opportunities to integrate interactive system simulations using the haptic channel
within electronic textbooks or other educational media for every day use in schools and
homes.
Another educational implication that is often overlooked is for educators to
consider the use of programming environments to allow learners to construct their own
representations of functional relationships. Many free, yet sophisticated programming
environments are available for student use including Scratch (MIT, 2010), Greenfoot
(Greenfoot, 2010) and Alice (Carnegie Mellon University, 2010). These tools were
designed to teach programming skills to novice programmers, enabling the creation of
simulations and animations without professional expertise. Lego Mindstorms (Lego,
2010) takes this experience a step further by providing the platform to create physical
robots that embody student designed programs, compelling them to make tangible the
relationships they want to learn about and study. Seeing and interacting with welldesigned technology tools, either virtually or physically, will help bring the awareness to
both causal and functional relationships that will foster a clarity through deeper reasoning
about complex systems.
107
REFERENCES
Apple, Inc. (2010). Retrieved from http://www.apple.com/ipad
Bar-Yam, Y. (2003). Dynamics of Complex Systems. Boulder, CO: Westview Press.
Barbas, A., & Psillos, D. (1997). Causal Reasoning as a Base for Advancing a Systemic
Approach to Simple Electrical Circuits. Research in Science Education, 27(3), 445-459.
Barbas, A., & Psillos, D. (2003). Evolution of Students' Reasoning about Microscopic Processes
in Electrostatics under the Influence of Interactive Simulations In D. Psillos & H.
Niedderer (Eds.), Teaching and learning in the science laboratory: Springer.
Besson, U. (2004). Some features of causal reasoning: common sense and physics teaching.
Research in Science & Technological Education, 22(1), 113-125.
Black, J. (2007). Imaginary Worlds. In M. A. Gluck & J. R. Anderson (Eds.), Memory and Mind:
A Festschrift for Gordon H. Bower. Mahwah, NJ: Lawrence Erlbaum Associates.
Black, J., Savich, H., Mandelman, S., & Youm, J. H. (2008). The Effect of Functional Relations
on Text Comprehension and Recall, Unpublished Study. Teachers College, Columbia
University.
Black, J. B. (1992). Types of Knowledge Representation, (CCT Report 92-3). New York, NY:
Teachers College, Columbia University.
Black, J.B. and Bern, H.S. (1981) Causal coherence and memory for events in narratives.
Journal of Verbal Learning and Verbal Behavior, 20, 267-275.
Black, J. B. and Bower, G.H. (1980) Story understanding as problem-solving. Poetics, 9, 223250.
Bullock, M , Gelman, R., & Baillargeon, R. (1982). The development of causal reasoning. In W.
108
J. Friedman (Ed.), The developmental psychology of time. New York:
Academic Press.
Carnegie Mellon University. (2010). Alice. Retrieved from http://www.alice.org
Chan, M. S., & Black, J. (2006). The role of presentation format on mental model acquisition. In
S. A. Barab, K. E. Hay & D. T. Hickey (Eds.), The Proceedings of the Seventh
International Conference of the Learning Sciences. Mahwah, NJ: Lawrence Erlbaum
Associates.
Chandrasekaran, B. (1993). Functional Representation: A Brief Historical Perspective. Applied
Artificial Intelligence, Special Issue on Functional Reasoning.
Chater, N., Tenenbaum, J. B., & Yuille, A. (2006). Probabilistic models of cognition: Conceptual
foundations. TRENDS in Cognitive Sciences, 10(7), 288-291.
Cheng, P. (1997). From Covariation to Causation: A Causal Power Theory. Psychological
Review, 104(2), 367-405.
Chi, M. T. H. (2000). Misunderstanding Emergent Processes as Causal. Paper presented at the
The Annual Meeting of the American Educational Research Association, New Orleans,
LA.
Chi, M. T. H., Bassok, M., Lewis, M., Reimann, P., & Glaser, R. (1989). Self-explanations: How
students study and use examples in learning to solve problems. Cognitive Science, 13,
145-182.
Cohen, L. B., & Oakes, L. M. (1993). How Infants Perceive a Simple Causal Event.
Developmental Psychology, 29(3), 421-433.
deKleer, J., & Brown, J. S. (1983). Assumptions and ambiguities in mechanistic mental models.
109
In D. Gentner & A. L. Stevens (Eds.), Mental Models (pp. 131-153).
Hillsdale, NJ: Lawrence Erlbaum Associates.
Dijkstra, E. (1979). Programming considered as a human activity. In E. Yourdon (Ed.), Classics
in Software. New York, NY: Yourdon Press.
Doerr, H. M. (1996). STELLA Ten Years Later: A Review of the Literature. International
Journal of Computers for Mathematical Learning, 1, 201-224.
Feltovich, P., Coulsen, R., Spiro, R., & Adami, J. (1992). Conceptual understanding and
stability and knowledge shieldfor fending off conceptual change. Springfield, IL:
Southern Illinois University School of Medicine.
Gelman, S. A., & Wellman, H. M. (1991). Insides and essence: Early understandings of the nonobvious. Cognition, 38, 213-244.
Gentner, D., & Stevens, A. L. (1983). Mental Models. Hillsdale, NJ: Lawrence Erlbaum
Associates.
Glymour, C , & Cheng, P. W. (1998). Causal Mechanism and Probability: A Normative
Approach. In M. Oaksfor & N. Chater (Eds.), Rational Models of Cognition: Oxford
University Press.
Glymour, C , & Danks, D. (2008). Reasons as Causes in Bayesian Epistemology. Journal of
Philosophy, 104(9), 464-474.
Goel, A., Garza, A. G. d. S., Grue, N., Murdock, J., Recker, M., & Govinderaj, T. (1996).
Towards designing learning environments: Exploring how devices work. In G. G. C.
Fraisson, & A. Lesgold (Ed.), Intelligent Tutoring Systems: Lecture notes in computer
science. NY: Springer.
110
Gopnik, A., & Glymour, C. (2002). Causal maps and Bayes nets: A cognitive and
computational account of theory-formation. In P. Carruthers, S. Stick & M. Siegal (Eds.),
The cognitive basis of science (pp. 117-133). Cambridge: Cambridge University Press.
Gopnik, A., & Nazzi, T. (2003). Words, kinds and causal powers: A theory theory perspective on
early naming and categorization. In D. Rakison & L. Oakes (Eds.), Early categorization.
Oxford: Oxford University Press.
Gopnik, A., & Schulz, L. E. (2004). Mechanisms of theory formation in young children.
TRENDS in Cognitive Sciences, S(8), 371-376.
Gopnik, A., & Sobel, D. M. (2000). Detecting blickets: How young children use information
about causal powers in categorization and induction. Child Development, 71, 1205-1222.
Greenfoot. (2010). Greenfoot - The Java Object World. Retrieved from http://www.greenfoot.org
Grotzer, T., & Bell, B. (1999). Negotiating the funnel: Guiding students toward understanding
elusive generative concepts. In L. Hetland & S. Veenema (Eds.), The Project Zero
classroom: Views on understanding: President and Fellows of Harvard College.
Grotzer, T., & Bell Basca, B. (2003). How does grasping the underlying causal structures of
ecosystems impact students' understanding? Journal of Biological Education, 38(1), 1629.
Grotzer, T., & Perkins, D. N. (2000). A Taxonomy of Causal Models: The Conceptual Leaps
Between Models and Students' Reflections on Them. Paper presented at the National
Association of Research in Science Teaching (NARST), New Orleans, LA.
Grotzer, T., & Sudbury, M. (2000). Moving Beyond Underlying Linear Causal Models of
Electrical Circuits. Paper presented at the National Association of Research in Science
Ill
Teaching (NARST), New Orleans, LA.
Hachey, A. (2005). An inquiry into the ontogeny of mental models and the etiology of
phenomenological inferencing. Unpublished Ph.D. Dissertation, Columbia University,
New York, NY.
Hachey, A., Tsuei, L., & Black, J. (2001). Computer-based Tools for the Development and
Investigation of Scientific Reasoning Skills: Fostering Mental-Model Thinking During
Design In C. Montgomerie & J. Vitelli (Eds.), Proceedings of World Conference on
Educational Multimedia, Hypermedia and Telecommunications. Finland: AACE.
Halbwachs, F. (1971). Reflexions sur la causalite physique. Causalite lineaire et causalite
circulaire en physique. In M. Bunge, F. Halbwachs, T. S. Kuh & J. Piaget (Eds.), Les
theories de la causalite. (Paris, PUF).
Heppner, F., & Grenander, U. (1990). A stochastic nonlinear model for coordinated bird flocks.
In S. Krasner (Ed.), The ubiquity of chaos (pp. 233-238). Washington, DC: AAAS
Publications.
Hmelo, C. E., Holton, D. L., & Kolodner, J. L. (2000). Designing to Learn About Complex
Systems. The Journal of the Learning Sciences, 9(3), 247-298.
Hmelo-Silver, C. E., & GreenPfeffer, M. (2004). Comparing expert and novice understanding of
a complex system from the perspective of structures, behaviors, and functions. Cognitive
science, 28, 127-138.
Hmelo-Silver, C. E., Marathe, S., & Liu, L. (2007). Fish Swim, Rocks Sit, and Lungs Breathe:
Expert-Novice Understanding of Complex Systems. The Journal of the Learning
Sciences, 16(3), 307-331.
112
Hume, D. (1739/1987). A Treatise of Human Nature (Second ed.). Oxford: Clarendon
Press.
Inagaki, K., & Hatano, G. (1993). Young children's understanding of the mind-body distinction.
Child Development, 64, 1534-1549.
isee systems, Inc. (2010). Stella, Systems Thinking for Education and Research. Retrieved from
http://^rww.iseesystems.corn/softwares/Education/StellaSoftware.aspx
Jacobson, M. J. (2000). Butterflies, traffic jams, and cheetahs: Problem solving and complex
systems. Paper presented at the The Annual Meeting of the American Educational
Research Association, New Orleans, LA.
Jacobson, M. J. (2001). Problem Solving, Cognition, and Complex Systems: Differences
between Experts and Novice. Complexity, 6(2), 1-9.
Jacobson, M. J., & Wilensky, U. (2006). Complex Systems in Education: Scientific and
Educational Importance and Implications for the Learning Sciences. The Journal of the
Learning Sciences, 15(1), 11-34.
Johnson-Laird, P. N. (1983). Mental Models. Cambridge, MA: Harvard University Press.
Jonassen, D. H., & Ionas, I. G. (2008). Designing effective supports for causal reasoning.
Educational Technolog Research and Development, 56(3), 287-308.
Kagan, J., Moss, H. A., & Sigel, I. E. (1963). Psychological Significance of Styles of
Conceptualization. Monographs of the Society for Research in Child Development. Basic
Cognitive Processes in Children: Report of the Second Conference Sponsored by the
Committee on Intellective Processes Research of the Social Science Research Council,
28(2), 73-112.
113
Kelley, H. H. (1967). Attribution theory in social psychology. In D. Levine (Ed.),
Nebraska Symposium on Motivation (Vol. 15, pp. 107-128). Lincoln: University of
Nebraska Press.
Kolodner, J. L., Crismond, D., Gray, J., Holbrook, J., & Puntambekar, S. (1998). Learning by
Design from Theory to Practice. In A. Bruckman, M. Guzdial, J. Kolodner & A. Ram
(Eds.), The Proceedings of the Third International Conference of the Learning Sciences
(pp. 16-22). Atlanta, GA: AACE.
Lego. (2010). Lego Mindstorms. Retrieved from http://mindstorms.lego.com/en-us/Default.aspx
Leslie, A. M. (1984). Spatiotemporal continuity and the perception of causality in infants.
Perception, 13, 287-305.
Lorenz, E. (1972). Predictability: Does the Flap of a Butterfly's Wings in Brazil Set off a
Tornado in Texas? Paper presented at the American Association for the Advancement of
Science.
Mandinach, E. (1989). Model-building and the use of computer simulation of dynamic systems.
Journal of Educational Computing Research, 5(2), 221-243.
Mandinach, E., & Cline, H. (1994). Classroom dynamics: Implementing a technology-based
learning environment. Hillsdale, NJ: Lawrence Erlbaum Associates.
Massachusetts Institute of Technology (MIT). (2010). Scratch. Retrieved from
http://scratch.mit.edu
MSNEncarta. (2008). How the Greenhouse Effect Works. Retrieved from
http://encarta.msn.eom/encyclopedia_761578504/Greenhouse_Effect.html#s2
National Research Council & National Committee on Science Education Standards and
114
Assessment. (1996). National Science Education Standards: The National
Academies Press.
Novick, L. R. (2008). Research on Causal Reasoning. Retrieved from
http://www. vanderbilt.edu/peabody/novick/causal.html
Novick, L. R., & Cheng, P. (2004). Assessing Interactive Causal Influence. Psychological
Review, 111(2), 455-485.
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Morgan Kaufmann.
Peat, F. D. (2002). From Certainty to Uncertainty: The Story of Science and Ideas in the TwentyFirst Century: Joseph Henry Press.
Perkins, D., & Grotzer, T. (2005). Dimensions of Causal Understanding: the Role of Complex
Causal Models in Students' Understanding of Science. Studies in Science Education, 41,
117-166.
Perkins, D. N. (1986). Knowledge as Design. Hillsdale, NJ: Lawrence Erlbaum Associates.
Piaget, J. (1930). The child's conception of physical causality. London: Routledge & Kegan Paul.
Resnick, M. (1996). Beyond the Centralized Mindset. The Journal of the Learning Sciences,
5(1), 1-22.
Schwartz, D. L., & Black, J. (1996). Shuttling Between Depictive Models and Abstract Rules:
Induction and Fallback. Cognitive Science, 20, 457-497.
Shultz, T. R. (1980). Rules of causal attribution. Monographs of the Society for Research in
Child Developmental Psychology, 47(1).
Simon, H. A. (1996). The Sciences of the Artificial (Third ed.). Cambridge, MA: The MIT Press.
Stevens, J. P. (2002). Applied Multivariate Statistics for the Social Sciences. Mahwah, NJ:
115
Lawrence Erlbaum Associates.
Sweeney, L. B. (2001). When a Butterfly Sneezes: A Guide for Helping Kids Explore
Interconnections in Our World Through Favorite Stories (Vol. 1). Waltham, MA:
Pegasus Communications, Inc.
Tenenbaum, J. B., Griffiths, T. L., & Kemp, C. (2006). Theory-based Bayesian models of
inductive learning and reasoning. TRENDS in Cognitive Sciences, 10(1), 309-318.
Tsuei, L. (2004). Using Simulation Diagrams to Facilitate Reasoning About Mechanisms and
Systems. Unpublished Ph.D. Dissertation, Columbia University, New York, NY.
Webb, M., & Hassell, D. (1988). Opportunities for computer based modelling and simulation in
secondary education. Paper presented at the Computers in Education, Amsterdam,
Netherlands.
Wellman, H. M., Hickling, A., & Schult, C. A. (1997). Young children's explanations:
Psychological, physical, and biological reasoning. In H. M. Wellman & K. Inagaki
(Eds.), New directions for child development: Children's theories (pp. 7-25). San
Francisco, CA: Jossey-Bass.
The White House (2010). Retrieved from http://www.whitehouse.gov/the-press-office/presidentobama-launches-educate-innovate-campaign-excellence-science-technologv-en.
Wilensky, U., & Resnick, M. (1999). Thinking in Levels: A Dynamic Systems Approach to
Making Sense of the World. Journal of Science Education and Technology, 5(1), 3-19.
Williams, M. D., Hollan, J. D., & Stevens, A. L. (1983). Human reasoning about a simple
physical system. In D. Gentner & A. L. Stevens (Eds.), Mental Models (pp. 131-153).
Hillsdale, NJ: Lawrence Erlbaum Associates.
116
Zuckerman, O., Grotzer, T., & Leahy, K. (2006). Flow Blocks as a Conceptual Bridge
Between Understanding the Structure and Behavior of a Complex Causal System. In S.
A. Barab, K. E. Hay & D. T. Hickey (Eds.), The Proceedings of the Seventh International
Conference of the Learning Sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
117
Appendix A: System Pretest and Posttest
The Greenhouse Effect
What do you know about the greenhouse effect?
How well do you feel you understand the greenhouse effect? (Circle One)
Not Well at all
Not Well
Average
Well
What is the role of the atmosphere in the greenhouse effect?
What is the role of the sun in the greenhouse effect?
Very Well
118
Appendix B: Causal Relationships Training
INTRODUCTION
Thank you for volunteering your time in this study. Your participation will help us learn about how
people can better understand a system. What do we mean by a system?
A system is a set of many interrelated parts that work together as a whole. If you think about it,
most things that we encounter daily are systems. For example, consider a computer. A computer
is made up of a keyboard, display, buttons, hard drive, processor and many other parts. Each part
has relationships t o other parts, and the parts and their relationships as a whole result in the overall
system, the computer. Other examples of systems include families (a social system), the stock
market (a financial system), cars (a mechanical system) and our body's immune system (a biological
system).
In this study, we will ask you to take a deeper focus on the concept of relationships. After all, it is
because parts are related that they belong in a particular system. W e hope that your attention to
these relationships will help you better learn about a new system.
I N T R O D U C T I O N T O RELATIONSHIPS
A relationship is a connection or association between t w o or more objects. There are different
types of relationships that exist in a system. W e ask you to explore one type of relationship here:
causal relationships.
CAUSAL RELATIONSHIPS
Let's say we have t w o objects and call them X and Y. A causal relationship exists between t w o
objects when any changes t o X also change Y. In other words, X is the cause of changes to Y, and
hence the term causal relationship. Another way to think about this relationship is in terms of cause
(changes in X ) and effect (changes in Y). This type of relationship can best be interpreted in an
"if/then" format "If X, then Y". The following are examples of causal relationships:
If photosynthesis occurs, then oxygen is released into the air.
If the people are unable t o vote, then the elections will not be fair.
N o w when you think about the parts in a system, for example, a computer system, think about the
causal relationships that you encounter.
•
•
•
If the power button is pressed, then the computer will turn on.
If the hard drive has a large capacity, then lots of data can be stored.
If there are many applications running, then the processor works harder.
For the remainder of this study, pay close attention t o causal relationships.
119
Appendix B: Functional Relationships Training
INTRODUCTION
Thank you for volunteering your time in this study. Your participation will help us learn about how
people can better understand a system. What do we mean by a system?
A system is a set of many interrelated parts that work together as a whole. If you think about it,
most things that we encounter daily are systems. For example, consider a computer. A computer
is made up of a keyboard, display, buttons, hard drive, processor and many other parts. Each part
has relationships t o other parts, and the parts and their relationships as a whole result in the overall
system, the computer. Other examples of systems include families (a social system), the stock
market (a financial system), cars (a mechanical system) and our body's immune system (a biological
system).
In this study, we will ask you t o take a deeper focus on the concept of relationships. After all, it is
because parts are related that they belong in a particular system. W e hope that your attention to
these relationships will help you better learn about a new system.
I N T R O D U C T I O N T O RELATIONSHIPS
A relationship is a connection or association between t w o or more objects. There are different
types of relationships that exist in a system. W e ask you to explore one type of relationship here:
functional relationships.
F U N C T I O N A L RELATIONSHIPS
A functional relationship describes how changes occur between t w o objects. To understand this
type of relationship, use as an analogy a simple algebraic equation that represents the relationship
between a circle's diameter, X, t o its radius, Y: X = 2Y
The function that describes the changes in X is 2Y When X changes, Y changes by a multiple of 2.
When Y changes, X changes by a multiple of ]li. Similarly, a functional relationship expresses the
details of how one object changes another object, like an algebraic function. The following are
examples of functional relationships:
The more it rains, the higher the water level in the lake, (function = amount of rain; water level is a
function of the amount of rain)
The more sugar that gets added t o the sauce, the sweeter it will be. (function = amount of sugar;
sweetness is a function of the amount of sugar added)
N o w when you think about the parts in a system, for example, a computer system, think about the
functional relationships that you encounter
•
•
•
The larger the hard drive, the more data that can be stored.
The more applications running, the harder the processor works.
The greater the resolution of a monitor, the clearer the display.
For the remainder of this study, pay close attention t o functional relationships.
120
Appendix C: Causal Relationship Intervention Worksheet for the Greenhouse Effect
FIND THE RELATIONSHIPS: GREENHOUSE EFFECT
Read the "How the Greenhouse Effect Works" article to complete this part of the study.
Based on the content in this article, create causal relationships between the following sets
of items. Feel free to refer to the article as often as you need. You will be asked about this
article later.
I. What is the causal relationship between: Sunlight and the Earth's atmosphere?
2. What is the causal relationship between: the Sun's energy and the Earth's surface?
3. What is the causal relationship between: Carbon dioxide and Infrared radiation?
4. What is the causal relationship between: Greenhouse gases and the Earth's
temperature?
121
Appendix C: Functional Relationship Intervention Worksheet for the Space Elevator
Find the Relationships: Space Elevator
Read the "The Space Elevator" article to complete this part of the study.
Based on the content in this article, create functional relationships between the following
sets of items. Feel free to refer to the article as often as you need. You will be asked about
this article later.
I.
What is the functional relationship between: the Carbon Composite Ribbon and the
Counterweight?
2. What is the functional relationship between: a Space Elevator and the Equator?
3. What is the functional relationship between: Developers and Safety?
4. What is the functional relationship between: Space Cargo and the Economy?
Appendix C: Control Group Intervention Worksheet for the Space Elevator
Find the Relationships: Space Elevator
Read the "The Space Elevator" article to complete this part of the study.
Based on the content in this article, take notes on the following items. Feel free to refer to
the article as often as you need. You will be asked about this article later.
1.
Carbon Composite Ribbon
2. Counterweight
3. Space Elevator
4. Equator
5. Developers
6. Safety
7. Space Cargo
8. Economy
Appendix D: Greenhouse Effect Causal Group Posttest Interview Protocol
INTERVIEW: GREENHOUSE EFFECT
1. Draw a picture to describe the greenhouse effect.
2. Describe your understanding of a causal relationship.
3. Describe your understanding of the greenhouse effect in as much detail as you
remember from the article you just read. (If participant has not already done so,
add this prompt.) Explain your drawing.
4. Let's look at the relationships that you came up with earlier from the greenhouse
effect article. What general steps or strategies did you use to come up these four
causal relationships?
5. Let's look at the relationship that you came up with between sunlight and the
Earth's atmosphere. What were the steps that helped you figure out this causal
relationship? What is the process that led you to this relationship? (Additional
prompts) How did you come up with this relationship? What strategies did you
use? How did you figure out the causal part?
6. Let's look at the relationship that you came up with between carbon dioxide and
infrared radiation. What were the steps that helped you figure out this causal
relationship? What is the process that led you to this relationship? (Additional
prompts) How did you come up with this relationship? What strategies did you
use? How did you figure out the causal part?
7. What do you think would happen to the greenhouse effect if the Earth's reflective
surface area were decreased? (Reminder prompt) Describe how you figured this
out.
124
Appendix D: Space Elevator Functional Group Posttest Interview Protocol
INTERVIEW: SPACE ELEVATOR
1.
Draw a picture to describe how the space elevator works.
2. Describe your understanding of a functional relationship.
3. Describe your understanding of how space elevators work in as much detail as you
remember from the article you just read. Feel free to use your drawing to help you
explain. (If participant has not already done so, add this prompt.) Explain your
drawing.
4. Let's look at the relationships that you came up with earlier from the space elevator
article. What general steps or strategies did you use to come up these four
functional relationships? What were you thinking about to help you come up with
the functions?
5. Let's look at the relationship that you came up with between a Space Elevator and
the Equator. What were the steps that helped you figure out this functional
relationship? How did you figure out the functional part? (Additional prompts)
What is the process that led you to this relationship? How did you come up with
this relationship? What strategies did you use?
6. Let's look at the relationship that you came up with between Space Cargo and the
Economy. What were the steps that helped you figure out this functional
relationship? What is the process that led you to this relationship? (Additional
prompts) How did you come up with this relationship? What strategies did you
use? How did you figure out the functional part?
7. What do you think would happen to the space elevator if it was anchored
somewhere other than the equator? (Reminder prompt) Describe how you figured
this out.
125
Appendix D: Greenhouse Effect Control Group Posttest Interview Protocol
INTERVIEW: GREENHOUSE EFFECT
1. Draw a picture to describe the greenhouse effect.
2.
Describe your understanding of the greenhouse effect in as much detail as you
remember from the article you just read. Feel free to use your drawing to help you
explain. (If participant has not already done so, add this prompt.) Explain your
drawing.
3. Let's look at the notes you took while reading this article. How did you decide
what notes to take? (Additional prompts) What strategies did you use?
4. What do you think would happen to the greenhouse effect if the Earth's reflective
surface area were decreased? (Reminder prompt) Describe how you figured this
out.
126
Appendix E: Demographic Questionnaire
Section I: General
Name:
Date:
Gender
Check One:
Male
|
[ Female
Age:
City, State/Country
of Residence
Occupation:
Education:
Major
Year in
School/Degree:
Name of University:
City, State/Country:
Section 2: Ethnicity
Please use the following numbers t o complete this question:
(1)
Asian, including Chinese, Japanese, Korean and others
(2)
Asian American
(3)
Black or African American
(4)
Hispanic or Latino, including Mexican American, Central American, and others
(5)
White, Caucasian, Anglo, European American; not Hispanic
(6)
American Indian/Native American
(7)
Mixed; Parents are from t w o different groups
(8)
Other (write in below):
Your Ethnicity:
SECTION 3: EXPERIENCE
1. How familiar were you with the how plants grow prior your participation in this study?
Not Familiar at all
Not Familiar
Average
Familiar
Very Familiar
O
O
O
O
O
2. How familiar were you with the digestive system prior your participation in this study?
Not Familiar at all
Not Familiar
Average
Familiar
Very Familiar
O
O
O
O
O
3. How familiar were you with the greenhouse effect prior your participation in this study?
Not Familiar at all
Not Familiar
Average
Familiar
Very Familiar
O
O
O
O
O
4. How familiar were you with thow hybrid cars work prior your participation in this study?
Not Familiar at all
Not Familiar
O
O
Section 4: Background
Average
O
Familiar
O
Very Familiar
O
I. List any psychology course you have taken:
2. List any prior experience with the topics presented in this study (greenhouse effect, space
elevator, Mars):
Appendix F: Greenhouse Effect Article
How the Greenhouse Effect Works
The greenhouse effect has warmed Earth for over 4 billion years. These atmospheric gases
have risen t o levels higher than at anytime in at least the last 650,000 years. As these gases build up
in the atmosphere, they trap more heat near Earth's surface, causing Earth's climate to become
warmer than it would naturally.
The greenhouse effect results from the interaction between sunlight and the layer of
greenhouse gases in the atmosphere that extends up to 100 km (60 mi) above Earth's surface.
Sunlight is composed of a range of radiant energies known as the solar spectrum, which includes
visible light, infrared light, gamma rays, X rays, and ultraviolet light. When the Sun's radiation reaches
Earth's atmosphere, some 25 percent of the energy is reflected back into space by clouds and other
atmospheric particles. About 20 percent is absorbed in the atmosphere. For instance, gas molecules
in the uppermost layers of the atmosphere absorb the Sun's gamma rays and X rays. The Sun's
ultraviolet radiation is absorbed by the ozone layer, located 19 t o 48 km (12 to 30 mi) above
Earth's surface.
About 50 percent of the Sun's energy, largely in the form of visible light, passes through the
atmosphere t o reach Earth's surface. Soils, plants, and oceans on Earth's surface absorb about 85
percent of this heat energy, while the rest is reflected back into the atmosphere—most effectively
by reflective surfaces such as snow, ice, and sandy deserts. In addition, some of the Sun's radiation
that is absorbed by Earth's surface becomes heat energy in the form of long-wave infrared radiation,
and this energy is released back into the atmosphere.
Certain gases in the atmosphere, including water vapor, carbon dioxide, methane, and
nitrous oxide, absorb this infrared radiant heat, temporarily preventing it from dispersing into space.
As these atmospheric gases warm, they in turn emit infrared radiation in all directions. Some of this
heat returns back t o Earth t o further warm the surface in what is known as the greenhouse effect,
and some of this heat is eventually released t o space. This heat transfer creates equilibrium between
the total amount of heat that reaches Earth from the Sun and the amount of heat that Earth
radiates out into space. This equilibrium or energy balance—the exchange of energy between
Earth's surface, atmosphere, and space—is important to maintain a climate that can support a wide
variety of life.
The heat-trapping gases in the atmosphere behave like the glass of a greenhouse. They let
much of the Sun's rays in, but keep most of that heat from directly escaping. Because of this, they
are called greenhouse gases. Without these gases, heat energy absorbed and reflected from Earth's
surface would easily radiate back out to space, leaving the planet with an inhospitable temperature
close t o - 1 9 ° C (2°F), instead of the present average surface temperature of 15°C (59°F).
To appreciate the importance of the greenhouse gases in creating a climate that helps
sustain most forms of life, compare Earth t o Mars and Venus. Mars has a thin atmosphere that
contains low concentrations of heat-trapping gases. As a result, Mars has a weak greenhouse effect
resulting in a largely frozen surface that shows no evidence of life. In contrast, Venus has an
atmosphere containing high concentrations of carbon dioxide. This heat-trapping gas prevents heat
radiated from the planet's surface from escaping into space, resulting in surface temperatures that
average 462°C (864°F)—too hot t o support life.
Appendix G: Space Elevator Article (Explicit Functional Relationships in Bold)
The Space Elevator
The space shuttle promised to carry things into space cheaply, but whether it is the
space shuttle or the non-reusable Russian spacecraft, the cost of a launch is approximately
$ 10,000 per pound. The space elevator is a new transportation system being developed
that could make travel to space a daily event and transform the global economy. A space
elevator made of a carbon composite ribbon anchored to an offshore ocean platform
would stretch to a small counterweight approximately 62,000 miles into space. Mechanical
lifters attached to the ribbon would climb the ribbon, carrying cargo and humans into
space, at only about $ 100 per pound.
To better understand the concept of a space elevator, think of the game tetherball
where a rope is attached at one end to a pole and at the other to a ball. In this analogy, the
rope is the carbon composite ribbon, the pole is the Earth and the ball is the
counterweight. Imagine the ball is placed in perpetual spin around the pole, fast enough to
keep the rope taut. With the space elevator, the counterweight spins around the Earth,
keeping the cable straight and allowing the robotic lifters to ride up and down the ribbon.
The lower the orbit of an Earth satellite the faster ft has to go. As one moves
toward the equator and the height of the orbit rises, a satellite can attain a stationary
position over a spot on the Earth. This is needed for a space elevator so its anchor will
need to be on the equator. The surface of the Earth spins faster as one moves toward the
equator and slower as one moves towards the poles. Thus, the energy to launch a satellite
into orbit becomes less as one moves toward the equator.
The robotic lifter will use the ribbon to guide its ascent into space. Rollers on the
lifter would clamp on to the ribbon and pull it through, enabling the lifter to climb up the
elevator. The space elevator will originate from a mobile platform in the equatorial Pacific,
to anchor the ribbon to Earth. At the top of the ribbon, there will be a heavy
counterweight that might be assembled from equipment used to build the ribbon including
the spacecraft used to launch it. The lifter will be powered by a laser system located on or
near the anchor. The laser will beam energy to photovoltaic cells attached to the lifter,
which will then convert that energy to electricity to power electric motors and carry cargo
ranging from satellites to humans.
The space elevator will be vulnerable to many dangers, including weather, space
debris and terrorists. Developers plan to build multiple space elevators. Each will be
cheaper than the previous and the more there are, the safer access to space will be. The
first space elevator will serve as a platform from which to build additional ones. Thus,
developers are ensuring that even if one space elevator encounters problems, others can
continue lifting payloads into space. Also, because the ribbon is anchored to a mobile
platform, the anchor can be moved to pull the ribbon out of the way of satellites and space
debris.
A space elevator would make trips to space more frequent and open up space to a
new era of development by significantly lowering the cost of putting cargo into space. The
more space elevators there are, the lower the cost of putting cargo into space and the
more the economy will be stimulated.
Appendix H: Training Examples Worksheet
List three examples of causal relationships found in everyday systems.
How familiar were you with causal relationships prior to this study? (Circle One)
Not Familiar at all
Not Familiar
Average
Familiar
Very Familiar
How well do you understand causal relationships now? (Circle One)
Not Well at all
Not Well
Average
Well
Very Well
131
Appendix I: Training Relationship Examples
Order
Item 1
Hern 2
Causal Relationship
1
Temperature
Pressure
If the temperature
increases (as
volume is held
constant), then
pressure increases.
2
Coffee
Sleep
3
Sun
Snow
If you drink coffee,
then it will affect
your sleep.
If the sun is out, the
snow will melt.
4
Exercise
Calories
If you exercise, then
you will burn
calories.
5
Supply
Demand
If supply goes up,
then demand goes
down.
6
Traffic
Commute
Time
If there is traffic,
then there will be a
long commute.
7
Rain
Water
Level of
Lake
If it rains, then the
lake will have water.
Functional
Relationship
When volume is
held constant,
pressure changes
proportionally with
temperature; as
temperature
increases, pressure
increases, and as
temperature
decreases, pressure
decreases.
The more coffee you
drink, the more your
sleep will be affected.
The greater the
warmth from the
sun, the faster the
snow will melt
The more exercise
you do, the more
calories you will
bum.
Supply is inversely
proportional t o
demand: as supply
goes up, demand
goes down, as supply
goes down, demand
goes up.
When there is more
traffic, longer
commute times will
result.
The more it rains,
the higher the water
level of the lake.
132
Appendix J: Transfer Task
DESIGN A LARGE CITY O N MARS
How would you design a large city on Mars to provide food, housing, goods, services, and
so on to your citizens so that there would be minimal shortages and surpluses?
Some facts about Mars:
• Fourth Planet from the Sun in our Solar System
• A terrestrial planet with a thin atmosphere
• Seasonal cycles are similar to that of the Earth
• Most likely planet to have liquid water besides Earth
133
Appendix K: Target Elements, Roles and Mechanisms
TABLE I. GREENHOUSE EFFECT SYSTEM TARGET LIST
Target Item
Notes
ELEMENTS
absorbant surfaces= soils, plants, oceans
atmosphere/air
carbon dioxide (C0 2 )
climate
Earth/Earth's surface
equilibrium/energy balance
gamma rays
gas molecules
glass
greenhouse
greenhouse gases/atmospheric gases, examples
include water vapor (H 2 0), carbon dioxide
(C0 2 ), methane (CH4), nitrous oxide (N 2 0)
ice
infrared radiation/infrared light/long-wave
infrared radiation
life
Mars
Methane (CH4)
nitrous oxide(NjO)
oceans
ozone layer
plants
If "absorbant surfaces" is mentioned, then assign a point for
this general Element.
If "absorbant surfaces" and examples of absorbant surfaces
are mentioned, and examples are identified correctly, then
assign a point for the general Element and a point for each
example.
If an example such as "plants" is mentioned, but not the
general Element "absorbant surfaces", then assign a point
for the Element representing the example only.
If "greenhouse gases" is mentioned, then assign a point for
this general Element.
If "greenhouse gases" and examples of greenhouse gases
are mentioned, and examples are identified correctly, then
assign a point for the general Element and a point for each
example.
If an example such as "carbon dioxide" is mentioned, but
not the general Element "greenhouse gases", then assign a
point for the Element representing the example only.
humans, people
Also accepting NO
sea
134
reflective surfaces= snow, ice, (sandy) deserts
(sandy) deserts
snow
soils
space
sun
sunlight/sun's radiation/heat/solar
spectrum/radiant energies/energy
temperature
If "reflective surfaces" is mentioned, then assign a point for
this general Element.
If "reflective surfaces" and examples of reflective surfaces
are mentioned, and examples are identified correctly, then
assign a point for the general Element and a point for each
example.
If an example such as "snow" is mentioned, but not the
general Element "reflective surfaces", then assign a point for
the Element representing the example only.
If the concept of warming or cooling is mentioned, then
assign a point.
ultravoilet light/ultravoilet radiation
Venus
visible light
water vapor
X rays
ROLES
any absorbant surfaces: The purpose of
absorbant surfaces is to absorb solar radiation
(85%) that reaches the surface of the Earth.
atmosphere: The purpose of the atmosphere is
to trap/absorb and emit solar radiation. The
atmosphere is made up of layers that absorb
different radiant energies.
clouds/atmospheric particles: The purpose of
clouds and atmospheric particles is to reflect
solar radiation (2596) back into space.
Earth/Earth's surface: The purpose of the
Earth's surface is to receive and emit solar
radiation.
equilibrium /energy balance: The purpose of
equilibrium is to maintain a climate that can
support a wide variety of life on Earth.
gas molecules: The purpose of gas molecules is
to absorb gamma rays and X rays.
glass of a greenhouse: The purpose of glass in a
greenhouse is to let the Sun's rays in, but keep
most of the heat from directly escaping.
greenhouse effect: The purpose of the
greenhouse effect is to warm the Earth (over
time).
If one of the absorbant surfaces is mentioned, then assign a
point for the "any absorbant surfaces" purpose and not the
"Earth/Earth's surface" purpose.
Exact percentages are not important to receive a point,
however, if percentages are cited, then they should be in
relative proportion to the correct percentage.
Exact percentages are not important to receive a point,
however, if percentages are cited, then they should be in
relative proportion to the correct percentage.
See notes for "any absorbant surfaces" and "any reflective
surfaces".
135
any greenhouse gases: The purpose of
greenhouse gases is to absorb heat/infrared
radiation.
Mars: The purpose of Mars is to provide an
example of a weak greenhouse effect.
ozone layer. The purpose of the ozone layer is
to absorb ultraviolet radiation.
any reflective surfaces: The purpose of reflective If one of the reflective surfaces is mentioned, then assign a
surfaces is to reflect radiation (15%) off the
point for the "any reflective surfaces" purpose and not the
surface of the Earth.
"Earth/Earth's surface" purpose.
Exact percentages are not important to receive a point,
however, if percentages are cited, then they should be in
relative proportion to the correct percentage.
sunlight/sun's radiation/heat/solar spectrum: The
purpose of sunlight is to provide heat energy.
Venus: The purpose of Venus is to provide an
example of a strong greenhouse effect.
MECHANISMS
equilibrium: The mechanism of equilibrium is
created through a balance between the total
amount of heat that reaches the Earth from
the Sun and the amount of heat that Earth
radiates out into space. It permits life to
flourish and creates stable climate patterns.
greenhouse effect: The mechanism of the
greenhouse effect results from the interaction
between sunlight and the layer of greenhouse
gases in the atmosphere; As these gases build
up in the atmosphere, they trap more heat
near Earth's surface, causing Earth's climate to
become warmer than it would naturally.
greenhouse effect on Mars: The mechanism of
the greenhouse effect on Mars is that the
atmosphere is too thin with low concentrations
of greenhouse gases to trap heat resulting in
mostly frozen surfaces and no evidence of life.
greenhouse effect on Venus: The mechanism of
the greenhouse effect on Venus is that the
atmosphere has high concentrations of
greenhouse gases (carbon dioxide) trapping
heat on its surface resulting in surface
temperatures too hot to support life.
greenhouse gases: The mechanism of
greenhouse gases is to absorb infrared
radiation temporarily to keep heat energy from
radiating back out to space and then emit them
out in all directions as the gases warm,
analogous to the glass of a greenhouse.
To receive a point, the participant should indicate some
interaction between heat from the Earth and Sun beyond
the Purpose defined for "equilibrium".
To receive a point, the participant should indicate some
interaction between gases in the atmosphere and the Earth
beyond the Purpose defined for "greenhouse effect".
To receive a point, the participant should indicate some
interaction between absorbing and emitting gases beyond
the Purpose defined for "greenhouse gases".
136
atmosphere (sunlight/sun's radiation/heat/solar Exact percentages are not important to receive a point,
spectrum and the atmosphere): The mechanism however, if percentages are cited, then they should be in
of sunlight and the atmosphere is that sunlight
relative proportion to the correct percentage.
is absorbed (about 20%) and emitted/reflected
(about 25%) by the Earth's atmosphere to
regulate Earth's surface temperature.
Earth's surface (sunlight/sun's radiation/heat/solarExact percentages are not important to receive a point,
spectrum and the Earth's surface): The
however, if percentages are cited, then they should be in
mechanism of sunlight and the Earth's surface is
relative proportion to the correct percentage.
that about 50% passes through the atmosphere
to the Earth's surface where about 85% is
absorbed by absorbant surfaces and about 15%
is reflected by reflective surfaces.
137
TABLE 2. SPACE ELEVATOR SYSTEM TARGET LIST
Target
Notes
ELEMENTS
ball
(carbon composite) ribbon
cargo
cost
counterweight
dangers
developers
Earth
Earth's (surface) rotation
economy
(electric) motors
energy (to launch a satellite)
equator
equipment (to build ribbon)
humans
laser (system)
(mechanical/robotic) lifter
(mobile offshore) platform/anchor
(non-reusable Russian) spacecraft/spacecraft
orbit (of satellite)
photovoltaic (PV)cells
pole
rollers
rope
(satellite) docking station
satellites
space
space debris
space shuttle
terrorists
tetherball
transportation system
weather
ROLES
ball: The purpose of ball is to provide an
analogy for the counterweight.
(carbon composite) ribbon: The purpose of the
carbon composite ribbon is to connect the
anchor on Earth to the counterweight in space.
counterweight: The purpose of the
counterweight is to perpetually spin fast
enough to keep the ribbon taut.
developers: The purpose of developers is to
plan and build space elevators.
Earth: The purpose of the Earth is to anchor
the ribbon like a pole in tetherball analogy.
If the concept of cheaper or more expensive is mentioned,
then assign a point.
risks
people
lifts
138
equator. The purpose of the equator is to
provide an optimal location for a satellite (to
attain a stationary position over a spot on the
Earth).
equipment (to build ribbon): The purpose of
equipment is to build the ribbon and for
potential use in assembling the counterweight.
laser (system): The purpose of the laser system
is to power the lifter.
(mechanical/robotic) lifter. The purpose of the
lifters is to carry things up and down the
ribbon.
(mobile offshore) platform/anchor. The purpose
of the mobile platform is to anchor the space
elevator to Earth and/or to pull the space
elevator out of the way of space debris and
satellites.
(electric) motors: The purpose of the motors is
to power the lifter.
Photovoltaic (PV) cells: The purpose of the
photovoltaic cells is to convert energy to
electricity.
pole: The purpose of a pole is to provide an
analogy for Earth.
rollers: The purpose of the rollers is to clamp
the lifter to the ribbon so the lifter can climb
up and down the ribbon.
rope: The purpose of rope is to provide an
analogy for the ribbon.
(satellite) docking station: The purpose of the
satellite docking station is to help build the
space elevator.
space elevator. The purpose of the space
elevator is to lower costs of carrying
cargo/travel to space.
(non-reusable Russian) spacecraft/spacecraft:
The purpose of the space shuttle and nonreusable Russian spacecrafts was to carry things
into space cheaply.
tetherball: The purpose of the tetherball game
is to serve as an analogy for the space elevator.
MECHANISMS
cost/economy. The mechanism of cost is that as To receive credit for this mechanism, the participant should
travel to space becomes more frequent (daily),
mention an interaction between the lowering of
then the cost of the space travel decreases, i.e.
costs/frequency of trips and stimulating the economy. This
$ 10,000/lb. vs. $ 100/lb. and the more the
is in contrast to the "space elevatori'purpose that only
economy will be stimulated.
mentions lowering costs.
Exact dollar amounts are not important to receive a point,
however, if dollar amounts are cited, then they should be in
relative proportion to the correct amounts.
139
Earth's (surface) rotation: The mechanism of the
Earth's surface rotation is that the Earth spins
fasters as one moves toward the equator and
slower as one moves towards the poles.
laser (system): The mechanism of the laser
system is that from a location on or near the
anchor, lasers beam energy to photovoltaic
cells attached to the lifter, which will then
convert that energy to electricity to power
electric motors and carry cargo.
multiple space elevators and danger. The
mechanism of multiple space elevators is that
the more space elevators there are, the safer
access to space will be and the less dangers
that will affect the functioning of the space
elevators. (The first space elevator will serve as
a platform from which to build additional ones.
Thus, developers are ensuring that even if one
space elevator encounters problems, others
can continue lifting payloads into space.)
multiple space elevators and cost: The
mechanism of multiple space elevators is the
cost decreases with the construction of
multiple space elevators leading to the
possibility of travel to space a daily event.
orbit (of satellite): The mechanism of a satellite's
orbit is that as one moves towards the
equator, the height of the orbit rises and since
the lower the orbit, the faster the satellite has
to go. It takes less energy for a satellite to
attain a stationary position over a spot on the
equator, and as a result, it is best for the
mobile platform should be anchored at the
equator.
space elevator. The mechanism of the space
elevator is analogous to the game of tetherball
where a carbon composite ribbon (rope) is
anchored to an offshore platform on Earth
(pole) and stretches (about 62,000 miles) to a
counterweight (ball) in space. Cargo and
humans are carried into space using mechanical
lifters that climb the ribbon.
140
Appendix L: Training Procedure for Systems Understanding Coding
PROCEDURE FOR PRETEST/POSTTEST SYSTEM SUMMARY QUESTION CODING
The pretest and posttest include an open-ended, free recall question in which the
participant is asked one of the following questions depending on the condition they have
been assigned:
"What do you know about the greenhouse effect?"
or
"What do you know about space elevators?"
These open-ended questions will be coded by identifying the Elements, Purpose and
Mechanisms that can be observed in participants' written responses where:
An element describes attributes, properties or parts of the system. Elements
answer the question "what is in the system?"
A role describes the purpose or role of an element in the system. Role answer the
question "why is something in the system?"
A mechanism describes the process or explains how elements work to achieve their
purpose. Mechanisms answer the question "how does something in the system
work?"
A target list of Elements, Role and Mechanisms for the greenhouse effect and space
elevator systems have been identified through a process of analysis and informal reliability
testing on a subset of the participant responses. These lists can be found in Table I and
Table 2. (Appendix K)
For each participant's response to the free recall question in the pretest and posttest,
determine which of the Elements, Role and Mechanisms from the respective target list are
present and record a point for each. When multiple terms have been listed and separated
by a "/", they are considered as acceptable synonyms for an Element. When terms are
presented in parentheses, they are considered optional to receive credit for an Element,
Role or Mechanism.
ASSIGNING POINTS
Element
• A point for an Element is assigned when the Element is stated literally or by a
synonym used in a context consistent with the contents of the article presented in
the study. For example, "ray" is acceptable for the Element "sunlight" if used in the
141
•
Role
•
context of the sun and "water" is acceptable for the Element "ocean" if used in
the context of a body of water on Earth.
A point for an Element can also be assigned if the Element is not stated literally, but
is obviously implied. For example, "cost" is implied when referring to things
becoming cheaper or more expensive and "temperature" is implied when referring
to the Earth getting warmer or colder.
To receive a point for Role, both the Element and the Role must be stated or
implied.
Mechanism
• A point for both Role and Mechanism can be assigned for the same Element. For
example, a point can be assigned to the Role of "equilibrium" and the Mechanism
for "equilibrium" if both are stated.
• A point for Mechanism should only be awarded if the participant has correctly
conveyed a level of detail or explanation beyond the Role indicating an interaction
between Elements of the system.
Appendix M: Training Procedure for Reasoning Strategies Coding
PROCEDURE FOR RELATIONSHIP STRATEGY INTERVIEW RESPONSE CODING
1.
Participant responses to the following question should be evaluated to see if any of
the above observations can be coded:
Causal and Functional Condition:
"Let's look at the relationships that you came up with earlier from the <space
elevator/greenhouse effect> article. What general steps or strategies did you use
to come up with these four <causal /functional> relationships?"
Control Condition:
"Let's look at the notes you took while reading this article. How did you decide
what notes to take? (Additional prompts) What strategies did you use?" in the
control condition.
2. For participants in the causal and functional conditions, responses to the two
questions that follow the first strategy question should also be included in the
coding process:
"Let's look at the relationship that you came up with between <object I > and
<object 2>. What were the steps that helped you figure out this <causal
/functional relationship? What is the process that led you to this relationship?
(Additional prompts) How did you come up with this relationship? What strategies
did you use? How did you figure out the <causal /functional part?"
3. Each participant response can be coded for multiple strategies. If there is an
apparent strategy that is not included in the list, code it "Other" and document the
strategy.
4. Note that participants were handed back their relationship worksheets for
reference during the interview. Code only participant responses that are beyond
reading or repeating content from the article or the relationships written on their
worksheets, i.e. limit to metacognitive thoughts.
NOTES
•
•
•
Strategy codes are not exclusively about strategies, but describe what the user was
thinking about or their process for coming up with their relationships or notes
One point (and only one point) should be given for each code that is identified
Coding should span across the responses to the general strategy question and
responses to the two specific follow-up questions
143
•
•
•
•
It is only the metacognitive thoughts that are stated during the interviews that
are the focus of this coding so correctness of the relationships or notes is not
important
Watch out for the ""cause" (i.e. cuz) vs. cause (i.e. cause and effect)
Treat transcriptions as literally as possible, i.e. do not make assumptions about their
strategies
Text that is in parentheses indicate non-verbal actions by the participant, for
example, laugh or pause, or verbal comments by the party that is not speaking, for
example, an "Ok" by the researcher when the participant is speaking
TABLE I. RELATIONSHIP STRATEGY CODES
CODE
AR
DEFINITION
EXAMPLE
Reading and searching the text of an
article for information such as
definitions, facts (including descriptive
information like percentages and other
numerical measures), key words/main
ideas
•
•
•
•
CE
FR
Thinking about one object causing
changes in another object in a linear,
one-way or sequential order, i.e. order
matters, like when following an "ifthen" format; if a functional
relationship is stated in an "if-then"
format but the emphasis is on
constructing an "if-then" statement
and not the nature of the relationship,
then the response should be coded as
CE
Thinking about dynamic, functional (or
the degree to which one object
changes when another changes)
•
•
•
•
"1 went back and reread and sort
of broke it down sentence by
sentence and made sure 1 was
understanding the steps that they
were taking to get to their
explanation..." (68)
"1 just thought about how it
worked and then 1 looked back in
the article..." (52)
".. .either 1 look for a definition or
something that, um, discussed that
topic" (71)
"...1 was mostly, um, looking for
like numbers and terms, something
that 1 wouldn't been able to
remember like right away." (65)
"1 try to identify which is the thing
that would be affected first and
then 1 described how the other
one affects it." (10)
"1 chose the first item and 1
thought how does this item, like
item "X", affect the second item
that was listed in bold..." (45)
".. .the fact that carbon dioxide
gets warm and then it emits
radiation, that was kind of the
function," (85)
".. .one thing increased the other
or, or decreased it or was the
same and tried to fit that to each
and/or mathematical relations such as
•
inverse, reciprocal or proportional
relationships; if a functional relationship
is stated in an "if-then" format but the
emphasis is on the nature of the
relationship, then the response should
be coded as FR
MS
Using a mental simulation or picture,
visualization or imagining
•
•
IN
Thinking about intermediary or
indirect relationships
•
•
DF
Having difficulty in deriving or finding
the relationship or information and
involving lots of effort and thought
possibly because relationship does not
exist or information is hard to
understand
•
•
one..." (19)
"...1 looked...how the
atmosphere can be changed by
sunlight or the opposite, it's
actually, 1 was thinking it doesn't
have to be that sunlight is to cause
the atmosphere it can be both
sides, so 1 was trying to think if
sunlight, if there is more sunlight
what would happen to
atmosphere or if atmosphere gets
thinner, what would happen to
sunlight..." (57)
"1 just imagined that process just
while, while 1 was reading the
article" (25)
"1 was trying to play out the
scenarios in my head of what a
change would look like..." (14)
".. .what was sort of difficult was
that sometimes it didn't appear
that the relationships were direct.
It seemed like there was
something else sort of in the
middle, or, or, that, that was in the
middle between the two objects
that 1 was supposed to evaluate
the relationship for" (86)
"...1 did have to go back to the
article sometimes and reread and
see what the different, um, like
mini-relationships are in between
there." (32)
"...then 1 had to look at like a
relationship between the two that
1 could create a cause and effect
from so it, it was a little bit more
actual thought process in it as
opposed to just rewriting basically
was on, in the article." (94)
".. .1 put a question mark by it
because 1 wasn't sure if there really
was a relationship because 1 don't
see one exactly impacting the
EZ
OM
Involving little strategy because
relationship or information was direct,
easy to find, required little thought or
taken straight from the article
•
Exploring the ability to manipulate or
change objects (of/in the relationship)
•
•
•
SY
Thinking about objects in relation to a
system or parts to a whole
•
•
OT
Other strategy (record these)
other." (56)
"...1 mean it was actually the
article that blatantly just said
that...." (31)
".. .the relationship between these
two things is very clearly shown,
shown in the, uh, the last sentence
of that, uh, article" (26)
"...1 was kind of like, well the
sunlight doesn't change, right, the
sun is not a, is not a, it's a constant
and the Earth's atmosphere, the
Earth's surface, they do not affect
the sun in any way, so how can
you even begin to describe this as
a functional relationship, um,
because you're basically just asking
like, like, it, how does a constant
thing affect something else that can
not possibly affect it, in turn..."
(13)
"1 can tell like they're more
sunlight, it's like, uh, the
atmosphere, um, the Earth's
atmosphere is going to absorb
more sunlight but actually there's
20%, uh, in terms of the amount,
might absorb more, but it doesn't
really, uh, absorb more than 20%
so 1 was a little bit confused about
this relationship." (25)
"... when 1 was prompted by a
word, 1 thought about the word in
relation to the whole system and
then 1 described how that feature
of the system functioned within in
the system." (53)
".. .it was supposed to be about
systems so 1 mean, uh, when 1
think systems, 1 think definitions
and, uh, their interactions with
each other." (54)
146
Appendix N: Training Procedure for What-If Coding
PROCEDURE FOR W H A T - I F CODING
The what-if task involves an open-ended question in which the participant is asked one of
the following questions:
Greenhouse System condition:
"What do you think would happen to the greenhouse effect if the Earth's reflective
surface area were decreased? (Reminder prompt) Describe how you figured this
out."
Space Elevator condition:
"What do you think would happen to the space elevator if it was anchored
somewhere other than the equator? (Reminder prompt) Describe how you figured
this out."
The response to this open-ended question will be coded on a scale of 0-3 where 0
is an incomplete response or one that is not complete enough to be evaluated, I is
a simple response and 3 is a complex response. A Medium (Level 2) response is
one that is average for the data set. Accordingly, a Low (Level I) response is one
that is below average and a High (Level 3) response is one that is above average.
TABLE I. RUBRIC A N D EXAMPLES
STATEMENT TYPE
Incomplete (Level 0)
• No response or incomplete
response
• Does not include any
relationships
Low (Level 1)
• Describes ONE
relationship, not including
those used in phrasing the
What-if Question, e.g.,
"decreased reflective
EXAMPLE
•
•
•
•
"Uh (long pause) so (long pause) so less reflected
energy would come up from the surface and, um,
they're less energy that, uh, green-effect house can
hold, uh, absorb, so less and less...I'm not sure" (25)
"Oh, um (long pause) but is that where the gravity is?
No? (laugh) You mean other then, anywhere else
besides the equator. But, in the article, it mentioned
how, that's (sigh)...oh god...urn (long pause). I'm
not sure." (82)
"More, more energy will be absorbed by the earth,
so it could become warmer (Relationship between
energ/ and earth). " (49)
"Eh, 1 think it will be more expensive to, 1 think as
much 1 understood it, 1 understood that it is possible
to do it, but it just will be, eh, more expensive to, to
147
surface area" or "moving
anchor from the equator"
•
Medium (Level 2)
• Describes T W O
relationships not including
those used in phrasing the
What-if Question
High (Level 3)
• Describes THREE or more
relationships not including
those used in phrasing the
What-if Question
• Or, mentions the concept
of equilibrium which alone
can qualify a response as a
High (e.g. regulating
temperature to maintain a
balance, keeping from
getting too hot or too cold
on Earth, orbit of Earth
faster at equator and
slower at the Poles)
activate it to do it. (Relationship between space
elevator and cost)" (16)
"If the Earth's reflective surface, sorry, reflective
surface was decreased, uh, 1 think there'll be too
much, too much heat trapped on the Earth's surface
so the Earth will become hotter (Relationship between
trapped heat and Earth)." (59)
•
"Hmmm, urn, it would probably, the greenhouse
effect would be more detrimental to the Earth then
because if the reflective surfaces decrease that means
the less, um, heat energy that will be reflected back
into space (Relationship between heat and reflective
surfaces) so pretty much all of it will be absorbed
instead of some of it being reflected back and then
the Earth will get hotter (Relationship between heat
aborption and Earth)." (85)
•
"Oh, it will require more energy to make it work.
Well the closer it is to the equator, it will require less
energy (Relationship between equator energy) and will
cost less (Relationship between equator and cost)."
(18)
•
"If the earth's reflective surface area were decreased,
wow, um. It's, it's.. .that's interesting.. .that's
interesting uh question because if the surface area
were decreased in order for it to balance out the
earth (Equilibrium concept) would have to absorb
more sunlight, um, so 1 could see that it would
(pause), 1 mean prob... probably the, the...the
green...greenhouse effect (laugh) (pause). Hmmm
(pause), 1 don't know one of two things that could
happen (laugh), either the... either the atmosphere
would, um, would...would continue to heat up and
we would become significantly warmer or there
were.. .there were the earth itself would have to
take most of.. .most of the brunt of these earth's
sunlight and, um, that particular, that.. .the.. .the earth
got to absorb more (sure)." (37)
148
•
•
"Ok, reflective surface area decreases, yes, uh-huh,
that means less, less amount of energy will be
radiated from the surface of the Earth (Relationship
between energy and Earth's surface) and that means
there'll be less amount of the, the, the infrared
radiation trapped by the greenhouse gases
(Relationship between infrared radiation and greenhouse
gases) even though we have greenhouse gases in the
atmosphere, but the amount of energy reflected back
from the Earth's surface will be less, therefore they
will less amount of, uh, infrared radiation which
means that the Earth will be much colder, the climate
will be much colder (Relationship between infrared
radiation and Earth's climate)." (I)
"Well, um, it would be more expensive to launch
the, uh, space elevator (Relationship between moving
space elevator and cost). Um, I'm sure that it must
also affect the physics of the space elevator. Maybe,
it, since it doesn't spin as fast, maybe it takes more
weight to hold up the ribbon, um, or less weight
maybe (Relationship between orbit at equator and
counterweight). So, you'd have to, you'd have to, it
would change the, uh, I think the, the logistics and the
dynamics of the, the physical properties of creating
the space elevator. Um, it may, if, if, if, uh, if, uh, if a
dock or a launch pad is created farther from the
equator, and it brings up the cost of creating this
system, then it might, the system might not even be
created at all. People would say well what's the
point in making the space elevator, uh, knowing that,
that we already have an alternative now. We already
have, um, space shuttles that are doing the job for
about the same amount of money. What's the point
even doing, creating this space elevator if it's going to
cost the same or more as the space shuttle, and it
probably, it probably wouldn't even be done at
al I... (Relationship between cost of space shuttle and
cost of space elevators)" (14)
Appendix O: Training Procedure for Transfer Task Coding
PROCEDURE FOR TRANSFER TASK CODING
The transfer task involves an open-ended question in which the participant is asked the
following question and given a 10-minute time limit to write their response:
"How would you design a large city on Mars to provide food, housing, goods,
services, and so on to your citizens so that there would be minimal shortages and
surpluses? Take ten minutes to design your city. Provide a written description and
on the second sheet, provide a drawing of your city."
The response to this open-ended question will be coded by identifying participants' written
responses as portraying one of four types of mental models: Incomplete, Low, Medium and
High (Table I). The mental models are considered to be increasing in complexity on a
scale of 0 to 3 and the overall responses should be evaluated against this scale.
REFERENCE
The following terms are used in the guidelines for defining each level and are presented
here for reference:
An Element describes attributes, properties or parts of the system. Elements
answer the question "what is in the system?"
A Role describes the purpose or role of an element in the system. Roles answer
the question "why is something in the system?"
A Mechanism describes the process or explains how elements work to achieve
their purpose. Mechanisms answer the question "how does something in the
system work?"
A Relationship is when two elements are related or linked together.
LEVELS OF COMPLEXITY SCALE
Level 0 - Incomplete
Guideline:
• A response with only Elements listed
• Too few Elements (e.g. one to three) with little or no description about them or their
Roles to determine a level of complexity.
150
•
The city design seems incomplete. However, if a participant was in the middle of a
sentence when the time-limit was reached, but you are able to judge a level of
complexity based on what was written prior to the cut off, then this is not an
Incomplete and should be coded based on what was written up to that point.
Example# I:
"Assume water exists! But not enough, therefore we need to supply food, goods from
earth. Gather all water to the center => water bank. The city is built under the protective
shield." (48)
Level I - Low Complexity
Guideline:
• Primarily made up of Elements, like a list and only loosely related through the purpose
of designing a city.
• Some Elements have basic Roles described.
• Though there may be a common theme presented to describe Elements, they are not
described in relation to each other and are represented independently.
• A Role and extra details are presented, but only to describe a single Element
Example# I: Participant describes a list of Elements and Roles in a list without an overarching
theme.
"Description:
• City will have a docking station (Element) for space elevators that will be linked to
Earth's docking station (Role for docking station).
• Elevators can be in use at any time to move food, supplies and goods (Elements)
between Earth and Mars (Role for Space Elevators).
• Station will have the capability to hold multiple elevators simultaneously.
• The city will have temperature controlled storage facilities (Element)
• City has a water treatment facility (Element).
• Citizens will be able to access food/goods/supplies at the storage facility
• The housing community will be within an enclosed temperature controlled area
(Element) so that the citizens can live within extreme temperatures (Role for
enclosed area)." (101)
Example#2: Participant provides only a basic list of Elements.
"put glasses (Element) between the mountains -> produce green house effect (Element);
find water and build up pipes to supply water (Element) to citizens; space stations (Element)
from an to earth; install 0 2 producer (Element) and air filters (Element)." ( I I )
Example#3: Participant lists Elements and Roles with the cold as a recurring theme, but does
not relate Elements to each other (e.g., houses, clothing, special heating, food are presented
independent of each other).
151
"The sun would not play a factor therefore more electricity (Element) would be need.
Houses (Element) would have to built in the canyons. Clothing (Element) would need to
be designed for warmth because of the cold (Role for clothes). Houses would have water
but would need special heating (Element) due to the extreme cold (Role for heating). Food
issues would need to be considered also because of the cold and transportation. Few
inhabitants would live on Mars." (84)
Level 2 - Medium Complexity
Guideline:
• Made up of several Elements and Roles
• Most Roles described in moderate detail and/or describing how the Roles can be
achieved
• Elements are presented in a meaningful sequence with some detail or by a common
theme and connected to each other with a few relationships between Elements.
(Note: If most of the Elements are related together, then assign the response a High
complexity.)
• A static or teleological view of a city is provided, i.e., things have a final purpose and no
fluidity in the city design is expressed.
Example# I: Participant provides a list of at least seven Elements, but only their basic roles are
described. If the Roles had been elaborated in further detail, then this response would have
been a High. If only a few Roles had been presented, then this response would have been a
Low.
"City of mars should have many energy-generating power plants (Element) because the city
will require excessive energy to heat (Role of power plants) the buildings & the houses
(Elements) in which people live (Role of buildings and houses). There should be stores
(Element) that provides (Role of stores) winter clothings (Element) & fat rich food supplies
(Element) to keep people warm (Role of winter clothing and fat rich food). The houses
should be with wooden materials (Element) rather than concrete so that the heat energy is
not lost easily (Role of wooden materials). People should excessively use fossil fuels
(Element) so that C 0 2 (Element) is released in the atmosphere (Role of fossil fuels). This
will help trap the radiations & heat the surface of mars to some extend [sic] (Role ofC02)."
(I)
Example#2: Though the participant only describes a few Elements (e.g., glass tubes, space
elevators, airport-like ports), Roles and detailed information are also described for these
Elements.
"People in Mars live all in glass tubes (Element). Those are huge tubes made of glass or
other transparent materials. They work as protectors of the Mars residents from the
extreme coldness outside the tubes (Role for glass tubes). They are all interconnected and
transparent so people don't have to get out of the places in any circumstances and don't
have to feel dark inside during the day time (More detail about role of glass tubes). The
connection btw the Earth (food, goods provider) and the Mars (Role for space elevator and
152
relationship from Mars to Earth) is the space elevator. (Element). Mars shall have
airport-like ports (Element) that receives food, goods and visitors that are carried from the
Earth via the space elevator (Role for ports and more detail on space elevator). Mars is not a
good place for planting but the Earth and the space elevators will do this job." (18)
Example#3: Participant describes basic Roles for most of the Elements.
1. Needs to be warmer.
2. Water (Element) can be used to absorb heat (Role of water)
3. Needs more greenhouse gases to radiate heat
4. Water for food = farming, drinking
This city would first need to be warmed to resemble the temperatures of Earth (Role of
replacing surfaces). We can do this by replacing reflective surfaces w/heat absorbing
surfaces (Elements). People could live underground in houses (Element) built under
canyons. Very industrial to release more C 0 2 (Element). We need to shoot greenhouse
gases (Element) into antmosphere [sic] to trap heat (Role of greenhouse gases/C02).
Eventually this should become more green to not become too hot (Concept of regulating
temperature, but not how to regulate). Farming communities (Element) live by water, like
ancient civilizations. Or if water is frozen, then they can live underneath the ground as
well." (21)
Level 3 - High Complexity
Guideline:
• Made up of Elements and a deeper elaboration of Roles (possibly even Mechanisms).
• Multiple relationships between Elements are described
• Equilibrium is considered to some degree and should make attempts to describe how
to achieve this beyond just that it is needed (e.g., how to regulate temperature vs. the
need for temperature regulation, how to provide for constant food supply vs. the need
for food). Equilibrium is involves a sense of balance (e.g., "growing enough food so that
everyone has some, but not so much that there is food wasted"). The description of
equilibrium alone qualifies a response as High.
• A systems perspective (of any type) is evident in the design where Elements are meant
to interact as a whole such as in an ecological (e.g., atmosphere, temperature and
climate relationships) or organizational and social sense (e.g., role of governing bodies
or role of people in city planning).
• A dynamic or changing/fluid view of the city is provided. Dynamic implies that the city
design is not final, i.e., that it could change, (e.g., "the amount of food necessary
depends on the number of people" where people and food are Elements that could
change overtime).
Example# I: Participant lists Elements and Roles that are related to each other, e.g., canals to
distribute water, housing near water, medical and social services near housing.
153
"I would put the city inside of a bubble (Element) made of material that would keep
oxygen and heat inside (Role of bubble). At the center of the city would be a large water
reservoir (Element) which would connect to canels [sic] (Element and relationship between
canals and water reservoir) to allow irrigation and other types of water use throughout the
city (Role of water reservoir). The city would be build into a canyon (Element) which would
also determine the city boundaries (Role of canyons). The housing (Element) would be
clustered near water (Relationship of housing to water) reserves and other services. There
would only be public transportation (Element) no individual would own cars, to cut down
on the need for gas stations (Role of public transportation). The transportation system
would be similar to the subway. Medical and social services (Element) would be clustered
in the housing areas (Relationship of services to housing areas)."
(8)
Example#2: Participant provides a detailed, meaningful description of Elements and accounts
for regulating temperature, an equilibrium concept.
"To establish a city on Mars with the hopes of sustaining life (Role of temperature), the
temperature (Element) of the planet would have to be dealt with since Mars has an
atmosphere, but is further away from the sun than earth is it is reasonable to increase the
levels of greenhouse gases artificially, as the natural process is much too slow, in order to
contain more of the sun's radiation in order to heat the surface of Mars (Relationship
between Mars and the sun, greenhouse gases and process of regulating temperature on Mars
surface). Once this feat is accomplished, soil (Element) should be tested to gauge how
sustainable crops (Element) will be on Mars. If needed, plantations (Element) from earth
can be transported to Mars in order to introduce plant life (Role of plantations). A system
of reservoirs (Element) will be stabled, using a supply of earth's water to sustain the crops
(Relationship between water and crops), until a source of liquid water is discovered. Mars'
canyons could provide the first step in reaching deep below Mars' surface to reach a water
source (More detail on water). As many cities on earth (Element) will be abandoned by the
time Mars is colonized, they can be transported to Mars, cities like Boise and the state of
Utah, in order to establish proper housing for Mars' first colony (Relationship between Mars
and Earth)." (19)
Example#3: Participant shows understanding of equilibrium for heat balance and 02 and C02
exchange.
"basic needs (List of Elements): power, air, shelter, water, food; build city under dome
(Element) or w/large airtight roof to keep air in (Role of dome); solar panels (Element) on
mountain for power (Role of solar panels); use greenhouse effect to regulate temperaturetransparent roof when under sunlight, for net gain of heat, then minimize heat loss by
sealing up when not in sunlight (Concept of heat balance and how to regulate temperature);
grow crops under domes heated same way; use power to condense oxygen out of
atmosphere-then reciprocal system betw (?), crops, 0 2 , C 0 2 (Concept of two-way system to
get 02 and COT); water (Element) biggest problem but can melt from snow if exists
snow/ice; need close to 100% recycling if possible; advantage: can vent waste C 0 2 into
atmosphere to begin terrafarming!! (Relationship between C02 and farming)" (13)
154
Example#4: Participant describes at least six Elements and details about Roles and
relationships for most of them.
"The mars city would be covered by a clear 'biodome' (Element) to maintain an earth-like
climate (Role of biodome). The climate would be created by a large lens (Element) on the
dome focusing sunlight onto a large body of collected water. By a (magical) balance of
holding and releasing water an earth-like climate would be produced (How role of lens will
be achieved). The hot water (Relationship between water and lens) will also power an
electric generator (Element) and provide heat to the buildings (Role of generator). A space
elevator (Element) external to the dome will provide the link between Earth (Element) and
Mars for trade (Role of space elevator and relationship between Mars to Earth). Initially it will
be a agricultural economy de to the novelty of extra terrestrial food (Element)." (10)
Distinguishing between Levels of Complexity
The open-ended nature of the Transfer task responses may make distinguishing between
two levels of complexity difficult at times. These guidelines are provided to help make a
decision between two levels:
•
•
Low responses will generally have minimal detail about the Elements they describe.
However, if a response contains some information about the Roles for Elements (e.g.,
[glass tubes] work as protectors of the Mars residents from the extreme coldness) or
includes extra details about some of the Elements (e.g., [glass tubes] are all
interconnected and transparent so people don't have to get out of the places in any
circumstances and don't have to feel dark inside during the day time), then consider the
ratio of Elements with Roles/information to those without. If there are more Elements
without Roles/extra details, then assign a Low. If there are more Elements with
Roles/extra details, then assign a Medium.
If there is a common theme that provides a context for the Elements in the response
(e.g., to provide heat or build a bubble city) and this theme is not used to relate or link
the Elements together (e.g., power plant to generate heat, clothes to stay warm,
housing with heaters), then assign a Low. If a common theme is used to relate or link
some, but not a majority, of the Elements together (e.g., power plant to generate
electricity, electricity used for heaters in the houses) or there is extra detail or
information about the Elements, then assign a Medium. If a common theme is used to
relate or link a majority of the Elements together, then assign a High. However, a High,
does not require a common theme; if there are many relationships between the
Elements (regardless of a common theme or not), then it should be a High. For
example, Example# I for High Complexity has relationships (without a theme) between
canals and water reservoir, water and housing and medical and social services to
housing. In contrast, if you look at Example# I for Low Complexity, the theme is "the
cold" and if there had been more statements like "housing requires special heaters to
protect them from the cold. The special heaters need electricity because the sun does
155
•
not play a factor on Mars which makes it cold." it would have been a High. A
Medium response might have been one that described how the cold affected designing
housing, clothes and food in more details (e.g., "Clothing would need to be designed for
warmth. They could invent a lightweight down material so it's warm but not bulky. To
keep the food warm there would powerful microwave ovens that could heat lots of
food quickly.") though not necessarily relating them together.
Responses that show any evidence of systems thinking, regardless of the number of
Elements and relationships between Elements, where the city represents a system or
any part of the city can be represented as system, should be assigned a High. Systems
thinking can be characterized by the following types of reasoning (Jacobson, 2000):
1. Having a sense that the whole-is-greater-than-the-parts
2. Seeing de-centralized control, i.e., control emerges from interactions in the
system not a single source
3. Believing there are multiple causes for effects in the system
4. Realizing small actions in the system can have big effects
5. Understanding that the actions or nature of the parts of the system are not
completely predictable
6. Having a non-teleological view (causes are not final) of phenomena
7. Recognizing equilibrium processes
Документ
Категория
Без категории
Просмотров
0
Размер файла
6 608 Кб
Теги
sdewsdweddes
1/--страниц
Пожаловаться на содержимое документа