close

Вход

Забыли?

вход по аккаунту

?

Evaluation of Multimodal Input for Entering Mathematical Equations

код для вставкиСкачать
Evaluation of Multimodal Input
for Entering Mathematical
Equations on the Computer
Lisa Anthony,
Jie Yang, Kenneth R. Koedinger
Human-Computer Interaction Institute
Carnegie Mellon University, Pittsburgh, PA
Computer-Based Math Tools
Maple V, Mathematica, Matlab;
Microsoft Equation Editor, …
Programming constructs
or special syntax
Linearization of mental
representation
MATHEMATICA
Difficult to revise structure
in template-based editors
Human-Computer Interaction Institute
MICROSOFT
EQUATION EDITOR
Anthony, Yang, Koedinger
Computer-Based Math Tools
Maple V, Mathematica, Matlab;
Microsoft Equation Editor, …
Problems:
в–єLarge
learning curve
в–єTied to keyboard and
mouse input
MATHEMATICA
MICROSOFT
EQUATION EDITOR
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Pen-Based Computing
Notations already exist for
paper-based math
Affordance for spatial
representation
Especially good for
students learning math
Problem: recognition
accuracy
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Solution: Multimodal Input
Increased robustness
в–єBetter
recognition accuracy with multiple input
streams (Oviatt, 1999; pen gestures + speech)
Consider both repair-only and simultaneous
input in both streams
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Motivation for User Study
Literature says typing is faster (Brown, 1988)
в–єCompared
inputting paragraphs of English text
в–єMath is different domain
Little evaluation done of pen-based equation
input
в–єSystems
constrained by recognition accuracy
(Smithies et al, 2001)
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Design
Input method
Keyboard-and-mouse
Used keyboard and mouse with Microsoft
Equation Editor (MSEE).
Handwriting-only
Wrote in Windows Journal program;
without recognition.
Speech-only
Dictated into microphone; without
recognition or visual feedback.
Handwriting-plusspeech
Both spoke and wrote in series or in
parallel (user choice); without recognition.
Equation complexity
“Recognition” means system tries to
interpret user input.
в–є Number
of characters in equation
► Number of “complex” symbols (e.g., √ and ∑)
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Participants
48 paid participants (27 male, 21 female)
Undergraduate/graduate, full-time/part-time
students at Carnegie Mellon
Native English speakers only
Most (33 of 48) had no experience with MSEE
before study
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Procedure
Entered math equations on TabletPC
в–є36
equations (7 + 2 practice per condition)
в–єConditions counterbalanced across participants
Instructions for each condition
в–єNo
prompting for specific ways of expressing
equations
►5 min “explore time” for MSEE
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Measures
Time per equation
Number of errors per
equation (corrected and
uncorrected)
User preferences before
and after session
Equation entry screen in
handwriting condition.
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Sample Stimuli
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Results
50
45
T im e (seco n d s)
40
35
30
25
20
15
10
5
0
K eyboard-
Handw riting
and-m ous e
only
S peec h only
Handw ritingplus -s peec h
C o n d itio n
Average time in seconds per
equation by condition. Error bars
show 95% confidence interval (CI).
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
User Study: Results
N u m b er o f E rro rs
2.5
2
1.5
1
0.5
0
K eyboard-
Handw riting
and-m ous e
only
S peec h only
Handw ritingplus -s peec h
C o n d itio n
Mean number of user errors made
per equation by condition. Error bars
show 95% CI.
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
Score (1 to 5)
Score (1 to 5)
User Study: Results
Keyboard-andmouse
Handwriting only
Speech only
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
Keyboardand-mouse
Handwriting
only
Speech only Handwritingplus-speech
Condition
Condition
Pre-test
Post-test
Preference questionnaire rankings of each condition on a
5-point Likert scale. Error bars show 95% CI.
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Conclusions
Handwriting faster, more efficient, and more
enjoyable to novice users than standard
keyboard-and-mouse
Handwriting-plus-speech faster and better
liked than keyboard-and-mouse
Handwriting-plus-speech not much worse
than handwriting alone, so multimodal
may be a winner for technology reasons
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Further Analyses
Transcription of spoken input as corpus for
generation of language model
Consistency across and within users in handwriting
and speech
в–є Ambiguity
resolution
в–є Self correction
в–є Pausing and synchronization in multimodal input
Greater variability within speech condition than
within handwriting condition
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Further Analyses
Transcription of spoken input as corpus for
generation of language model
Consistency across and within users in handwriting
and speech
в–є Ambiguity
resolution
в–є Self correction
в–є Pausing and synchronization in multimodal input
Greater variability within speech condition than
within handwriting condition
(Supported by Oviatt et al, 2005)
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Questions?
Project Webpage:
http://www.cs.cmu.edu/~lanthony/research/multimodal/
Pittsburgh Science of Learning Center:
http://www.learnlab.org/index.php
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
References
Brown, C.M.L.: Comparison of Typing and Handwriting in “Two-Finger
Typists.” Proceedings of the Human Factors Society (1988) 381–
385.
Oviatt, S.: Mutual Disambiguation of Recognition Errors in a Multimodal
Architecture. Proceedings of the CHI Conference (1999) 576–583.
Smithies, S., Novins, K., and Arvo, J.: Equation Entry and Editing via
Handwriting and Gesture Recognition. Behaviour and Information
Technology 20 (2001) 53–67.
Hausmann, R.G.M. and Chi, M.T.H.: Can a Computer Interface Support
Self-explaining? Cognitive Technology 7 (2002) 4–14.
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Other References in Paper
Anderson, J.R., Corbett, A.T., Koedinger, K.R., and Pelletier, R.: Cognitive
Tutors: Lessons Learned. The Journal of the Learning Sciences 4
(1995) 167–207.
Blostein, D. and Grbavec, A.: Recognition of Mathematical Notation. In
Handbook on Optical Character Recognition and Document
Analysis, Wang, P.S.P. and Bunke, H. (eds) (1996) 557–582.
Locke, J.L. and Fehr, F.S.: Subvocalization of Heard or Seen Words Prior
to Spoken or Written Recall. American Journal of Psychology 85
(1972) 63–68.
Microsoft.: Microsoft Word User’s Guide Version 6.0 (1993), Microsoft
Press.
Sweller, J.: Cognitive Load During Problem Solving: Effects on Learning.
Cognitive Science 12 (1988) 257–285.
Human-Computer Interaction Institute
Anthony, Yang, Koedinger
Документ
Категория
Презентации
Просмотров
8
Размер файла
900 Кб
Теги
1/--страниц
Пожаловаться на содержимое документа