close

Вход

Забыли?

вход по аккаунту

?

Chris Beauchamp ASI

код для вставки
How to Prepare Students Who Must Rewrite the
CPNRE: Evidence-based Best Practice
Chris Beauchamp, Senior Consultant
Canadian Association of Practical Nurse Educators
2 October 2012
1400 Blair Place, Suite 210, Ottawa, ON K1J 9B8 Canada Tel: 613-237-0241 • Fax: 613-237-6684 • www.asinc.ca
1400, place Blair, bureau 210, Ottawa (Ont.) K1J 9B8 Canada Tél. : 613-237-0241 • Téléc. : 613-237-6684 • www.asinc.ca
Outline
•
•
•
•
•
About ASI
Student resources
Study on repeat candidates
Implications
Acknowledgments
2
В© 2012 Assessment Strategies Inc.
Who is
Assessment Strategies Inc.
ASSESSMENT STRATEGIES INC.
A private, federally chartered corporation with more
than 40 years experience
Works with the provincial/territorial regulatory
authorities to develop and administer the CPNRE
3
В© 2012 Assessment Strategies Inc.
Student Resources
Web resources
• CPNRE website www.cpnre.ca
• Information on exam development process,
including contact information and FAQs
• Twitter @cpnre
• Information on the exam
• Timelines and deadlines
• Important announcements
5
В© 2012 Assessment Strategies Inc.
The Canadian PN Exam Prep Guide
• Fourth edition released in September 2011
• Retails for approximately $84.95 + taxes
• Available at most bookstores or through St. Joseph’s
Communications (1-888-562-5561)
6
В© 2012 Assessment Strategies Inc.
CPNRE Predictor Test
• Two equivalent tests of 100 questions each
• Students can purchase this product by going to the
website (www.cpnre.ca)
• Predictor Test I and Predictor Test II have different sets
of questions, but are similar in difficulty
• Once candidates have completed the test and submitted
their answers, they will be presented with their results.
Candidates can also review their questions
• In a recent study, the Predictor Test accurately predicted
performance on the CPNRE 89% of the time
7
В© 2012 Assessment Strategies Inc.
CPNRE Classroom Assessment
• Same methodology as the Predictor Test, except:
• Is geared more toward assessment rather than being a
learning tool
• Can only be purchased by educators for in-class
purposes
• Price ranges by the number of units purchased (from
$44.95 to $49.95, or lower for larger purchases)
• Must be administered in a secure setting (computer
lab, classroom, etc…).
• Students cannot review their questions
• Different questions than the Predictor Test, but similar
in style and difficulty
8
В© 2012 Assessment Strategies Inc.
Study findings:
Repeat candidate performance
Background
• The CPNRE is administered three times per year
(January, May and September)
• Candidates are typically given three opportunities to
pass the CPNRE
• Some provinces require remediation before a
candidate writes for the third time
Pass rates for Canadian-educated candidates
First time writers: 87%
First time repeater writers: 62%
Second time repeater writers: 62%
10
В© 2012 Assessment Strategies Inc.
Previous research
• Most previous research has shown score
gains for repeat writers (effect sizes
vary)
• Four possible explanations:
1. Real improvements in abilities
2. Changes in ability-irrelevant factors (e.g.,
test taking skills, study skills)
3. Changes in psychometric properties (e.g.,
candidate remembers questions)
4. Regression to the mean
11
В© 2012 Assessment Strategies Inc.
Methodology
• In 2009, the same form of the CPNRE was used for
all three administrations (last time this has occurred)
• The exam had 224 multiple-choice questions
Exam form characteristics
Competency areas
1. Professional, ethical and legal practice (65 questions)
2. Foundations of practice (113 questions)
3. Collaborative practice (46 questions)
Taxonomy
1. Knowledge/comprehension (35 questions)
2. Application (136 questions)
3. Critical thinking (53 questions)
12
В© 2012 Assessment Strategies Inc.
Hypotheses
1. First time writers score higher than
repeat writers
2. Retaking the CPNRE will lead to score
gains
3. Candidates who rewrite after an 8
month delay will outperform
candidates who rewrite after a 4 month
delay
4. Score gains vary across different
cognitive ability and competency levels
13
В© 2012 Assessment Strategies Inc.
Sample
Table 1. Candidate performance on the CPNRE in 2009
n
1st time
writers
Repeat
writers
Average
score
January 2009
1668
85.5%
14.5%
158.4
May 2009
1891
83.7%
16.3%
160.5
September 2009
2767
87.2%
12.8%
159.7
Total
6326
84.5%
14.5%
159.5
Administration
14
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #1
First time writers score higher than repeat writers
Table 2. Performance on the CPNRE (three admins combined; all
candidates)
Taxonomy
K/C
App
Competency category Total
CT
PEL
FP
CP
1st time
72.5% 74.1%
67.7%
74.7%
69.6%
75.3%
72.3%
Repeat
65.9% 66.8%
60.8%
67.7%
62.7%
67.2%
65.0%
Diff.
6.1%
7.7%
7.2%
6.9%
8.1%
7.3%
7.3%
15
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #1
First time writers score higher than repeat writers
Overall trends – all candidates
• Critical thinking questions were the most challenging.
Application questions were the least challenging
• Foundations of practice questions were the most challenging.
Collaborative practice questions were the least challenging
Differences between first time and repeat writers
• Overall, first time writers outperformed repeat writers
(statistically significant across all categories)
• Repeat writers struggled the most with critical thinking and
collaborative practice questions
Therefore: Hypothesis #1 is supported
16
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #2
Retaking the CPNRE will lead to score gains
1-time repeaters
First writing
Second writing
Score gain
60.4%
66.4%
6.0%
2-time repeaters
Second writing
Third writing
Score gain
59.6%
64.4%
4.8%
Note: All score gains are statistically significant
17
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #2
Retaking the CPNRE will lead to score gains
• Retaking the CPNRE leads to modest score
gains
• Score gains occur for 1-time repeaters and 2time repeaters
Therefore: Hypothesis #2 is supported
18
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #3
Candidates who rewrite after an 8 month delay will
outperform candidates who rewrite after a 4
month delay
• Performance of candidates who rewrote after a 4month delay were compared to candidates who
rewrote after an 8-month delay
Effect of time delay
4 month delay – 6.0% score gain
8 month delay – 5.7% score gain
* Difference is statistically significant
Therefore: Delaying the time to rewrite does not
improve performance and may hinder performance
19
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #4
Score gains vary across different cognitive ability
and competency levels
Table 3. Performance of repeat writers on the CPNRE
First attempt
Second attempt
Score gain
Knowledge
59.8%
67.9%
8.9%
Application
62.2%
68.0%
5.8%
Critical thinking
56.5%
61.2%
4.7%
Prof, ethical, legal
63.7%
68.8%
5.1%
Foundations
57.5%
63.9%
6.4%
Collaborative
63.8%
68.9%
5.1%
Taxonomy
Competency
20
В© 2012 Assessment Strategies Inc.
Results – Hypothesis #4
Score gains vary across different cognitive ability
and competency levels
• By taxonomy, the biggest score gains were in the
knowledge/comprehension taxonomy. The smallest
score gains were in the critical thinking taxonomy
• By competency category, the largest score gains
were in the foundations of practice category
Therefore: Score gains vary across different cognitive
ability and competency levels
21
В© 2012 Assessment Strategies Inc.
Implications
Summary
• Candidate performance was assessed on the 2009
CPNRE (administered three times)
Hypothesis
First time writers score higher than repeat
writers
Retaking the CPNRE will lead to score
gains
Candidates who rewrite after an 8 month
delay will outperform candidates who
rewrite after a 4 month delay
Score gains vary across different
cognitive ability and competency levels
23
Outcome
пѓј
пѓј
x
пѓј
В© 2012 Assessment Strategies Inc.
Implications
For all candidates
• Critical thinking questions were the most challenging while
application questions were the least challenging
• Foundations of practice questions were the most challenging
while collaborative practice questions were the least
challenging
Implications
• All candidates should pay special attention to critical
thinking questions
• Read the question thoroughly
• Focus on key words (e.g., “first”, “best”, “most appropriate”).
• Consider all options rather than selecting the first correct option
• Only a small portion of the CPNRE is based on
knowledge/comprehension questions
• Important not to waste time memorizing lists of information
• More important to understand how the facts can be applied
24
В© 2012 Assessment Strategies Inc.
Implications
For candidates who do not pass the CPNRE
• 87% of candidates pass on their first attempt
• The average score gain for candidates who rewrite the
CPNRE is 6.0 to 6.8%
• Most of these score gains are for knowledge/comprehension
questions and foundations of practice questions
Implications
• On average, candidates can expect score gains upon rewrite
• Data suggest that candidates who rewrite likely spend time
memorizing content
• The biggest opportunity for improvement is not in
knowledge/comprehension questions
• Candidates should focus on application, understanding and
critical thinking when studying
25
В© 2012 Assessment Strategies Inc.
Implications
For candidates who do not pass the CPNRE
• Waiting 8 months to rewrite does not lead to
improved performance compared to waiting 4
months
• During this additional delay, it is likely that their
course-based knowledge and exam-taking skills
degrade
Implications
• Unless there is a good reason to do so, candidates
should rewrite the CPNRE as soon as possible
26
В© 2012 Assessment Strategies Inc.
Acknowledgments
Dr. Han Bao, ASI
Assessment Strategies Inc.
www•asinc•ca
1400 Blair Place, Suite 210, Ottawa, ON K1J 9B8 Canada Tel: 613-237-0241 • Fax: 613-237-6684 • www.asinc.ca
1400, place Blair, bureau 210, Ottawa (Ont.) K1J 9B8 Canada Tél. : 613-237-0241 • Téléc. : 613-237-6684 • www.asinc.ca
Questions?
Chris Beauchamp, PhD
Manager, Business Development and Senior Consultant
Assessment Strategies Inc.
210-1400 Blair Place
Ottawa, ON K1J 9B8
1-888-900-0005 ext: 223
cbeaucha@asinc.ca
28
В© 2012 Assessment Strategies Inc.
Документ
Категория
Без категории
Просмотров
6
Размер файла
251 Кб
Теги
1/--страниц
Пожаловаться на содержимое документа