Presented by Jennifer Fager For University of Wisconsin-Superior Enhancement Day 1/19/2011 Advice for Assessing Student Learning in an Online Environment п‚— Multiple Choice and Short Answer Questions п‚— Not always the best technique in either online or ground formats п‚— Lend themselves to cheating more so in online п‚— Questions, regardless of ground or online, must be written well п‚— The вЂњtestвЂќ has been overused and relied upon unless the instructor is an expert in writing good test items Advice continued п‚— Use multiple forms of assessment п‚— Has corroborating evidence of student achievement п‚— More personalized assessments reduce the likelihood of plagiarism п‚— п‚— вЂњTerm paperвЂќ vs. application paper using personal examples General research paper vs. select topics focus п‚— Use examples more вЂњauthenticвЂќ in format п‚— Consider problems of the field п‚— Engage in вЂњconversationвЂ™sвЂќ online Advice continued п‚— Security options п‚— Lockdown browser п‚— п‚— Student canвЂ™t go to another website during the test NOTE: Most students now have multiple devices so they can go to their IPhones, blackberries, laptops, IPads, etc. to research the вЂњanswersвЂќ Advice Continued п‚— Set test time limits п‚— Even if the student gets help, time restricts the amount п‚— Can punish the slower readers/processors п‚— What do you want in terms of meaningful, thoughtful responses Advice continued п‚— Assume any online test is, by definition, open book п‚— And perhaps open notes, and open neighbors п‚— DonвЂ™t try to make it otherwise Advice continued п‚— Quiz/test software products are available п‚— Can generate multiple versions of tests п‚— Each student can receive a вЂњdifferentвЂќ test п‚— http://www.fau.edu/irm/instructional/respondus.php п‚— Have students explain their answers п‚— 25% have to explain вЂ�AвЂ™ answers, 25% вЂ�BвЂ™ answers, etc. Advise continued п‚— Other security ideas п‚— вЂњthumbвЂќ identification п‚— Webcams п‚— Face-to-face testing at a testing center Online Assessment Resources п‚— Commercial Assessment Data Management Tools вЂ“ п‚— E.g. TracDat, WeaveOnline, Chalk & Wire п‚— Curriculum Design Tools п‚— E.g. WIDS вЂ“ link outcomes to Performance Assessment Tasks п‚— Survey Instruments п‚— Survey Monkey п‚— Zoomerang п‚— Portfolios п‚— Course Management Systems Quiz and Analysis Functions п‚— Blackboard and others Resources Continued п‚— Google Docs п‚— Jing п‚— Reflective Instruments п‚— Leadership п‚— Personality Style п‚— Teamwork п‚— Merlot (has content template builder) п‚— Clickers п‚— AdobeConnect with polling features п‚— Lock-down Browsers for online testing Fundamental Questions п‚— What evidence do you have that students achieve your stated learning outcomes? п‚— How have you articulated these outcomes to students and adjunct faculty? п‚— In what ways do you analyze and use evidence of student learning? Assessment Process п‚— The same process is used for online programs and courses as for ground (traditional): п‚— Needs Assessment п‚— п‚— Needs of the profession Needs of the learners п‚— Articulate your expectations п‚— Measure achievement of expectations п‚— Collect and analyze data п‚— Use evidence to improve learning п‚— Assess effectiveness of improvement Measure Online Learning п‚— Direct Methods п‚— Mostly the same as ground п‚— Can use any student work products that can be saved in an electronic form п‚— Some challenges exist with lab work, oral performances, etc. п‚— Indirect Methods п‚— Again, same as ground п‚— Survey response rates are notoriously low Advantages of Online Assessment п‚— Artifact collection can be automated through the course management software п‚— Can work for ground as well п‚— Assignments, rubrics can be standardized and mapped to specific competencies/outcomes п‚— Again, can work for ground as well New Opportunities with Online п‚— Assess quality of discussions, group work п‚— Particularly effective with asynchronous п‚— Assess individual learning in more depth п‚— Student advising can be more robust п‚— Use of standardized rubrics can result in greater consistency in grading п‚— Particularly effective when combined with periodic instructor norming and use of anchor samples Discussion & Group Work п‚— Faculty can isolate and organize contributions to a threaded discussion and give feedback on how to improve п‚— Can archive individual contributions and group discussions п‚— Formative and summative п‚— Course, program, and institutional levels п‚— Create a record of instructor effectiveness п‚— Useful for annual review, tenure Individual Student Learning п‚— Individual learning/achievements can be tracked across the educational experience п‚— E-format allows for wide dissemination of collected data п‚— Current software packages provide various levels of data collection & dissemination Rubrics п‚— Clear articulation of expectations that are linked to specific course & program outcomes п‚— A means through which correspondence of online & ground rigor can be ensured (alignment) п‚— Increases consistency in grading across sections, courses, programs, even colleges п‚— Holistic vs. Analytic Example of a Successful Online Assessment Strategy п‚— Develop course/program maps which articulate learning outcomes and link them to specific courses and course assignments п‚— Collect and archive artifacts through the course management software п‚— Post rubrics through course management software for student and faculty use Ensure Validity, Reliability п‚— Tracking inter-rater reliability helps achieve consistency in grading п‚— Post rubrics, anchor samples for 24/7 faculty use п‚— Full-time/core faculty should formulate competencies, achievement standards & interpret results Sample Assessments п‚— Review the Video Gaming Assignment included at the end of this packet. п‚— Determine how this assignment could be adapted for online learning General Education Outcomes Example п‚— The ability and inclination to think and make connections across academic disciplines п‚— How would you assess this outcome in ground or online courses? Websites and Resources п‚— http://fod.msu.edu/oir/TeachWithTech/online- courses.asp п‚— http://www.uwstout.edu/soe/profdev/assess.cfm п‚— http://www.fgcu.edu/onlinedesign/index.html п‚— http://www.pvc.maricopa.edu/~lsche/resources/onlin eteaching.htm п‚— http://sloanconsortium.org/ Questions and Concerns п‚— What issues have you encountered when assessing online learning? п‚— What are your concerns regarding the use of online assessment techniques? п‚— What policies and/or procedures need to be in place to reduce or alleviate these concerns? Questions Still Burning? п‚— What else do you need to know? The Bottom Line? п‚— Assessment in online learning can be the same as assessment on groundвЂ”artifacts and tools can be the same even if the medium is different!