Report: “Exploring eAssessment”

Date: October 16th, 2012
Location: University of Lancaster
Details: “Exploring eAssessment”. This event was hosted as part of the eAssessment Association Regional Events and JISC Regional Support Centre Northwest.

• This post was first published on my private TEL blog, as we did not have a University blogging platform with which to share our work and experiences in the College. The post has been copied here as it was first published on October 16, 2012.

Today (October 16, 2012) I attended the JISC RSC (Regional Support Centre) North West and eAssessment Association event ‘Exploring eAssessment‘ in the lovely setting of Lancaster House Hotel.

With the event was billed as:

“With the pressure to show impact of e-Assessment in our institutions, it’s important to know that the technology is being applied in the most effective way. We have brought together speakers from near and far to share their experiences of how you can make a difference with e-Assessment within your own organisations.”

the schedule covered aspect of assessment such as ‘developing flexible e-Assessment spaces’, ‘quick wins for learner assessment’, ‘importance of learner tracking as a motivational tool’, as well as how to use QR Codes to “deliver assessment tasks in authentic spaces, allowing learners to interact with physical spaces while recording their actions”.

Note: I use ‘eAssessment’ and not ‘Eassessment’ or ‘E-assessment’ just because its my personal preference. Where I quote anything written I’ll use whatever version the author uses.

Importance of Learner Tracking as a Motivational Tool
Roger Clegg from Oldham College kicks off the day with this presentation on tracking learners and using this tracking for motivation.

• What works well and what isn’t working is more important than the bits in the middle, but large number of students involved in Oldham College’s Work Based Learning (WBL) studies increases the need for motivation.
• Online assessments centres used for WBL but had limited success due to location, room sizes, student access, resources available, costs, funding, etc. Moved to Moodle (and Ascentis) with fully online courses and eAssessment to reduce these problems (especially costs) and give students choice for eAssessment or centre-based assessment. Immediate results were positive and effective (and efficient use of resources, from all angles and stakeholders).
• Challenges included personalisation, dynamic curriculum, reduction in funding, common inspection framework, use of technology, WBL distance learners, and efficiency vs. effectiveness (doing more with less).
• Strategic targets based around Common Inspection Handbook introduced tough measures, particularly expectation of continuous assessment so learners/staff know where they are.
• Personalisation is a challenge, not based on size of cohort (although this does introduce it’s own complexities) but technology able to be used for different delivery/study methods and different levels/stages of progress through module materials and assessments.
• Students can input their own “active academic achievement/target” for review; while its not assessed it can help tutor direct and support student on their study ‘guided learning’ as well help struggling students (if the students use it?).
• Gantt chart developed dynamically from the system for assessments helps both students and tutor see progress and, if appropriate, alert both to delays or problems in meeting assessment deadlines:

Developing Flexible e-Aessessment Spaces
John Dermo from the University of Bradford shared his experience from developing and supporting an innovative eAssessment facility using client technology and virtual desktops.

• Developed room with 100+ computers used for summative assessment, only used for 6 weeks of the year, so had to come up with extra uses and purposes of the room for the rest of the year to obtain funding – sorry, Jon didn’t go into this any further even after introducing it as a challenge.
• QuestionMark and Blackboard brought together in eLearning Pathfinder Programme: “Embedding support processes for eAssessment”.
• Driver for focus on online assessment include Institution strategy on eAssessment, academic & curriculum framework, National Student Survey (NSS) results, increased use of Technology Enhanced Learning (TEL), and student expectations on use of technology.
• NSS feedback:

• PC terminal (not independent PC) cluster/room so students cannot ‘easily’ view and copy from another’s screen in an awkward space, also adhering to accessibility and ‘interesting’ layout – clusters of three terminals on each table in a modular design. Line-of-sight an important consideration when considering layout, but distance between students and terminals makes this problem negligible and no evidence of copying has been found in years since introduction of this room/layout

• PC terminal means only one programme available on PC and that is the assessment. PC terminal can be swapped between student and assessment vision for different uses of the room, but smartcard the student uses determines what ‘image’ is loaded when the student logs in. Different assessments can be used in the room at the same times based on the smartcard (resits).
• Benefits to eAssessment room include re-image between exams quickly, control software available when taking exam, easier to invigilate, nicer environment, easier to manage extra-time students (cordoned off 15 PCs in glassed off area).
• Challenges on managing this space include ownership, getting most out out of facility, managing door locks, expanding use of rooms for essay exams, server capacity for larger groups, etc.
• Future use of the room … well, do we still need these rooms? Certainly not for 200 or 500 students, but perhaps smaller more adaptable rooms and cleverer (is that a word?) assessment strategies in order where stakes not so high or spaces for BYOD (Bring Your Own Deivce).

Quick Wins for Learner Assessment
Judy Bloxham from JISC RSC reports on the Common Inspection Handbook and how it targets assessment for monitoring progress and enabling learners to understand how to improve, and tools that can help us achieve this.

• Why do we assess? Diagnostic, Formative, and Summative.
• We (assessors) think about progress tracking as important, students think about motivation as being important. Feedback and praise important in this motivation.
• Quizlet.com, and Quizard app flash cards and ‘game’ is good way for revision and interesting flash cards.
• mbclick.com classroom response online with immediate feedback (student and tutor) with anonymous results.
• Infuselearning.com online quiz maker and various styles and email results or download to spreadsheet.

• Nearpod demonstrated (but those on Android can’t access!) to great effect with projector showing instructor side of the app while we (obviously) see the student side with slides and questions.

Numbas
Bill Foster and his colleague from Newcastle University demonstrated Numbas, the browser based “open-source online assessment system” based around mathematics and equation based assessment(s).

• Practice time-solving problems students were expected to do led to development of Numbas as an open-source assessment system for continuous assessment. Need for control over system, add functionality, etc led to development of Numbas over commercially available systems, and needed to be SCORM-compliant.
• Design goals included being reliable, good for maths and complicated algorithms, easy to use and good looking (from student feedback), used by non-technical ‘authors’, emphasis on creating informative eAssessment and learning materials, and importance placed on feedback in the design and implementation.
• Why use? Easy to tailor to your own needs, full control over creation and dissemination of assessment, customisable from appearance to core functionality, online editor, browser-based, multi-platform, etc.

QRreative Assessment
Running throughout the day Jon Gill set up the assessment exercise for us to follow, scan and participate in his QR Code demonstration on “how these simple codes can be applied to almost any space … [and the] simple user interface allows staff to design their own assessment tasks”.

• Each QR Code scans to a system the requires registration (or logging in) from the first code, and each code thereafter your performance (and score) based on the codes and choices you make.
• The quiz/assessment (?) is only as effective as the questions you choose and direct the students to – asking the students to find a code on a certain object and ask if it’s full or half full (bottle of water) then registers different score, but is this really assessment or just a really good orienteering exercise? I’d call this a game farther than assessment, but I guess it’s more about how you set it up rather then the code itself.
• QR Code Safari is the history of a village, as created and played by school children, including pictures from history merging in to present day, as directed from and by QR Codes themselves.
• Useful for health & safety training, location-based exercises, etc … what do you think?