Perspectives of mathematical e-assessment on a pathway to institution-wide adoption
Chris Graham
School of Mathematics, Statistics & Physics Digital Learning Unit
This talk
Reflect on the extent to which mathematical e-assessment has been adopted at Newcastle University, UK
Give insight into perspectives of students and colleagues
Explore barriers to engaging with mathematical e-assessment
Our team at Newcastle
I lead the Digital Learning Unit in our University's School of Maths, Stats & Physics.
We have a unique set up: perspectives that I hope will be insightful (but this is also a caveat for relevance to other institutions).
The existence of our unit is thanks to the development of The Numbbas e-assessment system
Numbas at Newcastle
Numbas has been used at Newcastle since 2012.
Embedded in nearly 100 modules, primarily in Maths, Stats & Physics and the School of Engineering, with over 100,000 student assessment attempts each year.
A lot of formative use, but I have mainly been interested here in summative.
Perspectives
Over the past semester, I have been trying to get a snapshot of attitudes to e-assessment.
Several dialogues with key partners in the use of Numbas:
- Students
- Maths Transition Officer
- Director of Education
- Academic colleagues in our department and Engineering
- Chair of the Board of Examiners
- University Learning Enhancement & Technology Manager
Maths & Stats Stage 1
8 maths modules at stage 1. Of particular interest because of the scale of e-assessment use:
- 7 use Numbas for coursework
- 4 hybrid Numbas final exams, 4 written exams.
1 module assessment is fully written on pedagogic grounds.
Reflections: Students & Transition Officer
On e-assessment used for transition:
Powers transition material for our stage 1 with mini tests and a diagnostic assessment. Overwhelming positive from student perspective is the formative, infinite nature.
On how students see summative assessments:
Students see assessments as less onerous (don't need to write out tidy solutions) and appreciate much quicker marking
Opinion is that Numbas is most impactful in a hybrid mode. We don't need to see a lot of working, but it does need to be nurtured.
Maths & Stats Big Picture
Reflections: Director of Education
On the benefits of e-assessment:
- for staff: workload
- for students: timeliness of feedback
- for the School: modernity associated with its use
Examples of e-assessment used badly/well:
- Badly: Jeopardy of having lots of working for one answer box.
- Well: tailored feedback; error-carried and alternative answers.
Reflections: Academic colleagues
On the main challenges to adoption:
- Entrenched views
- Workload
- Awareness of what's possible
On written vs online exams:
One colleague reflected on retaining a written exam because they would like to see written mathematics. But in practice tend to be very soft on this in the exam: don't have time to mark anything except bottom line.
Reflections: Chair of the Board of Examiners
On written assessments being replaced by digital:
Good for establishing foundations, but important for students to do higher level mathematical writing for most exams. Very positive about hybrid solutions.
On the impact of AI:
Not very concerned except for small pockets, e.g. projects. In-course assessments are mostly much more about engagement.
On how externals view digital assessments:
Ours see written/digital as equals, considered on their own merit.
Wider University Context
University Exam Software
Newcastle use Inspera as a digital exam solution. Numbas is also an approved option:
We have muscled in to all of the exam processes here:
- Digital exams steering group
- Invigilator training
- Exam administration
- Exam support channels
28 exams with 6000 exam sittings in Numbas.
Exam sittings by format:
Reflections: University Learning and Enhancement Team Manager
On how they would manage mathematical exams without our support:
Would struggle to support mathematical exams with just Inspera: doesn't have the extensive features. Would not have expertise to support and would have been a struggled to interpret needs. Tender for exam software may have been different.
On concerns about non-Inspera exams:
Nervous about non-standard exams, without having people who they can call upon.
Conclusions
In-course assessment is settling to a hybrid of written and digital across all levels (course, module, single assessment)
There are still challenges engaging colleagues and ensuring that assessments are well designed to avoid student frustration
Specialist mathematical e-assessment exams are desirable and possible, but difficult to integrate at institutional level
Thanks for listening!
Thank you to Kate Henderson, Jon Goss, Chris Pearson, Martina Balagovic, Simon Lambert, Terry Charlton, Magda Carr, Ryan Doran and student representatives for dialogue discussions.
And, as ever, to my colleagues in the Digital Learning Team for all of their great work!