Adoption and attitudes of staff to mathematical e-assessment
Chris Graham
School of Mathematics, Statistics & Physics Digital Learning Unit
This talk
-
Reflect on the extent to which mathematical e-assessment has been adopted at Newcastle University, UK
-
Give insight into perspectives of key colleagues
-
Explore barriers to engaging with mathematical e-assessment
Hopefully this will raise questions and discussion: add your questions to the Matrix Live Chat.
Our team at Newcastle
Myself, Christian and others make up the Digital Learning Unit in our University's School of Maths, Stats & Physics.
We have a unique set up: perspectives that I hope will be insightful (but this is also a caveat for relevance to other institutions).
The existence of our unit is thanks to the development of Numbas...
From EAMS 2016!
L-R: Chris Graham, Christian Lawson-Perfect, Bill Foster
Our team has grown significantly since: from next month, 6 members!
Numbas at Newcastle
Numbas has been used at Newcastle since 2012.
Embedded in nearly 100 modules, primarily in Maths, Stats & Physics and the School of Engineering.
Over 100,000 student assessment attempts each year.
Part of the University digital exam solution.
Perspectives
In our School:
- Maths Transition Officer
- Director of Education
- Academic colleagues
- Chair of the Board of Examiners
And partners elsewhere in our University:
- Engineering Champion
- University Learning Enhancement & Technology Manager
Maths & Stats Stage 1
8 maths modules at stage 1:
- 7 use Numbas for coursework
- 4 hybrid Numbas final exams, 4 written exams.
1 module assessment is fully written on pedagogic grounds.
Reflections: Transition Officer
On how e-assessments are used for transition:
Powers transition material for our stage 1 with mini tests and a diagnostic assessment. Overwhelming positive from student perspective is the formative, infinite nature.
On how students see e-assessments:
Students see assessments as less onerous (don't need to write out tidy solutions) and appreciate much quicker marking
Opinion is that Numbas is most impactful in a hybrid mode. We don't need to see a lot of working, but it does need to be nurtured.
Maths & Stats Big Picture
Reflections: Director of Education
On the benefits of e-assessment:
- for staff: workload
- for students: timeliness of feedback
- for the School: modernity associated with its use
Examples of e-assessment used badly/well:
- Badly: Jeopardy of having lots of working for one answer box.
- Well: tailored feedback; error-carried and alternative answers.
On the main challenges to adoption:
- Entrenched views
- Workload
- Awareness of what's possible
Reflections: Academic colleagues
On written work:
One colleague reflected on retaining a written exam because they would like to see written mathematics. But in practice tend to be very soft on this in the exam: don't have time to mark anything except bottom line.
On Training opportunities:
"There's never a good time, but it's always a good time"
On getting support for writing questions:
With some experience of writing, would be happier doing it themselves than getting low level assistance (e.g. postgraduate).
Reflections: Chair of the Board of Examiners
On written assessments being replaced by digital:
Good for a limited number of topics: checking recall, basic computations, basis on which all knowledge is built, but important for students to do higher level mathematical writing for most exams. Very positive about hybrid solutions.
On the impact of AI:
Not very concerned except for small pockets, e.g. projects. In-course assessments are mostly much more about engagement.
On how externals view digital assessments:
Ours see written/digital as equals, considered on their own merit.
School of Engineering
Heavy users of Numbas: 55,000 student assessment attempts across 25 modules, with 20 final exams.
Engineering Champion promotes Numbas use in the School.
Reflections: Engineering Champion
On stage 1 exams:
Rely heavily on automatic assessment. Split between Numbas and Inspera, with the latter mostly MCQ questions.
On other Numbas applications:
Similar distribution to Maths at later stages, where they want more focused marking for longer questions.
Some very innovative use in e.g. lab-based activities. Lots of interest, but no awareness of, options for hybrid assessments.
On the future:
Lots of opportunities to coordinate further between our team and their Numbas Champion.
Wider University Context
Newcastle use Inspera as a digital exam solution. Numbas is also an approved option:
We have muscled in to all of the exam processes here:
- Digital exams steering group
- Invigilator training
- Exam administration
- Exam support channels
28 exams with 6000 exam sittings in Numbas.
Exam sittings by format:
Reflections: University Learning and Enhancement Team Manager
On how they would manage mathematical exams without our support:
Would struggle to support mathematical exams with just Inspera: doesn't have the extensive features. Would not have expertise to support and would have been a struggled to interpret needs. Tender for exam software may have been different.
On concerns about non-Inspera exams:
Nervous about non-standard exams, without having people who they can call upon.
Thanks for listening!
Thank you to Kate Henderson, Jon Goss, Chris Pearson, Martina Balagovic, Simon Lambert, Terry Charlton, Magda Carr and Ryan Doran for dialogue discussions.
And, as ever, to my colleagues in the Digital Learning Team for all of their great work!