Writing Assessments

This is my quick and opinionated guide on how to write assessments for those new to the topic. Feel free to write a comment below if you feel I have missed anything.

What is an assessment?

This guide is interested in Summative Assessments – which are used to score or grade students over a specific period. Typically at the conclusion “sum” of a course. Formative Assessments are usually not scored or graded, but help tutors and the student get a feel for their “forming” learning. The most popular assessments being essays, exams, and portfolios.

Traditionally an assessment is chosen very early on in the course design process. And often without a lot of thought, merely repeating the methods used previously. This has been the experience most of us have had during our own study.

This haphazard approach has given us assessments that:

  • don’t connect to real-world experiences
  • are marked inconsistent across different courses, assessments and tutors
  • assessed topics not taught
  • were too easy, too hard or just plain confusing to the students

To negate these issues, the new thoughts within Learning Design are:

  • to prioritize Authentic assessment design
  • use Ongoing Assessments
  • Backward Design the course so learning materials and assessments are written after the learning outcomes are defined
  • Constructive Alignment between the assessment, teaching strategies and intended learning outcomes

These four Learning Design theories underpin this guide.

Prelude

Before we design an assessment, a course’s Learning Outcomes have already been decided (or should have). These are the pragmatic and practical outcomes of the Course Aim. What do I want students to know how to do when they leave this course?

At an institution like NMIT we would find them in the official Course Descriptor document and clearly displayed in any course introduction.

These have usually been guided by the Cognitive Domain in Bloom’s Taxonomy (Bloom, Engelhart, Furst, & Krathwohl, 1956). Bloom’s Taxonomy helps define subject matter and the depth of learning the students should achieve.

Revised Bloom’s Taxonomy

Typically, learners progress through these stages to reach mastery of their subject. Learning Outcomes use appropriate synonyms (similar words) of the above verbs (action words) related to the subject being taught.

So if we were to (roughly) apply these stages to a hypothetical LEGO building course we might have something like this:

By the end of the course you will be able to:

  1. List the types of LEGO Bricks (Remember)
  2. Explain the purpose of LEGO (Understand)
  3. Use LEGO to build (Apply)
  4. Report on the success and failure of your LEGO build (Analyze)
  5. Critique the use of the LEGO system in creating the build (Evaluate)
  6. Design an alternative Brick system (Create)

Obviously you wouldn’t use all six levels or have such great leaps in difficulty within the one course. But this does demonstrate how Learning Outcomes are used and decided on at the beginning of course design.

The Triangle

It is helpful to always have this triangle in mind when developing a course.

  • Learning outcomes: What do I want students to know how to do when they leave this course?
  • Assessments: What kinds of tasks will reveal whether students have achieved the learning outcomes I have identified?
  • Learning Activities: What kinds of activities in and out of class will reinforce the learning outcomes and prepare students for assessments? From passive lectures to reading to group work etc.

Making sure that nothing is missed in connecting each corner. And also that nothing superfluous is added. Everything is as watertight as possible. If any future questions get asked about your choice of Learning Outcome, Assessment or Activity, you can easily point how it aligns with the needs of the two connected corners.

Writing Assessments

Selecting the right kind of assessment is a subject for another post. But at this stage we mostly know the Why (the Learning Outcomes) and the How (Learning Activities) to do our assessment. Now we need specifics.

But what is very key when writing the Assessment is that they need to be:

  • Clear and punchy
    Use Simple English, and don’t write in the passive.
  • Only as much information as is needed
    Student should be able to read and understand it within a paragraph. Don’t include learning within the task description.
  • Written for the student reader, not the institute
    Write for your audience (“You are” instead of “The student”)
  • Understandable by the marker
  • Consistent across the Programme
    Work with the other tutors

Elements of an Assessment

The common problem with Assessment Briefs is that too much information is crammed in and can get a bit muddled. It’s really important to separate the key elements:

When designed well, any individual elements could change without having a great effect with any of the others.

Objective

This is the description of what you want the student to do. As realistic to a real world scenario as possible.

This objective could be replaced each semester and it wouldn’t effect how the student is marked in the marking guide. Allowing for more variety for the tutors, making it harder for cheating and most importantly – allows flexibility to adapt to new culture and local events.

This needs to be displayed clearly on assessment pages, and not hidden away in documents.

Instructions

This is the pragmatic explanation of what needs to be submitted and how. i.e submit a 500 word document in the form below.

This doesn’t need to outline in detail any processes that are standard and have already been taught in class. This is just doubling up in information, and if something is slightly different – it could cause extra confusion. Make sure what you write here couldn’t be put in the course material and be linked to.

This is also a good place to link to any learning support links.

Marking Guide (or Marking Scheme)

The marking guide shows the student how the tutor will be checking for evidence that the learning outcomes are achieved within their submission.

There are multiple ways of creating these. The most common way of doing them now is by using a Marking Rubric.

The Marking Rubric

“A rubric is a scoring tool that lists the criteria for an assignment. The rubric must present as clearly as
possible the criteria for grading each task the instructor is requiring. Well-written rubrics help students
understand what they are expected to accomplish in an assignment, improve student performance as well
as monitor it, and help define quality. Rubrics assist in making the evaluation and feedback process more
effective, more objective, and more likely to result in deeper student learning. Using rubrics help students
with peer assessment (judge the quality of their own and others’ work) and reduce the amount of time
instructors spend evaluating student work.”

Konyu-Fogel, DuBois and Wallingford (2013; p.73)

What a rubric helps us to do is:

  • Improve consistency in grading across tutors and over time
  • Reduce the impact of subjectivity in grading
  • Provide a mechanism for checking the assessment brief
  • More effective use of tutor time in grading/marking

Elements of a Rubric

Writing criteria and descriptions

Just like with assessment descriptions the criteria and criteria descriptions need to be:

  • Clear and punchy – Use Simple English
  • Only as much information as is needed, remove repetition
  • Student should be able to read and understand it within a sentence.
  • Don’t include learning material. They already should have access to that knowledge before coming to this
  • Written for the student reader, not the institute. Write for your audience (“You are” instead of “The student”) – no one wants to be treated like a cog

There is a debate around the way skill levels increase in rubrics. Some prefer to have the highest grade be on the left, others on the far right. Those advocating the prior – like the idea that a student is presented with the best possible result first up. Others like myself see it as grammatically incorrect as we read increasing values from left-to-right, and that a student seeing a table would know to look first to the far right and then work backwards, if the wanted. This way also allows you to write cleaner descriptions as you can just build upon the prior description. ie:

Decreasing in value:

10520
Assessment submitted, on time and covering all the requirements in the assessment descriptionAssessment submitted on time but not covering all the requirements of the assessment descriptionAssessment submitted lateNo assessment submitted

Increasing in value:

02510
No assessment submittedSubmitted and on timewhile covering all the requirements in the assessment description

What ever method you choose – it needs to be the same method across all the courses that a student may be taking.

Subjectivity and Commenting

When we write rubric descriptions we try hard to remove any subjectivity. This is because when we mark it is based on evidence and not on our personal opinions. This can be very hard to do, especially when dealing with subjects that have a large aesthetic aspect to them (like the arts, design, beauty or music). What I consider “smart”, “stylish” or “beautiful” could be different to you, and more honestly, I am a tutor and not an international leader or taste maker in my field.

But our opinions and encouragements still count and matter, so this where commenting becomes very crucial to the rubric marking process. Comments are the place where you can be subjective, emotional and encouraging.

Moodle

At NMIT we use Moodle as our Learning Management System.

With the Moodle gradebook, you can either enter the students’ scores manually or grade the assessment using a rubric. You can write feedback comments, mark up a PDF of an assessment with comments or symbols, and adjust the weighting of assessments.

It has it’s own way of creating rubrics that are slightly different to standard practice. While it can be slightly difficult to set up, it allows you to quickly mark student’s work and to comment, even allowing for predetermined comments. Some tutors find Moodle’s Marking Guide as a more flexible alternative (and can also be considered a simple form of rubric)

RubricMarking Guide
The Rubric has preset levels for each criterion that the teacher can select from when marking that criterion.
Tutors just click on the appropriate descriptor for each criteria and the scores are automatically calculated in Moodle.
Only one score is possible for each level of performance (range of scores and half marks per criteria are not possible)
Marking guide lets you enter a number as the grade for a criterion.
Marking guide also lets you build a set of frequently used comments to use when marking.
It only contains the criteria and not the descriptors – tutors need to enter a score (and a feedback comment) for each criteria

Summary

I am a fan of using rubrics (especially Moodle’s version) because it automates the pragmatic parts of marking, freeing up time to write encouraging comments. It also clearly delineates between evidence and opinions, which is extra security if there is any future disagreement with the grades given.

On a personal level all students in my courses should ideally be able to get 100%. If they don’t – it would be because of extraneous conditions (sickness) or active disengagement. In fact if I could I would get rid of grading and only have:

  • Completed all Learning Outcomes
  • Completed all Learning Outcomes and with Distinction
  • Try again

And put stronger focus on portfolio development as the true decider on quality of work.

I hope this guide has been of use to you. Please leave comments below on anything I may have missed out on.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.