LH216V Develop the Learning by Using Grading Criteria. A course in teaching and learning in higher education directed to KTH teachers. The aim of the course is that the course participant should be able to use outcomes based criterion-referenced grading criteria and assessment aligned to the criteria in his or her own courses.
The course will include examples on how to replace written exams with other forms of assessment, even for courses with large student groups.
Next course offering in English will preliminary be in period 4, 2025.
Headings denoted with an asterisk ( * ) is retrieved from the course syllabus version Autumn 2020
Content and learning outcomes
Course contents
Positive effects of criterion-referenced grading criteria.
Pitfalls in connection with introduction of grading criteria.
How expected learning outcomes, grading criteria and examination are related to one another and to the students' learning.
Examples of how criterion-referenced grading criteria have been developed and are used in courses at KTH Royal Institute of Technology and Uppsala University.
How one writes criterion-referenced grading criteria.
Student perspective on grading criteria.
What is needed to make the grading criteria support the learning?
Combining assessments to a final mark.
Guidelines and support for criterion-referenced grading criteria at KTH Royal Institute of Technology
Work on grading criteria and modified assessment of the course participant's own course.
Intended learning outcomes
After passing the course, the course participant should be able to:
· demonstrate how intended learning outcomes, grading criteria and examination are related
· formulate criterion-referenced grading criteria for a course
· suggest how examination items should be designed so that they are assessed according to specified grading criteria
· critically review and reflect on one's own specified grading criteria and those of others
· give an account of how assessments can be weighed together to a final mark and reflect on the possible effects of this
· reflect on how the grading criteria can be used in one's own teaching
in order to be able to design criterion-referenced grading criteria and examination that can be used directly in an own course.
Learning activities
There will be four meetings, where meeting one and two are mandatory. There are preparations or assignments to be done before each meeting. After each meeting, the slides from the presentations will be published in the course room. A chronological view of all deadlines and meetings can be found under Syllabus/Kursöversikt.
Language
The main language of the course is English. However, in group discussions it is allowed to speak other languages as long as all in the group understand. The terminology is given in both Swedish and English (see Key concepts below), and you are allowed to write your assignments in either Swedish or English. We recommend that you use the same language in your assignments as the course you are writing your grading criteria for is using.
Detailed plan
Learning activities
Content
Preparations
Meeting 1 (mandatory)
Purpose of grading criteria, how grading criteria can be designed, examples of ILOs, outcomes based grading criteria and assessment, how grading criteria can support the learning in a course, your first own grading criteria.
See under "Preparations before course start" below.
Meeting 2 (mandatory)
Students' thoughts about grading criteria, a case study on grading criteria at Uppsala Universitet, discussions on and work with your submitted assignment 1, KTH regulations, combining grades to a final grade, pitfalls and advice.
As in every course, there are several concepts that you should be aware of. Below is a list of the key concepts of this course, with the terms given both in Swedish and English. Such a list has both a pedagogical and a terminological function and should be included in each course.
A course in teaching and learning in higher education, for example LH231V Teaching and Learning, is recommended but is not a requirement.
Specific preparations
Before the first meeting, please do the following preparations:
Read the article Chris Rust: Purposes and Principles of Assessment, available at academia.edu (also attached to the welcome email).
Review the concept of constructive alignment and read about how intended learning outcomes should be written. Please look at these slides to refresh your memory about this. If you need more explanation and examples, please look at the reading list on the last slide.
In this course you will be working with the design of grading criteria in a course. Decide which course you would like to work with, and go through the current intended learning outcomes (from the course syllabus). If your course is given in Swedish, you should look at the Swedish version of the intended learning outcomes (ILOs).
Check whether the intended learning outcomes are result oriented, assessable and at a reasonable level. According to the KTH regulations, all intended learning outcomes have to be assessed in the course, and only students fulfilling all intended learning outcomes should pass the course. Modify if needed the ILOs so that they are possible to assess.
Bring your (modified) intended learning outcomes to the first meeting (in the language of instruction of your course), or if you participate on-line: prepare to share them in a window in Zoom.
John Biggs and Catherine Tang: Teaching for Quality Learning at University, 4:th edition, Open University Press, McGraw Hill Education, 2011. Chapter 11 and 12. Available online through KTHB.
Graham Gibbs: Learning in TeamsGrading of team projects, among other things.
Phillip Dawson: Assessment rubrics: towards clearer and more replicable design, research and practice (available online through KTHB). Describes and discusses grading rubrics and gives 14 design elements related to rubrics. A useful paper, but a bit hard to read if you are not used to read educational research papers.
Lab Report Rubric. An example of a well designed grading rubric from MIT. The lab course is described, together with the ILOs. The scale 0-3 is applied on the eight types of criteria used. How the criteria relate to the ILOs is shown, together with the formula for the final scoring. All in all, this rubric looks like what you are expected to develop for your course within this course (LH216V).
Scoring rubrics for professional writing. Nine criteria and four scoring levels that can be useful for inspiration on how to work with marking (or peer marking) where writing skills are to be trained and assessed.
Scoring rubrics for professional presentations. Eight criteria and four scoring levels that can be useful for inspiration on how to work with marking (or peer marking) where oral presentation skills are to be trained and assessed.
INL1 - Assignment, 1.5 credits, Grading scale: P, F
Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.
The examiner may apply another examination format when re-examining individual students.
An individual assignment in the own subject area is included in the written assignment.
The section below is not retrieved from the course syllabus:
There are three mandatory assessments in the course. It is important that you submit the assignments before the stated deadline in order for the course to work. Please contact the course coordinator if you need to submit a hand-in late.
Assignment 1a (ILOs and draft of new grading critieria), submitted before 17 April at 19:00 in Canvas.
Assignment 1b (peer feedback to two peers), submitted before 20 April at 19:00 in Canvas.
Assignment 2 (final version of grading criteria etc.), submitted before 30 May 2023 at 19:00 in Canvas. Assessment (with respect to the grading criteria below) and constructive feedback on your assignment 2 will be given by the teachers before the final meeting. A submission fulfilling all criteria will pass. A submission not fulfilling any criterion will fail. A submission fulfilling at least one but not all criteria will receive Fx, and could be completed (see below).
Other requirements for final grade
Active participation at course meetings 1 and 2 is compulsory.
Grading criteria/assessment criteria
Since this is a pass/fail course, the grading criteria are close to the intended learning outcomes. ILOs 1, 3 and 4 have been split into two critera each.
account for assessable intended learning outcomes for your own course (ILO 1, assessed in assignment 1a (formatively) and assignment 2)
account for assessment tasks for your own course (ILO 3, assessed in assignment 2)
formulate outcomes based criterion-referenced grading criteria for your own course (ILO 2, assessed in assignment 1a (formatively) and assignment 2)
account for how intended learning outcomes, grading criteria, and assessment fit together (ILO 1 and 3, assessed in assignment 1a (formatively) and assignment 2)
review critically and reflect on others' grading criteria (ILO 4, assessed in assignment 1b)
review critically and reflect on ones own and others' feedback on grading criteria (ILO 4, assessed in assignment 2)
describe how to combine assessments to determine a final grade and reflect on which effects this can result in (ILO 5, assessed in assignment 2)
reflect on how the grading criteria can be used in your own teaching (ILO 6, assessed in assignment 2).
Opportunity to complete the requirements via supplementary examination
A submission of assignment 2 fulfilling at least one but not all criteria will receive Fx, and could be completed (submitting a new version in Canvas) within three weeks (29 June 2023) by submitting the new version in Canvas. A new version that is fulfilling all criteria will pass. A new version fulfilling all criteria except one or two may be completed one more time (latest 15 August 2023). Otherwise the submission will be failed.
Ethical approach
All members of a group are responsible for the group's work.
In any assessment, every student shall honestly disclose any help received and sources used.
In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.
Further information
Changes of the course before this course offering
The first meeting has been extended 45 minutes, and the presentation by Hans Havtun has been moved from the second to the first meeting.
An extra voluntary meeting is inserted between meeting 2 and the final meeting where a course participant who submit a draft of assignment 2 can receive feedback both from peers and teachers.
The course memo can be found in the course room at the end of the Syllabus page.
Advice from participants from the previous course offering in English
Ask questions, participate actively in all discussions, work systematically, keep deadlines, and participate in all meetings (even if not mandatory).
Please be aware of FX requirement in the course.
Make the ILOs and the grading as realistic as possible. Results can be used directly to courses.
Learn this stuff so you won't be one-upped by students who are playing the system.
Course evaluation and course analysis
By the end of meeting 1, a mini-evaluation will be carried out. At the end of the final meeting, we will evaluate the course together, constituting the course meeting of the course (as required by the KTH regulations for course evaluation and course analysis). After the final meeting, there will be an ordinary Learning Experience Questionnaire (LEQ).
