Monday, 14 November 2011

Validity theory

I attended one of SQA's occasional research seminars on Friday afternoon about the validity of assessment. It was presented by Paul Newton of Cambridge Assessment, part of Cambridge University.

Paul's talk was entitled: "Controversy surrounding modern validity theory". The validity of an assessment is a measure of its appropriateness. For example, assessing driving skills (solely) through a writing test would not be very valid.

The "controversy" relates to arguments surrounding classical validity theory versus modern validity theory. Classical theory focuses purely on the appropriateness of the test for its intended purpose; modern theory extends this to include all possible uses of the test. For example, a Higher question paper might be appropriate for its intended purpose (to measure students' knowledge of the associated curriculum) but not valid for some of its actual purposes (such as selection for university). Classical assessment theory would consider this examination valid (it's OK for what it was designed for), whereas modern theory would not (it's not OK for what it is used for).

It was a complex but interesting lecture. Classical validity theory creates lots of different types of validity (such as content validity, construct validity and predictive validity) whereas modern theory only has one (construct validity) but requires this to be tested for all possible uses. At the end of the presentation, my sympathies lay with classical validity theory since the complexities of measuring all of the possible effects of a test were too great.

These research seminars are presented by experts in the subject area, and are intended to keep SQA staff up-to-date with the latest developments in assessment and qualification design.


  1. Interesting post, and something all assessors and unit writers should consider. I have a niggling concern that project based assessments don't reflect real life very well in that they tend to run over elongated, unrealistic timescales that don't reflect real business practices. This particularly concerns me about graded unit projects, however a verifier once suggested that we could run graded units over just a few weeks with the time devoted exclusively to the project. While there are some timetabling issues with this, I like the idea and wonder if many colleges do this & what their experiences are?

  2. It sounds like an interesting idea, Colin. It would certainly be more realistic.