Tips, tricks & gyan about effective assessment questions. (Summary by authors)
Technology today offers many new opportunities for innovation in educational assess- ment through rich new assessment tasks and potentially powerful scoring, reporting and real-time feedback mechanisms. One potential limitation for realizing the benefits of computer-based assessment in both instructional assessment and large scale testing comes in designing questions and tasks with which computers can effectively interface (i.e., for scoring and score reporting purposes) while still gathering meaningful measurement evidence. This paper introduces a taxonomy or categorization of 28 innovative item types that may be useful in computer-based assessment. Organized along the degree of constraint on the respondent’s options for answering or interacting with the assessment item or task, the proposed taxonomy describes a set of iconic item types termed “intermediate constraint” items. These item types have responses that fall somewhere between fully constrained responses (i.e., the conventional multiple-choice question), which can be far too limiting to tap much of the potential of new information technologies, and fully constructed responses (i.e. the traditional essay), which can be a challenge for computers to meaningfully analyze even with today’s sophisticated tools. The 28 example types discussed in this paper are based on 7 categories of ordering involving successively decreasing response constraints from fully selected to fully constructed. Each category of constraint includes four iconic examples. The intended purpose of the proposed taxonomy is to provide a practical resource for assessment developers as well as a useful framework for the discussion of innovative assessment formats and uses in computer-based settings. (Abstract by author)
In England there is a national survey of all students just before they graduate from bachelor’s programmes called the National Student Survey (NSS). This produces a very influential ranking of all universities and colleges in England in terms of how the education they provide is perceived by their students. Top (or near the top) of this ranking every year is the Open University UK – well ahead of prestigious universities that are ranked many hundreds of places higher on world (research) rankings (although with one exception, to which I will return). This extraordinary fact deserves attention because it is normally assumed that open and distance learning has a series of educational limitations that make it inherently inferior to traditional face to face campusbased education. Somehow, the Open University UK must be embodying fundamental educational principles very effectively despite being a distance learning organisation. If we could spot what these principles were, they might be very useful to us. (Introduction by author)