Assessment and Testing Redux, II

salvēte iterum, sodālēs! In this post we’ll look at an important, but often overlooked, purpose for formal testing (as a category of assessment) … in fact, it’s a primary purpose for external assessments, whether they’re the infamous “standardized tests” in the United States or the national exams given in most other industrialized nations. That purpose, of course, is not so much to measure individual students’ learning as it is to make sure that schools, and teachers, are doing what they’re supposed to do: to assess programs as well as learners. In fact, when the College Board’s Advanced Placement examinations were first developed in the 1950’s, the idea was to identify schools, teachers, and students who were doing college-level work in high school … for the purpose of providing greater opportunity (college admission, for example, or placement into higher-level courses) for students who ordinarily wouldn’t have received such opportunities.

In short, one big purpose of testing is to do quality control … and some identification of quality that might otherwise go unidentified. Thomas Jefferson, when he proposed a system of free public schools for the state of Virginia, had a similar purpose of identifying quality. If I recall the quotation in Diane Ravitch’s books, he planned to “rake from the rubble” – or maybe it was “from the muck” – a few “boys of best quality” who would be trained to become leaders. Over time, the emphasis shifts between quality identification and quality control, though both are always present in the background of standarized or external measures of learning.

As I write this post in early 2010, external testing, especially in the United States, tends to be focused on quality control (from the perspective of the school or the teacher). Yes, testing also assesses an individual’s progress over time, but the focus is usually not on individual growth; it’s on a snapshot of student performance, and it’s aimed at determining whether factory-model schools are doing a good job with their product of student learning. Now, please don’t understand me! I think quality control is important! After all, no one wants a defective product, and no one wants a defective learning opportunity, either! Regardless of your underlying model of learning – regardless of whether learning is a factory, a retail store, a workshop, or some other model entirely – you still want learners to leave with a good command of the knowledge, skills, and understanding you intended to impart to them. But different models of learning will lead to very different approaches to quality control.

Let’s return, again, to our frequent distinction between the factory and the workshop, this time from the perspective of quality control. In a factory, as a rule, quality assurance is distinct from production. Even in a Japanese auto plant, where the workers are highly involved in assuring in-production quality control and can actually stop the production line if the see a quality problem, there are still some post-production spot-checks to make sure that everything is assembled properly. (As I write this, one manufacturer in particular is probably wishing it had taken more control of the supplier who sold it a vast number of defective pedals!) But in a workshop, where one or two apprentices are working on each product under the master craftsperson’s supervision, quality assurance is seamlessly linked with production. If there’s a problem, someone will notice it – and correct it – quickly. The hope, at least on the apprentices’ part, is that they will catch any defects before the master does; after all, the master probably has a bit of a temper! 🙂

As we return from our metaphorical excursion to the factory, where testing is essential, and the workshop, where it’s usually not necessary, what lessons about assessment can we bring with us?

  • By virtue of its size and structure, a factory-model school probably requires testing. The purpose of such testing is primarily to ensure the quality of the program, as the researchers at the University of Michigan pointed out in this link, to which I referred in a prior post.
  • If a learning community is not structured on the factory model, it may or may not require testing. If it’s clear from other assessments that the learners are learning and that their needs are being met, testing is probably not necessary. On the other hand, if there are learning problems, or if teachers aren’t meeting learners’ needs, testing might be one tool that could uncover the problems.

In any case, it’s probably most helpful to think of testing the way that doctors think of medical tests, or the way that sensible factory managers think of their quality assurance programs.

  • When a doctor receives alarming test results, her first response isn’t to punish the patient; it’s to solve the problem revealed by the test results. (Now, if the patient has contributed to the problem, she may need to have an unpleasant conversation with the patient – but the purpose is to correct, rather than to punish!)
  • If the QA results show that the product is defective, a sensible factory manager doesn’t punish the products! He’s much more likely to focus on fixing the process. (Now, if it turns out the workers haven’t been following the process, he may need to take some corrective action with them! Again, though, the purpose of these actions – painful as they may be to the workers at the time – is to fix the problem, not to punish the people.)

Similarly, if test results in a factory-model school reveal problems, a prudent administrator avoids shame and blame – whether of teachers or of students. It’s tempting, of course, to use shame and blame, but it doesn’t actually solve the problem. For example, if 89% of the students in Course A have failed to master Objective 1.7, the principal will probably get much better results from

  • finding out exactly what Objective 1.7 is, and how proficiency is to be demonstrated;
  • finding out whether the teachers of Course A had this understanding (if not, the fix is probably easy!);
  • finding out what the teachers did with Objective 1.7;
  • possibly finding out whether the students had trouble with related objectives in previous courses (if they did, it’s a cumulative problem, but at least you know it’s there and can take appropriate action!);
  • finding out whether Objective 1.7 has always been a problem for these teachers, regardless of the prior background of their students;
  • finding out what different strategies were used by teachers whose students did master Objective 1.7; and
  • sitting with the teachers as they figure out how to adapt their strategies (or adopt those from the more successful classes) to help more students with Objective 1.7.

Regardless of the educational model, all learners need ongoing assessments of their progress. These may or may not be “tests” as we traditionally envision them. Depending on the learning goal – and the criteria for mastery of that goal – a well-designed test can be a great assessment, but so can

  • an oral retelling of a story;
  • a set of illustrations;
  • an original video based on a story;
  • a new story, created by a participant, that retells an existing story from a new viewpoint, or that tells “what happened then” or “what happened before” or “what was happening at the same time, over there”;
  • a set of learner-designed comprehension questions (with possible answers) about a story;
  • a learner-designed quiz that asks “you” to find, analyze, or create examples of a grammatical concept in a passage;
  • a collection of links about a background concept;
  • a self-correcting, learner-designed quiz that asks “you” to understand the information presented in a collection of links;
  • a Venn Diagram, Bubble Map, or other graphic representation of similarities and differences between a Roman or Latin “thing” and a modern-day “thing”;
  • a thematic web of the type that Randy described in his recent comment;
  • a set of “big questions” about a topic that could be used in a Continuing Virtual Seminar;
  • a set of responses, by a learner, to Continuing Virtual Seminar questions;
  • a learner-designed rubric for any of these tasks, complete with the learner’s self-assessment; or
  • countless other tasks that I’ve never even imagined, but that Tres Columnae participants – and you, lectōrēs fidēlissimī – will eagerly propose and develop as the project goes along.

quid respondētis, amīcī?

  • What do you think about the idea of testing as quality control?
  • To what extent do you think such quality control is really necessary? Are there other ways besides testing to achieve quality control?
  • Is it possible to use a quality control test to give good results for individual learners to guide their progress? In other words, can you kill two birds with one stone when it comes to testing?
  • If so, how might we use these results? If not, what other types of assessment should be used for individual learners?
  • And is it possible for one activity to serve both as a learning opportunity and as an assessment?

I truly look forward to your responses! This is a critically important point, and one that we don’t talk enough about as a profession.

Tune in next time for a description of the Continuing Virtual Seminar, a portion of the Tres Columnae approach that will attempt to

  • test and assess learners’ progress with Connections and Comparisons, in particular;
  • function, simultaneously, as an assessment and a learning opportunity; and
  • build up the Community aspect of Tres Columnae – especially important since our learners will be geographically and chronologically dispersed.

In the meantime, please keep those emails and comments coming! et grātiās maximās omnibus legentibus et responentibus et responsūrīs!

Published in: on February 1, 2010 at 1:49 pm  Leave a Comment  

The URI to TrackBack this entry is:

RSS feed for comments on this post.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: