The Rise of the Machines

Elective 3 – E-Assessment of Writing

http://blog.modernmechanix.com/school-uses-machine-to-mark-all-examination-papers/

The Rise of the Machines

Sometimes when I am at school, I choose to grade a pile of papers or student journals while sitting in the faculty lounge.  It is a room with some space for me to get comfortable as I prepare to make a dent in a stack of assignments that I know will take me hours and hours to get through.  As I read through pages of student writing, correcting the same mistakes again and again, it is easy for me to get a little frustrated with my choice in profession.  This frustration only increases when a history teacher or science teacher strolls into the room with a stack of Scantron sheets and goes to the machine in the corner.  They are able to grade their tests in two minutes, while I am still trying to repair the introductory paragraph on the first paper.  I will joke with these teachers about running my stack of papers through the machine, and a part of me wishes that I could. 

The reading that I did this week showed me that this type of computer-assessment of writing is possible, but may not be the sophisticated enough to rely on entirely.  (Note: I also believe that Scantrons probably shouldn’t be used so prominently in the classes of some of my colleagues, but that is another conversation).

Vojak, Kline, Cope, McCarthey, & Kalantzis (2011) really provided me with a strong introduction to the world of electronic writing assessment, and I feel that I understand the potential benefits and drawbacks of the tools that are out there.  One line from the article that really got me thinking about this issues was the CCCC’s 2004 position statement on digital assessment:

Writing to a machine, they argue, “violates the social nature of writing” and is thus detrimental to students’ understanding of writing as a uniform of communication.  In addition, the CCCC questions the validity of machine scoring and the criteria on which machine scoring is based; the CCCC raises concerns the machine scoring of college papers will lead to high schools preparing students to write for machines rather than for human audiences. (p. 98)

Teaching to the standardized test has already infiltrated education enough, and so it would be very detrimental if teachers had to start teaching writing with a machine-based assessment in mind.  Vojak et al. established three key understandings about writing with this “social nature” in mind.  According to Vojak et al. writing is socially situated, functionally and formally diverse, meaning-making, and can be conveyed in multiple modalities.  With these key understandings in mind, it is clear that digital writing assessment cannot fully support a student’s understanding of the writing process.

When reading about the digital-assessment of writing, I was interested to read the strengths of some of the programs.  Their accuracy when compared to a human scorer is impressive (AES scores within a single point of human grader 97% of the time), and I like how programs like CPR incorporate peer scoring, teacher scoring, computer scoring, and self-scoring.  A lot of these programs are innovative and truly amazing, but they are still flawed.  The purpose of assessment is not to merely provide an accurate grade, but it should also help the student improve.  Without meaningful and direct feedback from a person, I don’t think students will have much motivation to improve.

However, these tools should not be neglected completely.  I think that teachers can use them as supplements.  These programs can be “used to facilitate students’ early drafting and revising process, followed by human feedback from both the teacher and peers in the later process” (Vojak et al., p. 104).  A little computer feedback on the front end of the writing process, might allow teachers to focus on other more important aspects of the writing later in the process.  With a little more focus on these other aspects, teachers might be able to more readily embrace the increasingly multi-modal nature of writing.

Cope, Kalantzis, McCarthey, Vojak, & Kline (2011) also made a lot of great points about digital writing tools.  Much of their article was a little challenging for me to fully grasp, but when I finished it I was blown away by one of their ideas.  They suggest more assessments rather than less.  Initially this suggestion might seem like a call for tests everyday, but that is not the case.  Rather, they suggest continual formative assessments that make summative assessments less daunting:

We imagine a time in the not-too-distant future when there is no practical distinction between formative and summative.  These two forms of assessment will only ever be used to measure the same thing, for learning (as you go) and progress in learning (retrospectively) will converge.  If and when the day arrives, there will be no need for the problematic end-of-program, high-stakes, or accountability defining tests that we have today.  Formative assessment will be pervasive—a repertoire of modes of machine-assisted social intelligence that provides continuous feedback to learners in a rapidly responsive manner and with sufficient substantive feedback.

I apologize for the length of the quote, but it really seems to provide an image of a utopian world of education.  Perhaps it is too ambitious, but I appreciate the attempt to move forward and dream about the possibilities.

Here is a petition I stumbled on this week when I was researching the topic.  I found it interesting:

http://humanreaders.org/petition/

For My Classroom
 I think that the best thing that I can do for my classroom would be to experiment with some of these writing programs this summer.  I am especially interested in Calibrated Peer Review (because it's free), but I am not yet very comfortable with any of these programs. I know that these programs will not "do the grading for me" but I think that they might be able to supplement my writing instruction in a way that is beneficial for my students.

Works Cited:

Cope, B., Kalantzis, M., McCarthey, S., Vojak, C., & Kline, S. (2011). Technology-Mediated Writing Assessments: Principles and Processes. Computers and Composition, 28(2), 79–96.

Vojak, C., Kline, S., Cope, B., McCarthey, S., & Kalantzis, M. (2011). New Spaces and Old Places: An Analysis of Writing Assessment Software. Computers and Composition, 28(2), 97–111.

(Note: Sorry for the length.  I wrote most of this response before watching this week's Voicethread.  I promise my next post will be shorter.)