Assessing Writing

Background and Theory

Works in this section will provide the reader with a foundation in basic assessment theory as well as the historical background necessary to fully appreciate the current theory and practices in writing assessment.

Cooper, Charles R. and Lee Odell, eds. Evaluating Writing: Describing, Measuring, Judging. Buffalo, NY.: State University of New York at Buffalo, 1977.

A compilation of essays which provided a comprehensive summary of the best information on the evaluation of writing at the time it was published. It was presented as a work to be used by writing researchers, curriculum evaluators, and classroom instructors at the secondary and college level. All the essays are concerned with direct measurements of writing.

Ebel, Robert L. and David A. Frisbie. Essentials of Educational Measurement. 4th ed. Englewood Cliffs, NJ. : Prentice-Hall, Inc., 1986.

Basic textbook for educational measurement addresses assessment issues of validity and reliability as well as test construction and interpretation of test scores. Chapter 8 focuses on the essay test.

Godshalk, Fred I., Francis Swineford, and William E. Coffman. The Measurement of Writing Ability. New York: College Entrance Examination Board, 1966.

This classic study proved the predictive validity and reliability of assessing writing by using objective or indirect measures while at the same time showing that validity and reliability can be slightly enhanced by the inclusion of a short essay read holistically. It is an important though somewhat technically complex work in the development of writing assessment.

Greenberg, Karen L., Harvey S. Wiener, and Richard A. Donovan, eds. Writing Assessment: Issues and Strategies. New York: Longman Inc., 1986.

The issues and strategies in writing assessment identified in this edited collection of essays grew out of the shared experiences of writing teachers attending the first two conferences of the National Testing Network in Writing (NTNW), 1983 and 1984. The essays in this book chronicle the history of writing assessment, identify current practices and problems in writing assessment, present research on writing assessment, and indicate the direction future work on writing assessment should take. Includes an annotated bibliography.

Ruth, Leo and Sandra Murphy. Designing Writing Tasks for the Assessment of Writing. Norwood, New Jersey: Ablex Publishing Corp., 1988.

The authors address what they consider to be a "neglected variable" in writing assessment and research--that of the specific writing task and its interpretation by the writer as well as by the reader/assessor. Little research has been done on what makes a good topic and the authors discovered that the topic, the method and language with which it is introduced to the writer, the rhetorical aspects demanded by the task and several other variables all have major effects upon the quantity and quality of writing being produced. The work includes guidelines for producing writing assignments.

White, Edward M. Teaching and Assessing Writing. San Francisco: Jossey-Bass Publishers, 1985.

White very clearly details all elements needed to develop a successful program of writing assessment from topic development to statistical analysis of results. This work includes a chapter on the specifics of conducting a holistic scoring reading session. Required reading for those working in writing assessment.

Current Issues

Items listed in this section address more recent issues in writing assessment and will provide the reader access to the current dialog on assessment issues.

Bers, Trudy H. and Kerry E. Smith. "Assessing Assessment Programs: The Theory and Practice of Examining Reliability and Validity of a Writing Placement Test." Community College Review 18.3 (1990): 17-27.

This article reports the results of examining an assessment program used for college freshman placement. The authors conclude that although there are real benefits for using holistically scored impromptu student essays to place incoming students, many other factors combine with writing ability to determine student success in freshman writing courses.

CCCC Committee on Assessment. "Writing Assessment: A Position Statement." College Composition and Communications 46.3 (1995) : 430-437.

This statement enumerates all the major issues involved in writing assessment, details the responsibilities of students, faculty, administrators, and legislators in participating in assessment decisions and programs, and includes selected references.

Hamp-Lyons, Liz and William Condon. "Questioning Assumptions about Portfolio- Based Assessment." College Composition and Communication 44.2 (1993) : 176-190.

A careful investigation of an established portfolio assessment used to determine exit scoring revealed some commonly held assumptions about portfolio assessment. After examining each of five assumptions, the authors conclude that increased accuracy is not an inherent virtue of portfolio assessment but must be continually pursued through constant attention to goals and systematic evaluation of the program.

Huot, Brian. "Reliability, Validity, and Holistic Scoring: What We Know and What We Need to Know." College Composition and Communication 41.2 (1990) : 201-213.

Huot claims that in current practice too much emphasis is put on the reliability of writing sample scoring leaving validity largely assumed. As holistic scoring practices have become widely accepted much work has been done to demonstrate the reliability of such measures; now Huot argues we need to turn the same critical attention to the validity of holistic scoring practices.

Jones, Elizabeth et al. National Assessment of College Learning: Identifying College Graduates' Essential Skills in Writing, Speech and Listening, and Critical Thinking. US. Department of Education, Office of Educational Research and Improvement, 1995.

This final project report outlines the activities undertaken by the National Center for Educational Statistics to first identify and then to reach consensus among three groups, faculty, employers, and policy makers, as to what constitutes advanced communication abilities in response the Goal 5 of the nation's education goals 2000. The work divides the areas of writing ability, speech communication, and critical thinking into subskills that are to be mastered. Although the report does not deal directly with assessment, it could represent a turn away from holistic approaches back to assessing individual subskills.

White, Edward M. "An Apologia for the Timed Impromptu Essay Test." College Composition and Communication 46.1 (1995) : 30-45.

In response to recent attacks on the use of the impromptu essay test, White reviews the strengths of the essay as compared with the indirect methods of assessment it replaced. Since the advancement of portfolio assessment, the impromptu essay test is seen less favorably than it was twenty years ago, but White defends it on several grounds placing it as more valid than the multiple choice alternatives and more efficient than the portfolio assessments. All three methods of writing assessment are evaluated.

-----. "Assessing Higher-order Thinking and Communication Skills in College Graduates through Writing." JGE: The Journal of General Education 42.2 (1993) : 105-122.

In this article White addresses in more concise form some of the same issues covered in his 1985 book. He presents the advantages and disadvantages of the three types of writing assessment: multiple choice tests, essay tests, and portfolio evaluation. He strongly favors portfolio evaluation since it is the only method of assessment that allows the writing process to be considered.

-----. Teaching and Assessing Writing. Second Edition. San Francisco: Jossey-Bass Publishers, 1994.

Updated edition of the 1985 work presents a general overview of the current teaching and assessing of writing. White addresses both classroom and large scale assessments and includes in this work portfolio assessment. White presents a balanced look with admonitions and advice about all the methodologies currently being used to assess writing.

-----. "Language and Reality in Writing Assessment." College Composition and Communication 41.2 (1990) : 187-200.

Writing assessment is perceived differently by writing instructors and by psychometricians. White gives three suggestions for resolving the conflict that results from these differing perceptions. He tells writing instructors to first recognize the limits of our perceptions about writing and assessment, to learn the language of those who hold differing perceptions, and lastly to listen more than we speak when first encountering differing perceptions.

Williamson, Michael M. and Brian A. Huot. Validating Holistic Scoring for Writing Assessment: Theoretical and Empirical Foundations. Cresskill, NJ: Hampton Press, Inc., 1993.

This current compilation of essays focuses on trends in the field of writing evaluation in the past several years since the widespread adaptation of holistic scoring. It looks at the limitations of current theory and practices in view of our growing awareness of the contextualized nature of writing. As the model for writing changes, so must the model for assessing writing.

Practices

This section includes writings on the implementation of assessment theory. Many entries are narratives detailing instructors' experiences with some facet of assessment.

Beach, Richard. "Demonstrating Techniques for Assessing Writing in the Writing Conference." College Composition and Communication 37.1 (1986) : 56-65.

In this interesting article Beach describes an informal assessment method used during writing conferences that teaches the student to critically assess his/her own writing. The technique involves a series of questions which require the student to address rhetorical issues of purpose, organization or support, audience, and finally effectiveness.

Belanoff, Pat and Marcia Dickson, eds. Portfolios: Process and Product. Portsmouth, NH: Boynton/Cook Publishers, 1991.

The first section of this book is a compilation of eight essays on portfolio usage for proficiency testing. The second section is concerned with the use of portfolios to assess programs, and the third section deals with the political issues involved in portfolio assessment.

Courts, Patrick L. and Kathleen H. McInerney. Assessment in Higher Education: Politics, Pedagogy, and Portfolios. Praeger, 1993.

Chapter 4 addresses portfolios as an assessment of student learning. The objectives in setting up this portfolio assessment were to improve instruction and to enhance student involvement and responsibility for learning.

Hamp-Lyons, Liz and William Condon. "Questioning Assumptions about Portfolio- Based Assessment." College Composition and Communication 44.2 (1993) : 176-190.

A careful investigation of an established portfolio assessment used to determine exit scoring revealed some commonly held assumptions about portfolio assessment. After examining each of five assumptions, the authors conclude that increased accuracy is not an inherent virtue of portfolio assessment but must be continually pursued through constant attention to goals and systematic evaluation of the program.

Harris, Charles B. "Mandated Testing and the Postsecondary English Department." ADE Bulletin 104 (1993) : 4-13.

In this two-part article Harris first chronicles the increasing reality of state mandated assessment at the post secondary level and then goes on to describe a modified version of the senior seminar which his institution, a mid-western four year university, is using to assess student growth in writing. Students majoring in English maintain a portfolio of their writings from beginning to end of their academic careers; these portfolios become the basis of the final assessment of students' intellectual and personal development as English majors.

Haswell, Richard and Susan Wyche-Smith. "Adventuring into Writing Assessment." College Composition and Communication 45.2 (1994) : 220-236.

This article narrates the experiences of two writing instructors who became involved first with a large scale writing placement assessment at a Pacific Northwest land grant university and later with rising junior exams at the same institution. They discovered and discuss some helpful rules of thumb for English faculty faced with assessment dilemmas. Their message is get involved in the process in order to get what you want from the assessment.

McNew, Janet and Cindy Malone. "Assessing the English Major: A Case Study." ADE Bulletin 107 (1994) : 28-30.

In this article which deals with curriculum or program assessment, the authors give some insight into the principles of sound assessment. Their advice is to begin with the goals or objectives and then determine through the assessment if those objectives have been met.

Odell, Lee. "Writing Assessment and Learning to Write: A Classroom Perspective." Theory and Practice in the Teaching of Writing: Rethinking the Discipline. Ed. Lee Odell. Carbondale: Southern Illinois University Press, 1993. 289-313.

Odell looks at the ways that two large scale assessment practices, primary trait scoring and portfolio assessment, can be combined and adapted to create an evaluation technique viable for classroom writing assignments that will meet student needs for meaningful writing assessment at the same time it promotes student growth in writing.

Odell, Lee and Sally Hampton. "Writing Assessment, Writing Instruction, and Teacher Professionalism." A Rhetoric of Doing: Essays on Written Discourse in Honor of James L. Kinneavy. Eds. Stephen P. Witte, Neil Nakadate, and Roger D. Cherry. Carbondale: Southern Illinois University Press, 1992. 276-290.

Working with the teachers and administrators of the Forth Worth Independent School District on making the best use of the Texas Assessment of Academic Skills, Odell and Hampton demonstrate that state-wide writing assessment can lead to improved writing instruction and increased teacher professionalism when careful attention is given to understanding the assessment tool, to analyzing student writing produced in response to the assessment, and to developing teacher in-service programs which enhance teacher effectiveness.

Sommer, Robert F. Teaching Writing to Adults. San Francisco: Jossey-Bass Publishers, 1989.

In his chapter on reading and responding, Sommer describes the traditional methods of writing evaluation and explains why they are counterproductive to the growth of the returning adult student. He suggests more non-traditional methods including holistic scoring and naturalistic methods.

Yancey, Kathleen Blake, ed. Portfolios in the Writing Classroom. Urbana, IL: National Council of Teachers of English, 1992.

This essay compilation chronicles the experiences of several writing instructors and program directors in dealing with portfolios in the classroom. This work presents multiple perspectives of portfolio usage and includes an annotated bibliography.

Additional Resources

The entries in this section list sources for further reading/research on assessment. In addition to these specific resources, all other entries include bibliographic information.

CCCC Committee on Assessment. "A Selected Bibliography on Postsecondary Writing Assessment, 1979-1991." College Composition and Communication 43.2 (1992) : 244-255.

This bibliography compiled for postsecondary writing instructors includes only articles from major scholarly publications for the dates listed. It is divided into the following categories: large scale assessment, classroom writing assessment, theory, politics and issues, methods and procedures, program assessment, and networking resources.
College Composition and Communications 46.3 (1995) : 430-437.

This statement enumerates all the major issues involved in writing assessment, details the responsibilities of students, faculty, administrators, and legislators in participating in assessment decisions and programs, and includes selected references.

Greenberg, Karen L., Harvey S. Wiener, and Richard A. Donovan, eds. Writing Assessment: Issues and Strategies. New York: Longman Inc., 1986.

The issues and strategies in writing assessment identified in this edited collection of essays grew out of the shared experiences of writing teachers attending the first two conferences of the National Testing Network in Writing (NTNW), 1983 and 1984. The essays in this book chronicle the history of writing assessment, identify current practices and problems in writing assessment, present research on writing assessment, and indicate the direction future work on writing assessment should take. Includes an annotated bibliography.

Yancey, Kathleen Blake, ed. Portfolios in the Writing Classroom. Urbana, IL: National Council of Teachers of English, 1992.

This essay compilation chronicles the experiences of several writing instructors and program directors in dealing with portfolios in the classroom. This work presents multiple perspectives of portfolio usage and includes an annotated bibliography.