Criterion, a web-based, instructor-led writing evaluation tool, is revolutionizing the ways professors and instructors at community colleges and minority serving institutions (MSIs) teach students necessary skills for writing competency.
“Disposable time is often a luxury MSI and community college students don’t have,” says Chrystal Anderson, a client relations specialist for ETS. “Criterion’s technology frees up time for institutions to focus on teaching, not correcting, and it gives more opportunities for students to practice and improve their individual writing skills.”
Many community colleges and MSIs face the added pressure of staying accountable for their students’ success, and “a large portion” of community college and MSI students are placed in remedial English courses that can be a “path that often contributes to these students never completing their degrees,” says Andrea King, director of project management for ETS.
ETS began to develop the writing evaluation tool in 2000 and the system is powered by ETS’ proprietary software engine called ‘e-rater.’ After a student completes an assignment given by an instructor, the e-rater scoring engine provides the student with annotated diagnostic feedback and a holistic score “based on level-specific models built from essays pre-scored by ETS-trained readers,” the Criterion website reads. The e-rater’s automated grading closely matches human scorers’ grading.
To access the tool, an institution, a specific department or a professor can order the Criterion service. Students must then log in to the online system using an access code provided by their instructor.
While Criterion cannot take the place of teacher instruction and feedback or grade specifically for content, using the tool in a writing course saves time, provides consistency for grading and can empower professors and instructors to focus on higher-level student learning, King says.
At Southwestern Christian College, Joyce Cathey, who is the institutional effectiveness chair, says faculty — especially those members outside the humanities division — can focus more on a student’s writing content and they “don’t have to worry about grading the conventions, grammar and organization . . . because they can feel certain that the Criterion tool is going to take care of all of the conventions.”
She says that administrators and faculty members at the school feel that students “get the best of both worlds,” by having the specialized content focus and annotated evaluation necessary to achieve student learning outcomes.
Yvonne Singleton, the director of the writing center at Voorhees College, says students come in to her center to use Criterion to evaluate essays and writing assignments for various courses.
Voorhees is a smaller institution in South Carolina and Singleton found that students attending the college sometimes lack the fundamental basics of the mechanics and grammar of writing.
Singleton complements the students’ use of Criterion with her own feedback and guidance at the writing center. “Sometimes I sit with them and I go over some changes and things that they need to make in their paper to make the corrections and make it more cohesive,” she says.
Keeping up with ETS’ notion that Criterion should not replace teacher instruction and feedback, Singleton says that the tool “does a good job of pointing out things, but sometimes, there are situations and things where Criterion will suggest a change when the change is not really necessary.”
As a result, Singleton builds her students’ writing confidence by telling them: “‘You have to read it to make sure that’s what you want. The program will make some suggestions sometimes that are not what you want to make changes to. You have to just know that that’s the right word, that’s the right phrase, that’s how you want to say it.’”
The Criterion representatives add that instructors of first-generation or high-risk students who may not have the study skills or self-motivation as other students can personalize their use of the tool by integrating it as a mandatory part of their course, altering Criterion’s grading scale, or creating their own prompts outside the expansive topic library.
Notably, instructors can create assignments using more than 400 prompts from old ETS tests and writing prompts within Criterion’s topic library. Writing levels for various topics range from the fourth-grade level to the graduate level, as well as prompts for the Test of English as a Foreign Language (TOEFL).
And because the tool is self-paced, students who work multiple jobs, or who may not have internet access at home, can practice at any time of the day and anywhere with internet access — such as a campus library. This is beneficial for students who have children, other family responsibilities or not having access to transportation to meet with their instructors outside of class, Anderson says.
Criterion’s setup provides leeway for students to address grammatical and mechanic issues on their own, Singleton says, adding that she has seen students who are concerned about making the necessary corrections to make their papers better, and students who “just want to complete the assignment and turn it in,” she says laughing. “But I tell them and try to encourage them that if you read the [Criterion] information and follow it and follow the prompts and the suggestions that the program gives you, in the long run, your writing will improve.”
This is the ultimate goal for ETS, especially for MSIs and community colleges.
“Writing is a learned skill that requires guided practice,” Anderson emphasizes. “That’s where ETS believes our Criterion online writing tool can help students and institutions overcome nonacademic obstacles that can hinder student success.”
Students whose professors use the Criterion tool no longer have to wait for that professor to grade stacks of papers for multiple firstand second-year English courses, and they can receive immediate writing feedback on an assignment in “15 seconds or less,” Anderson adds. The automated feedback frees up time for a professor to focus more in-depth on the quality, content and style of students’ work.
Focusing on five key categories — grammar, usage, mechanics, style and organization and development — Criterion structures its feedback in a manner helpful for students to complete multiple revisions of their writing assignment, ultimately strengthening their writing process and highlighting problem areas that may need more work.
Additionally, professors and instructors have the option of providing their own teacher comments in a student-teacher dialog box, and there is a new “Peer Review Tool” within the Criterion service that allows peer groups to view and comment on each student’s work. A Writer’s Handbook is also included for multiple education and language levels.
Despite decreasing funding and increasing scrutiny of higher education, more MSIs and community colleges are looking for ways to improve their effectiveness, “often measured by learning outcomes and student completion,” Anderson says. “Criterion’s reports help identify areas that need to be addressed, at the class or cohort level.”
The Criterion data is presented in graphic snapshots, giving professors and writing instructors clear visuals of their entire class’ or an individual student’s writing performance for an assignment or over a period of time. The service also gives higher education administrators the necessary information to benchmark their students’ writing proficiency and make important data-driven institutional decisions.
“Criterion was not designed for higher-ed institutions to use in high-stakes decision making, like admissions or course placement, for example,” Anderson says. However, the “data can help professors and department heads make more accurate curriculum and instruction decisions to address student needs.”
Administrators and officials at Southwestern Christian College are doing this by using Criterion to support the institution’s quality enhancement plan through writing, says Cathey, the institutional effectiveness chair. She says that writing was previously an area of weakness for students, but now to address that, every operational unit of the college must support the strategic plan to improve student writing outcomes. “That means that all of our courses will have some writing component” and all classes will have student learning outcomes.
“We use the writing component to review their work, and then refer them to our center for student success for additional supplemental instruction,” Cathey adds. “We find that that is moving us closer to mastery in the various components. And our center for student success is reporting that [students] can come in independently and know exactly what to ask for in terms of their needs, based upon the Criterion evaluation. It’s a win-win situation.”
A plan for the future
Criterion specialists cite a report by the Georgetown Center on Education and the Workforce that shows that a portion of well-paying jobs do not require college degrees. Nonetheless, the report reveals that individuals with college degrees or some college are obtaining higher portions of these opportunities, Anderson says.
King acknowledges that the ability to write well can help open career pathways. This comes as more employers in the workforce are now beginning to emphasize the importance of writing skills for today’s college graduates.
“When you have a job or something in that area that requires you to do a lot of writing, your writing must be clear and concise [so] that people who read it will understand what you’re writing,” says Singleton, the Voorhees writing center director. She adds that Criterion helps her students incorporate different phrases and sentence styles in their own writing, in addition to helping them become more aware of the clarity of their writing.
Moreover, the element of self-learning and time management that comes with using the Criterion service for writing assignments aligns with some of the skills and values that employers want. Cathey says Criterion creates readiness, confidence and brings out self-efficacy in students so that they are better able to articulate and market themselves when they enter the workplace.
“They feel confident in writing a personal statement,” Cathey says. “They feel comfortable in expressing their philosophy or ideologies or whatever they’re asked for in writing.”
As institutions prepare their students holistically for graduate school or the world of work, ETS hopes that the writing evaluation tool will help community college and MSI students. “Over time, our hope is that Criterion helps level the playing field by helping to strengthen these students’ writing skills, assist in keeping students on a completion pathway, and these students can be more prepared to compete in today’s workforce,” King says.
- This story also appears in the 21, 2017 print edition of Diverse.
Tiffany Pennamon can be reached at [email protected].
