Parents prepare their kids for a big testing day with the right breakfast and a hug of encouragement. How does Pearson help states and other education agencies develop test questions for a big test day?
An Extensive Review Process
Before any question makes it onto a student’s test, many experts inside and outside of Pearson have reviewed the item, tested it out, and determined it is fit to be used.
Every state (or local education agency) has a different process for developing tests—but we tend to follow a 9-step process.
STEP 1: Item Writing
An item, also known as a test question, is created by an item writer.
To build a pool of diverse, authentic test items, Pearson contracts with professional item writers. In general, item writers need the following qualifications:
- Teaching or assessment experience in the subject
- Know how to align test questions to standards
- Experience writing items/passages
Because every test is different, we give the writers expert training specific to the assessment and requirements and needs.
STEP 2: Internal Item Review
Once Pearson gets the item from the writer, Pearson assessment specialists review it to make sure it is a good item.
To maintain consistent quality, Pearson assessment specialists evaluate each item to verify they are clear, accurate, and meet expectations. Sometimes items get rejected or sent back to the item writer to be improved. But, if the item is acceptable, the specialist verifies that the item meets the required criteria and sends it to a few more experts:
- A research librarian fact checks the item to make sure it is accurate
- An editor reviews for clarity, style, and grammar
- A graphic designer adds art or graphics such as charts or tables
STEP 3: Content Committee Review
Experts and educators representing the state review the item to make sure it fits the criteria of a good test item for the test and for their students.
STEP 4: Bias Committee Review
Each item has to pass a bias review by the state working with Pearson, so that every student has to have an equal chance to answer the item correctly.
Items that do not measure up to standards for fairness and sensitivity can affect the credibility of an assessment and its results. Pearson avoids content that might offend, unfairly penalize, or offer an advantage to students based on personal characteristics or culture.
STEP 5: Final Internal Review
After the state reviews each item, Pearson incorporates that feedback and makes edits.
An item that doesn’t pass this review cannot make it to the next stage.
STEP 6: Field Test
Now, the item gets put on its first test, which is called a “field test.”
Students answer the item, but their responses don’t count toward their test scores or teacher evaluations. If it is a new program, field tests might be held separately. Once a program is established, field-test items are usually embedded in the operational test.
STEP 7: Data Review
Pearson experts and the state look at data from the field test to make sure the item is performing as expected and it is giving customers the information they need. For example, if a group of students is struggling with an item significantly more than expected, we can remove that item from the pool.
STEP 8: Operational Test
If a test item passes all these reviews, it can be put on a test where it counts toward student scores.
STEP 9: Retirement
Once an item has been used too many times, it is ready for retirement and is no longer used on tests. By refreshing the items, students see test questions that haven’t been used before and the old items can be released to the public.