Adaptive Tests
At Psiphas, our tests include adaptive tests: tests that adapt themselves to the test taker and identify their abilities with optimal precision.
The classical test
- A fixed-length test: every test taker is presented with the same number of items, in the same order, regardless of the test taker's ability level.
- The test score is usually the number of items the test taker answered correctly, regardless of the characteristics of those items (item difficulty, discrimination, etc.).
- For any given test taker, many of the items are too easy or too difficult and add little of significance to the estimate of that test taker's ability level.
The adaptive test
- Presents the test taker only with the items relevant to their ability level.
- Has a variable length.
- The content and order of the items vary from one test taker to the next; in theory, the tests given to two different test takers may not have a single item in common.
- In an adaptive test, the test taker's ability level is estimated by mathematical means, as the level that best explains the test taker's response pattern.
The adaptive test: the testing process
The adaptive testing process can be described schematically as follows:
-
A first item is presented to the test taker (this item may be the same for all test takers, selected from a group of items of a given difficulty level, or selected at random from the entire item bank).
-
The test taker answers the item.
-
The test taker's answer is checked, and it determines whether the next item will be more or less difficult than the current one.
-
The new item is presented to the test taker, who answers it.
-
Based on all the information gathered about the test taker (not only their answer to the last item), the test taker's ability level is estimated, along with the stability of that estimate. If the estimate is stable enough, the test ends. If not, another item is selected and presented to the test taker. This continues until a stable estimate of the test taker's ability is reached.
Advantages of the adaptive test
Test duration
Because the items presented to the test taker are relevant to their ability level, the test taker is spared items that are too easy or too difficult. In practice, an adaptive test includes about half as many items as a traditional test, and testing time is cut in half.
Improved test-taker experience
As noted above, the test taker is presented with items suited to their ability level, which strengthens their sense that the test is "fair."
Item "survival"
Each test taker is presented with the items relevant to them. As a result, items are generally less exposed, which extends the life of the test.
A nervous candidate
In a classical test, this candidate would be "penalized" for incorrect answers. In an adaptive test, the system adds questions until it finds the candidate's true ability range.
Testing from home
Testing from home has many advantages. The disadvantage is that there is no way of knowing whether the candidate took the test alone. The professional literature suggests the following: if you decide to proceed with the test taker, have them retake, as a verification test, the adaptive test on which they scored highest at home. The system remembers the questions given at home and will select different ones.
Preventing copying
Even if two test takers sitting next to each other receive the same test battery and work at the same pace, the items each of them receives are likely to differ from the other's (even if their ability levels are similar). This prevents copying.
Parallel tests
A score on an adaptive test is not tied to specific items (in other words, the same score can be reached with different items). As a result, administering "parallel" tests (a complex task under classical test theory) becomes very simple. This opens up new possibilities (such as unproctored testing from home, with the results of some of the tests verified in a proctored session).
Reducing the guessing problem
In a classical test, when only a few seconds remain, the candidate guesses, which adds "noise" to the test results. In an adaptive test, each question has its own time limit and the test taker does not know whether it is the last question, so this problem is reduced.
