Marine Hovhannisyan taking a pilot computer-based UEE mathematics test at Yerevan High School No. 42. Photo credit: Arevik Badalyan / World Bank
Marine Hovhannisyan is a 17-year-old student at Yerevan High School No. 42. She is aspiring to study at the Yerevan State University¡¯s department of Applied Mathematics. To be considered for university admission she needs to take the Unified Entrance Examination (UEE). Along with taking the paper-based version of the UEE, Marine is also taking part in a pilot of a computer-based version of the mathematics test that is part of the UEE. The pilot aims to show whether students¡¯ results would differ based on the version of the exam they take. According to Marine, ¡°taking the test on a computer has turned out to be useful for me. Sometimes, I accidentally leave out some of the test items and rush to the next parts of the test. The computer clearly showed the items I left untouched, thus, I marked them as well and eventually submitted a fully completed test.¡±
The UEE plays a vital role in Armenia¡¯s education system. This year alone, over 4,300 high school students will take the high-stakes exam, which decides whether they graduate from secondary school, and which higher-education institution they are admitted to. The Assessment and Testing Center oversees the UEE as well as large-scale national and international assessments in Armenia. In 2007, the Assessment and Testing Center piloted the UEE in Armenian Language and Literature, applying standardized test procedures and modern technologies. Since 2009, the UEE has covered all secondary-school subjects.
Objectivity in measuring learning outcomes, including through transparent test administration procedures, with same-day administration and grading across Armenia are core principles of the UEE. Still, it has had its challenges. Public confidence in the exam has been threatened by instances of inappropriate behavior during the examination process, and by limited comparability of examination results between subjects and across years. Exclusive reliance on multiple-choice questions limits the scope of testable content, and the necessity of private tutoring to pass some UEE subject tests raises equity concerns.
Armenia has focused on improving the overall efficiency of UEE administration, increasing public confidence, and improving test quality. After assessing needs, costs, challenges, and risks of various approaches (under the first Russia Education Aid for Development (READ) Trust Fund program), Armenia has committed to a gradual introduction of computer-based testing for the UEE under the READ Trust Fund.
Currently, a READ grant is supporting the design and piloting of computer-based testing in mathematics, a core UEE subject. The focus is on ensuring alignment of the computer-based test with both the State Standard for education and State Standard for mathematics curriculum. The second stage of the computer-based UEE pilot commenced in February 2018, covering over 700 students in the capital city of Yerevan ¨C including Marine ¨C who took both paper- and computer-based tests.
Students taking both paper-based and computer-based UEE tests as a pilot at Yerevan High School No. 42. Photo credit: Arevik Badalyan / World Bank
On the day of her computer-based test Marine came to the exam center where a workstation with a laptop was set up for her. The laptop is protected against security breaches and has software to prevent loss or misuse of student data. She logged into her personal account created in the system for all students taking the test. She received a short briefing on how to use the computer for answering UEE mathematics questions, as the computer-based format requires basic computer literacy skills. After this, Marine had 90 minutes to answer 40 multiple-choice and short-answer questions. To minimize cheating and increase fairness, four versions of the exam had been developed, so Marine¡¯s neighbors were seeing different questions on their screens. Grading of the computer-based test is done automatically, so Marine received her results immediately after she completed the test.
The next day, Marine arrived at the same location to take a paper-based UEE mathematics test, which also lasted for 90 minutes and measured the same skills but consisted of 40 different (and only multiple-choice) questions, with results announced within two hours after submission of the exam. When Marine turned in her paper-based exam, she was asked to complete a questionnaire that collected information on her computer literacy, her experience of taking the computer-based test, and her socio-economic status. This survey will be used to examine the impact of computer-based testing on students¡¯ performance motivation, attitudes toward paper-based versus computer-based tests, and test performance of Marine and other test takers like her.
Trained test administrators ensure integrity and honesty during the computer-based test. With the support of the READ Trust Fund program, Sona Karakhanyan, one of the test administrators in Marine¡¯s exam center, received prior training on how to administer the computer-based version of the UEE. On the day of the exam, Sona ensured that the workstations were ready for the test takers, checked in each test taker against the registered roster, and provided clear instructions about the testing process. During the test, Sona walked through the exam room and ensured that test takers adhered to the exam rules.
While the results of the computer-based testing UEE pilot are yet to be analyzed, it is expected that the introduction of computer-based testing across the country will help Armenia¡¯s students better demonstrate their knowledge and skills and ensure that their knowledge is assessed in a more efficient, fair, and objective way.
For an overview of accomplishments in Armenia under the first READ Trust Fund program, see the video below.
