- Transitioning assessment systems
The educational technology marketplace is being transformed by the introduction of “tablets.” There is an increasing expectation that schools should be able to use tablets for high stakes summative assessment. The PARCC and SBAC consortia are also looking toward allowing delivery of their assessments to tablets. However, there is important research to be conducted to understand the full impact of introducing tablets into secure, summative assessment programs. This session will include the perspective of two states (Virginia and Maryland) on the tablet technology landscape as well as the perspective of two vendors (Pearson and eMetric) who are researching issues of usability, comparability, and security.
Pearson will discuss its ongoing series of usability studies, cognitive labs, and score comparability research designed to thoughtfully promote a transition of high stakes summative assessment to tablets while maintaining a student experience that is optimized for tablet delivery and which does not disadvantage students testing on tablets. While tablets offer an attractive option for creating engaging assessment experiences, it is important to examine several key differences in how students interact with tablets that could potentially result in advantages or disadvantages relative to students taking the test on a desktop computer. Tablet features such as screen size, virtual keyboards, and touch screen interfaces require special consideration in a high stake summative environment. For example, the virtual keyboard takes up a sizable amount of screen real estate and may cover critical elements of the item to which a student is responding. In many cases it is possible to design solutions for the tablet which can mitigate these concerns by optimizing the user interface for navigation, tool functionality, and display of test content such as reading passages. In other cases redesign alone may be insufficient to achieve comparability of student experience. In these situations, states should proceed carefully and may want to consider allowing tablets for only certain subject areas or item types. Pearson presenters will show how research is guiding interface design and assessment item rendering strategies that take tablet-based delivery into consideration.
Virginia and Maryland are two states actively involved in the tablet transition and have participated with Pearson in the tablet research effort described above. Specifically, Virginia has participated in a cognitive lab/usability study conducted with 10” Android tablets designed to evaluate the feasibility of assessing writing on tablets. The study evaluated student responding to a writing prompt when using the onscreen virtual keyboard as well as an external wireless keyboard. Additionally, both Maryland and Virginia have participated in a usability study conducted with iPads and 7” Android tablets to evaluate student interactions with a number of item types ranging from multiple choice to short answer math items to technology enhanced items. Members of the Virginia and Maryland Departments of Education will share their experiences with the studies, discuss iPad and tablet activity in their respective states in general, and describe what they see as the challenges for moving high stakes summative assessment to tablets.
eMetric's presentation will include a brief discussion about how technology plays a role in supporting the secure testing across varying devices. They will discuss the architecture that allows items/tests to be created in a single format and yet supported on multiple devices. Specifically they will discuss how HTML5 can be used to produce a “write once, run anywhere” solution that offers the flexibility to accommodate devices across the classrooms of today and tomorrow. They will also demo a set of items to show how they can be displayed on a PC, iPad and Android tablet.