Preventing, Detecting, and Investigating Test Security Irregularities: A Comprehensive Guidebook On Test Security For States

Thursday, June 20, 2013: 8:30 AM-10:00 AM
National Harbor 6 (Gaylord National Resort and Convention Center)
  • TILSA Test Security Guidebook - NCSA presentation by John Olson - June 2013.ppt (254.0 kB)
  • NCSA Conference 2013 Presentation slides.ppt (1.9 MB)
  • NCSA_2013_TestSecurity_WV.pptx (1.7 MB)
  • Content Strands:
    1. Implementing state and federal programs and policy
    2. Improving data analysis, use, and reporting
    Test security of state assessments is an increasingly important topic as media across the nation buzz with stories of cheating in schools. As the stakes in state assessments increase, more instances of cheating or improper behavior by educators are being discovered. Cheating and test piracy pose major threats to the validity and credibility of state assessment programs.  The soundness of test results requires systematic efforts in the design, development, and maintenance of these programs. This session describes a new, invaluable resource for states to use in preventing, detecting, and investigating testing irregularities.

    Because of all the recent attention on improving the security of assessment programs, in 2012 the Technical Issues in Large Scale Assessment (TILSA) SCASS created a special project to address the topic, assist states in dealing with potential problems, and develop a Comprehensive Guidebook on Test Security for States.  TILSA state members and experts on the topic collaborated on the contents of the Guidebook so that the information is comprehensive in its scope and practical and useful to state assessment staff.

    Published in February 2013, the main purpose of the Guidebook is to provide resources that states can use for preventing and detecting irregularities, such as cheating and test piracy on state assessments, and for investigating suspected or confirmed examples of improper/unethical testing behavior by either students or educators.  Among the many issues that are addressed is the implementation of solid security procedures for high stakes assessments—at both state and local levels—and the need to improve the professional testing practices of teachers. The Guidebook also provides depth on key technical topics such as data forensics analysis so that state staff and others involved in testing will find it helpful in their ongoing work.

    The three main sections included in the Test Security Guidebook provide numerous details for security practices on the following activities:

    Prevention – Standards for test security aspects of the design, development, and operation of state assessment programs, both paper-and-pencil and computer-based, for both multiple-choice and performance items.

    Detection – Guidelines for planning, implementing, and interpreting data forensics analyses.

    Follow-Up Investigations – Strategies for states when planning/conducting investigations and actions they need to take based on findings from investigations.

    In addition, many practical documents and resources are included in the Guidebook, e.g.,: (1) sample checklists to improve security prevention processes for each phase of an assessment; (2) example outline of a Test Security Manual; and (3) model Investigations Kit with sample interview guides, forms, and suggested allocation of responsibilities.

    A crucial area where the Guidebook can be applied within states is with the promotion of professional best practices for school administrators and teachers.  An important objective of this project is to guide educators on appropriate ways to conduct testing in their schools, with the hope of improving the procedures being used and the overall validity and fairness of the assessment results.  Another objective is to help states meet the challenge of maintaining the integrity of their operational programs while transitioning to new tests with higher stakes.  A goal of this session is to provide a forum for states to share the best available resources and to help them make these transitions while improving security to improve the quality of their test data, use, and reporting.

    In this session, the TILSA-SCASS advisor will provide an overview of the project, two PIs will present details of the contents of the Guidebook, two state representatives will share insights on how they are improving security for their assessment programs, and a nationally-recognized expert will discuss implications for the field.  Copies of the Test Security Guidebook will be available for attendees.