The Critical Role of Policy in Maintaining Test Security

Thursday, June 20, 2013: 10:15 AM-11:45 AM
National Harbor 6 (Gaylord National Resort and Convention Center)
Presentations
  • NCSA 2013 - Clark Test Security Presentation Final.ppt (223.5 kB)
  • 2013 CCSSO MDE Security.pdf (141.4 kB)
  • TestSecurityPresentation_CCSSO_June2013_TamaraLewis.pdf (4.5 MB)
  • Florida.CCSSO.TestSecurity.Final.ppt (373.0 kB)
  • Handouts
  • Sample_Erasure_Analysis_Report.pdf (39.2 kB)
  • Content Strands:
    1. Implementing state and federal programs and policy
    ABSTRACT:
    Considering the increased reliance on test scores for high-stakes decisions, numerous recent high-profile cheating scandals, and occasionally mishandled responses to suspected cheating incidents, much attention has been placed on test security. Sound policy is the cornerstone of a responsible test security plan. During this session, representatives from several states and one testing organization will share perspectives on responsible test security policies. Experiences and recommendations will be offered by session participants, followed by feedback and perspectives by the session discussant.

    With allegations of misconduct being reported with alarming frequency in recent years, the topic of cheating on assessments has become pervasive in testing news. The stakes in testing outcomes continue to rise for both students and educators, with test scores sometimes being directly tied to students’ opportunities for grade promotion or graduation and teachers’ evaluations, which have at least partially contributed to the creation of new incentives for stakeholders to engage in misconduct in an effort to improve scores on state assessments. These incentives, coupled with sometimes lacking or outdated test security policies, have contributed to an apparent increase in reported cheating incidents in recent years.

    State educational agencies are working to address the issues, in part, by stepping up efforts to flag statistically anomalous test results—a potential indicator that test misconduct may have occurred. These statistical tools used to identify anomalous testing outcomes provide valuable information in the pursuit of test security, but as with any technique based on statistical methods, they carry noteworthy limitations as well. A mishandled response to statistically anomalous test data by the state carries potentially catastrophic consequences for all involved: students may miss out on significant opportunities, such as graduating from high school, teachers may suffer irreparable damage to their reputations and careers, and the educational agency may damage its credibility or even expose itself to the risk of costly lawsuits.

    Considering the objective of maintaining the validity of test scores and the risks associated with making false accusations of cheating, the importance of having a sound test security policy cannot be understated. Any actions taken by the state in response to statistically anomalous test results must be tempered by a proper appreciation of the strengths and weaknesses of these tools. Test security policies describing statistical screening tools to be used, additional evidence to be collected when misconduct is suspected, steps in conducting follow-up investigations, and appropriate punitive actions to be imposed when misconduct is confirmed must be established for the protection of both the state and stakeholders. The state’s test security policy must be clearly documented and communicated to and understood by stakeholders.

    This session will focus on the importance of policy in maintaining test security, with particular emphasis placed on policies surrounding responses to suspected incidents of test misconduct. Each state’s representative will overview their state’s policies on prevention, detection, and investigation of test misconduct, including information regarding the successes and challenges of these policies. States’ anecdotes and sharing of their test security policies will be profitably received as educational agencies tackle the task of preserving the integrity of test scores. A representative from a testing organization will provide a nontechnical presentation discussing pros and cons of some common statistical techniques, discussing how these strengths and limitations must be considered in drafting responsible test security policies. The discussant will offer an independent perspective on the interaction between flagging techniques and policy.

    Official:
    Discussant: