Standing in the Middle of the Bridge: Lessons Learned about Current AA-AAS and Implications for the Next Generation of Assessments

Thursday, June 20, 2013: 8:30 AM-10:00 AM
Chesapeake G-H (Gaylord National Resort and Convention Center)
Presentations
  • CCSSO2013_LEAAP.pdf (382.3 kB)
  • Content Strands:
    1. Transitioning special populations
    ABSTRACT:
    As states transition to new assessment systems, decisions about elements of current assessment systems that are still valued and worth carrying forward are needed. This session describes findings from research using three states’ alternate assessment (AA-AAS) scores and item features. Results will be presented on (1) evidence of growth in AA-AAS scores across three years, (2) features of AA-AAS items that differentiate proficient and non-proficient students, and (3) features of hypothesized content progressions in item sets. State partners will discuss the utility of the findings as they plan for transitions to new AA-AAS systems.

    Alternate assessments based on alternate achievement standards (AA-AAS) have been part of large-scale assessment systems for more than a decade and have undergone considerable changes since their inception. Despite continuous improvement of AA-AAS systems, there are areas in which AA-AASs do not perform as well as other large-scale assessments.  For instance, the emphasis has been to evaluate proficiency within a year. In some states, achievement is evaluated in part based on growth within that year. Until recently there has been limited attention paid to the growth of students who take AA-AAS across years (Cook & Eignor, 2009).  Yet the use of student growth is desirable because it offers a means to detect their progress even though they may not reach the next achievement level. Another challenge lies in identifying the features of AA-AAS items that can be systematically manipulated to assess a continuum of skills within a progression. Item mapping techniques (e.g., Zwick, Senturk, Wang, & Cooper Loomis, 2001) may be a next step in improving AA-AAS systems.

    This session will describe findings from an Enhanced Assessment Grant project designed to address the issues described above. The partner states have diverse AA-AAS systems, including portfolios, rater checklists, multiple choice, and performance task formats; their systems are based on grade level or extended content standards. The research focused on AA-AAS in grades 3-8 Reading/ELA and math.

    The session includes four components. The first describes the application of several statistical models for examining student growth across consecutive grades. Transition, growth to proficiency, and OLS regression models are applied to the data. Implications for monitoring and reporting on growth will be discussed.

    The second part of the session describes findings from item mapping analyses. We examined features of items that discriminate between proficient and non-proficient students and considered whether there were differences in those items across grade levels or for different subgroups of students.

    The third component of the session focuses more narrowly on hypothesized content progressions that emerged from earlier content analysis and expert panel reviews. Using smaller item sets, exploratory analyses of item features will be presented.

    The final component will feature a panel discussion by state partners on their interpretation of the findings from the related studies. Representatives from these states, which are also participants in a national AA-AAS consortium, will discuss how project findings have illuminated information within  their current assessment systems that may be necessary to consider as they bridge to their next generation of AA-AAS.

    This session is relevant to several audiences, including state education personnel, assessment system developers and researchers, and others who work on the inclusion of students with significant cognitive disabilities in accountability systems. Although consortia are developing new common AA-AAS, states may still customize those assessments or build AA-AAS that integrate local priorities with nationally-developed systems. This session will highlight the factors states consider as they plan for high quality AA-AAS systems in years to come.