A validity case for procedural assessment literacy for EFL teachers

Pill and Harding (2013) presented a cline identifying five stages of Language Assessment Literacy (LAL), from total inexperience (1) to complete expert (5). O’Sullivan (2016) suggests most English teaching practitioners are around the third (functional) stage, meaning key concepts are understood in broad strokes. However, moving all professionals towards more principled understanding (the fourth, or procedural stage) of LAL is both an important and neglected aspect of validation. This session is based upon a replication study of Fulcher (2012), who conducted a self-evaluative questionnaire on perceived awareness of LAL among various practitioners. My study replicated that questionnaire, conducted in this instance among 50 English teachers and examiners. It also addresses the limitation in the original study of only using self-evaluative data by additionally asking participants to write brief definitions of terminology such as “construct validity”. The results were analysed using summary item statistics and thematic coding in NVivo. This paper empirically consolidates the aforementioned conclusions of O’Sullivan (2016), by demonstrating i) that many teaching professionals are indeed at the functional stage, ii) there are key areas of LAL within which practitioners are not even aware of the nature of the knowledge gap, and iii) that addressing these gaps through LAL training would lead to an overall increase in test validity, including for instance improved classroom testing, or better communication with stakeholders. English teaching in Latin America is used as a context to suggest practical frameworks for LAL development. More information about New Directions: https://bit.ly/3hLCaXK