Our Research

Nearly 3.4 million children require speech and language services under the Individuals with Disabilities Education Act (IDEA) and are at risk of falling behind in their academic and social-emotional development without timely intervention by Speech and Language Pathologists (SLPs). Unfortunately, there is a significant shortage of SLPs and the COVID pandemic has further exacerbated this gap, making it almost impossible for SLPs to provide individualized services for children.

The NSF and IES AI Institute for Transforming Education for Children with Speech and Language Processing Challenges (or National AI Institute for Exceptional Education, in short) aims to close this gap by developing advanced AI technologies to scale SLPs’ availability and services such that no child in need of speech and language services is left behind. Towards this end, the Institute proposes to develop two novel AI solutions: (1) the AI Screener to enable universal early screening for all children, and (2) the AI Orchestrator to work with SLPs and teachers to provide individualized interventions for children with their formal Individualized Education Program (IEP).  In developing these solutions, the Institute will advance foundational AI technologies, enhance our understanding of children’s speech and language development, serve as a nexus point for all special education stakeholders, and represent a fundamental paradigm shift in how SLPs serve children in need of ability-based speech and language services.

The Institute’s research will lead to advancements in AI, human-AI interaction, and learning science to improve educational outcomes for children with speech and language related challenges. These advancements and solutions can also benefit children without disabilities extending the reach of ability-based learning interventions.  The Institute will engage participants across the Institute and related stakeholders to advance opportunities to meet their unique educational and workforce development needs.

The AI Screener is an edge-based solution that will be initially deployed in early childhood classrooms. It will analyze video and audio streams of children’s classroom interactions, derive conventional speech and language measures used by SLPs, and assess novel and hard to obtain automaticity measures.  The AI Orchestrator is a superset of the AI Screener with its main application in the public-school classrooms. It will help SLPs to administer a wide range of evidence-based interventions and assess their effects on meeting children’s individual IEP learning targets.  At the core of the Orchestrator is a robust multi-agent reinforcement learning framework that can evaluate the potential benefits of different intervention practices and recommend those most appropriate for each child.  Both solutions will push significant advances in self-supervised learning to address sparse and noisy data issues, multimodality perception, learning material rewriting and enrichment, and edge AI for real-time processing. The Institute will develop human-centered AI design methodologies to embody the solutions in a form appropriate for children’s learning. Most importantly, Learning Science will not only inform the initial prototyping and validation, but also continually derive unique insights from the field deployed solutions.