An NSF & IES AI Institute for Transforming Education for Children with Speech and Language Processing Challenges
The Project
The National AI Institute for Exceptional Education is developing advanced artificial intelligence technologies to scale speech-language pathologists’ availability and services so that every child’s speech and language needs are met. We are developing two novel AI solutions: (1) the AI Screener to enable universal early screening for all children, and (2) the AI Orchestrator to work with SLPs and teachers to provide individualized interventions for children with their formal Individualized Education Program (IEP).
The importance of speech and language in children’s academic and social-emotional development cannot be overemphasized. Much of children’s progress across all academic content areas such as social studies, math and science requires strong speech and language development.
There are wide educational gaps between children with delayed speech or language abilities and their peers. This is particularly concerning as more than one-half of children under the 13 categories in the Individuals with Disabilities Education Act—about 3.4 million children in the US—need speech and language services.
Solution 1: AI Screener
The AI Screener is an edge-based solution that will be deployed in early childhood classrooms. It analyzes video and audio streams of children’s classroom interactions, derive conventional speech and language measures used by SLPs, and assess novel and hard to obtain automaticity measures.
Solution 2: AI Orchestrator
The AI Orchestrator is a superset of the AI Screener with its main application in elementary school classrooms. It will help SLPs to administer a wide range of evidence-based interventions and assess their effects on meeting children’s individual IEP learning targets. At the core of the Orchestrator is a robust multi-agent reinforcement learning framework that can evaluate the potential benefits of different intervention practices and recommend those most appropriate for each child.
Outcomes
Both solutions promote significant advances in self-supervised learning to address sparse and noisy data issues, multimodality perception, learning material rewriting and enrichment, and edge AI for real-time processing.
Our human-centered AI design methodologies embody solutions in a form appropriate for children’s learning. Most importantly, learning science will inform the initial prototyping and validation and continually derive unique insights from the field deployed solutions.
We bring together interdisciplinary teams of renowned researchers and practitioners in the areas of foundational AI science and technologies, learning science for children with disabilities, and human-centered AI design. Our goal is to solve both the foundational AI challenges and address the societal challenge of scarcity of SLP service providers while addressing cross-disciplinary research questions.
We will contribute significant new knowledge in the following areas: (1) novel architectures and optimization techniques for robust self-supervised learning, (2) novel multi-agent reinforcement learning algorithms that are both robust and sample-efficient, (3) new SOTA algorithms and results on multimodality perception and understanding at the edge in an unconstrained environment, and (4) new understanding of some foundational learning science questions for children with speech and language related concerns.
Thrust 1
Self-supervised Robust Learning: Foundational AI research to address self-supervised robust learning with few and noisy labeled data.
Thrust 2
Multi-agent Reinforcement Learning: Foundational AI research to address the advancement of multi-agent-based reinforcement learning.
Thrust 3
Advancing Multimodality Perception at the Edge: Leverage some of the first two thrusts’ core AI technologies to further advance AI’s capabilities in multimodal perception.
Thrust 4
Advancing Human-centered AI Design Methodologies: Develop foundational AI technologies in collaboration with the human-centered AI design team.
Thrust 5
Advancing Socially and Clinically Appropriate AI Embodiment: Deploy the use-inspired AI solutions through socially and clinically appropriate AI embodiment research.
Thrust 6
Advancing Speech and Language Learning Science: Pair the Institute’s AI science researchers with SLP-domain researchers for every research thrust to facilitate cross-pollination of ideas. The new data and insights from AI will help advance learning science research for children with speech and language concerns.
Validation and Efficacy Methodologies
Throughout our research efforts, we will continuously inform, refine, and validate our methods, thus ensuring we remain focused on the most important and relevant research challenges.
