Using Novel Phrase-Level Explanations in a Softmax-Linked Additive Explainability Model for Transformers

STEM I is a year-long independent research course at Mass Academy of Math and Science at WPI. Students identify a research problem, write a grant proposal, design and execute a methodology, and present findings at regional science fairs. This page documents the research proposal and development process for the phrase-level SLALOM project.

AI systems like ChatGPT are being used in hospitals, courtrooms, and banks to help make important decisions, but nobody can see how these systems actually think. Existing methods for explaining AI decisions break sentences into individual words and score each one separately. But language doesn’t work that way — “not bad” means something completely different from “not” and “bad” separately. This project built a new method that explains AI decisions using whole phrases instead of individual words, so the explanations match how people actually read and understand language. The result is explanations that are both more accurate and easier for a human to understand.