Schools all across the country are struggling with how to keep their students safe, but a high school in Massachusetts is hoping that AI will help prevent future violence. Shawsheen Valley Technical High School has decided to partner with Social Sentinel, a data analytics company that claims it’s able to detect potentially violent behavior in students’ online activity before anything violent or dangerous actually happens. Ideally, the program will be able to identify at-risk students, similar to technology used by the Police to help monitor potentially dangerous students.
According to Social Sentinel founder and CEO, Gary Margolis, the AI program has a library of 450,000 violent indicators — including sings of depression, resentment, and isolation. The company also worked with linguists and psychologists to make sure the program would be able to recognize any social media or online behavior that might indicate a student planning to do harm to themselves or others.
“We went back, for example, and looked at the language that school shooters, as one example, have used in the past in various manifestos — what’s been published or that they’ve shared on social media,” Margolis told Boston public radio WBUR. “And we want to understand similarities and patterns. And we can teach computers, to an extent, how to identify some of that nuance.”
But while Margolis is confident about the program’s abilities, many experts are critical of a program that monitors student’s online activity in order to try and recognize “violent” behavior before violence actually takes place. Beyond the obvious challenges of properly defining what exactly can be counted as an indicator of violence, many see this entire thing as invasive and an inadequate excuse to force students to surrender their right to privacy.
“This is an expansion of the schools’ ability to police what students are doing inside of school or on campus to their outside-of-school conduct,” Kade Crockford, director of the Technology for Liberty Program at the ACLU of Massachusetts told WBUR. “In many cases across the country, schools have been using social media surveillance tools in ways that have harmed, specifically, students of color. So we certainly have concerns about technologies like this being used to expand what we call the school-to-prison pipeline.”