Twitch will use AI to detect people evading bans

Twitch is furthering its efforts to reduce harassment with a new tool that uses machine learning to detect people who may be attempting to evade bans. It’s the company’s latest addition to combat things like hate raids, where streamers’ chats are overrun with trolls sending hateful messages.

The new tool, called Suspicious User Detection, can identify users as “likely” or “possible” people who have evaded bans from a streamer’s channel. The machine learning model powering the tool identifies potential evaders by evaluating things such as their behavior and characteristics about their account and compares that information against accounts that have been banned from a streamer’s channel.

Messages from “likely” evaders won’t be sent to chat, but streamers and their mods can see them. Streamers and mods can choose to monitor a likely ban evader, which adds that user to a monitoring list and puts a message next to a user’s name noting the monitoring (as shown in the GIF below), or ban them. “Possible” evaders’ messages will appear in chat, but streamers / mods can opt to have those messages blocked from chat as well.

Twitch says it will turn Suspicious User Detection on by default, but streamers can tweak or turn off the tool if they want. Streamers and mods can also manually select to monitor users of whom they are suspicious.

“This tool was inspired in large part by community feedback around the need for better ways to curb ban evaders,” Alison Huffman, Twitch’s director of product for community health, said in a statement to The Verge. “When we were speaking with mods about their pain points, we heard that it can be hard to distinguish whether a user who chatted something that violated their channel’s norms was a harmful, repeat harasser or just a newer viewer who hadn’t learned that channel’s customs yet. As such, we designed this tool to give mods and creators more information about potential ban evaders so they could make more efficient and informed decisions within their channel.”

Suspicious User Detection seems like it could make a difference in silencing hateful individuals, especially if used in tandem with recently introduced controls that let a streamer require phone or email verification for accounts participating in chat. But it remains to be seen how effective Suspicious User Detection actually will be in practice or if ban evaders can find ways to get around the tool.

By The Verge