Using AI to reach users who need support in a time of crisisSep 01 2022
are incredibly proud to announce that our custom moderation AI,
created by the Swedish company Oterlu, is now helping our human
moderators detect potential expressions of self-harm. This is the
result of hard work and collaboration between Oterlu, Recolor’s
moderation team, and Recolor’s content strategy team, and will help
us find and assist users struggling with self-harmful thoughts or
did it start?
community team at Recolor already had an established flow to provide
users with support and resources when there were any indications or
expressions of potential self-harmful behavior. But like all manual
moderation, this relied heavily on reports from other users.
our striving for continual improvement, we wanted a better, faster way
of finding users who need help. Oterlu was able to expand the AI
model’s capabilities so that it can now flag in real time comments
that could indicate users need support.
additional layer of safety
The AI model from Oterlu that is used on the Recolor platform is not intended to operate on its own, but to be an additional tool for the community team. The AI flags publicly-posted comments that are then reviewed by the moderation team, which makes it faster to reach out with help and resources for those who may be in a crisis and need them.
We hope that this will inspire other platforms to focus on how to support those of their users who need help the most.