This post was originally published on this site.
Protecting user experiences is at the core of every good product. No one wants to enjoy something once just to have a poor interaction the next time you go to use it. Here at Stack Overflow we are dedicating time and resources to the improvement of user experiences, particularly where malicious content and system distributions are involved. The newly established Moderation Tooling team for the Public Platform are implementing new tools and systems to keep users from the unwanted and interruptive exposure of spam and securing vulnerabilities from bad actors.
Stopping spam before it hits the platform
Like much of the internet, spam has long been an unfortunate reality on Stack Overflow. Spam has many looks but ultimately is unwanted and unsolicited advertisement or promotion that disruptions the Q&A experience. Moderation Tooling built an all-new spam filtering system to combat spam before it is ever published to Stack’s network. The idea behind the tool is simple: if a new post looks very similar to content that has been recently removed for being spam, it’s likely spam too.
In the past we’ve tried naive, or even legacy text comparison approaches by using a regex blocklist of words or phrases that often accompanied spammy posts. However, this approach was difficult to maintain, requiring engineers to manually notice trends and update lists accordingly. It was also incredibly brittle to strike the right balance between blocking a spammer from adding a phone number while still allowing a programming question about how to validate one.
The new wave of spam detection is here! By using vector embeddings and cosine similarity, the team has been able to build a tool that has an incredibly low false positive rate, or not mistakenly removing legitimate messages as spam. This has led to a 50% reduction in the time spam stays live on the platform and allotted valuable time back to our moderators to help maintain other parts of the platform’s integrity.
Community Moderation
None of this would be possible without the dedicated members of our community helping us flag and identify what spam looks like on our platform. With special shouts to the folks behind Charcoal, who safeguard the site from bad actors on a daily, hourly, and even minute-by-minute basis. We’re very excited to be able to leverage the hard work they put in on identifying spam and automate it earlier into the pipeline.
Improving the moderation tools for a healthier platform
We are taking up our efforts to improve the site and improve the tools that our moderators have to maintain a healthy network. In May, the Moderation Tooling team was formed to bring those moderator asks and requests to life, and also the solutions needed to maintain a more secure user experience.
Our goal is to create a secure and positive environment for you to get your questions answered and share your knowledge. With new systems for stopping spam and detecting bad actors, the Moderation Tooling team is committed to making the platform safer, freeing up our community moderators and ensuring better Q&A for everyone. We believe that a clean, safe platform allows everyone to focus on what matters most: learning and building. We invite you to enjoy a spam-free network experience while continuing to ask, answer, and connect with confidence on a healthier, more secure Stack Overflow.
