Julietshen/update roadmap#52
Conversation
…res, and new plan for preparing ROOST projects for AI-powered safety
This section is a reasonable addition. There is potential for MCP sourced added context being beneficial in decision making. The details over in https://github.com/orgs/roostorg/discussions/44 look like a good start to formalizing a connection. I do note the mentions of threat model, threats, and trends in adversarial evolution in the discussion threads. There's definitely a potential that added context, despite hope for the opposite, actually undermines assessments whether through malicious code or subtle inherent bias. AI assist modes definitely change a threat model, even for something that aims to have a human in the loop, and this will need more analysis and guardrails. |
|
@haileyok what do you think? |
We run both office hours and working groups. Plus, link to the bare path so it will handle the changes in #54 cleanly.
There was a problem hiding this comment.
Okay! I went through largely focusing on how to be a bit more nuanced around "AI," especially in light of conversations with platforms and communities who may be turned off by generative AI. The gist is that I don't think we need to change the actual roadmap itself at all (as it's based on our conversations with adopters), but I just want to make sure we're considering multiple perspectives and not unintentionally shutting out some potential adopters while accurately communicating our plans and focus.
To be clear, each comment is a suggestion and I invite pushback/discussion!
|
|
||
| ## Preparing for AI-Powered Safety | ||
|
|
||
| As AI increasingly gets used by bad actors, it's critical that those who work on online safety have access to the same degree of technology to protect online communities. In addition, product improvements are shipping rapidly and adversarial behavior is evolving with them. AI can make safety capabilities accessible to teams that could never have built them from scratch. [Our blog post on deploying AI agents for safety](https://roost.tools/blog/how-to-deploy-ai-agents-for-safety/) covers the motivation and patterns in more depth, including production examples from Block and Notion. In order to prepare our projects for AI-powered features, there are two foundational components needed: |
There was a problem hiding this comment.
I have this persistent nagging issue with the breath and imprecision of "AI" as a term, roping in several different technologies. Maybe that's a lost battle, but I wanted to flag it.
I also worry that this could read at a glance too much like, "we're going all-in on AI!" versus "we're ensuring our tools work well with LLMs/agentic integrations should platforms choose to use them."
If I were a platform or organization opposed to the use of LLMs and/or generative AI (for example, several platforms designed for open source communities, artist/maker communities, writers, etc.), this could be a turn-off to me. So I want to make sure we're balancing things a bit:
- We know that generative AI use is increasingly used by bad actors
- Online safety teams need access to the best tools, including "AI" technologies like LLMs and agentic workflows if they choose
- If platforms/orgs do choose to use ROOST projects with AI technologies, these are things we know we need to do to improve our projects (and online safety tools in general) to enable this
There was a problem hiding this comment.
Thanks! I think I can also add some language introducing why we're adding this update, as a way to provide more detail to the previously very ambiguous "add AI features to our tools" that was previously in our roadmap.
I think using AI for safety is something we should not shy away from; we already have a disclaimer in the Roadmap that says that those who don't want to use AI can use earlier versions that are tagged as pre-AI (which we should plan/discuss, since we're getting closer to this on Osprey).
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Co-authored-by: Cassidy James <cassidyjames@roost.tools>
Updated the roadmap with infrastructure work, framing and language on how we plan to prepare ROOST's tools for AI-powered functionality, links to detailed features on the roadmap, and updated details on what versions are currently available.