I have no data yet. It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.
🚧
Welcome to the OSINTBuddy project where you can connect, combine, and get insight from unstructured and public data as results that can be explored step-by-step. An easy-to-use plugin system allows any Python developer to quickly integrate new data sources so you can focus on discovering, interacting, and visualizing what's important to you
osib-demo-video.mp4
Table of Contents
What is osintbuddy.com?
Not much, yet.
But here's the concept:
An almost incomprehensible amount of data is created every day. And each year, figures are growing at an ever-increasing rate. These data sources can be divided up into six different categories of information flow:
- Public government data
- Media (newspapers, magazines, radio)
- Internet (blogs, discussion groups, citizen media, etc)
- Professional and academic publications (budgets, hearings, telephone directories, websites, etc)
- Commercial data (commercial imagery, financial assessments, databases, etc)
- Grey literature (technical reports, preprints, patents, business documents, etc)
OSINT is a method of working with, assessing and ranking information — We are living in the information age and the volume of information OSINT research has to deal with (information explosion) and the issue of low quality data leaves researchers drowning. The majority of this data is unstructured introducing many challenges to analyzing it and producing actionable intelligence since most data analytics databases are designed for structured data. We are surrounded by information and nearly none of it is useful.
I've decided to do something about it. The rapid developments in technologies such as AI and big data analytics have opened new horizons for OSINT which weren't previously available. We want to put all of that information at your fingertips. We want actionable intelligence quickly and to the point, that's why we're building an open-source OSINT tool that's free. Free to use, free to modify, free to do with as you wish, and built with plain old web technologies anyone can learn. But this isn't really the project.
It's been three decades since its inception and the internets only proved to be a mirror of us, our society and politics. We face a series of significant challenges that directly threaten democratic values and processes. The spread of disinformation and misinformation, amplified by algorithms designed to prioritize engagement over truth, erodes trust in institutions and fuels social division. Online "echo chambers" and filter bubbles reinforce existing biases, increasing political polarization and hindering constructive dialogue. Big Tech monopolies shape our digital lives and authoritarianism gains ground, the question of who owns, designs, and controls technology has never been more urgent.
What are the technical frameworks that ensure that a technology is open and equitable; how do we “reconfigure” digital infrastructures to serve positive social transformation rather than corporate interests? This is a new chance for a truly free open source OSINT tool to be created for enthusiasts around the world to fight back against these issues and we need your help to design it, to program it, and to build it. We want to hear your suggestions, your ideas, and we're going to build it right in front of your eyes. The notion of a “needle in a haystack” is taken to the extreme on the internet. Let's build a magnet.
Caution
OSINTBuddy is currently experimental software. It's not quite ready for use yet! If you're interested in development you'll need the plugins docker service cloned: git clone https://github.com/osintbuddy/plugins osintbuddy-plugins to the root of this repo
| Repository | Description | Language | ETA |
|---|---|---|---|
osintbuddy |
The main web application and backend (you are here) | Rust, TypeScript/Preact | 3-6 months from alpha |
plugins |
the Python plugin system package that's on PyPi | Python | 3-6 months |
entities |
The default OSIB entity definitions | Python | 4-7 months |
- Visual Intelligence Made Simple: Intuitive graph-based interface transforms complex data relationships into clear, interactive visualizations.
- Extensible Plugin Architecture: a Python-based plugin system allows custom entities to pull from any data source.
- Self-hosted: Your data can stay under your control with full privacy and security
- Cost-Effective: Ditch the enterprise pricing, access advanced OSINT capabilities with free open source software.
- Open for contributions.
Works most of the time :)We're currently in the process of finishing up the rewrite, stay tuned!- And check out the open issues for a list of requested features (and known bugs).
We aspire to become more than just a data aggregation tool:
- Collective Intelligence - Collaborative workspaces for global knowledge sharing
- Real-time Monitoring - Automated insights and pattern detection
- History Graphs - Browser extension for seamless web exploration
- AI Integration - Swarm intelligence, sentiment analysis, and evolutionary algorithms
Once we iron out a few more bugs and a few more features we intend to setup a github actions workflow to build and package up the built frontend and Rust server. Stay tuned...
This Rust rewrite exists because I got tired of Python and I decided to rewrite the entire thing to learn Rust. Now it's faster and crashes less (hopefully).
If you want to start developing for OSINTBuddy, create or pick up an issue and follow these steps:
-
Clone the repo
git clone https://github.com/osintbuddy/osintbuddy.git cd osintbuddy # using ssh? # git clone git@github.com:osintbuddy/osintbuddy.git
-
Initialize core plugins for OSINTBuddy:
python3 -m venv venv . ./venv/bin/activate pip install ./osintbuddy-plugins/ ob init -
Start the stack
docker compose up db ui -
Migrate db and build and start web server with watch
cargo install sqlx-cli --no-default-features --features native-tls,postgres cargo install cargo-watch sqlx migrate run # ensure your Python venv with osintbuddy is activated and the default plugins initialized... # . ./venv/bin/activate # now we can run and watch the Rust server cargo watch -q -c -w src/ -x run
-
Access OSINTBuddy through the URLs provided for the frontend, backend, and documentation.
- URLs
- Frontend:
http://localhost:55173 - Docs:
http://localhost:55173/docs/overview - Backend:
http://localhost:48997/api
- Frontend:
↑License
We are using the GNU Affero General Public License v3.0 (AGPL) as we want to guarantee freedom of use, reuse, copy, modification and re-publication of modifications. This is a technopolitical decision encoded into the social contract. Suffice to say here that the Affero GPLv3 licence legally binds the service providers to give direct access to any user to the computer code that runs in a given instance.
Note: the OSINTBuddy PyPi package is MIT licensed. We understand some individuals and businesses may not want to share their plugins developed in-house.
↑Related Projects
Empowering investigators & cybersecurity enthusiasts to uncover truths, fight injustice, and create a safer world.
Start with one lead. Close cases 12x faster with OSINT.
LinkScope allows you to perform online investigations by representing information as discrete pieces of data, called Entities.
Discover and deliver actionable intelligence.
a browser extension that offers a real-time, on-page approach to analyzing web content – completely content and site agnostic
↑Contact
Open an issue if you need to get in touch with me or send an email to oss@osintbuddy.com.