Free Google-Maps lead-gen for no-website prospecting, with a full outreach workflow on top. Argora finds local businesses that have no real website while their same-category competitors do, ranks them by pitch-conversion probability, pushes them into the KunthiveOS database, and then turns each lead into a ready-to-send, multi-channel pitch (WhatsApp / call / email / walk-in) — so the whole client-acquisition funnel runs from one screen.
No Apify credits, no Google Places API. A maintained-style Playwright scraper does the extraction; the value lives in the lead-intelligence + outreach layer on top (sector presets, website split, ranking, competitor matching, message generation, an outreach log).
Documentation map
| Doc | Read it when |
|---|---|
| this README | setting up, first run, the daily workflow |
| BARRIERS.md | before scaling — what gets you throttled/banned, what never to build |
| SCHEMA.md | pushing to the KunthiveOS database — column mapping + dedup guarantees |
| docs/API.md | calling the local HTTP API from another app (KunthiveOS does) |
| docs/RECOVERY.md | a run got interrupted (CAPTCHA, crash, kill) — what survived, how to recover |
| docs/DEVELOPMENT.md | changing the code — module map, tests, fake-scrape mode, selector upkeep |
1. DISCOVER scrape Google Maps for a trade in an area (fol. 1–2)
2. RANK 5-dimension scoring → LEADS / COMPETITORS / ALL (fol. 3)
3. PUSH idempotent INSERT into KunthiveOS `leads` (fol. 4)
4. ENRICH transcribe a competitor's page (pricing, etc.) (fol. 5)
5. OUTREACH per-lead pitch + log the touch + follow-ups (fol. 6)
KunthiveOS (the sibling Supabase project) stays the system of record for leads. Argora is the cockpit that fills it and works it.
app.py ← THE one command — launches the web app + opens your browser
run.py ← CLI alternative: scrape a sector in an area, build the csvs
webapp/
server.py ← FastAPI backend: runs jobs, streams live logs (SSE),
SQL/DB API, page-extractor API, outreach API
static/
index.html ← the single-file UI (no build step) — six "folios"
leadfinder/
scraper.py ← FRAGILE. Drives Google Maps, pulls raw places. The SEL dict
is the ONE place to fix when Google changes its DOM.
analyze.py ← STABLE + TESTED. Raw JSON → LEADS / COMPETITORS / ALL csvs.
Auto-normalizes columns, so it ingests output from ANY engine.
ranking.py ← STABLE + TESTED. The 5-dimension lead-scoring algorithm
(score 0–100, tier hot/warm/cool/cold, tags). Single source
of truth; mirrors KunthiveOS/lib/scoring.ts.
sql_gen.py ← STABLE + TESTED. csv → Supabase `leads` INSERT sql (idempotent).
db.py ← Direct "no copy-paste" push to the KunthiveOS Postgres, plus
the optional outreach write-back (status/notes/activities).
extractor.py ← Single-URL page capture (text, else full-page screenshot).
outreach.py ← Pitch templating: lead + competitor → WhatsApp/call/email/
walk-in copy. Pure functions, no network (optional LLM slot).
outreach_log.py ← The lightweight local outreach log (JSON). NOT a CRM.
sectors.py ← Your trade presets (driving-school, real-estate, gym, law…).
data/raw/ ← raw JSON pulls (git-ignored)
data/leads/ ← the deliverables — csv (git-ignored)
data/sql/ ← generated Supabase sql (git-ignored)
data/extracts/ ← transcribed competitor pages (git-ignored)
data/outreach/ ← the outreach log (PII) (git-ignored)
data/last_job.json ← the last run's outcome + log (git-ignored)
The split is deliberate: the scraper is a maintenance treadmill (selectors rot, IPs throttle). The analysis + outreach layers — your real moat — never break when Google changes its markup.
- Python 3.10+ (the repo's venv was built with 3.14; anything 3.10+ is fine).
- git with access to this repo (
kunthive-Labs/Argora). - For the DB push / outreach write-back (optional): the KunthiveOS repo checked out as a sibling, or a Postgres connection string. See "Connect the KunthiveOS database" below.
# 1. Clone
git clone git@github.com:kunthive-Labs/Argora.git
cd argora
# 2. Create + activate a virtualenv
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# 3. Install dependencies
pip install -r requirements.txt
# 4. Install the Chromium browser Playwright drives
playwright install chromiumThe intelligence layer (ranking, analysis, SQL generation, outreach) and the server's job orchestration are covered by a 200+ case pytest suite — pure functions and mocked scrapes, no browser or network needed:
pip install -r requirements-dev.txt
pytestTo exercise the whole app end-to-end without touching Google, run it with the fake-scrape hook — canned places with realistic pacing:
ARGORA_FAKE_SCRAPE=1 python app.py # Windows: set ARGORA_FAKE_SCRAPE=1Windows notes: activate the venv with
.venv\Scripts\activate(shown above); everything else is identical — all file I/O is explicit UTF-8. If you pipe CLI output to a file on a legacy-codepage system and hit an encoding error on the★symbols, setPYTHONUTF8=1.
Two things do NOT come through git (they're git-ignored on purpose):
- Your scraped data in
data/— it's local business PII. You start the new machine with emptydata/folders and re-scrape (or copy thedata/folder across by hand if you want your history).- Database credentials — see the next section. You must reconnect the DB on the new machine before "Push to DB" / outreach write-back will work.
The DB push and the outreach write-back need a Postgres connection. Argora looks for it in this order — set up any one:
-
DATABASE_URLenv var (or a.envfile in the repo root):# .env — copy from .env.example, never committed DATABASE_URL=postgresql://postgres:PASSWORD@db.xxxx.supabase.co:5432/postgres?sslmode=require
(Supabase → Project Settings → Database → Connection string. Special characters in the password are handled automatically.)
-
The KunthiveOS sibling repo — if you also clone KunthiveOS next to this repo, Argora auto-reads its local
../KunthiveOS/.db-conn.json. Nothing to configure. -
KUNTHIVE_OS_DB_CONNenv var pointing at a.db-conn.jsonelsewhere.
Run once against the database before pushing Argora leads: apply KunthiveOS
migration 0010_leads_v2_rich_fields.sql (adds the 'argora' source value and
the rich columns). See SCHEMA.md for the full column mapping.
You can verify the connection from the UI (fol. 4, the wax seal turns green) or:
python -c "from leadfinder import db; print(db.status())"source .venv/bin/activate
python app.py # → http://127.0.0.1:8000 opens automatically
# options:
python app.py --port 9000 --no-openCtrl-C stops the server. One job runs at a time (a scrape/extract opens a
real browser).
| Folio | Name | What you do |
|---|---|---|
| fol. 1 | Open an entry | Pick trades (preset chips or write your own) + one or more localities (one per line — scraped in order, with a configurable rest between areas). Post the run. |
| fol. 2 | Posting | Watch the scrape live — each place streams in, with running lead/competitor/scraped/area counts. area 2/3 · gym @ HSR Layout phase lines show where the run is. |
| fol. 3 | The ledger | Browse/sort/filter any generated CSV; click a row for full detail; download. |
| fol. 4 | Post to the master | Pick a LEADS book → Draft the SQL (copy/download) or Post direct to KunthiveOS (idempotent, safe to re-run). |
| fol. 5 | Page extractor | Paste one exact URL → transcribe its text (or a full-page screenshot if it can't be copied). Use it to grab a competitor's pricing/services page. |
| fol. 6 | Outreach studio | Open a LEADS book → each lead becomes a ready-to-send pitch. Fire WhatsApp/Call/Email/walk-in, log the touch, set follow-ups. Due follow-ups surface as a banner at the top of the page and a red badge on this folio — click through to land on the lead's pitch card. |
Batching areas: list several localities in fol. 1 and the run works through them inside the one job slot, resting between areas (default 180 s — the BARRIERS.md pacing, adjustable under Posting terms). Each (trade, area) pair gets its own summary row and its own set of CSVs.
# one sector, one area — scrape + analyze in one go
python run.py driving-school "Jayanagar, Bengaluru"
# bigger area, custom cap
python run.py real-estate "Whitefield, Bengaluru" --max 120Output: data/leads/driving-school-jayanagar-bengaluru-LEADS.csv
(+ -COMPETITORS, -ALL).
Other CLI entry points:
# generate Supabase SQL from a LEADS csv
python -m leadfinder.sql_gen data/leads/gym-yelahanka-LEADS.csv \
--dataset argora/gym-yelahanka --out data/sql/gym-yelahanka.sql
# merge several area pulls into one deduped list
python -m leadfinder.scraper "real estate agent" "HSR Layout, Bengaluru" --out data/raw/re-hsr.json
python -m leadfinder.analyze "data/raw/re-*.json" --out data/leads/real-estate --sector real-estate
# transcribe one page
python -m leadfinder.extractor "https://example.com/pricing" --out data/extracts --mode auto- fol. 1–2 — pick a trade and list the session's neighbourhoods (headed, modest — see BARRIERS.md). The run paces itself area-to-area; you watch it stream in.
- fol. 4 — push the LEADS book into KunthiveOS (idempotent; never duplicates).
- fol. 6 — Outreach studio:
- Fill Your details once (name / business / phone) — it's kept in your browser and signs every message.
- Pick the LEADS book, Open work. Leads come ranked, each with a pitch that already names the competitor who outranks them.
- For each lead: hit WhatsApp (opens wa.me with the message pre-filled), Call (dials + a script you can expand), Email (paste an address, opens your mail client), or copy the text. Walk-in stops are grouped by PIN in the side panel (printable).
- Set the outcome (sent / no-answer / interested / callback / converted), add a note, optionally a +2d / +1w follow-up, and Log touch.
- Tick push to also flip the lead's status/notes in KunthiveOS (only when the DB is reachable).
- Untouched only hides leads you've already worked; Follow-ups due resurfaces the ones to chase.
The outreach log lives in data/outreach/log.json (local, git-ignored). It
stops you double-messaging and tracks follow-ups — it is not a second CRM;
KunthiveOS remains the system of record.
| File | Rows | Use |
|---|---|---|
…-LEADS.csv |
no real website + has phone, ranked | who you pitch |
…-COMPETITORS.csv |
same trade, HAS a website, ranked | pitch ammo + the competitor each lead is cited against |
…-ALL.csv |
everything, with website_status |
the full picture / QA |
Ranking is a 5-dimension score (0–100): web gap (40) · review volume (25) ·
rating trust (15) · reachability (10) · category urgency (10), plus
bonuses/tags (upgrade_pitch, iconic_local, premium_zone…) and a tier
(hot/warm/cool/cold). Google Sites / business.site auto-pages count as no
website — the strongest kind of lead. Full spec:
KunthiveOS/docs/handover-lead-ranking-algorithm.md.
Scraping gets interrupted: Google throws a CAPTCHA wall, the browser dies, the laptop sleeps, you hit Halt. Argora is built so none of that costs you the places already scraped:
- Write-through checkpointing — every extracted place is saved to
data/raw/<stem>.json(atomically) the moment it's scraped. Even a hard kill of the process loses at most the one card in flight. - CAPTCHA detection, graceful stop — Google's "unusual traffic" wall is detected and the run stops with a clear message. Partials are saved. (Per BARRIERS.md the right response is to stop for the day — Argora never tries to solve or evade it.)
- Per-sector isolation — one trade failing mid-batch doesn't kill the run; its partials are saved, the error is recorded on its summary row, and the remaining trades/areas continue.
-RECOVEREDbooks — CSVs built from an interrupted scrape are suffixed-RECOVEREDso you can tell a partial book from a complete one at a glance.- Survives a dead server — job progress is snapshotted to
data/last_job.jsonafter every finished target; reopen the app and fol. 2 shows what completed ("interrupted mid-run — partials saved to data/").
Full details + the manual recovery command: docs/RECOVERY.md.
BARRIERS.md is the must-read: what gets you throttled/banned and what never to
build. Short version:
- Google caps ~120 results per search. Scrape area-by-area, not one big query.
- Headed + gentle + spaced-out. Headless at volume is the easiest bot tell.
- No evasion tooling (CAPTCHA solvers, proxy rotation, login automation).
- Logged out, always. Never drive Maps signed into your Google account.
- Data stays local —
data/is git-ignored; it's business PII. - TRAI/DND: outreach is low-volume, relevant, legit — no bulk auto-dialers/spam.
- Selectors rot — when extraction goes blank, fix the
SELdict inscraper.py.
Edit leadfinder/sectors.py — copy a block, change query, exclude
(keywords that mean "wrong business"), and min_reviews. That's it. Or just
type a free-text trade in fol. 1.
The generated INSERT only ever INSERTs, and can never add anything twice —
guarded four ways (intra-batch dedup, WHERE NOT EXISTS on place-id or phone,
ON CONFLICT on the unique constraint, INSERT-only). Verified end-to-end against
Postgres 16. Full mapping + guarantees: SCHEMA.md.
maps-lead-gen is the Apify-based version (pay-per-event,
compass/crawler-google-places). This repo is the free Playwright version.
analyze.py here is column-compatible with that repo's Apify JSON, so you can
mix sources.