Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ Full release notes with details on each version: [GitHub Releases](https://githu
- Fix: non-English query sentences no longer pick wrong BFS seeds because their filler words were unfiltered (#1900). The query stopword set now covers German and the major Romance languages (curated to avoid clobbering English content words), so `Wie funktioniert die Authentifizierung?` seeds the keyword, not the stopwords.
- Fix: Python calls to an imported module now resolve (already shipped in 0.9.16); `.skill` files (Markdown-with-frontmatter agent files) are now classified as documents instead of being silently dropped as an unsupported extension (#1901).
- Fix: the `--postgres` missing-driver error now points at the correct PyPI package, `graphifyy[postgres]` (was the nonexistent `graphify[postgres]`) (#1906).
- Add: R (`.r`/`.R`) support — bespoke `extract_r` handles all five function-assignment forms (`<-`, `<<-`, `=`, `->`, `->>`), nested function scoping beneath the enclosing function, top-level vs function-level call attribution, member calls (`$`/`@`), package-qualified calls (`::`/`:::`), and pipes (`|>`/`%>%`). Named functions are assignments whose right- or left-hand side is an anonymous `function_definition`, so the generic extractor is not a good fit; the new bespoke extractor lives under `graphify/extractors/`, the established home for language-specific extractors (not the `extract.py` god node). `library()`/`require()`/`requireNamespace()` with a literal identifier or string emit `imports` edges; `pkg::fn`/`pkg:::fn` double as import evidence for `pkg`. `source("helper.r")` with a static `.r` path resolved against the caller file's directory emits `imports_from` when the target exists on disk (URLs, connections, variables, computed paths, and missing files are skipped — mirroring the Bash static-`source` policy). Unqualified bare calls funnel through the shared cross-file `raw_calls` resolver; an R language-family guard blocks binding to non-R definitions, so a cross-file candidate resolves `EXTRACTED` with `source()` import evidence or `INFERRED` without it, and ambiguous names remain unresolved. Ships under the optional `[r]` extra via `tree-sitter-language-pack` (the pack may fetch/cache the R grammar on first use; no source is uploaded and R itself is not executed). Install: `uv tool install "graphifyy[r]"`. R package metadata (`DESCRIPTION`/`NAMESPACE`/exports) and package-qualified call resolution are deferred to a follow-up (#9).
- Fix: `.R` files no longer trigger the "no AST extractor for `.r`/`.R`" warning (#1689) now that an `extract_r` dispatch exists; the missing-dependency path reuses the existing `#1745` install-extra warning with `pip install "graphifyy[r]"`. `Rscript`-shebang scripts (already detected as code) now dispatch to `extract_r` instead of falling through to the no-extractor warning.

## 0.9.16 (2026-07-14)

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ Codex users also need `multi_agent = true` under `[features]` in `~/.codex/confi
| `terraform` | Terraform / HCL `.tf`/`.tfvars`/`.hcl` AST extraction | `uv tool install "graphifyy[terraform]"` |
| `pascal` | Pascal / Delphi `.pas`/`.dpr`/`.dpk`/`.inc` AST extraction (more accurate `calls`/`inherits` edges; falls back to a regex extractor when absent) | `uv tool install "graphifyy[pascal]"` |
| `chinese` | Chinese query segmentation (jieba) | `uv tool install "graphifyy[chinese]"` |
| `r` | R `.r`/`.R` AST extraction (functions, calls, imports, static `source()`); ships via `tree-sitter-language-pack` which may fetch/cache the R grammar on first use (no source uploaded, R not executed) | `uv tool install "graphifyy[r]"` |
| `all` | Everything above | `uv tool install "graphifyy[all]"` |

</details>
Expand Down Expand Up @@ -327,7 +328,7 @@ To remove graphify from all platforms at once: `graphify uninstall` (add `--purg

| Type | Extensions |
|------|-----------|
| Code (36 tree-sitter grammars) | `.py .ts .mts .cts .js .jsx .tsx .mjs .go .rs .java .c .cpp .cc .cxx .h .hpp .cu .cuh .metal .rb .cs .kt .kts .scala .php .swift .lua .luau .toc .zig .ps1 .psm1 .psd1 .ex .exs .m .mm .jl .vue .svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json .dm .dme .dmi .dmm .dmf .sln .slnx .csproj .fsproj .vbproj .xaml .razor .cshtml` (`.dm`/`.dme` requires `uv tool install graphifyy[dm]`; `.mts`/`.cts` reuse the TypeScript grammar, `.cc`/`.cxx` and CUDA `.cu`/`.cuh` and Metal `.metal` reuse the C++ grammar) |
| Code (37 tree-sitter grammars) | `.py .ts .mts .cts .js .jsx .tsx .mjs .go .rs .java .c .cpp .cc .cxx .h .hpp .cu .cuh .metal .rb .cs .kt .kts .scala .php .swift .lua .luau .toc .zig .ps1 .psm1 .psd1 .ex .exs .m .mm .jl .r .R .vue .svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json .dm .dme .dmi .dmm .dmf .sln .slnx .csproj .fsproj .vbproj .xaml .razor .cshtml` (`.dm`/`.dme` requires `uv tool install graphifyy[dm]`; `.r`/`.R` requires `uv tool install "graphifyy[r]"`; `.mts`/`.cts` reuse the TypeScript grammar, `.cc`/`.cxx` and CUDA `.cu`/`.cuh` and Metal `.metal` reuse the C++ grammar) |
| Salesforce Apex | `.cls .trigger` (regex-based; classes, interfaces, enums, methods, triggers, SOQL/DML edges) |
| Terraform / HCL | `.tf .tfvars .hcl` (requires `uv tool install graphifyy[terraform]`) |
| MCP configs | `.mcp.json` `mcp.json` `mcp_servers.json` `claude_desktop_config.json` — extracts server nodes, package refs, env var requirements |
Expand Down
9 changes: 7 additions & 2 deletions graphify/extract.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@
from graphify.extractors.objc import _objc_local_var_types, extract_objc # noqa: E402,F401

from graphify.extractors.julia import extract_julia # noqa: E402,F401
from graphify.extractors.r import extract_r # noqa: E402,F401

_RECURSION_LIMIT = 10_000

Expand Down Expand Up @@ -1788,6 +1789,7 @@ def _lang_is_case_insensitive(source_file: object) -> bool:
".zig": "zig",
".ex": "elixir", ".exs": "elixir",
".jl": "julia",
".r": "r",
".dart": "dart",
".sh": "shell", ".bash": "shell",
".ps1": "powershell", ".psm1": "powershell", ".psd1": "powershell",
Expand Down Expand Up @@ -3877,6 +3879,7 @@ def add_existing_edge(edge: dict) -> None:
".m": extract_objc,
".mm": extract_objc,
".jl": extract_julia,
".r": extract_r,
".f": extract_fortran,
".F": extract_fortran,
".f90": extract_fortran,
Expand Down Expand Up @@ -3943,6 +3946,7 @@ def add_existing_edge(edge: dict) -> None:
".hcl": "terraform",
".dm": "dm",
".dme": "dm",
".r": "r",
}


Expand All @@ -3951,7 +3955,7 @@ def add_existing_edge(edge: dict) -> None:
# routes them to the CODE path via _shebang_interpreter; _get_extractor must
# honor the same signal or these files are classified as code and then silently
# dropped by extraction. Only interpreters with a real extractor are mapped —
# detect's wider set (perl, fish, tcsh, Rscript) stays unmapped and skipped.
# detect's wider set (perl, fish, tcsh) stays unmapped and skipped.
_SHEBANG_DISPATCH: dict[str, Any] = {
"python": extract_python,
"python2": extract_python,
Expand All @@ -3967,6 +3971,7 @@ def add_existing_edge(edge: dict) -> None:
"lua": extract_lua,
"php": extract_php,
"julia": extract_julia,
"Rscript": extract_r,
}


Expand Down Expand Up @@ -4403,7 +4408,7 @@ def extract(
)

# #1689: a file counted as code (extension in CODE_EXTENSIONS) but with no AST
# extractor wired up (e.g. .r/.R — there is no tree-sitter-r dispatch) silently
# extractor wired up (e.g. .ejs) silently
# contributes zero nodes. The #1666 warning above deliberately skips these (it
# only fires when an extractor exists), so surface them explicitly, grouped by
# extension, rather than reporting success as if the language were mapped.
Expand Down
2 changes: 2 additions & 0 deletions graphify/extractors/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from graphify.extractors.markdown import extract_markdown
from graphify.extractors.objc import extract_objc
from graphify.extractors.pascal import extract_pascal
from graphify.extractors.r import extract_r
from graphify.extractors.pascal_forms import extract_delphi_form, extract_lazarus_form
from graphify.extractors.powershell import extract_powershell, extract_powershell_manifest
from graphify.extractors.razor import extract_razor
Expand Down Expand Up @@ -52,6 +53,7 @@
"markdown": extract_markdown,
"objc": extract_objc,
"pascal": extract_pascal,
"r": extract_r,
"powershell": extract_powershell,
"powershell_manifest": extract_powershell_manifest,
"razor": extract_razor,
Expand Down
281 changes: 281 additions & 0 deletions graphify/extractors/r.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,281 @@
"""R extractor.

R's named functions are assignments whose left- or right-hand side is an
anonymous ``function_definition`` (``name <- function(...) ...`` and the
right-assignment forms ``function(...) -> name``), so a bespoke extractor is
required rather than the generic one. The contract mirrors the other
extractors: file/function nodes, ``contains``/``calls``/``imports`` edges and
``raw_calls`` for the shared cross-file bare-call resolver.

Package metadata (DESCRIPTION/NAMESPACE/exports) is intentionally out of scope
for this initial integration; package-qualified ``pkg::fn`` calls are recorded
as member raw facts only.
"""
from __future__ import annotations

from pathlib import Path
from typing import Any

from graphify.extractors.base import _LANGUAGE_BUILTIN_GLOBALS, _file_stem, _make_id


_ASSIGN_OPS_LEFT = frozenset({"<-", "<<-", "="})
_ASSIGN_OPS_RIGHT = frozenset({"->", "->>"})
_PKG_LOADERS = frozenset({"library", "require", "requireNamespace"})


def extract_r(path: Path) -> dict:
"""Extract functions, calls, imports, and static source() from a .r/.R file."""
try:
from tree_sitter_language_pack import get_language
from tree_sitter import Parser
except ImportError:
return {"nodes": [], "edges": [], "error": "tree-sitter-language-pack not installed"}

try:
language = get_language("r")
parser = Parser(language)
source = path.read_bytes()
tree = parser.parse(source)
root = tree.root_node
except Exception as e:
return {"nodes": [], "edges": [], "error": str(e)}

stem = _file_stem(path)
str_path = str(path)
del stem # current contract builds function ids from scope nids, not the file stem

nodes: list[dict] = []
edges: list[dict] = []
seen_ids: set[str] = set()
function_bodies: list[tuple[str, Any]] = []
pkg_imports_seen: set[tuple[str, str]] = set()

def _text(node) -> str:
return source[node.start_byte:node.end_byte].decode("utf-8", errors="replace")

def add_node(nid: str, label: str, line: int) -> None:
if nid not in seen_ids:
seen_ids.add(nid)
nodes.append({"id": nid, "label": label, "file_type": "code",
"source_file": str_path, "source_location": f"L{line}"})

def add_edge(src: str, tgt: str, relation: str, line: int,
confidence: str = "EXTRACTED", weight: float = 1.0,
context: str | None = None) -> None:
if not src or not tgt or src == tgt:
return
edge = {"source": src, "target": tgt, "relation": relation,
"confidence": confidence, "source_file": str_path,
"source_location": f"L{line}", "weight": weight}
if context:
edge["context"] = context
edges.append(edge)

file_nid = _make_id(str(path))
add_node(file_nid, path.name, 1)

def _op_of(binop) -> str:
for child in binop.children:
if child.type in _ASSIGN_OPS_LEFT or child.type in _ASSIGN_OPS_RIGHT:
return child.type
return ""

def _parent_type(node) -> str:
parent = node.parent
return parent.type if parent is not None else ""

def _named_children(node):
return [c for c in node.children if c.is_named]

def _function_body(fn_def):
for c in _named_children(fn_def):
if c.type != "parameters":
return c
return None

def walk(node, scope_nid: str) -> None:
if node.type == "binary_operator":
op = _op_of(node)
named = _named_children(node)
if op in _ASSIGN_OPS_LEFT and len(named) >= 2:
lhs, rhs = named[0], named[1]
in_call_arg = _parent_type(node) == "argument"
if (not in_call_arg and lhs.type == "identifier"
and rhs.type == "function_definition"):
name = _text(lhs)
line = node.start_point[0] + 1
func_nid = _make_id(scope_nid, name)
add_node(func_nid, f"{name}()", line)
add_edge(scope_nid, func_nid, "contains", line)
body = _function_body(rhs)
if body is not None:
function_bodies.append((func_nid, body))
walk(rhs, func_nid)
return
elif op in _ASSIGN_OPS_RIGHT and len(named) >= 2:
lhs, rhs = named[0], named[1]
if rhs.type == "identifier" and lhs.type == "function_definition":
name = _text(rhs)
line = node.start_point[0] + 1
func_nid = _make_id(scope_nid, name)
add_node(func_nid, f"{name}()", line)
add_edge(scope_nid, func_nid, "contains", line)
body = _function_body(lhs)
if body is not None:
function_bodies.append((func_nid, body))
walk(lhs, func_nid)
return
for c in node.children:
walk(c, scope_nid)
return

if node.type == "function_definition":
parent_op = _op_of(node.parent) if node.parent is not None and node.parent.type == "binary_operator" else ""
if parent_op in _ASSIGN_OPS_LEFT or parent_op in _ASSIGN_OPS_RIGHT:
for c in node.children:
walk(c, scope_nid)
return
body = None
for c in _named_children(node):
if c.type not in ("parameters",):
body = c
break
if (body is not None and body.type == "binary_operator"
and _op_of(body) in _ASSIGN_OPS_RIGHT
and _named_children(body)[-1].type == "identifier"):
body_named = _named_children(body)
rhs = body_named[-1]
name = _text(rhs)
line = node.start_point[0] + 1
func_nid = _make_id(scope_nid, name)
add_node(func_nid, f"{name}()", line)
add_edge(scope_nid, func_nid, "contains", line)
fbody = _function_body(node)
if fbody is not None:
function_bodies.append((func_nid, fbody))
if len(body_named) >= 2:
walk(body_named[0], func_nid)
return
for c in node.children:
walk(c, scope_nid)
return

for c in node.children:
walk(c, scope_nid)

walk(root, file_nid)

label_to_nid: dict[str, str] = {}
for n in nodes:
normalised = n["label"].strip("()").lstrip(".")
label_to_nid[normalised] = n["id"]

seen_call_pairs: set[tuple[str, str]] = set()
raw_calls: list[dict] = []

def _first_string(node) -> str | None:
for c in node.children:
if c.type == "string":
inner = c.children[1] if len(c.children) > 1 else c
return _text(inner)
if c.type == "argument":
return _first_string(c)
return None

def _first_identifier(node) -> str | None:
for c in node.children:
if c.type == "identifier":
return _text(c)
if c.type == "argument":
return _first_identifier(c)
return None

def _emit_pkg_import(scope_nid: str, pkg: str, line: int) -> None:
key = (scope_nid, pkg)
if pkg and key not in pkg_imports_seen:
pkg_imports_seen.add(key)
add_edge(scope_nid, _make_id(pkg), "imports", line, context="import")

def walk_calls(node, caller_nid: str) -> None:
if node.type == "function_definition":
return
if node.type != "call":
for c in node.children:
walk_calls(c, caller_nid)
return

line = node.start_point[0] + 1
fn_node = None
arguments_node = None
for c in node.children:
if c.type in ("identifier", "namespace_operator", "extract_operator"):
fn_node = c
elif c.type == "arguments":
arguments_node = c

if fn_node is not None and fn_node.type == "identifier":
callee = _text(fn_node)
if callee in _PKG_LOADERS:
if arguments_node is not None:
pkg = _first_identifier(arguments_node) or _first_string(arguments_node)
if pkg:
_emit_pkg_import(caller_nid, pkg, line)
for c in node.children:
walk_calls(c, caller_nid)
return
if callee == "source":
if arguments_node is not None:
raw = _first_string(arguments_node)
if raw and raw.lower().endswith(".r"):
if not raw.startswith(("http://", "https://", "ftp://")):
resolved = (path.parent / raw).resolve()
if resolved.exists():
add_edge(file_nid, _make_id(str(resolved)), "imports_from", line, context="import")
for c in node.children:
walk_calls(c, caller_nid)
return
is_member_call = False
elif fn_node is not None and fn_node.type == "namespace_operator":
named = _named_children(fn_node)
pkg = _text(named[0]) if named else ""
callee = _text(named[-1]) if named else ""
is_member_call = True
if pkg:
_emit_pkg_import(caller_nid, pkg, line)
elif fn_node is not None and fn_node.type == "extract_operator":
named = _named_children(fn_node)
callee = _text(named[-1]) if named else ""
is_member_call = True
else:
callee = ""
is_member_call = False

if callee and callee not in _LANGUAGE_BUILTIN_GLOBALS:
tgt_nid = label_to_nid.get(callee)
if tgt_nid and tgt_nid != caller_nid:
pair = (caller_nid, tgt_nid)
if pair not in seen_call_pairs:
seen_call_pairs.add(pair)
add_edge(caller_nid, tgt_nid, "calls", line,
confidence="EXTRACTED", weight=1.0, context="call")
else:
raw_calls.append({
"caller_nid": caller_nid,
"callee": callee,
"is_member_call": is_member_call,
"source_file": str_path,
"source_location": f"L{line}",
})

for c in node.children:
walk_calls(c, caller_nid)

walk_calls(root, file_nid)
for caller_nid, body in function_bodies:
walk_calls(body, caller_nid)

clean_edges = [e for e in edges if e["source"] in seen_ids and
(e["target"] in seen_ids or e["relation"] in ("imports", "imports_from"))]
return {"nodes": nodes, "edges": clean_edges, "raw_calls": raw_calls,
"input_tokens": 0, "output_tokens": 0}
Loading
Loading