tooling: turn line-capture and line-analyze into skills

.agents/commands/line-analyze.md → .agents/skills/line-analyze/SKILL.md

@@ -1,48 +1,70 @@
-Parse and analyze a LINE Chrome Extension network capture.
+---
+name: line-analyze
+description: Parses and redacts a LINE Chrome Extension network capture (CDP capture or Chrome net-log) and compares it with the beeper/line bridge code. Use after a line-capture run, or when given a LINE net-log or CDP capture file to analyze.
+---
+
+# LINE Analyze
+
+Turns a raw LINE Chrome Extension capture into redacted, readable findings and
+compares them with the bridge. Run the commands below from the repository root.
 
 ## Input
 
-Use the provided path to a net-log JSON file. If no path is provided, default to
+Use the provided path to a capture file. Prefer the CDP capture
+(`line-cdp-capture.json`): it includes SSE messages and request bodies. The
+Chrome net-log is a backup. If no path is provided, default to
+`/tmp/line-chrome-extension-capture/line-cdp-capture.json`, then
 `/tmp/line-chrome-extension-capture/line-net-log.json`.
 
 ## What to do
 
-1. Run the parser:
+1. Run the parser. This extracts LINE-related HTTP requests and writes a
+   `-parsed.json` file:
+
+   ```bash
+   python3 .agents/skills/line-analyze/scripts/parse-line-traffic.py <capture-file> [<parsed-output.json>]
    ```
-   python3 scripts/parse-line-traffic.py <net-log-file>
+
+2. Redact it before reading or quoting anything. This removes tokens, MIDs,
+   `picturePath` and raw body bytes:
+
+   ```bash
+   python3 .agents/skills/line-analyze/scripts/redact-line-capture.py <parsed-output.json>
    ```
-   This extracts LINE-related HTTP requests from the Chrome net-log and writes a `-parsed.json` file.
 
-2. Read the parsed output JSON file.
+   Read the raw files only locally, and only when the redacted output loses
+   information you need. Quote or document only redacted snippets.
 
-3. Analyze the traffic and present findings organized by category:
+3. Read the redacted output and present findings by category:
 
    - **Thrift RPC** (`/api/talk/thrift/`): The main LINE API. Requests use LINE's Thrift-over-HTTP protocol. The `x-lhm` header indicates the Thrift method name. Compare against methods in `pkg/line/methods.go`.
-
    - **Long-poll / SSE** (`/api/talk/long-polling/`, `/api/operation/receive`): Real-time event delivery. LF1 is used during login verification, JQ for login polling, and SSE for ongoing message/operation streaming. Compare against `pkg/line/sse.go` and `pkg/line/client.go`.
-
    - **Auth** (`/api/auth/`): Token refresh and login flows. Compare against login methods in `pkg/line/client.go` (loginV2, confirmE2EELogin, tokenRefresh).
-
    - **Media/OBS** (`obs.line-apps.com`): Media upload/download. Check `x-obs-params` header for operation details, `x-obs-oid` in responses. Compare against `pkg/line/client.go` (UploadOBS, DownloadOBS).
-
    - **Stickers/CDN** (`stickershop`, `scdn`): Sticker image downloads.
-
    - **Secondary API** (`legy-jp.line-apps.com`): Link previews and page info. Compare against `pkg/line/client.go` (GetPageInfo).
 
 4. For each interesting request, explain:
    - What the bridge equivalent is (or note if it's not implemented yet)
    - The request/response structure
    - Any headers that carry protocol-specific meaning
 
-5. Highlight any **unimplemented** endpoints or behaviors that differ from what the bridge currently does. Cross-reference with `pkg/line/methods.go` and `pkg/connector/`.
+5. Highlight any **unimplemented** endpoints or behaviors that differ from what
+   the bridge currently does. Cross-reference with `pkg/line/methods.go` and
+   `pkg/connector/`.
 
-6. After presenting the analysis, ask the user if they want to clean up the capture files (`/tmp/line-chrome-extension-capture/`). Warn that the raw net-log and parsed output contain sensitive data (auth tokens in `X-Line-Access` headers, message IDs, etc.) and should not be left on disk or committed to the repository. If the user agrees, delete the entire capture directory.
+6. After presenting the analysis, ask the user if they want to clean up the
+   capture files (`/tmp/line-chrome-extension-capture/`). Warn that the raw
+   capture and parsed output contain sensitive data (auth tokens in
+   `X-Line-Access` headers, message IDs, etc.) and must not be left on disk or
+   committed to a repository. If the user agrees, delete the entire capture
+   directory.
 
 ## LINE protocol context
 
 - The bridge identifies as LINE Chrome Extension: header `x-line-application` contains the app version string.
 - Thrift RPC uses HTTP POST to `line-chrome-gw.line-apps.com/api/talk/thrift/Talk/<Service>/<Method>`.
 - Auth token is in `x-line-access` header.
 - E2EE messages have `chunks` field containing encrypted content (base64 segments).
-- Operations have numeric `type` codes — see `pkg/connector/sync.go` and `pkg/connector/handle_message.go` for the operation type switch.
+- Operations have numeric `type` codes. See `pkg/connector/sync.go` and `pkg/connector/handle_message.go` for the operation type switch.
 - Body content in the parsed output may be base64-encoded binary (Thrift serialization). Note the method name from headers rather than trying to decode the binary body.

scripts/parse-line-traffic.py → .agents/skills/line-analyze/scripts/parse-line-traffic.py

@@ -2,7 +2,7 @@
 """Parse LINE network captures (CDP or net-log format).
 
 Usage:
-    python3 scripts/parse-line-traffic.py <capture.json> [output.json]
+    python3 .agents/skills/line-analyze/scripts/parse-line-traffic.py <capture.json> [output.json]
 
 Supports two input formats:
 - CDP capture (from line-cdp-capture.py): has SSE message bodies, Thrift response bodies
@@ -231,14 +231,17 @@ def parse_cdp_capture(data: dict) -> dict:
             entry["status"] = req["status"]
 
         # Extract interesting request headers
-        headers = req.get("headers", {})
+        # requestWillBeSentExtraInfo carries the headers Chrome really sent
+        # (Cookie, Origin, Sec-*, X-Line-*), which requestWillBeSent omits.
+        headers = {**req.get("headers", {}), **req.get("request_extra_headers", {})}
         interesting_req = {}
         for k, v in headers.items():
             if k.lower() in (
                 "content-type", "x-line-access", "x-line-application",
                 "x-lhm", "x-lpv", "x-lsr", "x-obs-params",
-                "x-line-channeltoken", "accept",
-            ):
+                "x-line-channeltoken", "accept", "origin", "user-agent",
+                "x-line-chrome-version", "x-line-mid",
+            ) or k.lower().startswith("x-line-") or k.lower().startswith("x-ls"):
                 interesting_req[k] = v
         if interesting_req:
             entry["request_headers"] = interesting_req
@@ -270,7 +273,19 @@ def parse_cdp_capture(data: dict) -> dict:
 
         # Include request body if available
         if "request_body" in req:
-            entry["request_body_text"] = req["request_body"]
+            body = req["request_body"]
+            if isinstance(body, dict):
+                # Raw captures keep the bytes in base64; the parsed output only
+                # keeps the text form and a size + sha256 for binary bodies.
+                if body.get("error"):
+                    entry["request_body_error"] = body["error"]
+                else:
+                    entry["request_body_size"] = body.get("size")
+                    entry["request_body_sha256"] = body.get("sha256")
+                    if body.get("text") is not None:
+                        entry["request_body_text"] = body["text"]
+            else:
+                entry["request_body_text"] = body
 
         entry["timestamp"] = req.get("timestamp", "")
         line_requests.append(entry)

scripts/redact-line-capture.py → .agents/skills/line-analyze/scripts/redact-line-capture.py

@@ -2,7 +2,7 @@
 """Redact sensitive values from LINE capture JSON files.
 
 Usage:
-    python3 scripts/redact-line-capture.py input.json [output.json]
+    python3 .agents/skills/line-analyze/scripts/redact-line-capture.py input.json [output.json]
 
 The default output path is input-redacted.json. This helper preserves JSON
 structure for protocol analysis while removing credentials, cookies, tokens,
@@ -39,6 +39,16 @@
     "refresh_token",
     "session",
     "certificate",
+    # Identifiers that point at a person, group or uploaded object.
+    "mid",
+    "chatmid",
+    "contactmid",
+    "picturepath",
+    "picturestatus",
+    "oid",
+    "x-obs-oid",
+    # Raw request-body bytes from the CDP recorder (uploads, Thrift payloads).
+    "base64",
 }
 
 SENSITIVE_KEY_FRAGMENTS = (
@@ -53,7 +63,8 @@
 
 EMAIL_RE = re.compile(r"(?i)[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}")
 JWT_RE = re.compile(r"eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}")
-MID_RE = re.compile(r"\b([ucr][0-9a-f]{20,})\b", re.IGNORECASE)
+# Not \b: "_" counts as a word character there, which hid MIDs after "_".
+MID_RE = re.compile(r"(?<![0-9A-Za-z])([ucr][0-9a-f]{20,})(?![0-9A-Za-z])", re.IGNORECASE)
 LONG_SECRET_RE = re.compile(r"^[A-Za-z0-9+/=_-]{96,}$")
 
 

.agents/skills/line-capture/SKILL.md

@@ -0,0 +1,75 @@
+---
+name: line-capture
+description: Launches Chrome with network capture (CDP recorder plus net-log) to record LINE Chrome Extension traffic for the beeper/line Matrix bridge, after checking the dependencies. Use when you need raw LINE Chrome Extension traffic for a feature, or when asked to capture, record or sniff LINE traffic.
+---
+
+# LINE Capture
+
+Records what the real LINE Chrome Extension sends, so the bridge can copy it.
+Run the commands below from the repository root.
+
+## What to do
+
+1. **Check dependencies.** Run `bash .agents/skills/line-capture/scripts/line-capture-preflight.sh`
+   (on Windows, from Git Bash). It prints the detected OS and an `OK` or
+   `MISSING` line per dependency. It does not say how to install anything:
+   choose the install method that fits the detected OS and the package managers
+   that are actually available (for example Homebrew on macOS; apt, dnf or
+   pacman on Linux; winget, Chocolatey or Scoop on Windows; `<python> -m pip
+   install --user` for Python packages, using the Python the preflight
+   reported). Show the user the missing items with the exact commands you plan
+   to run, ask them to confirm, then install and re-run the preflight until it
+   exits 0. Never install without that confirmation. The launcher refuses to
+   start while dependencies are missing.
+
+2. **Close any running Chrome instances first.** Chrome only supports one
+   instance per profile with remote debugging. Warn the user if Chrome is
+   running.
+
+3. Run the capture script from the checkout:
+
+   ```bash
+   bash .agents/skills/line-capture/scripts/line-chrome-capture.sh /tmp/line-chrome-extension-capture
+   ```
+
+   This launches Chrome with:
+   - Remote debugging on port 9222
+   - Full network logging to `/tmp/line-chrome-extension-capture/line-net-log.json`
+   - A persistent capture-only Chrome profile at
+     `~/.cache/line-chrome-extension-capture/chrome-debug-profile`
+   - Auto-loaded LINE and Codex extensions staged from the user's normal Chrome
+     profile into `~/.cache/line-chrome-extension-capture/extensions`
+   - A LINE extension version preflight:
+     - repo version from `pkg/line/client.go` `ExtensionVersion`
+     - installed/staged Chrome extension version from `manifest.json`
+     - a warning if those versions differ
+   - First-run Chrome prompts pre-skipped:
+     - "Make Google Chrome the default browser" stays unchecked
+     - "Automatically send usage statistics and crash reports" stays unchecked
+     - Chrome/Sync sign-in is skipped; use "Stay Signed Out" if it appears
+   - If the LINE extension is missing and the user asked for setup, relaunch
+     with `LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0`, open the official Web Store
+     listing for `ophjlpahpchlmihnnnihgmmeilfjmjjc`, verify LINE / LY
+     Corporation, click Add to Chrome, and accept the standard "Add extension"
+     confirmation
+
+4. Tell the user:
+   - Chrome opens on `about:blank`. Open LINE from the extension's toolbar icon
+     (puzzle-piece menu if it is not pinned), not by opening the extension's
+     `index.html` as a tab: that behaves differently from the real popup.
+     `LINE_CAPTURE_START_URL` overrides the start URL.
+   - Log in manually if the capture profile is not already logged in
+   - Perform whatever flow they want to analyze
+   - Do not open DevTools on the extension: LINE has `debugger;` traps that
+     pause it, and the recorder does not need DevTools
+   - Windows the extension opens (Settings, group popups) can start frozen
+     and blank, with LINE greyed out as "Debugger paused in another tab". The
+     recorder releases them automatically. If one still freezes, quit Chrome
+     and run the capture again.
+   - When done, quit Chrome completely (all windows, not only the tab)
+
+5. Once Chrome is closed, the capture files are ready. Continue with the
+   `line-analyze` skill.
+
+Raw captures are secrets (tokens, cookies, MIDs, message bodies). Never commit
+them. See the `line-implement` skill for the full security rules.