flyntor

<div align="center"> <img alt="flyntor" src="https://raw.githubusercontent.com/flyntor/flyntor/master/assets/flyntor.png" width="172"> <h1>flyntor</h1> <p><strong>Pull a web application's own source out of production and read it like a repository.</strong></p> <p> <img alt="licence MIT" src="https://img.shields.io/badge/licence-MIT-22C55E?style=flat-square&labelColor=0F172A"> <img alt="node 20.11+" src="https://img.shields.io/badge/node-%E2%89%A5%2020.11-22C55E?style=flat-square&labelColor=0F172A"> <a href="https://github.com/flyntor/flyntor/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/flyntor/flyntor/actions/workflows/ci.yml/badge.svg"></a> <img alt="no build step" src="https://img.shields.io/badge/build%20step-none-22C55E?style=flat-square&labelColor=0F172A"> <img alt="no telemetry" src="https://img.shields.io/badge/telemetry-none-22C55E?style=flat-square&labelColor=0F172A"> </p> <p> <a href="#install">Install</a> · <a href="#quickstart">Quickstart</a> · <a href="#how-it-works">How it works</a> · <a href="#what-a-crawler-misses">Why</a> · <a href="#commands">Commands</a> · <a href="#drive-it-from-an-agent">MCP</a> · <a href="#scope-is-not-optional">Scope</a> </p> </div> --- A web application hands its entire client to anyone who asks, then serves it back as a wall of minified JavaScript. flyntor reads the bundler's own chunk table, fetches the modules the page never loads, recovers source maps where they were shipped, and writes a source tree you can open in an editor. What you get is a directory of the target's real code. Not a crawl, not a URL list. The files, with the paths they had before the build.  <p align="center">_{Real CLI output against a controlled localhost fixture &middot; 4 chunks, including lazy chunks &rarr; 7 restored source files &rarr; 5 offline endpoint findings}</p> ## Install ```bash npm install -g flyntor ``` Or from a clone, if you want to read it first or change it: ```bash git clone https://github.com/flyntor/flyntor.git cd flyntor npm install npm link ``` ```console $ flyntor --version ``` `flyntor acquire`, `flyntor endpoints` and the rest now work from any directory. To install without touching `PATH`, skip `npm link` and run `node bin/flyntor.mjs` from the checkout. Node 20.11 or newer. There is no build step: the CLI is TypeScript and runs as TypeScript, so what you read in `src/` is exactly what executes. Point a tool at systems you are responsible for and you should be able to check that without reconstructing it from a bundle. `session capture` opens a real browser through `playwright-core`. If you have never used Playwright on this machine, run `npx playwright install chromium` once. Every other command works without it. ## Quickstart > [!NOTE] > `--scope` is mandatory on anything networked. Identify your traffic with `--ua` and > keep `--rate` inside the program's limit. ```bash # 1. Take the application apart. Scope is required; nothing leaves without it. flyntor acquire https://app.example.com/ \ --scope "*.example.com" \ --ua "YourHandle-Bugbounty" \ --rate 1 # 2. Read it. Free, offline, instant. flyntor endpoints flyntor secrets flyntor listeners # 3. Or read it yourself. $EDITOR corpus/app/ ``` `acquire` writes: ``` corpus/ MAP.md where every file came from, with its origin URL findings.jsonl one record per observation, appended by every lens app/ the target's own code, source tree reconstructed app/index.html the document itself, filed by the route it was served for vendor/ frameworks and libraries data/ locale tables, one kept per key space ``` ## How it works  | phase | what flyntor does | |---|---| | **Discover** | reads the bundler runtime and enumerates entry points, lazy chunks and source maps | | **Fetch** | requests only in-scope assets, at the configured rate, with an attributable User-Agent; keeps the document, `robots.txt` and `sitemap.xml` | | **Rebuild** | restores shipped source content, separates app from vendor code and reconstructs paths | | **Inspect** | runs local lenses over bundles, the document, API specs and crawler files, and writes evidence-backed records to `findings.jsonl` | ### What leaves your machine | command | network | sends | writes | |---|---|---|---| | `acquire`, `probe`, `dns`, `subdomains`, `session capture` | explicit | scoped requests or public CT lookups | corpus or local session data | | `endpoints`, `secrets`, `listeners`, other lenses, `graph`, `diff` | none | nothing | local findings or graph files | | `read`, `agent` | explicit provider API | selected corpus content | verified local findings | `flyntor mcp` is offline by default. Its network tools do not exist unless the server is started with `--allow-network`. ## What a crawler misses **The code the application does not load.** A page fetches only what its current route needs. The admin panel, the billing flow, and the module behind a role you do not hold sit on the origin as static files that nobody requests. flyntor reads the bundler runtime's chunk table and enumerates them. The naming scheme is read from the runtime rather than assumed, because builds disagree: | runtime expression | chunk naming | |---|---| | `a => a + ".js?ver=" + {…}[a]` | hash in a query parameter | | `e => "static/js/" + e + "." + {…}[e] + ".chunk.js"` | hash in the filename | | `e => e + ".js"` | no hash at all | **Source maps, where the build shipped them.** On one billing portal, 62 fetched assets became **620 files** with real `.vue` components at real paths, in two minutes. A minified chunk is evidence. The component it was built from is source. **An app whose session has lapsed.** An authenticated console redirects its document to a login page the moment the session expires. The chunks that document names are static assets needing no session at all. Point flyntor at the runtime script and it still gets the application. **Paths, rebuilt.** `apps_manager_src_app_modules_security_security_module_ts.js` becomes `app/apps/manager/src/app/modules/security/security.module.ts.js`. Reconstruction is best effort, since `_` stands for both `/` and `.`, so `MAP.md` keeps the origin URL. That is the fact; the tree is the convenience. **The document itself.** The entry page is the one file that did not come out of a build, and it is kept. Inline `<script>` bodies feed every lens as if they were a file of their own, with line numbers that point at the real document. On top of that the lenses read what only markup states: each `<form action>` with its field names and which are hidden, the hosts the page loads scripts and styles from, a `<meta>` CSP, and HTML comments, where a staging host or a commented-out route tends to survive a deploy. **What the site tells crawlers.** `robots.txt` and `sitemap.xml` are fetched through the same gate and kept when they are real, not the app's shell answering for a missing file. Every `Disallow` becomes an endpoint row, since a path the site asks crawlers to leave alone is a strong hint about where the surface is; every sitemap `<loc>` becomes one too. **API descriptions the build shipped.** An OpenAPI 3 or Swagger 2 document found in a corpus is parsed rather than pattern-matched: every path with its method, parameters, and whether the operation requires auth. An operation that overrides a document-level `security` block with an empty one is reported as stating no auth requirement, which is the row worth reading first. **A refusal to make a mess.** Requests are scope-checked, spaced, and carry the attribution a program requires. A `200` that is really the app's shell standing in for a missing asset is rejected rather than filed as source. Seven translation tables holding one key space between them are collapsed to one. ## Commands ### Get the source | | | |---|---| | `acquire <url>` | fetch a page or runtime, follow its chunk table, write the corpus | | `session capture --url <login>` | open a browser, log in yourself, keep the session | | `session status` | what is stored, and whether it has expired | ### Read it. No model, no network, no cost Each appends to `corpus/findings.jsonl`. Run these before you spend anything. | | | |---|---| | `hosts` | every host the application talks to, loads from, or links to | | `endpoints` | routes it calls, with the verb where the code states one; form actions; paths a spec declares | | `graphql` | operations and the fields they select | | `sinks` | untrusted input reaching somewhere dangerous | | `secrets` | credential-shaped literals | | `listeners` | `postMessage` handlers, and whether each checks origin | | `constants` | roles, permissions and flags the build baked in | | `graph` | module imports and lazy-loaded routes, to `corpus/graph.json` | | `diff <before> <after>` | what changed between two acquisitions | `diff` is worth a habit. Code shipped since your last acquisition is the code least likely to have been looked at by anyone, and a route that vanished from the frontend has very often not vanished from the server. ### Read it with a model These are the only commands that cost money. | | | |---|---| | `read` | every claim checked against the file's bytes; unverifiable ones discarded | | `agent ["question"]` | ask about the corpus, with tools that verify quotes | | `config` | model profiles, stored outside the corpus so no key travels with it | `read --dry-run` costs nothing and asks for nothing. Run it first. `read --from corpus/findings.jsonl` reads only the files the free lenses flagged, which on a real corpus was a 19x difference in spend. ### Reach the target | | | |---|---| | `probe <url>` | one request through the gate, for confirming a single hypothesis | | `dns` | resolve the hosts the corpus names; private addresses, dangling CNAMEs | | `subdomains <apex>` | certificate-transparency lookup, filtered to scope | `subdomains` sends nothing to the target. CT logs are a public record, so it cannot breach a rate limit or appear in anyone's access log. <details> <summary><b>Full option reference</b></summary> ``` --scope <a,b> hosts the program permits. Required on anything networked. "*.example.com" covers subdomains and the apex. --ua <suffix> appended to the User-Agent. Programs require attribution; supply it. --rate <n> requests per second, default 1 --limit <n> ceiling on files fetched, default 300 --cookies <file> a cookies.txt export --session use a session captured with "session capture" --out <dir> corpus directory, default ./corpus --plain lens output as one value per line on stdout, deduplicated --json lens output as one JSON record per line on stdout --vendor include vendor/ in a lens run, normally skipped --verbose every file, every request, and the model's reasoning --version print the installed version and exit acquire --limit, --rate, --cookies, --session, --ua graph --external include third-party modules read --lens authz|secrets|sinks|flows (omit for a full pass) --from <jsonl> read only the files that file names --resume skip files already read, matched on content --dry-run what it would cost, without spending it dns --hosts <a,b> names to resolve, instead of the corpus's --tls also check whether a strict client can build the chain mcp --allow-network offer the two tools that send traffic probe --ua <suffix> session capture --url <login url> [--as <name>] | status model options (read / agent / config only) --provider <p> deepseek | openai | openrouter | together | local --effort <e> low | medium | high | xhigh, where the provider supports it --max-cost <n> ceiling on spend, in USD. Real, not advisory. --key, --base-url, --model override the saved values for one run --max-files <n> default 5000 --concurrency <n> default 8 --max-tokens <n> --timeout <s> --yes approve requests without asking FLYNTOR_API_KEY, FLYNTOR_BASE_URL, FLYNTOR_MODEL, FLYNTOR_EFFORT ``` `flyntor help` prints the same thing with prose. </details> ## Pipe it into everything else Any lens takes `--plain` for one value per line, deduplicated, or `--json` for the whole record. Both put findings on stdout and move the renderer to stderr, so what a pipe receives is a list and nothing else. ```bash flyntor endpoints --plain | httpx -mc 200,401,403 flyntor hosts --plain | anew hosts.txt flyntor secrets --json | jq -r 'select(.state == "VERIFIED") | .value' flyntor dns --scope "*.example.com" --plain | anew dns-findings.txt flyntor subdomains example.com --scope "*.example.com" --json | jq -r .host ``` `dns` and `subdomains` take the same flags, so a whole sweep pipes end to end. <details> <summary><b>The record shape</b></summary> ```json { "kind": "ENDPOINT", "value": "/api/v2/workspaces/:id/members", "detail": "called from shipped code", "derived": false, "file": "app/src/features/workspace/members.service.ts", "line": 356, "snippet": "… const membersUrl = \"/api/v2/workspaces/:id/members\", …", "state": "VERIFIED", "reader": "inspect:endpoints" } ``` `state` is the honest part. `VERIFIED` means the quoted snippet was found in that file at that line. `RELOCATED` means it was found, at a different line. `CONTRADICTED` rows are discarded before they reach you. A claim nobody can point at is not a finding. </details> ## Drive it from an agent | | | |---|---| | `mcp` | serve the corpus tools over MCP on stdio | `flyntor mcp` puts the corpus behind an MCP server, so an agent you already run can drive it. ```bash claude mcp add flyntor -- node /path/to/flyntor/bin/flyntor.mjs mcp \ --scope "*.example.com" --out /path/to/corpus ``` Eleven tools, none of which sends traffic: `graph_query`, `endpoint_callers`, `params_for`, `curl_for`, `list_files`, `open_file`, `search`, `findings_query`, `corpus_diff`, `verify_snippet`, `inspect`. `--allow-network` adds `probe` and `resolve_hosts`. Without it they are not listed at all, so a model driving flyntor cannot reach a host whether or not that host is in scope. The gate lives in the server rather than in the prompt: the model proposes, `checkScope` disposes, and a refusal comes back as a tool result instead of as a request. There is a skill for agents at [`skill/flyntor.md`](skill/flyntor.md) covering the sweep order and what each lens is worth reading for. ## Scope is not optional > [!IMPORTANT] > This is a tool for security research you are permitted to perform: a bug bounty program, a > penetration test with a signed scope, or your own systems. `--scope` is required on every command that touches the network. Without it nothing leaves the machine, which is the whole point. A host found inside a bundle is not in scope by virtue of having been found. What that buys, concretely. On one engagement the free `hosts` lens named 14 hosts and `dns` refused 8 of them as out of scope, among them a third-party mapping API the program's rules put off limits. Up to that point the rule had been enforced by remembering it. Single deliberate requests, enforced scope, mandatory attribution, and no scanning behaviour anywhere in the tool. ## What it has actually found A tool's track record is more useful than its feature list. - **Internal RFC1918 addressing in public DNS** across ten in-scope hosts, and an **incomplete TLS chain** on an apex that browsers paper over and Node does not. Both reported. Neither is visible from any corpus. - **Two hosts referenced from production JS with no DNS record**, one leaking a non-production naming convention, that a manual sweep of the same target had missed. - **A bot-detection result worth knowing.** A *perfect* bare Chrome User-Agent got `403` where the same UA with an identifying suffix got `200`, repeatably. A flawless browser claim arriving on a non-browser TLS stack is a contradiction, and it is the cheapest heuristic there is. flyntor's User-Agent is browser-shaped but never anonymous. The lenses earn their keep by removing work as much as by finding it: 67 sinks dismissed in seconds because every row carries what feeds it, and 25 of 25 hosts correctly labelled as comment mentions rather than as traffic. ## Development ```bash npm test # 608 tests, 35 files npm run typecheck # tsc --noEmit, strict npm run recall # regression harness: 18 fixtures over real corpora ``` `recall` is the one that matters. It runs the extractors against saved corpora from real targets and fails if a route that used to be found stops being found, or if a route the code never states starts appearing. An invented URL is the worst thing this tool can produce, and it is the thing an ordinary test suite is least likely to catch. ## Licence MIT.