falconeye

# FalconEye **Free, self-hosted OSINT investigator's toolkit.** Eighteen focused modules in one interface: crypto wallet tracing, phishing kit fingerprinting, domain intelligence, Telegram OSINT, IP reputation, email header forensics with LLM-powered scam detection, Google dork generation, suspicious script deobfuscation, URL expansion and redirect chain analysis, QR code decoding, commercial prospect dossiers, reverse image search, username enumeration across ~950 platforms, Have I Been Pwned breach checks, global + PH/SEA ransomware victim tracking, and a fictional sock-puppet persona generator with dossier export. The home page carries a Philippines-focused threat pulse and a curated news strip. The IP Reputation and Email Header tabs also compose abuse reports to the responsible provider (RDAP contact lookup, with optional Mailgun send). Current version: **3.36.2** Live instance: [falconeye.osintph.info](https://falconeye.osintph.info) License: AGPL-3.0 --- ## Use FalconEye from Claude Code or Claude Desktop FalconEye ships an **MCP server**, so seven of the tabs are available as tools inside Claude Code and Claude Desktop. Point it at your own instance and ask for a lookup instead of opening a browser tab. ```bash claude mcp add falconeye -- /opt/falconeye/mcp-venv/bin/python -m app.mcp_server ``` | Tool | Input | |---|---| | `ip_reputation` | a public IPv4 or IPv6 address | | `domain_intel` | a hostname | | `email_header_analyze` | raw email headers, optionally with the body | | `script_decode` | obfuscated script source | | `url_expand` | a shortened or redirecting URL | | `qr_analyze` | a QR code image (file path or data URI) | | `ransomware_watch_search` | a victim, sector or group name | **It runs against your own instance, your own `.env` and your own API keys.** There is no hosted endpoint and no account: each tool calls the same route the browser tab calls, in-process, so your rate limits, the SSRF guard and the prompt-safety wrapping all apply unchanged, and the two LLM-backed tools spend your Anthropic budget under the existing daily cap. stdio transport only: no listener, no authentication, no key table, because the only caller is the user who launched the process. That also means it can drive a remote instance over SSH without opening a port. The person-centric tabs (username, phone, Telegram, reverse image search, sock puppet, dork generator) are deliberately not exposed. Setup, the Claude Desktop config block and the SSH pattern: **[docs/mcp.md](docs/mcp.md)**. --- ## What it does FalconEye is the workbench an OSINT investigator opens when a new lead arrives. Each tab does one thing well and connects to the others via one-click pivots, from "I have a wallet address" to "here are the related domains, email infrastructure, Telegram channel, and the script the phishing kit runs" without switching tools. ### Tabs | Tab | What it does | |---|---| | **Home** | Landing page with PH Threat Pulse widget, example cards that prefill other tabs, and a curated news strip | | **Crypto Workbench** | Trace Bitcoin, Ethereum, and USDT TRC20 addresses with D3 force-directed transaction graphs and labelled clusters | | **Phishing Scanner** | Fingerprint phishing kits by URL or pasted HTML; identify the kit family; extract IOCs. An opt-in **Deep kit report** toggle upgrades the same run to a full teardown: fetch the page, detect a client-rendered SPA shell, pull and hash every JS bundle, decode the obfuscated string table and resolve decoder call sites before searching, probe the operator relay path (status codes only, no socket opened), pull RDAP, CT and urlscan, score against a kit signature, and render one case report with a sourced lifecycle timeline and a copyable indicator block. The case host always comes from the URL you submitted: a cloaking kit that redirects the scanner to the brand it impersonates cannot redirect the investigation with it, and a fetch that leaves the submitted domain is reported as a cloaked redirect with the chain, scored on the redirect itself, with every skipped stage marked N/A rather than 0%. Pasting a JavaScript bundle instead of page source runs the same analysis offline, for a target that is already dead. | | **Domain Intelligence** | RDAP, DNS, certificate transparency logs (crt.sh + Cert Spotter fallback), RIPEstat ASN data | | **Telegram Inspector** | Scrape public Telegram channels (t.me/s/) for messages and extract IOCs (URLs, wallets, contact details) | | **IP Reputation** | Multi-source reputation with a consensus verdict (Clean/Suspicious/Malicious/Incomplete) from AbuseIPDB, VirusTotal, AlienVault OTX, Censys, and ThreatFox, plus Shodan InternetDB, GreyNoise, RIPEstat, URLhaus, reverse DNS. Merges Censys + Shodan ports and surfaces geolocation disagreement across sources. ASN Intelligence block (RIPEstat): operator identity, full announced-prefix footprint, and an expand-to-load routing view of upstream/downstream ASN relationships, the pivot from one abusive IP to the operator's whole network. Composes an abuse report to the hosting provider's RDAP abuse-c contact (copy, optional Mailgun send, or client-side PDF). Reputation reports export to PDF in the browser. | | **URL Expander** | Follow a short URL's full redirect chain hop-by-hop with per-hop status codes, TLS certificate details, server headers, and timing. Flags shortener depth, TLD switches, punycode hostnames, and non-standard ports. Every hop is re-validated through `safe_fetch`'s SSRF guard. One-click pivot to the Phishing Scanner. | | **QR Analyzer** | Decode one or more QR codes from an uploaded image or a base64 data URI (processed in memory, never stored). Categorizes payloads (HTTP, Bitcoin, Ethereum, UPI, WiFi, sms, tel, geo, text). One-click pivot of decoded URLs into the URL Expander. | | **Image** | Reverse image search via Google Lens and Yandex in parallel. Paste a URL or upload a file (JPEG, PNG, WebP, GIF, max 10 MB). Shows visual match grids, cross-source domain corroboration, and EXIF metadata for uploads. Requires a SearchAPI.io key. Results cached 24 hours in Redis. | | **Email Header** | Authentication checks (SPF/DKIM/DMARC), hop analysis with ASN attribution, body scam pattern detection. LLM-powered classification with validated, clamped output. Supports .eml and .msg file upload. Inline file-hash reputation (URLhaus + MalwareBazaar) on extracted hashes. Composes abuse reports to the sending IP's hoster and the sender domain's registrar. | | **Sock Puppet Generator** | Build a coherent fictional research persona for authorized OSINT use: a Cover (deterministic, from Faker plus an offline country-metadata layer covering all ISO countries) and an optional Legend (Claude Haiku back-story pinned to the Cover so it cannot contradict it). Non-Latin locales carry a Latin romanization twin per field (Legend-written, else `unidecode` fallback) so nothing renders blank; 6 to 8 human-looking username suggestions derived from the roman name and persona facts; per-field copy buttons, financial filler off by default, dossier PDF, and an operator OPSEC checklist. Fictional personas only, never impersonation. | | **Dork Generator** | LLM-powered Google search query generator with preset categories and free-form natural-language input | | **Script Decoder** | LLM-powered deobfuscation of suspicious PowerShell, JavaScript, VBA, Base64 blobs, and packed scripts. Returns deobfuscated code, IOCs, MITRE ATT&CK techniques, and detection suggestions | | **Prospect** | Commercial intelligence dossier for any domain: identity resolution, news, job postings, and Google Ads Transparency data. Requires a SearchAPI.io key. Results cached 6 hours in Redis. | | **Contact** | Feedback form for bug reports, feature requests, and new tab suggestions | | **Username** | Check where a username appears across ~950 platforms using vendored WhatsMyName + Sherlock data. Dual-engine with cross-validation (hits in both engines are higher confidence). Quick (~280 sites) and Full scans, adult sites off by default, CSV export, Telegram pivot. | | **Breach Check** | Have I Been Pwned integration: email breach + paste exposure lookup with full breach metadata, timeline, and paste history; client-side password check via K-anonymity (SHA-1 prefix only, password never leaves the browser); domain breach history lookup. Sensitive breaches redacted by default (toggle to reveal). Pivots to Username Enumeration, Domain Intelligence, and IP Reputation. Requires an `HIBP_API_KEY` (HIBP Core 1 subscription). Breach data attributed to Have I Been Pwned under CC BY 4.0. | | **Route Map** | Map the real network path to a target and draw it on a world map, starting from where you are. Enter a domain, URL or IP and a RIPE Atlas probe on your own network runs the traceroute; or paste a trace, or run one yourself and pipe it to a single-use upload link. Windows `tracert`, Unix `traceroute` and `mtr --report` are all parsed, format auto-detected. Hops are placed **hostname first**: CAIDA Hoiho's published ruleset, then a bundled carrier site-code table, and an IP geolocation database only as a fallback, because a backbone router's address geolocates to wherever its operator registered the prefix. Every placement is then checked against the speed of light (about 100 km per ms of round trip) and rejected if the measured RTT could not reach it, so a Tata router cannot be drawn in Harare at the same RTT as San Jose. Hops that cannot be placed are listed with the reason, never silently dropped. Annotates asymmetric return paths, ICMP rate limiting mistaken for packet loss, and a destination that does not answer ICMP. Your location is asked for through the browser's own permission popup, rounded to about 10 km in the browser, used for the render only and never stored; declining falls back to an offline city picker. Atlas measurements are public and the tab says so. Needs an `ATLAS_API_KEY` for the probe path; paste and upload work without one. | | **Ransomware Watch** | Global + PH/SEA ransomware victim tracking: world map, regional (PH/SG/MY/ID/TH/VN/HK/TW) comparison with month-over-month trend, latest victims with cross-source corroboration flags, group activity, leak-site mirror health, and a PH-relevant watchlist. Reads a locally scheduled collector's cache only for its core panels, never queries an upstream from the browser there. Two guarded exceptions: a company search (always live against ransomware.live, min 3 characters, rate-limited, degrades to a local partial-cache fallback if upstream is down) and an on-demand country lookup for any country outside the continuously-collected PH/SEA set (cached 24h, rate-limited). Every entry is labelled a claim, never a confirmed fact. Requires a `RANSOMWARE_LIVE_API_KEY` (ransomware.live PRO); RansomLook needs no key. Data attributed to ransomware.live (non-commercial terms) and RansomLook (CC BY 4.0). | ### LLM-powered tabs Four tabs use Anthropic's Claude Haiku 4.5: **Email Header**, **Dork Generator**, **Script Decoder**, and **Sock Puppet Generator** (the Legend back-story). The model is hardcoded in each router (not a configuration variable) and protected by four cost safeguards (the Sock Puppet Legend is capped per IP per 24 hours, default 25, configurable via `SOCKPUPPET_LLM_PER_DAY`, since bulk legend generation is its abuse case): 1. Anthropic Console spend limits at the API account level 2. Per-feature, per-IP daily rate limits (10 calls per 24-hour rolling window), keyed on `CF-Connecting-IP` so limits track real users behind Cloudflare, not edge nodes 3. Environment variable kill switches (`LLM_ANALYSIS_ENABLED`, `LLM_DORKGEN_ENABLED`, `LLM_DECODER_ENABLED`) 4. Prompt caching on system prompts (~90% input cost discount on repeated calls within 5 minutes) LLM JSON output is schema-validated and clamped before use: numeric fields are type-coerced and range-clamped, string fields are truncated, enum fields (severity, intent, risk_level) are validated against known values, and finding lists are filtered to allowed keys only. LLM output is presented in the UI as model opinion, not as verified verdict. Typical cost per LLM call: ~$0.003. --- ## Abuse Reporting The IP Reputation and Email Header tabs can turn an identified piece of hostile infrastructure into an abuse report to the responsible provider. FalconEye resolves the abuse contact via RDAP, prefills a category-appropriate report from what it already knows, and offers two actions: - **Copy to Clipboard**: always available, no configuration, no authentication. Paste into your own mail client and send from there. - **Send via Mailgun**: optional. Gated behind admin HTTP Basic Auth, rate-limited per IP / per recipient / globally, and restricted to recipients FalconEye itself resolved via RDAP. Every send is written to an append-only audit log. Compose-and-copy works out of the box. Enabling send requires reporter-identity and Mailgun environment variables plus an admin bcrypt hash. See **[docs/abuse-reporting.md](docs/abuse-reporting.md)** for the full setup guide, security posture, and current Mailgun free-tier state. --- ## Security posture FalconEye is a public, unauthenticated OSINT tool with no login. The following controls are in place as of v3.36.2: **SSRF prevention (Phishing Scanner + URL Expander).** All user-supplied URLs pass through the shared `safe_fetch` primitives before any HTTP request is made. `safe_fetch` resolves and validates every hop in a redirect chain independently against a complete blocklist: private/loopback/link-local/reserved/multicast/unspecified ranges (via the Python `ipaddress` stdlib), CGNAT (100.64.0.0/10), NAT64 (64:ff9b::/96), IPv4-mapped IPv6 (::ffff:a.b.c.d unwrapped before check), and the "this" network (0.0.0.0/8). The URL Expander re-runs this check (`resolve_and_check`) at the start of every hop and before its per-hop TLS grab, and rejects embedded userinfo; it does not add a second SSRF implementation. TLS certificate verification is enforced on all outbound fetches (`verify=True`). Response bodies are streamed and size-capped (10 MB by default, 2 MB per hop in the URL Expander) so a target cannot choose how much memory a fetch costs. Fixed-host API calls (Shodan, RDAP, Telegram, etc.) are not routed through `safe_fetch` as they are not SSRF surfaces. **Rate limiting.** All per-IP limits, including LLM cost controls and phishing scanner, are keyed on `get_client_ip()`, which honours the `CF-Connecting-IP` header **only when the direct TCP peer is inside a published Cloudflare range** (`app/utils/cloudflare_ips.py`) or an operator-configured `TRUSTED_PROXY_CIDRS` network. From any other peer the header is ignored and the limit keys on the real peer address, so a caller that reaches the origin directly cannot rotate a header to mint fresh counters. Header values that are not well-formed IP addresses are ignored rather than becoming their own bucket. This is deliberately independent of the nginx allowlist below: the limits hold even if the origin is exposed. The fallback for local development (no trusted peer, no header) is `request.client.host`. **XSS.** Attacker-controlled strings from Telegram channel metadata, RDAP registration fields, RSS feeds, and threat intelligence APIs are escaped with `escapeHtml()` / `escapeAttr()` before any DOM insertion. The existing escape helpers are used consistently; no `innerHTML` is assigned with unescaped external data. **Reputation coverage is stated, not assumed.** The IP Reputation verdict is only `CLEAN` when all five keyed sources actually answered. A source with no key, an error, a timeout or an exhausted quota is reported as unavailable and the verdict becomes `INCOMPLETE`, naming which source was not consulted. Before v3.33.2 such a source was indistinguishable from one that answered and found nothing, so the card could assert "No source flagged this IP" when nothing had been asked. Every verdict now carries an "N of M reputation sources responded" note, and the tab warns before a lookup when the instance has no keys at all. **Optional third-party source: Hudson Rock.** Off by default (`HUDSONROCK_ENABLED`). When on, Domain Intel and the Email Header risk assessment gain infostealer exposure data. Only stealer family names, compromise dates and counts are ever rendered: the client reduces every upstream response to an allowlist of safe fields, because Hudson Rock's schema also carries plaintext credentials, session cookies and victims' search history. Enabling it forwards your visitors' lookups to a third party, and the endpoints publish no rate limit and no terms of use, so it is treated as best effort. See "Hudson Rock" in [docs/deploy-runbook.md](docs/deploy-runbook.md). **MCP server (optional, local mode).** The server described under "Use FalconEye from Claude Code or Claude Desktop" above is not part of the web service and must not be exposed on a socket or a tunnel: stdio means the only caller is the user who launched the process, which is also why it carries no authentication. Its tools call the same routes the browser tabs call, so the rate limits, the SSRF guard and the prompt-safety wrapping apply unchanged, and nothing in it can be reached from the public instance. See [docs/mcp.md](docs/mcp.md). **Security headers.** The nginx server block sets: `Content-Security-Policy`, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: no-referrer`, and `Strict-Transport-Security` with a one-year max-age. The CSP retains `script-src 'unsafe-inline'` for now because the frontend uses inline event handlers; removing it requires a frontend refactor to `addEventListener` bindings. `script-src` also allows `cdn.tailwindcss.com` for Tailwind and `cdnjs.cloudflare.com` for D3. These live in `nginx/snippets/security-headers.conf`, which is included at server level **and** inside every location that sets an `add_header` of its own. nginx does not merge `add_header` across levels: a location defining any header of its own inherits none of the server-level ones, which silently stripped CSP, HSTS and `nosniff` from `/static/` and the favicon routes until v3.32.2. `tests/unit/test_nginx_config.py` fails if a location reintroduces that. **Error isolation.** Exception strings from httpx and upstream APIs are logged server-side with `log.exception` and never echoed to the client. Client responses get generic messages only (`"Upstream service unavailable."`). **Prompt injection.** Both LLM endpoints analyse input that is hostile by definition: obfuscated malware samples, scam email bodies. Every attacker-controlled value (the script decoder's `code` and `hint`, the email analyser's `sender_email` and body) is wrapped in `<untrusted_data>` fences by `app/utils/prompt_safety.py` before it reaches Claude, with any fence tag stripped from the value first so a sample cannot close its own fence and continue as operator framing. Both system prompts carry a trust-boundary instruction stating that fenced content is data to analyse and never an instruction to follow, and that an attempt to influence the verdict is itself an indicator to report. Untrusted input is never interpolated raw into a prompt. **LLM output validation.** Parsed LLM JSON is validated before any field is used in application logic. `clamp_int`, `safe_str`, and `validate_findings_list` in `app/utils/llm_response.py` prevent type errors from malformed model output reaching `max()`, `for` loops, or string operations. Free-text fields are additionally sanitised by `sanitize_llm_text` (control and ANSI sequences stripped, echoed fence markers removed, length clamped) before they are returned to the browser, so injected content cannot smuggle escapes or forged framing into the UI. `severity` and `intent` remain allowlist-validated, so an injected verdict cannot survive parsing. **Input validation.** All SQL uses parameterized queries. The only subprocess is `whois` in list-arg form over a strictly validated domain. No `eval`, `exec`, `pickle`, `yaml.load`, or `shell=True` anywhere in the codebase. **Origin protection.** nginx only accepts inbound connections from Cloudflare IP ranges (IPv4 and IPv6), via `nginx/snippets/cloudflare-origin-allow.conf`. It is a separate file so a deployment that is not behind Cloudflare can drop one `include` rather than edit the vhost; see [docs/deploy-runbook.md](docs/deploy-runbook.md). The gunicorn listener binds to `127.0.0.1:8000` only, and pins `--forwarded-allow-ips 127.0.0.1` so uvicorn trusts `X-Forwarded-For` from the local nginx alone. With `*` it would take the caller-supplied left-most entry and the peer address that Rate limiting depends on would become attacker-chosen. The same range list is enforced a second time in the application (see Rate limiting), so the two must be kept in step: a unit test fails if `nginx/snippets/cloudflare-origin-allow.conf` and `app/utils/cloudflare_ips.py` drift apart, or if the vhost stops including the snippet. **Unauthenticated endpoints.** `GET /api/scanner/history` is public and returns an explicit column allowlist from `phishing_scans`. `telegram_bot_id` (the live bot token extracted from a kit's exfiltration call) is never in that list; it is returned only to the caller who submitted the scan that found it. Any column added to the table later is withheld until it is listed on purpose. --- ## Stack - **Backend**: Python 3.10+ (3.11 recommended), FastAPI, Uvicorn, Gunicorn - **Database**: SQLite with WAL mode (caching and rate limits; no user data); Redis for Prospect and Image tab response caches - **Frontend**: Tailwind CSS via CDN, vanilla JavaScript, D3.js for graph visualizations - **Web server**: nginx with Cloudflare Origin Certificate; Cloudflare in front for TLS termination, DDoS protection, and edge caching - **LLM**: Anthropic Claude Haiku 4.5 via the official Python SDK (`anthropic` package, install separately, see requirements.txt note) - **Security**: CrowdSec community blocklist + firewall bouncer, fail2ban, ufw, SSH on non-standard port with keys-only auth Memory footprint at idle: ~120 MB RAM. Disk: ~50 MB for code + ~20 MB SQLite cache. --- ## Self-hosting ### Prerequisites - Ubuntu 22.04 or 24.04 VPS (1 vCPU, 1 GB RAM minimum; 2 GB recommended) - A domain name pointed at the VPS - Python 3.10 or later (3.11+ recommended) - System packages beyond Python. `pip` does not install these, and two of them fail only at runtime: ```bash sudo apt-get install -y --no-install-recommends \ python3 python3-pip python3-venv git redis-server libzbar0 whois ``` This list now lives in one place, `FE_SYSTEM_PACKAGES` in `scripts/lib/common.sh`, and both `provision.sh` and `upgrade.sh` install from it, so a package added for a new feature reaches existing boxes too. `libzbar0` is required: `pyzbar` loads it via ctypes for the QR Code tab, and because that router is imported at module level, without it the whole app fails to start with `ImportError: Unable to find zbar shared library`. On Ubuntu 24.04 the package is really `libzbar0t64`, but it provides `libzbar0`, so this name works on 22.04 and 24.04. `whois` is used as a subprocess by the Domain Intel tab and fails silently when absent. See [docs/deploy-runbook.md](docs/deploy-runbook.md) ("Prerequisites: system packages") for the full audit. - Optional: Cloudflare account for TLS termination and DDoS protection. The shipped nginx config assumes it. Deploying behind another proxy, or none, needs the steps in [docs/deploy-runbook.md](docs/deploy-runbook.md) ("Deploying without Cloudflare"), which also covers AWS and Cloudflare Tunnel. - Optional: Anthropic API key for LLM-powered tabs; the rest of the tool runs without it - **If you self-host publicly, set the operator identity.** Out of the box the page carries the upstream operator's name, inbox, blog and privacy policy, and before v3.33.0 the contact form posted to their Formspree endpoint. Set `OPERATOR_NAME`, `OPERATOR_URL`, `OPERATOR_CONTACT_EMAIL`, `OPERATOR_PRIVACY_EMAIL` and `SITE_ORIGIN`, or set `CONTACT_ENABLED=false` to remove the Contact page entirely (nav entry gone, panel gone from the HTML, `/contact` returns 404). See "Operator identity" in [docs/deploy-runbook.md](docs/deploy-runbook.md). The AGPL notice and the link to this repository are not affected by any of these: that is the licence. ### Quick install (automated) ```bash sudo bash scripts/provision.sh ``` The provision script: 1. Creates `/opt/falconeye/{data,venv}` with correct ownership 2. Clones the repository to `/opt/falconeye/app_src` 3. Creates a virtualenv and installs dependencies from `requirements.txt` 4. Initializes the SQLite database 5. Installs and enables the systemd service 6. Installs the nginx vhost config After provisioning, copy and configure the environment file: ```bash sudo cp /opt/falconeye/app_src/.env.example /opt/falconeye/.env sudo vi /opt/falconeye/.env # fill in your API keys sudo chmod 600 /opt/falconeye/.env sudo systemctl restart falconeye ``` ### Manual install ```bash # 0. System packages (pip does not install these; libzbar0 is required or the # app will not import at all) sudo apt-get update sudo apt-get install -y --no-install-recommends \ python3 python3-pip python3-venv git redis-server libzbar0 whois # 1. Clone sudo mkdir -p /opt/falconeye sudo chown $USER:$USER /opt/falconeye cd /opt/falconeye git clone https://github.com/osintph/falconeye.git app_src cd app_src # 2. Create venv and install deps python3 -m venv /opt/falconeye/venv /opt/falconeye/venv/bin/pip install -r requirements.txt # Note: anthropic and extract-msg are not in requirements.txt (see below). # Install them separately if you want LLM tabs or .msg file upload: # /opt/falconeye/venv/bin/pip install "anthropic>=0.25" "extract-msg>=0.28" # 3. Configure environment cp .env.example /opt/falconeye/.env vi /opt/falconeye/.env # set ANTHROPIC_API_KEY, IMAGE_UPLOAD_SECRET, etc. chmod 600 /opt/falconeye/.env # 4. Initialize the database FALCONEYE_DB=/opt/falconeye/data/falconeye.db \ /opt/falconeye/venv/bin/python scripts/db_init.py # 5. Smoke test before enabling anything. A missing native library fails here, # not in a gunicorn restart loop where the traceback scrolls past. /opt/falconeye/venv/bin/python -c "import app.main" && echo "imports OK" # 6. Install systemd unit. # The unit declares LogsDirectory=falconeye, so systemd creates # /var/log/falconeye owned by the service user before gunicorn starts. Create it # by hand too if you intend to run gunicorn directly before installing the unit; # without it gunicorn exits with PermissionError on the error log. sudo install -d -o ubuntu -g ubuntu -m 0755 /var/log/falconeye sudo cp falconeye.service /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable --now falconeye # 7. Install nginx vhost. # Copy the snippets FIRST: the vhost includes them, so in the other order # `nginx -t` fails and the reload is refused. sudo mkdir -p /etc/nginx/snippets sudo cp nginx/snippets/*.conf /etc/nginx/snippets/ sudo cp nginx/falconeye.conf /etc/nginx/sites-available/falconeye sudo ln -sf /etc/nginx/sites-available/falconeye /etc/nginx/sites-enabled/falconeye sudo nginx -t && sudo systemctl reload nginx # Not behind Cloudflare? The vhost denies every request until you remove or # replace the origin allow snippet. See "Deploying without Cloudflare" in # docs/deploy-runbook.md before going live. ``` ### Upgrading One command, on the box: ```bash sudo /opt/falconeye/app_src/scripts/upgrade.sh v3.34.4 # a specific tag sudo /opt/falconeye/app_src/scripts/upgrade.sh # the newest tag sudo /opt/falconeye/app_src/scripts/upgrade.sh --dry-run # show, change nothing ``` It backs up `.env`, the database and the installed systemd unit to `/opt/falconeye/backups/<timestamp>/`, checks out the tag, reinstalls dependencies **only if `requirements.txt` changed**, copies the systemd unit and reloads systemd **if the unit changed**, updates the nginx snippets (and any `conf.d` file you already had), applies whatever migrations or cache flushes the release declares, restarts the service and verifies `/health` reports the new version. If it does not, it fails with the rollback command already filled in. **It never overwrites your vhost.** `server_name`, your certificate paths and any rate-limit zones live in `/etc/nginx/sites-available/falconeye`, so when the shipped vhost changes the script prints the diff command instead of clobbering yours. It never touches `.env`. Rolling back is the same command with the previous tag. The steps it performs are written out in "Appendix: what upgrade.sh does" in [docs/deploy-runbook.md](docs/deploy-runbook.md), for the day you have to do one of them by hand. ### State to persist Everything in this table is state `git reset --hard` cannot put back. The list lives in `fe_state_paths()` in `scripts/lib/common.sh`; this table is generated from it and a test fails if the two disagree. **Essential** means losing it loses something this repository plus a working afternoon cannot recreate. **Convenience** means it can be rebuilt, at the cost of history or of an hour. | Path | Class | Why | |---|---|---| | `/opt/falconeye/.env` | **essential** | API keys, operator identity and the abuse-admin hash. Keys can be reissued; the file cannot be recovered. | | `/opt/falconeye/data/falconeye.db` | **essential** | The application database: the abuse report audit trail, rate-limit counters and every cache. | | `/opt/falconeye/private/telegram.session` | **essential** | Telegram MTProto session. Recreating it needs an interactive login as the account holder. | | `/etc/ssl/falconeye` | **essential** | Cloudflare Origin CA certificate and private key. The key is shown once at issue and cannot be downloaded again. | | `/etc/nginx/sites-available/falconeye` | **essential** | Your nginx vhost: server_name, certificate paths, and the log_format the site is configured for. | | `/opt/falconeye/data/ransomware.db` | convenience | Collected ransomware victims. The collector rebuilds it forward only, so postings that have rotated off the source are gone. | | `/opt/falconeye/private/ransomware_watchlist.txt` | convenience | The PH-relevant ransomware search terms, kept outside git on purpose. | | `/etc/nginx/conf.d/goaccess-logformat.conf` | convenience | Defines log_format goaccess_cf. If your vhost names that format, nginx will not start without this file. | | `/opt/falconeye/data/.release_actions_applied` | convenience | Which release actions have run. Losing it replays idempotent flushes, which is noisy rather than harmful. | Two things that are **not** on the list, deliberately: - `/opt/falconeye/venv` and `/opt/falconeye/mcp-venv` rebuild from `requirements.txt`. Back up the pin file, not the tree. - `/opt/falconeye/data/image_temp/` and `/opt/falconeye/data/prospect/` are scratch space the app cleans up after itself. Keep anything else of your own (private ops scripts, a second watchlist) in `FE_BACKUP_EXTRA`, a space-separated path list the backup includes and neither script ever writes to. #### Back up and restore ```bash sudo /opt/falconeye/app_src/scripts/backup.sh # -> /opt/falconeye/backups/ sudo /opt/falconeye/app_src/scripts/backup.sh --out /mnt/x # somewhere else sudo /opt/falconeye/app_src/scripts/backup.sh --dry-run # list, write nothing sudo /opt/falconeye/app_src/scripts/restore.sh /path/to/falconeye-state-<stamp>.tar.gz ``` The backup writes `falconeye-state-<timestamp>.tar.gz` at mode 0600 with a `.sha256` sidecar, and prints the checksum. **The archive holds the API keys and the Cloudflare Origin CA private key**, so copy it the way you would copy `.env`. The restore verifies that checksum, refuses any archive carrying a path outside the list above (it runs as root, and `tar` writes what it is told), stops the service before touching the database, keeps whatever it overwrites in `/opt/falconeye/backups/pre-restore-<timestamp>/`, then starts the service and checks `/health`. It does not touch the checkout: restoring state onto a newer release is the normal case, and `scripts/upgrade.sh` is what moves the code. #### Ephemeral root disks On a platform where the root filesystem is rebuilt on every deploy (container hosts, image-based VMs, some managed platforms), none of the paths above survive unless you put them somewhere that does: - **Mount a persistent volume at `/opt/falconeye`.** That covers `.env`, both databases, the Telegram session and the watchlist in one mount, which is why they all live under one prefix. Point `FALCONEYE_DB` and `TELEGRAM_SESSION_PATH` at the mounted paths if your layout differs. - **`/etc/ssl/falconeye` and the nginx vhost belong in the platform's config store**, not on the disk: a secret store or config map for the certificate and key, the vhost rendered at boot from your own template. They are the two essential paths that are *not* under `/opt/falconeye`, and the two most easily forgotten, because nothing fails until the next rebuild. - If your vhost names `goaccess_cf`, ship `conf.d/goaccess-logformat.conf` with it. nginx refuses to start on an undefined `log_format`, so the pair has to land together. A restore onto a fresh box is therefore: provision, mount or restore the volume, put the certificate and vhost back, then `scripts/restore.sh` for the rest. ### Required and optional API keys | Variable | Required | Used by | Notes | |---|---|---|---| | `ANTHROPIC_API_KEY` | For LLM tabs | Email Header, Dork Gen, Script Decoder, Sock Puppet Generator | Free $5 credit on signup. Set a monthly spend limit in the Console. | | `IMAGE_UPLOAD_SECRET` | If `IMAGE_SEARCH_ENABLED=true` | Image Search upload flow | Generate: `openssl rand -hex 32` | | `SEARCHAPI_KEY` | For Prospect and Image tabs | Prospect dossier, Image reverse search | Free trial at searchapi.io | | `GREYNOISE_API_KEY` | No | IP Reputation | Community tier is free | | `ABUSECH_AUTH_KEY` | No | Sandbox History, IP Reputation | Free at auth.abuse.ch | | `HIBP_API_KEY` | For Breach Check | Breach Check (email + paste search) | Have I Been Pwned Core 1 subscription (haveibeenpwned.com/API/Key). Domain/recent/browse-all lookups use HIBP's free endpoints and work without this key. | | `RANSOMWARE_LIVE_API_KEY` | For Ransomware Watch | Ransomware Watch collector only, not the live request path | ransomware.live PRO subscription. Used solely by the scheduled collector (`app/collectors/ransomware_collect.py`), never by the tab's own requests. A missing/invalid key is logged loudly and falls back to ransomware.live's keyless v2 API rather than emptying the tab; RansomLook data (group activity, mirror health, watchlist) needs no key at all. | If a key is missing, the relevant feature degrades gracefully (returns an error card) without crashing the rest of the tool. > **Note on `anthropic` and `extract-msg`:** These packages are not listed in `requirements.txt` because they were installed separately on the reference deployment. If you want the three LLM-powered tabs or `.msg` file upload to work, install them manually: `pip install "anthropic>=0.25" "extract-msg>=0.28"`. A future release will add them to `requirements.txt` with pinned versions. --- ## API endpoints All write endpoints accept JSON. Rate limits are enforced per real client IP (`CF-Connecting-IP` behind Cloudflare, `request.client.host` otherwise). | Endpoint | Method | Rate limit | |---|---|---| | `/api/crypto/lookup` | POST | None (upstream provider quotas apply) | | `/api/scanner/scan` | POST | 10/minute per IP | | `/api/scanner/kit-report` | POST | 4/minute per IP + 10/IP/24h | | `/api/domain/intel` | POST | None | | `/api/telegram/inspect` | POST | None | | `/api/ip/reputation` | POST | None | | `/api/sandbox/lookup` | POST | None | | `/api/email-header/analyze` | POST | LLM analysis: 10/IP/24h | | `/api/email-header/upload` | POST | None | | `/api/dork-generator/generate` | POST | 10/IP/24h | | `/api/script-decoder/decode` | POST | 10/IP/24h | | `/api/url/expand` | POST | 10/IP/24h | | `/api/qr/decode` | POST | 10/IP/24h | | `/api/news/feed` | GET | None | | `/api/threat-pulse` | GET | None | | `/api/image/search` | POST | 5/minute per IP | | `/api/prospect/investigate` | POST | 10/IP/day | | `/health` | GET | None | OpenAPI docs are disabled in production (`FALCONEYE_PUBLIC_DOCS=false` by default). --- ## Development ```bash cd /path/to/falconeye python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt pip install "anthropic>=0.25" "extract-msg>=0.28" # if you need LLM tabs locally # Run the dev server with hot reload uvicorn app.main:app --reload --host 127.0.0.1 --port 8000 ``` Then open `http://127.0.0.1:8000/`. Hot reload picks up Python and static file changes. ### Running tests ```bash source .venv/bin/activate pytest tests/ -v ``` Expected baseline: **122 passed, 3 skipped** (the 3 skips are JPEG EXIF fixture tests that need actual image files; 2 collection errors in test_routes.py modules are pre-existing Python 3.9 type-annotation incompatibilities that do not affect Python 3.10+ deployments). ### Project structure ``` falconeye/ ├── app/ │ ├── main.py # FastAPI entry point, router registration │ ├── mcp_server.py # MCP tools over stdio, local mode, see docs/mcp.md │ ├── config.py # Environment variable loading │ ├── routers/ # One file per tab / feature │ │ ├── crypto.py │ │ ├── scanner.py # Phishing Scanner: uses safe_fetch for user URLs │ │ ├── domain_intel.py │ │ ├── telegram_inspector.py │ │ ├── ip_intel.py │ │ ├── sandbox.py │ │ ├── email_header.py │ │ ├── dork_generator.py │ │ ├── script_decoder.py │ │ ├── news.py │ │ └── threat_pulse.py │ ├── scanner/ # Phishing Scanner analysis engine, see docs/kit-analysis.md │ │ ├── kit_analyzer.py # Deobfuscates string-obfuscated bundles; decodes the │ │ │ # table and resolves call sites BEFORE searching │ │ ├── rabbithunt_sig.py # Per-kit signature records + transparent weighted scorer │ │ ├── kit_acquire.py # Read-only acquisition, every request via safe_fetch │ │ ├── kit_report.py # Consolidated case report builder │ │ ├── cloudflare_detect.py │ │ └── ph_bank_indicators.py │ ├── prospect/ # Prospect tab (SearchAPI.io dossier) │ ├── image_search/ # Image Search tab (Google Lens + Yandex) │ ├── ransomware/ # Ransomware Watch tab: reads local SQLite only │ ├── collectors/ │ │ └── ransomware_collect.py # Scheduled collector (ransomware.live + RansomLook); run by an out-of-repo systemd timer, see docs/ransomware-watch-runbook.md │ └── utils/ │ ├── client_ip.py # CF-Connecting-IP extraction │ ├── safe_fetch.py # SSRF-safe HTTP fetcher for user-supplied URLs │ ├── llm_response.py # LLM JSON validation helpers │ ├── ssrf.py # Legacy validate_url (used by crypto.py) │ ├── domain.py │ ├── indicator.py │ └── telegram.py ├── app/static/ # Single-page app shell (index.html, app.js, etc.) ├── scripts/ │ ├── provision.sh # Automated VPS provisioning │ └── db_init.py # SQLite schema initializer ├── tools/ # Standalone CLI analysis tools (not imported by the app) │ ├── kitanalyze.py # Reference deobfuscator the tab's engine was ported from │ ├── runkit.sh # Reference acquisition sequence (raw curl; the app uses safe_fetch) │ ├── decode_kit.py # Minimal string-table decoder │ └── kitdecrypt.py # Decrypts captured kit blobs; CLI only, deliberately │ # never wired into an endpoint. Needs `cryptography`. ├── nginx/ │ ├── falconeye.conf # nginx vhost │ ├── snippets/ │ │ ├── cloudflare-origin-allow.conf # Origin lock; drop it if not behind Cloudflare │ │ └── security-headers.conf # CSP/HSTS/nosniff, included per location │ └── conf.d/ │ └── goaccess-logformat.conf # Optional GoAccess log format ├── falconeye.service # systemd unit ├── requirements.txt ├── .env.example ├── LICENSE ├── CONTRIBUTING.md └── README.md ``` ### Adding a new tab 1. Create `app/routers/your_tab.py` with a FastAPI router 2. Register it in `app/main.py` 3. Add the tab button and content section in `app/static/index.html` 4. Add the JS handler in `app/static/app.js` 5. Add `'your_tab'` to the `VALID_TABS` array for hash routing 6. Update this README's tab table and the endpoint list above If the tab fetches a user-supplied URL, route it through `safe_fetch` in `app/utils/safe_fetch.py`. Do not use `httpx.AsyncClient` directly on user input. If the tab uses an LLM, follow the pattern in `script_decoder.py`: hardcoded model constant, per-feature rate-limit table keyed on `CF-Connecting-IP`, environment kill switch, prompt caching, and output validation via `app/utils/llm_response.py`. --- ## Privacy posture FalconEye does not maintain user accounts. - Input is processed to produce analysis results and is not persisted beyond the moments needed - Results are cached for 24 hours by SHA256 hash of the input (keyed by content, not by user) - Uploaded .eml and .msg files are parsed in memory and discarded immediately - Source IP is stored only for rate-limit enforcement with automatic cleanup after 48 hours - No cookies set by FalconEye (Cloudflare sets its own security cookies) - No third-party advertising or analytics When you use the LLM-powered tabs, your input is sent to Anthropic's API. See [Anthropic's privacy policy](https://www.anthropic.com/legal/privacy). At time of writing, Anthropic does not train on API input by default. --- ## Roadmap - IoC enrichment pipeline (one URL in, full report from all tabs out) - Hash-based artifact deduplication across investigation history - Maltego transform export for the crypto graph - Public API tokens for trusted integrations --- ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). Bug reports and feature requests go to the [Contact tab](https://falconeye.osintph.info/#contact) or as a GitHub issue. --- ## License AGPL-3.0. Strong copyleft: if you run a modified version as a network service, you must offer the modified source to users of that service. See [LICENSE](LICENSE). --- ## Acknowledgments - abuse.ch (URLhaus, MalwareBazaar) - GreyNoise (IP intelligence) - Shodan InternetDB (free, no-key endpoint) - crt.sh, Cert Spotter (certificate transparency) - RDAP.org (RDAP queries) - RIPEstat (ASN data) - Blockstream, BlockCypher, TronGrid (blockchain APIs) - Anthropic Claude Haiku 4.5 (LLM analysis) - D3.js (graph visualization) - Tailwind CSS (styling) Built and maintained by [OSINT-PH](https://blog.osintph.info). - City list from [GeoNames](https://www.geonames.org/) (`cities15000`), used under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Bundled offline in the [routemap-engine](https://github.com/osintph/routemap-engine) package for the Route Map origin picker. - Router hostname geolocation from [CAIDA Hoiho](https://api.hoiho.caida.org/), the published ruleset behind the Route Map tab's hostname-first placement. - Carrier site codes derived from [Arelion's public looking glass](https://lg.twelve99.net/). The table ships in the routemap package; see its `routemap/engine/data/README.md` to regenerate it or add a carrier. - Traceroute measurements from [RIPE Atlas](https://atlas.ripe.net/). Measurements created by this tool are public.