osint-autopilot
End-to-end external OSINT engagement autopilot. Run the FULL osint-methodology pipeline to completion in ONE go for an authorized domain — engagement folder, Stages 1-5 (seed, expansion, enrichment, exposure, convergence), multi-agent per-host content+JS fan-out, headline verification, auto-generated findings, and a consolidated multi-tab .xlsx deliverable. Stops ONLY for Stage 6 (active exploitation) arming and out-of-scope sibling assets. Use whenever asked to "run OSINT / recon / attack-surface" on a target — do NOT deliver a thin passive first pass.
git clone --depth 1 https://github.com/elementalsouls/Claude-OSINT /tmp/osint-autopilot && cp -r /tmp/osint-autopilot/skills/osint-autopilot ~/.claude/skills/osint-autopilotSKILL.md
# OSINT Autopilot
Purpose: eliminate the "thin first pass, then user pushes for more" failure. When the user says *run OSINT/recon on `<domain>`* (authorized), execute the whole pipeline to completion and hand back the consolidated workbook — no stopping to ask except the two hard gates below.
Read the companion `osint-methodology` skill for the framework; this skill is the executable runbook. Feedback memory `osint-full-pipeline-default` governs the default.
## Hard gates (the ONLY reasons to stop and ask)
1. **Authorization** — if not already asserted, ask the one scope question, then proceed. (Here it's typically already signed.)
2. **Stage 6 (active exploitation)** — BOLA/IDOR, credential replay, injection confirmation. Never run under autopilot; present the ranked target queue and wait for arming.
3. **Out-of-scope siblings** — a newly-discovered sibling domain/brand (e.g. `<company>.io` when scope was `.com`). Flag; don't scan until confirmed.
Everything in Stages 1–5 runs without pausing for questions.
## Run sequence
Let `S = ~/.claude/skills/osint-autopilot/scripts`.
### 1. Deterministic recon (Stages 1–3 + harvest + breach + ports)
```
bash $S/recon_pipeline.sh <domain>
```
Creates `~/Research/engagements/<domain>/` and populates evidence (DNS/WHOIS/RDAP/email-sec, subdomain union via subfinder+certspotter+crt.sh, dig-resolve, HTTP probe, identity fabric, /version disclosure, S3 takeover check, gau+wayback+JS harvest, HudsonRock breach, bounded public-IP port scan), hashes everything, and splits responsive hosts into `buckets/bucket-NN`.
Host notes baked in: uses `dig`/`curl` (ProjectDiscovery `dnsx`/`httpx` segfault on this M1 — cgo m1cpu); nmap is time-capped at 10 min so it can't stall.
### 2. Multi-agent per-host fan-out (Stage 4 exposure)
Read `buckets/COUNT` and the `buckets/bucket-*` paths, then launch the workflow (`ffuf` must be on PATH — `go install github.com/ffuf/ffuf/v2@latest` if missing):
```
Workflow({ scriptPath: "<S>/host_enum.workflow.js",
args: { domain:"<domain>", engDir:"~/Research/engagements/<domain>" (absolute),
ffuf:"<gopath>/bin/ffuf", wordlist:"<S>/wordlist.txt",
buckets:[ absolute bucket-00 … bucket-NN ] } })
```
**Pass `args` as a real JSON object, NOT a stringified one** — the harness hands `args` to the script verbatim; a string makes `args.domain`/`args.buckets` undefined and only the synthesize agent runs. The script fails loudly if args are malformed. `engDir` must be absolute. `buckets` = absolute paths of every `buckets/bucket-*` file (read `buckets/COUNT`).
It fans out ffuf content-discovery + JS/endpoint analysis per bucket (agentType general-purpose), then synthesizes a ranked BOLA/IDOR target queue. Save `synthesis` to `evidence/stage6-content/SYNTHESIS.md`.
Optional screenshots: `gowitness scan file -f <urls> --screenshot-path evidence/stage6-screens --write-db` (Chrome required).
### 3. Verify headlines yourself (do NOT trust agent output blindly)
Re-curl every CONFIRMED-worthy claim before reporting it: exposed secrets (fetch the JS/vars file), `/version` disclosures, Cognito/Auth0 config, any "unauth admin 200". Downgrade anything you can't reproduce.
### 4. Auto-findings + workbook
`build_xlsx.py` requires the `openpyxl` pip package (`pip install -r $S/requirements.txt` if missing).
```
python3 $S/findings_gen.py <domain> # evidence -> findings/findings.csv (rules engine)
python3 $S/build_xlsx.py <domain> # findings.csv + evidence -> <domain>-osint-consolidated.xlsx
```
`findings_gen.py` encodes the rubric (non-prod exposure, CDN/WAF-bypass origins, internal-IP leak, DMARC/SPF, /version disclosure, JS secrets, S3 takeover, identity fabric, breach severity, WordPress, port exposure). Hand-add any nuanced findings the rules miss by editing `findings.csv`, then re-run `build_xlsx.py`.
### 5. Report
Give the consolidated `.xlsx` path + a short severity summary, then present the two open gates (Stage 6 queue + any out-of-scope siblings). Offer passive CVE mapping against disclosed versions as a next step.
## Deliverable
`~/Research/engagements/<domain>/`:
- `<domain>-osint-consolidated.xlsx` — 9–10 tabs (Summary, Findings, Subdomains, Live Hosts, API Endpoints, Secrets, Ports, Internal-IP-Leak, Screenshots)
- `findings/findings.csv` · `evidence/**` (+ `.sha256`) · `evidence/stage6-content/SYNTHESIS.md` · `run-log.jsonl`
## Time budget
Small org (<100 hosts): ~15–25 min wall-clock. Medium (100–1K): ~30–60 min, dominated by the fan-out workflow and gau harvest.
## Scale note
The fan-out is ~1 agent per 18 hosts. Keep total agents ≤15 (default guideline); for >270 responsive hosts, raise bucket size in `recon_pipeline.sh` (`split -l`) rather than agent count.Operational arsenal for external red-team and bug-bounty reconnaissance. Concrete wordlists (28 Swagger paths, 13 GraphQL paths, 35 high-risk ports, 6 missing-header findings, 15 always-on HTTP checks, 5 SAML paths, cloud bucket permutations, JS guess-paths, vendor product fingerprints for Citrix/F5/Pulse/Fortinet/Cisco/PaloAlto/VMware/Exchange, cloud-native service fingerprints, container/K8s exposure paths, CI/CD platform paths, documentation/wiki leak paths, WHOIS/RDAP, DNS record catalog, Wayback CDX recipes), 80-pattern secret-regex catalog (incl. modern AI API keys: Anthropic/OpenAI/HuggingFace/Cloudflare/DigitalOcean/npm/PyPI/Docker Hub/Atlassian/DataDog/Sentry/ngrok; plus a provider-expansion tier: Postman PMAK/GitLab/Square/Shopify/Mailchimp/PagerDuty/Asana/Databricks/Grafana/Terraform Cloud/Fastly/Algolia/Segment/Airtable/GCP+Google OAuth/Azure AD/Facebook OAuth/RubyGems/JFrog/Okta/Slack app-level/Dropbox/Doppler/HashiCorp Vault/Firebase Cloud Messaging), 80+ dork corpus across 9 categories, GitHub code-search dorks, copy-paste curl/httpie probes for every check, post-discovery enumeration workflows (AWS/GitHub/Slack/JWT/PMAK/Anthropic/OpenAI), endpoint interest scoring rubric (0–100), mobile app ownership confidence + APK static-analysis pipeline (acquisition, apktool/aapt2/jadx/androguard decompile, manifest exported-component/deep-link/misconfig extraction, Firebase config, network-security-config, embedded-secret scan), identity-fabric endpoints (Entra/Okta/ADFS/Google/SAML/M365 Teams+SharePoint+OneDrive+OAuth + user-enum), GraphQL field-suggestion enumeration when introspection disabled, 9 read-only secret validators (Postman/AWS/GitHub/Slack/Anthropic/OpenAI/npm/Atlassian/DataDog), Postman workspace search (verified endpoint), Stack Exchange sweep, public SaaS dorks, email security analysis (SPF/DMARC/DKIM/BIMI/MTA-STS/DNSSEC), origin-discovery / CDN bypass techniques, TLS deep audit (sslyze/testssl.sh/JA3/JA4), reverse-DNS sweep + IPv6 enum, vulnerability prioritization data sources (NVD/EPSS/CISA KEV/ExploitDB/Metasploit), 27 attack-path hint templates, 80+ severity-matrix examples, LinkedIn employee enumeration, job posting tech-stack analysis, Slack/Discord workspace discovery, package registry leak hunting (npm/PyPI/Docker Hub/Quay/GHCR), sat imagery for physical recon, tooling quick-install one-liners, sector-specific recon notes (healthcare/finance/ICS-SCADA/IoT/government), runnable stdlib-only secret_scan.py helper, plus the existing tool references for username/email/phone/people/social/breach/infrastructure/crypto/media/geospatial/AI/archiving/automation. Use when you need concrete probe paths, regexes, payloads, scoring rules, curl one-liners, and tool URLs for an authorized external recon engagement.
Comprehensive OSINT methodology for external red-team operations and authorized attack-surface assessments. Covers the 6-stage recon pipeline (seed → asset expansion → enrichment → exposure analysis → convergence → operator-armed active validation) with connector-resilience and stage-vs-gating discipline, asset-graph discipline, severity rubric, confidence upgrade workflows, time budgeting, identity-fabric mapping, breach×identity correlation with per-person identity dossiers, detectability tagging, detection-aware probing, WAF/CDN bypass, vulnerability prioritization, phishing infrastructure planning, bug bounty submission, and client deliverable templates. Use when planning or executing reconnaissance against authorized targets, mapping an organization's external attack surface, investigating a person/entity, or producing client deliverables.
Build, validate, and run the claude-osint skills repo — check SKILL.md frontmatter, run the secret_scan.py and h1_reference.py helpers, run sync-skill-content.sh, run the smoke test. Use when asked to run, build, test, validate, or smoke-test claude-osint or its OSINT skills/scripts.
Organization-grade cloud and supply-chain attack-surface discovery: S3/GCS/Azure Blob bucket discovery via observed-name mining (CNAME/cert-SAN/Wayback) and bounded two-class permutation (6 prefixes x 15 suffixes on trusted tokens, bounded target-bound expansion on subdomain stems), existence (HEAD/GET) vs public-listing confirmation, object-key triage into 9 value tiers (database dumps, credentials, IaC state, kubeconfig, VCS dirs, config, archives, PII, logs), dangling-CNAME bucket-takeover detection, and the ownership-gated severity model that stops an unattributable public bucket from becoming a false CRITICAL; the fully offline AWS-account-ID recovery from a leaked AKIA/ASIA/AROA access key (base32 decode, runnable stdlib Python, canonical test vector, AWS-documentation-example-ID screening); dependency-confusion confirmation for npm/PyPI (internal-signal classifier -- private-registry binding vs org-namespace match -- paired with a read-only public-registry 404 check and the npm scope-claimability nuance the public search API misses); and passive cloud-native/container/Kubernetes/CI control-plane fingerprinting (Lambda URLs, API Gateway, Cloud Run, App Service, kubelet/etcd/K8s API/dashboard, Jenkins/GitLab/Argo CD) as an org-attribution and exposure surface. Passive/discovery only -- no exploitation, no credential submission, no active control-plane confirmation (a stage-6 validate_cloud active tier is described but out of scope). Use when enumerating a target's cloud storage footprint, recovering an AWS account ID from a leaked key, confirming a supply-chain dependency-confusion vector, or fingerprinting cloud-native/K8s/CI infrastructure for an authorized external recon engagement.
Turns one-shot external recon into a continuous monitoring program. Covers the scheduled re-scan-and-diff loop (baseline snapshot -> interval sleep -> re-scan -> asset/finding delta -> threshold-gated webhook alert), the scan-to-scan diff engine (new/removed/changed assets by a tracked-attribute table, new/resolved findings by a stable cross-scan fingerprint), adversary CTI / chatter monitoring across six public feeds (ransomwatch, ransomware.live, HackerNews Algolia search, Reddit security-subreddit RSS, GitHub Gist code-search, public Telegram channel scraping) with a source-kind-aware severity engine (leak-site/forum/telegram/paste tiers, CRITICAL through INFO), literal/glob/regex watchlist pattern matching, full-corpus capture with retroactive rescan on new watchlist entries, infrastructure-tracking-over-time discipline (certificate-transparency, passive-DNS, port/service, and typosquat re-enumeration cadence, and what a genuine 'perimeter drift' event looks like in the diff output), a five-state finding-lifecycle state machine (open/triaged/risk_accepted/resolved/false_positive) with per-severity SLA and fingerprint-based cross-scan dedup and auto-resolve/reopen rules, the alert-fatigue trap where a lifecycle-unaware rule re-fires on an already-accepted finding, a durable retry/backoff alert-outbox pattern ('queued is not delivered'), and copy-paste bash-cron plus PowerShell-Scheduled-Task recipes for a re-scan+diff loop with the Slack-compatible webhook payload shape. Passive OSINT and analysis only -- no new active-intrusion technique. Use when setting up ongoing monitoring for a retainer or MSSP engagement, tuning alert thresholds to avoid fatigue, triaging a finding's lifecycle status, investigating adversary chatter about a brand, building a 'what changed on the perimeter since last week' report, or deciding whether a persisting finding should re-alert.
Rigorous, defensible email-spoofability verdict and SPF supply-chain risk analysis computed from published DNS alone. Deepens the record-level SPF/DMARC/DKIM/BIMI/MTA-STS/DNSSEC fetch recipes in the offensive-osint arsenal (§16.14) with the reasoning that section doesn't do: a priority-ordered composite verdict for whether an attacker can actually land header-From-spoofed mail in an inbox, and by which vector (exact-domain vs subdomain) — grounded in the single most-misunderstood distinction in email security: the envelope MAIL FROM that SPF authenticates vs the visible header From: that only DMARC governs. Explains precisely why SPF -all/~all alone is NOT spoof-proof without DMARC enforcement, and why SPF +all bypasses DMARC even under p=reject pct=100. Covers RFC 7208 §4.6.4's 10-DNS-lookup / 2-void-lookup PermError fail-open condition with a runnable stdlib-only lookup-counter script, plus the SPF-include-takeover supply-chain vector (an attacker re-registering a dead include inherits SPF-pass authority over the victim domain) with strict transient-vs-NXDOMAIN discrimination discipline so a temporary SERVFAIL is never mistaken for a takeover lead. Fully passive: DNS TXT reads only, no mail sent, no RCPT TO probe, no API keys. Use when auditing a domain's real spoofing resistance (not just its published records), explaining to a client why 'we have SPF -all' does not mean they're covered, investigating an SPF PermError or an unusually long include chain, evaluating a dead SPF include as a takeover lead, or writing a defensible spoofability finding for a deliverable.
FAIR-aligned exposure quantification: turns a pile of recon findings into a defensible 0-100 + A-F org risk score (Likelihood x Impact, three ownership-aware factors: exposure/threat/impact), an ownership + proof demotion cap so unproven or weakly-owned findings can't inflate the number, a $-denominated FAIR loss-magnitude estimate (IBM/Ponemon per-record cost bands, cross-source record dedup, threat-factor annualization), attack-path amplification (curated red-team chain catalog + generic graph-walk engine, with a kill-chain vs shared-fate honesty gate), and a board-ready one-pager deliverable (hero $ + letter grade + top-3 findings + top attack path + the ask). Extends osint-methodology's severity rubric and client deliverable templates with quantification. Passive analysis only -- operates on findings already collected, no target traffic, no API keys. Use when asked to score risk, quantify exposure, estimate breach cost, build a board report, translate technical findings to dollars, or explain why a grade or dollar figure came out the way it did.
Organization-grade identity-fabric mapping: tenant/federation fingerprinting and the pre-auth user-ENUMERATION oracle methodology — enumeration and fingerprint only, never credential submission. Covers domain-to-tenant resolution (Microsoft getuserrealm.srf Managed/Federated namespace check, Entra OIDC metadata tenant-GUID extraction, Autodiscover v2), keyless Microsoft tenant-federation mapping (GetFederationInformation SOAP -> sibling-domain discovery, discover-only ROE, FEDERATED_WITH provenance edge held out of attack-path pivoting), Okta org-slug derivation + OIDC fingerprint + governed custom-domain enumeration, ADFS passive/active fingerprint + version inference, Google Workspace MX-correlated detection, generic OIDC (Auth0/Keycloak/Ping Identity/OneLogin/Duo) discovery, SAML metadata (5 paths), Azure AD Seamless-SSO Negotiate-challenge detection, Microsoft Defender for Identity (MDI) sensor-API presence check, the user-enumeration oracle methodology for Microsoft GetCredentialType (IfExistsResult semantics: exists / doesn't-exist / exists-in-federated-tenant / throttled) and Okta /api/v1/authn (errorCode differential), Medium-detectability discipline with a hard 20-candidate-per-tenant cap and admin/role interest-based ranking, and name x confirmed-email-pattern login-candidate synthesis that FAILS CLOSED with zero output when no org pattern is confirmed. Grounded directly in a production ASM implementation's sso_idp.py, tenant_recon.py, and core/email_patterns.py modules. Deepens — does not duplicate — offensive-osint skill's Identity Fabric endpoint reference with the tenant-federation MAP, the oracle WORKFLOW, and the candidate-SYNTHESIS methodology that reference lacks. Use when fingerprinting an organization's identity provider, mapping its tenant/federation boundary, running an authorized pre-auth user-enumeration pass, or synthesizing login candidates from harvested names to feed that oracle — never for password spray, credential submission, or auth bypass.