run-claude-osint
Build, validate, and run the claude-osint skills repo — check SKILL.md frontmatter, run the secret_scan.py and h1_reference.py helpers, run sync-skill-content.sh, run the smoke test. Use when asked to run, build, test, validate, or smoke-test claude-osint or its OSINT skills/scripts.
git clone --depth 1 https://github.com/elementalsouls/Claude-OSINT /tmp/run-claude-osint && cp -r /tmp/run-claude-osint/.claude/skills/run-claude-osint ~/.claude/skills/run-claude-osintSKILL.md
# Run: claude-osint
`claude-osint` is **not an app** — it's a Claude *skills package*. Its product
is two `SKILL.md` files (`skills/offensive-osint/`, `skills/osint-methodology/`)
plus two runnable Python helpers under `skills/offensive-osint/scripts/`. There
is no GUI, server, or TUI. "Running it" means: the SKILL.md frontmatter parses
and is complete (that's what Claude loads), and the helper scripts work.
The driver is **`.claude/skills/run-claude-osint/smoke.sh`** — it does all of
that in one shot. **All paths below are relative to the repo root.**
## Prerequisites
Python 3 with PyYAML (already present on this container). If `import yaml` fails:
```bash
pip install pyyaml # or: apt-get install -y python3-yaml
```
The helpers are stdlib-only otherwise. `h1_reference.py` needs outbound HTTPS to
`hackerone.com`. No build step, no `npm install` — nothing to compile.
## Run (agent path) — the driver
From the repo root:
```bash
.claude/skills/run-claude-osint/smoke.sh # full smoke (exits 0 on pass)
.claude/skills/run-claude-osint/smoke.sh --no-net # skip the HackerOne live check
```
It runs six checks and prints a `==> PASS` / `==> FAIL` line (exit 0 / 1):
1. `py_compile` both helpers.
2. `secret_scan.py` detects AWS key + JWT from stdin (the CI canaries).
3. `secret_scan.py` recursively scans `skills/` and emits JSONL (~11 findings — example tokens in the SKILL.md docs).
4. Every `skills/*/SKILL.md` has valid YAML frontmatter with `name`/`description`/`version`/`triggers` and ≥5 triggers.
5. `sync-skill-content.sh --check` exits 0.
6. `h1_reference.py` fetches one live disclosed report (non-fatal if offline).
## Run individual pieces
Verified this session:
```bash
# Secret scanner — stdin
printf 'AKIAIOSFODNN7EXAMPLE\n' | python3 skills/offensive-osint/scripts/secret_scan.py
# -> {"pattern": "AWS_ACCESS_KEY", "severity": "critical", ...}
# Secret scanner — scan a directory tree (JSONL, one finding per line)
python3 skills/offensive-osint/scripts/secret_scan.py skills/
# HackerOne reference helper (needs network)
python3 skills/offensive-osint/scripts/h1_reference.py --top-voted --limit 3
python3 skills/offensive-osint/scripts/h1_reference.py --top-bounty --limit 3
python3 skills/offensive-osint/scripts/h1_reference.py --top-voted --query "XSS" --pages 3
# Install the skills for Claude to load
cp -r skills/* ~/.claude/skills/
```
## Test
CI (`.github/workflows/lint.yml`) runs four jobs: markdown lint
(`markdownlint-cli2`), the frontmatter check, the `secret_scan.py` smoke, and
`shellcheck ./scripts`. `smoke.sh` covers the latter two plus the helpers
directly. `markdownlint-cli2` and `shellcheck` are **not installed** on this
container; install with `npm i -g markdownlint-cli2` / `apt-get install -y
shellcheck` if you need to reproduce those jobs locally.
## Gotchas
- **`sync-skill-content.sh` is a no-op in a fresh clone.** It copies from
`docs/full-skills/`, which **is not in the repo** — the script prints
"⚠ Source missing … Skipping" and exits 0. The `skills/*/SKILL.md` files are
*already* the full inline content (offensive-osint ≈ 4,200 lines), so you do
**not** need to run sync after cloning. Don't be alarmed by the warnings.
- **`secret_scan.py` skips its own findings as noise filters.** It excludes
`.git`, `node_modules`, `__pycache__`, `.venv`, `dist`, `build`, `.cache`
and any file >10 MB. Scanning `skills/` returns the example/doc tokens (~11),
not real secrets.
- **`h1_reference.py` hits an undocumented public GraphQL endpoint.** It works
unauthenticated but is sensitive to HackerOne-side changes and rate limits;
treat a failure there as environmental, not a repo bug (hence non-fatal in
the driver). It caps at 50 results/page — use `--pages N` for breadth.
- **No screenshot / no window** — this repo has no visual surface. If a request
asks to "see it run," that means the `smoke.sh` PASS output, not a GUI.
## Troubleshooting
| Symptom | Fix |
|---|---|
| `ModuleNotFoundError: No module named 'yaml'` in check [4] | `pip install pyyaml` (only the frontmatter check needs it). |
| Check [6] shows `~ no network / HackerOne unreachable` | Expected when egress is blocked; run with `--no-net` to silence. Driver still passes. |
| `sync --check` shows "Source missing" | Expected — see Gotchas. Not a failure; exit code is 0. |Operational arsenal for external red-team and bug-bounty reconnaissance. Concrete wordlists (28 Swagger paths, 13 GraphQL paths, 35 high-risk ports, 6 missing-header findings, 15 always-on HTTP checks, 5 SAML paths, cloud bucket permutations, JS guess-paths, vendor product fingerprints for Citrix/F5/Pulse/Fortinet/Cisco/PaloAlto/VMware/Exchange, cloud-native service fingerprints, container/K8s exposure paths, CI/CD platform paths, documentation/wiki leak paths, WHOIS/RDAP, DNS record catalog, Wayback CDX recipes), 80-pattern secret-regex catalog (incl. modern AI API keys: Anthropic/OpenAI/HuggingFace/Cloudflare/DigitalOcean/npm/PyPI/Docker Hub/Atlassian/DataDog/Sentry/ngrok; plus a provider-expansion tier: Postman PMAK/GitLab/Square/Shopify/Mailchimp/PagerDuty/Asana/Databricks/Grafana/Terraform Cloud/Fastly/Algolia/Segment/Airtable/GCP+Google OAuth/Azure AD/Facebook OAuth/RubyGems/JFrog/Okta/Slack app-level/Dropbox/Doppler/HashiCorp Vault/Firebase Cloud Messaging), 80+ dork corpus across 9 categories, GitHub code-search dorks, copy-paste curl/httpie probes for every check, post-discovery enumeration workflows (AWS/GitHub/Slack/JWT/PMAK/Anthropic/OpenAI), endpoint interest scoring rubric (0–100), mobile app ownership confidence + APK static-analysis pipeline (acquisition, apktool/aapt2/jadx/androguard decompile, manifest exported-component/deep-link/misconfig extraction, Firebase config, network-security-config, embedded-secret scan), identity-fabric endpoints (Entra/Okta/ADFS/Google/SAML/M365 Teams+SharePoint+OneDrive+OAuth + user-enum), GraphQL field-suggestion enumeration when introspection disabled, 9 read-only secret validators (Postman/AWS/GitHub/Slack/Anthropic/OpenAI/npm/Atlassian/DataDog), Postman workspace search (verified endpoint), Stack Exchange sweep, public SaaS dorks, email security analysis (SPF/DMARC/DKIM/BIMI/MTA-STS/DNSSEC), origin-discovery / CDN bypass techniques, TLS deep audit (sslyze/testssl.sh/JA3/JA4), reverse-DNS sweep + IPv6 enum, vulnerability prioritization data sources (NVD/EPSS/CISA KEV/ExploitDB/Metasploit), 27 attack-path hint templates, 80+ severity-matrix examples, LinkedIn employee enumeration, job posting tech-stack analysis, Slack/Discord workspace discovery, package registry leak hunting (npm/PyPI/Docker Hub/Quay/GHCR), sat imagery for physical recon, tooling quick-install one-liners, sector-specific recon notes (healthcare/finance/ICS-SCADA/IoT/government), runnable stdlib-only secret_scan.py helper, plus the existing tool references for username/email/phone/people/social/breach/infrastructure/crypto/media/geospatial/AI/archiving/automation. Use when you need concrete probe paths, regexes, payloads, scoring rules, curl one-liners, and tool URLs for an authorized external recon engagement.
Comprehensive OSINT methodology for external red-team operations and authorized attack-surface assessments. Covers the 6-stage recon pipeline (seed → asset expansion → enrichment → exposure analysis → convergence → operator-armed active validation) with connector-resilience and stage-vs-gating discipline, asset-graph discipline, severity rubric, confidence upgrade workflows, time budgeting, identity-fabric mapping, breach×identity correlation with per-person identity dossiers, detectability tagging, detection-aware probing, WAF/CDN bypass, vulnerability prioritization, phishing infrastructure planning, bug bounty submission, and client deliverable templates. Use when planning or executing reconnaissance against authorized targets, mapping an organization's external attack surface, investigating a person/entity, or producing client deliverables.
Organization-grade cloud and supply-chain attack-surface discovery: S3/GCS/Azure Blob bucket discovery via observed-name mining (CNAME/cert-SAN/Wayback) and bounded two-class permutation (6 prefixes x 15 suffixes on trusted tokens, bounded target-bound expansion on subdomain stems), existence (HEAD/GET) vs public-listing confirmation, object-key triage into 9 value tiers (database dumps, credentials, IaC state, kubeconfig, VCS dirs, config, archives, PII, logs), dangling-CNAME bucket-takeover detection, and the ownership-gated severity model that stops an unattributable public bucket from becoming a false CRITICAL; the fully offline AWS-account-ID recovery from a leaked AKIA/ASIA/AROA access key (base32 decode, runnable stdlib Python, canonical test vector, AWS-documentation-example-ID screening); dependency-confusion confirmation for npm/PyPI (internal-signal classifier -- private-registry binding vs org-namespace match -- paired with a read-only public-registry 404 check and the npm scope-claimability nuance the public search API misses); and passive cloud-native/container/Kubernetes/CI control-plane fingerprinting (Lambda URLs, API Gateway, Cloud Run, App Service, kubelet/etcd/K8s API/dashboard, Jenkins/GitLab/Argo CD) as an org-attribution and exposure surface. Passive/discovery only -- no exploitation, no credential submission, no active control-plane confirmation (a stage-6 validate_cloud active tier is described but out of scope). Use when enumerating a target's cloud storage footprint, recovering an AWS account ID from a leaked key, confirming a supply-chain dependency-confusion vector, or fingerprinting cloud-native/K8s/CI infrastructure for an authorized external recon engagement.
Turns one-shot external recon into a continuous monitoring program. Covers the scheduled re-scan-and-diff loop (baseline snapshot -> interval sleep -> re-scan -> asset/finding delta -> threshold-gated webhook alert), the scan-to-scan diff engine (new/removed/changed assets by a tracked-attribute table, new/resolved findings by a stable cross-scan fingerprint), adversary CTI / chatter monitoring across six public feeds (ransomwatch, ransomware.live, HackerNews Algolia search, Reddit security-subreddit RSS, GitHub Gist code-search, public Telegram channel scraping) with a source-kind-aware severity engine (leak-site/forum/telegram/paste tiers, CRITICAL through INFO), literal/glob/regex watchlist pattern matching, full-corpus capture with retroactive rescan on new watchlist entries, infrastructure-tracking-over-time discipline (certificate-transparency, passive-DNS, port/service, and typosquat re-enumeration cadence, and what a genuine 'perimeter drift' event looks like in the diff output), a five-state finding-lifecycle state machine (open/triaged/risk_accepted/resolved/false_positive) with per-severity SLA and fingerprint-based cross-scan dedup and auto-resolve/reopen rules, the alert-fatigue trap where a lifecycle-unaware rule re-fires on an already-accepted finding, a durable retry/backoff alert-outbox pattern ('queued is not delivered'), and copy-paste bash-cron plus PowerShell-Scheduled-Task recipes for a re-scan+diff loop with the Slack-compatible webhook payload shape. Passive OSINT and analysis only -- no new active-intrusion technique. Use when setting up ongoing monitoring for a retainer or MSSP engagement, tuning alert thresholds to avoid fatigue, triaging a finding's lifecycle status, investigating adversary chatter about a brand, building a 'what changed on the perimeter since last week' report, or deciding whether a persisting finding should re-alert.
Rigorous, defensible email-spoofability verdict and SPF supply-chain risk analysis computed from published DNS alone. Deepens the record-level SPF/DMARC/DKIM/BIMI/MTA-STS/DNSSEC fetch recipes in the offensive-osint arsenal (§16.14) with the reasoning that section doesn't do: a priority-ordered composite verdict for whether an attacker can actually land header-From-spoofed mail in an inbox, and by which vector (exact-domain vs subdomain) — grounded in the single most-misunderstood distinction in email security: the envelope MAIL FROM that SPF authenticates vs the visible header From: that only DMARC governs. Explains precisely why SPF -all/~all alone is NOT spoof-proof without DMARC enforcement, and why SPF +all bypasses DMARC even under p=reject pct=100. Covers RFC 7208 §4.6.4's 10-DNS-lookup / 2-void-lookup PermError fail-open condition with a runnable stdlib-only lookup-counter script, plus the SPF-include-takeover supply-chain vector (an attacker re-registering a dead include inherits SPF-pass authority over the victim domain) with strict transient-vs-NXDOMAIN discrimination discipline so a temporary SERVFAIL is never mistaken for a takeover lead. Fully passive: DNS TXT reads only, no mail sent, no RCPT TO probe, no API keys. Use when auditing a domain's real spoofing resistance (not just its published records), explaining to a client why 'we have SPF -all' does not mean they're covered, investigating an SPF PermError or an unusually long include chain, evaluating a dead SPF include as a takeover lead, or writing a defensible spoofability finding for a deliverable.
FAIR-aligned exposure quantification: turns a pile of recon findings into a defensible 0-100 + A-F org risk score (Likelihood x Impact, three ownership-aware factors: exposure/threat/impact), an ownership + proof demotion cap so unproven or weakly-owned findings can't inflate the number, a $-denominated FAIR loss-magnitude estimate (IBM/Ponemon per-record cost bands, cross-source record dedup, threat-factor annualization), attack-path amplification (curated red-team chain catalog + generic graph-walk engine, with a kill-chain vs shared-fate honesty gate), and a board-ready one-pager deliverable (hero $ + letter grade + top-3 findings + top attack path + the ask). Extends osint-methodology's severity rubric and client deliverable templates with quantification. Passive analysis only -- operates on findings already collected, no target traffic, no API keys. Use when asked to score risk, quantify exposure, estimate breach cost, build a board report, translate technical findings to dollars, or explain why a grade or dollar figure came out the way it did.
Organization-grade identity-fabric mapping: tenant/federation fingerprinting and the pre-auth user-ENUMERATION oracle methodology — enumeration and fingerprint only, never credential submission. Covers domain-to-tenant resolution (Microsoft getuserrealm.srf Managed/Federated namespace check, Entra OIDC metadata tenant-GUID extraction, Autodiscover v2), keyless Microsoft tenant-federation mapping (GetFederationInformation SOAP -> sibling-domain discovery, discover-only ROE, FEDERATED_WITH provenance edge held out of attack-path pivoting), Okta org-slug derivation + OIDC fingerprint + governed custom-domain enumeration, ADFS passive/active fingerprint + version inference, Google Workspace MX-correlated detection, generic OIDC (Auth0/Keycloak/Ping Identity/OneLogin/Duo) discovery, SAML metadata (5 paths), Azure AD Seamless-SSO Negotiate-challenge detection, Microsoft Defender for Identity (MDI) sensor-API presence check, the user-enumeration oracle methodology for Microsoft GetCredentialType (IfExistsResult semantics: exists / doesn't-exist / exists-in-federated-tenant / throttled) and Okta /api/v1/authn (errorCode differential), Medium-detectability discipline with a hard 20-candidate-per-tenant cap and admin/role interest-based ranking, and name x confirmed-email-pattern login-candidate synthesis that FAILS CLOSED with zero output when no org pattern is confirmed. Grounded directly in a production ASM implementation's sso_idp.py, tenant_recon.py, and core/email_patterns.py modules. Deepens — does not duplicate — offensive-osint skill's Identity Fabric endpoint reference with the tenant-federation MAP, the oracle WORKFLOW, and the candidate-SYNTHESIS methodology that reference lacks. Use when fingerprinting an organization's identity provider, mapping its tenant/federation boundary, running an authorized pre-auth user-enumeration pass, or synthesizing login candidates from harvested names to feed that oracle — never for password spray, credential submission, or auth bypass.
Org-grade attack-surface mapping: given a company's legal identity, discover its ENTIRE owned internet footprint — corporate family -> owned domains -> owned netblocks/ASN -> live assets — with attribution discipline, not just DNS breadth. The org-first attribution pyramid (legal entity -> LEI/registration -> corporate family -> owned domains -> owned netblocks/ASN -> live assets). Corporate-identity resolution via the GLEIF LEI API (legal name -> LEI, exact-LEI direct-children expansion, downward-only depth-capped BFS, NEVER a name re-resolution), SEC-EDGAR full-text search + Exhibit-21 subsidiary entity names, OpenCorporates entity corroboration, Wikidata SPARQL corporate graph (P856/P355/P749/P1830). Domain attribution via reverse-WHOIS (WhoisXML preview-then-purchase quota guard, SecurityTrails associated-domains — both paid), crt.sh O= certificate-transparency organization pivot (keyless), infrastructure correlation (shared NS/MX/SaaS-TXT, netblock membership, reverse-DNS PTR), and the independent-evidence combiner (1-prod(1-w_i), rule of three, OwnerTier NONE/WEAK/MODERATE/STRONG/CONFIRMED) — discover-only `related:` candidates are NEVER auto-scanned. Netblock/ASN attribution via org-first RIR queries (ARIN Whois-RWS org-handle search, RIPE DB organisation + inverse-org search) that recover 'dark netblocks' with no DNS link to the seed, ASN discovery (RIPEstat + BGPView union), the HYPERSCALER-SCOPE GUARD (never attribute a whole AWS/GCP/Azure/Cloudflare announced range to a tenant — keep only the seed-containing block, tag shared_hosting_cdn), and org-identity-seeded internet-scan-index queries (Shodan/Censys/ZoomEye/FOFA/BinaryEdge org: filters + an always-on keyless crt.sh fallback). Promote-to-scan triage ranks forgotten discover-only netblocks by remote-access exposure (gateway-vendor/KEV/control-plane/datastore port scoring) into an operator queue. Attribution confidence rubric + anti-patterns (namesake grafting via GLEIF name re-resolution, single-signal ownership, hyperscaler over-attribution, privacy-WHOIS pivot poisoning, RIR org-name collisions). Passive/keyless-first OSINT only — every paid-key dependency is enrichment on top of a keyless core, never a hard requirement. Use when mapping an organization's full corporate-family internet footprint, resolving a legal entity to its LEI/subsidiaries, discovering domains/netblocks/ASNs an org owns beyond its one seed domain, auditing M&A/shadow-IT sprawl, or scoping an engagement that starts from a company NAME rather than a domain.