industry-scanner

# Industry Scanner

Daily deep-research agent that scans the internet for everything relevant to a client's industry, then generates strategic GTM opportunities based on what it finds.

## Quick Start

```
Run an industry scan for <client>. Use the config at clients/<client>/config/industry-scanner.json.
```

Or for a weekly deeper scan:

```
Run a weekly industry scan for <client> with --lookback 7.
```

## Inputs

- **Client name** — determines which config and context files to load
- **Lookback period** (optional) — `1` for daily (default), `7` for weekly deep scan
- **Focus area** (optional) — limit scan to specific categories (e.g., "competitors only", "events only")

---

## Step-by-Step Process

### Phase 1: Load Configuration

1. Read `clients/<client>/config/industry-scanner.json` — this contains all the keywords, sources, competitors, and URLs to scan
2. Read `clients/<client>/context.md` — need the ICP, value props, and positioning to generate relevant strategies
3. Set the lookback period: use `1` day for daily scans, `7` for weekly, or whatever the user specifies
4. Note today's date for the output filename

If no client config exists, ask the user for the key inputs and offer to create one from the example at `skills/industry-scanner/config/example-config.json`.

### Phase 2: Data Collection

Run these data sources in parallel where possible. Skip any source that isn't configured. For each source, use the existing skill's CLI or tool as documented.

**IMPORTANT:** Run as many of these bash commands in parallel as possible to minimize total scan time. Sources are independent of each other.

#### 2A. Web Search (built-in WebSearch tool)

Run 5-8 web searches combining the configured `web_search_queries` with time-sensitive modifiers. Examples:

- `"<industry keyword> news this week"`
- `"<competitor name> shutdown OR closing OR acquired 2026"`
- `"<industry> conference 2026 speaker applications"`
- `"<industry keyword> new regulation OR policy change"`
- `"<competitor name> layoffs OR pivot OR rebrand"`

Also search for each competitor name directly to catch any recent news.

#### 2B. Industry Blogs & Publications

```bash
python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
  --urls "<comma-separated blog_urls from config>" \
  --days <lookback> --output json
```

Read `skills/blog-feed-monitor/SKILL.md` for full CLI reference.

#### 2C. Reddit

For each configured subreddit, run:

```bash
python3 skills/reddit-post-finder/scripts/search_reddit.py \
  --subreddit "<comma-separated subreddits from config>" \
  --keywords "<comma-separated reddit_keywords from config>" \
  --days <lookback> --sort hot --output json
```

Also run a separate search with `--sort top --time week` to catch high-engagement posts.

Read `skills/reddit-post-finder/SKILL.md` for full CLI reference.

#### 2D. Twitter/X

For each configured Twitter query:

```bash
python3 skills/twitter-mention-tracker/scripts/search_twitter.py \
  --query "<twitter_query>" \
  --since <yesterday-YYYY-MM-DD> --until <today-YYYY-MM-DD> \
  --max-tweets 30 --output json
```

Read `skills/twitter-mention-tracker/SKILL.md` for full CLI reference.

#### 2E. LinkedIn

Search each configured LinkedIn keyword via the linkedin-post-research skill.

Delegate to the `linkedin-post-research` skill (uses the `apimaestro~linkedin-posts-search-scraper-no-cookies` Apify actor). Search each keyword with `date_posted: "past-day"` (or `"past-week"` for weekly scans).

Read `skills/linkedin-post-research/SKILL.md` for the full Apify workflow.

#### 2F. Hacker News

```bash
python3 skills/hacker-news-scraper/scripts/search_hn.py \
  --query "<hn_query>" --days <lookback> --output json
```

Run once per configured `hn_queries` entry. Read `skills/hacker-news-scraper/SKILL.md` for full CLI reference.

#### 2G. RSS News Feeds

If the client has an accounting-news-monitor (or similar) configured:

```bash
python3 skills/accounting-news-monitor/scripts/monitor_news.py \
  --new-only --days <lookback> --output json
```

Read `skills/accounting-news-monitor/SKILL.md` for full CLI reference.

#### 2H. Newsletter Inbox

If the client has newsletter monitoring configured:

```bash
python3 skills/newsletter-monitor/scripts/scan_newsletters.py \
  --days <lookback> --output json
```

Read `skills/newsletter-monitor/SKILL.md` for full CLI reference.

#### 2I. Review Sites

For each configured review URL:

```bash
python3 skills/review-site-scraper/scripts/scrape_reviews.py \
  --platform <platform> --url "<review_url>" \
  --days <lookback> --max-reviews 20 --output json
```

Read `skills/review-site-scraper/SKILL.md` for full CLI reference.

### Phase 3: Consolidate & Categorize

After all data collection completes, consolidate the results:

1. **Deduplicate** — items appearing across multiple sources (e.g., a news story on both a blog and Reddit). Keep the richest version but note multi-source appearance (higher signal).

2. **Categorize** each item into one of these types:

| Category | What to Look For |
|----------|-----------------|
| **Competitor News** | Shutdowns, launches, funding, pivots, negative reviews, leadership changes, pricing changes |
| **Industry Events** | Upcoming conferences, webinars, meetups, speaker slots, CFPs, award nominations |
| **Market Trends** | Viral discussions, hot topics, emerging themes, sentiment shifts, adoption data |
| **Regulatory / Policy** | New regulations, compliance changes, government actions, standards updates |
| **People Moves** | Key hires, departures, promotions at competitors or target companies |
| **Technology** | New product launches, integrations, platform changes, deprecations |
| **Funding / M&A** | Acquisitions, mergers, funding rounds, PE investments, IPO signals |
| **Pain Points** | People publicly complaining about problems the client solves |
| **Content Opportunities** | Trending content, viral posts, gaps in existing coverage, unanswered questions |

3. **Rate relevance** — Hig