job-scraper

# Job Scraper

Search for job postings across LinkedIn and Indeed using Apify. Find open roles by keyword, location, company, or job type. Use for hiring signal detection, GTM research, or competitive intelligence.

No LinkedIn cookies. No Indeed login. Just search queries in, structured job data out.

## When to Auto-Load

Load this skill when:
- User says "find jobs", "who is hiring", "what roles is [company] hiring for"
- User wants hiring signals ("find companies growing their AI team")
- User wants competitive intelligence ("what is [competitor] hiring for")
- User says "job search", "open roles", "job listings", "job postings"

## Prerequisites

### Apify API Token

Required for both LinkedIn and Indeed scraping. Set in `.env`:

```
APIFY_API_TOKEN=your_token_here
```

No LinkedIn cookies, Indeed login, or any platform credentials needed. That's the only setup.

---

## Sources

This skill searches two job platforms via Apify actors:

| Source | Apify Actor | Best For | Cost |
|--------|------------|----------|------|
| **LinkedIn** | `automation-lab/linkedin-jobs-scraper` | B2B, tech, SaaS, enterprise roles. Has seniority level, job function, industries. | ~$0.002/job |
| **Indeed** | `borderline/indeed-scraper` | Broadest coverage. Richest data — salary, company details, ratings, contacts, street addresses. | ~$0.004/job |

### Source Selection Logic

**Do NOT ask the user which source to use unless genuinely ambiguous.** Decide based on context:

1. **User specifies a source** → use that source only.
2. **Context strongly suggests one source:**
   - B2B/tech/SaaS roles, enterprise companies, seniority-level filtering → **LinkedIn**
   - Hourly/blue-collar roles, local/retail jobs, salary-focused search → **Indeed**
   - Company hiring research ("what is Stripe hiring for") → **LinkedIn** (better company filtering)
3. **No clear signal** → search **both sources**, deduplicate results by job title + company name, present combined results.

After deciding, **tell the user which source(s) you're searching and why.** Don't ask — inform.

---

## Workflow

### Phase 0: Understand the Request

Extract from the user's message:
- **Search term** — job title, role, or keyword (required)
- **Location** — city, state, country, or "Remote" (optional)
- **Company** — specific company name (optional)
- **Recency** — "recent", "last week", "last 30 days" (optional)
- **Job type** — fulltime, parttime, contract, internship (optional)
- **Remote** — whether to filter for remote jobs (optional)
- **Result count** — how many results they want (default: 25)

If anything is ambiguous, pick reasonable defaults and tell the user what you chose. Do not ask clarifying questions for things you can reasonably infer.

### Phase 1: Search

#### LinkedIn — `automation-lab/linkedin-jobs-scraper`

**API call:**
```bash
curl -X POST "https://api.apify.com/v2/acts/automation-lab~linkedin-jobs-scraper/runs?token=$APIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "searchQuery": "AI engineer",
    "location": "San Francisco",
    "maxItems": 25
  }'
```

**Input fields:**
| Field | Type | Description |
|-------|------|-------------|
| `searchQuery` | string | Job title or keywords (required) |
| `location` | string | City, state, or country (optional) |
| `maxItems` | integer | Max jobs to return (default: 50) |

**Polling for results:**
```bash
# Check run status (poll every 10s)
curl "https://api.apify.com/v2/acts/automation-lab~linkedin-jobs-scraper/runs/{RUN_ID}?token=$APIFY_API_TOKEN"

# When status is SUCCEEDED, fetch results
curl "https://api.apify.com/v2/datasets/{DATASET_ID}/items?token=$APIFY_API_TOKEN"
```

**Output fields per job:**
- `title` — Job title
- `companyName` — Company name
- `companyLinkedinUrl` — Company LinkedIn page
- `companyLogo` — Logo URL
- `location` — City, state
- `salary` — Salary text (when available)
- `employmentType` — Full-time, Part-time, Contract, etc.
- `seniorityLevel` — Entry, Mid-Senior, Director, Executive, etc.
- `jobFunction` — Engineering, Sales, Marketing, etc.
- `industries` — Industry classification
- `descriptionText` — Full job description (plain text)
- `descriptionHtml` — Full job description (HTML)
- `applicantsCount` — Number of applicants
- `postedAt` — When posted (e.g., "6 days ago")
- `url` — Direct link to the LinkedIn job posting
- `applyUrl` — Direct apply URL

#### Indeed — `borderline/indeed-scraper`

**API call:**
```bash
curl -X POST "https://api.apify.com/v2/acts/borderline~indeed-scraper/runs?token=$APIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "AI engineer",
    "location": "San Francisco, CA",
    "country": "us",
    "maxResults": 25
  }'
```

**Input fields:**
| Field | Type | Description |
|-------|------|-------------|
| `query` | string | Job title or keywords (required) |
| `location` | string | City and state (optional) |
| `country` | string | Lowercase 2-letter country code (required). Common: `us`, `uk`, `ca`, `de`, `fr`, `in`, `au` |
| `maxResults` | integer | Max jobs to return |

**Important:** The `country` field is required for Indeed. If the user doesn't specify a country, default to `us`. Use lowercase 2-letter codes only.

**Output fields per job:**
- `title` — Job title
- `companyName` — Company name
- `companyDescription` — Company description
- `companyNumEmployees` — Company size
- `companyRevenue` — Company revenue range
- `companyUrl` — Company Indeed page
- `location` — Object with `city`, `postalCode`, `country`, `formattedAddressShort`, `latitude`, `longitude`, `streetAddress`
- `salary` — Object with `salaryCurrency`, `salaryMin`, `salaryMax`, `salaryText`, `salaryType` (hourly/yearly)
- `descriptionText` — Full job description (plain text)
- `descriptionHtml` — Full job description (HTML)
- `datePublished` — Posted date (YYYY-MM-DD)
- `age` — Human-readable age ("24 days ago")
- `expired` — Whether job is still active
- `isRemote` — Remote flag
- `jobType` — E