Job Scraper

Job board monitoring for privacy-focused and open-source companies.

Quick Reference

Item	Value
URL	https://jobs.novanexus.ca
Port	8085
Containers	job-scraper-scheduled, job-dashboard
Data	./data/jobs.db, ./data/dashboard.html

Development Workflow

After making code changes, rebuild and deploy:

cd ~/job-scraper && sudo docker compose build && sudo docker compose up -d

Common Commands

# View status
sudo docker compose ps

# View logs
sudo docker compose logs -f scraper-scheduled

# Run scraper manually (one-time)
sudo docker compose run --rm scraper

# Restart services
sudo docker compose restart

# Rebuild without cache (if having issues)
sudo docker compose build --no-cache && sudo docker compose up -d

Configuration

config.yaml - Companies to monitor, filters, notifications
After config changes: sudo docker compose restart scraper-scheduled

Email Notifications

Uses msmtp with system config (~/.msmtprc). The container mounts this file.

To test email manually:

sudo docker compose run --rm scraper

Files

main.py - CLI entry point, scheduling
db.py - SQLite database operations
notify.py - Console, msmtp, email, Slack notifications
dashboard.py - HTML dashboard generator
scrapers/ - Platform-specific scrapers (greenhouse, lever, ashby)

Documentation

Full docs: ~/maple-docs/docs/04-services/job-scraper.md

1.4 KiB Raw Blame History