.
├── cmd/outward-web/main.go    # web UI entrypoint
├── cmd/scrappr/main.go       # binary entrypoint
├── internal/app              # bootstrapping and output writing
├── internal/logx             # colored emoji logger
├── internal/model            # dataset models
├── internal/scraper          # crawl flow, parsing, queueing, retries
├── internal/webui            # embedded web server + static UI
├── go.mod
├── go.sum
└── outward_data.json         # generated output

Run

go run ./cmd/scrappr

go run ./cmd/outward-web

What It Does

Crawls item and crafting pages from outward.fandom.com
Uses browser-like headers and rotating user agents
Limits crawl depth and queue size to avoid drifting into junk pages
Retries temporary failures with short backoff
Prints colored emoji logs for queueing, requests, responses, parsing, retries, and periodic status
Stores legacy and portable infobox fields, primary item image URLs, recipes, effects, and raw content tables for later processing
Saves resumable checkpoints into .cache/scrape-state.json on a timer, during progress milestones, and on Ctrl+C
Writes a stable, sorted JSON dataset to outward_data.json
Serves a local craft-planner UI backed by recipes from outward_data.json

Tuning

Scraper defaults live in internal/scraper/config.go.

Lower or raise RequestDelay / RequestJitter
Tighten or relax MaxQueuedPages
Adjust RequestTimeout, MaxRetries, ProgressEvery, AutosaveEvery, and AutosavePages