This commit is contained in:
2026-03-15 18:23:58 +02:00
parent 6bf221de3f
commit ad3385c63b
8 changed files with 319995 additions and 8 deletions

View File

@@ -6,11 +6,13 @@ Small Go scraper for the Outward Fandom wiki.
```text
.
├── cmd/outward-web/main.go # web UI entrypoint
├── cmd/scrappr/main.go # binary entrypoint
├── internal/app # bootstrapping and output writing
├── internal/logx # colored emoji logger
├── internal/model # dataset models
├── internal/scraper # crawl flow, parsing, queueing, retries
├── internal/webui # embedded web server + static UI
├── go.mod
├── go.sum
└── outward_data.json # generated output
@@ -22,6 +24,10 @@ Small Go scraper for the Outward Fandom wiki.
go run ./cmd/scrappr
```
```bash
go run ./cmd/outward-web
```
## What It Does
- Crawls item and crafting pages from `outward.fandom.com`
@@ -32,6 +38,7 @@ go run ./cmd/scrappr
- Stores legacy and portable infobox fields, primary item image URLs, recipes, effects, and raw content tables for later processing
- Saves resumable checkpoints into `.cache/scrape-state.json` on a timer, during progress milestones, and on `Ctrl+C`
- Writes a stable, sorted JSON dataset to `outward_data.json`
- Serves a local craft-planner UI backed by recipes from `outward_data.json`
## Tuning