warpbox-dev/docs/Design Docs/Warpbox.dev Design & Requirements – Self‑Hosted File Transfer Platform(1).md

Warpbox.dev Design & Requirements – Self‑Hosted File Transfer Platform

1. Vision and Goals

Warpbox.dev is a self‑hosted, open‑source file transfer and lightweight file hosting service inspired by PsiTransfer, transfer.sh, and Gofile, but built entirely in Go with a minimal vanilla JS/CSS frontend. It must be easy to rebrand and run as a multi‑tenant or single‑tenant instance (e.g., warpbox.dev) while also supporting fully self‑hosted private deployments via Docker.[^1][^2][^3]

Core goals:

• Anonymous, frictionless uploads (similar to PsiTransfer / transfer.sh).[^2][^3][^1]
• Optional “account mode” with folders, file management, and dashboards similar to Gofile.[^4][^5][^6]
• Admin console for moderation, statistics, and storage management.[^7]
• First‑class CLI and curl upload support for automation.[^8][^9][^2]
• High‑quality link previews (Discord, social, chat apps) with working image/video embeds where possible via Open Graph/Twitter Card metadata.[^10][^11][^12]
• Simple, secure API for custom clients, including ShareX integration.

2. High‑Level Feature Set

2.1 Anonymous Upload Mode

Anonymous upload mode mimics PsiTransfer and transfer.sh: anyone can upload without an account and immediately receive a shareable link.[^3][^1][^2]

Required features:

• Drag‑and‑drop web UI for uploading one or many files.
• Upload very large files (configurable max size; resumable where possible).
• Generated “bucket”/“upload” ID with random token in URL (e.g., /d/{id} or /b/{id}).
• Optional one‑time download links and max downloads / max days, similar to transfer.sh/PsiTransfer.[^13][^2][^3]
• Automatic expiration and cleanup based on configured policies.
• Optional password‑protected buckets (AES or equivalent; password known only to the user, stored hashed/encrypted server‑side), like PsiTransfer’s password‑protected download list.[^14][^13][^3]
• Option to download all files in a bucket as zip/tar.gz.[^13][^3]

Nice‑to‑have:

• Simple frontpage with “quick upload”, recent uploads for the current browser session only, and copy‑link buttons.
• Human‑readable short URLs in addition to long random IDs.

2.2 Authenticated User Mode ("Self‑Hosted Gofile")

Authenticated mode should approximate a lightweight, self‑hosted Gofile: users can register/log in, create folders, and manage uploads.[^5][^6][^15][^4]

Required features:

• User accounts (email + password, or OAuth providers as optional later).
• Folder hierarchy for each user (root + nested folders).
• Upload files into specific folders from the web UI.
• Per‑file metadata: filename, size, content‑type, upload date, last accessed, owner, visibility (public/unlisted/private), expiration, password.
• Per‑folder metadata: name, created at, parent folder, total size, number of files.
• File actions: rename, move, delete, set expiration, set visibility, set password.
• Folder actions: rename, move, delete (with safety checks), share folder as “public gallery”.
• User dashboard: list of uploads, sort/filter, basic stats (total storage, bandwidth used, recent activity).

Nice‑to‑have:

• Guest tokens like Gofile uses for pseudo‑accounts attached to a browser session.[^16]
• Tagging of files and folders.
• Simple search (by filename, extension, MIME type, tags) within a user’s space.

2.3 Admin Backend & Management

Admin capabilities should go beyond PsiTransfer’s simple /admin bucket list and provide real operational visibility.[^1][^7][^13]

Required features:

• Admin authentication (separate role from regular users; ideally role‑based access control with roles such as admin, moderator, support).
• Admin dashboard with:
  • Total files, total size on disk, total bandwidth used.
  • Daily/weekly/monthly uploads and downloads.
  • Top files by size, downloads, or bandwidth.
  • Top users by storage usage and bandwidth.
• Bucket/file management:
  • Search by ID, filename, user, IP, or date range.
  • View metadata and status of an upload bucket.
  • Force delete file/bucket.
  • Extend or shorten expiration date.
  • Lock or disable a user account.
• Abuse/moderation tools:
  • Flag content as suspected TOS violation.
  • Mark user or IP as blocked (no further uploads).
  • Optionally integrate with external abuse reporting or virus scanning services (later).

Nice‑to‑have:

• Audit logs of admin actions.
• Config UI for system‑wide settings: max file size, default expiration, storage backend, mail settings, etc.

3. Upload and Download Flows

3.1 Web Uploads

Web uploads should behave similarly to PsiTransfer: responsive UI, multiple files, resumable uploads where possible.[^14][^3][^13]

Core behaviors:

• Drag‑and‑drop multiple files or folder selection.
• Progress bars per file and aggregated progress.
• Resumable uploads for large files using tus.io or a similar resumable upload protocol, which is widely used in PsiTransfer‑like tools and other self‑hosted solutions.[^17][^14]
• Graceful error handling: network failures, timeouts, file too large, storage quota exceeded.

3.2 CLI / Curl Uploads

Curl upload support is a hard requirement and should be as simple as transfer.sh’s interface.[^18][^9][^8][^2]

Required behaviors:

• Basic upload:

  curl --upload-file ./file.ext https://warpbox.dev/file.ext
  

  Returns a direct URL on stdout (e.g., https://warpbox.dev/d/{id}/file.ext).[^19][^2]

• Multiple files via multipart form:

  curl -F file=@file1 -F file=@file2 https://warpbox.dev/upload
  

• HTTP headers or query parameters for options, similar to transfer.sh:

  • Max-Downloads – max download count before expiration.
  • Max-Days – max lifetime in days.[^2]
  • Optional X-Warpbox-Password – to set a download password.
  • Optional X-Warpbox-Folder – target folder for authenticated uploads.

• Simple one‑liner helper function in docs (bash/zsh/fish/PowerShell) inspired by transfer.sh examples.[^9][^2]

Nice‑to‑have:

• warpbox CLI wrapper (Go binary) as an alternative to raw curl.

3.3 Download Behavior

Downloads must be simple but efficient and support range requests for partial downloads/resume.[^14]

Required behaviors:

• Support HTTP range requests (Accept-Ranges) so download managers and browsers can resume incomplete downloads.[^14]
• Correct Content-Type, Content-Length, and Content-Disposition headers.
• Optional streaming mode for media (see section 5) when content type is video/* or image/*.
• Respect max‑downloads and expiration on every request.

4. API Design

4.1 General Principles

The API should be a simple JSON‑over‑HTTP REST API secured with token‑based authentication for privileged operations.

General conventions:

• Base path: /api/v1.
• Authentication:
  • User API tokens (per account) for user endpoints.
  • Admin API tokens (different scope) for admin endpoints.
• Request/response: JSON bodies for metadata, multipart/form‑data for file upload.
• Pagination for list endpoints (page, pageSize or limit/offset).

4.2 Public/Anonymous Endpoints

• POST /api/v1/upload

  • Anonymous upload.
  • Accepts multipart form file field(s).
  • Optional form fields or headers: max_downloads, max_days, password, filename.
  • Response: bucket ID, list of file URLs, expiration.

• GET /api/v1/buckets/{id}

  • Returns limited metadata about anonymous bucket if public.

• DELETE /api/v1/buckets/{id} (with delete token)

  • If bucket was created anonymously, return a management/delete token in the response URL (similar to Gofile management links and other anonymous file hosts).[^4]

4.3 Authenticated User Endpoints

• POST /api/v1/auth/login
• POST /api/v1/auth/register
• POST /api/v1/auth/refresh
• GET /api/v1/me – profile, usage quotas.

Folders:

• GET /api/v1/folders – list root folders.
• POST /api/v1/folders – create folder.
• GET /api/v1/folders/{id} – folder details + paginated files.
• PATCH /api/v1/folders/{id} – rename/move.
• DELETE /api/v1/folders/{id}.

Files:

• POST /api/v1/folders/{id}/upload – upload into folder via multipart.
• GET /api/v1/files/{id} – file metadata.
• PATCH /api/v1/files/{id} – change filename, visibility, expiration, password, folder.
• DELETE /api/v1/files/{id}.

4.4 Admin Endpoints

• GET /api/v1/admin/stats/overview

  • Global metrics: total size, total files, uploads/downloads per day.[^7]

• GET /api/v1/admin/files / GET /api/v1/admin/buckets

  • Search/filter by user, size, date, IP, status.

• PATCH /api/v1/admin/files/{id}

  • Force delete, change expiration, lock.

• GET /api/v1/admin/users

• PATCH /api/v1/admin/users/{id} – change roles, lock user.

• GET /api/v1/admin/logs – optional; audit logs.

5. Embeds and Social Link Previews

5.1 Discord and Social Embeds Basics

Discord and other platforms primarily rely on Open Graph meta tags (og:*) and Twitter Card tags for rich embeds.[^11][^12][^20][^10]

Key tags supported by Discord:

• og:title – title of the page.
• og:description – description text under the title.
• og:image – preview image.
• og:url – canonical URL.
• og:site_name – small text above the title.
• Optional theme-color – sets the colored left border of the embed in Discord.[^12][^11]

Discord also reads Twitter Card tags, with twitter:card recommended as summary_large_image for large previews.[^10][^12]

5.2 Behavior for Image and Video Files

Image embeds:

• For image/* files, serve a landing page (/v/{id}) that includes:
  • og:type = "website".
  • og:title = filename or user‑supplied title.
  • og:description = short description or “Shared via Warpbox”.
  • og:image = direct URL of the image.
  • og:url = landing page URL.
  • og:site_name = brand/site name.
  • theme-color = configurable brand color.[^11][^12][^10]

Video embeds:

• Many platforms restrict video embeds to a whitelist of providers; Discord only allows video playback via OG/OEmbed for a small set of providers like YouTube.[^21][^20][^12]
• For generic video files (video/*), fully inline video playback in Discord is not reliable, but the best practice is:
  • Set og:image to a generated thumbnail.
  • Use og:title, og:description, og:url as above.
  • Provide a direct download/stream link in the embed description.

Document and generic files:

• Use a generic icon preview or first‑page thumbnail (for PDFs) as og:image.

5.3 OEmbed Considerations

Discord supports oEmbed but only for whitelisted providers; for most self‑hosted sites, Open Graph + Twitter Card tags are the primary mechanism.[^20][^12][^11]

Nice‑to‑have:

• Provide an OEmbed endpoint (/oembed) to maximize compatibility with clients that do support custom providers.

6. ShareX and Desktop Integration

6.1 ShareX Custom Uploader

ShareX can integrate with custom uploaders via .sxcu configuration files, which specify the request URL, file field name, headers, and URL patterns.[^22][^23][^24][^25]

Requirements:

• HTTP endpoint suitable for ShareX:
  • POST https://warpbox.dev/api/v1/upload for anonymous.
  • POST https://warpbox.dev/api/v1/folders/{id}/upload for authenticated.
• Accept multipart form field file or sharex (configurable) with the uploaded file.[^25]
• Support Authorization header with API key/token, as many setups do.[^24][^22][^25]
• Respond with JSON containing a direct link, e.g. { "url": "https://warpbox.dev/d/{id}/file.ext" }.

The project should ship example .sxcu files and documentation on how to import them, matching typical ShareX guides.[^23][^22][^25]

6.2 Warpbox.exe Helper

Warpbox should ship a small warpbox.exe for Windows and an equivalent shell script for Linux/macOS.

Concept:

• warpbox.exe is a simple CLI wrapper around curl that:

  • Reads configuration (server URL, API key, default folder) from a config file.
  • Provides basic commands: warpbox upload file.ext, warpbox url etc.
  • Optionally generates a .sxcu file tailored to the instance.

• For Linux/macOS, a warpbox.sh shell script:

  • Validates presence of curl.
  • Wraps example curl commands.

• Documentation:

  • Step‑by‑step instructions to set up ShareX with Warpbox using the .sxcu file and API token, based on common patterns in existing tutorials.[^22][^23][^24][^25]

7. Architecture Overview (Golang + Vanilla JS/CSS)

7.1 Backend Architecture (Go)

Core stack:

• Language: Go.
• HTTP framework: standard library net/http or lightweight router (e.g., chi, gorilla/mux) as an implementation detail.
• Persistence options:
  • PostgreSQL or MySQL for structured data (users, files, folders, tokens, stats).
  • Optional SQLite for single‑user or small instances.
• Storage backend:
  • Local filesystem (mount volume in Docker) for simple setups.
  • Pluggable S3‑compatible backend for scalable deployments, similar to how transfer.sh supports S3.[^26]

Major components:

• API server: handles REST endpoints, authentication, and business logic.
• Upload engine: streaming uploads to disk/S3; optional resumable uploads (tus.io‑like protocol implementation or existing Go tus library).[^17][^14]
• Download server: efficient range‑enabled file serving.
• Admin module: stats aggregation, moderation, storage quotas.
• Background workers:
  • Expiration/cleanup of old files and buckets.
  • Pre‑generation of thumbnails for images/videos.
  • Periodic stats aggregation for dashboard.

7.2 Frontend Architecture (Vanilla JS/CSS)

The frontend should be framework‑free to minimize footprint and keep rebranding simple.

Key pages:

• Landing / anonymous upload.
• Auth pages: login, register, password reset.
• User dashboard (folders/files grid or list view).
• File/bucket details view.
• Admin dashboard.

Implementation notes:

• Use vanilla JS fetch API for calling the backend.
• Minimal build tooling; optionally a simple bundler for minification.
• CSS:
  • Modular BEM‑style classes, or simple utility classes.
  • Dark/light themes with CSS variables for easy rebranding.

8. Deployment, Docker, and From‑Scratch Image

8.1 Docker Image Strategy

Warpbox should ship an official Docker image with a multi‑stage build, which is common best practice for Go services.[^26]

Recommended Dockerfile approach:

1. Builder stage

   • Base: golang:1.x-alpine.
   • Fetch dependencies, build statically linked binary with CGO_ENABLED=0.

2. Runner stage

   • Base: FROM scratch.
   • Copy binary from builder.
   • Copy static assets (HTML, JS, CSS) into an internal directory.
   • Expose port (e.g., 8080).
   • Set entrypoint to the Go binary.

This keeps the runtime image extremely small while supporting a fully self‑contained binary.[^26]

8.2 Configuration and Environment

• Configuration via environment variables:

  • WARPBOX_DB_DSN, WARPBOX_STORAGE_PATH or S3 credentials.
  • WARPBOX_MAX_FILE_SIZE, WARPBOX_DEFAULT_EXPIRATION_DAYS.
  • WARPBOX_BRAND_NAME, WARPBOX_BRAND_COLOR for rebranding.
  • WARPBOX_ADMIN_EMAIL, WARPBOX_ADMIN_PASSWORD for bootstrapping.

• Provide Docker Compose examples for:

  • Warpbox + Postgres + Traefik/NGINX reverse proxy.
  • Warpbox with S3 backend.

9. Security, Privacy, and Abuse Handling

9.1 Security Baselines

• All access via HTTPS (left to reverse proxy, but documented).
• Strong password hashing (Argon2id, bcrypt) for user credentials.
• CSRF protection for web UI forms (if cookies used).
• Strict API authentication with bearer tokens over HTTPS.
• Rate limiting for anonymous uploads to prevent abuse.

Nice‑to‑have:

• Content‑type and file extension validation.
• Optional integration with virus scanning (e.g., ClamAV or third‑party services), similar to transfer.sh’s optional virus scanning feature.[^26]

9.2 Privacy and Anonymity

• Anonymous uploads should not require registration and may log minimal metadata (IP, user agent) only for abuse prevention.
• Clear configuration for retention of logs and metadata.
• Optional “privacy mode” where IPs are hashed or truncated.

9.3 Abuse and Legal Considerations

• Admin tools to quickly remove illegal or infringing content and ban abusive users/IPs.[^7]
• TOS/Acceptable Use Policy recommended for public instances.

10. SEO and Discoverability

Even though Warpbox links are primarily shared directly, basic SEO ensures public pages index correctly.

Required SEO features:

• Proper <title> and <meta name="description"> tags on public/view pages.
• Canonical URLs using >.
• Open Graph and Twitter Card tags as described in section 5.[^12][^10]
• robots.txt with sensible defaults (e.g., disallow admin endpoints, allow public galleries).
• Sitemap for discoverable public galleries (optional, enabled only for public instances).

11. Rebranding and Theming

Warpbox should be easy to fork and rebrand, or to configure branding at runtime.

Required:

• All exposed names, titles, and meta tags use configurable brand name.
• Primary colors defined as CSS variables and customizable via config.
• Logo and favicon paths configurable (e.g., mount custom assets in Docker volume).

Nice‑to‑have:

• Simple theme system (JSON/YAML theme configs) to quickly switch branding.

12. Quality‑of‑Life Features and Future Enhancements

Potential QoL features inspired by existing self‑hosted file sharing platforms and user expectations:[^13][^1][^7][^26]

• Drag‑and‑drop upload that works across desktop and mobile.
• Clipboard helpers: one‑click copy link, copy embed code.
• QR code generation for download URLs.
• Simple “text paste” mode for sharing snippets (Gist‑like).
• Optional email notification to the uploader when a file is first downloaded.
• Light‑weight analytics for individual files (download count, last download time).
• Multi‑language support (i18n).
• Public folder galleries with grid view of images/videos.

These are not necessary for v1 but can significantly improve the perceived polish and usability of Warpbox.

13. Minimal Viable Product (MVP) Scope

To keep the first version shippable, the following is a suggested MVP:

• Anonymous file uploads (web + curl) with:
  • Max file size config.
  • Expiration (by days) and optional max downloads.
  • Basic landing page with Open Graph image preview.
• Simple user accounts with:
  • Single‑level “folders” or collections.
  • Upload into folders via web UI.
  • Basic dashboard listing files and folders.
• Admin panel:
  • Login.
  • Global stats (total size, total files, recent uploads).
  • Search and delete files.
• API:
  • POST /api/v1/upload for anonymous.
  • Basic authenticated upload and file management endpoints.
• Docker multi‑stage build with FROM scratch runner and minimal configuration variables.

Future iterations can then add:

• Resumable uploads.
• Advanced admin analytics.
• ShareX presets and warpbox.exe.
• S3 backend and multi‑node scaling.
• Thumbnails, galleries, and richer embeds.

---

References

1. GitHub - psi-4ward/psitransfer: Simple open source self-hosted file ... - Simple open source self-hosted file sharing solution. It's an alternative to paid services like Drop...

2. transfer.sh - Easy and fast file sharing from the command-line.

3. PSiTransfer - A Simple Open Source Self-hosted File Sharing Solution - PSiTransfer is a simple and open source file sharing utility used to share your files locally or glo...

4. Gofile.io 匿名文件分享平台，不限文件大小和下载速度- 网盘 - Gofile Introduction Gofile.io is a free and anonymous file sharing platform where users can quickly ...

5. Understanding Gofile: A Gateway to Anonymous File Sharing - Oreate AIwww.oreateai.com › blog › understanding-gofile-a-gateway-to-anonymous... - Gofile.io offers free anonymous file sharing with no data limits while emphasizing privacy through e...

6. Gofile.io: Free, anonymous file sharing with no limits - AlternativeTo - Enjoy free, anonymous file sharing without data limits. Upload, store, and stream media files at max...

7. Erugo - Self-Hosted File Sharing Platform - Erugo is a powerful, self-hosted file-sharing platform that puts you in complete control of your dat...

8. transfer.sh command - github.com/dutchcoders ...

9. transfer.sh/examples.md at main · dutchcoders/transfer.sh - Easy and fast file sharing from the command-line. Contribute to dutchcoders/transfer.sh development ...

10. How to Add Social Media Embeds To Your Website (Open Graph ... - The configuration for these embeds is done using tags, usually in the header of your site. Th...

11. Discord Website Embeds - Nora's Hideout - GitHub Pages - A Discord website embed is composed of several HTML meta tags, and optionally, an OEmbed JSON file. ...

12. Discord Embed Meta Tags: Complete Reference - opengraphplus.com - Discord supports oEmbed, but only for whitelisted providers (YouTube, Twitch, Spotify, etc.). For mo...

13. PsiTransfer is a Free Self-hosted File Transfer Solution - MEDevel.com - Features. No accounts, no logins; Mobile friendly responsive interface; Supports many and very big f...

14. PsiTransfer - psi.cx - Simple open source self-hosted file sharing solution with robust, resumeable up- and downloads of la...

15. Send Large Files - Up to 5GB Free - Gofile - Gofile is the simplest way to send your files around the world. We host the Internet!

16. KasimKaizer/gofileioupload: A simple package to upload ... - A simple package to upload files to gofile.io. Contribute to KasimKaizer/gofileioupload development ...

17. Self hosted file transfer that can resume from connection interruptions. - Hi, I'm new to this subreddit and hope this is the right place to ask this. I am trying to host my o...

18. Upload files from command line using tranfer.sh - Ever stuck on a remote server with some file? Needed to upload/download file from remote server? Fin...

19. GitHub - jahands/transfer.sh: Easy and fast file sharing from the command-line. - Easy and fast file sharing from the command-line. Contribute to jahands/transfer.sh development by c...

20. What standard does Discord use when generating embeds from links? - What meta-tags (I suppose it's meta tags, anyway) does Discord look for when generating things like ...

21. How can I add an OG meta tag that embeds a video? - Stack Overflow - I have been trying to embed a video in my HTML page using OG Meta Tag and it does not work apparentl...

22. ShareX | S.EE Docs - Open ShareX · Go to Destinations > Custom uploader settings · Click Import, then either: Select the ...

23. Setting up ShareX and Image Uploaders | New Day RP - Here are a few general guides to using ShareX with an uploader service, along with general recommend...

24. Custom domain for your images with ShareX - DEV Community - To upload your images, you need to make it request on your server and especially on a file. There ar...

25. ShareX CUSTOM Domain/URL Setup - YouTube - How to setup ShareX with your own custom domain or URL, this guide details the configuration and req...

26. transfer.sh Deploy Guide - Zeabur - Key Features · Upload files from terminal with curl, wget, or any HTTP client · Built-in web upload ...