4 Commits

Author SHA1 Message Date
4eacb4cde2 fix(handlers): bypass box creation limits for batched uploads
All checks were successful
Build and Publish Docker Image / deploy (push) Successful in 1m42s
Update `createOrAppendBox` to accept the upload policy and admin status, allowing policy enforcement to be handled during the box creation/append decision process. This ensures that appending files to an existing batch does not incorrectly trigger daily or active box creation limits, as no new box is being created.

Also, add unit tests to verify that batched uploads successfully bypass both daily and active box creation caps.
2026-06-01 00:20:18 +03:00
71d9b9db7e perf(backend): optimize ban lookups and prune upload group map
- Optimize the ban matching middleware by using a read-only transaction (`db.View`) for the initial scan, avoiding the single bbolt write lock on every request when no ban matches.
- Implement periodic pruning of stale entries in the upload grouper map to prevent unbounded memory growth over time.
- Avoid redundant parsing of the `max_days` form value in the upload handler.
2026-06-01 00:12:43 +03:00
01996c0445 feat(policy): support unlimited values in user policies and box expiry
- Update user policy and user update handlers to accept -1 as an unlimited value for MaxDays, DailyBoxes, ActiveBoxes, and ShortWindowRequests.
- Introduce `optionalIntAllowUnlimited` helper and update `optionalMBAllowZero` to support -1.
- Use `boxExpiryLabel` helper across admin, dashboard, and download handlers to properly format expiration dates, supporting boxes that never expire.
2026-05-31 22:40:48 +03:00
adb1a12dfd feat(upload): support batching via header and update ShareX config
Introduce support for grouping multiple sequential file uploads into a single box using the `X-Warpbox-Batch` header. This is particularly useful for ShareX multi-file selections, which are sent as separate back-to-back requests.

Additionally, this change:
- Updates the ShareX configuration template to opt-in to batching by default.
- Switches ShareX configuration placeholders to the modern `{json:...}` format.
- Adds `thumbnailUrl` to the upload response schema and documents its usage.
2026-05-31 22:27:43 +03:00
24 changed files with 657 additions and 155 deletions

View File

@@ -182,7 +182,7 @@ Curl and custom uploaders can use the same endpoint:
# Terminal-friendly output: one plain box URL. # Terminal-friendly output: one plain box URL.
curl -F file=@./report.pdf http://localhost:8080/api/v1/upload curl -F file=@./report.pdf http://localhost:8080/api/v1/upload
# JSON output with boxUrl, manageUrl, deleteUrl, zipUrl, and file entries. # JSON output with boxUrl, thumbnailUrl, manageUrl, deleteUrl, zipUrl, and file entries.
curl -F sharex=@./screenshot.png \ curl -F sharex=@./screenshot.png \
-H 'Accept: application/json' \ -H 'Accept: application/json' \
http://localhost:8080/api/v1/upload http://localhost:8080/api/v1/upload
@@ -190,6 +190,19 @@ curl -F sharex=@./screenshot.png \
The upload endpoint accepts multipart fields named `file` and `sharex`. ShareX users can start The upload endpoint accepts multipart fields named `file` and `sharex`. ShareX users can start
from `examples/sharex/warpbox-anonymous.sxcu`; update `RequestURL` to match your instance URL. from `examples/sharex/warpbox-anonymous.sxcu`; update `RequestURL` to match your instance URL.
Authenticated uploads (your account's limits) add an `Authorization: Bearer <token>` header — mint
a token under **Account → Access tokens**. The JSON response uses ShareX placeholders
`{json:boxUrl}` (URL), `{json:thumbnailUrl}` (thumbnail), `{json:deleteUrl}` (deletion), and
`{json:error}` (error message).
### Grouping multiple files into one box (`X-Warpbox-Batch`)
By default every uploaded file becomes its own box. To put several files in a **single** box, send
the opt-in `X-Warpbox-Batch` header: requests that share the same header value (scoped per account,
or per IP for anonymous uploads) within 20s are appended to the same box. This lets a multi-file
ShareX selection — which ShareX sends as separate back-to-back requests — land as one shareable
link. The shipped `.sxcu` sets `X-Warpbox-Batch: sharex`; remove that header for one box per file.
Requests without the header behave exactly as before.
## Stage 4 Accounts + Personal Boxes ## Stage 4 Accounts + Personal Boxes

View File

@@ -414,6 +414,80 @@ func TestLayeredUploadLimits(t *testing.T) {
} }
} }
func TestBatchedUploadAppendBypassesDailyBoxCreationCap(t *testing.T) {
app, cleanup := newTestApp(t)
defer cleanup()
policy := testPolicy(t, app)
policy.AnonymousDailyBoxes = 1
policy.AnonymousActiveBoxes = 10
if err := app.settingsService.UpdateUploadPolicy(policy); err != nil {
t.Fatalf("UpdateUploadPolicy returned error: %v", err)
}
first := multipartUploadRequest(t, "/api/v1/upload", "file", "first.txt", "hello")
first.Header.Set("Accept", "application/json")
first.Header.Set(uploadBatchHeader, "sharex-test")
firstResponse := httptest.NewRecorder()
app.Upload(firstResponse, first)
if firstResponse.Code != http.StatusCreated {
t.Fatalf("first batched status = %d, body = %s", firstResponse.Code, firstResponse.Body.String())
}
second := multipartUploadRequest(t, "/api/v1/upload", "file", "second.txt", "hello")
second.Header.Set("Accept", "application/json")
second.Header.Set(uploadBatchHeader, "sharex-test")
secondResponse := httptest.NewRecorder()
app.Upload(secondResponse, second)
if secondResponse.Code != http.StatusCreated {
t.Fatalf("second batched status = %d, body = %s", secondResponse.Code, secondResponse.Body.String())
}
third := multipartUploadRequest(t, "/api/v1/upload", "file", "third.txt", "hello")
third.Header.Set("Accept", "application/json")
thirdResponse := httptest.NewRecorder()
app.Upload(thirdResponse, third)
if thirdResponse.Code != http.StatusTooManyRequests {
t.Fatalf("non-batched status = %d, body = %s", thirdResponse.Code, thirdResponse.Body.String())
}
}
func TestBatchedUploadAppendBypassesActiveBoxCreationCap(t *testing.T) {
app, cleanup := newTestApp(t)
defer cleanup()
policy := testPolicy(t, app)
policy.AnonymousDailyBoxes = 10
policy.AnonymousActiveBoxes = 1
if err := app.settingsService.UpdateUploadPolicy(policy); err != nil {
t.Fatalf("UpdateUploadPolicy returned error: %v", err)
}
first := multipartUploadRequest(t, "/api/v1/upload", "file", "first.txt", "hello")
first.Header.Set("Accept", "application/json")
first.Header.Set(uploadBatchHeader, "active-cap")
firstResponse := httptest.NewRecorder()
app.Upload(firstResponse, first)
if firstResponse.Code != http.StatusCreated {
t.Fatalf("first batched status = %d, body = %s", firstResponse.Code, firstResponse.Body.String())
}
second := multipartUploadRequest(t, "/api/v1/upload", "file", "second.txt", "hello")
second.Header.Set("Accept", "application/json")
second.Header.Set(uploadBatchHeader, "active-cap")
secondResponse := httptest.NewRecorder()
app.Upload(secondResponse, second)
if secondResponse.Code != http.StatusCreated {
t.Fatalf("second batched status = %d, body = %s", secondResponse.Code, secondResponse.Body.String())
}
third := multipartUploadRequest(t, "/api/v1/upload", "file", "third.txt", "hello")
third.Header.Set("Accept", "application/json")
thirdResponse := httptest.NewRecorder()
app.Upload(thirdResponse, third)
if thirdResponse.Code != http.StatusTooManyRequests {
t.Fatalf("non-batched status = %d, body = %s", thirdResponse.Code, thirdResponse.Body.String())
}
}
func TestUserPolicyOverrideChangesUploadEnforcement(t *testing.T) { func TestUserPolicyOverrideChangesUploadEnforcement(t *testing.T) {
app, cleanup := newTestApp(t) app, cleanup := newTestApp(t)
defer cleanup() defer cleanup()

View File

@@ -1004,10 +1004,10 @@ func (a *App) AdminUpdateUserPolicy(w http.ResponseWriter, r *http.Request) {
MaxUploadMB: optionalMB(r.FormValue("max_upload_mb")), MaxUploadMB: optionalMB(r.FormValue("max_upload_mb")),
DailyUploadMB: optionalMB(r.FormValue("daily_upload_mb")), DailyUploadMB: optionalMB(r.FormValue("daily_upload_mb")),
StorageQuotaMB: optionalMBAllowZero(r.FormValue("storage_quota_mb")), StorageQuotaMB: optionalMBAllowZero(r.FormValue("storage_quota_mb")),
MaxDays: optionalInt(r.FormValue("max_days")), MaxDays: optionalIntAllowUnlimited(r.FormValue("max_days")),
DailyBoxes: optionalInt(r.FormValue("daily_boxes")), DailyBoxes: optionalIntAllowUnlimited(r.FormValue("daily_boxes")),
ActiveBoxes: optionalInt(r.FormValue("active_boxes")), ActiveBoxes: optionalIntAllowUnlimited(r.FormValue("active_boxes")),
ShortWindowRequests: optionalInt(r.FormValue("short_window_requests")), ShortWindowRequests: optionalIntAllowUnlimited(r.FormValue("short_window_requests")),
} }
if backendID := r.FormValue("storage_backend_id"); backendID != "" { if backendID := r.FormValue("storage_backend_id"); backendID != "" {
if _, err := a.uploadService.Storage().BackendConfig(backendID); err != nil { if _, err := a.uploadService.Storage().BackendConfig(backendID); err != nil {
@@ -1036,10 +1036,10 @@ func (a *App) AdminUpdateUser(w http.ResponseWriter, r *http.Request) {
MaxUploadMB: optionalMB(r.FormValue("max_upload_mb")), MaxUploadMB: optionalMB(r.FormValue("max_upload_mb")),
DailyUploadMB: optionalMB(r.FormValue("daily_upload_mb")), DailyUploadMB: optionalMB(r.FormValue("daily_upload_mb")),
StorageQuotaMB: optionalMBAllowZero(r.FormValue("storage_quota_mb")), StorageQuotaMB: optionalMBAllowZero(r.FormValue("storage_quota_mb")),
MaxDays: optionalInt(r.FormValue("max_days")), MaxDays: optionalIntAllowUnlimited(r.FormValue("max_days")),
DailyBoxes: optionalInt(r.FormValue("daily_boxes")), DailyBoxes: optionalIntAllowUnlimited(r.FormValue("daily_boxes")),
ActiveBoxes: optionalInt(r.FormValue("active_boxes")), ActiveBoxes: optionalIntAllowUnlimited(r.FormValue("active_boxes")),
ShortWindowRequests: optionalInt(r.FormValue("short_window_requests")), ShortWindowRequests: optionalIntAllowUnlimited(r.FormValue("short_window_requests")),
} }
if backendID := r.FormValue("storage_backend_id"); backendID != "" { if backendID := r.FormValue("storage_backend_id"); backendID != "" {
if _, err := a.uploadService.Storage().BackendConfig(backendID); err != nil { if _, err := a.uploadService.Storage().BackendConfig(backendID); err != nil {
@@ -1206,7 +1206,7 @@ func (a *App) adminBoxes(limit int) ([]adminBoxView, error) {
ID: box.ID, ID: box.ID,
Owner: owner, Owner: owner,
CreatedAt: box.CreatedAt.Format("Jan 2 15:04"), CreatedAt: box.CreatedAt.Format("Jan 2 15:04"),
ExpiresAt: box.ExpiresAt.Format("Jan 2 15:04"), ExpiresAt: boxExpiryLabel(box.ExpiresAt, "Jan 2 15:04"),
FileCount: box.FileCount, FileCount: box.FileCount,
TotalSizeLabel: box.TotalSizeLabel, TotalSizeLabel: box.TotalSizeLabel,
DownloadCount: box.DownloadCount, DownloadCount: box.DownloadCount,
@@ -1296,7 +1296,8 @@ func optionalMBAllowZero(value string) *float64 {
return nil return nil
} }
parsed, err := strconv.ParseFloat(value, 64) parsed, err := strconv.ParseFloat(value, 64)
if err != nil || parsed < 0 { // 0 and -1 both mean unlimited; reject other negatives.
if err != nil || (parsed < 0 && parsed != -1) {
return nil return nil
} }
return &parsed return &parsed
@@ -1313,6 +1314,18 @@ func optionalInt(value string) *int {
return &parsed return &parsed
} }
// optionalIntAllowUnlimited is like optionalInt but also accepts -1 (unlimited).
func optionalIntAllowUnlimited(value string) *int {
if value == "" {
return nil
}
parsed, err := strconv.Atoi(value)
if err != nil || (parsed <= 0 && parsed != -1) {
return nil
}
return &parsed
}
func formatMB(value float64) string { func formatMB(value float64) string {
return strconv.FormatFloat(value, 'f', -1, 64) + " MB" return strconv.FormatFloat(value, 'f', -1, 64) + " MB"
} }

View File

@@ -17,6 +17,7 @@ type apiDocsData struct {
ShareXExampleURL string ShareXExampleURL string
ShareXDownloadURL string ShareXDownloadURL string
ShareXFileFieldName string ShareXFileFieldName string
ShareXGroupWindow string
} }
func (a *App) APIDocs(w http.ResponseWriter, r *http.Request) { func (a *App) APIDocs(w http.ResponseWriter, r *http.Request) {
@@ -33,6 +34,7 @@ func (a *App) APIDocs(w http.ResponseWriter, r *http.Request) {
ShareXExampleURL: a.cfg.BaseURL + "/api/v1/upload", ShareXExampleURL: a.cfg.BaseURL + "/api/v1/upload",
ShareXDownloadURL: a.cfg.BaseURL + "/api/v1/sharex/warpbox-anonymous.sxcu", ShareXDownloadURL: a.cfg.BaseURL + "/api/v1/sharex/warpbox-anonymous.sxcu",
ShareXFileFieldName: "sharex", ShareXFileFieldName: "sharex",
ShareXGroupWindow: uploadGroupWindow.String(),
}, },
}) })
} }
@@ -47,11 +49,16 @@ func (a *App) ShareXAnonymousConfig(w http.ResponseWriter, r *http.Request) {
"RequestURL": a.cfg.BaseURL + "/api/v1/upload", "RequestURL": a.cfg.BaseURL + "/api/v1/upload",
"Headers": map[string]string{ "Headers": map[string]string{
"Accept": "application/json", "Accept": "application/json",
// Group a multi-file selection (sent as back-to-back requests) into
// one box. Remove this header for one box per file.
uploadBatchHeader: "sharex",
}, },
"Body": "MultipartFormData", "Body": "MultipartFormData",
"FileFormName": "sharex", "FileFormName": "sharex",
"URL": "$json:boxUrl$", "URL": "{json:boxUrl}",
"DeletionURL": "$json:manageUrl$", "ThumbnailURL": "{json:thumbnailUrl}",
"DeletionURL": "{json:deleteUrl}",
"ErrorMessage": "{json:error}",
}) })
} }
@@ -112,8 +119,9 @@ func (a *App) UploadResponseSchema(w http.ResponseWriter, r *http.Request) {
"boxId": map[string]any{"type": "string"}, "boxId": map[string]any{"type": "string"},
"boxUrl": map[string]any{"type": "string", "format": "uri"}, "boxUrl": map[string]any{"type": "string", "format": "uri"},
"zipUrl": map[string]any{"type": "string", "format": "uri"}, "zipUrl": map[string]any{"type": "string", "format": "uri"},
"thumbnailUrl": map[string]any{"type": "string", "format": "uri", "description": "Thumbnail of the most recently uploaded file (placeholder until generated)."},
"manageUrl": map[string]any{"type": "string", "format": "uri", "description": "Private bearer URL for managing/deleting this upload. Returned only at upload time."}, "manageUrl": map[string]any{"type": "string", "format": "uri", "description": "Private bearer URL for managing/deleting this upload. Returned only at upload time."},
"deleteUrl": map[string]any{"type": "string", "format": "uri", "description": "Private bearer POST URL for deleting this upload. Returned only at upload time."}, "deleteUrl": map[string]any{"type": "string", "format": "uri", "description": "Private bearer URL for deleting this upload (GET or POST). Returned only at upload time."},
"expiresAt": map[string]any{"type": "string", "format": "date-time"}, "expiresAt": map[string]any{"type": "string", "format": "date-time"},
"files": map[string]any{ "files": map[string]any{
"type": "array", "type": "array",
@@ -125,6 +133,7 @@ func (a *App) UploadResponseSchema(w http.ResponseWriter, r *http.Request) {
"name": map[string]any{"type": "string"}, "name": map[string]any{"type": "string"},
"size": map[string]any{"type": "string"}, "size": map[string]any{"type": "string"},
"url": map[string]any{"type": "string", "format": "uri"}, "url": map[string]any{"type": "string", "format": "uri"},
"thumbnailUrl": map[string]any{"type": "string", "format": "uri"},
}, },
}, },
}, },

View File

@@ -18,6 +18,7 @@ type App struct {
settingsService *services.SettingsService settingsService *services.SettingsService
banService *services.BanService banService *services.BanService
rateLimiter *rateLimiter rateLimiter *rateLimiter
uploadGroups *uploadGrouper
} }
func NewApp(cfg config.Config, logger *slog.Logger, renderer *web.Renderer, uploadService *services.UploadService, authService *services.AuthService, settingsService *services.SettingsService, banService *services.BanService) *App { func NewApp(cfg config.Config, logger *slog.Logger, renderer *web.Renderer, uploadService *services.UploadService, authService *services.AuthService, settingsService *services.SettingsService, banService *services.BanService) *App {
@@ -30,6 +31,7 @@ func NewApp(cfg config.Config, logger *slog.Logger, renderer *web.Renderer, uplo
settingsService: settingsService, settingsService: settingsService,
banService: banService, banService: banService,
rateLimiter: newRateLimiter(), rateLimiter: newRateLimiter(),
uploadGroups: newUploadGrouper(),
} }
} }
@@ -112,6 +114,9 @@ func (a *App) RegisterRoutes(mux *http.ServeMux) {
mux.HandleFunc("GET /d/{boxID}/deleted", a.ManageDeleted) mux.HandleFunc("GET /d/{boxID}/deleted", a.ManageDeleted)
mux.HandleFunc("GET /d/{boxID}/manage/{token}", a.ManageBox) mux.HandleFunc("GET /d/{boxID}/manage/{token}", a.ManageBox)
mux.HandleFunc("POST /d/{boxID}/manage/{token}/delete", a.ManageDeleteBox) mux.HandleFunc("POST /d/{boxID}/manage/{token}/delete", a.ManageDeleteBox)
// GET variant so ShareX (which issues a GET to the configured DeletionURL)
// can delete a box via its secret one-time delete token.
mux.HandleFunc("GET /d/{boxID}/manage/{token}/delete", a.ManageDeleteBox)
mux.HandleFunc("POST /d/{boxID}/unlock", a.UnlockBox) mux.HandleFunc("POST /d/{boxID}/unlock", a.UnlockBox)
mux.HandleFunc("GET /d/{boxID}/zip", a.DownloadZip) mux.HandleFunc("GET /d/{boxID}/zip", a.DownloadZip)
mux.HandleFunc("GET /d/{boxID}/f/{fileID}", a.DownloadFile) mux.HandleFunc("GET /d/{boxID}/f/{fileID}", a.DownloadFile)

View File

@@ -82,7 +82,7 @@ func (a *App) Dashboard(w http.ResponseWriter, r *http.Request) {
FileCount: len(row.Box.Files), FileCount: len(row.Box.Files),
Size: row.TotalSizeLabel, Size: row.TotalSizeLabel,
CreatedAt: row.Box.CreatedAt.Format("Jan 2 15:04"), CreatedAt: row.Box.CreatedAt.Format("Jan 2 15:04"),
ExpiresAt: row.Box.ExpiresAt.Format("Jan 2 15:04"), ExpiresAt: boxExpiryLabel(row.Box.ExpiresAt, "Jan 2 15:04"),
URL: "/d/" + row.Box.ID, URL: "/d/" + row.Box.ID,
}) })
} }

View File

@@ -78,7 +78,7 @@ func (a *App) DownloadPage(w http.ResponseWriter, r *http.Request) {
} }
} }
expiresLabel := box.ExpiresAt.Format("Jan 2, 2006 15:04 MST") expiresLabel := boxExpiryLabel(box.ExpiresAt, "Jan 2, 2006 15:04 MST")
title := "Shared files on Warpbox" title := "Shared files on Warpbox"
description := fmt.Sprintf("%d file%s shared via Warpbox · expires %s", len(box.Files), plural(len(box.Files)), expiresLabel) description := fmt.Sprintf("%d file%s shared via Warpbox · expires %s", len(box.Files), plural(len(box.Files)), expiresLabel)
if locked && box.Obfuscate { if locked && box.Obfuscate {
@@ -337,6 +337,21 @@ func unlockCookieName(boxID string) string {
return "warpbox_unlock_" + strings.NewReplacer("-", "_", ".", "_").Replace(boxID) return "warpbox_unlock_" + strings.NewReplacer("-", "_", ".", "_").Replace(boxID)
} }
// neverExpires reports whether a box's expiry is far enough out to be treated as
// "forever" (set via the unlimited / -1 expiry option).
func neverExpires(t time.Time) bool {
return time.Until(t) > 50*365*24*time.Hour
}
// boxExpiryLabel formats a box's expiry with the given layout, rendering
// "forever" boxes as "Never" instead of a meaningless far-future date.
func boxExpiryLabel(t time.Time, layout string) string {
if neverExpires(t) {
return "Never"
}
return t.Format(layout)
}
func absoluteURL(r *http.Request, path string) string { func absoluteURL(r *http.Request, path string) string {
if strings.HasPrefix(path, "http://") || strings.HasPrefix(path, "https://") { if strings.HasPrefix(path, "http://") || strings.HasPrefix(path, "https://") {
return path return path

View File

@@ -82,7 +82,7 @@ func (a *App) managePageData(box services.Box, token string) managePageData {
Token: token, Token: token,
FileCount: len(box.Files), FileCount: len(box.Files),
TotalSize: helpers.FormatBytes(totalSize), TotalSize: helpers.FormatBytes(totalSize),
ExpiresLabel: box.ExpiresAt.Format("Jan 2, 2006 15:04 MST"), ExpiresLabel: boxExpiryLabel(box.ExpiresAt, "Jan 2, 2006 15:04 MST"),
DownloadCount: box.DownloadCount, DownloadCount: box.DownloadCount,
MaxDownloads: box.MaxDownloads, MaxDownloads: box.MaxDownloads,
Protected: a.uploadService.IsProtected(box), Protected: a.uploadService.IsProtected(box),

View File

@@ -75,6 +75,10 @@ func (a *App) homeExpiryOptions(settings services.UploadPolicySettings, user ser
unlimited = true unlimited = true
case loggedIn: case loggedIn:
maxDays = a.settingsService.EffectivePolicyForUser(settings, user).MaxDays maxDays = a.settingsService.EffectivePolicyForUser(settings, user).MaxDays
// A negative per-user MaxDays override means unlimited retention.
if maxDays < 0 {
unlimited = true
}
} }
return buildExpiryOptions(maxDays, unlimited) return buildExpiryOptions(maxDays, unlimited)
} }
@@ -103,6 +107,10 @@ func buildExpiryOptions(maxDays int, unlimited bool) ([]expiryOption, int) {
if len(options) == 0 { if len(options) == 0 {
options = append(options, expiryOption{Minutes: capMinutes, Label: expiryLabel(capMinutes)}) options = append(options, expiryOption{Minutes: capMinutes, Label: expiryLabel(capMinutes)})
} }
// Unlimited uploaders can pick "never expires" (sentinel -1) after the ladder.
if unlimited {
options = append(options, expiryOption{Minutes: -1, Label: "Unlimited (never expires)"})
}
// Default to 24h when available, otherwise the smallest option offered. // Default to 24h when available, otherwise the smallest option offered.
defaultMinutes := options[0].Minutes defaultMinutes := options[0].Minutes
@@ -154,5 +162,9 @@ func (a *App) homeUploadPolicyLabels(settings services.UploadPolicySettings, use
if policy.StorageQuotaSet { if policy.StorageQuotaSet {
quota = services.FormatMegabytesLabel(policy.StorageQuotaMB) quota = services.FormatMegabytesLabel(policy.StorageQuotaMB)
} }
return maxUpload, "Daily cap: " + services.FormatMegabytesLabel(policy.DailyUploadMB) + " · Storage quota: " + quota + " · " + strconv.Itoa(policy.MaxDays) + " day max." expiryLimit := strconv.Itoa(policy.MaxDays) + " day max."
if policy.MaxDays < 0 {
expiryLimit = "no expiry limit."
}
return maxUpload, "Daily cap: " + services.FormatMegabytesLabel(policy.DailyUploadMB) + " · Storage quota: " + quota + " · " + expiryLimit
} }

View File

@@ -36,7 +36,7 @@ func (a *App) Upload(w http.ResponseWriter, r *http.Request) {
} }
effectivePolicy := a.effectiveUploadPolicy(settings, user, loggedIn) effectivePolicy := a.effectiveUploadPolicy(settings, user, loggedIn)
rateKey := uploadRateKey(r, user, loggedIn) rateKey := uploadRateKey(r, user, loggedIn)
if !isAdminUpload && !a.rateLimiter.Allow("upload:"+rateKey, effectivePolicy.ShortRequests, effectivePolicy.ShortWindow, time.Now().UTC()) { if !isAdminUpload && effectivePolicy.ShortRequests > 0 && !a.rateLimiter.Allow("upload:"+rateKey, effectivePolicy.ShortRequests, effectivePolicy.ShortWindow, time.Now().UTC()) {
a.logger.Warn("upload rate limited", "source", "user-upload", "severity", "warn", "code", 4290, "ip", uploadClientIP(r), "user_id", user.ID) a.logger.Warn("upload rate limited", "source", "user-upload", "severity", "warn", "code", 4290, "ip", uploadClientIP(r), "user_id", user.ID)
helpers.WriteJSONError(w, http.StatusTooManyRequests, "too many upload requests, please slow down") helpers.WriteJSONError(w, http.StatusTooManyRequests, "too many upload requests, please slow down")
return return
@@ -77,22 +77,39 @@ func (a *App) Upload(w http.ResponseWriter, r *http.Request) {
return return
} }
} }
maxDays := parseInt(r.FormValue("max_days")) // Unlimited expiry: admins, or users whose effective MaxDays is negative.
unlimitedExpiry := isAdminUpload || effectivePolicy.MaxDays < 0
rawMaxDays := parseInt(r.FormValue("max_days"))
maxDays := rawMaxDays
if maxDays <= 0 { if maxDays <= 0 {
maxDays = min(7, effectivePolicy.MaxDays) maxDays = 7
if effectivePolicy.MaxDays > 0 && effectivePolicy.MaxDays < maxDays {
maxDays = effectivePolicy.MaxDays
} }
if !isAdminUpload && maxDays > effectivePolicy.MaxDays { }
if !unlimitedExpiry && maxDays > effectivePolicy.MaxDays {
a.logger.Warn("upload rejected expiration days", "source", "user-upload", "severity", "warn", "code", 4131, "ip", uploadClientIP(r), "user_id", user.ID, "requested_days", maxDays, "max_days", effectivePolicy.MaxDays) a.logger.Warn("upload rejected expiration days", "source", "user-upload", "severity", "warn", "code", 4131, "ip", uploadClientIP(r), "user_id", user.ID, "requested_days", maxDays, "max_days", effectivePolicy.MaxDays)
helpers.WriteJSONError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("expiration cannot exceed %d days", effectivePolicy.MaxDays)) helpers.WriteJSONError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("expiration cannot exceed %d days", effectivePolicy.MaxDays))
return return
} }
expiresMinutes := parseInt(r.FormValue("expires_minutes")) expiresMinutes := parseInt(r.FormValue("expires_minutes"))
if expiresMinutes > 0 && !isAdminUpload && expiresMinutes > effectivePolicy.MaxDays*24*60 { // A negative expires_minutes (or max_days) is the "never expires" request.
// Only honour it for unlimited uploaders; otherwise it's an invalid value.
if expiresMinutes < 0 || rawMaxDays < 0 {
if !unlimitedExpiry {
a.logger.Warn("upload rejected unlimited expiration", "source", "user-upload", "severity", "warn", "code", 4133, "ip", uploadClientIP(r), "user_id", user.ID)
helpers.WriteJSONError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("expiration cannot exceed %d days", effectivePolicy.MaxDays))
return
}
expiresMinutes = -1
} else if expiresMinutes > 0 && !unlimitedExpiry && expiresMinutes > effectivePolicy.MaxDays*24*60 {
a.logger.Warn("upload rejected expiration minutes", "source", "user-upload", "severity", "warn", "code", 4132, "ip", uploadClientIP(r), "user_id", user.ID, "requested_minutes", expiresMinutes, "max_days", effectivePolicy.MaxDays) a.logger.Warn("upload rejected expiration minutes", "source", "user-upload", "severity", "warn", "code", 4132, "ip", uploadClientIP(r), "user_id", user.ID, "requested_minutes", expiresMinutes, "max_days", effectivePolicy.MaxDays)
helpers.WriteJSONError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("expiration cannot exceed %d days", effectivePolicy.MaxDays)) helpers.WriteJSONError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("expiration cannot exceed %d days", effectivePolicy.MaxDays))
return return
} }
result, err := a.uploadService.CreateBox(files, services.UploadOptions{ opts := services.UploadOptions{
MaxDays: maxDays, MaxDays: maxDays,
ExpiresInMinutes: expiresMinutes, ExpiresInMinutes: expiresMinutes,
MaxDownloads: parseInt(r.FormValue("max_downloads")), MaxDownloads: parseInt(r.FormValue("max_downloads")),
@@ -103,14 +120,20 @@ func (a *App) Upload(w http.ResponseWriter, r *http.Request) {
SkipSizeLimit: isAdminUpload || effectivePolicy.MaxUploadMB < 0, SkipSizeLimit: isAdminUpload || effectivePolicy.MaxUploadMB < 0,
CreatorIP: uploadClientIP(r), CreatorIP: uploadClientIP(r),
StorageBackendID: effectivePolicy.StorageBackendID, StorageBackendID: effectivePolicy.StorageBackendID,
}) }
result, boxesAdded, status, policyMessage, err := a.createOrAppendBox(r, user, loggedIn, effectivePolicy, files, opts, !isAdminUpload)
if policyMessage != "" {
a.logger.Warn("upload rejected by policy", "source", "quota", "severity", "warn", "code", status, "ip", uploadClientIP(r), "user_id", user.ID, "message", policyMessage, "bytes", totalBytes, "files", len(files))
helpers.WriteJSONError(w, status, policyMessage)
return
}
if err != nil { if err != nil {
a.logger.Warn("upload failed", "source", "user-upload", "severity", "warn", "code", 4001, "ip", uploadClientIP(r), "user_id", user.ID, "error", err.Error()) a.logger.Warn("upload failed", "source", "user-upload", "severity", "warn", "code", 4001, "ip", uploadClientIP(r), "user_id", user.ID, "error", err.Error())
helpers.WriteJSONError(w, http.StatusBadRequest, err.Error()) helpers.WriteJSONError(w, http.StatusBadRequest, err.Error())
return return
} }
if !isAdminUpload { if !isAdminUpload {
if err := a.recordUploadUsage(r, user, loggedIn, totalBytes, 1); err != nil { if err := a.recordUploadUsage(r, user, loggedIn, totalBytes, boxesAdded); err != nil {
a.logger.Warn("failed to record upload usage", "source", "quota", "severity", "warn", "code", 4402, "error", err.Error()) a.logger.Warn("failed to record upload usage", "source", "quota", "severity", "warn", "code", 4402, "error", err.Error())
} }
if err := a.settingsService.CleanupUsage(time.Now().UTC(), settings.UsageRetentionDays); err != nil { if err := a.settingsService.CleanupUsage(time.Now().UTC(), settings.UsageRetentionDays); err != nil {
@@ -130,6 +153,77 @@ func (a *App) Upload(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintln(w, result.BoxURL) _, _ = fmt.Fprintln(w, result.BoxURL)
} }
// createOrAppendBox creates a new box. It only ever appends to an existing box
// when the request opts in via the X-Warpbox-Batch header: requests sharing the
// same batch value (per account, or per IP for anonymous) within
// uploadGroupWindow are folded into one box. Without the header the behaviour is
// identical to creating a fresh box every time. Returns the result and how many
// boxes were created (1 for a new box, 0 for an append) for usage accounting.
func (a *App) createOrAppendBox(r *http.Request, user services.User, loggedIn bool, policy services.EffectiveUploadPolicy, files []*multipart.FileHeader, opts services.UploadOptions, enforceBoxLimits bool) (services.UploadResult, int, int, string, error) {
batch := strings.TrimSpace(r.Header.Get(uploadBatchHeader))
if batch == "" {
if enforceBoxLimits {
if status, message := a.checkBoxCreationPolicy(r, user, loggedIn, policy); message != "" {
return services.UploadResult{}, 0, status, message, nil
}
}
result, err := a.uploadService.CreateBox(files, opts)
if err != nil {
return services.UploadResult{}, 0, 0, "", err
}
return result, 1, 0, "", nil
}
// Group key is scoped to the uploader so batches never cross accounts/IPs.
identity := "ip:" + uploadClientIP(r)
if loggedIn {
identity = "user:" + user.ID
}
entry := a.uploadGroups.entryFor(identity + "|" + batch)
// Hold the per-key lock across the whole create/append so concurrent batched
// uploads serialise into the same box instead of racing.
entry.mu.Lock()
defer entry.mu.Unlock()
if entry.boxID != "" && time.Since(entry.at) < uploadGroupWindow {
if box, err := a.uploadService.GetBox(entry.boxID); err == nil && a.batchBoxMatches(box, user, loggedIn, r) && a.uploadService.CanDownload(box) == nil {
if result, err := a.uploadService.AppendFiles(entry.boxID, files, opts); err == nil {
// Re-attach the manage/delete URLs from the box's creation so every
// upload in the batch returns a working deletion URL.
result.ManageURL = entry.manageURL
result.DeleteURL = entry.deleteURL
entry.at = time.Now()
return result, 0, 0, "", nil
}
}
}
if enforceBoxLimits {
if status, message := a.checkBoxCreationPolicy(r, user, loggedIn, policy); message != "" {
return services.UploadResult{}, 0, status, message, nil
}
}
result, err := a.uploadService.CreateBox(files, opts)
if err != nil {
return services.UploadResult{}, 0, 0, "", err
}
entry.boxID = result.BoxID
entry.manageURL = result.ManageURL
entry.deleteURL = result.DeleteURL
entry.at = time.Now()
return result, 1, 0, "", nil
}
// batchBoxMatches guards that a batched append only ever touches a box owned by
// the same uploader (account for logged-in users, creator IP for anonymous).
func (a *App) batchBoxMatches(box services.Box, user services.User, loggedIn bool, r *http.Request) bool {
if loggedIn {
return box.OwnerID == user.ID
}
return box.OwnerID == "" && box.CreatorIP == uploadClientIP(r)
}
func (a *App) checkUploadPolicy(r *http.Request, user services.User, loggedIn bool, settings services.UploadPolicySettings, policy services.EffectiveUploadPolicy, files []*multipart.FileHeader, totalBytes int64) (int, string) { func (a *App) checkUploadPolicy(r *http.Request, user services.User, loggedIn bool, settings services.UploadPolicySettings, policy services.EffectiveUploadPolicy, files []*multipart.FileHeader, totalBytes int64) (int, string) {
if len(files) == 0 { if len(files) == 0 {
return 0, "" return 0, ""
@@ -151,16 +245,6 @@ func (a *App) checkUploadPolicy(r *http.Request, user services.User, loggedIn bo
if policy.DailyUploadMB > 0 && usage.UploadedBytes+totalBytes > services.MegabytesToBytes(policy.DailyUploadMB) { if policy.DailyUploadMB > 0 && usage.UploadedBytes+totalBytes > services.MegabytesToBytes(policy.DailyUploadMB) {
return http.StatusTooManyRequests, "anonymous daily upload limit reached" return http.StatusTooManyRequests, "anonymous daily upload limit reached"
} }
if usage.UploadedBoxes+1 > policy.DailyBoxes {
return http.StatusTooManyRequests, "anonymous daily box limit reached"
}
activeBoxes, err := a.uploadService.ActiveBoxCountForIP(uploadClientIP(r))
if err != nil {
return http.StatusInternalServerError, "active box limit could not be checked"
}
if activeBoxes+1 > policy.ActiveBoxes {
return http.StatusTooManyRequests, "anonymous active box limit reached"
}
if status, message := a.checkStorageBackendCapacity(policy.StorageBackendID, settings, totalBytes); message != "" { if status, message := a.checkStorageBackendCapacity(policy.StorageBackendID, settings, totalBytes); message != "" {
return status, message return status, message
} }
@@ -174,16 +258,6 @@ func (a *App) checkUploadPolicy(r *http.Request, user services.User, loggedIn bo
if policy.DailyUploadMB > 0 && usage.UploadedBytes+totalBytes > services.MegabytesToBytes(policy.DailyUploadMB) { if policy.DailyUploadMB > 0 && usage.UploadedBytes+totalBytes > services.MegabytesToBytes(policy.DailyUploadMB) {
return http.StatusTooManyRequests, "daily upload limit reached" return http.StatusTooManyRequests, "daily upload limit reached"
} }
if usage.UploadedBoxes+1 > policy.DailyBoxes {
return http.StatusTooManyRequests, "daily box limit reached"
}
activeBoxes, err := a.uploadService.ActiveBoxCountForUser(user.ID)
if err != nil {
return http.StatusInternalServerError, "active box limit could not be checked"
}
if activeBoxes+1 > policy.ActiveBoxes {
return http.StatusTooManyRequests, "active box limit reached"
}
activeStorage, err := a.uploadService.UserActiveStorageUsed(user.ID) activeStorage, err := a.uploadService.UserActiveStorageUsed(user.ID)
if err != nil { if err != nil {
return http.StatusInternalServerError, "storage quota could not be checked" return http.StatusInternalServerError, "storage quota could not be checked"
@@ -197,6 +271,42 @@ func (a *App) checkUploadPolicy(r *http.Request, user services.User, loggedIn bo
return 0, "" return 0, ""
} }
func (a *App) checkBoxCreationPolicy(r *http.Request, user services.User, loggedIn bool, policy services.EffectiveUploadPolicy) (int, string) {
now := time.Now().UTC()
if !loggedIn {
usage, err := a.settingsService.UsageForIP(uploadClientIP(r), now)
if err != nil {
return http.StatusInternalServerError, "upload usage could not be checked"
}
if policy.DailyBoxes > 0 && usage.UploadedBoxes+1 > policy.DailyBoxes {
return http.StatusTooManyRequests, "anonymous daily box limit reached"
}
activeBoxes, err := a.uploadService.ActiveBoxCountForIP(uploadClientIP(r))
if err != nil {
return http.StatusInternalServerError, "active box limit could not be checked"
}
if policy.ActiveBoxes > 0 && activeBoxes+1 > policy.ActiveBoxes {
return http.StatusTooManyRequests, "anonymous active box limit reached"
}
return 0, ""
}
usage, err := a.settingsService.UsageForUser(user.ID, now)
if err != nil {
return http.StatusInternalServerError, "upload usage could not be checked"
}
if policy.DailyBoxes > 0 && usage.UploadedBoxes+1 > policy.DailyBoxes {
return http.StatusTooManyRequests, "daily box limit reached"
}
activeBoxes, err := a.uploadService.ActiveBoxCountForUser(user.ID)
if err != nil {
return http.StatusInternalServerError, "active box limit could not be checked"
}
if policy.ActiveBoxes > 0 && activeBoxes+1 > policy.ActiveBoxes {
return http.StatusTooManyRequests, "active box limit reached"
}
return 0, ""
}
func (a *App) recordUploadUsage(r *http.Request, user services.User, loggedIn bool, totalBytes int64, boxes int) error { func (a *App) recordUploadUsage(r *http.Request, user services.User, loggedIn bool, totalBytes int64, boxes int) error {
now := time.Now().UTC() now := time.Now().UTC()
if loggedIn { if loggedIn {

View File

@@ -0,0 +1,76 @@
package handlers
import (
"sync"
"time"
)
// uploadGroupWindow is how long after a batched upload a follow-up upload with
// the same X-Warpbox-Batch value (and same account/IP) is folded into the same
// box. ShareX sends a multi-file selection as separate back-to-back requests;
// the batch header lets it land them in one box.
const uploadGroupWindow = 20 * time.Second
// uploadBatchHeader is the opt-in request header. Without it, uploads behave
// exactly as before (one box per request). With it, requests sharing the same
// value (per account/IP) within uploadGroupWindow are grouped into one box.
const uploadBatchHeader = "X-Warpbox-Batch"
// uploadGroupPruneInterval is how often entryFor drops stale entries so the map
// can't grow without bound (one key per account/IP + batch value otherwise).
const uploadGroupPruneInterval = 5 * time.Minute
// uploadGrouper tracks the most recent box per batch key so opt-in batched
// uploads land in a single box. Each key has its own lock, which also serialises
// that key's concurrent uploads so they append to the same box instead of racing
// to create several.
type uploadGrouper struct {
mu sync.Mutex
entries map[string]*uploadGroupEntry
lastPrune time.Time
}
type uploadGroupEntry struct {
mu sync.Mutex
boxID string
manageURL string
deleteURL string
at time.Time
}
func newUploadGrouper() *uploadGrouper {
return &uploadGrouper{entries: make(map[string]*uploadGroupEntry)}
}
func (g *uploadGrouper) entryFor(key string) *uploadGroupEntry {
g.mu.Lock()
defer g.mu.Unlock()
g.pruneLocked(time.Now())
entry, ok := g.entries[key]
if !ok {
entry = &uploadGroupEntry{at: time.Now()}
g.entries[key] = entry
}
return entry
}
// pruneLocked drops entries whose last use is well past the grouping window so
// the map stays bounded to recently-active keys. Callers must hold g.mu. Entries
// currently in use are kept to avoid removing one a request is about to
// populate.
func (g *uploadGrouper) pruneLocked(now time.Time) {
if now.Sub(g.lastPrune) < uploadGroupPruneInterval {
return
}
g.lastPrune = now
for key, entry := range g.entries {
if !entry.mu.TryLock() {
continue
}
stale := now.Sub(entry.at) > 2*uploadGroupWindow
entry.mu.Unlock()
if stale {
delete(g.entries, key)
}
}
}

View File

@@ -0,0 +1,24 @@
package handlers
import (
"testing"
"time"
)
func TestUploadGroupPrunesFailedEntries(t *testing.T) {
g := newUploadGrouper()
entry := g.entryFor("ip:203.0.113.1|failed")
entry.mu.Lock()
entry.at = time.Now().Add(-3 * uploadGroupWindow)
entry.mu.Unlock()
g.lastPrune = time.Now().Add(-uploadGroupPruneInterval)
_ = g.entryFor("ip:203.0.113.1|next")
if _, ok := g.entries["ip:203.0.113.1|failed"]; ok {
t.Fatalf("stale failed entry was not pruned")
}
if _, ok := g.entries["ip:203.0.113.1|next"]; !ok {
t.Fatalf("new entry was not created")
}
}

View File

@@ -26,10 +26,20 @@ func Bans(logger *slog.Logger, bans *services.BanService, trustedProxies []strin
return return
} }
settings, err := bans.Settings()
if err != nil {
logger.Error("ban settings load failed", "source", "ban", "severity", "error", "code", 5004, "ip", ip, "error", err.Error())
next.ServeHTTP(w, r)
return
}
if !settings.AutoBanEnabled {
next.ServeHTTP(w, r)
return
}
if pattern, err := bans.MaliciousPattern(r.URL.Path); err != nil { if pattern, err := bans.MaliciousPattern(r.URL.Path); err != nil {
logger.Error("malicious path check failed", "source", "ban", "severity", "error", "code", 5002, "ip", ip, "error", err.Error()) logger.Error("malicious path check failed", "source", "ban", "severity", "error", "code", 5002, "ip", ip, "error", err.Error())
} else if pattern != "" { } else if pattern != "" {
if result, err := bans.RecordAbuse(ip, services.AbuseKindMaliciousPath, r.URL.Path, banThreshold(bans, services.AbuseKindMaliciousPath), now); err != nil { if result, err := bans.RecordAbuse(ip, services.AbuseKindMaliciousPath, r.URL.Path, settings.MaliciousPathThreshold, now); err != nil {
logger.Error("malicious path event failed", "source", "ban", "severity", "error", "code", 5003, "ip", ip, "path", r.URL.Path, "error", err.Error()) logger.Error("malicious path event failed", "source", "ban", "severity", "error", "code", 5003, "ip", ip, "path", r.URL.Path, "error", err.Error())
} else if result.Enabled { } else if result.Enabled {
logger.Warn("malicious path requested", "source", "ban", "severity", "warn", "code", 4302, "ip", ip, "path", r.URL.Path, "pattern", pattern, "count", result.Event.Count) logger.Warn("malicious path requested", "source", "ban", "severity", "warn", "code", 4302, "ip", ip, "path", r.URL.Path, "pattern", pattern, "count", result.Event.Count)
@@ -48,18 +58,3 @@ func Bans(logger *slog.Logger, bans *services.BanService, trustedProxies []strin
}) })
} }
} }
func banThreshold(bans *services.BanService, kind string) int {
settings, err := bans.Settings()
if err != nil {
return 0
}
switch kind {
case services.AbuseKindAdminLogin:
return settings.AdminLoginFailureThreshold
case services.AbuseKindUserLogin:
return settings.UserLoginFailureThreshold
default:
return settings.MaliciousPathThreshold
}
}

View File

@@ -79,6 +79,26 @@ func TestBansMiddlewareAutoBansMaliciousPaths(t *testing.T) {
} }
} }
func TestBansMiddlewareSkipsAutoBanWhenDisabled(t *testing.T) {
bans := newMiddlewareBanService(t)
handler := Chain(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
}), Bans(slog.New(slog.NewTextHandler(io.Discard, nil)), bans, nil))
for i := 0; i < 5; i++ {
request := httptest.NewRequest(http.MethodGet, "/.env", nil)
request.RemoteAddr = "203.0.113.23:6070"
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code == http.StatusForbidden {
t.Fatalf("request %d was blocked while auto-ban disabled", i+1)
}
}
if _, ok, err := bans.Match("203.0.113.23", time.Now().UTC()); err != nil || ok {
t.Fatalf("disabled auto-ban Match = %v, %v", ok, err)
}
}
func newMiddlewareBanService(t *testing.T) *services.BanService { func newMiddlewareBanService(t *testing.T) *services.BanService {
t.Helper() t.Helper()
root := t.TempDir() root := t.TempDir()

View File

@@ -862,20 +862,20 @@ func validateUserPolicy(policy UserPolicy) error {
if policy.DailyUploadMB != nil && ((*policy.DailyUploadMB < 0 && *policy.DailyUploadMB != -1) || *policy.DailyUploadMB == 0) { if policy.DailyUploadMB != nil && ((*policy.DailyUploadMB < 0 && *policy.DailyUploadMB != -1) || *policy.DailyUploadMB == 0) {
return fmt.Errorf("daily upload override must be positive or -1 for unlimited") return fmt.Errorf("daily upload override must be positive or -1 for unlimited")
} }
if policy.StorageQuotaMB != nil && *policy.StorageQuotaMB < 0 { if policy.StorageQuotaMB != nil && *policy.StorageQuotaMB < 0 && *policy.StorageQuotaMB != -1 {
return fmt.Errorf("storage quota override cannot be negative") return fmt.Errorf("storage quota override must be 0/positive or -1 for unlimited")
} }
if policy.MaxDays != nil && *policy.MaxDays <= 0 { if policy.MaxDays != nil && *policy.MaxDays <= 0 && *policy.MaxDays != -1 {
return fmt.Errorf("expiration override must be positive") return fmt.Errorf("expiration override must be positive or -1 for unlimited")
} }
if policy.DailyBoxes != nil && *policy.DailyBoxes <= 0 { if policy.DailyBoxes != nil && *policy.DailyBoxes <= 0 && *policy.DailyBoxes != -1 {
return fmt.Errorf("daily box override must be positive") return fmt.Errorf("daily box override must be positive or -1 for unlimited")
} }
if policy.ActiveBoxes != nil && *policy.ActiveBoxes <= 0 { if policy.ActiveBoxes != nil && *policy.ActiveBoxes <= 0 && *policy.ActiveBoxes != -1 {
return fmt.Errorf("active box override must be positive") return fmt.Errorf("active box override must be positive or -1 for unlimited")
} }
if policy.ShortWindowRequests != nil && *policy.ShortWindowRequests <= 0 { if policy.ShortWindowRequests != nil && *policy.ShortWindowRequests <= 0 && *policy.ShortWindowRequests != -1 {
return fmt.Errorf("short-window request override must be positive") return fmt.Errorf("short-window request override must be positive or -1 for unlimited")
} }
return nil return nil
} }

View File

@@ -312,7 +312,11 @@ func (s *BanService) Match(ip string, now time.Time) (MatchedBan, bool, error) {
} }
now = now.UTC() now = now.UTC()
var matched BanRecord var matched BanRecord
err := s.db.Update(func(tx *bbolt.Tx) error { var matchedKey []byte
// Read-only scan first: the common case (no match) only takes a concurrent
// read transaction, instead of grabbing the single bbolt write lock on every
// request that flows through the ban middleware.
err := s.db.View(func(tx *bbolt.Tx) error {
bucket := tx.Bucket(bansBucket) bucket := tx.Bucket(bansBucket)
return bucket.ForEach(func(key, value []byte) error { return bucket.ForEach(func(key, value []byte) error {
if matched.ID != "" { if matched.ID != "" {
@@ -325,20 +329,37 @@ func (s *BanService) Match(ip string, now time.Time) (MatchedBan, bool, error) {
if !record.Active(now) || !banTargetMatches(record.Normalized, parsed) { if !record.Active(now) || !banTargetMatches(record.Normalized, parsed) {
return nil return nil
} }
matched = record
matchedKey = append([]byte(nil), key...) // key bytes are only valid within the txn
return nil
})
})
if err != nil || matched.ID == "" {
return MatchedBan{Ban: matched, IP: ip}, matched.ID != "", err
}
// On a hit, record the match time in a short write transaction.
matched.LastMatchedAt = &now
matched.UpdatedAt = now
_ = s.db.Update(func(tx *bbolt.Tx) error {
bucket := tx.Bucket(bansBucket)
data := bucket.Get(matchedKey)
if data == nil {
return nil
}
var record BanRecord
if err := json.Unmarshal(data, &record); err != nil {
return nil
}
record.LastMatchedAt = &now record.LastMatchedAt = &now
record.UpdatedAt = now record.UpdatedAt = now
next, err := json.Marshal(record) next, err := json.Marshal(record)
if err != nil { if err != nil {
return err
}
if err := bucket.Put(key, next); err != nil {
return err
}
matched = record
return nil return nil
}
return bucket.Put(matchedKey, next)
}) })
}) return MatchedBan{Ban: matched, IP: ip}, true, nil
return MatchedBan{Ban: matched, IP: ip}, matched.ID != "", err
} }
func (r BanRecord) Active(now time.Time) bool { func (r BanRecord) Active(now time.Time) bool {
@@ -498,6 +519,11 @@ func (s *BanService) RecordAbuse(ip, kind, detail string, threshold int, now tim
if err != nil || !triggered { if err != nil || !triggered {
return AbuseResult{Event: event, Triggered: false, Enabled: true}, err return AbuseResult{Event: event, Triggered: false, Enabled: true}, err
} }
if matched, ok, err := s.Match(ip, now); err != nil {
return AbuseResult{}, err
} else if ok {
return AbuseResult{Event: event, Ban: matched.Ban, Triggered: true, Enabled: true}, nil
}
reason := fmt.Sprintf("%s threshold reached: %s", strings.ReplaceAll(kind, "_", " "), detail) reason := fmt.Sprintf("%s threshold reached: %s", strings.ReplaceAll(kind, "_", " "), detail)
ban, err = s.createBan(ip, reason, BanSourceAuto, "", now.Add(time.Duration(settings.AutoBanDurationHours)*time.Hour), now) ban, err = s.createBan(ip, reason, BanSourceAuto, "", now.Add(time.Duration(settings.AutoBanDurationHours)*time.Hour), now)
if err != nil { if err != nil {

View File

@@ -79,6 +79,17 @@ func TestBanServiceAutoBanThresholdsAndDisabled(t *testing.T) {
if err != nil || !result.Triggered || result.Ban.ID == "" { if err != nil || !result.Triggered || result.Ban.ID == "" {
t.Fatalf("RecordAbuse threshold = %+v, %v", result, err) t.Fatalf("RecordAbuse threshold = %+v, %v", result, err)
} }
again, err := bans.RecordAbuse("203.0.113.8", AbuseKindMaliciousPath, "/.env", 3, now.Add(4*time.Minute))
if err != nil || !again.Triggered || again.Ban.ID != result.Ban.ID {
t.Fatalf("RecordAbuse duplicate = %+v, %v", again, err)
}
records, err := bans.ListBans()
if err != nil {
t.Fatalf("ListBans returned error: %v", err)
}
if len(records) != 1 {
t.Fatalf("ban count = %d, want 1", len(records))
}
} }
func TestBanServiceMaliciousPathRules(t *testing.T) { func TestBanServiceMaliciousPathRules(t *testing.T) {

View File

@@ -85,6 +85,7 @@ type UploadResult struct {
BoxID string `json:"boxId"` BoxID string `json:"boxId"`
BoxURL string `json:"boxUrl"` BoxURL string `json:"boxUrl"`
ZipURL string `json:"zipUrl"` ZipURL string `json:"zipUrl"`
ThumbnailURL string `json:"thumbnailUrl"`
ManageURL string `json:"manageUrl"` ManageURL string `json:"manageUrl"`
DeleteURL string `json:"deleteUrl"` DeleteURL string `json:"deleteUrl"`
ExpiresAt string `json:"expiresAt"` ExpiresAt string `json:"expiresAt"`
@@ -96,6 +97,7 @@ type ResultFile struct {
Name string `json:"name"` Name string `json:"name"`
Size string `json:"size"` Size string `json:"size"`
URL string `json:"url"` URL string `json:"url"`
ThumbnailURL string `json:"thumbnailUrl"`
} }
type AdminStats struct { type AdminStats struct {
@@ -196,14 +198,21 @@ func (s *UploadService) CreateBox(files []*multipart.FileHeader, opts UploadOpti
if len(files) == 0 { if len(files) == 0 {
return UploadResult{}, fmt.Errorf("no files were uploaded") return UploadResult{}, fmt.Errorf("no files were uploaded")
} }
if opts.MaxDays <= 0 {
opts.MaxDays = 7
}
now := time.Now().UTC() now := time.Now().UTC()
expiresAt := now.Add(time.Duration(opts.MaxDays) * 24 * time.Hour) var expiresAt time.Time
if opts.ExpiresInMinutes > 0 { switch {
case opts.ExpiresInMinutes < 0 || opts.MaxDays < 0:
// "Forever" — a date far enough out that the box effectively never
// expires. No schema change; CanDownload/cleanup keep working as-is.
expiresAt = now.AddDate(100, 0, 0)
case opts.ExpiresInMinutes > 0:
expiresAt = now.Add(time.Duration(opts.ExpiresInMinutes) * time.Minute) expiresAt = now.Add(time.Duration(opts.ExpiresInMinutes) * time.Minute)
default:
days := opts.MaxDays
if days <= 0 {
days = 7
}
expiresAt = now.Add(time.Duration(days) * 24 * time.Hour)
} }
box := Box{ box := Box{
@@ -226,15 +235,66 @@ func (s *UploadService) CreateBox(files []*multipart.FileHeader, opts UploadOpti
box.PasswordHash = hash box.PasswordHash = hash
} }
backend, err := s.storage.Backend(box.StorageBackendID) if err := s.writeFilesToBox(&box, files, opts); err != nil {
return UploadResult{}, err
}
if err := s.SaveBox(box); err != nil {
return UploadResult{}, err
}
s.logger.Info("upload complete",
"source", "user-upload",
"severity", "user_activity",
"code", 2001,
"box_id", box.ID,
"file_count", len(box.Files),
)
return s.resultForBox(box, deleteToken), nil
}
// AppendFiles adds files to an existing box (used to group a ShareX multi-file
// selection into a single box). The box keeps its original expiry, password and
// other settings; only the new files are written.
func (s *UploadService) AppendFiles(boxID string, files []*multipart.FileHeader, opts UploadOptions) (UploadResult, error) {
if len(files) == 0 {
return UploadResult{}, fmt.Errorf("no files were uploaded")
}
box, err := s.GetBox(boxID)
if err != nil { if err != nil {
return UploadResult{}, err return UploadResult{}, err
} }
if err := s.writeFilesToBox(&box, files, opts); err != nil {
return UploadResult{}, err
}
if err := s.SaveBox(box); err != nil {
return UploadResult{}, err
}
s.logger.Info("upload appended",
"source", "user-upload",
"severity", "user_activity",
"code", 2001,
"box_id", box.ID,
"added", len(files),
"file_count", len(box.Files),
)
return s.resultForBox(box, ""), nil
}
// writeFilesToBox streams each uploaded file into the box's storage backend and
// appends the file metadata to box.Files. The box's StorageBackendID determines
// where files land, so it works for both new and existing boxes.
func (s *UploadService) writeFilesToBox(box *Box, files []*multipart.FileHeader, opts UploadOptions) error {
backend, err := s.storage.Backend(box.StorageBackendID)
if err != nil {
return err
}
for _, header := range files { for _, header := range files {
if !opts.SkipSizeLimit { if !opts.SkipSizeLimit {
if err := s.ValidateSize(header.Size); err != nil { if err := s.ValidateSize(header.Size); err != nil {
return UploadResult{}, err return err
} }
} }
@@ -245,7 +305,7 @@ func (s *UploadService) CreateBox(files []*multipart.FileHeader, opts UploadOpti
file, err := header.Open() file, err := header.Open()
if err != nil { if err != nil {
return UploadResult{}, err return err
} }
fileID := randomID(8) fileID := randomID(8)
@@ -263,7 +323,7 @@ func (s *UploadService) CreateBox(files []*multipart.FileHeader, opts UploadOpti
if err := s.writeUploadedObject(context.Background(), backend, objectKey, file, header.Size, maxSize, contentType); err != nil { if err := s.writeUploadedObject(context.Background(), backend, objectKey, file, header.Size, maxSize, contentType); err != nil {
file.Close() file.Close()
return UploadResult{}, err return err
} }
file.Close() file.Close()
@@ -278,20 +338,7 @@ func (s *UploadService) CreateBox(files []*multipart.FileHeader, opts UploadOpti
UploadedAt: time.Now().UTC(), UploadedAt: time.Now().UTC(),
}) })
} }
return nil
if err := s.SaveBox(box); err != nil {
return UploadResult{}, err
}
s.logger.Info("upload complete",
"source", "user-upload",
"severity", "user_activity",
"code", 2001,
"box_id", box.ID,
"file_count", len(box.Files),
)
return s.resultForBox(box, deleteToken), nil
} }
func (s *UploadService) GetBox(id string) (Box, error) { func (s *UploadService) GetBox(id string) (Box, error) {
@@ -730,13 +777,22 @@ func (s *UploadService) resultForBox(box Box, deleteToken string) UploadResult {
Name: file.Name, Name: file.Name,
Size: helpers.FormatBytes(file.Size), Size: helpers.FormatBytes(file.Size),
URL: fmt.Sprintf("%s/d/%s/f/%s", s.baseURL, box.ID, file.ID), URL: fmt.Sprintf("%s/d/%s/f/%s", s.baseURL, box.ID, file.ID),
ThumbnailURL: fmt.Sprintf("%s/d/%s/thumb/%s", s.baseURL, box.ID, file.ID),
}) })
} }
// The box-level thumbnail points at the most recently added file, so a
// per-file ShareX upload previews the file it just sent.
thumbnailURL := fmt.Sprintf("%s/d/%s/og-image.jpg", s.baseURL, box.ID)
if len(files) > 0 {
thumbnailURL = files[len(files)-1].ThumbnailURL
}
result := UploadResult{ result := UploadResult{
BoxID: box.ID, BoxID: box.ID,
BoxURL: fmt.Sprintf("%s/d/%s", s.baseURL, box.ID), BoxURL: fmt.Sprintf("%s/d/%s", s.baseURL, box.ID),
ZipURL: fmt.Sprintf("%s/d/%s/zip", s.baseURL, box.ID), ZipURL: fmt.Sprintf("%s/d/%s/zip", s.baseURL, box.ID),
ThumbnailURL: thumbnailURL,
ExpiresAt: box.ExpiresAt.Format(time.RFC3339), ExpiresAt: box.ExpiresAt.Format(time.RFC3339),
Files: files, Files: files,
} }

Binary file not shown.

After

Width:  |  Height:  |  Size: 423 B

View File

@@ -36,6 +36,22 @@
font-size: 1rem; font-size: 1rem;
} }
.docs-card h3 {
margin: 1.35rem 0 0;
font-size: 0.9rem;
font-weight: 650;
color: var(--foreground);
}
/* Highlights where the API token goes in the ShareX config snippet. */
.sxcu-highlight {
background: #fde047;
color: #1a1a1a;
font-weight: 700;
padding: 0 0.2rem;
border-radius: 3px;
}
.docs-card p { .docs-card p {
margin: 0.65rem 0 0; margin: 0.65rem 0 0;
color: var(--muted-foreground); color: var(--muted-foreground);

View File

@@ -60,7 +60,7 @@
</main> </main>
<footer class="site-footer"> <footer class="site-footer">
<span>{{.AppName}} · {{.AppVersion}} · {{.CurrentYear}} · self-hosted</span> <span>{{.AppName}} · {{.AppVersion}} · {{.CurrentYear}}</span>
<label class="theme-picker"> <label class="theme-picker">
<span>Theme</span> <span>Theme</span>
<select data-theme-select aria-label="Site theme"> <select data-theme-select aria-label="Site theme">

View File

@@ -52,7 +52,7 @@
<div class="table-header"> <div class="table-header">
<div> <div>
<h2>Identity and limits</h2> <h2>Identity and limits</h2>
<p>Blank limit fields inherit the global user defaults. Use -1 for unlimited upload size or daily upload caps. Storage quota set to 0 means unlimited.</p> <p>Blank limit fields inherit the global user defaults. Use <code>-1</code> for unlimited in any limit field — upload size, daily caps, storage quota, max expiration (the box can then last forever), daily boxes, active boxes, and short-window requests. Storage quota <code>0</code> also means unlimited.</p>
</div> </div>
</div> </div>
<form class="settings-form" action="/admin/users/{{.Data.UserEdit.ID}}/edit" method="post"> <form class="settings-form" action="/admin/users/{{.Data.UserEdit.ID}}/edit" method="post">
@@ -89,10 +89,10 @@
<h3 class="settings-section-title">Upload limits</h3> <h3 class="settings-section-title">Upload limits</h3>
<label><span>Max upload size (MB)</span><input name="max_upload_mb" value="{{.Data.UserEdit.MaxUploadMB}}" placeholder="inherit"></label> <label><span>Max upload size (MB)</span><input name="max_upload_mb" value="{{.Data.UserEdit.MaxUploadMB}}" placeholder="inherit"></label>
<label><span>Daily upload cap (MB)</span><input name="daily_upload_mb" value="{{.Data.UserEdit.DailyUploadMB}}" placeholder="inherit"></label> <label><span>Daily upload cap (MB)</span><input name="daily_upload_mb" value="{{.Data.UserEdit.DailyUploadMB}}" placeholder="inherit"></label>
<label><span>Max expiration (days)</span><input type="number" min="1" name="max_days" value="{{.Data.UserEdit.MaxDays}}" placeholder="inherit"></label> <label><span>Max expiration (days)</span><input type="number" min="-1" name="max_days" value="{{.Data.UserEdit.MaxDays}}" placeholder="inherit"></label>
<label><span>Daily boxes</span><input type="number" min="1" name="daily_boxes" value="{{.Data.UserEdit.DailyBoxes}}" placeholder="inherit"></label> <label><span>Daily boxes</span><input type="number" min="-1" name="daily_boxes" value="{{.Data.UserEdit.DailyBoxes}}" placeholder="inherit"></label>
<label><span>Active boxes</span><input type="number" min="1" name="active_boxes" value="{{.Data.UserEdit.ActiveBoxes}}" placeholder="inherit"></label> <label><span>Active boxes</span><input type="number" min="-1" name="active_boxes" value="{{.Data.UserEdit.ActiveBoxes}}" placeholder="inherit"></label>
<label><span>Short-window requests</span><input type="number" min="1" name="short_window_requests" value="{{.Data.UserEdit.ShortWindowRequests}}" placeholder="inherit"></label> <label><span>Short-window requests</span><input type="number" min="-1" name="short_window_requests" value="{{.Data.UserEdit.ShortWindowRequests}}" placeholder="inherit"></label>
</div> </div>
<button class="button button-primary" type="submit">Save user</button> <button class="button button-primary" type="submit">Save user</button>

View File

@@ -36,11 +36,12 @@
<article class="card docs-card"> <article class="card docs-card">
<div class="card-content"> <div class="card-content">
<h2>JSON response</h2> <h2>JSON response</h2>
<p>The raw delete token is returned only once inside <code>manageUrl</code> and <code>deleteUrl</code>. Keep those links private.</p> <p>The raw delete token is returned only once inside <code>manageUrl</code> and <code>deleteUrl</code>. Keep those links private. On error the body is <code>{ "error": "message" }</code> with a non-2xx status (e.g. rate limited or over a limit).</p>
<pre><code>{ <pre><code>{
"boxId": "abc123", "boxId": "abc123",
"boxUrl": "{{.Data.BaseURL}}/d/abc123", "boxUrl": "{{.Data.BaseURL}}/d/abc123",
"zipUrl": "{{.Data.BaseURL}}/d/abc123/zip", "zipUrl": "{{.Data.BaseURL}}/d/abc123/zip",
"thumbnailUrl": "{{.Data.BaseURL}}/d/abc123/thumb/file123",
"manageUrl": "{{.Data.BaseURL}}/d/abc123/manage/private-token", "manageUrl": "{{.Data.BaseURL}}/d/abc123/manage/private-token",
"deleteUrl": "{{.Data.BaseURL}}/d/abc123/manage/private-token/delete", "deleteUrl": "{{.Data.BaseURL}}/d/abc123/manage/private-token/delete",
"expiresAt": "2026-06-05T12:00:00Z", "expiresAt": "2026-06-05T12:00:00Z",
@@ -49,7 +50,8 @@
"id": "file123", "id": "file123",
"name": "report.pdf", "name": "report.pdf",
"size": "2.4 MiB", "size": "2.4 MiB",
"url": "{{.Data.BaseURL}}/d/abc123/f/file123" "url": "{{.Data.BaseURL}}/d/abc123/f/file123",
"thumbnailUrl": "{{.Data.BaseURL}}/d/abc123/thumb/file123"
} }
] ]
}</code></pre> }</code></pre>
@@ -59,22 +61,44 @@
<article class="card docs-card"> <article class="card docs-card">
<div class="card-content"> <div class="card-content">
<h2>ShareX setup</h2> <h2>ShareX setup</h2>
<p>Import the uploader, then add your API key to upload as your account — with your account's size, daily, and retention limits — instead of as an anonymous guest.</p>
<h3>1 · Import the uploader</h3>
<ol class="docs-steps"> <ol class="docs-steps">
<li>Download the instance config: <a href="/api/v1/sharex/warpbox-anonymous.sxcu"><code>/api/v1/sharex/warpbox-anonymous.sxcu</code></a>.</li> <li>Download <a href="/api/v1/sharex/warpbox-anonymous.sxcu"><code>warpbox-anonymous.sxcu</code></a>.</li>
<li>Or open the tracked template at <code>{{.Data.ShareXExamplePath}}</code> and change <code>RequestURL</code> to <code>{{.Data.ShareXExampleURL}}</code>.</li> <li>In ShareX: <code>Destinations → Custom uploader settings → Import → From file</code>, then pick the <code>.sxcu</code>.</li>
<li>Keep <code>FileFormName</code> as <code>{{.Data.ShareXFileFieldName}}</code>.</li>
<li>Import the <code>.sxcu</code> file into ShareX as a custom uploader.</li>
<li>Upload a file. ShareX will use <code>boxUrl</code> as the public URL and <code>manageUrl</code> as the deletion URL.</li>
</ol> </ol>
<h3>2 · Add your API key (upload as your account)</h3>
<ol class="docs-steps">
<li>Create a personal access token under <a href="/account/settings">Account → Access tokens</a> and copy it.</li>
<li>In <code>Custom uploader settings</code>, select the Warpbox uploader and open the <code>Headers</code> section.</li>
<li>Add a header — Name <code>Authorization</code>, Value <code>Bearer &lt;your token&gt;</code>.</li>
</ol>
<p class="muted-copy">Without that header, uploads stay anonymous. With it, they're attributed to your account and use your account's limits.</p>
<pre><code>{ <pre><code>{
"Version": "1.0.0",
"Name": "Warpbox (my account)",
"DestinationType": "ImageUploader, FileUploader, TextUploader",
"RequestMethod": "POST", "RequestMethod": "POST",
"RequestURL": "{{.Data.ShareXExampleURL}}", "RequestURL": "{{.Data.ShareXExampleURL}}",
"Headers": { "Accept": "application/json" }, "Headers": {
"Accept": "application/json",
"Authorization": "Bearer <span class="sxcu-highlight">YOUR_API_TOKEN</span>",
"X-Warpbox-Batch": "sharex"
},
"Body": "MultipartFormData", "Body": "MultipartFormData",
"FileFormName": "{{.Data.ShareXFileFieldName}}", "FileFormName": "{{.Data.ShareXFileFieldName}}",
"URL": "$json:boxUrl$", "URL": "{json:boxUrl}",
"DeletionURL": "$json:manageUrl$" "ThumbnailURL": "{json:thumbnailUrl}",
"DeletionURL": "{json:deleteUrl}",
"ErrorMessage": "{json:error}"
}</code></pre> }</code></pre>
<h3>Grouping multiple files into one box</h3>
<p>Grouping is <strong>opt-in via the <code>X-Warpbox-Batch</code> request header</strong> — without it, every file becomes its own box (the default). When the header is present, uploads sharing the same value (per account, or per IP for anonymous) within {{.Data.ShareXGroupWindow}} of each other are added to the <strong>same</strong> box, so a multi-file ShareX selection produces one shareable link instead of one per file. The shipped config sets <code>X-Warpbox-Batch: sharex</code>; remove that header for one box per file.</p>
<p class="muted-copy">The response also exposes <code>{json:thumbnailUrl}</code> for ShareX previews, <code>{json:deleteUrl}</code> for the deletion URL, and <code>{json:error}</code> so ShareX surfaces messages like rate limiting.</p>
</div> </div>
</article> </article>

View File

@@ -5,10 +5,13 @@
"RequestMethod": "POST", "RequestMethod": "POST",
"RequestURL": "https://warpbox.dev/api/v1/upload", "RequestURL": "https://warpbox.dev/api/v1/upload",
"Headers": { "Headers": {
"Accept": "application/json" "Accept": "application/json",
"X-Warpbox-Batch": "sharex"
}, },
"Body": "MultipartFormData", "Body": "MultipartFormData",
"FileFormName": "sharex", "FileFormName": "sharex",
"URL": "$json:boxUrl$", "URL": "{json:boxUrl}",
"DeletionURL": "$json:manageUrl$" "ThumbnailURL": "{json:thumbnailUrl}",
"DeletionURL": "{json:deleteUrl}",
"ErrorMessage": "{json:error}"
} }