darren.maverick/fm_be
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init push

Browse Source
This commit is contained in:
darren
2026-07-16 22:16:45 +07:00
commit 8b068bdb10
1021 changed files with 332816 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

435
README.md Normal file
View File

@@ -0,0 +1,435 @@
# Wucher Backend
Wucher is a Go backend built with GoFiber. The runtime uses MySQL for persistence and transient auth state, Amazon SQS for asynchronous email jobs, and Amazon SES v2 for email delivery.
## Prerequisites
- Go 1.25+
- MySQL
- LocalStack or AWS SQS/SES access for queue testing
## Project Structure
- `cmd/api/` API bootstrap
- `cmd/worker/` SQS worker bootstrap
- `internal/app/` application wiring
- `internal/resilience/` circuit breaker executor and dependency error classifiers
- `internal/service/` business logic
- `internal/transport/http/` GoFiber handlers and middleware
- `internal/repository/mysql/` MySQL repositories and transient stores
- `internal/repository/sqs/` SQS publisher/consumer adapter
- `internal/queue/` queue contracts, serializer, retry policy, and worker loop
## Runtime Architecture
- HTTP handlers stay thin and delegate to services.
- Auth and other business writes persist to MySQL first.
- API/service code never sends email synchronously.
- Email jobs are written to `email_outbox_messages` when outbox is enabled, or published directly to SQS when outbox is disabled.
- Worker publishes outbox rows to the SQS send queue, consumes SQS messages with long polling, and sends email through Amazon SES v2.
- SES bounce, complaint, and delivery tracking should use a separate flow:
- SES Configuration Set
- SNS topic subscription
- dedicated SQS event queue
- Circuit breakers are applied only at outbound dependency boundaries:
- Microsoft Entra token exchange
- SES delivery
- SQS publish/consume/ack calls
- Transient auth state previously kept in Redis is now stored in MySQL tables:
- `auth_runtime_tokens`
- `queue_idempotency_records`
`gorm` auto-migration creates these tables on startup.
## Getting Started
Install dependencies:
```bash
go mod tidy
```
Create local config from the example:
```bash
cp .env.example .env
```
Run the API:
```bash
go run ./cmd/api
```
Run the worker:
```bash
go run ./cmd/worker
```
Worker monitor endpoints:
```bash
curl http://127.0.0.1:9090/health
curl http://127.0.0.1:9090/metrics
curl http://127.0.0.1:9091/health
curl http://127.0.0.1:9091/metrics
```
- `/metrics` exposes Prometheus format for API and worker processes.
- `/health` remains JSON health/worker snapshot.
## Environment Variables
Example values are in `.env` for local development.
Important groups:
- Email delivery
- `EMAIL_PROVIDER`
- `EMAIL_MOCK_DIR`
- MySQL
- `MYSQL_DSN`
- `MYSQL_MAX_OPEN_CONNS`
- `MYSQL_MAX_IDLE_CONNS`
- `MYSQL_CONN_MAX_LIFETIME`
- `MYSQL_CONN_MAX_IDLE_TIME`
- Auth
- `AUTH_EMAIL_VERIFY_URL`
- `AUTH_PASSWORD_RESET_URL`
- `AUTH_SECURITY_PIN_RESET_URL`
- `AUTH_TOTP_CHALLENGE_TTL`
- `AUTH_SSO_STATE_TTL`
- `AUTH_JWT_ACCESS_SECRET`
- `AUTH_JWT_REFRESH_SECRET`
- `AUTH_WEBAUTHN_*`
- Queue / worker
- `QUEUE_MESSAGE_SCHEMA_VERSION`
- `QUEUE_ENABLE_LEGACY_PAYLOAD_FALLBACK`
- `QUEUE_IDEMPOTENCY_KEY_PREFIX`
- `QUEUE_IDEMPOTENCY_TTL`
- `QUEUE_PROCESSING_TTL`
- `QUEUE_WORKER_CONCURRENCY`
- `QUEUE_WORKER_MAX_BATCH_SIZE`
- `QUEUE_WORKER_PROCESS_TIMEOUT`
- `QUEUE_WORKER_BACKOFF_MIN`
- `QUEUE_WORKER_BACKOFF_MAX`
- `QUEUE_WORKER_PERMANENT_FAILURE_DELAY`
- `QUEUE_OUTBOX_ENABLED`
- `QUEUE_OUTBOX_POLL_INTERVAL`
- `QUEUE_OUTBOX_BATCH_SIZE`
- `QUEUE_OUTBOX_DISPATCH_TIMEOUT`
- `QUEUE_OUTBOX_LOCK_TTL`
- `QUEUE_OUTBOX_MAX_ATTEMPTS`
- File queue / file worker
- `FILE_QUEUE_MESSAGE_SCHEMA_VERSION`
- `FILE_QUEUE_IDEMPOTENCY_KEY_PREFIX`
- `FILE_QUEUE_IDEMPOTENCY_TTL`
- `FILE_QUEUE_PROCESSING_TTL`
- `FILE_QUEUE_WORKER_CONCURRENCY`
- `FILE_QUEUE_WORKER_MAX_BATCH_SIZE`
- `FILE_QUEUE_WORKER_PROCESS_TIMEOUT`
- `FILE_QUEUE_WORKER_SHUTDOWN_TIMEOUT`
- `FILE_QUEUE_WORKER_BACKOFF_MIN`
- `FILE_QUEUE_WORKER_BACKOFF_MAX`
- `FILE_QUEUE_WORKER_PERMANENT_FAILURE_DELAY`
- `FILE_QUEUE_WORKER_MONITOR_ADDR`
- `FILE_QUEUE_WORKER_HEALTH_ERROR_THRESHOLD`
- `FILE_QUEUE_WORKER_DEPTH_POLL_INTERVAL`
- `FILE_QUEUE_OUTBOX_ENABLED`
- `FILE_QUEUE_OUTBOX_POLL_INTERVAL`
- `FILE_QUEUE_OUTBOX_BATCH_SIZE`
- `FILE_QUEUE_OUTBOX_DISPATCH_TIMEOUT`
- `FILE_QUEUE_OUTBOX_LOCK_TTL`
- `FILE_QUEUE_OUTBOX_MAX_ATTEMPTS`
- Circuit breakers
- `CB_DEFAULT_ENABLED`
- `CB_DEFAULT_MAX_REQUESTS`
- `CB_DEFAULT_INTERVAL`
- `CB_DEFAULT_BUCKET_PERIOD`
- `CB_DEFAULT_TIMEOUT`
- `CB_DEFAULT_MIN_REQUESTS`
- `CB_DEFAULT_FAILURE_RATIO`
- `CB_DEFAULT_CONSECUTIVE_FAILURES`
- `CB_MICROSOFT_SSO_ENABLED`
- `CB_SES_ENABLED`
- `CB_SQS_ENABLED`
- AWS / SES
- `AWS_REGION`
- `SES_REGION`
- `SES_FROM`
- `SES_CONFIGURATION_SET`
- `SES_SEND_TIMEOUT`
- `SES_ENDPOINT`
- `SES_ALLOW_INSECURE_ENDPOINT`
- AWS / SQS
- Email queue:
- `AWS_REGION`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `SQS_ENDPOINT`
- `SQS_ALLOW_INSECURE_ENDPOINT`
- `SQS_QUEUE_URL`
- `SQS_QUEUE_TYPE`
- `SQS_MESSAGE_GROUP_ID`
- `SQS_MAX_MESSAGES`
- `SQS_WAIT_TIME_SECONDS`
- `SQS_VISIBILITY_TIMEOUT`
- File queue:
- `FILE_SQS_REGION`
- `FILE_SQS_ENDPOINT`
- `FILE_SQS_QUEUE_URL`
- `FILE_SQS_QUEUE_TYPE`
- `FILE_SQS_MESSAGE_GROUP_ID`
- `FILE_SQS_MAX_MESSAGES`
- `FILE_SQS_WAIT_TIME_SECONDS`
- `FILE_SQS_VISIBILITY_TIMEOUT`
- `FILE_SQS_ALLOW_INSECURE_ENDPOINT`
- File storage (S3)
- `FILE_STORAGE_PROVIDER`
- `FILE_S3_BUCKET`
- `FILE_S3_REGION`
- `FILE_S3_ENDPOINT`
- `FILE_S3_FORCE_PATH_STYLE`
- `FILE_S3_ALLOW_INSECURE_ENDPOINT`
- `FILE_S3_AUTO_CREATE_BUCKET`
- `FILE_OBJECT_KEY_PREFIX`
- `FILE_S3_PRESIGN_PUT_TTL`
- `FILE_S3_PRESIGN_GET_TTL`
- WOPI / Collabora
- `WOPI_TOKEN_SECRET`
- `WOPI_PROOF_SECRET`
- `WOPI_DISCOVERY_URL`
- `WOPI_PUBLIC_BASE_URL`
- `WOPI_EDIT_ACTION_URL`
- `WOPI_TOKEN_TTL`
- `WOPI_MAX_BODY_BYTES`
- `WOPI_REQUIRE_PROOF`
- Worker mode selector
- `WORKER_JOB=email`
- `WORKER_JOB=file_processing`
## WOPI / Collabora Hardening
- Keep WOPI endpoints behind the dedicated `/wopi` router and do not expose them through the session-authenticated app middleware chain.
- Use HTTPS end to end in production.
- Restrict network access to the Collabora host or private network allowlist.
- Set Collabora `aliasgroup`/host configuration so it can only call back to this backend.
- Keep `WOPI_TOKEN_SECRET` and `WOPI_PROOF_SECRET` in env or a secrets manager and rotate them periodically.
- Scope IAM to the minimum S3 bucket/prefix needed by file storage.
- Keep direct S3 URLs disabled for editor access; the backend should proxy file bytes.
- Ensure backend logs capture open/save/exit-save events for auditability.
- Verify `/hosting/discovery` reachability and S3 connectivity in health checks before enabling Collabora traffic.
Per-dependency overrides are supported with the same shape as the default block:
- `CB_MICROSOFT_SSO_*`
- `CB_SES_*`
- `CB_SQS_*`
Example: `CB_SES_TIMEOUT=45s` or `CB_SQS_CONSECUTIVE_FAILURES=3`.
## Circuit Breakers
- Breakers are scoped per outbound dependency, not per HTTP endpoint and not global middleware.
- MySQL repositories and internal business logic are intentionally not wrapped.
- Only dependency-health failures count toward opening the breaker.
- Microsoft SSO: network errors, timeouts, HTTP `429`, and `5xx`
- SES: network errors, throttling, and AWS server faults
- SQS: network errors, throttling, and AWS server faults
- Client/config/input errors are excluded so they do not open the breaker.
- State transitions are logged through the standard application logger.
## Queue Behavior
- Long polling is enabled through `SQS_WAIT_TIME_SECONDS`.
- Messages are deleted only after successful processing or explicit ack of duplicates.
- Transient failures use `ChangeMessageVisibility` with backoff.
- Permanent failures are left unacked so SQS redrive policy can move them to the DLQ.
- The default producer schema is `v2`; legacy payload fallback remains available for older messages.
- Configure DLQ on the queue itself in AWS/LocalStack. The application no longer publishes directly to a DLQ URL.
## Prometheus Metrics
Scrape endpoints:
- API: `http://<api-host>:8080/metrics`
- Email worker monitor: `http://<worker-host>:9090/metrics`
- File worker monitor: `http://<worker-host>:9091/metrics`
Example Prometheus scrape config:
```yaml
scrape_configs:
- job_name: "wucher-api"
static_configs:
- targets: ["127.0.0.1:8080"]
- job_name: "wucher-email-worker"
static_configs:
- targets: ["127.0.0.1:9090"]
- job_name: "wucher-file-worker"
static_configs:
- targets: ["127.0.0.1:9091"]
```
Metrics and labels:
- `app_aws_requests_total{component,service,operation,result}`
- `app_aws_request_duration_seconds{component,service,operation}`
- `app_aws_errors_total{component,service,operation,error_class}` where `error_class` is `4xx`, `5xx`, or `other`
- `app_aws_transferred_bytes_total{component,service,operation,direction}`
- `app_queue_messages_received_total{component,queue_name}`
- `app_queue_messages_deleted_total{component,queue_name}`
- `app_queue_visibility_changes_total{component,queue_name,result}`
- `app_queue_send_total{component,queue_name,result}`
- `app_queue_retries_total{component,queue_name,result}`
- `app_queue_message_bytes_total{component,queue_name,direction}`
- `app_queue_depth{component,queue_name}`
- `app_worker_jobs_processed_total{component,job_type,result}`
- `app_worker_job_duration_seconds{component,job_type}`
- `app_email_send_total{component,provider,result}`
- `app_email_send_duration_seconds{component,provider}`
- `app_file_upload_total{component,result}`
- `app_file_delete_total{component,result}`
- `app_file_head_total{component,result}`
- `app_file_presign_download_total{component,result}`
PromQL examples:
- Error rate per AWS operation:
```promql
sum by (component, service, operation) (rate(app_aws_requests_total{result="error"}[5m]))
/
clamp_min(sum by (component, service, operation) (rate(app_aws_requests_total[5m])), 1e-9)
```
- P95 latency per AWS operation:
```promql
histogram_quantile(
0.95,
sum by (le, component, service, operation) (
rate(app_aws_request_duration_seconds_bucket[5m])
)
)
```
- Queue depth:
```promql
max by (component, queue_name) (app_queue_depth)
```
- Worker success/failure count:
```promql
sum by (component, job_type, result) (increase(app_worker_jobs_processed_total[15m]))
```
- SES send failure rate:
```promql
sum by (component) (rate(app_email_send_total{provider="ses", result="error"}[5m]))
/
clamp_min(sum by (component) (rate(app_email_send_total{provider="ses"}[5m])), 1e-9)
```
- S3 4xx/5xx error rate:
```promql
sum by (operation, error_class) (rate(app_aws_errors_total{service="s3"}[5m]))
```
- S3 GetObject transferred bytes (estimated from presign download flow):
```promql
sum by (component) (rate(app_aws_transferred_bytes_total{service="s3",operation="getobject",direction="download"}[5m]))
```
- SQS message bytes in/out:
```promql
sum by (component, queue_name, direction) (rate(app_queue_message_bytes_total[5m]))
```
- SQS retry/requeue count:
```promql
sum by (component, queue_name, result) (increase(app_queue_retries_total[15m]))
```
## File Manager
- Queue separation:
- Email queue: `app-queue` (+ `app-queue-dlq`)
- File queue: `app-file-queue` (+ `app-file-queue-dlq`)
- `FILE_SQS_QUEUE_URL` must be different from `SQS_QUEUE_URL`.
- API startup validates queue separation and refuses startup on misconfiguration.
- Upload writes file metadata + file outbox durably; file queue publish is handled asynchronously by the file outbox dispatcher.
Local run checklist:
```bash
# 1) Start localstack (queue + bucket bootstrap script will run)
cd ../localstack && docker compose up -d
WORKER_JOB=file_processing go run ./cmd/worker
```
## SES Delivery Notes
- `EMAIL_PROVIDER=mock` is the recommended local/dev default when you do not have real AWS SES access.
- Mock delivery writes one JSON file per delivered email into `EMAIL_MOCK_DIR`.
- `SES_FROM` must be a verified identity or domain in Amazon SES.
- `SES_ENDPOINT` should be blank in production and only overridden for local/dev stacks such as LocalStack.
- Do not point SES events to the main send queue. Use `SES_CONFIGURATION_SET` with SNS -> separate SQS event queue.
- The codebase includes an SES event parser for SNS-wrapped notifications, but an event consumer is not started by default.
## Migration Notes
Detailed migration notes are in [docs/queue-sqs-migration.md](docs/queue-sqs-migration.md).
Data migration tracking wiki is in [docs/data-migration-wiki.md](docs/data-migration-wiki.md).
Main breaking changes:
- Redis runtime support has been removed from the codebase.
- Redis env vars are obsolete and should be removed from deployment config.
- SMTP runtime env/config has been removed in favor of SES.
- Worker now relies on MySQL for queue idempotency and auth/runtime state persistence.
- SQS must have a redrive policy and DLQ configured outside the application.
## CI / Build Selection
The pipeline always builds and pushes the `api` image on every push to `development` or `staging`.
To also build and push the worker or file-worker image, include the corresponding tag anywhere in the commit message:
| Tag | Image built | Docker target |
|---|---|---|
| `#PUSH-WORKER` | `mybitlab/wfm:<tag>-worker` | `worker` |
| `#PUSH-FILE-WORKER` | `mybitlab/wfm:<tag>-file-worker` | `file-worker` |
Both tags can appear in the same commit message to build all three in one pipeline run:
```
ci: deploy all services #PUSH-WORKER #PUSH-FILE-WORKER
```
Omitting a tag skips that image entirely — it will not be built or pushed.
## Testing
Run the full test suite:
```bash
go test ./...
```
Generate coverage:
```bash
go test ./... -coverprofile=coverage.out
go tool cover -func=coverage.out
go tool cover -html=coverage.out -o coverage.html
```