Files
2026-02-04 20:23:48 +01:00
..

API Gateway - ToolsPlatform Backend

Central API gateway for the ToolsPlatform project, handling authentication, file uploads, user management, and payment webhooks.

Architecture Overview

Client Request
    ↓
Fastify Server (port 4000)
    ↓
Middleware Pipeline:
    → CORS + Helmet (Security)
    → Rate Limiting (Redis)
    → Authentication (Keycloak JWT)
    → User Loading (Database)
    → Tier Checking (FREE/PREMIUM)
    ↓
Route Handlers
    ↓
Services:
    → User Service (Postgres)
    → Storage Service (MinIO)
    → Subscription Service (Postgres)
    → Feature Flag Service (ENV + Postgres)

Tech Stack

  • Runtime: Node.js 20.x + TypeScript 5.x
  • Framework: Fastify 5.x
  • Database: PostgreSQL (via Prisma ORM)
  • Cache/Queue: Redis (ioredis + BullMQ)
  • Storage: MinIO (S3-compatible)
  • Auth: Keycloak (JWT with JWKS)
  • Logging: Pino (structured JSON logging)
  • Validation: Zod (runtime type validation)

Project Structure

backend/
├── src/
│   ├── config/          # Configuration loaders (database, redis, minio)
│   ├── middleware/      # Fastify middleware (auth, tier checking, file size)
│   ├── services/        # Business logic (user, storage, subscription)
│   ├── routes/          # API endpoints (health, user, upload, webhooks)
│   ├── types/           # TypeScript types and interfaces
│   ├── utils/           # Utilities (errors, logger, validation, hash)
│   ├── plugins/         # (Future: extracted plugins)
│   ├── app.ts           # Fastify application builder
│   └── index.ts         # Entry point
├── prisma/
│   └── schema.prisma    # Database schema
├── tests/
│   ├── integration/     # Integration tests
│   └── unit/            # Unit tests
├── dist/                # Compiled JavaScript (generated)
├── package.json
├── tsconfig.json
└── README.md

Prerequisites

Before running the API Gateway, ensure these services are running (from Phase 2):

  • PostgreSQL 15+
  • Redis 7+
  • MinIO
  • Keycloak 23+

Start them via Docker Compose from the project root:

cd ..
docker-compose up -d postgres redis minio keycloak

Environment Setup

Important: Environment files live at the PROJECT ROOT (one level up from backend/).

  1. Copy the single template file:

    # From project root
    cp .env.example .env.development
    
  2. Configure Keycloak:

    • Access Keycloak at http://localhost:8180
    • Create realm: toolsplatform
    • Create client: api-gateway
    • Get client secret and update .env.development
  3. Configure Database:

    • Database URL should match Docker Compose settings
    • Default: postgresql://toolsuser:toolspass@localhost:5432/toolsdb

Installation

# Install dependencies
npm install

# Generate Prisma client
npx prisma generate

# Run database migrations
npm run db:migrate

# (Optional) Seed database
npm run db:seed

Development Commands

# Start development server (hot reload)
npm run dev

# Build for production
npm run build

# Start production server
npm start

# Database commands
npm run db:migrate       # Run migrations
npm run db:push          # Push schema changes (dev only)
npm run db:seed          # Seed database
npm run db:studio        # Open Prisma Studio

# Testing (future)
npm test

API Endpoints

Health Monitoring

  • GET /health - Basic health check
  • GET /health/detailed - Detailed dependency health check

User Management (Authenticated)

  • GET /api/v1/user/profile - Get current user profile
  • GET /api/v1/user/limits - Get tier-specific limits

File Uploads

  • POST /api/v1/upload - Upload file (authenticated, tier-based limits)
  • POST /api/v1/upload/anonymous - Upload file (anonymous, 15MB limit)

Payment Webhooks

  • POST /api/v1/webhooks/paddle - Paddle Billing webhooks (transactions, subscriptions)

Documentation

  • GET /docs - Swagger UI (OpenAPI documentation). Optional: set SWAGGER_ENABLED=false to disable; SWAGGER_ADMIN_ONLY=true (default) restricts access to admin users (Bearer token or ?token=... in browser).

Authentication

The API uses JWT tokens from Keycloak for authentication.

Getting a token:

  1. Authenticate with Keycloak at http://localhost:8180/realms/toolsplatform/protocol/openid-connect/token
  2. Include token in requests: Authorization: Bearer <token>

Example:

# Get token
TOKEN=$(curl -X POST "http://localhost:8180/realms/toolsplatform/protocol/openid-connect/token" \
  -d "client_id=api-gateway" \
  -d "client_secret=YOUR_SECRET" \
  -d "username=user@example.com" \
  -d "password=password" \
  -d "grant_type=password" | jq -r '.access_token')

# Use token
curl -H "Authorization: Bearer $TOKEN" http://localhost:4000/api/v1/user/profile

Tier System

Users are assigned one of two tiers:

  • FREE: 15MB file uploads, ads enabled, single file processing
  • PREMIUM: 200MB file uploads, no ads, batch processing, priority queue

Tiers are synced from Keycloak roles (premium-user role = PREMIUM tier).

Feature Flags

Feature flags control monetization and rollout:

Environment-based (simple toggles in .env):

  • FEATURE_ADS_ENABLED
  • FEATURE_PAYMENTS_ENABLED
  • FEATURE_PREMIUM_TOOLS_ENABLED
  • FEATURE_REGISTRATION_ENABLED

Database-based (complex targeting):

  • User-specific targeting
  • Tier-specific targeting
  • Rollout percentage control

Error Handling

All errors include a requestId for support tracking:

{
  "error": "Forbidden",
  "message": "This feature requires a Premium subscription",
  "requestId": "abc-123-def-456",
  "upgradeUrl": "/pricing"
}

Common error codes:

  • 401 Unauthorized - Missing/invalid JWT token
  • 403 Forbidden - Insufficient permissions (tier restriction)
  • 413 Payload Too Large - File exceeds size limit
  • 429 Too Many Requests - Rate limit exceeded
  • 503 Service Unavailable - Feature disabled or dependency down

Logging

Structured JSON logs via Pino:

{
  "level": "info",
  "time": "2026-01-26T10:30:00.000Z",
  "requestId": "abc-123",
  "method": "POST",
  "url": "/api/v1/upload",
  "statusCode": 200,
  "responseTime": "125ms",
  "userId": "user-uuid",
  "msg": "Request completed"
}

Log levels:

  • debug - Development only, verbose output
  • info - Request/response logs, service operations
  • warn - Rate limit warnings, degraded service
  • error - Errors, exceptions, failures

Rate Limiting

Redis-backed token bucket algorithm:

  • Limit: 100 requests per minute per client
  • Key: User ID (authenticated) or IP address (anonymous)
  • Response: 429 Too Many Requests with Retry-After header

Security

  • Helmet: Security headers (CSP, HSTS, X-Frame-Options)
  • CORS: Configurable origins (dev: all, prod: specific)
  • Rate Limiting: Abuse prevention
  • JWT Validation: JWKS-based RS256 signature verification
  • Input Sanitization: Filename and user input sanitization
  • IP Hashing: Privacy-preserving anonymous tracking

Performance

  • Connection Pooling: Prisma connection pool
  • Redis Caching: Rate limit state, session data
  • Multipart Streaming: Efficient file uploads
  • Lazy User Sync: Database writes only on first login or tier change

Monitoring

Health Checks:

# Quick check
curl http://localhost:4000/health

# Detailed check (tests all dependencies)
curl http://localhost:4000/health/detailed

Metrics (future):

  • Request count by endpoint
  • Response time percentiles
  • Error rates
  • Rate limit violations
  • File upload sizes

Troubleshooting

Server won't start

  • Check all environment variables are set
  • Verify database connection: npm run db:push
  • Check Docker services are running: docker ps

Authentication fails

  • Verify Keycloak is accessible
  • Check client secret in .env.development
  • Test token manually: See "Authentication" section

File uploads fail

  • Check MinIO is running: docker ps | grep minio
  • Verify bucket exists: Access MinIO console at http://localhost:9001
  • Check file size limits for your tier

Rate limit issues

  • Redis must be running: docker ps | grep redis
  • Rate limit: 100 req/min per client
  • Use different IP or wait 1 minute

Development Tips

  1. Use Prisma Studio for database inspection:

    npm run db:studio
    
  2. Test with Swagger UI at http://localhost:4000/docs

  3. Monitor logs in development:

    npm run dev | grep ERROR
    
  4. Reset database:

    npm run db:push -- --force-reset
    npm run db:seed
    

Next Steps

  • Run integration tests (Phase 6)
  • Deploy to staging environment
  • Set up monitoring and alerts
  • Configure production environment variables

Support

For issues or questions, check:

  1. Logs with requestId for error tracking
  2. Health endpoint for dependency status
  3. Swagger docs for API reference
  4. Quickstart guide for setup help