docs: add README.md, update .gitignore, include design and implementation docs

2026-04-02 16:10:43 +08:00
parent 85346ce805
commit ccf76fea95
4 changed files with 4024 additions and 0 deletions
--- a/docs/superpowers/specs/2026-04-02-agent-fox-design.md
+++ b/docs/superpowers/specs/2026-04-02-agent-fox-design.md
@@ -0,0 +1,335 @@
+# Agent Fox - API Documentation MCP Service
+
+## Context
+
+Developers using LLMs (Claude, GPT, etc.) often need to reference API documentation while coding. Currently, they either paste entire API docs into the context (wasting tokens) or manually copy relevant sections. Agent Fox solves this by providing an MCP service that lets LLMs efficiently query API documentation through multi-level retrieval, minimizing token consumption while maximizing usefulness.
+
+**Target**: Developer-facing SaaS product with user authentication and multi-tenant isolation.
+
+## Architecture Overview
+
+### Monorepo Structure (pnpm workspace)
+
+```
+agent-fox/
+├── packages/
+│   ├── web/          # React frontend (Vite + TailwindCSS + shadcn/ui)
+│   ├── server/       # Express backend API
+│   ├── mcp/          # MCP service (independent Express process)
+│   └── shared/       # Shared types + Prisma client
+├── prisma/           # Prisma schema + migrations
+├── package.json
+├── pnpm-workspace.yaml
+└── tsconfig.base.json
+```
+
+- `server` and `mcp` are independently deployable processes sharing the same PostgreSQL database via the `shared` Prisma client.
+- `web` is a static SPA served separately (or via CDN).
+
+### Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Frontend | React 19 + Vite + TailwindCSS + shadcn/ui + React Router + TanStack Query |
+| Backend API | Express + TypeScript + Zod (validation) |
+| MCP Service | `@modelcontextprotocol/server` + `@modelcontextprotocol/express` + `@modelcontextprotocol/node` |
+| Database | PostgreSQL + Prisma ORM |
+| OpenAPI Parsing | `@apidevtools/swagger-parser` |
+| Auth | JWT (access + refresh) + bcrypt + Passport.js (GitHub/Google OAuth) |
+| Language | TypeScript throughout |
+
+## Data Model
+
+### User
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID | Primary key |
+| email | String | Unique, for email/password auth |
+| passwordHash | String? | Nullable for OAuth-only users |
+| name | String | Display name |
+| avatarUrl | String? | Profile picture |
+| createdAt | DateTime | |
+| updatedAt | DateTime | |
+
+### OAuthAccount
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID | Primary key |
+| userId | UUID | FK → User |
+| provider | String | "github" or "google" |
+| providerAccountId | String | External account ID |
+| createdAt | DateTime | |
+
+### Project
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID | Primary key, exposed as project ID |
+| userId | UUID | FK → User (owner) |
+| name | String | Project display name |
+| description | String? | Optional description |
+| baseUrl | String? | API base URL |
+| openApiSpec | JSONB | Full dereferenced OpenAPI document |
+| openApiVersion | String | e.g., "3.0.3", "3.1.0" |
+| apiKeyHash | String | Hashed API key for MCP access |
+| createdAt | DateTime | |
+| updatedAt | DateTime | |
+
+### Module
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID | Primary key |
+| projectId | UUID | FK → Project |
+| name | String | Module name (from tag or path prefix) |
+| description | String? | Module description |
+| sortOrder | Int | Display order |
+| source | Enum | "tag", "path_prefix", "manual" |
+| createdAt | DateTime | |
+| updatedAt | DateTime | |
+
+### Endpoint
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID | Primary key |
+| projectId | UUID | FK → Project |
+| moduleId | UUID | FK → Module |
+| method | String | HTTP method (GET, POST, PUT, DELETE, etc.) |
+| path | String | URL path (e.g., /api/users/{id}) |
+| summary | String? | Short description |
+| description | String? | Detailed description |
+| operationId | String? | OpenAPI operationId |
+| parameters | JSONB | Path, query, header parameters |
+| requestBody | JSONB? | Request body schema |
+| responses | JSONB | Response schemas by status code |
+| tags | String[] | Original OpenAPI tags |
+| deprecated | Boolean | Whether the endpoint is deprecated |
+| createdAt | DateTime | |
+| updatedAt | DateTime | |
+
+## MCP Multi-Level Retrieval Design (Core Feature)
+
+The MCP service exposes 5 tools that enable LLMs to progressively drill down into API documentation, minimizing token usage at each step.
+
+### Tool Definitions
+
+#### 1. `get_project_overview`
+- **Description** (shown to LLM): "Get an overview of this API project including its name, version, base URL, and a summary of available modules with endpoint counts. Call this first to understand what the API offers. This is usually sufficient to decide which module to drill into."
+- **Input**: (none — projectId comes from the MCP connection URL)
+- **Output**: `{ name, description, version, baseUrl, totalEndpoints, modules: [{ id, name, endpointCount }] }`
+- **Estimated tokens**: ~200
+- **Note**: This is the recommended entry point. It provides a compact overview including module names and counts, enough for LLMs to decide next steps.
+
+#### 2. `list_modules`
+- **Description**: "List all API modules/groups with their descriptions. Each module contains related endpoints. Use this when you need module descriptions to decide which module to explore."
+- **Input**: (none)
+- **Output**: `[{ id, name, description, endpointCount }]`
+- **Estimated tokens**: ~100-300
+- **Note**: Differs from `get_project_overview` by including module descriptions. Use when the module name alone isn't enough to determine relevance.
+
+#### 3. `list_endpoints`
+- **Description**: "List all endpoints in a specific module. Returns method, path, and summary for each endpoint. Use get_endpoint_detail to get full information about a specific endpoint."
+- **Input**: `{ moduleId: string }`
+- **Output**: `[{ id, method, path, summary, deprecated }]`
+- **Estimated tokens**: ~200-500
+
+#### 4. `get_endpoint_detail`
+- **Description**: "Get complete details for a specific endpoint including parameters, request body schema, response schemas, and examples. Use this when you need to understand exactly how to call an endpoint."
+- **Input**: `{ endpointId: string }`
+- **Output**: `{ method, path, summary, description, parameters, requestBody, responses, deprecated }`
+- **Estimated tokens**: ~500-2000
+
+#### 5. `search_endpoints`
+- **Description**: "Search for endpoints by keyword. Searches across path, summary, description, operationId, and parameter names. Optionally filter by module. Returns matching endpoint summaries."
+- **Input**: `{ keyword: string, moduleId?: string }`
+- **Output**: `[{ id, method, path, summary, moduleName, deprecated }]`
+- **Estimated tokens**: ~200-500
+
+### Retrieval Flow Example
+
+```
+LLM wants to call "create user" endpoint:
+  1. get_project_overview()                    → ~200 tokens (see all modules)
+  2. list_endpoints({ moduleId: "users" })     → ~300 tokens (see user endpoints)
+  3. get_endpoint_detail({ endpointId: "..." }) → ~800 tokens (get full details)
+  Total: ~1,300 tokens (vs 10,000+ for full doc dump)
+```
+
+### MCP Authentication
+
+- MCP endpoint URL: `https://host/mcp/:projectId`
+- Auth via `Authorization: Bearer <project-api-key>` header
+- API key is validated against the project's `apiKeyHash`
+- Each project has its own isolated MCP instance
+
+### MCP Transport
+
+Support both transport protocols:
+- **Streamable HTTP** (new standard): POST/GET/DELETE on `/mcp/:projectId`
+- **SSE** (legacy): GET with SSE on `/mcp/:projectId/sse`, POST on `/mcp/:projectId/messages`
+
+Use `@modelcontextprotocol/express` middleware with session management for stateful connections.
+
+## Frontend Pages
+
+### 1. Auth Pages
+- Login: email/password form + GitHub/Google OAuth buttons
+- Register: email/password form + OAuth
+
+### 2. Projects List Page
+- Card grid of user's projects
+- Each card: project name, version, endpoint count, created date
+- Create project button → import flow
+
+### 3. Project Detail Page (Tabbed)
+
+**Tab: Documentation Preview**
+- Interactive API doc browser (similar to Swagger UI)
+- Left sidebar: module list (collapsible)
+- Main area: endpoint list grouped by module, expandable to show details
+- Supports try-it-out (optional, future feature)
+
+**Tab: Module Management**
+- View auto-generated modules
+- Drag-and-drop reorder
+- Move endpoints between modules
+- Create/rename/delete modules
+
+**Tab: MCP Integration**
+- MCP service URL (copyable)
+- API Key display (masked, with copy and rotate buttons)
+- Configuration snippet for Claude Code, Cursor, etc. (copyable JSON)
+- Connection status indicator
+
+**Tab: Settings**
+- Project name/description editing
+- Re-import OpenAPI document (with diff preview)
+- Danger zone: delete project
+
+### Import Flow
+1. User uploads JSON/YAML file OR pastes URL
+2. Backend validates with `@apidevtools/swagger-parser`
+3. Backend dereferences all `$ref` pointers
+4. Parses tags → Module records, paths → Endpoint records
+5. Endpoints without tags auto-grouped by path prefix (first segment)
+6. Preview shown to user with module/endpoint breakdown
+7. User confirms → data saved
+
+## Backend API
+
+### Auth Routes
+```
+POST   /api/auth/register           # Email registration
+POST   /api/auth/login              # Email login → returns JWT pair
+POST   /api/auth/refresh            # Refresh access token
+GET    /api/auth/github              # GitHub OAuth redirect
+GET    /api/auth/google              # Google OAuth redirect
+GET    /api/auth/callback/:provider  # OAuth callback
+```
+
+### Project Routes
+```
+GET    /api/projects                 # List user's projects
+POST   /api/projects                 # Create project (upload OpenAPI doc)
+GET    /api/projects/:id             # Get project details
+PUT    /api/projects/:id             # Update project metadata
+DELETE /api/projects/:id             # Delete project
+```
+
+### Module/Endpoint Routes
+```
+GET    /api/projects/:id/modules           # List modules
+PUT    /api/projects/:id/modules/:mid      # Update module (rename, reorder)
+POST   /api/projects/:id/modules           # Create manual module
+DELETE /api/projects/:id/modules/:mid      # Delete module
+PATCH  /api/projects/:id/endpoints/:eid    # Move endpoint to different module
+```
+
+### Import/Key Routes
+```
+POST   /api/projects/:id/reimport          # Re-import OpenAPI document
+POST   /api/projects/:id/api-key/rotate    # Rotate API key
+```
+
+### Authentication Design
+- **User auth**: JWT dual-token (access: 15min, refresh: 7d)
+- **Password**: bcrypt hashed
+- **OAuth**: Passport.js strategies for GitHub and Google
+- **MCP auth**: Project-level API key (independent from user JWT)
+- API key format: `afk_` prefix + 32-char random string
+- API key stored as bcrypt hash in database
+
+## Deployment (Docker Compose)
+
+All services containerized and orchestrated via Docker Compose for one-command deployment.
+
+### Services
+
+```yaml
+services:
+  postgres:        # PostgreSQL 16
+  server:          # Backend API (Express) - port 3000
+  mcp:             # MCP service (Express) - port 3001
+  web:             # Frontend (Nginx serving static build) - port 80
+  redis:           # Optional: session store / rate limiting cache
+```
+
+### Container Details
+
+| Service | Base Image | Notes |
+|---------|-----------|-------|
+| `postgres` | `postgres:16-alpine` | Persistent volume for data, init scripts for DB creation |
+| `server` | `node:20-alpine` | Multi-stage build: build → runtime only |
+| `mcp` | `node:20-alpine` | Same multi-stage build pattern |
+| `web` | `node:20-alpine` → `nginx:alpine` | Build stage + Nginx serve stage |
+| `redis` | `redis:7-alpine` | Optional, for rate limiting and session cache |
+
+### Docker Files
+
+```
+agent-fox/
+├── docker-compose.yml           # Orchestration
+├── docker-compose.dev.yml       # Dev overrides (hot reload, debug ports)
+├── packages/
+│   ├── web/Dockerfile
+│   ├── server/Dockerfile
+│   └── mcp/Dockerfile
+└── .env.example                 # Environment variable template
+```
+
+### Environment Variables
+
+Managed via `.env` file (git-ignored), with `.env.example` as template:
+- `DATABASE_URL` — PostgreSQL connection string
+- `JWT_SECRET` — JWT signing key
+- `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET` — GitHub OAuth
+- `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET` — Google OAuth
+- `MCP_BASE_URL` — Public URL for MCP service
+- `REDIS_URL` — Optional Redis connection
+
+### Dev vs Prod
+
+- **Dev** (`docker-compose -f docker-compose.yml -f docker-compose.dev.yml up`): Source mounted as volumes, hot reload enabled, debug ports exposed
+- **Prod** (`docker-compose up`): Optimized multi-stage builds, no source mounting, Nginx serves frontend
+
+## Error Handling
+
+- All API responses follow `{ success: boolean, data?: T, error?: { code, message } }` format
+- Zod validation on all inputs
+- MCP tools return structured error messages that help LLMs self-correct
+- Rate limiting on MCP endpoints to prevent abuse
+
+## Verification Plan
+
+### Unit Tests
+- OpenAPI parsing: validate correct module/endpoint extraction from sample docs
+- MCP tools: verify each tool returns correct data shape and respects scoping
+- Auth: test JWT generation, validation, refresh flow
+
+### Integration Tests
+- Full import flow: upload OpenAPI doc → verify modules/endpoints created correctly
+- MCP retrieval flow: simulate LLM calling tools in sequence
+- Auth flow: register → login → access protected routes
+
+### Manual Testing
+- Import Petstore OpenAPI sample → verify preview and module grouping
+- Configure Claude Code with generated MCP config → verify tools work
+- Test search with various keywords → verify relevance