文档对齐 React 技术栈,并支持渐进式数据库迁移
将 README 与部署文档从仅 init 改为 001 基线 + 增量迁移约定,并补充 create/status 脚手架与序号校验。
This commit is contained in:
+84
-45
@@ -1,24 +1,17 @@
|
||||
# order-site-backend
|
||||
|
||||
快手轻量后端服务。
|
||||
|
||||
后端负责订单入库、履约任务编排、kuaishou-lewan / 91 卡券 / 行业电子凭证配置、后台管理接口和数据库迁移。默认入口就是快手轻量后端。
|
||||
快手轻量后端:订单入库、履约任务编排、91 卡券 / kuaishou-cloud / kuaishou-feifei / 行业电子凭证对接、后台管理接口,以及数据库迁移。
|
||||
|
||||
## 快速启动
|
||||
|
||||
先安装 Node 依赖:
|
||||
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
复制并填写环境变量后启动:
|
||||
|
||||
```bash
|
||||
cp ../../.env.mac-docker.example ../../.env
|
||||
npm run dev
|
||||
```
|
||||
|
||||
推荐优先用仓库根目录 Docker 开发环境(含 PostgreSQL 与 Caddy)。
|
||||
|
||||
生产构建:
|
||||
|
||||
```bash
|
||||
@@ -36,61 +29,107 @@ npm test
|
||||
|
||||
## 配置
|
||||
|
||||
后端采用工作区根目录 `.env` 做环境配置:
|
||||
环境变量来自工作区根目录 `.env`:
|
||||
|
||||
- 通用默认值:`src/config/defaults.ts`
|
||||
- 本地/部署覆盖:工作区根目录 `.env`
|
||||
- 默认值:`src/config/defaults.ts`
|
||||
- 覆盖:根目录 `.env`
|
||||
|
||||
常用配置:
|
||||
常用项:
|
||||
|
||||
- 服务端口 `PORT`(Docker 部署中由根目录 `.env` 的 `BACKEND_PORT` 映射)
|
||||
- 日志级别 `LOG_LEVEL=debug|info|warn|error`
|
||||
- 数据库连接 `DATABASE_URL`
|
||||
- 后台登录密钥 `ADMIN_SESSION_SECRET`
|
||||
- 默认后台用户 `ADMIN_DEFAULT_USERS_JSON`
|
||||
- 91 卡券开放接口凭据 `KAQUAN91_USER_ID` / `KAQUAN91_SECRET`
|
||||
| 变量 | 说明 |
|
||||
| --- | --- |
|
||||
| `PORT` | 服务端口(Compose 中常由 `BACKEND_PORT` 映射) |
|
||||
| `LOG_LEVEL` | `debug` / `info` / `warn` / `error` |
|
||||
| `DATABASE_URL` | PostgreSQL 连接串 |
|
||||
| `ADMIN_SESSION_SECRET` | 后台登录态签名密钥 |
|
||||
| `ADMIN_DEFAULT_USERS_JSON` | 默认后台用户 |
|
||||
| `KAQUAN91_USER_ID` / `KAQUAN91_SECRET` | 91 卡券凭据 |
|
||||
|
||||
## 接口
|
||||
|
||||
- `GET /health`:健康检查
|
||||
- `GET /health/live`:存活检查
|
||||
- `GET /health/ready`:就绪检查
|
||||
- `/api/v1/open/91/*`:91 卡券开放回调
|
||||
- `/api/v1/claim/*`:领取页接口
|
||||
- `/api/v1/admin/*`:后台管理接口
|
||||
| 路径 | 说明 |
|
||||
| --- | --- |
|
||||
| `GET /health` | 健康检查 |
|
||||
| `GET /health/live` | 存活 |
|
||||
| `GET /health/ready` | 就绪 |
|
||||
| `/api/v1/open/91/*` | 91 卡券开放接口 |
|
||||
| `/api/v1/open/kuaishou-industry/*` | 快手行业电子凭证 |
|
||||
| `/api/v1/open/kuaishou-feifei/*` | 发货平台 feifei 回调等 |
|
||||
| `/api/v1/claim/*` | 领取页接口 |
|
||||
| `/api/v1/admin/*` | 后台管理 |
|
||||
| `/s/*` | 短链跳转 |
|
||||
|
||||
## 目录
|
||||
|
||||
- `src/index.ts`:服务入口
|
||||
- `src/app.ts`:Express 应用组装
|
||||
- `src/routes`:API 路由
|
||||
- `src/services/order`:订单、商品匹配和履约任务编排
|
||||
- `src/services/fulfillment`:kuaishou-lewan 履约编排
|
||||
- `src/services/admin`:后台读写、权限、配置管理
|
||||
- `src/repositories`:数据库访问
|
||||
- `src/db/migrations`:新库初始化脚本
|
||||
| 路径 | 职责 |
|
||||
| --- | --- |
|
||||
| `src/index.ts` | 服务入口 |
|
||||
| `src/app.ts` | Express 组装 |
|
||||
| `src/routes` | API 路由 |
|
||||
| `src/services/order` | 订单与任务同步 |
|
||||
| `src/services/fulfillment` | 履约路由与执行器 |
|
||||
| `src/services/platforms` | 外部平台实现 |
|
||||
| `src/services/admin` | 后台读写与配置 |
|
||||
| `src/repositories` | 数据库访问 |
|
||||
| `src/db/migrations` | SQL 迁移 |
|
||||
|
||||
## 数据库迁移
|
||||
|
||||
策略:**001 基线 + 后续渐进增量**。
|
||||
|
||||
| 文件 | 作用 |
|
||||
| --- | --- |
|
||||
| `src/db/migrations/001_init.sql` | 空库完整基线结构 |
|
||||
| `src/db/migrations/002_*.sql` 起 | 增量变更 |
|
||||
|
||||
规则:
|
||||
|
||||
1. 已应用的迁移文件禁止改内容。
|
||||
2. 结构变更只新增文件,不回写 `001_init.sql`。
|
||||
3. 启动时自动执行未应用迁移;记录在 `schema_migrations`。
|
||||
4. 文件名必须为 `NNN_name.sql`(三位序号)。
|
||||
|
||||
命令:
|
||||
|
||||
```bash
|
||||
# 对齐状态
|
||||
npm run db:migrate:status
|
||||
|
||||
# 手动 up
|
||||
npm run db:migrate
|
||||
|
||||
# 创建下一份增量(生成 00N_xxx.sql 模板)
|
||||
npm run db:migrate:create -- add_something
|
||||
```
|
||||
|
||||
创建后编辑 SQL,与业务代码一并提交。Docker 开发环境示例:
|
||||
|
||||
```bash
|
||||
docker compose -f ../../docker-compose.dev.yml exec -T backend npm run db:migrate:create -- add_something
|
||||
docker compose -f ../../docker-compose.dev.yml exec -T backend npm run db:migrate
|
||||
```
|
||||
|
||||
## Docker 开发
|
||||
|
||||
开发版 Compose 会把源码挂载进容器:
|
||||
开发 Compose 会挂载源码:
|
||||
|
||||
- `postgres` 容器提供开发库,后端通过 `DATABASE_URL` 连接
|
||||
- 后端容器启动时执行依赖安装和 `npm run dev`
|
||||
- 平时改 `src/` 或 `.env`,一般不需要重建镜像
|
||||
- `postgres` 提供开发库
|
||||
- 后端启动时安装依赖并 `npm run dev`
|
||||
- 改 `src/` 或 `.env` 通常不必重建镜像
|
||||
|
||||
只有下面几类改动通常还需要重新构建 backend:
|
||||
需要重建 backend 镜像的情况:
|
||||
|
||||
- `deploy/docker/backend*.Dockerfile`
|
||||
- Node 版本或系统层依赖
|
||||
- `package.json` / `package-lock.json`
|
||||
- `deploy/docker/backend*.Dockerfile` 变更
|
||||
- Node 版本或系统依赖变更
|
||||
- `package.json` / `package-lock.json` 变更
|
||||
|
||||
## 运行产物
|
||||
|
||||
运行日志默认保存在:
|
||||
日志默认:
|
||||
|
||||
- `data/logs/app-YYYY-MM-DD.log`
|
||||
- `data/logs/integration-YYYY-MM-DD.log`
|
||||
|
||||
日志按天切分,默认自动清理 7 天前的旧日志。成功的 `/health`、`/health/live`、`/health/ready` 探活请求默认不写 access log。
|
||||
按天切分,默认清理过期日志。成功的 `/health*` 探活默认不写 access log。
|
||||
|
||||
`data/*.json` 为本地运行配置,可能包含账号、Cookie、token 或平台配置,默认不提交;请复制 `data/*.example.json` 后填写真实值。
|
||||
`data/*.json` 为本地运行配置,可能含敏感信息,默认不提交;请从 `*.example.json` 复制后填写。
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
"cleanup:dev-data": "tsx scripts/cleanup-dev-data.ts",
|
||||
"build": "tsc -p tsconfig.build.json && rm -rf dist/db/migrations && mkdir -p dist/db && cp -R src/db/migrations dist/db/migrations",
|
||||
"db:migrate": "tsx src/db/migrate.ts",
|
||||
"db:migrate:create": "tsx scripts/create-migration.ts",
|
||||
"db:migrate:status": "tsx scripts/migration-status.ts",
|
||||
"dev": "tsx watch --clear-screen=false src/index.ts",
|
||||
"format": "prettier --write .",
|
||||
"mock:claim": "tsx scripts/mock-kuaishou-cloud-claim.ts",
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
/**
|
||||
* 创建下一份渐进式数据库迁移文件。
|
||||
*
|
||||
* 用法:
|
||||
* npm run db:migrate:create -- add_order_note
|
||||
* npm run db:migrate:create -- "add order note"
|
||||
*
|
||||
* 规则:
|
||||
* - 001_init.sql 是基线,只用于空库首次建表
|
||||
* - 后续结构变更一律新增 002_xxx.sql / 003_xxx.sql ...
|
||||
* - 已应用到任何环境的迁移文件禁止修改内容
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const CURRENT_DIR = path.dirname(fileURLToPath(import.meta.url))
|
||||
const MIGRATIONS_DIR = path.join(CURRENT_DIR, '../src/db/migrations')
|
||||
const MIGRATION_FILE_PATTERN = /^(\d{3})_(.+)\.sql$/i
|
||||
|
||||
function slugify(input: string): string {
|
||||
return input
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '_')
|
||||
.replace(/^_+|_+$/g, '')
|
||||
.replace(/_+/g, '_')
|
||||
}
|
||||
|
||||
function listMigrationFiles(): string[] {
|
||||
if (!fs.existsSync(MIGRATIONS_DIR)) {
|
||||
throw new Error(`迁移目录不存在: ${MIGRATIONS_DIR}`)
|
||||
}
|
||||
|
||||
return fs
|
||||
.readdirSync(MIGRATIONS_DIR)
|
||||
.filter((name) => MIGRATION_FILE_PATTERN.test(name))
|
||||
.sort((a, b) => a.localeCompare(b, 'en'))
|
||||
}
|
||||
|
||||
function resolveNextVersion(files: string[]): number {
|
||||
let maxVersion = 0
|
||||
for (const file of files) {
|
||||
const match = file.match(MIGRATION_FILE_PATTERN)
|
||||
if (!match) {
|
||||
continue
|
||||
}
|
||||
maxVersion = Math.max(maxVersion, Number(match[1]))
|
||||
}
|
||||
return maxVersion + 1
|
||||
}
|
||||
|
||||
function buildTemplate(version: string, name: string): string {
|
||||
const stamp = new Date().toISOString()
|
||||
return `-- ${version}_${name}.sql
|
||||
-- 创建时间: ${stamp}
|
||||
-- 说明: TODO 填写本次结构变更目的
|
||||
--
|
||||
-- 渐进迁移约定:
|
||||
-- 1. 只写增量 SQL(ALTER / CREATE INDEX / 数据回填等)
|
||||
-- 2. 禁止修改已经提交并应用过的历史迁移文件
|
||||
-- 3. 空库会按文件名顺序执行 001_init.sql → 后续增量迁移
|
||||
-- 4. 线上/开发库启动时自动应用尚未记录在 schema_migrations 中的文件
|
||||
|
||||
-- 在此编写增量 SQL
|
||||
`
|
||||
}
|
||||
|
||||
function main() {
|
||||
const rawName = process.argv.slice(2).join(' ').trim()
|
||||
if (!rawName || rawName.startsWith('-')) {
|
||||
console.error('用法: npm run db:migrate:create -- <migration_name>')
|
||||
console.error('示例: npm run db:migrate:create -- add_task_priority')
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
const name = slugify(rawName)
|
||||
if (!name) {
|
||||
console.error('迁移名称无效,请使用英文/数字/下划线描述,例如 add_task_priority')
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
const existing = listMigrationFiles()
|
||||
const nextVersion = resolveNextVersion(existing)
|
||||
if (nextVersion > 999) {
|
||||
console.error('迁移序号已超过 999,请调整命名策略')
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
const version = String(nextVersion).padStart(3, '0')
|
||||
const fileName = `${version}_${name}.sql`
|
||||
const filePath = path.join(MIGRATIONS_DIR, fileName)
|
||||
|
||||
if (fs.existsSync(filePath)) {
|
||||
console.error(`迁移文件已存在: ${fileName}`)
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
const conflict = existing.find((item) => item.startsWith(`${version}_`))
|
||||
if (conflict) {
|
||||
console.error(`迁移序号冲突: ${version} 已被 ${conflict} 占用`)
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
fs.writeFileSync(filePath, buildTemplate(version, name), 'utf8')
|
||||
console.info(`[db:migrate:create] 已创建 ${path.relative(process.cwd(), filePath)}`)
|
||||
console.info('[db:migrate:create] 编辑 SQL 后提交;后端启动或 npm run db:migrate 会自动应用')
|
||||
}
|
||||
|
||||
main()
|
||||
@@ -0,0 +1,102 @@
|
||||
/**
|
||||
* 查看本地迁移文件与数据库 schema_migrations 的对齐状态。
|
||||
*
|
||||
* 用法:
|
||||
* npm run db:migrate:status
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { closeDb, query } from '../src/db/client.js'
|
||||
|
||||
const CURRENT_DIR = path.dirname(fileURLToPath(import.meta.url))
|
||||
const MIGRATIONS_DIR = path.join(CURRENT_DIR, '../src/db/migrations')
|
||||
const MIGRATION_FILE_PATTERN = /^(\d{3})_.+\.sql$/i
|
||||
|
||||
function listLocalMigrations(): string[] {
|
||||
if (!fs.existsSync(MIGRATIONS_DIR)) {
|
||||
return []
|
||||
}
|
||||
|
||||
return fs
|
||||
.readdirSync(MIGRATIONS_DIR)
|
||||
.filter((name) => MIGRATION_FILE_PATTERN.test(name))
|
||||
.sort((a, b) => a.localeCompare(b, 'en'))
|
||||
}
|
||||
|
||||
function normalizeMigrationName(value: unknown): string {
|
||||
const raw = String(value || '').trim()
|
||||
if (!raw) {
|
||||
return ''
|
||||
}
|
||||
return raw.replace(/\.sql$/i, '')
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const localFiles = listLocalMigrations()
|
||||
const localNames = localFiles.map((file) => file.replace(/\.sql$/i, ''))
|
||||
|
||||
let appliedNames = new Set<string>()
|
||||
try {
|
||||
const result = await query<{ name: string | null; filename: string | null; run_on: string | null }>(
|
||||
`
|
||||
SELECT name, filename, run_on
|
||||
FROM schema_migrations
|
||||
ORDER BY COALESCE(filename, name), id
|
||||
`,
|
||||
)
|
||||
|
||||
appliedNames = new Set(
|
||||
result.rows
|
||||
.map((row) => normalizeMigrationName(row.filename || row.name))
|
||||
.filter(Boolean),
|
||||
)
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
if (/schema_migrations/i.test(message) && /does not exist|不存在/i.test(message)) {
|
||||
console.warn('[db:migrate:status] schema_migrations 尚不存在,视为尚未应用任何迁移')
|
||||
} else {
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
const pending = localNames.filter((name) => !appliedNames.has(name))
|
||||
const orphaned = [...appliedNames].filter((name) => !localNames.includes(name))
|
||||
|
||||
console.info('[db:migrate:status] 本地迁移文件')
|
||||
if (localFiles.length === 0) {
|
||||
console.info(' (无)')
|
||||
} else {
|
||||
for (const file of localFiles) {
|
||||
const name = file.replace(/\.sql$/i, '')
|
||||
const mark = appliedNames.has(name) ? 'applied' : 'pending'
|
||||
console.info(` [${mark}] ${file}`)
|
||||
}
|
||||
}
|
||||
|
||||
console.info(`[db:migrate:status] 已应用 ${appliedNames.size} 个,待应用 ${pending.length} 个`)
|
||||
if (pending.length > 0) {
|
||||
console.info('[db:migrate:status] 待应用:')
|
||||
for (const name of pending) {
|
||||
console.info(` - ${name}.sql`)
|
||||
}
|
||||
}
|
||||
|
||||
if (orphaned.length > 0) {
|
||||
console.warn('[db:migrate:status] 数据库中存在本地没有的迁移记录(可能是旧环境残留):')
|
||||
for (const name of orphaned) {
|
||||
console.warn(` - ${name}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main()
|
||||
.catch((error) => {
|
||||
console.error(error)
|
||||
process.exitCode = 1
|
||||
})
|
||||
.finally(async () => {
|
||||
await closeDb()
|
||||
})
|
||||
@@ -1,3 +1,14 @@
|
||||
/**
|
||||
* 数据库迁移入口。
|
||||
*
|
||||
* 策略:
|
||||
* - 001_init.sql:空库基线,建立完整结构
|
||||
* - 002_xxx.sql 起:渐进增量迁移,禁止改已应用历史文件
|
||||
* - 启动时自动 up;也可用 npm run db:migrate 手动执行
|
||||
* - 新建迁移:npm run db:migrate:create -- <name>
|
||||
* - 查看状态:npm run db:migrate:status
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
@@ -12,10 +23,14 @@ const CURRENT_DIR = path.dirname(fileURLToPath(import.meta.url))
|
||||
const MIGRATIONS_DIR = path.join(CURRENT_DIR, 'migrations')
|
||||
const MIGRATIONS_TABLE = 'schema_migrations'
|
||||
const SHORT_NUMERIC_PREFIX_MESSAGE = /^Can't determine timestamp for \d+$/
|
||||
const MIGRATION_FILE_PATTERN = /^(\d{3})_.+\.sql$/i
|
||||
|
||||
export async function runDatabaseMigrations() {
|
||||
assertMigrationFilesOrdered()
|
||||
await ensureNodePgMigrateMetadataCompatibility()
|
||||
|
||||
console.info(`[db:migrate] 迁移目录: ${MIGRATIONS_DIR}`)
|
||||
|
||||
await runner({
|
||||
databaseUrl: createMigrationDatabaseConfig(),
|
||||
dir: MIGRATIONS_DIR,
|
||||
@@ -45,6 +60,44 @@ export async function runDatabaseMigrations() {
|
||||
await syncLegacyMigrationColumns()
|
||||
}
|
||||
|
||||
function assertMigrationFilesOrdered() {
|
||||
if (!fs.existsSync(MIGRATIONS_DIR)) {
|
||||
throw new Error(`迁移目录不存在: ${MIGRATIONS_DIR}`)
|
||||
}
|
||||
|
||||
const files = fs
|
||||
.readdirSync(MIGRATIONS_DIR)
|
||||
.filter((name) => name.endsWith('.sql'))
|
||||
.sort((a, b) => a.localeCompare(b, 'en'))
|
||||
|
||||
if (files.length === 0) {
|
||||
throw new Error(`迁移目录为空: ${MIGRATIONS_DIR}`)
|
||||
}
|
||||
|
||||
const invalid = files.filter((name) => !MIGRATION_FILE_PATTERN.test(name))
|
||||
if (invalid.length > 0) {
|
||||
throw new Error(
|
||||
`迁移文件命名必须为 NNN_name.sql(三位序号),非法文件: ${invalid.join(', ')}`,
|
||||
)
|
||||
}
|
||||
|
||||
const versions = files.map((name) => Number(name.slice(0, 3)))
|
||||
for (let index = 1; index < versions.length; index += 1) {
|
||||
if (versions[index] < versions[index - 1]) {
|
||||
throw new Error(`迁移序号乱序: ${files[index - 1]} 之后出现 ${files[index]}`)
|
||||
}
|
||||
if (versions[index] === versions[index - 1]) {
|
||||
throw new Error(`迁移序号重复: ${files[index - 1]} 与 ${files[index]}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (files[0] !== '001_init.sql') {
|
||||
console.warn(
|
||||
`[db:migrate] 警告: 首个迁移不是 001_init.sql(当前为 ${files[0]}),请确认是否刻意调整基线`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
async function ensureNodePgMigrateMetadataCompatibility() {
|
||||
await query(`
|
||||
CREATE TABLE IF NOT EXISTS ${MIGRATIONS_TABLE} (
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
-- 初始化完整业务库结构。
|
||||
-- 本项目当前不维护历史测试库的逐步升级链路;新环境只需要执行这一份 init 迁移。
|
||||
-- 001_init.sql —— 空库基线迁移(完整业务库结构)。
|
||||
--
|
||||
-- 约定:
|
||||
-- 1. 仅在空库首次迁移时执行本文件,建立全量基线结构。
|
||||
-- 2. 后续结构变更不要改本文件,改为新增 002_xxx.sql / 003_xxx.sql 等渐进迁移。
|
||||
-- 3. 新环境按文件名顺序执行:001_init → 后续增量;已有环境只应用尚未记录的增量。
|
||||
-- 4. 已应用到任何共享/生产环境的迁移文件禁止修改内容。
|
||||
|
||||
CREATE EXTENSION IF NOT EXISTS pgcrypto;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user