perf(admin): 打手等级筛选用 UNION 形式、概览加短缓存、静态配置接口加缓存头
三处都出自审计清单,改动互不重叠。
1) 打手等级筛选(生产实测 1119ms)
原写法是对 worker_users 的两个相关 EXISTS(接单人 / 拼单参与人),规划器无法用它
驱动索引,只能顺序扫完整张 work_orders。改成先把等级解析成打手 id
(worker_users(level_id) 有索引,等级只有 6 个、解析出几百个 id),再走索引。
注意:**不能**写成 `assigned_worker_id = ANY(...) OR id IN (...)` —— 实测 1852ms,
比原写法还慢(OR 右支要逐行回表取 id)。用 UNION 形式:
wo.id IN (SELECT id FROM work_orders WHERE assigned_worker_id = ANY(...) UNION
SELECT work_order_id FROM work_order_shares WHERE worker_id = ANY(...) AND status != 'cancelled')
实测 61ms(count),语义与原来完全一致(含"空数组=过滤掉全部")。
三个入口 listWorkOrders / countWorkOrders / getWorkOrderStatistics 都先解析;
未解析时保留原相关 EXISTS 兜底。
2) 平台概览短缓存(5 秒)
一次并发 5 条 count(4 条打 work_orders,一条是运营态 CASE),而池只有 8 条连接,
后台刷新时与其他请求互相争抢(抓包显示整批 9 个请求同时发出、它自身 167ms)。
看板计数滞后几秒无感,用 WorkerListCache 缓存并合并并发。
3) 三个静态配置接口加 HTTP 缓存头(private, max-age=300)
接单分类、快手行业店铺、自定义店铺此前没有任何 Cache-Control,每次刷新都真发一次
请求(抓包里 4 个这类请求)。带 private 是因为这些是带鉴权的后台接口。
**dev-mock/status 故意不加**:它是可切换状态,缓存会让"切了开关界面没变"。
验证:后端 547 项单测全绿(新增 4 项守卫,分别锁住 UNION 形式、三个入口都解析、
概览缓存、以及 dev-mock 不得加缓存)。
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
import assert from 'node:assert/strict'
|
||||
import { readFile } from 'node:fs/promises'
|
||||
import test from 'node:test'
|
||||
|
||||
import { setHttpCache } from './http-cache.js'
|
||||
|
||||
test('setHttpCache 必须带 private(后台接口带鉴权,禁止共享缓存留存响应)', () => {
|
||||
const headers: Record<string, string> = {}
|
||||
const res = {
|
||||
setHeader: (name: string, value: string) => {
|
||||
headers[name] = value
|
||||
},
|
||||
}
|
||||
let called = false
|
||||
setHttpCache(300)(
|
||||
{} as never,
|
||||
res as never,
|
||||
(() => {
|
||||
called = true
|
||||
}) as never,
|
||||
)
|
||||
assert.equal(called, true, '必须把请求放行到下一个处理器')
|
||||
assert.equal(headers['Cache-Control'], 'private, max-age=300')
|
||||
})
|
||||
|
||||
test('静态配置接口必须挂上 HTTP 缓存头', async () => {
|
||||
const cases: Array<[string, string]> = [
|
||||
['routes/admin/worker-platform/config.ts', "router.get(\n '/worker-platform/categories',"],
|
||||
['routes/admin/kuaishou-industry.ts', "router.get(\n '/kuaishou-industry/shops',"],
|
||||
[
|
||||
'routes/admin/worker-platform/orders.ts',
|
||||
"router.get(\n '/worker-platform/orders/mock/custom-shops',",
|
||||
],
|
||||
]
|
||||
for (const [file, anchor] of cases) {
|
||||
const src = await readFile(new URL('../' + file, import.meta.url), 'utf8')
|
||||
assert.ok(src.includes('setHttpCache('), `${file} 未挂缓存头:后台刷新会重复拉取这些静态配置`)
|
||||
const routeBlock = src.slice(src.indexOf(anchor), src.indexOf(anchor) + 320)
|
||||
assert.ok(routeBlock.includes('setHttpCache('), `${file} 的缓存头没挂在该路由上`)
|
||||
}
|
||||
})
|
||||
|
||||
test('dev-mock/status 不能加 HTTP 缓存(它是可切换状态,缓存会让开关看起来失效)', async () => {
|
||||
const src = await readFile(new URL('../routes/admin/dev-mock.ts', import.meta.url), 'utf8')
|
||||
const block = src.slice(
|
||||
src.indexOf("'/dev-mock/status'"),
|
||||
src.indexOf("'/dev-mock/status'") + 320,
|
||||
)
|
||||
assert.ok(
|
||||
!block.includes('setHttpCache('),
|
||||
'dev-mock/status 缓存后,POST 切换开关再读会拿到旧值,界面看起来没生效',
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,19 @@
|
||||
import type { NextFunction, Request, Response } from 'express'
|
||||
|
||||
/**
|
||||
* 给「几乎不变」的配置类接口加 HTTP 缓存头。
|
||||
*
|
||||
* 背景:后台刷新页面时前端会重新拉一遍这些静态配置(接单分类、快手行业店铺、
|
||||
* 自定义店铺、dev-mock 状态)。它们此前没有任何 Cache-Control,于是每次刷新都真的
|
||||
* 走一次网络 —— 生产抓包显示一次刷新里有 4 个这类请求,且都和业务查询挤在同一个
|
||||
* 连接池窗口内。
|
||||
*
|
||||
* 必须带 `private`:这些是带鉴权的后台接口,禁止任何共享/中间缓存留存响应。
|
||||
*/
|
||||
export function setHttpCache(maxAgeSeconds: number) {
|
||||
const headerValue = `private, max-age=${Math.max(0, Math.floor(maxAgeSeconds))}`
|
||||
return (_req: Request, res: Response, next: NextFunction): void => {
|
||||
res.setHeader('Cache-Control', headerValue)
|
||||
next()
|
||||
}
|
||||
}
|
||||
@@ -457,6 +457,13 @@ export type ListInput = {
|
||||
levelId?: number
|
||||
/** 后台待验收筛选:普通单按接单人,拼单按参与打手等级。 */
|
||||
workerLevelIds?: number[]
|
||||
/**
|
||||
* 已解析的打手 id(由 workerLevelIds 解析而来)。
|
||||
* 等级筛选若直接写成型别 EXISTS,规划器无法索引 → 全表扫 work_orders;
|
||||
* 先解析成 id 就能走 `assigned_worker_id = ANY(ids)` 与拼单表 worker_id 索引。
|
||||
* 注意语义:**空数组表示"过滤掉全部"**,必须与"未提供"(undefined)区分。
|
||||
*/
|
||||
workerIds?: number[]
|
||||
categoryId?: number
|
||||
/** 后台打手列表按提现黑名单筛选。 */
|
||||
isWithdrawalBlacklisted?: boolean | null | undefined
|
||||
|
||||
@@ -259,12 +259,14 @@ export async function listWorkOrders({
|
||||
createdTo = '',
|
||||
}: ListInput = {}): Promise<{ items: WorkOrderRow[]; total: number }> {
|
||||
const effectiveStatuses = statuses.length > 0 ? statuses : status.trim() ? [status.trim()] : []
|
||||
const resolvedWorkerIds = await resolveWorkerLevelFilterIds(workerLevelIds)
|
||||
const { joins, whereClause, params } = buildWorkOrderWhere({
|
||||
statuses: effectiveStatuses,
|
||||
workOrderId,
|
||||
keyword,
|
||||
workerId,
|
||||
workerLevelIds,
|
||||
...(resolvedWorkerIds ? { workerIds: resolvedWorkerIds } : {}),
|
||||
categoryId,
|
||||
workerSharingId,
|
||||
hallCandidateLimit,
|
||||
@@ -346,7 +348,11 @@ export async function listWorkOrders({
|
||||
|
||||
/** 只返回筛选范围总数,不加载工单 JSON、取消事件和当前页关联字段。 */
|
||||
export async function countWorkOrders(input: ListInput = {}): Promise<number> {
|
||||
const { joins, whereClause, params } = buildWorkOrderWhere(input)
|
||||
const resolvedWorkerIds = await resolveWorkerLevelFilterIds(input.workerLevelIds)
|
||||
const { joins, whereClause, params } = buildWorkOrderWhere({
|
||||
...input,
|
||||
...(resolvedWorkerIds ? { workerIds: resolvedWorkerIds } : {}),
|
||||
})
|
||||
return countWorkOrdersWithWhere(joins, whereClause, params)
|
||||
}
|
||||
|
||||
@@ -380,7 +386,11 @@ const WORK_ORDER_PAYMENT_AMOUNT_SQL = `
|
||||
`
|
||||
|
||||
export async function getWorkOrderStatistics(input: ListInput = {}): Promise<WorkOrderStatistics> {
|
||||
const { joins, whereClause, params } = buildWorkOrderWhere(input)
|
||||
const resolvedWorkerIds = await resolveWorkerLevelFilterIds(input.workerLevelIds)
|
||||
const { joins, whereClause, params } = buildWorkOrderWhere({
|
||||
...input,
|
||||
...(resolvedWorkerIds ? { workerIds: resolvedWorkerIds } : {}),
|
||||
})
|
||||
// 仅勾「VIP 待补图」时才拼进 SQL。087 的覆盖索引故意不含 acceptance_json:
|
||||
// 这个筛选项会让统计回表变慢,是有意降级。不要把 acceptance_json 补进索引,
|
||||
// 否则默认统计(不勾这个筛选)会退回宽表堆访问。
|
||||
@@ -744,12 +754,46 @@ export async function listWorkOrdersByIds(ids: number[]): Promise<WorkOrderRow[]
|
||||
return result.rows
|
||||
}
|
||||
|
||||
/**
|
||||
* 把「打手等级筛选」解析成打手 id 列表。
|
||||
*
|
||||
* 等级筛选原本写成对 worker_users 的相关 EXISTS(两处:接单人、拼单参与人),
|
||||
* 规划器无法用它驱动索引 —— 只能顺序扫完整张 work_orders(生产实测 1119ms / 每次数百 MB)。
|
||||
* 先解析成 id 后,条件变成 `assigned_worker_id = ANY(ids)` 与
|
||||
* `work_order_shares.worker_id = ANY(ids)`,两侧都有索引,规划器可走 BitmapOr。
|
||||
*
|
||||
* 等级数量是个位数、worker_users(level_id) 有索引,解析本身是一次索引扫描。
|
||||
* 返回 null 表示"没有等级筛选";返回空数组表示"该等级下没有打手,应过滤掉全部"——
|
||||
* 这两者语义不同,调用方必须用 null 判断而不是 length。
|
||||
*/
|
||||
export async function resolveWorkerLevelFilterIds(
|
||||
workerLevelIds: unknown,
|
||||
): Promise<number[] | null> {
|
||||
const levels = Array.isArray(workerLevelIds)
|
||||
? [
|
||||
...new Set(
|
||||
workerLevelIds
|
||||
.map((level) => Number(level))
|
||||
.filter((level) => Number.isFinite(level) && level > 0),
|
||||
),
|
||||
]
|
||||
: []
|
||||
if (levels.length === 0) return null
|
||||
|
||||
const result = await query<{ id: number }>(
|
||||
'SELECT id FROM worker_users WHERE level_id = ANY($1::bigint[])',
|
||||
[levels],
|
||||
)
|
||||
return result.rows.map((row) => Number(row.id)).filter((id) => id > 0)
|
||||
}
|
||||
|
||||
export function buildWorkOrderWhere({
|
||||
statuses = [],
|
||||
workOrderId = 0,
|
||||
keyword = '',
|
||||
workerId = 0,
|
||||
workerLevelIds = [],
|
||||
workerIds,
|
||||
categoryId = 0,
|
||||
workerSharingId = 0,
|
||||
hallCandidateLimit = 0,
|
||||
@@ -916,7 +960,25 @@ export function buildWorkOrderWhere({
|
||||
)
|
||||
)`)
|
||||
}
|
||||
if (workerLevelIds.length > 0) {
|
||||
if (workerIds) {
|
||||
// 已解析成打手 id。这里必须用「id 子集 + IN」而不是
|
||||
// `assigned_worker_id = ANY(...) OR id IN (...)`:后者让 OR 的右支逐行回表取 id,
|
||||
// 生产实测 1852ms,比原来的相关 EXISTS(555ms)还慢;UNION 形式分别走
|
||||
// idx_work_orders_worker_status 与拼单表 worker_id 索引,实测 61ms。
|
||||
// 空数组要走到这里(表示"过滤掉全部"),所以判断的是 null/undefined 而不是 length。
|
||||
params.push(workerIds)
|
||||
filters.push(`wo.id IN (
|
||||
SELECT level_assigned.id
|
||||
FROM work_orders level_assigned
|
||||
WHERE level_assigned.assigned_worker_id = ANY($${params.length}::bigint[])
|
||||
UNION
|
||||
SELECT level_share.work_order_id
|
||||
FROM work_order_shares level_share
|
||||
WHERE level_share.worker_id = ANY($${params.length}::bigint[])
|
||||
AND level_share.status != 'cancelled'
|
||||
)`)
|
||||
} else if (workerLevelIds.length > 0) {
|
||||
// 兜底:调用方没有预先解析等级时仍保留原语义(相关 EXISTS,无法走索引)。
|
||||
params.push(workerLevelIds)
|
||||
filters.push(`(
|
||||
EXISTS (
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Router } from 'express'
|
||||
import { setHttpCache } from '../../middleware/http-cache.js'
|
||||
import {
|
||||
approveAdminKuaishouIndustryRefund,
|
||||
checkAdminKuaishouIndustryVoucherAvailable,
|
||||
@@ -41,6 +42,7 @@ router.get(
|
||||
router.get(
|
||||
'/kuaishou-industry/shops',
|
||||
requireAdminRoles(['admin', 'operator', 'support']),
|
||||
setHttpCache(300),
|
||||
createJsonHandler(() => listAdminKuaishouIndustryShops(), {
|
||||
successMessage: 'ok',
|
||||
errorMessage: '查询快手行业店铺失败',
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Router } from 'express'
|
||||
import { setHttpCache } from '../../../middleware/http-cache.js'
|
||||
|
||||
import {
|
||||
deleteAdminWorkCategory,
|
||||
@@ -89,6 +90,8 @@ router.delete(
|
||||
router.get(
|
||||
'/worker-platform/categories',
|
||||
requireAdminRoles(['admin', 'operator', 'support']),
|
||||
// 分类是低频配置:给浏览器 5 分钟本地缓存,避免每次刷新都发一次请求
|
||||
setHttpCache(300),
|
||||
createJsonHandler(() => listAdminWorkCategories(), {
|
||||
successMessage: 'ok',
|
||||
errorMessage: '读取接单分类失败',
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Router } from 'express'
|
||||
import { setHttpCache } from '../../../middleware/http-cache.js'
|
||||
|
||||
import {
|
||||
acceptAdminWorkOrders,
|
||||
@@ -49,6 +50,7 @@ router.get(
|
||||
router.get(
|
||||
'/worker-platform/orders/mock/custom-shops',
|
||||
requireAdminRoles(['admin', 'operator', 'support']),
|
||||
setHttpCache(300),
|
||||
createJsonHandler(() => ({ items: listAdminMockCustomShops() }), {
|
||||
successMessage: 'ok',
|
||||
errorMessage: '读取自定义店铺失败',
|
||||
|
||||
@@ -2,9 +2,25 @@ import { WORK_ORDER_STATUS } from '../../domain/work-order-status.js'
|
||||
import { countWorkOrders, listWorkerUsers } from '../../repositories/worker-platform/index.js'
|
||||
import { refreshStoredFileUrl } from '../file-storage/file-storage-service.js'
|
||||
import { publishWorkerHallRealtimeRefresh } from '../realtime/realtime-event-service.js'
|
||||
import { WorkerListCache } from './worker-list-cache.js'
|
||||
import { getWorkerHallConfig, saveWorkerHallConfig } from './worker-hall-config-service.js'
|
||||
|
||||
/**
|
||||
* 平台概览的短缓存。
|
||||
*
|
||||
* 这个接口一次并发 5 条 count(其中 4 条打 work_orders,一条还是运营态 CASE),
|
||||
* 而数据库连接池只有 8 条 —— 后台刷新页面时会和其他请求挤在一起(生产抓包显示
|
||||
* 整批 9 个请求同时发出,这个接口自身耗时 167ms 且独占 5 条连接)。
|
||||
* 它是看板计数,滞后几秒无感,因此按 5 秒缓存并合并并发请求。
|
||||
*/
|
||||
const ADMIN_SUMMARY_CACHE_TTL_MS = 5_000
|
||||
const adminSummaryCache = new WorkerListCache(ADMIN_SUMMARY_CACHE_TTL_MS, 4)
|
||||
|
||||
export async function getAdminWorkerPlatformSummary() {
|
||||
return adminSummaryCache.getOrSet('summary', computeAdminWorkerPlatformSummary)
|
||||
}
|
||||
|
||||
async function computeAdminWorkerPlatformSummary() {
|
||||
const hallConfig = getWorkerHallConfig()
|
||||
const [
|
||||
pendingWorkers,
|
||||
|
||||
@@ -2808,3 +2808,44 @@ test('打手端「我的订单」必须走"先取 id 再取详情"的延迟分
|
||||
)
|
||||
assert.ok(src.includes('rowsById.get(id)'), 'ANY() 不保证顺序,必须按 id 顺序还原分页结果')
|
||||
})
|
||||
|
||||
test('打手等级筛选必须先把等级解析成打手 id,并用 UNION 形式(不能用 OR + IN 子查询)', async () => {
|
||||
const { readFile } = await import('node:fs/promises')
|
||||
const src = await readFile(
|
||||
new URL('../../repositories/worker-platform/work-order-query-repo.ts', import.meta.url),
|
||||
'utf8',
|
||||
)
|
||||
assert.ok(
|
||||
src.includes('resolveWorkerLevelFilterIds'),
|
||||
'缺少等级→打手 id 的解析:相关 EXISTS 无法走索引,会顺序扫整张 work_orders',
|
||||
)
|
||||
assert.ok(
|
||||
/SELECT id FROM worker_users WHERE level_id = ANY/.test(src),
|
||||
'解析必须只查 worker_users(level_id)(该列有 idx_worker_users_level)',
|
||||
)
|
||||
const branch = src.slice(
|
||||
src.indexOf('if (workerIds) {'),
|
||||
src.indexOf('} else if (workerLevelIds.length > 0) {'),
|
||||
)
|
||||
assert.ok(branch.includes('UNION'), '必须用 UNION 形式(生产实测 61ms)')
|
||||
assert.ok(
|
||||
!/OR\s+wo\.id IN/.test(branch),
|
||||
'不能写成 `assigned_worker_id = ANY(...) OR id IN (...)`:OR 右支逐行回表,实测 1852ms 反而更慢',
|
||||
)
|
||||
// 三个入口都要先解析再建 where
|
||||
const resolved = src.split('resolveWorkerLevelFilterIds(').length - 1
|
||||
assert.ok(resolved >= 4, `解析调用点应为 1 处定义 + 3 个入口,实际 ${resolved}`)
|
||||
})
|
||||
|
||||
test('平台概览必须有短缓存(一次并发 5 条 count,池只有 8 条连接)', async () => {
|
||||
const { readFile } = await import('node:fs/promises')
|
||||
const src = await readFile(
|
||||
new URL('./admin-worker-platform-summary-service.ts', import.meta.url),
|
||||
'utf8',
|
||||
)
|
||||
assert.ok(src.includes('new WorkerListCache('), '缺少概览缓存:一次刷新会独占 5 条数据库连接')
|
||||
assert.ok(
|
||||
src.includes("adminSummaryCache.getOrSet('summary'"),
|
||||
'概览必须走 getOrSet 合并并发请求',
|
||||
)
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user