feat(finance): P1 人工商品归一,按内部 SKU 生成 man 商品并回填

- 新服务 manual-product-normalization-service:skuNick → outSkuSn → sku_code 兜底,按内部 SKU 生成 man:<sku> 商品
- 订单项按 id 分批(2 万/批)回填,避免 15s 语句超时;同步补齐人工成本快照的商品维度
- 新增 POST /cost-center/manual-products/normalize(支持 dryRun/日期窗口),成本中心「标准商品」加「归一人工商品」按钮
- 每日 cost_finance_sync 接入最近 N 天的增量归一
- 渠道映射未归一队列增加人工单提示:人工单不能用平台 sku_code 建映射
- 文档同步:P1 标记完成,补充归一接口契约与销售事实重算结果
This commit is contained in:
yml2213
2026-10-24 16:54:55 +08:00
parent e5122d116a
commit a78cc1f984
8 changed files with 425 additions and 7 deletions
@@ -59,6 +59,7 @@ import {
syncAdminAffiliateDashProducts,
syncAdminWorkOrderTemplateCostVersions,
} from '../../services/cost-center/template-bootstrap-service.js'
import { normalizeManualProducts } from '../../services/cost-center/manual-product-normalization-service.js'
import type { AdminSession } from '../../services/admin/admin-auth-service.js'
import {
syncActualWorkOrderCostSnapshots,
@@ -198,6 +199,30 @@ router.post(
}),
)
router.post(
'/cost-center/manual-products/normalize',
requireAdminPermission('finance.config'),
createJsonHandler(
(req) =>
normalizeManualProducts({
dateFrom: bodyOf(req).dateFrom as string | undefined,
dateTo: bodyOf(req).dateTo as string | undefined,
dryRun: bodyOf(req).dryRun === true,
}),
{
successMessage: '人工商品归一完成',
errorMessage: '归一人工商品失败',
scope: '[admin/cost-center/manual-products/normalize]',
audit: (_req, data) => ({
action: 'cost_manual_products_normalized',
targetType: 'product_catalog',
targetId: '',
data: data as Record<string, unknown>,
}),
},
),
)
router.get(
'/cost-center/products',
requireAdminPermission('finance.view'),
@@ -0,0 +1,273 @@
/**
* P1 人工商品归一:把人工履约订单项归到 `man:<内部SKU>` 标准商品。
*
* 为什么不用渠道映射:人工单的「内部 SKU」是快手 spec_json.skuNick / 拼多多
* spec_json.outSkuSn,与平台 sku_code 不是同一个值(91kaquan 实测 0 条相同),
* 所以按 provider + shop + sku_code 建映射抓不到这些单。这里直接按内部 SKU 归一。
*
* 幂等:商品按 canonical_sku 去重;订单项只更新 canonical_product_id IS NULL 的行,
* 并按 id 分批(每批 2 万)避免撞 DATABASE_STATEMENT_TIMEOUT_MS。
*/
import { query } from '../../db/client.js'
import { nowIso } from '../../utils/time.js'
const MANUAL_PRODUCT_CATEGORY = '人工客服'
const MANUAL_PRODUCT_UNIT = '件'
const NORMALIZE_BATCH_SIZE = 20000
export type NormalizeManualProductsInput = {
dateFrom?: string | undefined
dateTo?: string | undefined
dryRun?: boolean
resolveExceptions?: boolean
}
export type NormalizeManualProductsResult = {
dryRun: boolean
range: { fromDate: string; toDate: string } | null
distinctSkuCount: number
pendingItemCount: number
createdProductCount: number
updatedItemCount: number
updatedSnapshotCount: number
resolvedExceptionCount: number
remainingItemCount: number
}
/** 内部 SKU:快手 skuNick / 拼多多 outSkuSn / 平台 sku_code 兜底。 */
const MANUAL_SKU_KEY_SQL = `
COALESCE(
NULLIF(TRIM(oi.spec_json->>'skuNick'), ''),
NULLIF(TRIM(oi.spec_json->>'outSkuSn'), ''),
NULLIF(TRIM(oi.sku_code), '')
)
`
export async function normalizeManualProducts(
input: NormalizeManualProductsInput = {},
): Promise<NormalizeManualProductsResult> {
const fromDate = normalizeDate(input.dateFrom)
const toDate = normalizeDate(input.dateTo)
const hasRange = Boolean(fromDate || toDate)
const dryRun = input.dryRun === true
const now = nowIso()
const joinSql = hasRange ? 'JOIN orders o ON o.id = oi.order_id' : ''
// 日期条件按传入的起始占位符编号生成,便于不同语句复用。
function buildRangeClause(startIndex: number) {
const params: unknown[] = []
const parts: string[] = []
if (fromDate) {
parts.push(
`(COALESCE(o.paid_at, o.created_at) AT TIME ZONE 'Asia/Shanghai')::date >= $${startIndex + params.length}::date`,
)
params.push(fromDate)
}
if (toDate) {
parts.push(
`(COALESCE(o.paid_at, o.created_at) AT TIME ZONE 'Asia/Shanghai')::date <= $${startIndex + params.length}::date`,
)
params.push(toDate)
}
return { sql: parts.length ? `AND ${parts.join(' AND ')}` : '', params }
}
const candidateRange = buildRangeClause(1)
const pendingWhere = `
oi.fulfillment_mode = 'manual'
AND oi.canonical_product_id IS NULL
AND ${MANUAL_SKU_KEY_SQL} IS NOT NULL
${candidateRange.sql}
`
const candidatesResult = await query<{
sku_key: string
canonical_name: string
item_count: number
}>(
`
WITH item_keys AS (
SELECT
${MANUAL_SKU_KEY_SQL} AS sku_key,
NULLIF(TRIM(oi.sku_name), '') AS sku_name
FROM order_items oi
${joinSql}
WHERE ${pendingWhere}
)
SELECT
k.sku_key,
LEFT(
COALESCE(NULLIF(TRIM(r.display_name), ''), MIN(k.sku_name), k.sku_key),
120
) AS canonical_name,
COUNT(*)::int AS item_count
FROM item_keys k
LEFT JOIN manual_sku_registry r ON r.sku_key = k.sku_key
GROUP BY k.sku_key, r.display_name
ORDER BY COUNT(*) DESC, k.sku_key ASC
`,
candidateRange.params,
)
const candidates = candidatesResult.rows.map((row) => ({
skuKey: row.sku_key,
canonicalSku: `man:${row.sku_key}`,
canonicalName: row.canonical_name || row.sku_key,
itemCount: Number(row.item_count || 0),
}))
const pendingItemCount = candidates.reduce((sum, item) => sum + item.itemCount, 0)
if (dryRun) {
return {
dryRun: true,
range: hasRange ? { fromDate: fromDate || '', toDate: toDate || '' } : null,
distinctSkuCount: candidates.length,
pendingItemCount,
createdProductCount: 0,
updatedItemCount: 0,
updatedSnapshotCount: 0,
resolvedExceptionCount: 0,
remainingItemCount: pendingItemCount,
}
}
let createdProductCount = 0
if (candidates.length > 0) {
const inserted = await query(
`
INSERT INTO product_catalog (
canonical_sku, canonical_name, category, unit_name, status,
attributes_json, created_at, updated_at
)
SELECT
input.canonical_sku,
input.canonical_name,
$2,
$3,
'active',
jsonb_build_object('source', 'manual_normalization', 'internalSku', input.sku_key),
$4::timestamptz,
$4::timestamptz
FROM unnest($1::text[], $5::text[], $6::text[])
AS input(canonical_sku, canonical_name, sku_key)
ON CONFLICT (canonical_sku) DO NOTHING
RETURNING id
`,
[
candidates.map((item) => item.canonicalSku),
MANUAL_PRODUCT_CATEGORY,
MANUAL_PRODUCT_UNIT,
now,
candidates.map((item) => item.canonicalName),
candidates.map((item) => item.skuKey),
],
)
createdProductCount = inserted.rowCount || 0
}
// 回填订单项:按 id 分批,每批重新取日期参数,避免单条语句超过 15s。
let updatedItemCount = 0
let lastId = 0
for (let batch = 0; batch < 40; batch += 1) {
const range = buildRangeClause(3)
const updated = await query(
`
WITH target AS (
SELECT oi.id, p.id AS product_id
FROM order_items oi
${joinSql}
JOIN product_catalog p
ON p.canonical_sku = 'man:' || ${MANUAL_SKU_KEY_SQL}
WHERE oi.fulfillment_mode = 'manual'
AND oi.canonical_product_id IS NULL
AND oi.id > $1
${range.sql}
ORDER BY oi.id
LIMIT $2
)
UPDATE order_items oi
SET canonical_product_id = t.product_id,
updated_at = NOW()
FROM target t
WHERE oi.id = t.id
AND oi.canonical_product_id IS NULL
RETURNING oi.id
`,
[lastId, NORMALIZE_BATCH_SIZE, ...range.params],
)
const rowCount = updated.rowCount || 0
if (rowCount === 0) break
updatedItemCount += rowCount
const ids = updated.rows as Array<{ id: number }>
lastId = Number(ids[ids.length - 1]?.id || lastId)
if (rowCount < NORMALIZE_BATCH_SIZE) break
}
// 已有实际成本快照补上商品维度,让毛利能按商品归集。
let updatedSnapshotCount = 0
if (updatedItemCount > 0) {
const updatedSnapshots = await query(
`
UPDATE cost_snapshots cs
SET canonical_product_id = oi.canonical_product_id
FROM order_items oi
WHERE cs.order_item_id = oi.id
AND cs.canonical_product_id IS NULL
AND oi.fulfillment_mode = 'manual'
AND oi.canonical_product_id IS NOT NULL
`,
)
updatedSnapshotCount = updatedSnapshots.rowCount || 0
}
let resolvedExceptionCount = 0
if (input.resolveExceptions !== false && updatedItemCount > 0) {
const resolved = await query(
`
UPDATE cost_exceptions ce
SET status = 'resolved', resolved_at = $1::timestamptz
WHERE ce.status = 'open'
AND ce.subject_type = 'order_item'
AND ce.reason = 'product_mapping_missing'
AND EXISTS (
SELECT 1 FROM order_items oi
WHERE oi.id = ce.subject_id AND oi.canonical_product_id IS NOT NULL
)
`,
[now],
)
resolvedExceptionCount = resolved.rowCount || 0
}
const remainingRange = buildRangeClause(1)
const remaining = await query<{ total: number }>(
`
SELECT COUNT(*)::int AS total
FROM order_items oi
${joinSql}
WHERE oi.fulfillment_mode = 'manual'
AND oi.canonical_product_id IS NULL
${remainingRange.sql}
`,
remainingRange.params,
)
return {
dryRun: false,
range: hasRange ? { fromDate: fromDate || '', toDate: toDate || '' } : null,
distinctSkuCount: candidates.length,
pendingItemCount,
createdProductCount,
updatedItemCount,
updatedSnapshotCount,
resolvedExceptionCount,
remainingItemCount: Number(remaining.rows[0]?.total || 0),
}
}
function normalizeDate(value: unknown): string {
const text = String(value || '').trim()
if (!text) return ''
if (!/^\d{4}-\d{2}-\d{2}$/.test(text)) return ''
return text
}
@@ -3,11 +3,12 @@ import {
syncActualWorkOrderCostSnapshots,
syncEstimatedOrderItemCostSnapshots,
} from '../cost-center/cost-snapshot-sync-service.js'
import { normalizeManualProducts } from '../cost-center/manual-product-normalization-service.js'
import { rebuildSalesDailyFacts } from '../finance/sales-facts-service.js'
import { reconcilePendingFulfillmentModes } from '../fulfillment/fulfillment-mode-service.js'
/**
* 财务闭环同步:履约方式兜底 -> 预估成本快照 -> 工单实际快照 -> 销售事实重算。
* 财务闭环同步:履约方式兜底 -> 人工商品归一 -> 预估成本快照 -> 工单实际快照 -> 销售事实重算。
* 只处理最近 N 天,避免定时任务扫全量历史。
*/
export async function runCostFinanceSyncJob(job: JsonObject = {}) {
@@ -19,6 +20,12 @@ export async function runCostFinanceSyncJob(job: JsonObject = {}) {
maxBatches: 10,
reason: 'cost_finance_sync',
})
// 人工单按内部 SKU 归一;异常收口交给预估同步统一处理,这里只补商品维度。
const manualProducts = await normalizeManualProducts({
dateFrom: shiftDays(-(days - 1)),
dateTo: today(),
resolveExceptions: false,
})
const estimated = await syncEstimatedOrderItemCostSnapshots({
dateFrom: shiftDays(-(days - 1)),
@@ -37,12 +44,15 @@ export async function runCostFinanceSyncJob(job: JsonObject = {}) {
status: 'ok',
message:
`履约方式 ${fulfillment.updatedCount} 项 / ` +
`人工商品 ${manualProducts.updatedItemCount} 项 / ` +
`成本快照 ${estimated.createdCount} 商品预估 / ` +
`${actual.directRewardCount + actual.upgradedCount} 工单实际 / ` +
`${actual.estimatedRewardCount} 工单在途,销售事实 ${facts.insertedCount} 行`,
days,
fulfillmentModeUpdated: fulfillment.updatedCount,
fulfillmentModeRemaining: fulfillment.remainingCount,
manualItemsNormalized: manualProducts.updatedItemCount,
manualProductsCreated: manualProducts.createdProductCount,
estimatedCreated: estimated.createdCount,
actualCreated: actual.directRewardCount + actual.upgradedCount,
estimatedRewardCreated: actual.estimatedRewardCount,
@@ -733,6 +733,15 @@ export function ChannelMappingPanel() {
/>
)}
{unmappedGroups.some((row) => row.fulfillmentMode === 'manual') ? (
<Alert
type="warning"
showIcon
message="人工客服的未归一建议按内部 SKU 批量归一"
description="人工单的内部 SKU(快手 skuNick / 拼多多 outSkuSn)和平台 sku_code 不是同一个值,手建渠道映射通常抓不到历史单。请到「标准商品 → 归一人工商品」,按内部 SKU 生成 man: 商品并回填。"
/>
) : null}
<Space wrap>
<Input
allowClear
@@ -38,6 +38,7 @@ import {
createAdminProductCatalog,
deleteAdminProductCatalog,
fetchAdminProductCatalog,
normalizeAdminManualProducts,
previewAdminTemplateProductBootstrap,
updateAdminProductCatalog,
} from '@/services/admin'
@@ -66,6 +67,7 @@ export function ProductCatalogPanel() {
const [bootstrapPreviewing, setBootstrapPreviewing] = useState(false)
const [bootstrapApplying, setBootstrapApplying] = useState(false)
const [backfillingWorkOrderSkus, setBackfillingWorkOrderSkus] = useState(false)
const [normalizingManualProducts, setNormalizingManualProducts] = useState(false)
const [productPage, setProductPage] = useState(1)
const [productPageSize, setProductPageSize] = useState(ADMIN_DEFAULT_PAGE_SIZE)
const [productModalOpen, setProductModalOpen] = useState(false)
@@ -227,6 +229,52 @@ export function ProductCatalogPanel() {
}
}
async function normalizeManualProductCatalog() {
setNormalizingManualProducts(true)
try {
const preview = await normalizeAdminManualProducts({ dryRun: true })
const data = preview.data
if (data.distinctSkuCount === 0) {
message.success('没有待归一的人工订单项')
return
}
modal.confirm({
title: '归一人工商品',
content: (
<Space direction="vertical" size={8}>
<span>
按内部 SKU(快手 skuNick / 拼多多 outSkuSn)新建最多{' '}
<strong>{data.distinctSkuCount}</strong> 个 <code>man:</code> 标准商品。
</span>
<span>
把 <strong>{data.pendingItemCount}</strong>{' '}
个人工订单项归一到对应商品,并补齐已有成本快照的商品维度。
</span>
<span className="muted">已归一过的订单项不会改动;重复执行是幂等的。</span>
</Space>
),
okText: '开始归一',
cancelText: '取消',
onOk: async () => {
const response = await normalizeAdminManualProducts({})
const result = response.data
message.success(
`已新建 ${result.createdProductCount} 个商品、归一 ${result.updatedItemCount} 个订单项${
result.resolvedExceptionCount > 0
? `、收口 ${result.resolvedExceptionCount} 条异常`
: ''
},剩余 ${result.remainingItemCount} 个未归一`,
)
refreshAll()
},
})
} catch (error) {
message.error(error instanceof Error ? error.message : '归一人工商品失败')
} finally {
setNormalizingManualProducts(false)
}
}
function openCreateProduct() {
setEditingProduct(null)
productForm.resetFields()
@@ -463,6 +511,13 @@ export function ProductCatalogPanel() {
>
按工单回填商品
</Button>
<Button
icon={<SyncOutlined />}
loading={normalizingManualProducts}
onClick={() => void normalizeManualProductCatalog()}
>
归一人工商品
</Button>
<Button
icon={<ReloadOutlined />}
loading={productsQuery.isFetching}
@@ -132,6 +132,20 @@ export function fetchAdminChannelProductMappings(params?: Record<string, unknown
return apiGet<ChannelProductMappingListResult>('/api/v1/admin/cost-center/mappings', params)
}
export function normalizeAdminManualProducts(payload: Record<string, unknown>) {
return apiPost<{
dryRun: boolean
range: { fromDate: string; toDate: string } | null
distinctSkuCount: number
pendingItemCount: number
createdProductCount: number
updatedItemCount: number
updatedSnapshotCount: number
resolvedExceptionCount: number
remainingItemCount: number
}>('/api/v1/admin/cost-center/manual-products/normalize', payload)
}
export function fetchAdminChannelMappingDiagnostics(params?: Record<string, unknown>) {
return apiGet<ChannelMappingDiagnostics>('/api/v1/admin/cost-center/mappings/diagnostics', params)
}
+31
View File
@@ -112,6 +112,37 @@ AND (
)
```
### 4.1 渠道映射诊断与未归一队列
| 接口 | 说明 |
| --- | --- |
| `GET /api/v1/admin/cost-center/mappings/diagnostics` | 标准商品数 / 映射数 / 有映射商品数 / 未归一订单项 / 未归一 SKU 组;按 `fulfillment_mode` 拆分聚合 |
| `GET /api/v1/admin/cost-center/mappings/unmapped` | 未归一队列,按「来源 + 店铺 + 平台 SKU + 履约方式」聚合,支持 `keyword` / `provider` / `fulfillmentMode` 与分页 |
| `POST /api/v1/admin/cost-center/mappings/backfill` | 回填订单项映射;返回 `resolvedExceptionCount`(已归一的 `product_mapping_missing` 异常自动收口) |
### 4.2 人工商品归一(P1)
`POST /api/v1/admin/cost-center/manual-products/normalize`,`finance.config`。
```jsonc
// 请求(全部可选;dryRun=true 只预览)
{ "dateFrom": "2026-07-10", "dateTo": "2026-10-14", "dryRun": false }
// 返回
{
"distinctSkuCount": 1116, // 待处理的内部 SKU 数
"pendingItemCount": 91606, // 待归一的订单项
"createdProductCount": 1116, // 新建的 man: 商品
"updatedItemCount": 91606, // 回填的订单项
"updatedSnapshotCount": 4, // 补了商品维度的成本快照
"resolvedExceptionCount": 91606,
"remainingItemCount": 0
}
```
键的取值:`spec_json->>'skuNick'`(快手)→ `spec_json->>'outSkuSn'`(拼多多)→ `sku_code` 兜底;
商品 `canonical_sku = man:<内部SKU>`,`canonical_name` 优先取 `manual_sku_registry.display_name`,否则取订单里最常见的 `sku_name`。
每日 `cost_finance_sync` 任务会对最近 N 天执行同一归一(不重复收口异常)。
## 五、成本版本接口
路径不变,字段不变:
@@ -151,17 +151,17 @@
| --- | --- | --- |
| P0 | 统一履约事实(`fulfillment_mode`)+ 成本模型正交 | **已完成(本地,未部署生产)** |
| P0.5 | 人工成本链路收敛为「人工录单 → 扣备用金 → 写快照」(下线登记/审批/报销) | **已完成(本地,未部署生产)** |
| P1 | 人工商品归一(`man:<skuId>`)+ 上线后人工成本全覆盖 | 未实施 |
| P1 | 人工商品归一(`man:<skuId>`)+ 上线后人工成本全覆盖 | **已完成(本地,未部署生产)** |
| P2 | 退款冲减与异常闭环 | 未实施 |
| P3 | 影子总账与对账 | 未实施 |
P0 的具体迁移、回填、代码清单与验收 SQL 见 [成本中心与履约模型](成本中心与履约模型.md);接口契约见 [成本中心API契约](成本中心API契约.md)。
> 下一步只做 P1,不要同时改退款或总账。
> 下一步只做 P2,不要同时改总账。
---
## 七、当前实施进度(2026-10-23)
## 七、当前实施进度(2026-10-24 更新)
| 能力 | 状态 | 证据 |
| --- | --- | --- |
@@ -176,12 +176,13 @@ P0 的具体迁移、回填、代码清单与验收 SQL 见 [成本中心与履
| 成本模型与履约方式正交(去 `carrier_type`) | **已实施(本地库已改造)** | 迁移 `103–107` 重写;`cost_items.cost_model` |
| 12 个迁移合并为 5 个 | **已实施** | `103_product_catalog` ~ `107_order_item_fulfillment_mode` |
| 人工成本链路收敛为「人工录单 → 扣备用金 → 写快照」 | **已实施(本地库已改造)** | 迁移 `108_drop_manual_expense_records`;下线登记单与审批报销 |
| 人工商品归一 `man:<skuId>` | **未实施** | P1 |
| 人工商品归一 `man:<skuId>` | **已实施(本地库已归一)** | `manual-product-normalization-service` + `POST /cost-center/manual-products/normalize`;1,116 个 `man:` 商品、91,606 个人工订单项归一,人工未归一 = 0 |
| 人工商品归一后的销售事实 | **已实施** | 94 天分 30 天窗口重算:26,507 行;人工 ¥1,219.20 万收入全部按 `man:` 商品归集 |
| 退款冲减、异常闭环 | **未实施** | §六 P2 |
| 影子总账、对账、关账 | **未实施** | §六 P3 |
> 分支事实:以上实现已快进合并到 `dev`(tip `43f402cf`);迁移 `103–109` **未推送远端、未部署生产**。
> 生产口径仍以 `main` 为准。
> 历史平台仍有 53,901 个订单项(¥389.12 万)未归一,不属于 P1 范围;人工历史成本按既定口径不补录,因此人工毛利当前是“收入已归商品、成本仅新单有”的状态。
> 分支事实:以上实现已快进合并到 `dev`;迁移 `103–111` **未推送远端、未部署生产**。生产口径仍以 `main` 为准。
---