initial: project docs and design spec
This commit is contained in:
103
docs/superpowers/specs/2026-04-21-order-bak-design.md
Normal file
103
docs/superpowers/specs/2026-04-21-order-bak-design.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# Order Backup CLI — Design Spec
|
||||
|
||||
## Overview
|
||||
|
||||
CLI tool to back up completed (Recived) orders from production Azure Table by time range to CSV files, then optionally delete backed-up records. Operates on production storage, so all operations are throttled to minimize performance impact.
|
||||
|
||||
## Scale
|
||||
|
||||
- ~12,000 orders/day
|
||||
- ~6.5M total rows (1.5 years)
|
||||
- Backup targets: Recived orders only (state=1)
|
||||
|
||||
## Configuration
|
||||
|
||||
File: `~/.config/order-bak/config.toml`
|
||||
|
||||
```toml
|
||||
[azure]
|
||||
connection_string = "DefaultEndpointsProtocol=..."
|
||||
|
||||
[scan]
|
||||
page_size = 500
|
||||
delay_ms = 200
|
||||
scan_dir = "./scan"
|
||||
|
||||
[delete]
|
||||
batch_size = 100
|
||||
delay_ms = 300
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### `scan`
|
||||
|
||||
```
|
||||
order-bak scan
|
||||
```
|
||||
|
||||
Full table scan with continuation token pagination. No server-side filter — save all orders. Save results to `scan_dir/scan-<date>.csv`.
|
||||
|
||||
- Single-threaded, sequential with configurable page size and delay
|
||||
- Default: 500 rows/page, 200ms inter-page delay
|
||||
- Estimated time: ~43 minutes for 6.5M rows
|
||||
|
||||
### `backup`
|
||||
|
||||
```
|
||||
order-bak backup --start 2025-01-01 --end 2025-06-30
|
||||
```
|
||||
|
||||
Read from local scan CSV file (default: latest `scan-*.csv` in `scan_dir`, overridable with `--scan <path>`), filter by `state == Recived` and `OrderTime` in `[start, end)`, output one CSV per date under `backups/`.
|
||||
|
||||
- Pure local operation, no Azure access
|
||||
- Print summary after completion: total matched, per-day distribution
|
||||
|
||||
### `delete`
|
||||
|
||||
```
|
||||
order-bak delete --csv backups/2025-01-01.csv
|
||||
```
|
||||
|
||||
Read PartitionKey and RowKey from CSV. Group by PartitionKey, batch delete using Azure Table Transactions (max 100 per transaction, same partition). Print record count and require user confirmation before deleting.
|
||||
|
||||
## Output Directory Structure
|
||||
|
||||
```
|
||||
scan/
|
||||
└── scan-2026-04-21.csv # scan output (all orders)
|
||||
|
||||
backups/
|
||||
├── 2025-01-01.csv # backup output (per-date CSV)
|
||||
├── 2025-01-02.csv
|
||||
└── ...
|
||||
```
|
||||
|
||||
All CSV files share the same columns: `PartitionKey, RowKey, OrderTime, ReceiveTime, revenue, platformOrder, prodCount, pfId, prodId, prodPrice, state, ver, customData`
|
||||
|
||||
## Azure Table Schema (reference)
|
||||
|
||||
Table name: `order`
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| PartitionKey | string | User ID (pfId) |
|
||||
| RowKey | string | Order ID |
|
||||
| OrderTime | DateTime | Order creation time (UTC) |
|
||||
| ReceiveTime | DateTime | Confirmation time (UTC) |
|
||||
| revenue | double | Charged RMB amount |
|
||||
| platformOrder | string | Platform order ID |
|
||||
| prodCount | int | Product count |
|
||||
| pfId | string | Platform ID |
|
||||
| prodId | string | Product ID |
|
||||
| prodPrice | double | Product price |
|
||||
| state | byte | 0=Payed, 1=Recived |
|
||||
| ver | int | Schema version |
|
||||
| customData | string | JSON blob |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- Python, managed with `uv` (installable via `uv tool`)
|
||||
- Azure Data Tables SDK (`azure-data-tables`)
|
||||
- TOML config via `tomllib` (stdlib in Python 3.11+)
|
||||
- CSV output via `csv` stdlib module
|
||||
Reference in New Issue
Block a user