|
|
|
|
@ -0,0 +1,415 @@
|
|
|
|
|
# AmazingData 金融数据服务平台 - 系统文档
|
|
|
|
|
|
|
|
|
|
## 一、系统概述
|
|
|
|
|
|
|
|
|
|
AmazingData 金融数据服务平台是一个基于 FastAPI + Vue3 的金融数据管理系统,提供股票、期货等金融数据的查询、缓存和管理功能。
|
|
|
|
|
|
|
|
|
|
### 技术栈
|
|
|
|
|
|
|
|
|
|
| 组件 | 技术 | 版本 |
|
|
|
|
|
|------|------|------|
|
|
|
|
|
| 后端框架 | FastAPI | 0.100+ |
|
|
|
|
|
| 前端框架 | Vue 3 + Element Plus | 3.x |
|
|
|
|
|
| 数据库 | PostgreSQL / SQLite | 15+ / 3.x |
|
|
|
|
|
| 缓存 | Redis | 7.x |
|
|
|
|
|
| SDK | AmazingData | 最新版 |
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 二、数据库设计
|
|
|
|
|
|
|
|
|
|
### 2.1 数据库初始化
|
|
|
|
|
|
|
|
|
|
系统使用 SQLAlchemy ORM,支持 PostgreSQL 和 SQLite。数据库初始化脚本位于 `database/init.sql`。
|
|
|
|
|
|
|
|
|
|
### 2.2 主要数据表结构
|
|
|
|
|
|
|
|
|
|
#### 用户表 (users)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE users (
|
|
|
|
|
id SERIAL PRIMARY KEY,
|
|
|
|
|
username VARCHAR(50) UNIQUE NOT NULL,
|
|
|
|
|
password VARCHAR(255) NOT NULL,
|
|
|
|
|
email VARCHAR(100),
|
|
|
|
|
role VARCHAR(20) DEFAULT 'user',
|
|
|
|
|
is_active BOOLEAN DEFAULT TRUE,
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
|
|
|
);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### SDK配置表 (sdk_configs)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE sdk_configs (
|
|
|
|
|
id SERIAL PRIMARY KEY,
|
|
|
|
|
name VARCHAR(100) NOT NULL,
|
|
|
|
|
username VARCHAR(100) NOT NULL,
|
|
|
|
|
password VARCHAR(100) NOT NULL,
|
|
|
|
|
host VARCHAR(100) NOT NULL,
|
|
|
|
|
port INTEGER DEFAULT 8600,
|
|
|
|
|
local_path VARCHAR(255),
|
|
|
|
|
is_default BOOLEAN DEFAULT FALSE,
|
|
|
|
|
is_active BOOLEAN DEFAULT TRUE,
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
|
|
|
);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 缓存任务表 (cache_tasks)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE cache_tasks (
|
|
|
|
|
id SERIAL PRIMARY KEY,
|
|
|
|
|
task_name VARCHAR(200) NOT NULL,
|
|
|
|
|
task_type VARCHAR(50) NOT NULL,
|
|
|
|
|
security_type VARCHAR(20) NOT NULL,
|
|
|
|
|
period_type VARCHAR(10),
|
|
|
|
|
start_date DATE NOT NULL,
|
|
|
|
|
end_date DATE NOT NULL,
|
|
|
|
|
code_list TEXT,
|
|
|
|
|
status VARCHAR(20) DEFAULT 'pending',
|
|
|
|
|
progress DECIMAL(5,2) DEFAULT 0,
|
|
|
|
|
total_count INTEGER DEFAULT 0,
|
|
|
|
|
success_count INTEGER DEFAULT 0,
|
|
|
|
|
error_count INTEGER DEFAULT 0,
|
|
|
|
|
error_message TEXT,
|
|
|
|
|
created_by INTEGER REFERENCES users(id),
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
started_at TIMESTAMP,
|
|
|
|
|
completed_at TIMESTAMP,
|
|
|
|
|
|
|
|
|
|
CONSTRAINT cache_tasks_task_type_check
|
|
|
|
|
CHECK (task_type IN ('detect_missing', 'cache_data', 'sync_data', 'detect_all_missing', 'cache_all_data', 'fill_missing_data'))
|
|
|
|
|
);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 缓存任务详情表 (cache_task_details)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE cache_task_details (
|
|
|
|
|
id BIGSERIAL PRIMARY KEY,
|
|
|
|
|
task_id INTEGER REFERENCES cache_tasks(id) ON DELETE CASCADE,
|
|
|
|
|
code VARCHAR(20) NOT NULL,
|
|
|
|
|
trade_date DATE NOT NULL,
|
|
|
|
|
expected_count INTEGER DEFAULT 0,
|
|
|
|
|
actual_count INTEGER DEFAULT 0,
|
|
|
|
|
is_missing BOOLEAN DEFAULT FALSE,
|
|
|
|
|
status VARCHAR(20) DEFAULT 'pending',
|
|
|
|
|
error_message TEXT,
|
|
|
|
|
processed_at TIMESTAMP,
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
|
|
|
);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 股票日线K线表 (stock_kline_daily)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE stock_kline_daily (
|
|
|
|
|
id BIGSERIAL PRIMARY KEY,
|
|
|
|
|
code VARCHAR(20) NOT NULL,
|
|
|
|
|
trade_date DATE NOT NULL,
|
|
|
|
|
open DECIMAL(10,3),
|
|
|
|
|
high DECIMAL(10,3),
|
|
|
|
|
low DECIMAL(10,3),
|
|
|
|
|
close DECIMAL(10,3),
|
|
|
|
|
volume BIGINT,
|
|
|
|
|
amount DECIMAL(18,2),
|
|
|
|
|
adj_factor DECIMAL(10,5),
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
|
|
|
|
|
UNIQUE(code, trade_date)
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
CREATE INDEX idx_stock_kline_code ON stock_kline_daily(code);
|
|
|
|
|
CREATE INDEX idx_stock_kline_date ON stock_kline_daily(trade_date);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 期货日线K线表 (future_kline_daily)
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
CREATE TABLE future_kline_daily (
|
|
|
|
|
id BIGSERIAL PRIMARY KEY,
|
|
|
|
|
code VARCHAR(20) NOT NULL,
|
|
|
|
|
trade_date DATE NOT NULL,
|
|
|
|
|
open DECIMAL(10,3),
|
|
|
|
|
high DECIMAL(10,3),
|
|
|
|
|
low DECIMAL(10,3),
|
|
|
|
|
close DECIMAL(10,3),
|
|
|
|
|
volume BIGINT,
|
|
|
|
|
amount DECIMAL(18,2),
|
|
|
|
|
settle DECIMAL(10,3),
|
|
|
|
|
open_interest BIGINT,
|
|
|
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
|
|
|
|
|
|
UNIQUE(code, trade_date)
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
CREATE INDEX idx_future_kline_code ON future_kline_daily(code);
|
|
|
|
|
CREATE INDEX idx_future_kline_date ON future_kline_daily(trade_date);
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 三、Redis 缓存设计
|
|
|
|
|
|
|
|
|
|
### 3.1 Redis 客户端
|
|
|
|
|
|
|
|
|
|
Redis 客户端位于 `app/core/redis_client.py`,提供以下方法:
|
|
|
|
|
|
|
|
|
|
| 方法 | 功能 | 参数 |
|
|
|
|
|
|------|------|------|
|
|
|
|
|
| `get(key)` | 获取缓存 | key: 缓存键 |
|
|
|
|
|
| `set(key, value, expire)` | 设置缓存 | key, value, expire(秒) |
|
|
|
|
|
| `delete(key)` | 删除缓存 | key: 缓存键 |
|
|
|
|
|
| `exists(key)` | 检查键是否存在 | key: 缓存键 |
|
|
|
|
|
| `is_connected()` | 检查连接状态 | 无 |
|
|
|
|
|
|
|
|
|
|
### 3.2 缓存键设计
|
|
|
|
|
|
|
|
|
|
| 缓存键 | 数据内容 | 有效期 | 使用位置 |
|
|
|
|
|
|--------|----------|--------|----------|
|
|
|
|
|
| `code_list:{type}:{contract}` | 代码列表 | 12小时 | `cache_service.get_all_codes()` |
|
|
|
|
|
| `future_varieties` | 期货品种列表 | 24小时 | `cache_service.get_future_varieties()` |
|
|
|
|
|
| `trading_calendar:{market}` | 交易日历 | 1年 | `cache_service.get_trading_calendar_cached()` |
|
|
|
|
|
| `main_contract:{variety}` | 单个主力合约 | 1小时 | `cache_service.get_main_contract_cached()` |
|
|
|
|
|
| `all_main_contracts` | 所有主力合约 | 1小时 | `cache_service.get_all_main_contracts_cached()` |
|
|
|
|
|
|
|
|
|
|
### 3.3 Redis 使用位置汇总
|
|
|
|
|
|
|
|
|
|
| 文件 | 方法 | 缓存键 | 有效期 |
|
|
|
|
|
|------|------|--------|--------|
|
|
|
|
|
| `cache_service.py` | `get_all_codes()` | `code_list:{type}:{contract}` | 12小时 |
|
|
|
|
|
| `cache_service.py` | `get_future_varieties()` | `future_varieties` | 24小时 |
|
|
|
|
|
| `cache_service.py` | `get_trading_calendar_cached()` | `trading_calendar:{market}` | 1年 |
|
|
|
|
|
| `cache_service.py` | `get_main_contract_cached()` | `main_contract:{variety}` | 1小时 |
|
|
|
|
|
| `cache_service.py` | `get_all_main_contracts_cached()` | `all_main_contracts` | 1小时 |
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 四、部署指南
|
|
|
|
|
|
|
|
|
|
### 4.1 环境要求
|
|
|
|
|
|
|
|
|
|
- Python 3.11+
|
|
|
|
|
- Node.js 18+
|
|
|
|
|
- PostgreSQL 15+ (或 SQLite 3.x)
|
|
|
|
|
- Redis 7+
|
|
|
|
|
|
|
|
|
|
### 4.2 后端部署
|
|
|
|
|
|
|
|
|
|
#### 1. 安装依赖
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
cd backend
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 2. 配置环境变量
|
|
|
|
|
|
|
|
|
|
创建 `.env` 文件:
|
|
|
|
|
|
|
|
|
|
```env
|
|
|
|
|
DATABASE_URL=postgresql://user:password@localhost:5432/amazing_data
|
|
|
|
|
REDIS_URL=redis://localhost:6379/0
|
|
|
|
|
SECRET_KEY=your-secret-key
|
|
|
|
|
DEBUG=False
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 3. 初始化数据库
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# PostgreSQL
|
|
|
|
|
psql -U postgres -d amazing_data -f database/init.sql
|
|
|
|
|
|
|
|
|
|
# 或使用 SQLAlchemy 自动创建
|
|
|
|
|
python -c "from app.db.base import Base; from app.db.session import engine; Base.metadata.create_all(bind=engine)"
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 4. 启动服务
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# 开发模式
|
|
|
|
|
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
|
|
|
|
|
|
|
|
|
# 生产模式
|
|
|
|
|
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### 4.3 前端部署
|
|
|
|
|
|
|
|
|
|
#### 1. 安装依赖
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
cd frontend
|
|
|
|
|
npm install
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 2. 配置环境
|
|
|
|
|
|
|
|
|
|
创建 `.env` 文件:
|
|
|
|
|
|
|
|
|
|
```env
|
|
|
|
|
VITE_API_BASE_URL=http://localhost:8000/api/v1
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 3. 开发模式
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
npm run dev
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 4. 生产构建
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
npm run build
|
|
|
|
|
npm run preview
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### 4.4 Docker 部署 (可选)
|
|
|
|
|
|
|
|
|
|
创建 `docker-compose.yml`:
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
version: '3.8'
|
|
|
|
|
|
|
|
|
|
services:
|
|
|
|
|
postgres:
|
|
|
|
|
image: postgres:15
|
|
|
|
|
environment:
|
|
|
|
|
POSTGRES_DB: amazing_data
|
|
|
|
|
POSTGRES_USER: admin
|
|
|
|
|
POSTGRES_PASSWORD: password
|
|
|
|
|
volumes:
|
|
|
|
|
- postgres_data:/var/lib/postgresql/data
|
|
|
|
|
ports:
|
|
|
|
|
- "5432:5432"
|
|
|
|
|
|
|
|
|
|
redis:
|
|
|
|
|
image: redis:7
|
|
|
|
|
ports:
|
|
|
|
|
- "6379:6379"
|
|
|
|
|
|
|
|
|
|
backend:
|
|
|
|
|
build: ./backend
|
|
|
|
|
ports:
|
|
|
|
|
- "8000:8000"
|
|
|
|
|
depends_on:
|
|
|
|
|
- postgres
|
|
|
|
|
- redis
|
|
|
|
|
environment:
|
|
|
|
|
DATABASE_URL: postgresql://admin:password@postgres:5432/amazing_data
|
|
|
|
|
REDIS_URL: redis://redis:6379/0
|
|
|
|
|
|
|
|
|
|
frontend:
|
|
|
|
|
build: ./frontend
|
|
|
|
|
ports:
|
|
|
|
|
- "3000:3000"
|
|
|
|
|
depends_on:
|
|
|
|
|
- backend
|
|
|
|
|
|
|
|
|
|
volumes:
|
|
|
|
|
postgres_data:
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
启动:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
docker-compose up -d
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 五、系统配置
|
|
|
|
|
|
|
|
|
|
### 5.1 配置文件
|
|
|
|
|
|
|
|
|
|
配置文件位于 `backend/app/config.py`:
|
|
|
|
|
|
|
|
|
|
| 配置项 | 默认值 | 说明 |
|
|
|
|
|
|--------|--------|------|
|
|
|
|
|
| `DATABASE_URL` | sqlite:///./amazing_data.db | 数据库连接 |
|
|
|
|
|
| `REDIS_URL` | redis://localhost:6379/0 | Redis连接 |
|
|
|
|
|
| `SECRET_KEY` | your-secret-key | JWT密钥 |
|
|
|
|
|
| `ACCESS_TOKEN_EXPIRE_HOURS` | 24 | Token有效期 |
|
|
|
|
|
| `CACHE_DEFAULT_DAYS` | 365 | 默认缓存天数 |
|
|
|
|
|
| `CACHE_BATCH_SIZE` | 100 | 批量处理大小 |
|
|
|
|
|
|
|
|
|
|
### 5.2 SDK 配置
|
|
|
|
|
|
|
|
|
|
通过管理界面配置 SDK 连接参数:
|
|
|
|
|
|
|
|
|
|
- 用户名
|
|
|
|
|
- 密码
|
|
|
|
|
- 服务器地址
|
|
|
|
|
- 端口号
|
|
|
|
|
- 本地路径
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 六、监控与维护
|
|
|
|
|
|
|
|
|
|
### 6.1 日志
|
|
|
|
|
|
|
|
|
|
日志配置位于 `backend/app/core/logging.py`,日志文件存储在 `logs/` 目录。
|
|
|
|
|
|
|
|
|
|
### 6.2 健康检查
|
|
|
|
|
|
|
|
|
|
- API健康检查: `GET /health`
|
|
|
|
|
- Redis连接检查: `redis_client.is_connected()`
|
|
|
|
|
- SDK连接检查: `sdk_manager.get_default_connection().is_connected()`
|
|
|
|
|
|
|
|
|
|
### 6.3 缓存清理
|
|
|
|
|
|
|
|
|
|
定期清理过期缓存:
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
# 清理代码列表缓存
|
|
|
|
|
redis_client.delete("code_list:stock:all")
|
|
|
|
|
|
|
|
|
|
# 清理交易日历缓存
|
|
|
|
|
redis_client.delete("trading_calendar:SH")
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 七、安全注意事项
|
|
|
|
|
|
|
|
|
|
1. **JWT密钥**: 生产环境必须更换 `SECRET_KEY`
|
|
|
|
|
2. **数据库密码**: 使用强密码
|
|
|
|
|
3. **Redis密码**: 生产环境建议启用 Redis 认证
|
|
|
|
|
4. **HTTPS**: 生产环境建议启用 HTTPS
|
|
|
|
|
5. **API限流**: 建议添加请求限流中间件
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 八、常见问题
|
|
|
|
|
|
|
|
|
|
### Q1: Redis连接失败
|
|
|
|
|
|
|
|
|
|
检查 Redis 服务是否启动:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
redis-cli ping
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Q2: SDK连接失败
|
|
|
|
|
|
|
|
|
|
检查 SDK 配置是否正确,用户名密码是否有效。
|
|
|
|
|
|
|
|
|
|
### Q3: 数据库连接失败
|
|
|
|
|
|
|
|
|
|
检查数据库服务是否启动,连接字符串是否正确。
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 九、版本历史
|
|
|
|
|
|
|
|
|
|
| 版本 | 日期 | 更新内容 |
|
|
|
|
|
|------|------|----------|
|
|
|
|
|
| 1.0.0 | 2024-01 | 初始版本 |
|
|
|
|
|
| 1.1.0 | 2024-04 | 添加Redis缓存支持 |
|
|
|
|
|
| 1.2.0 | 2024-04 | 添加一键检测/补齐功能 |
|