Lxy
/
market-data-service
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: 初始化代码，目前实现go和python两种后台方式，前端提供admin配置及测试页面，其余全是接口访问

Browse Source
master
Lxy 3 months ago
commit 7cf4848f81
83 changed files with 19688 additions and 0 deletions
Show Stats Download Patch File Download Diff File

712
DEPLOY.md
Unescape View File

@ -0,0 +1,712 @@
# 统一行情数据服务 - 部署文档
本文档详细说明行情数据服务的部署方式，支持 **Go** 和 **Python** 两种实现。
## 实现方式选择
| 特性 | Go实现 | Python实现 |
|------|--------|------------|
| 性能 | ⭐⭐⭐ 高性能 | ⭐⭐ 良好 |
| 开发效率 | ⭐⭐ 中等 | ⭐⭐⭐ 高 |
| 数据源对接 | ⭐⭐ 需自行封装 | ⭐⭐⭐ Tushare原生支持 |
| 生态丰富度 | ⭐⭐ 中等 | ⭐⭐⭐ 丰富 |
| 部署复杂度 | ⭐⭐⭐ 单二进制 | ⭐⭐ 依赖较多 |
**推荐**:
- 生产环境高并发场景选择 **Go实现**
- 快速原型、数据源对接优先选择 **Python实现**
---
# 一、Go 实现部署
## 1.1 部署架构
```
┌─────────────────────────────────────────────────────────────┐
│ 负载均衡器 (可选) │
│ Nginx/HAProxy │
└─────────────────────┬───────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
┌───────▼──────┐ ┌────▼─────┐ ┌────▼─────┐
│ 服务实例1 │ │ 服务实例2 │ │ 服务实例3 │
│ :8080 │ │ :8080 │ │ :8080 │
└───────┬──────┘ └────┬─────┘ └────┬─────┘
│ │ │
└─────────────┼────────────┘
│
┌─────────────▼──────────────┐
│ PostgreSQL 主从 │
│ (分区表 + 读写分离) │
└────────────────────────────┘
```
## 1.2 环境要求
### 服务器配置
| 组件 | 最低配置 | 推荐配置 |
|------|---------|---------|
| 应用服务 | 2核4G | 4核8G |
| PostgreSQL | 4核8G | 8核16G |
| 存储 | 200GB SSD | 500GB SSD |
| 网络 | 10Mbps | 100Mbps |
### 软件依赖
- **操作系统**: Ubuntu 20.04+ / CentOS 8+ / Debian 11+
- **Go**: 1.21+
- **PostgreSQL**: 15+ (支持分区表)
- **Nginx**: 1.18+ (可选，用于负载均衡)
## 1.3 部署步骤
### 安装依赖
```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y postgresql-15 postgresql-contrib-15 nginx
# 安装Go 1.21
wget https://go.dev/dl/go1.21.6.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
```
### 配置PostgreSQL
```bash
# 启动PostgreSQL
sudo systemctl start postgresql
sudo systemctl enable postgresql
# 创建数据库和用户
sudo -u postgres psql -c "CREATE USER marketdata WITH PASSWORD 'your_password';"
sudo -u postgres psql -c "CREATE DATABASE marketdata OWNER marketdata;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE marketdata TO marketdata;"
# 启用分区表支持
sudo -u postgres psql -d marketdata -c "CREATE EXTENSION IF NOT EXISTS pg_partman;"
# 配置允许远程连接 (可选)
sudo sed -i "s/#listen_addresses = 'localhost'/listen_addresses = '*'/g" /etc/postgresql/15/main/postgresql.conf
echo "host all all 0.0.0.0/0 scram-sha-256" | sudo tee -a /etc/postgresql/15/main/pg_hba.conf
sudo systemctl restart postgresql
```
### 部署应用
```bash
# 创建应用目录
sudo mkdir -p /opt/market-data-service
sudo chown $USER:$USER /opt/market-data-service
# 解压项目
tar -xzf market-data-service.tar.gz -C /opt/
cd /opt/market-data-service
# 下载依赖
go mod download
# 构建
make build-all
# 创建配置文件目录
mkdir -p /opt/market-data-service/config
mkdir -p /opt/market-data-service/logs
```
### 配置环境变量
创建 `/opt/market-data-service/.env`:
```bash
# 应用配置
export PORT=8080
export GIN_MODE=release
export API_KEY=your_secure_api_key_here
# 数据库配置
export DATABASE_URL="postgres://marketdata:your_password@localhost:5432/marketdata?sslmode=disable"
# Tushare配置
export TUSHARE_TOKEN=your_tushare_token_here
# 日志配置
export LOG_LEVEL=info
export LOG_FILE=/opt/market-data-service/logs/app.log
```
### 初始化数据库
```bash
cd /opt/market-data-service
source .env
# 执行数据库初始化脚本
psql $DATABASE_URL -f memory/2026-03-07-database-schema.sql
# 同步基础数据
cd /opt/market-data-service
./bin/market-data-sync -type stocks
./bin/market-data-sync -type futures
./bin/market-data-sync -type calendar -start 20240101 -end 20241231
```
### 创建Systemd服务
创建 `/etc/systemd/system/market-data-service.service`:
```ini
[Unit]
Description=Market Data Service
After=network.target postgresql.service
[Service]
Type=simple
User=marketdata
Group=marketdata
WorkingDirectory=/opt/market-data-service
EnvironmentFile=/opt/market-data-service/.env
ExecStart=/opt/market-data-service/bin/market-data-service
Restart=always
RestartSec=5
StandardOutput=append:/opt/market-data-service/logs/service.log
StandardError=append:/opt/market-data-service/logs/error.log
[Install]
WantedBy=multi-user.target
```
```bash
# 创建用户
sudo useradd -r -s /bin/false marketdata
sudo chown -R marketdata:marketdata /opt/market-data-service
# 启动服务
sudo systemctl daemon-reload
sudo systemctl enable market-data-service
sudo systemctl start market-data-service
# 查看状态
sudo systemctl status market-data-service
sudo journalctl -u market-data-service -f
```
### Docker部署 (Go)
```dockerfile
# Dockerfile
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o bin/market-data-service ./cmd/server
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /root/
COPY --from=builder /app/bin/market-data-service .
COPY --from=builder /app/config ./config
EXPOSE 8080
CMD ["./market-data-service"]
```
---
# 二、Python 实现部署
## 2.1 部署架构
Python实现的部署架构与Go相同，但运行环境需要Python解释器和依赖库。
```
┌─────────────────────────────────────────────────────────────┐
│ Nginx (反向代理) │
└─────────────────────┬───────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
┌───────▼──────┐ ┌────▼─────┐ ┌────▼─────┐
│ Python服务1 │ │ Python │ │ Python │
│ Uvicorn │ │ 服务2 │ │ 服务3 │
│ :8080 │ │ :8081 │ │ :8082 │
└───────┬──────┘ └────┬─────┘ └────┬─────┘
│ │ │
└─────────────┼────────────┘
│
┌─────────────▼──────────────┐
│ PostgreSQL │
└────────────────────────────┘
```
## 2.2 环境要求
- **操作系统**: Ubuntu 20.04+ / CentOS 8+ / Windows 10+
- **Python**: 3.10+
- **PostgreSQL**: 15+
- **Nginx**: 1.18+ (可选)
## 2.3 部署步骤
### 安装Python环境
```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y python3.10 python3.10-venv python3-pip postgresql-15 nginx
# CentOS/RHEL
sudo yum install -y python310 python310-pip postgresql15-server nginx
```
### 创建虚拟环境
```bash
# 创建应用目录
sudo mkdir -p /opt/python-market-data-service
sudo chown $USER:$USER /opt/python-market-data-service
cd /opt/python-market-data-service
# 创建虚拟环境
python3.10 -m venv venv
# 激活虚拟环境
source venv/bin/activate
# 升级pip
pip install --upgrade pip
```
### 安装依赖
```bash
# 复制项目文件
cp -r /path/to/python_market_data_service/* /opt/python-market-data-service/
# 安装依赖
pip install -r requirements.txt
# 安装Tushare（需单独安装）
pip install tushare
```
### 配置环境变量
创建 `/opt/python-market-data-service/.env`:
```bash
# 应用配置
export PORT=8080
export DATABASE_URL="postgresql://marketdata:your_password@localhost:5432/marketdata"
export TUSHARE_TOKEN="your_tushare_token"
export API_KEY="your_api_key"
# Python路径
export PYTHONPATH=/opt/python-market-data-service
```
### 初始化数据库
```bash
source venv/bin/activate
source .env
# 使用Python初始化数据库（SQLAlchemy会自动创建表）
python -c "from app.repositories.database import init_db; init_db()"
# 或使用SQL脚本
psql $DATABASE_URL -f memory/2026-03-07-database-schema.sql
```
### 同步基础数据
```bash
source venv/bin/activate
source .env
# 同步股票列表
python scripts/sync_data.py --type stocks
# 同步期货列表
python scripts/sync_data.py --type futures
# 同步交易日历
python scripts/sync_data.py --type calendar --start 20240101 --end 20241231
```
### 启动服务
**方式1: 直接启动（开发模式）**
```bash
source venv/bin/activate
python -m app.main
```
**方式2: 使用Uvicorn（生产模式）**
```bash
source venv/bin/activate
# 单进程
uvicorn app.main:app --host 0.0.0.0 --port 8080
# 多进程（推荐）
uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
```
**方式3: 使用Gunicorn（更高性能）**
```bash
source venv/bin/activate
# 4个worker
gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8080
```
### 创建Systemd服务 (Python)
创建 `/etc/systemd/system/python-market-data-service.service`:
```ini
[Unit]
Description=Python Market Data Service
After=network.target postgresql.service
[Service]
Type=simple
User=marketdata
Group=marketdata
WorkingDirectory=/opt/python-market-data-service
Environment=PATH=/opt/python-market-data-service/venv/bin
EnvironmentFile=/opt/python-market-data-service/.env
ExecStart=/opt/python-market-data-service/venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
StandardOutput=append:/opt/python-market-data-service/logs/service.log
StandardError=append:/opt/python-market-data-service/logs/error.log
[Install]
WantedBy=multi-user.target
```
```bash
# 创建用户和日志目录
sudo useradd -r -s /bin/false marketdata
sudo mkdir -p /opt/python-market-data-service/logs
sudo chown -R marketdata:marketdata /opt/python-market-data-service
# 启动服务
sudo systemctl daemon-reload
sudo systemctl enable python-market-data-service
sudo systemctl start python-market-data-service
# 查看状态
sudo systemctl status python-market-data-service
sudo journalctl -u python-market-data-service -f
```
### Docker部署 (Python)
创建 `Dockerfile`:
```dockerfile
# Dockerfile for Python
FROM python:3.10-slim
WORKDIR /app
# 安装系统依赖
RUN apt-get update && apt-get install -y \
gcc \
postgresql-client \
&& rm -rf /var/lib/apt/lists/*
# 复制依赖文件
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install tushare
# 复制应用代码
COPY . .
# 暴露端口
EXPOSE 8080
# 启动命令
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
```
创建 `docker-compose.yml`:
```yaml
version: '3.8'
services:
app:
build: .
ports:
- "8080:8080"
environment:
- PORT=8080
- DATABASE_URL=postgresql://marketdata:password@postgres:5432/marketdata
- TUSHARE_TOKEN=${TUSHARE_TOKEN}
- API_KEY=${API_KEY}
depends_on:
- postgres
restart: always
volumes:
- ./logs:/app/logs
postgres:
image: postgres:15-alpine
environment:
- POSTGRES_USER=marketdata
- POSTGRES_PASSWORD=password
- POSTGRES_DB=marketdata
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
restart: always
volumes:
postgres_data:
```
```bash
# 构建并启动
docker-compose up -d
# 查看日志
docker-compose logs -f app
# 同步数据
docker-compose exec app python scripts/sync_data.py --type stocks
```
---
# 三、Nginx配置（通用）
## 3.1 配置Nginx
创建 `/etc/nginx/sites-available/market-data-service`:
```nginx
upstream market_data_backend {
server 127.0.0.1:8080;
# 多实例时添加
# server 127.0.0.1:8081;
# server 127.0.0.1:8082;
}
server {
listen 80;
server_name api.marketdata.example.com;
# WebSocket支持
location /v1/stream {
proxy_pass http://market_data_backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-API-Key $http_x_api_key;
proxy_read_timeout 86400;
}
# REST API
location /v1/ {
proxy_pass http://market_data_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-API-Key $http_x_api_key;
# 限流
limit_req zone=api_limit burst=20 nodelay;
}
# 管理后台
location /admin {
proxy_pass http://market_data_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# 健康检查
location /health {
proxy_pass http://market_data_backend/v1/admin/health;
access_log off;
}
}
# 限流配置
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
```
```bash
sudo ln -s /etc/nginx/sites-available/market-data-service /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx
```
---
# 四、监控与维护（通用）
## 4.1 查看服务状态
```bash
# Go服务
sudo systemctl status market-data-service
# Python服务
sudo systemctl status python-market-data-service
# 实时日志
sudo tail -f /opt/market-data-service/logs/service.log
# 或
sudo tail -f /opt/python-market-data-service/logs/service.log
# 数据库连接数
psql $DATABASE_URL -c "SELECT count(*) FROM pg_stat_activity;"
```
## 4.2 备份策略
```bash
#!/bin/bash
# 数据库备份脚本
BACKUP_DIR=/opt/backups
DATE=$(date +%Y%m%d_%H%M%S)
pg_dump $DATABASE_URL | gzip > $BACKUP_DIR/marketdata_$DATE.sql.gz
# 保留最近7天备份
find $BACKUP_DIR -name "marketdata_*.sql.gz" -mtime +7 -delete
```
## 4.3 水平扩展
**Go:**
```bash
PORT=8081 ./bin/market-data-service &
PORT=8082 ./bin/market-data-service &
PORT=8083 ./bin/market-data-service &
```
**Python:**
```bash
# 使用不同端口启动多个实例
uvicorn app.main:app --host 0.0.0.0 --port 8081 &
uvicorn app.main:app --host 0.0.0.0 --port 8082 &
uvicorn app.main:app --host 0.0.0.0 --port 8083 &
```
---
# 五、故障排查
## 5.1 常见问题
**问题1**: 服务无法启动，提示数据库连接失败
```bash
# 检查PostgreSQL状态
sudo systemctl status postgresql
# 测试连接
psql $DATABASE_URL -c "SELECT 1;"
```
**问题2**: Python服务依赖安装失败
```bash
# 检查Python版本
python3 --version # 需 >= 3.10
# 安装系统依赖（Ubuntu）
sudo apt install -y python3-dev libpq-dev gcc
# 重新安装
pip install -r requirements.txt --force-reinstall
```
**问题3**: WebSocket连接断开
```bash
# 检查连接数
netstat -an | grep :8080 | wc -l
# 查看错误日志
sudo tail -f /opt/market-data-service/logs/error.log
# 或
sudo tail -f /opt/python-market-data-service/logs/service.log
```
**问题4**: 数据同步失败
```bash
# Go
./bin/market-data-sync -type stocks 2>&1 | head -20
# Python
python scripts/sync_data.py --type stocks 2>&1 | head -20
```
---
# 六、性能优化
## 6.1 数据库优化
```sql
-- 添加额外索引
CREATE INDEX CONCURRENTLY idx_klines_symbol_ts ON stock.klines_1m(symbol_id, ts DESC);
```
## 6.2 连接池配置
**Go:**
```bash
export DB_MAX_OPEN_CONNS=100
export DB_MAX_IDLE_CONNS=10
```
**Python:**
```python
# 在 app/repositories/database.py 中修改
engine = create_engine(
config.database.database_url,
pool_size=10,
max_overflow=20,
pool_pre_ping=True,
)
```
## 6.3 查询缓存
高频查询结果可缓存到Redis，Python实现已预留Redis配置。
---
# 七、安全建议
1. **API Key管理**: 定期更换API Key，使用环境变量存储
2. **数据库安全**: 使用强密码，限制远程访问，启用SSL
3. **网络安全**: 配置防火墙，仅开放必要端口
4. **日志脱敏**: 日志中避免输出敏感信息
---
**文档结束**

208
Makefile
Unescape View File

@ -0,0 +1,208 @@
.PHONY: all build run test clean help
# 变量
BINARY_NAME=market-data-service
SYNC_BINARY=market-data-sync
DOCKER_IMAGE=market-data-service:latest
# Python项目路径
PYTHON_DIR=python_market_data_service
VENV_PATH=$(PYTHON_DIR)/venv
PYTHON_BIN=$(VENV_PATH)/bin/python
# Windows下使用不同的路径
ifeq ($(OS),Windows_NT)
PYTHON_BIN=$(VENV_PATH)/Scripts/python
endif
# 默认目标
all: build
# ==================== Go 命令 ====================
# 构建服务
build:
go build -o bin/$(BINARY_NAME) ./cmd/server
# 构建同步工具
build-sync:
go build -o bin/$(SYNC_BINARY) ./cmd/sync
# 构建全部
build-all: build build-sync
# 运行服务（Go）
run:
go run ./cmd/server
# 运行同步工具 - 同步股票列表
sync-stocks:
go run ./cmd/sync -type stocks
# 运行同步工具 - 同步期货列表
sync-futures:
go run ./cmd/sync -type futures
# 运行同步工具 - 同步交易日历
sync-calendar:
go run ./cmd/sync -type calendar -start 20240101 -end 20241231
# 运行同步工具 - 同步K线数据
sync-klines:
go run ./cmd/sync -type klines -symbol 000001.SZ -start 20240301 -end 20240307 -freq 1d
# Go测试
test:
go test -v ./...
# 下载Go依赖
deps:
go mod download
go mod tidy
# ==================== Python 命令 ====================
# 创建Python虚拟环境
py-venv:
cd $(PYTHON_DIR) && python -m venv venv
@echo "虚拟环境已创建，请激活:"
@echo " Windows: $(PYTHON_DIR)\venv\Scripts\activate"
@echo " Linux/Mac: source $(PYTHON_DIR)/venv/bin/activate"
# 安装Python依赖
py-deps:
$(PYTHON_BIN) -m pip install --upgrade pip
$(PYTHON_BIN) -m pip install -r $(PYTHON_DIR)/requirements.txt
$(PYTHON_BIN) -m pip install tushare
# 运行Python服务（开发模式）
py-run:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m app.main
# 运行Python服务（Uvicorn，热重载）
py-run-uvicorn:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m uvicorn app.main:app --reload --port 8080
# 运行Python服务（生产模式）
py-run-prod:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
# Python数据同步 - 股票
py-sync-stocks:
cd $(PYTHON_DIR) && $(PYTHON_BIN) scripts/sync_data.py --type stocks
# Python数据同步 - 期货
py-sync-futures:
cd $(PYTHON_DIR) && $(PYTHON_BIN) scripts/sync_data.py --type futures
# Python数据同步 - 交易日历
py-sync-calendar:
cd $(PYTHON_DIR) && $(PYTHON_BIN) scripts/sync_data.py --type calendar --start 20240101 --end 20241231
# Python数据同步 - K线
py-sync-klines:
cd $(PYTHON_DIR) && $(PYTHON_BIN) scripts/sync_data.py --type klines --symbol 000001.SZ --start 20240301 --end 20240307 --freq 1d
# Python测试
py-test:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m pytest tests/ -v
# Python代码格式化
py-fmt:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m black app/ --line-length 100
# Python类型检查
py-lint:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -m mypy app/ --ignore-missing-imports
# ==================== 数据库命令 ====================
# 初始化数据库（Go方式，使用SQL脚本）
db-init:
psql $(DATABASE_URL) -f memory/2026-03-07-database-schema.sql
# 初始化数据库（Python方式，使用SQLAlchemy）
db-init-py:
cd $(PYTHON_DIR) && $(PYTHON_BIN) -c "from app.repositories.database import init_db; init_db()"
# ==================== Docker 命令 ====================
# Docker构建（Go版本）
docker-build:
docker build -t $(DOCKER_IMAGE) .
# Docker构建（Python版本）
docker-build-py:
docker build -t $(DOCKER_IMAGE)-python -f $(PYTHON_DIR)/Dockerfile $(PYTHON_DIR)
# Docker运行
docker-run:
docker run -d -p 8080:8080 --name market-data $(DOCKER_IMAGE)
# ==================== 其他命令 ====================
# 格式化Go代码
fmt:
go fmt ./...
# Go代码检查
lint:
golangci-lint run
# 清理构建产物
clean:
rm -rf bin/
rm -rf $(PYTHON_DIR)/__pycache__
rm -rf $(PYTHON_DIR)/app/__pycache__
find $(PYTHON_DIR) -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
# 查看帮助
help:
@echo "╔════════════════════════════════════════════════════════════════╗"
@echo "║ 统一行情数据服务 - Makefile 命令帮助 ║"
@echo "╚════════════════════════════════════════════════════════════════╝"
@echo ""
@echo "【Go 命令】"
@echo " make build - 构建 Go 服务二进制文件"
@echo " make build-sync - 构建 Go 数据同步工具"
@echo " make build-all - 构建所有 Go 二进制文件"
@echo " make run - 运行 Go 服务（开发模式）"
@echo " make deps - 下载 Go 依赖"
@echo " make test - 运行 Go 测试"
@echo ""
@echo "【Python 命令】"
@echo " make py-venv - 创建 Python 虚拟环境"
@echo " make py-deps - 安装 Python 依赖"
@echo " make py-run - 运行 Python 服务（直接）"
@echo " make py-run-uvicorn - 运行 Python 服务（Uvicorn热重载）"
@echo " make py-run-prod - 运行 Python 服务（生产模式）"
@echo " make py-test - 运行 Python 测试"
@echo ""
@echo "【数据同步 - Go】"
@echo " make sync-stocks - 同步股票列表"
@echo " make sync-futures - 同步期货列表"
@echo " make sync-calendar - 同步交易日历"
@echo " make sync-klines - 同步K线数据"
@echo ""
@echo "【数据同步 - Python】"
@echo " make py-sync-stocks - 同步股票列表"
@echo " make py-sync-futures - 同步期货列表"
@echo " make py-sync-calendar - 同步交易日历"
@echo " make py-sync-klines - 同步K线数据"
@echo ""
@echo "【数据库】"
@echo " make db-init - 初始化数据库（Go SQL脚本）"
@echo " make db-init-py - 初始化数据库（Python SQLAlchemy）"
@echo ""
@echo "【Docker】"
@echo " make docker-build - 构建 Docker 镜像（Go）"
@echo " make docker-build-py - 构建 Docker 镜像（Python）"
@echo ""
@echo "【其他】"
@echo " make clean - 清理构建产物"
@echo " make fmt - 格式化 Go 代码"
@echo " make py-fmt - 格式化 Python 代码（black）"
@echo " make help - 显示本帮助"
@echo ""
@echo "【快速开始】"
@echo " 1. Go实现: make deps && make run"
@echo " 2. Python实现: make py-venv && make py-deps && make py-run-uvicorn"

395
PROGRESS.md
Unescape View File

@ -0,0 +1,395 @@
# 统一行情数据服务 - 项目进度报告
**项目**: 统一行情数据服务
**版本**: v1.0
**日期**: 2026-03-08
**状态**: P0 + P1 阶段已完成，支持 Go 和 Python 双实现
---
## 一、项目概览
### 1.1 项目目标
建设统一行情数据聚合层，对接多方数据源，提供标准化的数据接口，降低业务方接入成本。
### 1.2 核心功能
- ✅ 多周期K线查询（1m/5m/15m/30m/60m/1d/1w/1month）
- ✅ 股票复权支持（前复权qfq/后复权hfq）
- ✅ 数据源热切换（Tushare适配完成，预留Wind接口）
- ✅ 股票/期货双轨设计
- ✅ WebSocket实时订阅
- ✅ 数据质量监控
- ✅ 交易日历查询
- ✅ 期货合约查询
- ✅ **双语言实现**（Go + Python）
---
## 二、完成进度
### 2.1 整体进度: 75%
```
P0 (基础功能) ████████████████████ 100% 已完成
P1 (期货+适配器) ████████████████████ 100% 已完成
P2 (WebSocket) ████████████████████ 100% 已完成
P3 (监控+管理) ██████████████░░░░░░ 70% 部分完成
Python实现 █████████████████░░░ 90% 已完成
期货主力连续 ░░░░░░░░░░░░░░░░░░░░ 0% 预留未实现
```
### 2.2 双实现状态对比
| 功能模块 | Go实现 | Python实现 | 说明 |
|----------|--------|------------|------|
| HTTP API | ✅ 100% | ✅ 100% | 接口完全一致 |
| WebSocket | ✅ 100% | ✅ 100% | 协议兼容 |
| 数据库访问 | ✅ 100% | ✅ 100% | SQLAlchemy ORM |
| Tushare适配 | ✅ 100% | ✅ 100% | Python原生支持 |
| 配置热加载 | ✅ 100% | ✅ 100% | Pydantic支持 |
| 数据同步工具 | ✅ 100% | ✅ 100% | 功能一致 |
| 管理后台UI | ✅ 100% | ✅ 100% | HTML嵌入 |
| 数据质量监控 | ✅ 100% | ✅ 100% | 功能一致 |
| Docker部署 | ✅ 100% | ✅ 100% | 双版本支持 |
### 2.3 详细清单
#### 数据层 (100%)
| 模块 | Go | Python | 说明 |
|------|----|--------|------|
| PostgreSQL Schema | ✅ | ✅ | stock/futures双Schema，5周期K线表，交易日历表 |
| 分区表设计 | ✅ | ✅ | 按月分区，已验证 |
| 股票Repository | ✅ | ✅ | CRUD + 批量插入 |
| 期货Repository | ✅ | ✅ | CRUD + 持仓量字段 |
| 交易日历存储 | ✅ | ✅ | 双市场独立日历 |
| SQLAlchemy模型 | N/A | ✅ | Python ORM模型 |
#### 适配器层 (90%)
| 模块 | Go | Python | 说明 |
|------|----|--------|------|
| 适配器接口定义 | ✅ | ✅ | DataSourceAdapter + Factory |
| Tushare客户端 | ✅ | ✅ | 支持股票/期货日线+分钟线 |
| Tushare适配器 | ✅ | ✅ | Python使用tushare库 |
| Wind适配器 | ⏳ | ⏳ | 接口预留，待实现 |
| 数据源热切换 | ✅ | ✅ | 配置表 + 切换接口 |
#### 服务层 (95%)
| 模块 | Go | Python | 说明 |
|------|----|--------|------|
| 股票Service | ✅ | ✅ | K线查询、标的列表、交易日历 |
| 期货Service | ✅ | ✅ | K线查询、标的列表、合约查询 |
| 复权计算 | ⏳ | ⏳ | 接口预留，系数表已创建 |
| 主力连续合约 | ❌ | ❌ | 表结构预留，首期不实现 |
| 管理Service | ✅ | ✅ | 数据源切换、健康检查 |
| 配置Service | ✅ | ✅ | 热加载、回调机制 |
| 适配器Service | ✅ | ✅ | 工厂模式管理 |
| 测试Service | ✅ | ✅ | API/WS测试 |
#### API层 (100%)
| 接口 | Go | Python | 方法 | 状态 |
|------|----|--------|------|------|
| 股票K线查询 | ✅ | ✅ | GET /v1/stock/klines/:symbol | ✅ |
| 股票标的列表 | ✅ | ✅ | GET /v1/stock/symbols | ✅ |
| 股票批量查询 | ✅ | ✅ | POST /v1/stock/klines/batch | ✅ |
| 股票交易日历 | ✅ | ✅ | GET /v1/stock/trading-dates | ✅ |
| 期货K线查询 | ✅ | ✅ | GET /v1/futures/klines/:symbol | ✅ |
| 期货标的列表 | ✅ | ✅ | GET /v1/futures/symbols | ✅ |
| 期货批量查询 | ✅ | ✅ | POST /v1/futures/klines/batch | ✅ |
| 期货合约查询 | ✅ | ✅ | GET /v1/futures/contracts | ✅ |
| 期货交易日历 | ✅ | ✅ | GET /v1/futures/trading-dates | ✅ |
| 主力连续K线 | ⏳ | ⏳ | GET /v1/futures/continuous/:underlying | 预留 |
| 数据源状态 | ✅ | ✅ | GET /v1/admin/source/status | ✅ |
| 数据源切换 | ✅ | ✅ | POST /v1/admin/source/switch | ✅ |
| 历史补录 | ✅ | ✅ | POST /v1/admin/backfill | ✅ |
| 健康检查 | ✅ | ✅ | GET /v1/admin/health | ✅ |
| WebSocket订阅 | ✅ | ✅ | WS /v1/stream | ✅ |
#### 基础设施 (90%)
| 模块 | Go | Python | 说明 |
|------|----|--------|------|
| WebSocket Hub | ✅ | ✅ | 支持100并发，心跳保活 |
| 数据质量监控 | ✅ | ✅ | 每日检查，缺失告警 |
| 数据同步工具 | ✅ | ✅ | 支持stocks/futures/calendar/klines |
| API认证 | ✅ | ✅ | X-API-Key Header认证 |
| 限流 | ⏳ | ⏳ | Nginx层可配置 |
| 日志 | ✅ | ✅ | 结构化日志输出 |
| 配置管理 | ✅ | ✅ | 环境变量 + 文件 |
| Docker部署 | ✅ | ✅ | Dockerfile + docker-compose |
---
## 三、项目结构
### Go 项目结构
```
market-data-service/
├── api/ # API接口定义
│ ├── types.go # 类型定义 (580行)
│ ├── router.go # 路由注册 (350行)
│ ├── admin_types.go # 管理后台类型 (270行)
│ └── admin_router.go # 管理后台路由 (680行)
├── internal/
│ ├── handler/ # Handler实现
│ ├── service/ # 业务服务层
│ ├── repository/ # 数据访问层
│ ├── websocket/ # WebSocket服务
│ └── monitor/ # 数据质量监控
├── adapter/ # 数据源适配器
├── cmd/ # 程序入口
├── pkg/ # 公共包
├── docs/ # 文档
└── config.json # 配置文件
总代码量: ~4700行 (Go代码)
```
### Python 项目结构
```
python_market_data_service/
├── app/
│ ├── main.py # FastAPI主应用
│ ├── api/ # API路由
│ │ ├── routes.py # 主要API路由
│ │ └── admin_routes.py # 管理后台路由
│ ├── core/ # 核心模块
│ │ ├── config.py # 配置管理（Pydantic）
│ │ ├── errors.py # 错误定义
│ │ └── logger.py # 日志工具
│ ├── models/ # 数据模型（Pydantic）
│ │ ├── types.py # 基础类型
│ │ └── admin_types.py # 管理后台类型
│ ├── repositories/ # 数据访问层（SQLAlchemy）
│ │ ├── database.py # 数据库连接
│ │ ├── models.py # ORM模型
│ │ ├── stock_repository.py
│ │ └── futures_repository.py
│ ├── services/ # 业务逻辑层
│ │ ├── stock_service.py
│ │ ├── futures_service.py
│ │ ├── admin_service.py
│ │ ├── config_service.py
│ │ ├── adapter_service.py
│ │ └── test_service.py
│ ├── adapters/ # 数据源适配器
│ │ ├── base.py # 适配器基类
│ │ └── tushare_adapter.py
│ ├── websocket/ # WebSocket服务
│ │ └── server.py
│ └── monitor/ # 数据质量监控
│ └── monitor.py
├── scripts/
│ └── sync_data.py # 数据同步工具
├── requirements.txt # Python依赖
├── pyproject.toml # 项目配置
├── config.json # 配置文件（与Go相同）
├── README.md # Python项目说明
└── MIGRATION_GUIDE.md # 迁移对照指南
总代码量: ~5500行 (Python代码)
```
---
## 四、实现方式对比
### 技术栈对照
| 组件 | Go实现 | Python实现 | 对比说明 |
|------|--------|------------|----------|
| Web框架 | Gin | FastAPI | FastAPI自动生成文档 |
| WebSocket | Gorilla | FastAPI原生 | API一致 |
| 数据库 | database/sql | SQLAlchemy | ORM vs 原生SQL |
| 配置 | 自定义JSON | Pydantic Settings | Python类型安全 |
| 模型 | Go structs | Pydantic Models | Python验证更强 |
| 并发 | Goroutines | asyncio | 协程模型类似 |
### 性能对比（预估）
| 指标 | Go | Python | 说明 |
|------|----|--------|------|
| 单核QPS | ~10,000 | ~3,000 | Go编译型优势 |
| 内存占用 | ~50MB | ~100MB | Python解释器开销 |
| 启动时间 | ~100ms | ~2s | Python加载依赖 |
| 开发效率 | 中等 | 高 | Python生态丰富 |
### 推荐使用场景
| 场景 | 推荐实现 | 理由 |
|------|----------|------|
| 生产高并发 | Go | 性能更优，资源占用低 |
| 快速原型 | Python | 开发快，Tushare原生支持 |
| 数据源对接 | Python | Tushare等库生态更好 |
| 学习研究 | Python | 代码更易读，调试方便 |
---
## 五、已知问题与限制
### 5.1 当前限制
1. **复权计算**: 已实现接口，复权系数需从数据源同步
2. **主力连续合约**: 首期不实现，表结构预留
3. **实时Tick**: Tushare不支持实时推送，WebSocket推送需对接其他数据源
4. **限流**: 目前仅在Nginx层配置，应用层未实现细粒度限流
### 5.2 双实现差异
1. **Go**: 需要手动管理连接池和并发
2. **Python**: 依赖SQLAlchemy连接池，异步需async/await
3. **数据同步**: Go使用命令行参数，Python使用argparse
### 5.3 待优化项
1. **查询性能**: 大时间范围查询需添加缓存层
2. **并发处理**: WebSocket Hub可优化为更细粒度的锁
3. **错误处理**: 部分错误信息可更详细
4. **监控完善**: 需添加Prometheus指标暴露
---
## 六、测试覆盖
### 6.1 已测试
- ✅ 数据库连接和基本CRUD（双实现）
- ✅ Tushare API调用（双实现）
- ✅ HTTP API路由（双实现）
- ✅ WebSocket连接和订阅（双实现）
- ✅ 配置热加载（双实现）
### 6.2 待测试
- ⏳ 完整集成测试
- ⏳ 压力测试 (并发查询)
- ⏳ 数据同步大批量测试
- ⏳ 故障恢复测试
- ⏳ 双实现兼容性测试
---
## 七、上线 checklist
### 7.1 必需项 (Blocking)
- [x] 数据库Schema初始化
- [x] 基础数据同步 (股票/期货/日历)
- [x] API接口可用性验证
- [x] 部署文档完成
- [x] Python实现完成
### 7.2 建议项 (Non-blocking)
- [ ] 复权系数数据导入
- [ ] 监控告警配置 (钉钉/邮件)
- [ ] 性能压测报告
- [ ] 灾备方案
- [ ] 运维手册
---
## 八、后续规划
### P3 阶段 (剩余10%)
| 任务 | 工时 | 优先级 | Go | Python |
|------|------|--------|----|--------|
| 复权系数计算完善 | 1天 | 高 | ⏳ | ⏳ |
| 监控告警集成 | 1天 | 中 | ⏳ | ⏳ |
| Prometheus指标 | 1天 | 中 | ⏳ | ⏳ |
| 管理后台UI完善 | 1天 | 低 | ✅ | ✅ |
| 性能测试对比 | 1天 | 中 | ⏳ | ⏳ |
### P4 阶段 (增强)
- Wind数据源适配（双实现）
- Redis缓存层
- 期货主力连续合约实现
- 分布式部署支持
- gRPC接口支持
---
## 九、使用说明
### 9.1 快速启动 (Go)
```bash
# 1. 配置环境变量
export TUSHARE_TOKEN="your_token"
export DATABASE_URL="postgres://..."
# 2. 初始化数据库
psql $DATABASE_URL -f memory/2026-03-07-database-schema.sql
# 3. 同步基础数据
make sync-stocks
make sync-futures
make sync-calendar
# 4. 启动服务
make run
```
### 9.2 快速启动 (Python)
```bash
# 1. 安装依赖
pip install -r requirements.txt
pip install tushare
# 2. 配置环境变量
export TUSHARE_TOKEN="your_token"
export DATABASE_URL="postgresql://..."
# 3. 初始化数据库
python -c "from app.repositories.database import init_db; init_db()"
# 4. 同步基础数据
python scripts/sync_data.py --type stocks
python scripts/sync_data.py --type futures
python scripts/sync_data.py --type calendar
# 5. 启动服务
python -m app.main
# 或
uvicorn app.main:app --reload
```
### 9.3 API调用示例
```bash
# 查询股票K线（接口完全一致）
curl "http://localhost:8080/v1/stock/klines/000001.SZ?start=20250301&end=20250307&freq=1d" \
-H "X-API-Key: your_key"
# WebSocket连接（协议相同）
wscat -H "X-API-Key: your_key" -c ws://localhost:8080/v1/stream
> {"action":"subscribe","symbols":["000001.SZ"]}
```
---
## 十、总结
**当前状态**: 核心功能已完成，Go和Python双实现可用
**主要成果**:
1. ✅ 完整的股票/期货双轨数据服务（Go）
2. ✅ 完整的股票/期货双轨数据服务（Python）
3. ✅ Tushare数据源适配（双实现）
4. ✅ WebSocket实时订阅（双实现）
5. ✅ 数据质量监控（双实现）
6. ✅ 管理后台（双实现）
7. ✅ 完整的部署文档
**双实现价值**:
- **Go**: 生产环境高性能部署
- **Python**: 快速开发、数据源对接、学习研究
- **兼容**: 接口完全一致，可无缝切换
**风险提示**:
1. 复权功能需补充系数数据
2. 未经过大规模压力测试
3. 主力连续合约首期不可用
**建议**:
- 生产环境推荐使用 **Go实现**
- 开发测试可使用 **Python实现**
- 可先在小范围试用，验证稳定性后逐步扩大使用范围
---
**文档结束**

400
QUICKSTART.md
Unescape View File

@ -0,0 +1,400 @@
# 统一行情数据服务 - 快速启动指南
本文档提供最快的启动方式，支持 **Go** 和 **Python** 双实现。
---
## 30秒快速启动（Python）
```bash
# 1. 进入Python项目目录
cd python_market_data_service
# 2. 创建并激活虚拟环境
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# 3. 安装依赖
pip install -r requirements.txt
# 4. 启动服务
python -m app.main
```
访问 http://localhost:8080/admin
---
## 30秒快速启动（Go）
```bash
# 1. 下载依赖
go mod download
# 2. 启动服务
go run ./cmd/server
```
访问 http://localhost:8080/admin
---
## 完整启动步骤
### 前置条件
| 组件 | 版本 | 安装命令 |
|------|------|----------|
| Go | 1.21+ | [下载安装](https://go.dev/dl/) |
| Python | 3.10+ | [下载安装](https://www.python.org/) |
| PostgreSQL | 15+ | `docker run -d -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:15` |
### 方式一：Python实现（推荐开发）
#### 步骤1：环境准备
```bash
cd python_market_data_service
# 创建虚拟环境
python -m venv venv
# 激活虚拟环境
# Windows:
venv\Scripts\activate
# Linux/Mac:
source venv/bin/activate
# 安装依赖
pip install -r requirements.txt
pip install tushare
```
#### 步骤2：配置环境变量
```bash
# 必需
export TUSHARE_TOKEN="your_tushare_token" # 从 https://tushare.pro 获取
export DATABASE_URL="postgresql://postgres:postgres@localhost:5432/marketdata"
# 可选
export PORT=8080
export API_KEY=your-api-key
```
#### 步骤3：初始化数据库
```bash
# 方式1：自动创建表（推荐）
python -c "from app.repositories.database import init_db; init_db()"
# 方式2：使用SQL脚本
psql $DATABASE_URL -f ../memory/2026-03-07-database-schema.sql
```
#### 步骤4：启动服务
```bash
# 开发模式（热重载）
python -m app.main
# 或使用Uvicorn
uvicorn app.main:app --reload --port 8080
# 生产模式
uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
```
#### 步骤5：验证
```bash
# 健康检查
curl http://localhost:8080/v1/admin/health
# 访问管理后台
open http://localhost:8080/admin
# API文档（Python特有）
open http://localhost:8080/docs
```
---
### 方式二：Go实现（推荐生产）
#### 步骤1：环境准备
```bash
# 检查Go版本
go version # 需 >= 1.21
# 设置国内镜像（推荐）
go env -w GOPROXY=https://goproxy.cn,direct
# 下载依赖
go mod download
```
#### 步骤2：配置环境变量
```bash
# 必需
export TUSHARE_TOKEN="your_tushare_token"
export DATABASE_URL="postgres://postgres:postgres@localhost:5432/marketdata?sslmode=disable"
# 可选
export PORT=8080
export GIN_MODE=debug
export CONFIG_PATH=./config.json
```
#### 步骤3：初始化数据库
```bash
# 创建数据库
createdb marketdata
# 执行初始化脚本
psql $DATABASE_URL -f memory/2026-03-07-database-schema.sql
```
#### 步骤4：启动服务
```bash
# 开发模式
go run ./cmd/server
# 或使用Makefile
make run
# 生产模式
make build
./bin/market-data-service
```
#### 步骤5：验证
```bash
# 健康检查
curl http://localhost:8080/v1/admin/health
# 访问管理后台
open http://localhost:8080/admin
```
---
## Docker启动（最简单）
### Python版本
```bash
cd python_market_data_service
# 构建并启动
docker-compose up -d
# 查看日志
docker-compose logs -f app
```
### Go版本
```bash
# 构建镜像
docker build -t market-data-service .
# 运行
docker run -d \
-p 8080:8080 \
-e TUSHARE_TOKEN=your_token \
-e DATABASE_URL=postgres://... \
market-data-service
```
---
## 数据同步
### Python
```bash
# 同步股票列表
python scripts/sync_data.py --type stocks
# 同步期货列表
python scripts/sync_data.py --type futures
# 同步交易日历
python scripts/sync_data.py --type calendar --start 20240101 --end 20241231
# 同步K线数据
python scripts/sync_data.py --type klines --symbol 000001.SZ --start 20240301 --end 20240307 --freq 1d
```
### Go
```bash
# 同步股票列表
go run ./cmd/sync -type stocks
# 同步期货列表
go run ./cmd/sync -type futures
# 同步交易日历
go run ./cmd/sync -type calendar -start 20240101 -end 20241231
# 同步K线数据
go run ./cmd/sync -type klines -symbol 000001.SZ -start 20240301 -end 20240307 -freq 1d
```
---
## 常用命令速查
### 服务管理
| 操作 | Python | Go |
|------|--------|----|
| 启动服务 | `python -m app.main` | `go run ./cmd/server` |
| 热重载 | `uvicorn app.main:app --reload` | `fresh` 或 `air` |
| 生产启动 | `uvicorn app.main:app --workers 4` | `./bin/market-data-service` |
| 后台运行 | `nohup python -m app.main &` | `nohup ./market-server &` |
### 数据库
```bash
# 连接数据库
psql postgresql://postgres:postgres@localhost:5432/marketdata
# 查看表
\dt stock.*
\dt futures.*
# 查看股票列表
SELECT * FROM stock.symbols LIMIT 10;
```
### 测试API
```bash
# 健康检查
curl http://localhost:8080/v1/admin/health
# 查询股票K线
curl "http://localhost:8080/v1/stock/klines/000001.SZ?start=20250301&end=20250307&freq=1d" \
-H "X-API-Key: demo-api-key-2024"
# 热加载配置
curl -X POST http://localhost:8080/v1/admin/system/reload
# WebSocket测试
wscat -c ws://localhost:8080/v1/stream -H "X-API-Key: demo-api-key-2024"
> {"action":"subscribe","symbols":["000001.SZ"]}
```
---
## 故障排查
### 端口被占用
```bash
# 查找占用8080的进程
# Windows:
netstat -ano | findstr :8080
taskkill /PID <PID> /F
# Linux/Mac:
lsof -i :8080
kill -9 <PID>
# 或使用其他端口
# Python: uvicorn app.main:app --port 8081
# Go: PORT=8081 go run ./cmd/server
```
### 数据库连接失败
```bash
# 检查PostgreSQL是否运行
# Windows: 服务管理器查看 postgresql-x64-15
# Linux: sudo systemctl status postgresql
# Docker快速启动PostgreSQL
docker run -d --name postgres \
-e POSTGRES_PASSWORD=postgres \
-p 5432:5432 \
postgres:15
```
### Python依赖问题
```bash
# 升级pip
pip install --upgrade pip
# 重新安装依赖
pip install -r requirements.txt --force-reinstall
# 使用国内镜像
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### Go依赖问题
```bash
# 清理缓存
go clean -cache
# 重新下载
go mod download
# 整理依赖
go mod tidy
```
---
## 生产部署
### 使用Systemd（Linux）
**Python:**
```ini
# /etc/systemd/system/python-market-data.service
[Unit]
Description=Python Market Data Service
After=network.target
[Service]
Type=simple
User=marketdata
WorkingDirectory=/opt/python-market-data-service
Environment=PATH=/opt/python-market-data-service/venv/bin
ExecStart=/opt/python-market-data-service/venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8080
Restart=always
[Install]
WantedBy=multi-user.target
```
**Go:**
```ini
# /etc/systemd/system/market-data.service
[Unit]
Description=Market Data Service
After=network.target
[Service]
Type=simple
User=marketdata
WorkingDirectory=/opt/market-data-service
ExecStart=/opt/market-data-service/bin/market-data-service
Restart=always
[Install]
WantedBy=multi-user.target
```
启用服务：
```bash
sudo systemctl daemon-reload
sudo systemctl enable market-data.service
sudo systemctl start market-data.service
sudo systemctl status market-data.service
```
---
## 下一步
- [完整部署文档](DEPLOY.md) - 详细部署指南
- [开发指南](docs/development-guide.md) - 如何开发新功能
- [API文档](docs/admin-api-quick-reference.md) - API接口参考
---
**提示**: 开发环境推荐Python（启动快、热重载），生产环境推荐Go（性能高、资源省）。

357
README.md
Unescape View File

@ -0,0 +1,357 @@
# 统一行情数据服务
提供股票和期货的标准化行情数据查询服务，支持多周期K线、复权计算、数据源热切换等功能。
**🎉 重大更新**: 现已支持 **Go** 和 **Python** 双实现，所有接口完全一致！
| 实现 | 推荐场景 | 目录 |
|------|----------|------|
| **Go** | 生产环境、高并发 | `market-data-service/` (本目录) |
| **Python** | 快速开发、原型验证 | `python_market_data_service/` |
---
## 特性
- **多周期K线支持**：1m/5m/15m/30m/60m/1d/1w/1month
- **股票复权支持**：前复权(qfq)/后复权(hfq)
- **数据源热切换**：支持Wind、Tushare等多个数据源动态切换
- **双轨设计**：股票和期货接口独立，数据存储隔离
- **WebSocket实时订阅**：支持实时行情推送
- **数据质量监控**：自动检测数据缺失并告警
- **交易日历**：支持查询股票和期货的交易日历
- **期货合约查询**：根据品种获取可交易合约列表
- **双语言实现**：Go高性能 + Python快速开发
---
## 技术栈
### Go实现
- **语言**: Go 1.21+
- **Web框架**: Gin
- **WebSocket**: Gorilla WebSocket
- **数据库**: PostgreSQL 15+ (原生SQL)
- **数据源**: Tushare
### Python实现
- **语言**: Python 3.10+
- **Web框架**: FastAPI
- **WebSocket**: FastAPI原生
- **数据库**: PostgreSQL 15+ (SQLAlchemy ORM)
- **数据源**: Tushare (原生支持)
---
## 项目结构
### Go实现
```
market-data-service/
├── api/ # API接口定义
│ ├── types.go # 请求/响应类型定义
│ ├── router.go # HTTP路由注册
│ ├── admin_types.go # 管理后台类型定义
│ └── admin_router.go # 管理后台路由
├── internal/ # 内部实现
│ ├── handler/ # HTTP Handler实现
│ ├── service/ # 业务逻辑层
│ ├── repository/ # 数据访问层
│ ├── websocket/ # WebSocket服务
│ └── monitor/ # 数据质量监控
├── adapter/ # 数据源适配器框架
│ ├── adapter.go # 适配器接口定义
│ └── tushare/ # Tushare适配器实现
├── pkg/ # 公共包
├── cmd/ # 程序入口
├── config.example.json # 配置文件示例
├── go.mod
├── Makefile
└── README.md
```
### Python实现
```
python_market_data_service/
├── app/
│ ├── main.py # FastAPI主应用
│ ├── api/ # API路由 (FastAPI)
│ ├── core/ # 核心模块 (配置、日志、错误)
│ ├── models/ # 数据模型 (Pydantic)
│ ├── repositories/ # 数据访问 (SQLAlchemy)
│ ├── services/ # 业务逻辑层
│ ├── adapters/ # 数据源适配器
│ ├── websocket/ # WebSocket服务
│ └── monitor/ # 数据质量监控
├── scripts/ # 数据同步工具
├── requirements.txt # Python依赖
├── pyproject.toml # 项目配置
├── config.json # 配置文件 (与Go相同)
├── README.md # Python项目说明
└── MIGRATION_GUIDE.md # 迁移对照指南
```
---
## 快速开始
**最快方式**: 查看 [QUICKSTART.md](QUICKSTART.md) - 30秒启动指南
### 方式一：Go实现（推荐生产环境）
#### 1. 环境准备
- Go 1.21+
- PostgreSQL 15+
- Tushare Token (从 [Tushare官网](https://tushare.pro) 获取)
#### 2. 配置环境变量
```bash
export TUSHARE_TOKEN="your_tushare_token"
export DATABASE_URL="postgres://user:password@localhost:5432/marketdata?sslmode=disable"
export PORT="8080"
export GIN_MODE="debug"
```
#### 3. 初始化数据库
```bash
# 创建数据库
createdb marketdata
# 执行初始化脚本
psql $DATABASE_URL -f memory/2026-03-07-database-schema.sql
```
#### 4. 下载依赖并启动
```bash
make deps
make run
```
### 方式二：Python实现（推荐开发环境）
#### 1. 环境准备
- Python 3.10+
- PostgreSQL 15+
- Tushare Token
#### 2. 安装依赖
```bash
cd python_market_data_service
# 创建虚拟环境
python -m venv venv
source venv/bin/activate # Linux/Mac
# 或
venv\Scripts\activate # Windows
# 安装依赖
pip install -r requirements.txt
pip install tushare
```
#### 3. 配置环境变量
```bash
export TUSHARE_TOKEN="your_tushare_token"
export DATABASE_URL="postgresql://user:password@localhost:5432/marketdata"
export PORT="8080"
```
#### 4. 初始化数据库并启动
```bash
# 初始化数据库（SQLAlchemy自动创建表）
python -c "from app.repositories.database import init_db; init_db()"
# 启动服务
python -m app.main
# 或使用 uvicorn
uvicorn app.main:app --reload --port 8080
```
服务将启动在 `http://localhost:8080`
---
## API接口
> **注意**: Go和Python实现的API接口完全一致！
### 股票接口
| 接口 | 方法 | 说明 |
|------|------|------|
| `/v1/stock/klines/:symbol` | GET | 查询K线数据 |
| `/v1/stock/symbols` | GET | 查询标的列表 |
| `/v1/stock/klines/batch` | POST | 批量查询K线 |
| `/v1/stock/trading-dates` | GET | 获取交易日历 |
**查询K线示例**:
```bash
curl "http://localhost:8080/v1/stock/klines/000001.SZ?start=20250301&end=20250307&freq=1d&adjust=qfq" \
-H "X-API-Key: your_api_key"
```
### 期货接口
| 接口 | 方法 | 说明 |
|------|------|------|
| `/v1/futures/klines/:symbol` | GET | 查询K线数据 |
| `/v1/futures/symbols` | GET | 查询标的列表 |
| `/v1/futures/klines/batch` | POST | 批量查询K线 |
| `/v1/futures/continuous/:underlying` | GET | 查询主力连续合约（预留） |
| `/v1/futures/trading-dates` | GET | 获取交易日历 |
| `/v1/futures/contracts` | GET | 获取品种合约列表 |
### 管理后台
服务启动后，访问 `http://localhost:8080/admin` 进入管理后台。
**Python特有**: 自动生成API文档
- Swagger UI: `http://localhost:8080/docs`
- ReDoc: `http://localhost:8080/redoc`
---
## 数据同步工具
### Go实现
```bash
# 同步股票列表
go run ./cmd/sync -type stocks
# 同步期货列表
go run ./cmd/sync -type futures
# 同步交易日历
go run ./cmd/sync -type calendar -start 20240101 -end 20241231
# 同步K线数据
go run ./cmd/sync -type klines -symbol 000001.SZ -start 20240301 -end 20240307 -freq 1d
```
### Python实现
```bash
# 同步股票列表
python scripts/sync_data.py --type stocks
# 同步期货列表
python scripts/sync_data.py --type futures
# 同步交易日历
python scripts/sync_data.py --type calendar --start 20240101 --end 20241231
# 同步K线数据
python scripts/sync_data.py --type klines --symbol 000001.SZ --start 20240301 --end 20240307 --freq 1d
```
---
## 部署
### Docker部署
**Go实现:**
```bash
docker build -t market-data-service:go .
docker run -p 8080:8080 market-data-service:go
```
**Python实现:**
```bash
cd python_market_data_service
docker build -t market-data-service:python .
docker run -p 8080:8080 market-data-service:python
```
详细部署文档请参考 [DEPLOY.md](./DEPLOY.md)
---
## 实现方式对比
| 特性 | Go | Python |
|------|----|--------|
| 性能 | ⭐⭐⭐ 高性能 | ⭐⭐ 良好 |
| 开发效率 | ⭐⭐ 中等 | ⭐⭐⭐ 高 |
| 数据源生态 | ⭐⭐ 需自行封装 | ⭐⭐⭐ Tushare原生支持 |
| 内存占用 | ⭐⭐⭐ 低 (~50MB) | ⭐⭐ 中等 (~100MB) |
| 启动速度 | ⭐⭐⭐ 快 (~100ms) | ⭐⭐ 中等 (~2s) |
| 类型安全 | ⭐⭐⭐ 编译期检查 | ⭐⭐ 运行时检查 (Pydantic) |
| 部署复杂度 | ⭐⭐⭐ 单二进制 | ⭐⭐ 依赖较多 |
| API文档 | ⭐⭐ 需手动维护 | ⭐⭐⭐ 自动生成 |
**推荐**:
- **生产环境**: Go实现
- **开发测试**: Python实现
- **数据源对接**: Python实现（生态更好）
---
## 文档
### 🚀 启动部署
- [QUICKSTART.md](./QUICKSTART.md) - **30秒快速启动指南**（推荐先看）
- [启动指南](./docs/startup-guide.md) - 完整的启动教程
- [部署文档](./DEPLOY.md) - 详细部署指南（含Docker和Systemd）
### 📚 开发文档
- [开发指南](./docs/development-guide.md) - 开发新功能指南
- [架构设计](./docs/architecture.md) - 系统架构文档
- [API速查表](./docs/admin-api-quick-reference.md) - API快速参考
- [Python迁移指南](./python_market_data_service/MIGRATION_GUIDE.md) - Go到Python迁移对照
### 📋 项目信息
- [项目进度](./PROGRESS.md) - 开发进度和计划
---
## 常见问题
### Q: 应该选择Go还是Python实现？
A:
- 如果需要**生产环境高并发**，选择**Go**
- 如果需要**快速开发、原型验证**，选择**Python**
- 如果需要**频繁对接数据源**，选择**Python**（生态更好）
### Q: 两个实现的API是否兼容？
A: **完全兼容**。所有接口、请求参数、响应格式完全一致，客户端可无缝切换。
### Q: 数据库Schema是否相同？
A: **相同**。两个实现使用相同的数据库Schema，可共享数据。
### Q: 如何实现股票复权？
A: 查询时传入 `adjust=qfq` 或 `adjust=hfq` 参数，服务会自动应用复权系数。
### Q: 期货主力连续合约何时支持？
A: 主力连续合约功能在预留表结构中，将在后续版本实现。
---
## 贡献
欢迎提交Issue和PR！
- Go实现相关: 提交到本仓库
- Python实现相关: 提交到 `python_market_data_service/` 目录
---
## License
MIT

76
adapter/adapter.go
Unescape View File

@ -0,0 +1,76 @@
package adapter
import (
"time"
)
// DataSourceAdapter 数据源适配器接口
type DataSourceAdapter interface {
// Connect 建立连接
Connect(config map[string]string) error
// SubscribeTicks 订阅实时Tick
SubscribeTicks(symbols []string, callback TickCallback) error
// FetchKLines 拉取历史K线
FetchKLines(symbol, start, end, freq string) ([]KLineData, error)
// FetchSymbols 获取标的列表
FetchSymbols(assetType string) ([]SymbolInfo, error)
// FetchTradingCalendar 获取交易日历
FetchTradingCalendar(exchange, start, end string) ([]TradeCalData, error)
// HealthCheck 健康检查
HealthCheck() error
// Close 关闭连接
Close() error
}
// TickCallback Tick数据回调
type TickCallback func(symbol string, tick TickData)
// TickData Tick数据
type TickData struct {
Symbol string
Price float64
Volume int64
Time int64
}
// KLineData K线数据
type KLineData struct {
Symbol string
Time int64
Open float64
High float64
Low float64
Close float64
Volume int64
Amount float64
OpenInterest int64
}
// SymbolInfo 标的信息
type SymbolInfo struct {
SymbolID string
Name string
Exchange string
Underlying string
ContractMonth string
ListDate string
DelistDate string
}
// TradeCalData 交易日历数据
type TradeCalData struct {
Date time.Time
IsTradingDay bool
HasNightSession bool
}
// AdapterFactory 适配器工厂
type AdapterFactory interface {
Create(name string) (DataSourceAdapter, error)
}

293
adapter/tushare/adapter.go
Unescape View File

@ -0,0 +1,293 @@
package tushare
import (
"context"
"fmt"
"strings"
"time"
"market-data-service/adapter"
)
// Adapter Tushare数据源适配器
type Adapter struct {
client *Client
config map[string]string
}
// NewAdapter 创建Tushare适配器
func NewAdapter() *Adapter {
return &Adapter{}
}
// Connect 建立连接
func (a *Adapter) Connect(config map[string]string) error {
token, ok := config["token"]
if !ok || token == "" {
return fmt.Errorf("tushare token is required")
}
a.client = NewClient(token)
if baseURL, ok := config["base_url"]; ok && baseURL != "" {
a.client.SetBaseURL(baseURL)
}
a.config = config
return nil
}
// SubscribeTicks 订阅实时Tick（Tushare不支持实时推送，返回错误）
func (a *Adapter) SubscribeTicks(symbols []string, callback adapter.TickCallback) error {
return fmt.Errorf("tushare does not support real-time tick subscription")
}
// FetchKLines 拉取历史K线
func (a *Adapter) FetchKLines(symbol, start, end, freq string) ([]adapter.KLineData, error) {
// 判断是股票还是期货
if strings.Contains(symbol, ".SH") || strings.Contains(symbol, ".SZ") || strings.Contains(symbol, ".BJ") {
return a.fetchStockKLines(symbol, start, end, freq)
}
return a.fetchFuturesKLines(symbol, start, end, freq)
}
// fetchStockKLines 获取股票K线
func (a *Adapter) fetchStockKLines(symbol, start, end, freq string) ([]adapter.KLineData, error) {
// 转换symbol格式: 000001.SZ -> 000001.SZ
tsCode := symbol
switch freq {
case "1d", "", "day":
return a.fetchStockDaily(tsCode, start, end)
case "1m", "5m", "15m", "30m", "60m":
return a.fetchStockMinute(tsCode, start, end, freq)
default:
return nil, fmt.Errorf("unsupported frequency: %s", freq)
}
}
// fetchStockDaily 获取股票日线
func (a *Adapter) fetchStockDaily(tsCode, start, end string) ([]adapter.KLineData, error) {
data, err := a.client.GetStockDaily(tsCode, start, end)
if err != nil {
return nil, err
}
result := make([]adapter.KLineData, len(data))
for i, d := range data {
tradeDate, _ := time.Parse("20060102", d.TradeDate)
result[i] = adapter.KLineData{
Symbol: d.TSCode,
Time: tradeDate.Unix(),
Open: d.Open,
High: d.High,
Low: d.Low,
Close: d.Close,
Volume: int64(d.Volume * 100), // 手 -> 股
Amount: d.Amount * 1000, // 千元 -> 元
}
}
return result, nil
}
// fetchStockMinute 获取股票分钟线
func (a *Adapter) fetchStockMinute(tsCode, start, end, freq string) ([]adapter.KLineData, error) {
// 去掉'm'后缀
freqNum := strings.TrimSuffix(freq, "m")
data, err := a.client.GetStockMinute(tsCode, start, end, freqNum)
if err != nil {
return nil, err
}
result := make([]adapter.KLineData, len(data))
for i, d := range data {
// 解析时间格式: 2025-03-07 09:31:00
tradeTime, _ := time.Parse("2006-01-02 15:04:05", d.TradeTime)
result[i] = adapter.KLineData{
Symbol: d.TSCode,
Time: tradeTime.Unix(),
Open: d.Open,
High: d.High,
Low: d.Low,
Close: d.Close,
Volume: int64(d.Volume * 100),
Amount: d.Amount * 1000,
}
}
return result, nil
}
// fetchFuturesKLines 获取期货K线
func (a *Adapter) fetchFuturesKLines(symbol, start, end, freq string) ([]adapter.KLineData, error) {
switch freq {
case "1d", "", "day":
return a.fetchFuturesDaily(symbol, start, end)
case "1m", "5m", "15m", "30m", "60m":
return a.fetchFuturesMinute(symbol, start, end, freq)
default:
return nil, fmt.Errorf("unsupported frequency: %s", freq)
}
}
// fetchFuturesDaily 获取期货日线
func (a *Adapter) fetchFuturesDaily(tsCode, start, end string) ([]adapter.KLineData, error) {
data, err := a.client.GetFuturesDaily(tsCode, start, end)
if err != nil {
return nil, err
}
result := make([]adapter.KLineData, len(data))
for i, d := range data {
tradeDate, _ := time.Parse("20060102", d.TradeDate)
result[i] = adapter.KLineData{
Symbol: d.TSCode,
Time: tradeDate.Unix(),
Open: d.Open,
High: d.High,
Low: d.Low,
Close: d.Close,
Volume: int64(d.Volume),
Amount: d.Amount * 10000, // 万元 -> 元
OpenInterest: int64(d.OpenInterest),
}
}
return result, nil
}
// fetchFuturesMinute 获取期货分钟线
func (a *Adapter) fetchFuturesMinute(tsCode, start, end, freq string) ([]adapter.KLineData, error) {
freqNum := strings.TrimSuffix(freq, "m")
data, err := a.client.GetFuturesMinute(tsCode, start, end, freqNum)
if err != nil {
return nil, err
}
result := make([]adapter.KLineData, len(data))
for i, d := range data {
tradeTime, _ := time.Parse("2006-01-02 15:04:05", d.TradeTime)
result[i] = adapter.KLineData{
Symbol: d.TSCode,
Time: tradeTime.Unix(),
Open: d.Open,
High: d.High,
Low: d.Low,
Close: d.Close,
Volume: int64(d.Volume),
Amount: d.Amount * 10000,
OpenInterest: int64(d.OpenInterest),
}
}
return result, nil
}
// FetchSymbols 获取标的列表
func (a *Adapter) FetchSymbols(assetType string) ([]adapter.SymbolInfo, error) {
switch assetType {
case "stock":
return a.fetchStockSymbols()
case "futures":
return a.fetchFuturesSymbols()
default:
return nil, fmt.Errorf("unsupported asset type: %s", assetType)
}
}
// fetchStockSymbols 获取股票列表
func (a *Adapter) fetchStockSymbols() ([]adapter.SymbolInfo, error) {
data, err := a.client.GetStockBasic()
if err != nil {
return nil, err
}
result := make([]adapter.SymbolInfo, 0, len(data))
for _, d := range data {
// 只返回上市状态的
if d.ListStatus != "L" {
continue
}
result = append(result, adapter.SymbolInfo{
SymbolID: d.TSCode,
Name: d.Name,
Exchange: d.Exchange,
})
}
return result, nil
}
// fetchFuturesSymbols 获取期货列表
func (a *Adapter) fetchFuturesSymbols() ([]adapter.SymbolInfo, error) {
data, err := a.client.GetFuturesBasic("")
if err != nil {
return nil, err
}
result := make([]adapter.SymbolInfo, len(data))
for i, d := range data {
result[i] = adapter.SymbolInfo{
SymbolID: d.TSCode,
Name: d.Name,
Exchange: d.Exchange,
}
}
return result, nil
}
// FetchTradingCalendar 获取交易日历
func (a *Adapter) FetchTradingCalendar(exchange, start, end string) ([]adapter.TradeCalData, error) {
// Tushare交易所代码映射
exchangeMap := map[string]string{
"SH": "SSE",
"SZ": "SZSE",
"SHFE": "SHFE",
"DCE": "DCE",
"CZCE": "CZCE",
"CFFEX": "CFFEX",
"INE": "INE",
}
tsExchange, ok := exchangeMap[exchange]
if !ok {
tsExchange = "SSE"
}
data, err := a.client.GetTradeCal(tsExchange, start, end, -1)
if err != nil {
return nil, err
}
result := make([]adapter.TradeCalData, len(data))
for i, d := range data {
calDate, _ := time.Parse("20060102", d.CalDate)
result[i] = adapter.TradeCalData{
Date: calDate,
IsTradingDay: d.IsOpen == 1,
}
}
return result, nil
}
// HealthCheck 健康检查
func (a *Adapter) HealthCheck() error {
if a.client == nil {
return fmt.Errorf("client not initialized")
}
// 尝试获取交易日历作为健康检查
now := time.Now()
start := now.AddDate(0, 0, -7).Format("20060102")
end := now.Format("20060102")
_, err := a.client.GetTradeCal("SSE", start, end, 1)
return err
}
// Close 关闭连接
func (a *Adapter) Close() error {
// Tushare是HTTP接口，无需关闭
return nil
}
// GetClient 获取底层客户端（用于直接调用Tushare API）
func (a *Adapter) GetClient() *Client {
return a.client
}

531
adapter/tushare/client.go
Unescape View File

@ -0,0 +1,531 @@
package tushare
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"time"
)
const (
DefaultBaseURL = "http://api.tushare.pro"
)
// Client Tushare API客户端
type Client struct {
token string
baseURL string
client *http.Client
}
// NewClient 创建Tushare客户端
func NewClient(token string) *Client {
return &Client{
token: token,
baseURL: DefaultBaseURL,
client: &http.Client{Timeout: 30 * time.Second},
}
}
// SetBaseURL 设置基础URL（用于测试）
func (c *Client) SetBaseURL(baseURL string) {
c.baseURL = baseURL
}
// Request 通用请求结构
type Request struct {
APIName string `json:"api_name"`
Token string `json:"token"`
Params map[string]interface{} `json:"params"`
Fields string `json:"fields,omitempty"`
}
// Response 通用响应结构
type Response struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data *Data `json:"data"`
}
// Data 响应数据
type Data struct {
Fields []string `json:"fields"`
Items [][]interface{} `json:"items"`
}
// Error 实现error接口
func (r *Response) Error() string {
return fmt.Sprintf("tushare error: code=%d, msg=%s", r.Code, r.Msg)
}
// IsSuccess 判断是否成功
func (r *Response) IsSuccess() bool {
return r.Code == 0
}
// Query 执行查询
func (c *Client) Query(apiName string, params map[string]interface{}, fields string) (*Response, error) {
reqBody := Request{
APIName: apiName,
Token: c.token,
Params: params,
Fields: fields,
}
jsonData, err := json.Marshal(reqBody)
if err != nil {
return nil, fmt.Errorf("marshal request failed: %w", err)
}
resp, err := c.client.Post(c.baseURL, "application/json", bytes.NewBuffer(jsonData))
if err != nil {
return nil, fmt.Errorf("http request failed: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("read response body failed: %w", err)
}
var result Response
if err := json.Unmarshal(body, &result); err != nil {
return nil, fmt.Errorf("unmarshal response failed: %w", err)
}
if !result.IsSuccess() {
return &result, result
}
return &result, nil
}
// ToMapList 将Data转换为map列表
func (d *Data) ToMapList() []map[string]interface{} {
if d == nil || len(d.Fields) == 0 || len(d.Items) == 0 {
return nil
}
result := make([]map[string]interface{}, len(d.Items))
for i, item := range d.Items {
m := make(map[string]interface{})
for j, field := range d.Fields {
if j < len(item) {
m[field] = item[j]
}
}
result[i] = m
}
return result
}
// StockDaily 股票日线数据
type StockDaily struct {
TSCode string `json:"ts_code"` // 股票代码
TradeDate string `json:"trade_date"` // 交易日期
Open float64 `json:"open"` // 开盘价
High float64 `json:"high"` // 最高价
Low float64 `json:"low"` // 最低价
Close float64 `json:"close"` // 收盘价
PreClose float64 `json:"pre_close"` // 昨收价
Change float64 `json:"change"` // 涨跌额
PctChange float64 `json:"pct_chg"` // 涨跌幅
Volume float64 `json:"vol"` // 成交量（手）
Amount float64 `json:"amount"` // 成交额（千元）
}
// GetStockDaily 获取股票日线数据
func (c *Client) GetStockDaily(tsCode, startDate, endDate string) ([]StockDaily, error) {
params := map[string]interface{}{
"ts_code": tsCode,
"start_date": startDate,
"end_date": endDate,
}
resp, err := c.Query("daily", params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]StockDaily, len(items))
for i, item := range items {
result[i] = StockDaily{
TSCode: getString(item, "ts_code"),
TradeDate: getString(item, "trade_date"),
Open: getFloat64(item, "open"),
High: getFloat64(item, "high"),
Low: getFloat64(item, "low"),
Close: getFloat64(item, "close"),
PreClose: getFloat64(item, "pre_close"),
Change: getFloat64(item, "change"),
PctChange: getFloat64(item, "pct_chg"),
Volume: getFloat64(item, "vol"),
Amount: getFloat64(item, "amount"),
}
}
return result, nil
}
// StockMinute 股票分钟线数据
type StockMinute struct {
TSCode string `json:"ts_code"`
TradeTime string `json:"trade_time"`
Open float64 `json:"open"`
High float64 `json:"high"`
Low float64 `json:"low"`
Close float64 `json:"close"`
Volume float64 `json:"vol"`
Amount float64 `json:"amount"`
}
// GetStockMinute 获取股票分钟线数据
func (c *Client) GetStockMinute(tsCode, startDate, endDate string, freq string) ([]StockMinute, error) {
apiName := "stk_mins" // 默认1分钟
switch freq {
case "5":
apiName = "stk_mins5"
case "15":
apiName = "stk_mins15"
case "30":
apiName = "stk_mins30"
case "60":
apiName = "stk_mins60"
}
params := map[string]interface{}{
"ts_code": tsCode,
"start_date": startDate,
"end_date": endDate,
}
resp, err := c.Query(apiName, params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]StockMinute, len(items))
for i, item := range items {
result[i] = StockMinute{
TSCode: getString(item, "ts_code"),
TradeTime: getString(item, "trade_time"),
Open: getFloat64(item, "open"),
High: getFloat64(item, "high"),
Low: getFloat64(item, "low"),
Close: getFloat64(item, "close"),
Volume: getFloat64(item, "vol"),
Amount: getFloat64(item, "amount"),
}
}
return result, nil
}
// FuturesDaily 期货日线数据
type FuturesDaily struct {
TSCode string `json:"ts_code"`
TradeDate string `json:"trade_date"`
Open float64 `json:"open"`
High float64 `json:"high"`
Low float64 `json:"low"`
Close float64 `json:"close"`
PreClose float64 `json:"pre_close"`
Change float64 `json:"change"`
PctChange float64 `json:"pct_chg"`
Volume float64 `json:"vol"`
Amount float64 `json:"amount"`
OpenInterest float64 `json:"oi"`
OiChange float64 `json:"oi_chg"`
}
// GetFuturesDaily 获取期货日线数据
func (c *Client) GetFuturesDaily(tsCode, startDate, endDate string) ([]FuturesDaily, error) {
params := map[string]interface{}{
"ts_code": tsCode,
"start_date": startDate,
"end_date": endDate,
}
resp, err := c.Query("fut_daily", params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]FuturesDaily, len(items))
for i, item := range items {
result[i] = FuturesDaily{
TSCode: getString(item, "ts_code"),
TradeDate: getString(item, "trade_date"),
Open: getFloat64(item, "open"),
High: getFloat64(item, "high"),
Low: getFloat64(item, "low"),
Close: getFloat64(item, "close"),
PreClose: getFloat64(item, "pre_close"),
Change: getFloat64(item, "change"),
PctChange: getFloat64(item, "pct_chg"),
Volume: getFloat64(item, "vol"),
Amount: getFloat64(item, "amount"),
OpenInterest: getFloat64(item, "oi"),
OiChange: getFloat64(item, "oi_chg"),
}
}
return result, nil
}
// FuturesMinute 期货分钟线数据
type FuturesMinute struct {
TSCode string `json:"ts_code"`
TradeTime string `json:"trade_time"`
Open float64 `json:"open"`
High float64 `json:"high"`
Low float64 `json:"low"`
Close float64 `json:"close"`
Volume float64 `json:"vol"`
Amount float64 `json:"amount"`
OpenInterest float64 `json:"oi"`
}
// GetFuturesMinute 获取期货分钟线数据
func (c *Client) GetFuturesMinute(tsCode, startDate, endDate string, freq string) ([]FuturesMinute, error) {
apiName := "fut_mins" // 默认1分钟
switch freq {
case "5":
apiName = "fut_mins5"
case "15":
apiName = "fut_mins15"
case "30":
apiName = "fut_mins30"
case "60":
apiName = "fut_mins60"
}
params := map[string]interface{}{
"ts_code": tsCode,
"start_date": startDate,
"end_date": endDate,
}
resp, err := c.Query(apiName, params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]FuturesMinute, len(items))
for i, item := range items {
result[i] = FuturesMinute{
TSCode: getString(item, "ts_code"),
TradeTime: getString(item, "trade_time"),
Open: getFloat64(item, "open"),
High: getFloat64(item, "high"),
Low: getFloat64(item, "low"),
Close: getFloat64(item, "close"),
Volume: getFloat64(item, "vol"),
Amount: getFloat64(item, "amount"),
OpenInterest: getFloat64(item, "oi"),
}
}
return result, nil
}
// StockBasic 股票基础信息
type StockBasic struct {
TSCode string `json:"ts_code"`
Symbol string `json:"symbol"`
Name string `json:"name"`
Area string `json:"area"`
Industry string `json:"industry"`
FullName string `json:"fullname"`
EnName string `json:"enname"`
CNName string `json:"cnspell"`
Market string `json:"market"`
Exchange string `json:"exchange"`
CurrType string `json:"curr_type"`
ListStatus string `json:"list_status"`
ListDate string `json:"list_date"`
DelistDate string `json:"delist_date"`
IsHS string `json:"is_hs"`
}
// GetStockBasic 获取股票基础信息
func (c *Client) GetStockBasic() ([]StockBasic, error) {
resp, err := c.Query("stock_basic", map[string]interface{}{"list_status": "L"}, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]StockBasic, len(items))
for i, item := range items {
result[i] = StockBasic{
TSCode: getString(item, "ts_code"),
Symbol: getString(item, "symbol"),
Name: getString(item, "name"),
Area: getString(item, "area"),
Industry: getString(item, "industry"),
FullName: getString(item, "fullname"),
EnName: getString(item, "enname"),
CNName: getString(item, "cnspell"),
Market: getString(item, "market"),
Exchange: getString(item, "exchange"),
CurrType: getString(item, "curr_type"),
ListStatus: getString(item, "list_status"),
ListDate: getString(item, "list_date"),
DelistDate: getString(item, "delist_date"),
IsHS: getString(item, "is_hs"),
}
}
return result, nil
}
// FuturesBasic 期货合约基础信息
type FuturesBasic struct {
TSCode string `json:"ts_code"`
Symbol string `json:"symbol"`
Name string `json:"name"`
Exchange string `json:"exchange"`
FutCode string `json:"fut_code"`
Multiplier float64 `json:"multiplier"`
TradeUnit string `json:"trade_unit"`
PerUnit float64 `json:"per_unit"`
DeliveryDate string `json:"delivery_date"`
ListDate string `json:"list_date"`
DelistDate string `json:"delist_date"`
}
// GetFuturesBasic 获取期货合约基础信息
func (c *Client) GetFuturesBasic(exchange string) ([]FuturesBasic, error) {
params := map[string]interface{}{}
if exchange != "" {
params["exchange"] = exchange
}
resp, err := c.Query("fut_basic", params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]FuturesBasic, len(items))
for i, item := range items {
result[i] = FuturesBasic{
TSCode: getString(item, "ts_code"),
Symbol: getString(item, "symbol"),
Name: getString(item, "name"),
Exchange: getString(item, "exchange"),
FutCode: getString(item, "fut_code"),
Multiplier: getFloat64(item, "multiplier"),
TradeUnit: getString(item, "trade_unit"),
PerUnit: getFloat64(item, "per_unit"),
DeliveryDate: getString(item, "delivery_date"),
ListDate: getString(item, "list_date"),
DelistDate: getString(item, "delist_date"),
}
}
return result, nil
}
// TradeCal 交易日历
type TradeCal struct {
Exchange string `json:"exchange"`
CalDate string `json:"cal_date"`
IsOpen int `json:"is_open"`
PretradeDate string `json:"pretrade_date"`
}
// GetTradeCal 获取交易日历
func (c *Client) GetTradeCal(exchange, startDate, endDate string, isOpen int) ([]TradeCal, error) {
params := map[string]interface{}{
"exchange": exchange,
"start_date": startDate,
"end_date": endDate,
}
if isOpen >= 0 {
params["is_open"] = isOpen
}
resp, err := c.Query("trade_cal", params, "")
if err != nil {
return nil, err
}
items := resp.Data.ToMapList()
result := make([]TradeCal, len(items))
for i, item := range items {
result[i] = TradeCal{
Exchange: getString(item, "exchange"),
CalDate: getString(item, "cal_date"),
IsOpen: getInt(item, "is_open"),
PretradeDate: getString(item, "pretrade_date"),
}
}
return result, nil
}
// FuturesHolding 期货持仓排名
type FuturesHolding struct {
TSCode string `json:"ts_code"`
TradeDate string `json:"trade_date"`
Symbol string `json:"symbol"`
Broker string `json:"broker"`
Vol int64 `json:"vol"`
VolChange int64 `json:"vol_chg"`
LongHld int64 `json:"long_hld"`
LongChange int64 `json:"long_chg"`
ShortHld int64 `json:"short_hld"`
ShortChange int64 `json:"short_chg"`
}
// 辅助函数
func getString(m map[string]interface{}, key string) string {
if v, ok := m[key]; ok && v != nil {
switch val := v.(type) {
case string:
return val
case []byte:
return string(val)
default:
return fmt.Sprintf("%v", v)
}
}
return ""
}
func getFloat64(m map[string]interface{}, key string) float64 {
if v, ok := m[key]; ok && v != nil {
switch val := v.(type) {
case float64:
return val
case int:
return float64(val)
case int64:
return float64(val)
case string:
var f float64
fmt.Sscanf(val, "%f", &f)
return f
}
}
return 0
}
func getInt(m map[string]interface{}, key string) int {
if v, ok := m[key]; ok && v != nil {
switch val := v.(type) {
case int:
return val
case int64:
return int(val)
case float64:
return int(val)
case string:
var i int
fmt.Sscanf(val, "%d", &i)
return i
}
}
return 0
}

1430
api/admin_router.go
View File

File diff suppressed because it is too large Load Diff

254
api/admin_types.go
Unescape View File

@ -0,0 +1,254 @@
// Package api 管理后台相关类型定义
package api
import (
"time"
)
// ============================================
// 配置管理类型
// ============================================
// ConfigType 配置类型
type ConfigType string
const (
ConfigTypeServer ConfigType = "server" // 服务器配置
ConfigTypeDatabase ConfigType = "database" // 数据库配置
ConfigTypeRedis ConfigType = "redis" // Redis配置
ConfigTypeSource ConfigType = "source" // 数据源配置
ConfigTypeMonitor ConfigType = "monitor" // 监控配置
ConfigTypeLog ConfigType = "log" // 日志配置
)
// ConfigItem 配置项
type ConfigItem struct {
Key string `json:"key"` // 配置键
Value interface{} `json:"value"` // 配置值
Type string `json:"type"` // 值类型: string/int/bool/json
Description string `json:"description"` // 配置说明
Editable bool `json:"editable"` // 是否可编辑
Required bool `json:"required"` // 是否必填
}
// ConfigSection 配置分组
type ConfigSection struct {
Name string `json:"name"` // 分组名称
Type ConfigType `json:"type"` // 分组类型
Description string `json:"description"` // 分组说明
Items []ConfigItem `json:"items"` // 配置项列表
}
// ConfigListRequest 获取配置列表请求
type ConfigListRequest struct {
Type ConfigType `json:"type" form:"type"` // 配置类型筛选
}
// ConfigListData 配置列表响应
type ConfigListData struct {
Sections []ConfigSection `json:"sections"` // 配置分组列表
Version string `json:"version"` // 配置版本
Updated time.Time `json:"updated"` // 最后更新时间
}
// ConfigUpdateRequest 更新配置请求
type ConfigUpdateRequest struct {
Type ConfigType `json:"type" validate:"required"` // 配置类型
Items map[string]interface{} `json:"items" validate:"required"` // 更新的配置项
}
// ConfigUpdateData 更新配置响应
type ConfigUpdateData struct {
Success bool `json:"success"` // 是否成功
NeedRestart bool `json:"need_restart"` // 是否需要重启
Message string `json:"message"` // 提示信息
}
// ============================================
// 适配器管理类型
// ============================================
// AdapterInfo 适配器信息
type AdapterInfo struct {
Name string `json:"name"` // 适配器名称
Type string `json:"type"` // 适配器类型
Version string `json:"version"` // 版本
Description string `json:"description"` // 描述
Status AdapterStatus `json:"status"` // 状态
Config map[string]string `json:"config"` // 当前配置
LastError string `json:"last_error,omitempty"` // 最后错误
UpdatedAt time.Time `json:"updated_at"` // 更新时间
}
// AdapterStatus 适配器状态
type AdapterStatus string
const (
AdapterStatusActive AdapterStatus = "active" // 已激活
AdapterStatusStandby AdapterStatus = "standby" // 待命
AdapterStatusDisabled AdapterStatus = "disabled" // 已禁用
AdapterStatusError AdapterStatus = "error" // 错误
)
// AdapterListData 适配器列表响应
type AdapterListData struct {
Adapters []AdapterInfo `json:"adapters"` // 适配器列表
}
// AdapterToggleRequest 启用/禁用适配器请求
type AdapterToggleRequest struct {
Name string `json:"name" validate:"required"` // 适配器名称
Enable bool `json:"enable"` // 是否启用
}
// AdapterConfigUpdateRequest 更新适配器配置请求
type AdapterConfigUpdateRequest struct {
Name string `json:"name" validate:"required"` // 适配器名称
Config map[string]string `json:"config" validate:"required"` // 配置
}
// ============================================
// 系统管理类型
// ============================================
// SystemStatusData 系统状态数据
type SystemStatusData struct {
Status string `json:"status"` // 系统状态: running/stopping/restarting
Version string `json:"version"` // 系统版本
StartTime time.Time `json:"start_time"` // 启动时间
Uptime string `json:"uptime"` // 运行时长
GoVersion string `json:"go_version"` // Go版本
MemoryUsage MemoryInfo `json:"memory"` // 内存使用
Goroutines int `json:"goroutines"` // Goroutine数量
}
// MemoryInfo 内存信息
type MemoryInfo struct {
Alloc uint64 `json:"alloc"` // 已分配内存
TotalAlloc uint64 `json:"total_alloc"` // 累计分配
Sys uint64 `json:"sys"` // 系统内存
NumGC uint32 `json:"num_gc"` // GC次数
}
// RestartRequest 重启服务请求
type RestartRequest struct {
Force bool `json:"force"` // 是否强制重启
}
// ReloadRequest 热加载配置请求
type ReloadRequest struct {
ConfigType ConfigType `json:"config_type"` // 指定加载的配置类型，空表示全部
}
// ReloadData 热加载响应
type ReloadData struct {
Success bool `json:"success"`
Message string `json:"message"`
}
// ============================================
// 接口测试类型
// ============================================
// APITestCase 接口测试用例
type APITestCase struct {
ID string `json:"id"` // 用例ID
Name string `json:"name"` // 用例名称
Method string `json:"method"` // HTTP方法
Path string `json:"path"` // 请求路径
Description string `json:"description"` // 描述
Params map[string]string `json:"params"` // 默认参数
Body interface{} `json:"body"` // 请求体
}
// APITestCategory 测试分类
type APITestCategory struct {
Name string `json:"name"` // 分类名称
Items []APITestCase `json:"items"` // 测试用例
}
// APITestListData 接口测试列表响应
type APITestListData struct {
Categories []APITestCategory `json:"categories"` // 分类列表
BaseURL string `json:"base_url"` // 基础URL
}
// APITestRequest 执行接口测试请求
type APITestRequest struct {
ID string `json:"id" validate:"required"` // 用例ID
Params map[string]string `json:"params"` // 自定义参数
Body interface{} `json:"body"` // 自定义请求体
}
// APITestResult 接口测试结果
type APITestResult struct {
ID int `json:"id"` // 测试ID
CaseID string `json:"case_id"` // 用例ID
Name string `json:"name"` // 用例名称
Success bool `json:"success"` // 是否成功
StatusCode int `json:"status_code"` // HTTP状态码
Latency int64 `json:"latency"` // 延迟(ms)
Request interface{} `json:"request"` // 请求信息
Response interface{} `json:"response"` // 响应信息
Error string `json:"error,omitempty"` // 错误信息
Timestamp time.Time `json:"timestamp"` // 测试时间
}
// ============================================
// WebSocket测试类型
// ============================================
// WSTestCase WebSocket测试用例
type WSTestCase struct {
ID string `json:"id"` // 用例ID
Name string `json:"name"` // 用例名称
Description string `json:"description"` // 描述
Action string `json:"action"` // 动作类型
Symbols []string `json:"symbols"` // 订阅标的
}
// WSTestListData WebSocket测试列表响应
type WSTestListData struct {
Cases []WSTestCase `json:"cases"` // 测试用例
WSURL string `json:"ws_url"` // WebSocket地址
}
// WSTestRequest WebSocket测试请求
type WSTestRequest struct {
ID string `json:"id" validate:"required"` // 用例ID
Symbols []string `json:"symbols"` // 自定义标的
}
// WSTestResult WebSocket测试结果
type WSTestResult struct {
ID string `json:"id"` // 测试ID
CaseID string `json:"case_id"` // 用例ID
Success bool `json:"success"` // 是否成功
Latency int64 `json:"latency"` // 连接延迟(ms)
Messages []WSMessage `json:"messages"` // 收到的消息
Error string `json:"error,omitempty"`
Timestamp time.Time `json:"timestamp"`
}
// WSMessage WebSocket消息
type WSMessage struct {
Type string `json:"type"` // 消息类型
Data interface{} `json:"data"` // 消息内容
Timestamp time.Time `json:"timestamp"` // 时间
}
// ============================================
// 测试历史记录类型
// ============================================
// TestHistoryRequest 获取测试历史请求
type TestHistoryRequest struct {
Type string `json:"type" form:"type"` // 测试类型: api/ws
Limit int `json:"limit" form:"limit"` // 数量限制
}
// TestHistoryData 测试历史数据
type TestHistoryData struct {
APITests []APITestResult `json:"api_tests"` // API测试历史
WSTests []WSTestResult `json:"ws_tests"` // WebSocket测试历史
}

418
api/router.go
Unescape View File

@ -0,0 +1,418 @@
package api
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/gin-gonic/gin/binding"
)
// Router API路由注册
type Router struct {
handler Handler
}
// NewRouter 创建路由
func NewRouter(handler Handler) *Router {
return &Router{handler: handler}
}
// Register 注册所有路由
func (r *Router) Register(engine *gin.Engine) {
// 公开接口（无需认证）
public := engine.Group("/v1")
{
public.GET("/admin/health", r.healthCheck)
}
// 需要认证的接口
api := engine.Group("/v1")
api.Use(r.authMiddleware())
{
// 股票接口
stock := api.Group("/stock")
{
stock.GET("/klines/:symbol", r.queryStockKLines)
stock.GET("/symbols", r.listStockSymbols)
stock.POST("/klines/batch", r.batchQueryStockKLines)
stock.GET("/trading-dates", r.getStockTradingDates)
}
// 期货接口
futures := api.Group("/futures")
{
futures.GET("/klines/:symbol", r.queryFuturesKLines)
futures.GET("/symbols", r.listFuturesSymbols)
futures.POST("/klines/batch", r.batchQueryFuturesKLines)
futures.GET("/continuous/:underlying", r.queryContinuousKLines)
futures.GET("/trading-dates", r.getFuturesTradingDates)
futures.GET("/contracts", r.getFuturesContracts)
}
// 管理接口
admin := api.Group("/admin")
{
admin.GET("/source/status", r.getDataSourceStatus)
admin.POST("/source/switch", r.switchDataSource)
admin.POST("/backfill", r.backfillData)
}
}
}
// authMiddleware API认证中间件
func (r *Router) authMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
apiKey := c.GetHeader("X-API-Key")
if apiKey == "" {
c.JSON(http.StatusUnauthorized, ErrorResponse{
Code: 401,
Message: "缺少API Key",
})
c.Abort()
return
}
// TODO: 验证API Key有效性
// 可以将用户信息存入context
c.Set("api_key", apiKey)
c.Next()
}
}
// ============================================
// 股票接口实现
// ============================================
func (r *Router) queryStockKLines(c *gin.Context) {
var req KLineQueryRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
req.Symbol = c.Param("symbol")
resp, err := r.handler.QueryKLines(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) listStockSymbols(c *gin.Context) {
var req SymbolListRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.ListSymbols(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) batchQueryStockKLines(c *gin.Context) {
var req BatchKLineRequest
if err := c.ShouldBindBodyWith(&req, binding.JSON); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.BatchQueryKLines(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
// ============================================
// 期货接口实现
// ============================================
func (r *Router) queryFuturesKLines(c *gin.Context) {
var req KLineQueryRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
req.Symbol = c.Param("symbol")
resp, err := r.handler.QueryKLines(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) listFuturesSymbols(c *gin.Context) {
var req SymbolListRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.ListSymbols(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) batchQueryFuturesKLines(c *gin.Context) {
var req BatchKLineRequest
if err := c.ShouldBindBodyWith(&req, binding.JSON); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.BatchQueryKLines(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) queryContinuousKLines(c *gin.Context) {
underlying := c.Param("underlying")
var req KLineQueryRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.QueryContinuousKLines(c.Request.Context(), underlying, &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
// ============================================
// 管理接口实现
// ============================================
func (r *Router) getDataSourceStatus(c *gin.Context) {
resp, err := r.handler.GetDataSourceStatus(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) switchDataSource(c *gin.Context) {
var req SourceSwitchRequest
if err := c.ShouldBindBodyWith(&req, binding.JSON); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.SwitchDataSource(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, ErrorResponse{
Code: 422,
Message: "数据源切换失败",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) backfillData(c *gin.Context) {
var req BackfillRequest
if err := c.ShouldBindBodyWith(&req, binding.JSON); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.BackfillData(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "补录任务失败",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusAccepted, resp)
}
func (r *Router) healthCheck(c *gin.Context) {
resp, err := r.handler.HealthCheck(c.Request.Context())
if err != nil {
c.JSON(http.StatusServiceUnavailable, ErrorResponse{
Code: 503,
Message: "服务不可用",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
// ============================================
// 新增接口：交易日历和期货合约
// ============================================
func (r *Router) getStockTradingDates(c *gin.Context) {
var req TradingDatesRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.GetTradingDates(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) getFuturesTradingDates(c *gin.Context) {
var req TradingDatesRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.GetTradingDates(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}
func (r *Router) getFuturesContracts(c *gin.Context) {
var req FuturesContractsRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{
Code: 400,
Message: "参数错误",
Detail: err.Error(),
})
return
}
resp, err := r.handler.GetContractsByUnderlying(c.Request.Context(), &req)
if err != nil {
c.JSON(http.StatusInternalServerError, ErrorResponse{
Code: 500,
Message: "服务器内部错误",
Detail: err.Error(),
})
return
}
c.JSON(http.StatusOK, resp)
}

372
api/types.go
Unescape View File

@ -0,0 +1,372 @@
// Package api 定义HTTP API接口
package api
import (
"context"
"errors"
"time"
)
// 错误定义
var (
ErrRateLimit = errors.New("rate limit exceeded")
)
// ============================================
// 基础类型定义
// ============================================
// Frequency K线周期
type Frequency string
const (
Freq1M Frequency = "1m"
Freq5M Frequency = "5m"
Freq15M Frequency = "15m"
Freq30M Frequency = "30m"
Freq60M Frequency = "60m"
Freq1D Frequency = "1d"
Freq1W Frequency = "1w"
Freq1Month Frequency = "1month"
)
// AdjustType 复权类型
type AdjustType string
const (
AdjustNone AdjustType = "" // 不复权
AdjustQFQ AdjustType = "qfq" // 前复权
AdjustHFQ AdjustType = "hfq" // 后复权
)
// AssetClass 资产类别
type AssetClass string
const (
AssetStock AssetClass = "stock"
AssetFutures AssetClass = "futures"
AssetAll AssetClass = "all"
)
// SymbolType 标的类型
type SymbolType string
const (
SymbolTypeStock SymbolType = "stock"
SymbolTypeFutures SymbolType = "futures"
SymbolTypeIndex SymbolType = "index"
SymbolTypeETF SymbolType = "etf"
SymbolTypeContinuous SymbolType = "continuous"
)
// Exchange 交易所
type Exchange string
const (
// 股票交易所
ExchangeSZ Exchange = "SZ" // 深交所
ExchangeSH Exchange = "SH" // 上交所
ExchangeBJ Exchange = "BJ" // 北交所
// 期货交易所
ExchangeCFFEX Exchange = "CFFEX" // 中金所
ExchangeSHFE Exchange = "SHFE" // 上期所
ExchangeDCE Exchange = "DCE" // 大商所
ExchangeCZCE Exchange = "CZCE" // 郑商所
ExchangeINE Exchange = "INE" // 上期能源
ExchangeGFEX Exchange = "GFEX" // 广期所
)
// DataSourceStatus 数据源状态
type DataSourceStatus string
const (
DataSourceHealthy DataSourceStatus = "healthy"
DataSourceDegraded DataSourceStatus = "degraded"
DataSourceDown DataSourceStatus = "down"
)
// ============================================
// K线数据模型
// ============================================
// KLineItem 单条K线数据
type KLineItem struct {
Time time.Time `json:"time"` // 时间戳
Open float64 `json:"open"` // 开盘价
High float64 `json:"high"` // 最高价
Low float64 `json:"low"` // 最低价
Close float64 `json:"close"` // 收盘价
Volume int64 `json:"volume"` // 成交量
Amount float64 `json:"amount"` // 成交额
// 期货特有字段
OpenInterest *int64 `json:"open_interest,omitempty"` // 持仓量
Settlement *float64 `json:"settlement,omitempty"` // 结算价
// 股票特有字段
AdjFactor *float64 `json:"adj_factor,omitempty"` // 复权系数
}
// KLineData K线响应数据
type KLineData struct {
Symbol string `json:"symbol"` // 标的代码
Name string `json:"name"` // 标的名称
Freq Frequency `json:"freq"` // 周期
Adjust AdjustType `json:"adjust"` // 复权类型
Count int `json:"count"` // 数据条数
Items []KLineItem `json:"items"` // K线数据
}
// KLIneQueryRequest K线查询请求
type KLineQueryRequest struct {
Symbol string `json:"symbol" validate:"required"` // 标的代码
Start string `json:"start" validate:"required,len=8"` // 开始日期 YYYYMMDD
End string `json:"end" validate:"required,len=8"` // 结束日期 YYYYMMDD
Freq Frequency `json:"freq"` // 周期，默认1d
Adjust AdjustType `json:"adjust"` // 复权类型
}
// BatchKLineRequest 批量K线查询请求
type BatchKLineRequest struct {
Symbols []string `json:"symbols" validate:"required,max=100"` // 标的列表，最多100只
Start string `json:"start" validate:"required,len=8"`
End string `json:"end" validate:"required,len=8"`
Freq Frequency `json:"freq"`
Adjust AdjustType `json:"adjust"`
}
// BatchKLineResult 批量查询单条结果
type BatchKLineResult struct {
Symbol string `json:"symbol"`
Success bool `json:"success"`
Error string `json:"error,omitempty"`
Data *KLineSubData `json:"data,omitempty"`
}
// KLineSubData 批量查询中的数据部分
type KLineSubData struct {
Count int `json:"count"`
Items []KLineItem `json:"items"`
}
// BatchKLineData 批量K线响应数据
type BatchKLineData struct {
Results []BatchKLineResult `json:"results"`
}
// ============================================
// 标的模型
// ============================================
// Symbol 标的详细信息
type Symbol struct {
SymbolID string `json:"symbol_id"` // 统一标的编码
SymbolType SymbolType `json:"symbol_type"` // 标的类型
Exchange Exchange `json:"exchange"` // 交易所
Name string `json:"name"` // 名称
NameEN string `json:"name_en,omitempty"` // 英文名称
ListDate *time.Time `json:"list_date,omitempty"` // 上市日期
DelistDate *time.Time `json:"delist_date,omitempty"` // 退市日期
// 股票特有
Industry string `json:"industry,omitempty"` // 行业分类
// 期货特有
Underlying string `json:"underlying,omitempty"` // 标的资产
ContractMonth string `json:"contract_month,omitempty"` // 合约月份
ContinuousType string `json:"continuous_type,omitempty"` // 连续合约类型
Status string `json:"status"` // active/suspended/delisted
}
// SymbolListRequest 标的列表请求
type SymbolListRequest struct {
Exchange Exchange `json:"exchange" form:"exchange"` // 交易所筛选
Keyword string `json:"keyword" form:"keyword"` // 关键词搜索
Page int `json:"page" form:"page"` // 页码，默认1
Size int `json:"size" form:"size"` // 每页数量，默认20，最大100
}
// SymbolListData 标的列表响应数据
type SymbolListData struct {
Total int `json:"total"` // 总数
Page int `json:"page"` // 当前页
Size int `json:"size"` // 每页数量
Items []Symbol `json:"items"` // 标的列表
}
// ============================================
// 管理接口模型
// ============================================
// DataSourceInfo 数据源信息
type DataSourceInfo struct {
ActiveSource string `json:"active_source"` // 当前激活源
StandbySources []string `json:"standby_sources"` // 待命源列表
LastTickTS *time.Time `json:"last_tick_ts"` // 最后Tick时间
Status DataSourceStatus `json:"status"` // 数据源状态
}
// DataSourceStatusData 数据源状态响应
type DataSourceStatusData struct {
Stock DataSourceInfo `json:"stock"`
Futures DataSourceInfo `json:"futures"`
}
// SourceSwitchRequest 数据源切换请求
type SourceSwitchRequest struct {
AssetClass AssetClass `json:"asset_class" validate:"required,oneof=stock futures all"`
Source string `json:"source" validate:"required"` // 目标数据源
SyncBackfill bool `json:"sync_backfill"` // 是否同步补录
StartDate string `json:"start_date,omitempty"` // 补录开始日期 YYYYMMDD
}
// BackfillRequest 历史数据补录请求
type BackfillRequest struct {
AssetClass AssetClass `json:"asset_class" validate:"required,oneof=stock futures"`
Symbols []string `json:"symbols" validate:"required"` // 标的列表，空数组表示全部
Start string `json:"start" validate:"required,len=8"`
End string `json:"end" validate:"required,len=8"`
Freqs []Frequency `json:"freqs" validate:"required"` // 需要补录的周期
Source string `json:"source,omitempty"` // 指定数据源，默认当前激活源
}
// ============================================
// 交易日历模型
// ============================================
// TradingDatesRequest 可交易日期查询请求
type TradingDatesRequest struct {
Start string `json:"start" form:"start" validate:"required,len=8"` // 开始日期 YYYYMMDD
End string `json:"end" form:"end" validate:"required,len=8"` // 结束日期 YYYYMMDD
}
// TradingDatesData 可交易日期响应数据
type TradingDatesData struct {
Start string `json:"start"` // 查询开始日期
End string `json:"end"` // 查询结束日期
TotalDays int `json:"total_days"` // 总天数
TradingDays int `json:"trading_days"` // 交易日数量
TradingDates []string `json:"trading_dates"` // 交易日列表 [YYYYMMDD]
}
// ============================================
// 期货合约模型
// ============================================
// FuturesContractsRequest 获取期货合约请求
type FuturesContractsRequest struct {
Underlying string `json:"underlying" form:"underlying" validate:"required"` // 品种代码，如 Cu, RB, M
Exchange string `json:"exchange" form:"exchange"` // 交易所筛选，可选
}
// FuturesContractInfo 期货合约信息
type FuturesContractInfo struct {
SymbolID string `json:"symbol_id"` // 合约代码，如 Cu2504.SHFE
SymbolType string `json:"symbol_type"` // futures
Exchange Exchange `json:"exchange"` // 交易所
Name string `json:"name"` // 合约名称
Underlying string `json:"underlying"` // 品种代码，如 Cu
ContractMonth string `json:"contract_month"` // 合约月份，如 2504
ListDate *time.Time `json:"list_date"` // 上市日期
DelistDate *time.Time `json:"delist_date"` // 退市日期
Status string `json:"status"` // active/expired
}
// FuturesContractsData 期货合约列表响应
type FuturesContractsData struct {
Underlying string `json:"underlying"` // 品种代码
Count int `json:"count"` // 合约数量
Items []FuturesContractInfo `json:"items"` // 合约列表
}
// ============================================
// Response 通用响应结构
type Response struct {
Code int `json:"code"` // 0表示成功，非0表示错误
Message string `json:"message"` // 提示信息
Data interface{} `json:"data"` // 响应数据
}
// ErrorResponse 错误响应
type ErrorResponse struct {
Code int `json:"code"`
Message string `json:"message"`
Detail string `json:"detail,omitempty"`
}
// SuccessResponse 成功响应
type SuccessResponse struct {
Code int `json:"code"`
Message string `json:"message"`
Data interface{} `json:"data,omitempty"`
}
// HealthResponse 健康检查响应
type HealthResponse struct {
Status string `json:"status"`
Timestamp time.Time `json:"timestamp"`
}
// ============================================
// Handler 接口定义
// ============================================
// StockHandler 股票相关接口
type StockHandler interface {
// QueryKLines 查询股票K线
QueryKLines(ctx context.Context, req *KLineQueryRequest) (*Response, error)
// ListSymbols 查询股票标的列表
ListSymbols(ctx context.Context, req *SymbolListRequest) (*Response, error)
// BatchQueryKLines 批量查询股票K线
BatchQueryKLines(ctx context.Context, req *BatchKLineRequest) (*Response, error)
// GetTradingDates 获取股票交易日历
GetTradingDates(ctx context.Context, req *TradingDatesRequest) (*Response, error)
}
// FuturesHandler 期货相关接口
type FuturesHandler interface {
// QueryKLines 查询期货K线
QueryKLines(ctx context.Context, req *KLineQueryRequest) (*Response, error)
// ListSymbols 查询期货标的列表
ListSymbols(ctx context.Context, req *SymbolListRequest) (*Response, error)
// BatchQueryKLines 批量查询期货K线
BatchQueryKLines(ctx context.Context, req *BatchKLineRequest) (*Response, error)
// QueryContinuousKLines 查询主力连续合约K线（预留）
QueryContinuousKLines(ctx context.Context, underlying string, req *KLineQueryRequest) (*Response, error)
// GetTradingDates 获取期货交易日历
GetTradingDates(ctx context.Context, req *TradingDatesRequest) (*Response, error)
// GetContractsByUnderlying 根据品种获取可交易合约
GetContractsByUnderlying(ctx context.Context, req *FuturesContractsRequest) (*Response, error)
}
// AdminHandler 管理接口
type AdminHandler interface {
// GetDataSourceStatus 获取数据源状态
GetDataSourceStatus(ctx context.Context) (*Response, error)
// SwitchDataSource 切换数据源
SwitchDataSource(ctx context.Context, req *SourceSwitchRequest) (*Response, error)
// BackfillData 历史数据补录
BackfillData(ctx context.Context, req *BackfillRequest) (*Response, error)
// HealthCheck 健康检查
HealthCheck(ctx context.Context) (*HealthResponse, error)
}
// Handler 聚合所有Handler接口
type Handler interface {
StockHandler
FuturesHandler
AdminHandler
}

165
cmd/server/main.go
Unescape View File

@ -0,0 +1,165 @@
package main
import (
"context"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/gin-gonic/gin"
"market-data-service/api"
"market-data-service/internal/handler"
"market-data-service/internal/monitor"
"market-data-service/internal/repository"
"market-data-service/internal/service"
"market-data-service/internal/websocket"
)
func main() {
// 配置
port := getEnv("PORT", "8080")
dbURL := getEnv("DATABASE_URL", "postgres://user:password@localhost:5432/marketdata?sslmode=disable")
configPath := getEnv("CONFIG_PATH", "./config.json")
// 设置运行模式
ginMode := getEnv("GIN_MODE", "debug")
gin.SetMode(ginMode)
// 连接数据库
db, err := repository.NewDB(dbURL)
if err != nil {
log.Fatalf("Failed to connect to database: %v", err)
}
defer db.Close()
// 初始化配置服务
configService, err := service.NewConfigService(configPath)
if err != nil {
log.Printf("Warning: Failed to load config from %s: %v", configPath, err)
configService, _ = service.NewConfigService("")
}
// 初始化适配器服务
adapterService := service.NewAdapterService()
// 初始化测试服务
testService := service.NewTestService()
// 初始化Repository
stockRepo := repository.NewStockRepository(db)
futuresRepo := repository.NewFuturesRepository(db)
// 初始化Service
stockService := service.NewStockService(stockRepo)
futuresService := service.NewFuturesService(futuresRepo)
adminService := service.NewAdminService(db)
// 初始化Handler
h := handler.NewHandler(stockService, futuresService, adminService)
// 初始化管理后台Handler
adminHandler := handler.NewAdminHandlerImpl(configService, adapterService, testService)
// 初始化WebSocket Hub
hub := websocket.NewHub()
go hub.Run()
// 初始化WebSocket Server
wsServer := websocket.NewServer(hub)
// 初始化数据质量监控
alertSender := &monitor.LogAlertSender{}
dataMonitor := monitor.NewMonitor(db, stockRepo, futuresRepo, alertSender)
// 启动每日检查定时任务
ctx := context.Background()
dataMonitor.StartDailyCheckCron(ctx)
// 创建Gin引擎
router := gin.New()
router.Use(gin.Recovery())
router.Use(loggerMiddleware())
// 注册API路由
apiRouter := api.NewRouter(h)
apiRouter.Register(router)
// 注册WebSocket路由
router.GET("/v1/stream", wsServer.HandleWebSocket)
// 注册管理后台路由
adminRouter := api.NewAdminRouter(h, adminHandler, adminHandler, adminHandler)
adminRouter.Register(router)
// 创建HTTP服务器
srv := &http.Server{
Addr: ":" + port,
Handler: router,
}
// 启动服务器（异步）
go func() {
log.Printf("Server starting on port %s", port)
log.Printf("Admin dashboard: http://localhost:%s/admin", port)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("Server failed to start: %v", err)
}
}()
// 等待中断信号
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
<-quit
log.Println("Shutting down server...")
// 优雅关闭
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := srv.Shutdown(shutdownCtx); err != nil {
log.Printf("Server forced to shutdown: %v", err)
}
log.Println("Server exited")
}
// loggerMiddleware 日志中间件
func loggerMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now()
path := c.Request.URL.Path
raw := c.Request.URL.RawQuery
c.Next()
latency := time.Since(start)
clientIP := c.ClientIP()
method := c.Request.Method
statusCode := c.Writer.Status()
if raw != "" {
path = path + "?" + raw
}
log.Printf("[GIN] %v | %3d | %13v | %15s | %-7s %s",
start.Format("2006/01/02 - 15:04:05"),
statusCode,
latency,
clientIP,
method,
path,
)
}
}
func getEnv(key, defaultValue string) string {
if value := os.Getenv(key); value != "" {
return value
}
return defaultValue
}

255
cmd/sync/main.go
Unescape View File

@ -0,0 +1,255 @@
// Package sync 数据同步工具
package main
import (
"context"
"flag"
"fmt"
"log"
"os"
"time"
"market-data-service/adapter/tushare"
"market-data-service/api"
"market-data-service/internal/repository"
)
func main() {
var (
syncType = flag.String("type", "", "同步类型: stocks, futures, calendar, klines")
startDate = flag.String("start", "", "开始日期 YYYYMMDD")
endDate = flag.String("end", "", "结束日期 YYYYMMDD")
symbol = flag.String("symbol", "", "标的代码")
underlying = flag.String("underlying", "", "期货品种代码")
freq = flag.String("freq", "1d", "K线周期: 1m/5m/15m/30m/60m/1d")
)
flag.Parse()
if *syncType == "" {
flag.Usage()
os.Exit(1)
}
// 配置
tushareToken := os.Getenv("TUSHARE_TOKEN")
if tushareToken == "" {
log.Fatal("TUSHARE_TOKEN environment variable is required")
}
dbURL := os.Getenv("DATABASE_URL")
if dbURL == "" {
dbURL = "postgres://user:password@localhost:5432/marketdata?sslmode=disable"
}
// 连接数据库
db, err := repository.NewDB(dbURL)
if err != nil {
log.Fatalf("Failed to connect to database: %v", err)
}
defer db.Close()
// 初始化Tushare客户端
client := tushare.NewClient(tushareToken)
ctx := context.Background()
switch *syncType {
case "stocks":
syncStocks(ctx, client, db)
case "futures":
syncFutures(ctx, client, db)
case "calendar":
syncCalendar(ctx, client, db, *startDate, *endDate)
case "klines":
syncKLines(ctx, client, db, *symbol, *startDate, *endDate, *freq)
default:
log.Fatalf("Unknown sync type: %s", *syncType)
}
}
// syncStocks 同步股票基础信息
func syncStocks(ctx context.Context, client *tushare.Client, db *repository.DB) {
log.Println("Syncing stock basic info...")
data, err := client.GetStockBasic()
if err != nil {
log.Fatalf("Failed to get stock basic: %v", err)
}
repo := repository.NewStockRepository(db)
symbols := make([]api.Symbol, 0, len(data))
for _, d := range data {
if d.ListStatus != "L" {
continue // 只同步上市状态的
}
listDate, _ := time.Parse("20060102", d.ListDate)
symbols = append(symbols, api.Symbol{
SymbolID: d.TSCode,
SymbolType: api.SymbolTypeStock,
Exchange: api.Exchange(d.Exchange),
Name: d.Name,
NameEN: d.EnName,
Industry: d.Industry,
ListDate: &listDate,
Status: "active",
})
}
if err := repo.SaveSymbols(ctx, symbols); err != nil {
log.Fatalf("Failed to save symbols: %v", err)
}
log.Printf("Synced %d stocks", len(symbols))
}
// syncFutures 同步期货基础信息
func syncFutures(ctx context.Context, client *tushare.Client, db *repository.DB) {
log.Println("Syncing futures basic info...")
data, err := client.GetFuturesBasic("")
if err != nil {
log.Fatalf("Failed to get futures basic: %v", err)
}
repo := repository.NewFuturesRepository(db)
symbols := make([]api.Symbol, 0, len(data))
for _, d := range data {
listDate, _ := time.Parse("20060102", d.ListDate)
delistDate, _ := time.Parse("20060102", d.DelistDate)
status := "active"
if time.Now().After(delistDate) {
status = "expired"
}
symbols = append(symbols, api.Symbol{
SymbolID: d.TSCode,
SymbolType: api.SymbolTypeFutures,
Exchange: api.Exchange(d.Exchange),
Name: d.Name,
Underlying: d.FutCode,
ContractMonth: d.Symbol[len(d.FutCode):],
ListDate: &listDate,
DelistDate: &delistDate,
Status: status,
})
}
if err := repo.SaveSymbols(ctx, symbols); err != nil {
log.Fatalf("Failed to save symbols: %v", err)
}
log.Printf("Synced %d futures", len(symbols))
}
// syncCalendar 同步交易日历
func syncCalendar(ctx context.Context, client *tushare.Client, db *repository.DB, start, end string) {
if start == "" {
start = time.Now().AddDate(0, 0, -30).Format("20060102")
}
if end == "" {
end = time.Now().AddDate(0, 6, 0).Format("20060102")
}
log.Printf("Syncing trading calendar from %s to %s...", start, end)
// 同步股票交易日历（上交所）
stockData, err := client.GetTradeCal("SSE", start, end, -1)
if err != nil {
log.Fatalf("Failed to get stock calendar: %v", err)
}
stockRepo := repository.NewStockRepository(db)
stockDates := make([]api.TradeCalData, len(stockData))
for i, d := range stockData {
calDate, _ := time.Parse("20060102", d.CalDate)
stockDates[i] = api.TradeCalData{
Date: calDate,
IsTradingDay: d.IsOpen == 1,
}
}
if err := stockRepo.SaveTradingCalendar(ctx, stockDates); err != nil {
log.Fatalf("Failed to save stock calendar: %v", err)
}
// 同步期货交易日历（使用相同的，实际可能需要单独配置）
futuresRepo := repository.NewFuturesRepository(db)
if err := futuresRepo.SaveTradingCalendar(ctx, stockDates); err != nil {
log.Fatalf("Failed to save futures calendar: %v", err)
}
log.Printf("Synced %d calendar days", len(stockDates))
}
// syncKLines 同步K线数据
func syncKLines(ctx context.Context, client *tushare.Client, db *repository.DB, symbol, start, end, freq string) {
if symbol == "" {
log.Fatal("symbol is required for klines sync")
}
if start == "" {
start = time.Now().AddDate(0, 0, -7).Format("20060102")
}
if end == "" {
end = time.Now().Format("20060102")
}
log.Printf("Syncing %s klines for %s from %s to %s...", freq, symbol, start, end)
adapter := tushare.NewAdapter()
if err := adapter.Connect(map[string]string{"token": os.Getenv("TUSHARE_TOKEN")}); err != nil {
log.Fatalf("Failed to connect adapter: %v", err)
}
data, err := adapter.FetchKLines(symbol, start, end, freq)
if err != nil {
log.Fatalf("Failed to fetch klines: %v", err)
}
// 转换为api.KLineItem并保存
items := make([]api.KLineItem, len(data))
for i, d := range data {
ts := time.Unix(d.Time, 0)
items[i] = api.KLineItem{
Time: ts,
Open: d.Open,
High: d.High,
Low: d.Low,
Close: d.Close,
Volume: d.Volume,
Amount: d.Amount,
}
if d.OpenInterest > 0 {
oi := d.OpenInterest
items[i].OpenInterest = &oi
}
}
// 判断股票还是期货并保存
if isStock(symbol) {
repo := repository.NewStockRepository(db)
if err := repo.SaveKLines(ctx, api.Frequency(freq), items); err != nil {
log.Fatalf("Failed to save stock klines: %v", err)
}
} else {
repo := repository.NewFuturesRepository(db)
if err := repo.SaveKLines(ctx, api.Frequency(freq), symbol, items); err != nil {
log.Fatalf("Failed to save futures klines: %v", err)
}
}
log.Printf("Synced %d klines", len(items))
}
// isStock 判断是否为股票代码
func isStock(symbol string) bool {
return contains(symbol, ".SH") || contains(symbol, ".SZ") || contains(symbol, ".BJ")
}
func contains(s, substr string) bool {
return len(s) >= len(substr) && s[len(s)-len(substr):] == substr
}

46
config.example.json
Unescape View File

@ -0,0 +1,46 @@
{
"server": {
"port": 8080,
"mode": "debug",
"api_key": "your-api-key-here"
},
"database": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "password",
"database": "marketdata"
},
"redis": {
"host": "localhost",
"port": 6379,
"password": "",
"db": 0
},
"sources": {
"stock": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token",
"base_url": "https://api.tushare.pro"
}
}
}
},
"futures": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token",
"base_url": "https://api.tushare.pro"
}
}
}
}
}
}

46
config.json
Unescape View File

@ -0,0 +1,46 @@
{
"server": {
"port": 8080,
"mode": "debug",
"api_key": "demo-api-key-2024"
},
"database": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "postgres",
"database": "marketdata"
},
"redis": {
"host": "localhost",
"port": 6379,
"password": "",
"db": 0
},
"sources": {
"stock": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token-here",
"base_url": "https://api.tushare.pro"
}
}
}
},
"futures": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token-here",
"base_url": "https://api.tushare.pro"
}
}
}
}
}
}

190
docs/README.md
Unescape View File

@ -0,0 +1,190 @@
# 管理后台文档目录
本文档目录包含行情数据服务管理后台的完整开发文档。
## 📢 重要更新
**2026-03-08**: 项目已支持 **Go** 和 **Python** 双实现，所有文档已更新以反映两种实现方式。
| 实现方式 | 目录 | 适用场景 |
|----------|------|----------|
| **Go** | 根目录 `market-data-service/` | 生产环境、高性能需求 |
| **Python** | `python_market_data_service/` | 快速开发、数据源对接 |
---
## 🚀 快速开始
**新手？从这里开始：**
1. **[QUICKSTART.md](../QUICKSTART.md)** - **最快启动指南**（推荐先看）
2. **[startup-guide.md](./startup-guide.md)** - 完整启动教程
3. **[DEPLOY.md](../DEPLOY.md)** - 生产部署指南
---
## 文档列表
### 1. 入门文档
| 文档 | 说明 | 适合读者 |
|------|------|----------|
| [QUICKSTART.md](../QUICKSTART.md) | **最快启动指南** - 30秒启动服务 | **所有用户** |
| [startup-guide.md](./startup-guide.md) | **完整启动指南** - 详细步骤 | **所有用户** |
| [go-installation-guide.md](./go-installation-guide.md) | Go 安装指南 | Go用户 |
| [python-installation-guide.md](#) | Python 安装指南 | Python用户 |
### 2. 部署文档
| 文档 | 说明 | 适合读者 |
|------|------|----------|
| [DEPLOY.md](../DEPLOY.md) | **生产部署指南** - 含Docker和Systemd | 运维人员 |
| [startup-guide.md](./startup-guide.md) | 启动和配置说明 | 开发者 |
### 3. 开发文档
| 文档 | 说明 | 适合读者 |
|------|------|----------|
| [admin-dashboard-development.md](./admin-dashboard-development.md) | 管理后台完整开发文档 | 开发者、架构师 |
| [admin-api-quick-reference.md](./admin-api-quick-reference.md) | API接口速查表 | 前端开发者、测试人员 |
| [architecture.md](./architecture.md) | 架构设计文档 | 架构师、技术负责人 |
| [development-guide.md](./development-guide.md) | 开发指南 - 如何开发新功能 | 开发者 |
### 4. 实现方式对比
| 特性 | Go实现 | Python实现 |
|------|--------|------------|
| 性能 | ⭐⭐⭐ 高 | ⭐⭐ 良好 |
| 开发效率 | ⭐⭐ 中等 | ⭐⭐⭐ 高 |
| 数据源生态 | ⭐⭐ 需封装 | ⭐⭐⭐ 原生支持 |
| 部署复杂度 | ⭐⭐⭐ 简单 | ⭐⭐ 依赖多 |
---
## 快速导航
### 如果你是...
**⏱️ 想最快启动服务：**
1. **必读** [QUICKSTART.md](../QUICKSTART.md) - 30秒启动指南
2. 根据提示选择 Go 或 Python 方式
3. 访问 `http://localhost:8080/admin`
**🔧 还没有安装环境：**
- **Go用户**: 阅读 [go-installation-guide.md](./go-installation-guide.md)
- Windows用户可直接运行 `scripts/install-go-windows.ps1`
- Linux/Mac用户运行 `scripts/install-go-linux.sh`
- **Python用户**:
1. 安装Python 3.10+：`python --version`
2. 创建虚拟环境：`python -m venv venv`
3. 激活环境：`source venv/bin/activate` (Linux) 或 `venv\Scripts\activate` (Windows)
4. 安装依赖：`pip install -r python_market_data_service/requirements.txt`
**🚀 第一次使用，需要启动服务：**
1. **必读** [QUICKSTART.md](../QUICKSTART.md) - 最快的启动方式
2. 或阅读 [startup-guide.md](./startup-guide.md) - 完整的启动指南
3. 按步骤配置环境、启动服务
4. 访问 `http://localhost:8080/admin`
**📖 第一次接触这个项目：**
1. 先阅读 [QUICKSTART.md](../QUICKSTART.md) 快速体验
2. 选择实现方式（推荐Go用于生产，Python用于开发）
3. 阅读对应的安装指南
4. 阅读 [admin-dashboard-development.md](./admin-dashboard-development.md) 的"功能概述"章节
5. 查看 [architecture.md](./architecture.md) 了解系统架构
**🔌 需要调用管理后台API：**
1. 查看 [admin-api-quick-reference.md](./admin-api-quick-reference.md)
2. 参考其中的cURL示例（接口在Go和Python中完全一致）
**💻 需要开发新功能：**
1. 阅读 [development-guide.md](./development-guide.md)
2. 参考"开发新功能"章节中的场景示例
3. 注意Go和Python的实现差异
**🏗️ 需要进行架构设计：**
1. 查看 [architecture.md](./architecture.md)
2. 参考"设计决策"和"扩展点设计"章节
3. 查看 `python_market_data_service/MIGRATION_GUIDE.md` 了解双实现对等关系
**🚢 需要部署到生产环境：**
1. 阅读 [DEPLOY.md](../DEPLOY.md) - 详细部署指南
2. 选择 Go 或 Python 部署方式
3. 参考Systemd和Docker部署章节
---
## 相关文件
### Go实现代码分布
```
market-data-service/
├── api/
│ ├── admin_types.go # 类型定义
│ └── admin_router.go # 路由 + HTML页面
├── internal/
│ ├── handler/
│ │ └── admin.go # Handler实现
│ └── service/
│ ├── config.go # 配置服务
│ ├── adapter.go # 适配器服务
│ └── test.go # 测试服务
├── cmd/
│ └── server/
│ └── main.go # 主程序入口
├── QUICKSTART.md # 快速启动指南
├── DEPLOY.md # 部署文档
└── docs/ # 本文档目录
```
### Python实现代码分布
```
python_market_data_service/
├── app/
│ ├── api/
│ │ ├── routes.py # 主要API路由
│ │ └── admin_routes.py # 管理后台路由
│ ├── models/
│ │ ├── types.py # 基础类型（Pydantic）
│ │ └── admin_types.py # 管理后台类型
│ ├── services/
│ │ ├── config_service.py # 配置服务
│ │ ├── adapter_service.py # 适配器服务
│ │ └── test_service.py # 测试服务
│ └── main.py # FastAPI主应用
├── scripts/
│ └── sync_data.py # 数据同步工具
├── QUICKSTART.md # 快速启动指南（根目录）
├── MIGRATION_GUIDE.md # Go到Python迁移对照
└── README.md # Python项目说明
```
---
## 更新记录
| 日期 | 版本 | 说明 |
|------|------|------|
| 2026-03-07 | v1.0 | 初始版本，包含完整管理后台功能文档 |
| 2026-03-07 | v1.1 | 添加启动指南文档 |
| 2026-03-07 | v1.2 | 添加 Go 安装指南和自动安装脚本 |
| 2026-03-08 | v2.0 | **重大更新**: 添加Python实现支持，文档结构重组 |
| 2026-03-08 | v2.1 | 添加QUICKSTART.md快速启动指南 |
---
## 贡献指南
如需更新文档：
1. 修改对应 Markdown 文件
2. 更新本文档的更新记录
3. 确保文档中的代码示例可运行（Go和Python双版本）
4. 保持文档间的链接有效性
5. 如涉及到实现差异，请在文档中标注Go和Python的不同
---
**文档结束**

261
docs/admin-api-quick-reference.md
Unescape View File

@ -0,0 +1,261 @@
# 管理后台 API 速查表
**基础URL**: `http://localhost:8080`
**认证方式**: `X-Admin-Token` Header（当前版本暂未强制验证）
---
## 系统管理
### 获取系统状态
```http
GET /v1/admin/system/status
```
### 热加载配置
```http
POST /v1/admin/system/reload
Content-Type: application/json
{
"config_type": "source" // server|database|redis|source
}
```
### 重启服务
```http
POST /v1/admin/system/restart
Content-Type: application/json
{
"force": false
}
```
---
## 配置管理
### 获取配置列表
```http
GET /v1/admin/config?type={config_type}
# 示例
GET /v1/admin/config?type=server
GET /v1/admin/config?type=database
GET /v1/admin/config?type=redis
GET /v1/admin/config?type=source
```
### 更新配置
```http
PUT /v1/admin/config
Content-Type: application/json
# 服务器配置
{
"type": "server",
"items": {
"port": 8080,
"mode": "release",
"api_key": "new-key"
}
}
# 数据库配置
{
"type": "database",
"items": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "password",
"database": "marketdata"
}
}
# 数据源配置
{
"type": "source",
"items": {
"stock_active": "tushare",
"futures_active": "tushare"
}
}
```
---
## 适配器管理
### 获取适配器列表
```http
GET /v1/admin/adapters
```
### 启用/禁用适配器
```http
POST /v1/admin/adapters/toggle
Content-Type: application/json
{
"name": "tushare",
"enable": true
}
```
### 更新适配器配置
```http
PUT /v1/admin/adapters/config
Content-Type: application/json
{
"name": "tushare",
"config": {
"token": "your-new-token",
"base_url": "https://api.tushare.pro"
}
}
```
---
## 接口测试
### 获取API测试列表
```http
GET /v1/admin/tests/api
```
**测试用例ID列表**:
| ID | 名称 | 说明 |
|----|------|------|
| `stock_klines` | 查询股票K线 | GET /v1/stock/klines/{symbol} |
| `stock_symbols` | 查询股票列表 | GET /v1/stock/symbols |
| `stock_batch` | 批量查询股票K线 | POST /v1/stock/klines/batch |
| `stock_calendar` | 查询交易日历 | GET /v1/stock/trading-dates |
| `futures_klines` | 查询期货K线 | GET /v1/futures/klines/{symbol} |
| `futures_symbols` | 查询期货列表 | GET /v1/futures/symbols |
| `futures_batch` | 批量查询期货K线 | POST /v1/futures/klines/batch |
| `futures_contracts` | 查询合约列表 | GET /v1/futures/contracts |
| `futures_calendar` | 查询期货交易日历 | GET /v1/futures/trading-dates |
| `admin_health` | 健康检查 | GET /v1/admin/health |
| `admin_source_status` | 数据源状态 | GET /v1/admin/source/status |
### 执行API测试
```http
POST /v1/admin/tests/api/run
Content-Type: application/json
{
"id": "stock_klines",
"params": {
"symbol": "000001.SZ",
"start": "20260201",
"end": "20260307",
"freq": "1d"
}
}
```
### 获取WebSocket测试列表
```http
GET /v1/admin/tests/ws
```
**测试用例ID列表**:
| ID | 名称 | 说明 |
|----|------|------|
| `ws_subscribe_stock` | 订阅股票行情 | subscribe [000001.SZ] |
| `ws_subscribe_futures` | 订阅期货行情 | subscribe [CU2504.SHFE] |
| `ws_subscribe_multi` | 批量订阅 | subscribe 多标的 |
| `ws_unsubscribe` | 取消订阅 | unsubscribe [000001.SZ] |
### 执行WebSocket测试
```http
POST /v1/admin/tests/ws/run
Content-Type: application/json
{
"id": "ws_subscribe_stock",
"symbols": ["000001.SZ", "000002.SZ"]
}
```
### 获取测试历史
```http
GET /v1/admin/tests/history?type={type}&limit={limit}
# 示例
GET /v1/admin/tests/history # 全部历史
GET /v1/admin/tests/history?type=api # API测试历史
GET /v1/admin/tests/history?type=ws # WebSocket测试历史
GET /v1/admin/tests/history?limit=50 # 最近50条
```
---
## 响应格式
### 成功响应
```json
{
"code": 0,
"message": "success",
"data": { ... }
}
```
### 错误响应
```json
{
"code": 500,
"message": "错误描述",
"detail": "详细错误信息"
}
```
---
## 使用示例
### cURL 示例
```bash
# 获取系统状态
curl "http://localhost:8080/v1/admin/system/status"
# 热加载配置
curl -X POST "http://localhost:8080/v1/admin/system/reload" \
-H "Content-Type: application/json" \
-d '{"config_type": "source"}'
# 获取适配器列表
curl "http://localhost:8080/v1/admin/adapters"
# 启用适配器
curl -X POST "http://localhost:8080/v1/admin/adapters/toggle" \
-H "Content-Type: application/json" \
-d '{"name": "tushare", "enable": true}'
# 执行API测试
curl -X POST "http://localhost:8080/v1/admin/tests/api/run" \
-H "Content-Type: application/json" \
-d '{
"id": "stock_klines",
"params": {"symbol": "000001.SZ", "start": "20260201", "end": "20260307"}
}'
```
---
## 管理后台页面
直接访问：`http://localhost:8080/admin`
页面功能：
- 系统概览：实时状态监控
- 配置管理：可视化配置编辑
- 数据源适配：适配器管理
- 接口测试：一键测试所有接口

713
docs/admin-dashboard-development.md
Unescape View File

@ -0,0 +1,713 @@
# 管理后台开发文档
**版本**: v1.0
**日期**: 2026-03-07
**作者**: AI Assistant
---
## 目录
1. [功能概述](#一功能概述)
2. [架构设计](#二架构设计)
3. [核心模块](#三核心模块)
4. [API接口文档](#四api接口文档)
5. [前端实现](#五前端实现)
6. [使用指南](#六使用指南)
7. [待完善事项](#七待完善事项)
---
## 一、功能概述
管理后台为行情数据服务提供统一的可视化管理界面，支持配置热加载、数据源适配器管理、接口测试等功能。
### 1.1 功能模块
| 模块 | 功能描述 | 状态 |
|------|----------|------|
| 系统概览 | 实时监控系统状态、内存使用、运行时长 | ✅ 已完成 |
| 配置管理 | 在线修改配置、热加载、配置持久化 | ✅ 已完成 |
| 数据源适配 | 适配器管理、启用/禁用、配置更新 | ✅ 已完成 |
| 接口测试 | API测试、WebSocket测试、历史记录 | ✅ 已完成 |
### 1.2 技术特性
- **热加载**: 配置修改后无需重启服务即可生效
- **实时状态**: 系统运行状态定时刷新（5秒间隔）
- **单页应用**: 纯前端实现，无页面刷新
- **响应式设计**: 适配桌面端浏览器
---
## 二、架构设计
### 2.1 整体架构
```
┌─────────────────────────────────────────────────────────────┐
│ 前端层 │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ 系统概览 │ │ 配置管理 │ │ 数据源适配 │ 接口测试 │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │
│ 内嵌HTML/JS │
└─────────────────────────┬───────────────────────────────────┘
│ HTTP API
┌─────────────────────────┼───────────────────────────────────┐
│ ▼ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ AdminRouter │ │
│ │ (api/admin_router.go) │ │
│ └──────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ AdminHandlerImpl │ │
│ │ (internal/handler/admin.go) │ │
│ └──────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Config │ │Adapter │ │Test │ │
│ │Service │ │Service │ │Service │ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ 服务层│
└─────────────────────────────────────────────────────────────┘
```
### 2.2 目录结构
```
market-data-service/
├── api/
│ ├── admin_types.go # 管理后台类型定义 (270行)
│ └── admin_router.go # 管理后台路由 + HTML页面 (680行)
├── internal/
│ ├── handler/
│ │ └── admin.go # 管理后台Handler (180行)
│ └── service/
│ ├── config.go # 配置管理服务 (330行)
│ ├── adapter.go # 适配器管理服务 (220行)
│ └── test.go # 测试服务 (350行)
├── pkg/
│ └── config/
│ └── config.go # 配置结构定义
├── cmd/server/
│ └── main.go # 服务入口（集成管理后台）
└── docs/
└── admin-dashboard-development.md # 本文档
```
---
## 三、核心模块
### 3.1 配置管理服务 (ConfigService)
**文件**: `internal/service/config.go`
#### 功能
- 配置加载与保存（JSON格式）
- 热加载配置（支持按类型选择性加载）
- 配置变更回调机制
- 系统状态监控
#### 核心接口
```go
type ConfigService interface {
GetConfigList(ctx context.Context, req *api.ConfigListRequest) (*api.ConfigListData, error)
UpdateConfig(ctx context.Context, req *api.ConfigUpdateRequest) (*api.ConfigUpdateData, error)
ReloadConfig(ctx context.Context, req *api.ReloadRequest) (*api.ConfigUpdateData, error)
GetSystemStatus(ctx context.Context) (*api.SystemStatusData, error)
GetCurrentConfig() *config.Config
}
```
#### 配置类型
| 配置类型 | 说明 | 热加载支持 |
|----------|------|-----------|
| server | 服务器端口、运行模式、API Key | 部分需重启 |
| database | 数据库连接配置 | 需重启 |
| redis | Redis连接配置 | 需重启 |
| source | 数据源适配器配置 | ✅ 支持 |
#### 使用示例
```go
// 创建配置服务
configService, err := service.NewConfigService("./config.json")
// 获取配置列表
data, err := configService.GetConfigList(ctx, &api.ConfigListRequest{Type: "server"})
// 更新配置
result, err := configService.UpdateConfig(ctx, &api.ConfigUpdateRequest{
Type: "server",
Items: map[string]interface{}{
"port": 8080,
"mode": "release",
},
})
// 热加载配置
result, err := configService.ReloadConfig(ctx, &api.ReloadRequest{ConfigType: "source"})
```
---
### 3.2 适配器管理服务 (AdapterService)
**文件**: `internal/service/adapter.go`
#### 功能
- 适配器工厂注册
- 适配器启用/禁用
- 适配器配置管理
- 适配器实例管理
#### 适配器状态
| 状态 | 说明 |
|------|------|
| active | 已激活，正在使用 |
| standby | 待命，可用但未激活 |
| disabled | 已禁用 |
| error | 发生错误 |
#### 核心接口
```go
type AdapterService interface {
GetAdapterList(ctx context.Context) (*api.AdapterListData, error)
ToggleAdapter(ctx context.Context, req *api.AdapterToggleRequest) error
UpdateAdapterConfig(ctx context.Context, req *api.AdapterConfigUpdateRequest) error
GetActiveAdapter(assetClass string) (adapter.DataSourceAdapter, error)
RegisterAdapter(name string, factory AdapterFactory)
}
```
#### 内置适配器
| 适配器 | 类型 | 状态 | 说明 |
|--------|------|------|------|
| tushare | HTTP | 已实现 | Tushare Pro 数据接口 |
| wind | WebSocket | 预留 | Wind 金融终端（待实现） |
---
### 3.3 测试服务 (TestService)
**文件**: `internal/service/test.go`
#### 功能
- API接口测试用例管理
- WebSocket连接测试
- 测试历史记录
- 实时测试执行
#### 测试分类
**API测试**:
- 股票接口：K线查询、标的列表、批量查询、交易日历
- 期货接口：K线查询、标的列表、批量查询、合约查询
- 管理接口：健康检查、数据源状态
**WebSocket测试**:
- 订阅股票行情
- 订阅期货行情
- 批量订阅
- 取消订阅
#### 核心接口
```go
type TestService interface {
GetAPITestList(ctx context.Context) (*api.APITestListData, error)
RunAPITest(ctx context.Context, baseURL string, req *api.APITestRequest) (*api.APITestResult, error)
GetWSTestList(ctx context.Context) (*api.WSTestListData, error)
RunWSTest(ctx context.Context, wsURL string, req *api.WSTestRequest) (*api.WSTestResult, error)
GetTestHistory(ctx context.Context, req *api.TestHistoryRequest) (*api.TestHistoryData, error)
}
```
---
## 四、API接口文档
### 4.1 系统管理接口
#### 获取系统状态
```
GET /v1/admin/system/status
```
**响应示例**:
```json
{
"code": 0,
"data": {
"status": "running",
"version": "1.0.0",
"start_time": "2026-03-07T13:00:00Z",
"uptime": "1小时30分钟",
"go_version": "go1.21.0",
"memory": {
"alloc": 10485760,
"total_alloc": 52428800,
"sys": 33554432,
"num_gc": 5
},
"goroutines": 15
}
}
```
#### 热加载配置
```
POST /v1/admin/system/reload
Content-Type: application/json
{
"config_type": "source" // 可选：server/database/redis/source
}
```
**响应示例**:
```json
{
"code": 0,
"data": {
"success": true,
"message": "配置热加载成功"
}
}
```
#### 重启服务
```
POST /v1/admin/system/restart
Content-Type: application/json
{
"force": false
}
```
---
### 4.2 配置管理接口
#### 获取配置列表
```
GET /v1/admin/config?type=server
```
**响应示例**:
```json
{
"code": 0,
"data": {
"sections": [
{
"name": "服务器配置",
"type": "server",
"description": "HTTP服务器相关配置",
"items": [
{
"key": "port",
"value": 8080,
"type": "int",
"description": "服务端口",
"editable": true,
"required": true
}
]
}
],
"version": "1.0.0",
"updated": "2026-03-07T13:00:00Z"
}
}
```
#### 更新配置
```
PUT /v1/admin/config
Content-Type: application/json
{
"type": "server",
"items": {
"port": 8080,
"mode": "release",
"api_key": "new-key"
}
}
```
**响应示例**:
```json
{
"code": 0,
"data": {
"success": true,
"need_restart": false,
"message": "配置更新成功"
}
}
```
---
### 4.3 适配器管理接口
#### 获取适配器列表
```
GET /v1/admin/adapters
```
**响应示例**:
```json
{
"code": 0,
"data": {
"adapters": [
{
"name": "tushare",
"type": "http",
"version": "1.0.0",
"description": "Tushare Pro 金融数据接口",
"status": "active",
"config": {
"token": "***",
"base_url": "https://api.tushare.pro"
},
"updated_at": "2026-03-07T13:00:00Z"
}
]
}
}
```
#### 启用/禁用适配器
```
POST /v1/admin/adapters/toggle
Content-Type: application/json
{
"name": "tushare",
"enable": true
}
```
#### 更新适配器配置
```
PUT /v1/admin/adapters/config
Content-Type: application/json
{
"name": "tushare",
"config": {
"token": "new-token",
"base_url": "https://api.tushare.pro"
}
}
```
---
### 4.4 接口测试管理
#### 获取API测试列表
```
GET /v1/admin/tests/api
```
**响应示例**:
```json
{
"code": 0,
"data": {
"categories": [
{
"name": "股票接口",
"items": [
{
"id": "stock_klines",
"name": "查询股票K线",
"method": "GET",
"path": "/v1/stock/klines/{symbol}",
"description": "查询指定股票的K线数据",
"params": {
"symbol": "000001.SZ",
"start": "20260201",
"end": "20260307",
"freq": "1d"
}
}
]
}
],
"base_url": "http://localhost:8080"
}
}
```
#### 执行API测试
```
POST /v1/admin/tests/api/run
Content-Type: application/json
{
"id": "stock_klines",
"params": {
"symbol": "000001.SZ"
}
}
```
**响应示例**:
```json
{
"code": 0,
"data": {
"id": 1234567890,
"case_id": "stock_klines",
"name": "查询股票K线",
"success": true,
"status_code": 200,
"latency": 45,
"request": {
"method": "GET",
"url": "http://localhost:8080/v1/stock/klines/000001.SZ?start=20260201&end=20260307&freq=1d"
},
"response": {
"code": 0,
"data": {...}
},
"timestamp": "2026-03-07T13:00:00Z"
}
}
```
#### 获取WebSocket测试列表
```
GET /v1/admin/tests/ws
```
#### 执行WebSocket测试
```
POST /v1/admin/tests/ws/run
Content-Type: application/json
{
"id": "ws_subscribe_stock",
"symbols": ["000001.SZ", "000002.SZ"]
}
```
#### 获取测试历史
```
GET /v1/admin/tests/history?type=api&limit=20
```
---
## 五、前端实现
### 5.1 技术方案
- **纯HTML/CSS/JS**: 无前端框架依赖
- **单页应用**: 前端路由切换，无页面刷新
- **嵌入式部署**: HTML代码内嵌在Go文件中
### 5.2 页面结构
```
/admin
├── 系统概览 (dashboard)
│ ├── 状态卡片（运行状态、运行时长、版本、Goroutines）
│ ├── 内存使用详情
│ └── 操作按钮（热加载、重启）
│
├── 配置管理 (config)
│ ├── 服务器配置
│ ├── 数据库配置
│ ├── Redis配置
│ └── 数据源配置
│
├── 数据源适配 (adapters)
│ └── 适配器列表表格
│
└── 接口测试 (tests)
├── API测试页签
├── WebSocket测试页签
└── 测试历史页签
```
### 5.3 前端API封装
```javascript
// API请求封装
async function apiRequest(method, path, data = null) {
const options = {
method,
headers: {
'Content-Type': 'application/json',
'X-Admin-Token': localStorage.getItem('adminToken') || ''
}
};
if (data) {
options.body = JSON.stringify(data);
}
const response = await fetch(state.baseURL + path, options);
return response.json();
}
```
---
## 六、使用指南
### 6.1 启动服务
```bash
# 设置配置文件路径
export CONFIG_PATH="./config.json"
# 启动服务
go run ./cmd/server
# 或使用Makefile
make run
```
### 6.2 访问管理后台
浏览器访问：`http://localhost:8080/admin`
### 6.3 配置热加载示例
```bash
# 1. 修改 config.json 文件
# 2. 调用热加载API
curl -X POST "http://localhost:8080/v1/admin/system/reload" \
-H "Content-Type: application/json" \
-d '{"config_type": "source"}'
# 3. 或在管理后台点击"热加载配置"按钮
```
### 6.4 适配器管理流程
```
1. 进入"数据源适配"页面
2. 查看已注册的适配器列表
3. 点击"启用"/"禁用"按钮切换状态
4. 点击适配器名称可编辑配置
```
---
## 七、待完善事项
### 7.1 功能增强
| 优先级 | 功能 | 说明 |
|--------|------|------|
| P1 | 用户认证 | 实现基于Token的管理员认证机制 |
| P1 | 操作日志 | 记录所有配置变更和管理操作 |
| P2 | 配置版本 | 支持配置历史版本回滚 |
| P2 | 批量操作 | 支持批量启用/禁用适配器 |
| P3 | 数据可视化 | 添加数据源调用统计图表 |
| P3 | 告警管理 | 集成钉钉/邮件告警配置 |
### 7.2 安全加固
| 优先级 | 功能 | 说明 |
|--------|------|------|
| P1 | 访问控制 | 基于IP白名单的访问限制 |
| P1 | HTTPS支持 | 管理后台强制HTTPS访问 |
| P2 | 密码加密 | 配置文件中的敏感信息加密存储 |
| P2 | 审计日志 | 详细的操作审计日志 |
### 7.3 性能优化
| 优先级 | 功能 | 说明 |
|--------|------|------|
| P2 | 配置缓存 | 热点配置缓存，减少磁盘IO |
| P3 | 前端优化 | 静态资源CDN、代码分割 |
| P3 | WebSocket推送 | 系统状态通过WebSocket实时推送 |
### 7.4 已知问题
1. **重启服务**: 当前仅返回成功响应，实际重启需外部脚本配合
2. **并发安全**: 配置热加载时可能存在短暂的数据竞争
3. **测试隔离**: API测试直接调用生产接口，需增加测试模式
### 7.5 扩展建议
1. **插件系统**: 支持第三方适配器动态加载
2. **多租户**: 支持多用户、多权限级别的管理
3. **移动端**: 开发响应式移动端管理界面
4. **国际化**: 支持中英文切换
---
## 附录
### A. 配置文件示例
```json
{
"server": {
"port": 8080,
"mode": "debug",
"api_key": "your-api-key"
},
"database": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "password",
"database": "marketdata"
},
"redis": {
"host": "localhost",
"port": 6379,
"password": "",
"db": 0
},
"sources": {
"stock": {
"active": "tushare"
},
"futures": {
"active": "tushare"
}
}
}
```
### B. 相关文件清单
| 文件 | 说明 | 行数 |
|------|------|------|
| `api/admin_types.go` | 类型定义 | ~270 |
| `api/admin_router.go` | 路由 + HTML | ~680 |
| `internal/handler/admin.go` | Handler实现 | ~180 |
| `internal/service/config.go` | 配置服务 | ~330 |
| `internal/service/adapter.go` | 适配器服务 | ~220 |
| `internal/service/test.go` | 测试服务 | ~350 |
| `pkg/config/config.go` | 配置结构 | ~60 |
---
**文档结束**

460
docs/architecture.md
Unescape View File

@ -0,0 +1,460 @@
# 管理后台架构设计文档
## 重要说明
本文档适用于 **Go** 和 **Python** 双实现。两者架构设计保持一致，仅技术栈不同：
- **Go**: Gin + 原生SQL + Goroutine
- **Python**: FastAPI + SQLAlchemy + asyncio
---
## 1. 系统架构图
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ 客户端层 │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Web Browser │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Dashboard │ │ Config │ │ Adapter │ │ Test │ │ │
│ │ │ Page │ │ Page │ │ Page │ │ Page │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ http://localhost:8080/admin │
└─────────────────────────────────────┬───────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 接入层 │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Router (Gin/FastAPI) │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │ │
│ │ │ API Router │ │ Admin Router │ │ WebSocket Handler │ │ │
│ │ │(api/router.go)│ │(api/admin_ │ │ (/v1/stream) │ │ │
│ │ │ (routes.py) │ │ router.go) │ │ │ │ │
│ │ │ │ │(admin_routes)│ │ │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────────────────┘ │ │
│ │ │ │ │ │
│ └──────────┼────────────────┼─────────────────────────────────────────┘ │
│ │ │ │
└─────────────┼────────────────┼─────────────────────────────────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 业务层 │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Service Layer │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────────────────────┐ │ │
│ │ │ Stock/Futures │ │ AdminHandlerImpl │ │ │
│ │ │ Services │ │(admin.go / admin_routes.py) │ │ │
│ │ └─────────────────┘ └──────────────┬──────────────────┘ │ │
│ │ │ │ │
│ └──────────────────────────────────────────────┼──────────────────────┘ │
│ │ │
└─────────────────────────────────────────────────┼──────────────────────────┘
│
┌───────────────────────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 服务层 │
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ ConfigService │ │AdapterService │ │ TestService │ │
│ │(config.go) │ │(adapter.go) │ │ (test.go) │ │
│ │(config_service)│ │(adapter_service)│ │(test_service) │ │
│ │ │ │ │ │ │ │
│ │ • 配置加载 │ │ • 适配器注册 │ │ • API测试 │ │
│ │ • 热加载 │ │ • 启用/禁用 │ │ • WS测试 │ │
│ │ • 状态监控 │ │ • 配置管理 │ │ • 历史记录 │ │
│ └───────┬───────┘ └───────┬───────┘ └───────┬───────┘ │
│ │ │ │ │
│ └────────────────────┼────────────────────┘ │
│ │ │
└────────────────────────────────┼───────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 数据层 │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌───────────┐ │
│ │ config.json │ │ Database │ │ Adapter │ │ Memory │ │
│ │ (文件) │ │ (PostgreSQL)│ │ Factory │ │ Cache │ │
│ │ │ │ │ │ │ │ │ │
│ │ 持久化配置 │ │ 数据源配置 │ │ 适配器实例 │ │ 测试历史 │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### 技术栈对比
| 层级 | Go实现 | Python实现 |
|------|--------|------------|
| 接入层 | Gin Router | FastAPI Router |
| 业务层 | Go Interfaces | Python Protocols |
| 服务层 | Go Structs + Methods | Python Classes |
| 数据层 | database/sql | SQLAlchemy ORM |
| 配置 | JSON + 自定义解析 | Pydantic Settings |
---
## 2. 模块关系图
### 配置管理模块
```
┌─────────────────────────────────────────────────────────────────┐
│ 配置管理模块 │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ConfigService │ │
│ │ (Go / Python) │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ │ │
│ │ │ Load Config │───►│ Store Config│◄───│UpdateConfig│ │ │
│ │ │ (JSON File) │ │ (Memory) │ │ (API) │ │ │
│ │ └─────────────┘ └──────┬──────┘ └────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────┐ │ │
│ │ │ Reload │ │ │
│ │ │ (Hot Swap) │ │ │
│ │ └──────┬──────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────┐ │ │
│ │ │ Callbacks │ │ │
│ │ │ (Notify All)│ │ │
│ │ └─────────────┘ │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
**Go实现**:
```go
// 使用 sync.RWMutex 保证并发安全
type ConfigServiceImpl struct {
config *config.Config
mu sync.RWMutex
callbacks map[api.ConfigType][]func()
}
```
**Python实现**:
```python
# 使用 threading.RLock 保证并发安全
class ConfigService:
def __init__(self):
self.config = get_config()
self.lock = threading.RLock()
self.callbacks: Dict[ConfigType, List[Callable]] = {}
```
---
### 适配器管理模块
```
┌─────────────────────────────────────────────────────────────────┐
│ 适配器管理模块 │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ AdapterService │ │
│ │ (Go / Python) │ │
│ │ ┌──────────────┐ ┌─────────────────────┐ │ │
│ │ │ Register │────────►│ Factory Map │ │ │
│ │ │ (Add Factory)│ │ [name]factoryFunc │ │ │
│ │ └──────────────┘ └─────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────┐ ┌─────────────────────┐ │ │
│ │ │ Toggle │────────►│ Active Adapters │ │ │
│ │ │(Enable/Disable)│ │ [name]adapterInstance│ │ │
│ │ └──────────────┘ └─────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────┐ ┌─────────────────────┐ │ │
│ │ │Update Config │────────►│ Adapter Config │ │ │
│ │ └──────────────┘ │ [name]configMap │ │ │
│ │ └─────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
**Go实现**:
```go
type AdapterServiceImpl struct {
factories map[string]AdapterFactory
configs map[string]*adapterConfig
activeAdapters map[string]adapter.DataSourceAdapter
}
```
**Python实现**:
```python
class AdapterService:
def __init__(self):
self.factories: Dict[str, Callable] = {}
self.configs: Dict[str, dict] = {}
self.active_adapters: Dict[str, DataSourceAdapter] = {}
```
---
### 测试管理模块
```
┌─────────────────────────────────────────────────────────────────┐
│ 测试管理模块 │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ TestService │ │
│ │ (Go / Python) │ │
│ │ ┌────────────────┐ ┌──────────────────────┐ │ │
│ │ │ API Test │ │ Test Cases (JSON) │ │ │
│ │ │ • Build Request│◄───────│ • stock_klines │ │ │
│ │ │ • Execute HTTP │ │ • futures_klines │ │ │
│ │ │ • Parse Result │ │ • admin_health │ │ │
│ │ └───────┬────────┘ └──────────────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌────────────────┐ ┌──────────────────────┐ │ │
│ │ │ WS Test │ │ Test History │ │ │
│ │ │ • Dial WS │◄───────│ (In-Memory Cache) │ │ │
│ │ │ • Send Msg │ │ • api_tests [] │ │ │
│ │ │ • Recv Msg │ │ • ws_tests [] │ │ │
│ │ └────────────────┘ └──────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
---
## 3. 数据流图
### 3.1 配置热加载流程
```
User/Admin Admin API ConfigService File System
│ │ │ │
│ 1.修改 config.json │ │ │
│────────────────────────────►│ │ │
│ │ │ │
│ 2.点击"热加载" │ │ │
│────────────────────────────►│ │ │
│ │ 3.POST /system/reload │ │
│ │──────────────────────────►│ │
│ │ │ 4.读取 config.json │
│ │ │───────────────────────►│
│ │ │◄───────────────────────│
│ │ │ 5.解析并更新内存配置 │
│ │ │ │
│ │ │ 6.触发回调函数 │
│ │◄──────────────────────────│ │
│◄────────────────────────────│ 7.返回成功 │ │
│ │ │ │
```
### 3.2 适配器切换流程
```
User Admin API AdapterService Adapter Instance
│ │ │ │
│ 1.选择适配器并点击"启用" │ │ │
│──────────────────────────►│ │ │
│ │ 2.POST /adapters/toggle │ │
│ │─────────────────────────►│ │
│ │ │ 3.创建新实例 │
│ │ │─────────────────────►│
│ │ │◄─────────────────────│
│ │ │ 4.调用 Connect() │
│ │ │ │
│ │ │ 5.更新 activeAdapters│
│ │◄─────────────────────────│ │
│◄──────────────────────────│ 6.返回成功 │ │
```
---
## 4. 关键设计决策
### 4.1 配置存储
| 方案 | 优点 | 缺点 | Go选择 | Python选择 |
|------|------|------|--------|------------|
| JSON文件 | 简单、易编辑、无依赖 | 无事务支持 | ✅ | ✅ |
| 数据库存储 | 支持事务、历史版本 | 增加依赖 | ❌ | ❌ |
| etcd/consul | 分布式、高可用 | 引入新组件 | ❌ | ❌ |
**Python增强**: 使用 Pydantic 进行类型验证和自动解析
### 4.2 前端实现
| 方案 | 优点 | 缺点 | 选择 |
|------|------|------|------|
| 纯HTML/JS | 无依赖、部署简单 | 功能受限 | ✅ |
| Vue/React | 功能强大、生态丰富 | 需构建、体积大 | ❌ |
| 独立前端项目 | 前后端分离 | 部署复杂 | ❌ |
### 4.3 数据库访问
| 方案 | Go | Python | 说明 |
|------|----|--------|------|
| 原生SQL | ✅ database/sql | ❌ | Go标准方式 |
| ORM | ❌ | ✅ SQLAlchemy | Python生态标准 |
| SQL Builder | ⏳ 可选 | ⏳ 可选 | 未来考虑 |
---
## 5. 扩展点设计
### 5.1 新增适配器
**Go**:
```go
// 1. 实现适配器接口
type MyAdapter struct { ... }
func (a *MyAdapter) Connect(config map[string]string) error { ... }
func (a *MyAdapter) HealthCheck() error { ... }
// ... 实现其他方法
// 2. 注册到服务
func init() {
adapterService.RegisterAdapter("myadapter", func() adapter.DataSourceAdapter {
return &MyAdapter{}
})
}
```
**Python**:
```python
# 1. 实现适配器接口
class MyAdapter(DataSourceAdapter):
async def connect(self, config: dict) -> None:
pass
async def health_check(self) -> bool:
return True
# ... 实现其他方法
# 2. 注册到服务
adapter_service.register_adapter("myadapter", lambda: MyAdapter())
```
### 5.2 新增配置类型
**Go**:
```go
// 1. 扩展配置结构
type Config struct {
// ... 现有配置
Custom CustomConfig `json:"custom"`
}
// 2. 在 ConfigService 中添加处理逻辑
```
**Python**:
```python
# 1. 扩展配置结构
class Config(BaseModel):
# ... 现有配置
custom: CustomConfig = Field(default_factory=CustomConfig)
# 2. Pydantic 自动处理验证和解析
```
---
## 6. 部署架构
```
┌─────────────────────────────────────────────────────────────┐
│ 生产环境部署 │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Nginx / LB │ │
│ │ (SSL终止、静态资源缓存) │ │
│ └──────────────────┬──────────────────────────────────┘ │
│ │ │
│ ┌──────────┴──────────┐ │
│ │ │ │
│ ┌───────▼───────┐ ┌────────▼────────┐ │
│ │ Market Data │ │ Market Data │ │
│ │ Service 1 │ │ Service 2 │ │
│ │ │ │ │ │
│ │ • Go 或 │ │ • Go 或 │ │
│ │ Python │ │ Python │ │
│ │ • /v1/api │ │ • /v1/api │ │
│ │ • /admin │ │ • /admin │ │
│ │ • /v1/stream │ │ • /v1/stream │ │
│ └───────┬───────┘ └────────┬────────┘ │
│ │ │ │
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────────────▼──────────────────┐ │
│ │ PostgreSQL Cluster │ │
│ │ │ │
│ │ • data_source_config (数据源配置) │ │
│ │ • 其他业务表... │ │
│ └─────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 混合部署建议
```
┌─────────────────────────────────────────────────────────────┐
│ 推荐混合部署架构 │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Go 实现 │ │ Python 实现 │ │
│ │ (生产环境) │ │ (开发/测试) │ │
│ │ │ │ │ │
│ │ • 高并发API │ │ • 数据同步工具 │ │
│ │ • WebSocket │ │ • 管理后台 │ │
│ │ • 核心服务 │ │ • 快速原型 │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │
│ └───────────┬───────────┘ │
│ │ │
│ ┌────────────▼────────────┐ │
│ │ PostgreSQL │ │
│ │ (共享数据库) │ │
│ └─────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## 7. 监控点
| 模块 | 监控项 | Go方式 | Python方式 |
|------|--------|--------|------------|
| ConfigService | 配置加载耗时 | 日志 | 日志 |
| ConfigService | 配置热加载次数 | 日志 | 日志 |
| AdapterService | 适配器健康状态 | 接口/日志 | 接口/日志 |
| AdapterService | 适配器切换次数 | 日志 | 日志 |
| TestService | 测试执行次数 | 内存/日志 | 内存/日志 |
| TestService | 测试成功率 | 计算 | 计算 |
| WebSocket | 连接数 | metrics | metrics |
| Database | 查询耗时 | SQL日志 | SQLAlchemy事件 |
---
## 8. 实现差异汇总
| 方面 | Go | Python |
|------|----|--------|
| 并发模型 | Goroutines | asyncio |
| 锁机制 | sync.Mutex | threading.Lock/asyncio.Lock |
| 数据库 | database/sql + pq | SQLAlchemy + psycopg2 |
| Web框架 | Gin | FastAPI |
| 类型系统 | Struct + Interface | Pydantic Model |
| 配置解析 | encoding/json | Pydantic Settings |
| 错误处理 | error interface | Exception |
| 日志 | log | logging |
---
**文档结束**

988
docs/development-guide.md
Unescape View File

@ -0,0 +1,988 @@
# 管理后台开发指南
本文档适用于 **Go** 和 **Python** 双实现。示例代码会标注对应的实现语言。
---
## 快速开始
### 1. 环境准备
#### Go 环境
```bash
# 确保Go版本 >= 1.21
go version
# 安装依赖
cd market-data-service
go mod download
```
#### Python 环境
```bash
# 确保Python版本 >= 3.10
python --version
# 创建虚拟环境
cd python_market_data_service
python -m venv venv
# 激活虚拟环境
# Linux/Mac:
source venv/bin/activate
# Windows:
venv\Scripts\activate
# 安装依赖
pip install -r requirements.txt
pip install tushare
```
### 2. 启动服务
#### Go
```bash
# 方式1：直接运行
go run ./cmd/server
# 方式2：使用配置文件
export CONFIG_PATH="./config.json"
go run ./cmd/server
# 方式3：Makefile
make run
```
#### Python
```bash
# 方式1：直接运行（开发模式）
python -m app.main
# 方式2：使用uvicorn（推荐）
uvicorn app.main:app --reload --port 8080
# 方式3：生产模式
uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
```
### 3. 访问管理后台
浏览器打开：`http://localhost:8080/admin`
API文档：`http://localhost:8080/docs` (仅Python/FastAPI自动生成)
---
## 开发新功能
### 场景1：添加新的配置项
假设需要添加一个新的缓存配置项：
#### Go 实现
**步骤1**: 修改配置结构 (`pkg/config/config.go`)
```go
type Config struct {
// ... 现有配置
Cache CacheConfig `json:"cache"`
}
type CacheConfig struct {
Enabled bool `json:"enabled"`
TTL int `json:"ttl"` // 缓存过期时间（秒）
MaxSize int `json:"max_size"` // 最大缓存条目数
}
```
**步骤2**: 在配置服务中添加新分组 (`internal/service/config.go`)
```go
func (s *ConfigServiceImpl) GetConfigList(...) {
// ... 现有配置分组
// 添加缓存配置分组
sections = append(sections, api.ConfigSection{
Name: "缓存配置",
Type: api.ConfigType("cache"), // 添加新的类型常量
Description: "数据缓存相关配置",
Items: []api.ConfigItem{
{
Key: "enabled",
Value: s.config.Cache.Enabled,
Type: "bool",
Description: "是否启用缓存",
Editable: true,
Required: false,
},
{
Key: "ttl",
Value: s.config.Cache.TTL,
Type: "int",
Description: "缓存过期时间（秒）",
Editable: true,
Required: false,
},
{
Key: "max_size",
Value: s.config.Cache.MaxSize,
Type: "int",
Description: "最大缓存条目数",
Editable: true,
Required: false,
},
},
})
}
```
**步骤3**: 添加配置更新处理
```go
func (s *ConfigServiceImpl) UpdateConfig(...) {
switch req.Type {
// ... 现有 case
case api.ConfigType("cache"):
if enabled, ok := req.Items["enabled"]; ok {
s.config.Cache.Enabled = enabled.(bool)
}
if ttl, ok := req.Items["ttl"]; ok {
s.config.Cache.TTL = int(ttl.(float64))
}
if maxSize, ok := req.Items["max_size"]; ok {
s.config.Cache.MaxSize = int(maxSize.(float64))
}
}
// 保存配置
if err := s.saveConfig(); err != nil {
return nil, err
}
return &api.ConfigUpdateData{
Success: true,
NeedRestart: false, // 缓存配置支持热加载
Message: "配置更新成功",
}, nil
}
```
**步骤4**: 添加配置变更回调（可选）
```go
// 在初始化时注册回调
configService.RegisterCallback(api.ConfigType("cache"), func() {
// 重新初始化缓存
cache.Init(configService.GetCurrentConfig().Cache)
})
```
#### Python 实现
**步骤1**: 修改配置结构 (`app/core/config.py`)
```python
class CacheConfig(BaseModel):
"""缓存配置"""
enabled: bool = False
ttl: int = 3600 # 缓存过期时间（秒）
max_size: int = 1000 # 最大缓存条目数
class Config(BaseModel):
"""主配置类"""
# ... 现有配置
cache: CacheConfig = Field(default_factory=CacheConfig)
```
**步骤2**: 在配置服务中添加新分组 (`app/services/config_service.py`)
```python
from app.models import ConfigType
class ConfigService:
def get_config_list(self, req: ConfigListRequest) -> ConfigListData:
sections = []
# ... 现有配置分组
# 添加缓存配置分组
if not req.type or req.type == ConfigType.CACHE:
sections.append(ConfigSection(
name="缓存配置",
type=ConfigType.CACHE,
description="数据缓存相关配置",
items=[
ConfigItem(
key="enabled",
value=self.config.cache.enabled,
type="bool",
description="是否启用缓存",
editable=True,
required=False
),
ConfigItem(
key="ttl",
value=self.config.cache.ttl,
type="int",
description="缓存过期时间（秒）",
editable=True,
required=False
),
ConfigItem(
key="max_size",
value=self.config.cache.max_size,
type="int",
description="最大缓存条目数",
editable=True,
required=False
),
]
))
return ConfigListData(sections=sections, ...)
```
**步骤3**: 添加配置更新处理
```python
def update_config(self, req: ConfigUpdateRequest) -> ConfigUpdateData:
need_restart = False
with self.lock:
if req.type == ConfigType.CACHE:
if "enabled" in req.items:
self.config.cache.enabled = bool(req.items["enabled"])
if "ttl" in req.items:
self.config.cache.ttl = int(req.items["ttl"])
if "max_size" in req.items:
self.config.cache.max_size = int(req.items["max_size"])
# 保存到文件
try:
save_config(self.config)
self._trigger_callbacks(req.type)
return ConfigUpdateData(
success=True,
need_restart=False, # 缓存配置支持热加载
message="配置更新成功"
)
except Exception as e:
return ConfigUpdateData(
success=False,
message=f"配置保存失败: {e}"
)
```
**步骤4**: 添加配置变更回调（可选）
```python
# 在初始化时注册回调
config_service.register_callback(ConfigType.CACHE, lambda:
cache.init(config_service.get_current_config().cache)
)
```
---
### 场景2：添加新的数据源适配器
假设需要添加一个名为 "mydata" 的适配器：
#### Go 实现
**步骤1**: 创建适配器实现 (`adapter/mydata/adapter.go`)
```go
package mydata
import (
"market-data-service/adapter"
)
type Adapter struct {
client *Client
config map[string]string
}
func NewAdapter() *Adapter {
return &Adapter{}
}
func (a *Adapter) Connect(config map[string]string) error {
a.config = config
// 初始化客户端连接
a.client = NewClient(config["api_key"])
return nil
}
func (a *Adapter) SubscribeTicks(symbols []string, callback adapter.TickCallback) error {
// 实现实时数据订阅
return nil
}
func (a *Adapter) FetchKLines(symbol, start, end, freq string) ([]adapter.KLineData, error) {
// 实现K线数据获取
return nil, nil
}
func (a *Adapter) FetchSymbols(assetType string) ([]adapter.SymbolInfo, error) {
// 实现标的列表获取
return nil, nil
}
func (a *Adapter) FetchTradingCalendar(exchange, start, end string) ([]adapter.TradeCalData, error) {
// 实现交易日历获取
return nil, nil
}
func (a *Adapter) HealthCheck() error {
// 实现健康检查
return nil
}
func (a *Adapter) Close() error {
// 实现资源释放
return nil
}
```
**步骤2**: 注册适配器 (`internal/service/adapter.go`)
```go
import "market-data-service/adapter/mydata"
func (s *AdapterServiceImpl) registerBuiltinAdapters() {
// ... 现有适配器注册
// 注册 MyData 适配器
s.RegisterAdapter("mydata", func() adapter.DataSourceAdapter {
return mydata.NewAdapter()
})
// 设置元数据
s.metadata["mydata"] = &adapterMetadata{
Name: "mydata",
Type: "http",
Version: "1.0.0",
Description: "MyData 金融数据接口",
UpdatedAt: time.Now(),
}
// 默认配置
s.configs["mydata"] = &adapterConfig{
Enabled: false,
Config: map[string]string{
"api_key": "",
"base_url": "https://api.mydata.com",
},
}
}
```
#### Python 实现
**步骤1**: 创建适配器实现 (`app/adapters/mydata_adapter.py`)
```python
from typing import List
from app.adapters.base import (
DataSourceAdapter, TickData, KLineData,
SymbolInfo, TradeCalData, TickCallback
)
class MyDataAdapter(DataSourceAdapter):
"""MyData数据源适配器"""
def __init__(self):
self.client = None
self.config = {}
async def connect(self, config: dict) -> None:
self.config = config
# 初始化客户端连接
self.client = MyDataClient(config["api_key"])
async def subscribe_ticks(self, symbols: List[str], callback: TickCallback) -> None:
# 实现实时数据订阅
pass
async def fetch_klines(
self, symbol: str, start: str, end: str, freq: str
) -> List[KLineData]:
# 实现K线数据获取
return []
async def fetch_symbols(self, asset_type: str) -> List[SymbolInfo]:
# 实现标的列表获取
return []
async def fetch_trading_calendar(
self, exchange: str, start: str, end: str
) -> List[TradeCalData]:
# 实现交易日历获取
return []
async def health_check(self) -> bool:
# 实现健康检查
return True
async def close(self) -> None:
# 实现资源释放
pass
```
**步骤2**: 注册适配器 (`app/services/adapter_service.py`)
```python
from app.adapters.mydata_adapter import MyDataAdapter
class AdapterService:
def _register_builtin_adapters(self):
# ... 现有适配器注册
# 注册 MyData 适配器
self.register_adapter("mydata", lambda: MyDataAdapter())
# 设置元数据
self.metadata["mydata"] = {
"name": "mydata",
"type": "http",
"version": "1.0.0",
"description": "MyData 金融数据接口",
"updated_at": datetime.now()
}
# 默认配置
self.configs["mydata"] = {
"enabled": False,
"config": {
"api_key": "",
"base_url": "https://api.mydata.com"
}
}
```
**步骤3**: 前端会自动从 `/v1/admin/adapters` 接口获取适配器列表，无需额外修改。
---
### 场景3：添加新的接口测试用例
#### Go 实现
**步骤1**: 在测试服务中添加用例 (`internal/service/test.go`)
```go
func (s *TestServiceImpl) GetAPITestList(...) {
categories := []api.APITestCategory{
// ... 现有分类
{
Name: "自定义接口",
Items: []api.APITestCase{
{
ID: "custom_endpoint",
Name: "自定义端点测试",
Method: "GET",
Path: "/v1/custom/{id}",
Description: "测试自定义端点",
Params: map[string]string{
"id": "123",
},
},
{
ID: "custom_post",
Name: "自定义POST接口",
Method: "POST",
Path: "/v1/custom/create",
Description: "测试创建接口",
Body: map[string]interface{}{
"name": "test",
"value": 100,
},
},
},
},
}
return &api.APITestListData{
Categories: categories,
BaseURL: "",
}, nil
}
```
#### Python 实现
**步骤1**: 在测试服务中添加用例 (`app/services/test_service.py`)
```python
def get_api_test_list(self) -> APITestListData:
categories = [
# ... 现有分类
APITestCategory(
name="自定义接口",
items=[
APITestCase(
id="custom_endpoint",
name="自定义端点测试",
method="GET",
path="/v1/custom/{id}",
description="测试自定义端点",
params={"id": "123"}
),
APITestCase(
id="custom_post",
name="自定义POST接口",
method="POST",
path="/v1/custom/create",
description="测试创建接口",
body={"name": "test", "value": 100}
),
]
),
]
return APITestListData(categories=categories, base_url="")
```
**步骤2**: 前端会自动显示新的测试用例，无需修改前端代码。
---
## 调试技巧
### 1. 查看配置变更日志
#### Go
```go
// 在 ConfigService 中添加日志
func (s *ConfigServiceImpl) UpdateConfig(...) {
log.Printf("[Config] Updating config type: %s", req.Type)
log.Printf("[Config] Update items: %+v", req.Items)
// ... 更新逻辑
log.Printf("[Config] Config updated successfully, need restart: %v", needRestart)
}
```
#### Python
```python
# 在 ConfigService 中添加日志
from app.core.logger import info
def update_config(self, req: ConfigUpdateRequest) -> ConfigUpdateData:
info(f"[Config] Updating config type: {req.type}")
info(f"[Config] Update items: {req.items}")
# ... 更新逻辑
info(f"[Config] Config updated successfully, need restart: {need_restart}")
```
### 2. 适配器调试
#### Go
```go
// 在适配器中添加详细日志
func (a *MyAdapter) FetchKLines(...) {
log.Printf("[MyAdapter] FetchKLines called: symbol=%s, start=%s, end=%s, freq=%s",
symbol, start, end, freq)
// ... 实现逻辑
log.Printf("[MyAdapter] FetchKLines completed: got %d records", len(result))
return result, nil
}
```
#### Python
```python
# 在适配器中添加详细日志
from app.core.logger import info
async def fetch_klines(self, symbol: str, start: str, end: str, freq: str) -> List[KLineData]:
info(f"[MyAdapter] FetchKLines called: symbol={symbol}, start={start}, end={end}, freq={freq}")
# ... 实现逻辑
info(f"[MyAdapter] FetchKLines completed: got {len(result)} records")
return result
```
### 3. 使用调试器
#### Go (Delve)
```bash
# 启动调试模式
dlv debug ./cmd/server
# 在关键位置设置断点
(dlv) break internal/service/config.go:100
(dlv) break internal/service/adapter.go:150
# 运行
(dlv) continue
```
#### Python (pdb 或 IDE)
```python
# 代码中插入断点
import pdb; pdb.set_trace()
# 或使用IPython
from IPython import embed; embed()
```
或使用 VS Code / PyCharm 的图形化调试。
---
## 测试
### 单元测试示例
#### Go
```go
// internal/service/config_test.go
package service
import (
"context"
"testing"
"market-data-service/api"
)
func TestConfigService_UpdateConfig(t *testing.T) {
// 创建临时配置文件
tmpFile := t.TempDir() + "/config.json"
service, err := NewConfigService(tmpFile)
if err != nil {
t.Fatal(err)
}
// 测试更新配置
req := &api.ConfigUpdateRequest{
Type: api.ConfigTypeServer,
Items: map[string]interface{}{
"port": 9090,
},
}
result, err := service.UpdateConfig(context.Background(), req)
if err != nil {
t.Fatal(err)
}
if !result.Success {
t.Errorf("expected success, got: %v", result.Message)
}
// 验证配置已更新
cfg := service.GetCurrentConfig()
if cfg.Server.Port != 9090 {
t.Errorf("expected port 9090, got: %d", cfg.Server.Port)
}
}
```
#### Python
```python
# tests/test_config_service.py
import pytest
from app.services.config_service import ConfigService
from app.models import ConfigUpdateRequest, ConfigType
def test_config_service_update():
# 创建服务
service = ConfigService()
# 测试更新配置
req = ConfigUpdateRequest(
type=ConfigType.SERVER,
items={"port": 9090}
)
result = service.update_config(req)
assert result.success is True
assert service.get_current_config().server.port == 9090
```
运行测试：
```bash
# Go
go test ./internal/service/...
# Python
pytest tests/
```
### API 测试
```bash
# 使用 httpie 测试
http GET localhost:8080/v1/admin/system/status
http POST localhost:8080/v1/admin/config \
type=server \
items:='{"port": 8080}'
# 使用 curl 测试
curl -X POST "http://localhost:8080/v1/admin/tests/api/run" \
-H "Content-Type: application/json" \
-d '{"id": "stock_klines", "params": {"symbol": "000001.SZ"}}'
```
---
## 常见问题
### Q1: 配置热加载不生效？
**检查点**:
1. 配置文件路径是否正确
2. 配置类型是否支持热加载（数据库配置需重启）
3. 回调函数是否正确注册
**Go**:
```go
// 检查配置是否正确保存
config, _ := service.GetConfigList(ctx, &api.ConfigListRequest{})
fmt.Printf("Current config: %+v\n", config)
```
**Python**:
```python
# 检查配置是否正确保存
config = service.get_config_list(ConfigListRequest())
print(f"Current config: {config}")
```
### Q2: 适配器无法启用？
**检查点**:
1. 适配器是否已注册
2. 配置项是否完整（如 token）
3. Connect 方法是否返回错误
**Go**:
```go
// 添加调试日志
func (s *AdapterServiceImpl) ToggleAdapter(...) {
log.Printf("[Adapter] Toggling adapter: %s, enable: %v", req.Name, req.Enable)
cfg, ok := s.configs[req.Name]
if !ok {
log.Printf("[Adapter] Adapter not found: %s", req.Name)
return fmt.Errorf("adapter not found: %s", req.Name)
}
log.Printf("[Adapter] Current config: %+v", cfg)
// ...
}
```
**Python**:
```python
# 添加调试日志
def toggle_adapter(self, req: AdapterToggleRequest) -> None:
info(f"[Adapter] Toggling adapter: {req.name}, enable: {req.enable}")
if req.name not in self.configs:
info(f"[Adapter] Adapter not found: {req.name}")
raise ValueError(f"Adapter not found: {req.name}")
info(f"[Adapter] Current config: {self.configs[req.name]}")
# ...
```
### Q3: 前端页面空白？
**检查点**:
1. 浏览器控制台是否有错误
2. API 接口是否返回正确数据
3. 静态资源是否正确加载
```bash
# 检查管理后台路由是否正常
curl http://localhost:8080/admin
# 检查API接口
curl http://localhost:8080/v1/admin/system/status
```
### Q4: Python 依赖安装失败？
**解决方案**:
```bash
# 1. 升级pip
pip install --upgrade pip
# 2. 安装系统依赖（Ubuntu/Debian）
sudo apt install -y python3-dev libpq-dev gcc
# 3. 使用国内镜像
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
# 4. 单独安装有问题的包
pip install sqlalchemy psycopg2-binary --force-reinstall
```
---
## 性能优化建议
### 1. 配置缓存
#### Go
```go
// 添加配置缓存层
type CachedConfigService struct {
inner ConfigService
cache *config.Config
mu sync.RWMutex
ttl time.Duration
lastUpdate time.Time
}
func (s *CachedConfigService) GetCurrentConfig() *config.Config {
s.mu.RLock()
if time.Since(s.lastUpdate) < s.ttl {
defer s.mu.RUnlock()
return s.cache
}
s.mu.RUnlock()
// 缓存过期，重新加载
s.mu.Lock()
defer s.mu.Unlock()
s.cache = s.inner.GetCurrentConfig()
s.lastUpdate = time.Now()
return s.cache
}
```
#### Python
```python
# 添加配置缓存层
from functools import lru_cache
import time
class CachedConfigService:
def __init__(self, inner: ConfigService, ttl: int = 60):
self.inner = inner
self.cache = None
self.ttl = ttl
self.last_update = 0
self.lock = threading.RLock()
def get_current_config(self):
with self.lock:
if time.time() - self.last_update < self.ttl:
return self.cache
self.cache = self.inner.get_current_config()
self.last_update = time.time()
return self.cache
```
### 2. 数据库连接池
#### Go
```go
// 在 repository/database.go 中配置
db.SetMaxOpenConns(100)
db.SetMaxIdleConns(10)
db.SetConnMaxLifetime(time.Hour)
```
#### Python
```python
# 在 app/repositories/database.py 中配置
engine = create_engine(
config.database.database_url,
pool_size=10,
max_overflow=20,
pool_pre_ping=True,
pool_recycle=3600,
)
```
---
## 最佳实践
### 1. 配置管理
- 敏感信息（密码、token）使用 `********` 掩码显示
- 重要配置变更记录操作日志
- 配置验证在更新前进行
### 2. 适配器开发
- 实现完整的 `HealthCheck` 方法
- 处理网络超时和重试
- 使用连接池管理资源
### 3. 接口测试
- 使用参数化测试数据
- 测试失败时记录详细错误信息
- 清理测试产生的临时数据
### 4. 代码组织
**Go**:
- 按功能分层（api/internal/pkg/cmd）
- 使用接口定义依赖
- 错误处理要明确
**Python**:
- 使用类型注解
- 遵循PEP 8规范
- 合理使用异步
---
## 参考资源
### Go
- [Gin 框架文档](https://gin-gonic.com/docs/)
- [Go 并发模式](https://go.dev/blog/pipelines)
- [PostgreSQL 文档](https://www.postgresql.org/docs/)
### Python
- [FastAPI 文档](https://fastapi.tiangolo.com/)
- [SQLAlchemy 文档](https://docs.sqlalchemy.org/)
- [Pydantic 文档](https://docs.pydantic.dev/)
---
**文档结束**

450
docs/go-installation-guide.md
Unescape View File

@ -0,0 +1,450 @@
# Go 环境安装指南
**版本**: Go 1.21.6
**适用系统**: Windows / Linux / macOS
---
## 目录
1. [Windows 安装](#一windows-安装)
2. [Linux 安装](#二linux-安装)
3. [macOS 安装](#三macos-安装)
4. [验证安装](#四验证安装)
5. [配置 GOPROXY](#五配置-goproxy)
6. [常见问题](#六常见问题)
---
## 一、Windows 安装
### 方法一：使用 PowerShell 脚本（推荐）
1. **打开 PowerShell**（以管理员身份）
2. **运行安装脚本**
```powershell
cd d:\fs_workspace\market-data-service\scripts
.\install-go-windows.ps1
```
3. **等待安装完成**，脚本会自动：
- 下载 Go 1.21.6 安装包
- 执行安装
- 配置环境变量
- 设置国内镜像
4. **重新打开 PowerShell**，验证安装
```powershell
go version
```
### 方法二：手动安装
1. **下载安装包**
访问官方下载页面：
```
https://go.dev/dl/go1.21.6.windows-amd64.msi
```
2. **运行安装程序**
双击下载的 `.msi` 文件，按向导完成安装
3. **验证安装**
打开命令提示符，运行：
```cmd
go version
```
应输出：
```
go version go1.21.6 windows/amd64
```
### 环境变量配置
如果手动安装后 `go` 命令不可用，需要手动配置环境变量：
1. **右键"此电脑"** → **属性** → **高级系统设置**
2. **环境变量** → **系统变量** → **Path**
3. **添加以下路径**：
```
C:\Program Files\Go\bin
```
4. **新建用户变量**：
- 变量名：`GOPATH`
- 变量值：`%USERPROFILE%\go`
5. **重启命令提示符**
---
## 二、Linux 安装
### 方法一：使用安装脚本（推荐）
1. **打开终端**
2. **运行安装脚本**
```bash
cd /path/to/market-data-service/scripts
chmod +x install-go-linux.sh
./install-go-linux.sh
```
3. **使环境变量生效**
```bash
source ~/.bashrc
# 或
source ~/.zshrc
```
4. **验证安装**
```bash
go version
```
### 方法二：使用包管理器
**Ubuntu/Debian:**
```bash
# 添加 PPA
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt update
# 安装 Go
sudo apt install golang-1.21
# 创建软链接
sudo ln -s /usr/lib/go-1.21/bin/go /usr/local/bin/go
```
**CentOS/RHEL:**
```bash
# 使用 EPEL
sudo yum install epel-release
sudo yum install golang
# 或下载二进制
cd /tmp
wget https://go.dev/dl/go1.21.6.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz
```
### 方法三：手动安装
1. **下载安装包**
```bash
cd /tmp
wget https://go.dev/dl/go1.21.6.linux-amd64.tar.gz
```
2. **解压到 /usr/local**
```bash
sudo tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz
```
3. **配置环境变量**
编辑 `~/.bashrc` 或 `~/.zshrc`：
```bash
export PATH=$PATH:/usr/local/go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
export GOPROXY=https://goproxy.cn,direct
```
4. **使配置生效**
```bash
source ~/.bashrc
```
5. **验证安装**
```bash
go version
```
---
## 三、macOS 安装
### 方法一：使用 Homebrew（推荐）
1. **安装 Homebrew**（如未安装）
```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
2. **安装 Go**
```bash
brew install go@1.21
```
3. **添加到 PATH**
```bash
echo 'export PATH="/opt/homebrew/opt/go@1.21/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc
```
4. **验证安装**
```bash
go version
```
### 方法二：使用安装脚本
1. **下载并运行脚本**
```bash
cd /path/to/market-data-service/scripts
chmod +x install-go-linux.sh
./install-go-linux.sh
```
2. **使环境变量生效**
```bash
source ~/.zshrc
```
### 方法三：手动安装
1. **下载安装包**
```bash
cd /tmp
curl -L -o go1.21.6.darwin-amd64.tar.gz https://go.dev/dl/go1.21.6.darwin-amd64.tar.gz
```
2. **解压**
```bash
sudo tar -C /usr/local -xzf go1.21.6.darwin-amd64.tar.gz
```
3. **配置环境变量**
编辑 `~/.zshrc`：
```bash
export PATH=$PATH:/usr/local/go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
export GOPROXY=https://goproxy.cn,direct
```
4. **使配置生效**
```bash
source ~/.zshrc
```
---
## 四、验证安装
### 4.1 基本验证
```bash
# 查看 Go 版本
go version
# 输出: go version go1.21.6 xxx/xxx
# 查看 Go 环境
go env
# 查看特定变量
go env GOPATH
go env GOROOT
go env GOPROXY
```
### 4.2 运行测试程序
创建测试文件 `hello.go`：
```go
package main
import "fmt"
func main() {
fmt.Println("Hello, Go!")
fmt.Printf("Go version: %s\n", runtime.Version())
}
```
运行：
```bash
go run hello.go
```
应输出：
```
Hello, Go!
Go version: go1.21.6
```
### 4.3 编译测试
```bash
# 编译
go build -o hello hello.go
# 运行
./hello # Linux/macOS
hello.exe # Windows
```
---
## 五、配置 GOPROXY
### 为什么要配置？
Go 默认使用国外代理，在国内下载依赖可能很慢或失败。
### 配置方法
**临时配置（当前终端）**
```bash
# Windows
go env -w GOPROXY=https://goproxy.cn,direct
# Linux/macOS
export GOPROXY=https://goproxy.cn,direct
```
**永久配置**
Windows:
```powershell
[Environment]::SetEnvironmentVariable("GOPROXY", "https://goproxy.cn,direct", "User")
```
Linux/macOS:
```bash
echo 'export GOPROXY=https://goproxy.cn,direct' >> ~/.bashrc
source ~/.bashrc
```
### 验证配置
```bash
go env GOPROXY
# 输出: https://goproxy.cn,direct
```
### 其他可用代理
| 代理地址 | 说明 |
|----------|------|
| `https://goproxy.cn` | 七牛云，国内推荐 |
| `https://goproxy.io` | 官方，全球可用 |
| `https://mirrors.aliyun.com/goproxy/` | 阿里云 |
| `https://proxy.golang.org` | Google 官方 |
---
## 六、常见问题
### Q1: 安装后 `go` 命令不可用
**原因**: 环境变量未配置或需要重启终端
**解决**:
```bash
# Windows: 重新打开 PowerShell
# Linux/macOS:
source ~/.bashrc
# 或
source ~/.zshrc
```
### Q2: 下载依赖超时
**原因**: 未配置 GOPROXY 或网络问题
**解决**:
```bash
go env -w GOPROXY=https://goproxy.cn,direct
```
### Q3: 权限不足（Linux/macOS）
**解决**:
```bash
# 使用 sudo 运行安装脚本
sudo ./install-go-linux.sh
# 或手动解压到用户目录
tar -C $HOME/.local -xzf go1.21.6.linux-amd64.tar.gz
export PATH=$PATH:$HOME/.local/go/bin
```
### Q4: 如何卸载 Go
**Windows:**
1. 控制面板 → 程序和功能 → 卸载 Go
2. 删除环境变量中的 Go 相关配置
**Linux/macOS:**
```bash
# 删除安装目录
sudo rm -rf /usr/local/go
# 删除环境变量配置（编辑 ~/.bashrc 或 ~/.zshrc）
```
### Q5: 安装多个 Go 版本
使用 `g` 版本管理器：
```bash
# 安装 g
go install github.com/voidint/g@latest
# 安装特定版本
g install 1.21.6
g install 1.20.0
# 切换版本
g use 1.21.6
```
---
## 七、下一步
安装完成 Go 后，可以继续：
1. **返回项目目录**
```bash
cd d:\fs_workspace\market-data-service
```
2. **下载依赖**
```bash
go mod download
```
3. **启动服务**
```bash
go run ./cmd/server
```
4. **访问管理后台**
```
http://localhost:8080/admin
```
---
## 参考资源
- [Go 官方下载](https://go.dev/dl/)
- [Go 官方安装文档](https://go.dev/doc/install)
- [Go 国内镜像](https://goproxy.cn/)
---
**文档结束**

686
docs/startup-guide.md
Unescape View File

@ -0,0 +1,686 @@
# 行情数据服务启动指南
**版本**: v2.0
**日期**: 2026-03-08
**适用系统**: Windows / Linux / macOS
**支持实现**: Go / Python
---
## 📢 重要说明
本项目支持 **Go** 和 **Python** 双实现：
| 实现方式 | 推荐场景 | 目录 |
|----------|----------|------|
| **Go** | 生产环境、高并发 | `market-data-service/` |
| **Python** | 快速开发、原型验证 | `python_market_data_service/` |
本文档包含两种实现方式的启动说明，请按需选择。
---
## 目录
1. [快速选择指南](#快速选择指南)
2. [环境要求](#一环境要求)
3. [Go 实现启动](#二go-实现启动)
4. [Python 实现启动](#三python-实现启动)
5. [访问管理后台](#四访问管理后台)
6. [常见问题](#五常见问题)
7. [API 测试](#六api-测试)
---
## 快速选择指南
### 选择 Go 实现，如果你：
- 需要部署到生产环境
- 有高并发性能要求
- 偏好编译型语言的稳定性
### 选择 Python 实现，如果你：
- 需要快速验证功能
- 需要频繁调试和修改
- 需要更好的数据源生态支持（Tushare原生）
---
## 一、环境要求
### 1.1 必需组件
| 组件 | Go实现 | Python实现 | 说明 |
|------|--------|------------|------|
| Go | 1.21+ | - | Go编程语言运行时 |
| Python | - | 3.10+ | Python解释器 |
| PostgreSQL | 15+ | 15+ | 数据存储（可选） |
### 1.2 检查环境
```bash
# Go实现 - 检查 Go 版本
go version
# 输出: go version go1.21.x windows/amd64
# Python实现 - 检查 Python 版本
python --version
# 输出: Python 3.10.x
# 检查 PostgreSQL（可选）
psql --version
# 输出: psql (PostgreSQL) 15.x
```
---
## 二、Go 实现启动
### 2.1 安装 Go
**Windows:**
1. 下载安装包: https://go.dev/dl/go1.21.6.windows-amd64.msi
2. 双击安装，按向导完成
3. 打开新的命令提示符验证: `go version`
**Linux:**
```bash
# 下载并解压
wget https://go.dev/dl/go1.21.6.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz
# 添加到 PATH
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version
```
**macOS:**
```bash
brew install go
go version
```
### 2.2 安装依赖
```bash
cd market-data-service
# 设置国内镜像（推荐）
go env -w GOPROXY=https://goproxy.cn,direct
# 下载依赖
go mod download
# 验证依赖
go mod verify
```
### 2.3 配置文件
项目使用 JSON 格式的配置文件，默认路径为 `./config.json`。
**配置文件示例**: `config.json`
```json
{
"server": {
"port": 8080,
"mode": "debug",
"api_key": "demo-api-key-2024"
},
"database": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "postgres",
"database": "marketdata"
},
"sources": {
"stock": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token-here",
"base_url": "https://api.tushare.pro"
}
}
}
},
"futures": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "your-tushare-token-here",
"base_url": "https://api.tushare.pro"
}
}
}
}
}
}
```
### 2.4 启动服务
**方式一：直接运行（开发模式）**
```bash
cd market-data-service
# 设置环境变量（可选）
set PORT=8080
set CONFIG_PATH=./config.json
# 启动服务
go run ./cmd/server
```
**预期输出:**
```
[GIN-debug] [WARNING] Running in "debug" mode.
2026/03/08 14:00:00 Server starting on port 8080
2026/03/08 14:00:00 Admin dashboard: http://localhost:8080/admin
```
**方式二：编译后运行（生产模式）**
```bash
set GIN_MODE=release
go build -o market-server.exe ./cmd/server
# 运行
.\market-server.exe
```
**方式三：使用 Makefile**
```bash
# 查看可用命令
make help
# 启动服务
make run
# 编译
make build
```
---
## 三、Python 实现启动
### 3.1 安装 Python
**Windows:**
1. 下载安装包: https://www.python.org/downloads/windows/
2. 选择 Python 3.10+，安装时勾选 "Add to PATH"
3. 验证: `python --version`
**Linux:**
```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y python3.10 python3.10-venv python3-pip
# CentOS/RHEL
sudo yum install -y python310 python310-pip
python3 --version
```
**macOS:**
```bash
brew install python@3.10
python3 --version
```
### 3.2 创建虚拟环境
```bash
cd python_market_data_service
# 创建虚拟环境
python -m venv venv
# 激活虚拟环境
# Windows:
venv\Scripts\activate
# Linux/Mac:
source venv/bin/activate
```
### 3.3 安装依赖
```bash
# 升级pip
pip install --upgrade pip
# 安装依赖
pip install -r requirements.txt
# 安装Tushare（需单独安装）
pip install tushare
```
### 3.4 配置文件
Python实现使用与Go相同的 `config.json` 配置文件。
**环境变量（可选）**:
```bash
# Windows
set PORT=8080
set DATABASE_URL="postgresql://postgres:postgres@localhost:5432/marketdata"
set TUSHARE_TOKEN="your_token"
# Linux/Mac
export PORT=8080
export DATABASE_URL="postgresql://postgres:postgres@localhost:5432/marketdata"
export TUSHARE_TOKEN="your_token"
```
### 3.5 初始化数据库
```bash
# 使用Python初始化（SQLAlchemy会自动创建表）
python -c "from app.repositories.database import init_db; init_db()"
# 或使用SQL脚本（与Go相同）
psql postgresql://postgres:postgres@localhost:5432/marketdata -f memory/2026-03-07-database-schema.sql
```
### 3.6 启动服务
**方式一：直接运行（开发模式）**
```bash
# 确保在虚拟环境中
source venv/bin/activate # Linux/Mac
# 或
venv\Scripts\activate # Windows
# 启动服务
python -m app.main
```
**预期输出:**
```
INFO: Uvicorn running on http://0.0.0.0:8080
INFO: Application startup complete.
Admin dashboard: http://localhost:8080/admin
```
**方式二：使用Uvicorn（推荐）**
```bash
# 开发模式（热重载）
uvicorn app.main:app --reload --port 8080
# 生产模式
uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
```
**方式三：使用Gunicorn（Linux/Mac）**
```bash
gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8080
```
### 3.7 同步基础数据（可选）
```bash
# 同步股票列表
python scripts/sync_data.py --type stocks
# 同步期货列表
python scripts/sync_data.py --type futures
# 同步交易日历
python scripts/sync_data.py --type calendar --start 20240101 --end 20241231
```
---
## 四、访问管理后台
### 4.1 打开管理后台
服务启动后，在浏览器中访问：
```
http://localhost:8080/admin
```
### 4.2 功能导航
| 菜单 | 功能描述 | 主要操作 |
|------|----------|----------|
| **📈 系统概览** | 查看系统运行状态 | 查看状态卡片、内存使用、执行热加载/重启 |
| **⚙️ 配置管理** | 在线修改系统配置 | 编辑服务器/数据库/Redis/数据源配置 |
| **🔌 数据源适配** | 管理数据适配器 | 启用/禁用适配器、修改适配器配置 |
| **🧪 接口测试** | 测试API和WebSocket | 运行接口测试、查看测试历史 |
### 4.3 API文档（Python特有）
FastAPI自动生成API文档：
- Swagger UI: `http://localhost:8080/docs`
- ReDoc: `http://localhost:8080/redoc`
### 4.4 首次使用步骤
**步骤1: 查看系统状态**
1. 打开 `http://localhost:8080/admin`
2. 默认进入"系统概览"页面
3. 查看运行状态、运行时长、内存使用等信息
**步骤2: 配置数据源（可选）**
1. 点击左侧"配置管理"
2. 找到"数据源配置"部分
3. 输入 Tushare Token
4. 点击"保存配置"
**步骤3: 测试接口**
1. 点击左侧"接口测试"
2. 选择"API测试"页签
3. 点击任意测试用例的"运行测试"按钮
4. 查看测试结果
---
## 五、常见问题
### 5.1 端口被占用
**错误信息:**
```
listen tcp :8080: bind: Only one usage of each socket address...
```
**解决方案:**
**Go:**
```bash
set PORT=8081
go run ./cmd/server
```
**Python:**
```bash
set PORT=8081
python -m app.main
# 或
uvicorn app.main:app --port 8081
```
### 5.2 数据库连接失败
**Go:**
```bash
# 启动 PostgreSQL 服务
# Windows: 服务管理器启动 postgresql-x64-15
# Linux: sudo systemctl start postgresql
# 或使用 Docker
docker run -d --name postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:15
```
**Python:**
```bash
# 检查SQLAlchemy连接字符串格式
# 应该是: postgresql://user:password@host:port/database
# 测试连接
python -c "from app.repositories.database import engine; print('OK')"
```
### 5.3 依赖问题
**Go - 依赖下载失败:**
```bash
go env -w GOPROXY=https://goproxy.cn,direct
go mod download
```
**Python - 依赖安装失败:**
```bash
# 升级pip
pip install --upgrade pip
# 安装系统依赖（Ubuntu/Debian）
sudo apt install -y python3-dev libpq-dev gcc
# 使用国内镜像
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### 5.4 Python特有 - 模块导入错误
**错误信息:**
```
ModuleNotFoundError: No module named 'app'
```
**解决方案:**
```bash
# 确保在项目根目录
cd python_market_data_service
# 设置PYTHONPATH
export PYTHONPATH=$(pwd) # Linux/Mac
# 或
set PYTHONPATH=%cd% # Windows
# 或者使用 -m 方式运行
python -m app.main
```
### 5.5 管理后台页面空白
**排查步骤:**
1. **检查服务是否正常运行**
```bash
curl http://localhost:8080/v1/admin/health
# 应返回: {"status":"healthy",...}
```
2. **检查端口是否正确**
```bash
# Windows
netstat -an | findstr LISTENING
# Linux/Mac
netstat -tlnp | grep 8080
```
3. **检查浏览器控制台**
- F12 打开开发者工具
- 查看 Console 是否有报错
### 5.6 热加载不生效
**Go:**
```bash
# 确认配置文件路径
echo %CONFIG_PATH%
# 调用热加载 API 测试
curl -X POST http://localhost:8080/v1/admin/system/reload
```
**Python:**
```bash
# Python自动支持热重载（开发模式）
# 修改代码后服务会自动重启
# 配置热加载
curl -X POST http://localhost:8080/v1/admin/system/reload
```
---
## 六、API 测试
### 6.1 使用 curl 测试
**健康检查:**
```bash
curl http://localhost:8080/v1/admin/health
```
**系统状态查询:**
```bash
curl http://localhost:8080/v1/admin/system/status
```
**热加载配置:**
```bash
curl -X POST http://localhost:8080/v1/admin/system/reload \
-H "Content-Type: application/json" \
-d '{"config_type": "source"}'
```
### 6.2 使用 httpie 测试（推荐）
```bash
# 安装 httpie
pip install httpie
# 系统状态
http :8080/v1/admin/system/status
# 热加载
http POST :8080/v1/admin/system/reload config_type=source
# 获取适配器列表
http :8080/v1/admin/adapters
# 执行测试
http POST :8080/v1/admin/tests/api/run \
id=stock_klines \
params:='{"symbol": "000001.SZ"}'
```
---
## 七、生产环境部署
### 7.1 Go 实现
```bash
# 设置生产模式
set GIN_MODE=release
# 编译
set GOOS=linux
set GOARCH=amd64
go build -ldflags="-s -w" -o market-server ./cmd/server
# 运行
./market-server
```
### 7.2 Python 实现
```bash
# 安装生产依赖
pip install -r requirements.txt
# 使用Gunicorn（Linux/Mac）
gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8080
# 或使用Uvicorn
uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 4
```
### 7.3 Docker 部署
**Go:**
```dockerfile
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o market-server ./cmd/server
FROM alpine:latest
COPY --from=builder /app/market-server .
EXPOSE 8080
CMD ["./market-server"]
```
**Python:**
```dockerfile
FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8080
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
```
---
## 附录
### A. 目录结构
**Go实现:**
```
market-data-service/
├── api/ # API 定义
├── cmd/ # 程序入口
│ └── server/
│ └── main.go # 主程序
├── config.json # 配置文件
├── docs/ # 文档目录
├── internal/ # 内部实现
└── pkg/ # 公共包
```
**Python实现:**
```
python_market_data_service/
├── app/ # 应用代码
│ ├── api/ # API路由
│ ├── core/ # 核心模块
│ ├── models/ # 数据模型
│ ├── repositories/ # 数据访问
│ ├── services/ # 业务服务
│ ├── adapters/ # 数据源适配器
│ ├── websocket/ # WebSocket服务
│ └── main.py # 主程序
├── scripts/ # 工具脚本
├── config.json # 配置文件
├── requirements.txt # Python依赖
└── README.md # 项目说明
```
### B. 默认端口
| 服务 | 端口 | 说明 |
|------|------|------|
| HTTP API | 8080 | REST API 服务 |
| WebSocket | 8080 | 共用 HTTP 端口 |
| PostgreSQL | 5432 | 数据库 |
### C. 相关文档
- [部署文档](../DEPLOY.md)
- [开发文档](./admin-dashboard-development.md)
- [API 速查表](./admin-api-quick-reference.md)
- [架构设计](./architecture.md)
- [开发指南](./development-guide.md)
- [Python迁移指南](../python_market_data_service/MIGRATION_GUIDE.md)
---
**文档结束**

37
go.mod
Unescape View File

@ -0,0 +1,37 @@
module market-data-service
go 1.21
require (
github.com/gin-gonic/gin v1.9.1
github.com/google/uuid v1.6.0
github.com/gorilla/websocket v1.5.1
github.com/lib/pq v1.10.9
)
require (
github.com/bytedance/sonic v1.9.1 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/crypto v0.9.0 // indirect
golang.org/x/net v0.10.0 // indirect
golang.org/x/sys v0.8.0 // indirect
golang.org/x/text v0.9.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

236
internal/handler/admin.go
Unescape View File

@ -0,0 +1,236 @@
// Package handler 管理后台Handler实现
package handler
import (
"context"
"fmt"
"market-data-service/api"
"market-data-service/internal/service"
)
// AdminHandlerImpl 管理后台Handler实现
type AdminHandlerImpl struct {
configService service.ConfigService
adapterService service.AdapterService
testService service.TestService
}
// NewAdminHandlerImpl 创建管理后台Handler
func NewAdminHandlerImpl(
configService service.ConfigService,
adapterService service.AdapterService,
testService service.TestService,
) *AdminHandlerImpl {
return &AdminHandlerImpl{
configService: configService,
adapterService: adapterService,
testService: testService,
}
}
// Ensure interfaces are implemented
var _ api.ConfigHandler = (*AdminHandlerImpl)(nil)
var _ api.AdapterHandler = (*AdminHandlerImpl)(nil)
var _ api.TestHandler = (*AdminHandlerImpl)(nil)
// ============================================
// 配置管理接口实现
// ============================================
// GetConfigList 获取配置列表
func (h *AdminHandlerImpl) GetConfigList(ctx context.Context, req *api.ConfigListRequest) (*api.Response, error) {
data, err := h.configService.GetConfigList(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// UpdateConfig 更新配置
func (h *AdminHandlerImpl) UpdateConfig(ctx context.Context, req *api.ConfigUpdateRequest) (*api.Response, error) {
data, err := h.configService.UpdateConfig(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ReloadConfig 热加载配置
func (h *AdminHandlerImpl) ReloadConfig(ctx context.Context, req *api.ReloadRequest) (*api.Response, error) {
data, err := h.configService.ReloadConfig(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// GetSystemStatus 获取系统状态
func (h *AdminHandlerImpl) GetSystemStatus(ctx context.Context) (*api.Response, error) {
data, err := h.configService.GetSystemStatus(ctx)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ============================================
// 适配器管理接口实现
// ============================================
// GetAdapterList 获取适配器列表
func (h *AdminHandlerImpl) GetAdapterList(ctx context.Context) (*api.Response, error) {
data, err := h.adapterService.GetAdapterList(ctx)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ToggleAdapter 启用/禁用适配器
func (h *AdminHandlerImpl) ToggleAdapter(ctx context.Context, req *api.AdapterToggleRequest) (*api.Response, error) {
if err := h.adapterService.ToggleAdapter(ctx, req); err != nil {
return &api.Response{
Code: 500,
Message: err.Error(),
}, nil
}
return &api.Response{
Code: 0,
Message: "success",
}, nil
}
// UpdateAdapterConfig 更新适配器配置
func (h *AdminHandlerImpl) UpdateAdapterConfig(ctx context.Context, req *api.AdapterConfigUpdateRequest) (*api.Response, error) {
if err := h.adapterService.UpdateAdapterConfig(ctx, req); err != nil {
return &api.Response{
Code: 500,
Message: err.Error(),
}, nil
}
return &api.Response{
Code: 0,
Message: "success",
}, nil
}
// ============================================
// 测试管理接口实现
// ============================================
// GetAPITestList 获取API测试列表
func (h *AdminHandlerImpl) GetAPITestList(ctx context.Context) (*api.Response, error) {
data, err := h.testService.GetAPITestList(ctx)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// RunAPITest 执行API测试
func (h *AdminHandlerImpl) RunAPITest(ctx context.Context, req *api.APITestRequest) (*api.Response, error) {
// 获取基础URL
baseURL := "http://localhost:8080"
if cfg := h.configService.GetCurrentConfig(); cfg != nil && cfg.Server.Port != 0 {
baseURL = fmt.Sprintf("http://localhost:%d", cfg.Server.Port)
}
data, err := h.testService.RunAPITest(ctx, baseURL, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// GetWSTestList 获取WebSocket测试列表
func (h *AdminHandlerImpl) GetWSTestList(ctx context.Context) (*api.Response, error) {
data, err := h.testService.GetWSTestList(ctx)
if err != nil {
return nil, err
}
// 设置WebSocket URL
wsURL := "ws://localhost:8080/v1/stream"
if cfg := h.configService.GetCurrentConfig(); cfg != nil && cfg.Server.Port != 0 {
wsURL = "ws://localhost:" + string(rune(cfg.Server.Port)) + "/v1/stream"
}
data.WSURL = wsURL
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// RunWSTest 执行WebSocket测试
func (h *AdminHandlerImpl) RunWSTest(ctx context.Context, req *api.WSTestRequest) (*api.Response, error) {
// 获取WebSocket URL
wsURL := "ws://localhost:8080/v1/stream"
if cfg := h.configService.GetCurrentConfig(); cfg != nil && cfg.Server.Port != 0 {
wsURL = fmt.Sprintf("ws://localhost:%d/v1/stream", cfg.Server.Port)
}
data, err := h.testService.RunWSTest(ctx, wsURL, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// GetTestHistory 获取测试历史
func (h *AdminHandlerImpl) GetTestHistory(ctx context.Context, req *api.TestHistoryRequest) (*api.Response, error) {
data, err := h.testService.GetTestHistory(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}

325
internal/handler/handler.go
Unescape View File

@ -0,0 +1,325 @@
// Package handler HTTP Handler实现
package handler
import (
"context"
"fmt"
"market-data-service/api"
"market-data-service/internal/service"
)
// Handler HTTP处理器实现
type Handler struct {
stockService service.StockService
futuresService service.FuturesService
adminService service.AdminService
}
// NewHandler 创建Handler
func NewHandler(
stockService service.StockService,
futuresService service.FuturesService,
adminService service.AdminService,
) *Handler {
return &Handler{
stockService: stockService,
futuresService: futuresService,
adminService: adminService,
}
}
// 确保Handler实现了api.Handler接口
var _ api.Handler = (*Handler)(nil)
// ============================================
// 股票接口实现
// ============================================
// QueryKLines 查询股票K线
func (h *Handler) QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.Response, error) {
// 参数校验
if err := validateKLineRequest(req); err != nil {
return nil, err
}
// 调用Service层
data, err := h.stockService.QueryKLines(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ListSymbols 查询股票标的列表
func (h *Handler) ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.Response, error) {
// 设置默认值
if req.Page <= 0 {
req.Page = 1
}
if req.Size <= 0 {
req.Size = 20
}
if req.Size > 100 {
req.Size = 100
}
data, err := h.stockService.ListSymbols(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// BatchQueryKLines 批量查询股票K线
func (h *Handler) BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.Response, error) {
if err := validateBatchKLineRequest(req); err != nil {
return nil, err
}
data, err := h.stockService.BatchQueryKLines(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ============================================
// 期货接口实现
// ============================================
// QueryKLines 查询期货K线（期货Handler）
func (h *Handler) QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.Response, error) {
if err := validateKLineRequest(req); err != nil {
return nil, err
}
data, err := h.futuresService.QueryKLines(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ListSymbols 查询期货标的列表
func (h *Handler) ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.Response, error) {
if req.Page <= 0 {
req.Page = 1
}
if req.Size <= 0 {
req.Size = 20
}
if req.Size > 100 {
req.Size = 100
}
data, err := h.futuresService.ListSymbols(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// BatchQueryKLines 批量查询期货K线
func (h *Handler) BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.Response, error) {
if err := validateBatchKLineRequest(req); err != nil {
return nil, err
}
data, err := h.futuresService.BatchQueryKLines(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// QueryContinuousKLines 查询主力连续合约K线（预留）
func (h *Handler) QueryContinuousKLines(ctx context.Context, underlying string, req *api.KLineQueryRequest) (*api.Response, error) {
// TODO: 首期返回空数据或错误提示
return &api.Response{
Code: 0,
Message: "success",
Data: &api.KLineData{
Symbol: underlying + ".MAIN",
Name: underlying + "主力连续",
Freq: req.Freq,
Count: 0,
Items: []api.KLineItem{},
},
}, nil
}
// ============================================
// 管理接口实现
// ============================================
// GetDataSourceStatus 获取数据源状态
func (h *Handler) GetDataSourceStatus(ctx context.Context) (*api.Response, error) {
data, err := h.adminService.GetDataSourceStatus(ctx)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// SwitchDataSource 切换数据源
func (h *Handler) SwitchDataSource(ctx context.Context, req *api.SourceSwitchRequest) (*api.Response, error) {
if err := h.adminService.SwitchDataSource(ctx, req); err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "数据源切换成功",
Data: nil,
}, nil
}
// BackfillData 历史数据补录
func (h *Handler) BackfillData(ctx context.Context, req *api.BackfillRequest) (*api.Response, error) {
// 异步启动补录任务
taskID, err := h.adminService.BackfillData(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "补录任务已启动",
Data: map[string]string{
"task_id": taskID,
},
}, nil
}
// HealthCheck 健康检查
func (h *Handler) HealthCheck(ctx context.Context) (*api.HealthResponse, error) {
// TODO: 检查数据库连接、数据源连通性等
return h.adminService.HealthCheck(ctx)
}
// ============================================
// 新增接口：交易日历和期货合约
// ============================================
// GetTradingDates 获取股票交易日历
func (h *Handler) GetTradingDates(ctx context.Context, req *api.TradingDatesRequest) (*api.Response, error) {
if err := validateTradingDatesRequest(req); err != nil {
return nil, err
}
// 通过请求路径判断是股票还是期货，这里简化处理
// 实际可以通过gin的context或其他方式判断
data, err := h.stockService.GetTradingDates(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// GetContractsByUnderlying 根据品种获取可交易合约
func (h *Handler) GetContractsByUnderlying(ctx context.Context, req *api.FuturesContractsRequest) (*api.Response, error) {
if req.Underlying == "" {
return nil, fmt.Errorf("underlying不能为空")
}
data, err := h.futuresService.GetContractsByUnderlying(ctx, req)
if err != nil {
return nil, err
}
return &api.Response{
Code: 0,
Message: "success",
Data: data,
}, nil
}
// ============================================
// 私有辅助函数
// ============================================
func validateKLineRequest(req *api.KLineQueryRequest) error {
if req.Symbol == "" {
return fmt.Errorf("symbol不能为空")
}
if req.Start == "" || len(req.Start) != 8 {
return fmt.Errorf("start日期格式错误，应为YYYYMMDD")
}
if req.End == "" || len(req.End) != 8 {
return fmt.Errorf("end日期格式错误，应为YYYYMMDD")
}
if req.Freq == "" {
req.Freq = api.Freq1D
}
return nil
}
func validateBatchKLineRequest(req *api.BatchKLineRequest) error {
if len(req.Symbols) == 0 {
return fmt.Errorf("symbols不能为空")
}
if len(req.Symbols) > 100 {
return fmt.Errorf("单次查询标的数不能超过100")
}
if req.Start == "" || len(req.Start) != 8 {
return fmt.Errorf("start日期格式错误，应为YYYYMMDD")
}
if req.End == "" || len(req.End) != 8 {
return fmt.Errorf("end日期格式错误，应为YYYYMMDD")
}
if req.Freq == "" {
req.Freq = api.Freq1D
}
return nil
}
func validateTradingDatesRequest(req *api.TradingDatesRequest) error {
if req.Start == "" || len(req.Start) != 8 {
return fmt.Errorf("start日期格式错误，应为YYYYMMDD")
}
if req.End == "" || len(req.End) != 8 {
return fmt.Errorf("end日期格式错误，应为YYYYMMDD")
}
return nil
}

22
internal/model/model.go
Unescape View File

@ -0,0 +1,22 @@
package model
// Model 领域模型定义
// Symbol 标的模型
type Symbol struct {
SymbolID string
Name string
// TODO: 补充字段
}
// KLine K线模型
type KLine struct {
Symbol string
Open float64
High float64
Low float64
Close float64
Volume int64
Amount float64
// TODO: 补充字段
}

256
internal/monitor/monitor.go
Unescape View File

@ -0,0 +1,256 @@
package monitor
import (
"context"
"fmt"
"log"
"time"
"market-data-service/api"
"market-data-service/internal/repository"
)
// Monitor 数据质量监控
type Monitor struct {
db *repository.DB
stockRepo *repository.StockRepository
futuresRepo *repository.FuturesRepository
sender AlertSender
}
// AlertSender 告警发送接口
type AlertSender interface {
SendAlert(title, content string) error
}
// NewMonitor 创建监控器
func NewMonitor(db *repository.DB, stockRepo *repository.StockRepository, futuresRepo *repository.FuturesRepository, sender AlertSender) *Monitor {
return &Monitor{
db: db,
stockRepo: stockRepo,
futuresRepo: futuresRepo,
sender: sender,
}
}
// CheckResult 检查结果
type CheckResult struct {
Symbol string
Freq string
CheckDate string
CheckType string
Status string // pass/fail
ExpectCount int
ActualCount int
Detail string
}
// DailyCheck 每日数据质量检查
func (m *Monitor) DailyCheck(ctx context.Context, checkDate string) error {
log.Printf("Starting daily data quality check for %s", checkDate)
// 检查股票数据
if err := m.checkStockData(ctx, checkDate); err != nil {
log.Printf("Stock data check failed: %v", err)
}
// 检查期货数据
if err := m.checkFuturesData(ctx, checkDate); err != nil {
log.Printf("Futures data check failed: %v", err)
}
log.Printf("Daily data quality check completed")
return nil
}
// checkStockData 检查股票数据质量
func (m *Monitor) checkStockData(ctx context.Context, checkDate string) error {
// 1. 获取所有活跃股票
symbols, _, err := m.stockRepo.ListSymbols(ctx, &api.SymbolListRequest{
Page: 1,
Size: 10000,
})
if err != nil {
return err
}
// 2. 检查1分钟线完整性（240条）
for _, symbol := range symbols {
result := m.checkKLineCompleteness(ctx, "stock", symbol.SymbolID, "1m", checkDate, 240)
if err := m.saveCheckResult(ctx, "stock", result); err != nil {
log.Printf("Failed to save check result: %v", err)
}
}
return nil
}
// checkFuturesData 检查期货数据质量
func (m *Monitor) checkFuturesData(ctx context.Context, checkDate string) error {
// 获取所有活跃期货合约
symbols, _, err := m.futuresRepo.ListSymbols(ctx, &api.SymbolListRequest{
Page: 1,
Size: 10000,
})
if err != nil {
return err
}
// 检查1分钟线完整性
for _, symbol := range symbols {
result := m.checkKLineCompleteness(ctx, "futures", symbol.SymbolID, "1m", checkDate, 240)
if err := m.saveCheckResult(ctx, "futures", result); err != nil {
log.Printf("Failed to save check result: %v", err)
}
}
return nil
}
// checkKLineCompleteness 检查K线完整性
func (m *Monitor) checkKLineCompleteness(ctx context.Context, assetType, symbol, freq, checkDate string, expectCount int) CheckResult {
result := CheckResult{
Symbol: symbol,
Freq: freq,
CheckDate: checkDate,
CheckType: "missing",
Status: "pass",
}
// 解析日期
start, _ := time.Parse("20060102", checkDate)
end := start.Add(24 * time.Hour).Add(-time.Second)
var actualCount int
var err error
if assetType == "stock" {
items, err := m.stockRepo.GetKLines(ctx, symbol, api.Frequency(freq), start, end, api.AdjustNone)
if err == nil {
actualCount = len(items)
}
} else {
items, err := m.futuresRepo.GetKLines(ctx, symbol, api.Frequency(freq), start, end)
if err == nil {
actualCount = len(items)
}
}
if err != nil {
result.Status = "fail"
result.Detail = fmt.Sprintf("Error querying data: %v", err)
return result
}
result.ExpectCount = expectCount
result.ActualCount = actualCount
// 判断缺失情况
if actualCount < expectCount {
result.Status = "fail"
result.Detail = fmt.Sprintf("Data missing: expected %d, actual %d", expectCount, actualCount)
// 发送告警
if m.sender != nil {
title := fmt.Sprintf("[%s] Data Missing Alert", assetType)
content := fmt.Sprintf("Symbol: %s, Date: %s, Expected: %d, Actual: %d",
symbol, checkDate, expectCount, actualCount)
m.sender.SendAlert(title, content)
}
}
return result
}
// saveCheckResult 保存检查结果
func (m *Monitor) saveCheckResult(ctx context.Context, assetType string, result CheckResult) error {
// 使用Repository保存到data_quality_checks表
query := fmt.Sprintf(`
INSERT INTO %s.data_quality_checks
(check_date, symbol_id, freq, check_type, status, expect_count, actual_count, detail)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (check_date, symbol_id, freq, check_type) DO UPDATE SET
status = EXCLUDED.status,
expect_count = EXCLUDED.expect_count,
actual_count = EXCLUDED.actual_count,
detail = EXCLUDED.detail,
created_at = NOW()
`, assetType)
_, err := m.db.ExecContext(ctx, query,
result.CheckDate, result.Symbol, result.Freq,
result.CheckType, result.Status, result.ExpectCount,
result.ActualCount, result.Detail)
return err
}
// GetQualityReport 获取数据质量报告
func (m *Monitor) GetQualityReport(ctx context.Context, assetType, checkDate string) (*QualityReport, error) {
query := fmt.Sprintf(`
SELECT
COUNT(*) as total_checks,
SUM(CASE WHEN status = 'pass' THEN 1 ELSE 0 END) as pass_count,
SUM(CASE WHEN status = 'fail' THEN 1 ELSE 0 END) as fail_count
FROM %s.data_quality_checks
WHERE check_date = $1
`, assetType)
var report QualityReport
report.CheckDate = checkDate
report.AssetType = assetType
row := m.db.QueryRowContext(ctx, query, checkDate)
err := row.Scan(&report.TotalChecks, &report.PassCount, &report.FailCount)
if err != nil {
return nil, err
}
if report.TotalChecks > 0 {
report.PassRate = float64(report.PassCount) / float64(report.TotalChecks) * 100
}
return &report, nil
}
// QualityReport 数据质量报告
type QualityReport struct {
AssetType string `json:"asset_type"`
CheckDate string `json:"check_date"`
TotalChecks int `json:"total_checks"`
PassCount int `json:"pass_count"`
FailCount int `json:"fail_count"`
PassRate float64 `json:"pass_rate"`
}
// LogAlertSender 日志告警发送器
type LogAlertSender struct{}
// SendAlert 发送告警到日志
func (s *LogAlertSender) SendAlert(title, content string) error {
log.Printf("[ALERT] %s: %s", title, content)
return nil
}
// StartDailyCheckCron 启动每日检查定时任务
func (m *Monitor) StartDailyCheckCron(ctx context.Context) {
go func() {
for {
// 计算到下一个盘后的时间（假设15:30）
now := time.Now()
next := time.Date(now.Year(), now.Month(), now.Day(), 15, 35, 0, 0, now.Location())
if next.Before(now) {
next = next.Add(24 * time.Hour)
}
timer := time.NewTimer(next.Sub(now))
<-timer.C
// 执行检查
checkDate := now.Format("20060102")
if err := m.DailyCheck(ctx, checkDate); err != nil {
log.Printf("Daily check failed: %v", err)
}
}
}()
}

334
internal/repository/futures.go
Unescape View File

@ -0,0 +1,334 @@
package repository
import (
"context"
"database/sql"
"fmt"
"strings"
"time"
"market-data-service/api"
)
// FuturesRepository 期货数据仓库
type FuturesRepository struct {
db *DB
}
// NewFuturesRepository 创建期货Repository
func NewFuturesRepository(db *DB) *FuturesRepository {
return &FuturesRepository{db: db}
}
// GetKLines 获取K线数据
func (r *FuturesRepository) GetKLines(ctx context.Context, symbol string, freq api.Frequency, start, end time.Time) ([]api.KLineItem, error) {
tableName := fmt.Sprintf("futures.klines_%s", freq)
query := fmt.Sprintf(`
SELECT ts, open, high, low, close, volume, amount, open_interest
FROM %s
WHERE symbol_id = $1 AND ts >= $2 AND ts <= $3
ORDER BY ts ASC
`, tableName)
rows, err := r.db.QueryContext(ctx, query, symbol, start, end)
if err != nil {
return nil, err
}
defer rows.Close()
var items []api.KLineItem
for rows.Next() {
var item api.KLineItem
var oi sql.NullInt64
if err := rows.Scan(
&item.Time, &item.Open, &item.High, &item.Low, &item.Close, &item.Volume, &item.Amount, &oi); err != nil {
return nil, err
}
if oi.Valid {
item.OpenInterest = &oi.Int64
}
items = append(items, item)
}
return items, rows.Err()
}
// SaveKLines 保存K线数据
func (r *FuturesRepository) SaveKLines(ctx context.Context, freq api.Frequency, symbol string, items []api.KLineItem) error {
if len(items) == 0 {
return nil
}
tableName := fmt.Sprintf("futures.klines_%s", freq)
// 使用批量插入
valueStrs := make([]string, 0, len(items))
args := make([]interface{}, 0, len(items)*8)
argIdx := 1
for _, item := range items {
valueStrs = append(valueStrs, fmt.Sprintf("($%d, $%d, $%d, $%d, $%d, $%d, $%d, $%d)",
argIdx, argIdx+1, argIdx+2, argIdx+3, argIdx+4, argIdx+5, argIdx+6, argIdx+7))
args = append(args, symbol, item.Time, item.Open, item.High, item.Low, item.Close, item.Volume, item.Amount)
if item.OpenInterest != nil {
args = append(args, *item.OpenInterest)
} else {
args = append(args, nil)
}
argIdx += 8
}
query := fmt.Sprintf(`
INSERT INTO %s (symbol_id, ts, open, high, low, close, volume, amount, open_interest)
VALUES %s
ON CONFLICT (symbol_id, ts) DO UPDATE SET
open = EXCLUDED.open,
high = EXCLUDED.high,
low = EXCLUDED.low,
close = EXCLUDED.close,
volume = EXCLUDED.volume,
amount = EXCLUDED.amount,
open_interest = EXCLUDED.open_interest
`, tableName, strings.Join(valueStrs, ","))
_, err := r.db.ExecContext(ctx, query, args...)
return err
}
// ListSymbols 查询标的列表
func (r *FuturesRepository) ListSymbols(ctx context.Context, req *api.SymbolListRequest) ([]api.Symbol, int, error) {
whereClause := "WHERE 1=1"
args := []interface{}{}
argIdx := 1
if req.Exchange != "" {
whereClause += fmt.Sprintf(" AND exchange = $%d", argIdx)
args = append(args, req.Exchange)
argIdx++
}
if req.Underlying != "" {
whereClause += fmt.Sprintf(" AND underlying = $%d", argIdx)
args = append(args, req.Underlying)
argIdx++
}
if req.Keyword != "" {
whereClause += fmt.Sprintf(" AND (symbol_id ILIKE $%d OR name ILIKE $%d)", argIdx, argIdx)
args = append(args, "%"+req.Keyword+"%")
argIdx++
}
// 查询总数
countQuery := "SELECT COUNT(*) FROM futures.symbols " + whereClause
var total int
if err := r.db.QueryRowContext(ctx, countQuery, args...).Scan(&total); err != nil {
return nil, 0, err
}
// 查询数据
query := fmt.Sprintf(`
SELECT symbol_id, symbol_type, exchange, name, underlying, contract_month, list_date, delist_date, status
FROM futures.symbols
%s
ORDER BY symbol_id
LIMIT $%d OFFSET $%d
`, whereClause, argIdx, argIdx+1)
args = append(args, req.Size, (req.Page-1)*req.Size)
rows, err := r.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var symbols []api.Symbol
for rows.Next() {
var s api.Symbol
var listDate, delistDate sql.NullTime
if err := rows.Scan(
&s.SymbolID, &s.SymbolType, &s.Exchange, &s.Name, &s.Underlying,
&s.ContractMonth, &listDate, &delistDate, &s.Status); err != nil {
return nil, 0, err
}
if listDate.Valid {
s.ListDate = &listDate.Time
}
if delistDate.Valid {
s.DelistDate = &delistDate.Time
}
symbols = append(symbols, s)
}
return symbols, total, rows.Err()
}
// GetContractsByUnderlying 根据品种获取合约
func (r *FuturesRepository) GetContractsByUnderlying(ctx context.Context, underlying string, exchange string) (*api.FuturesContractsData, error) {
query := `
SELECT symbol_id, symbol_type, exchange, name, underlying, contract_month, list_date, delist_date, status
FROM futures.symbols
WHERE underlying = $1 AND status = 'active'
`
args := []interface{}{underlying}
if exchange != "" {
query += " AND exchange = $2"
args = append(args, exchange)
}
query += " ORDER BY contract_month ASC"
rows, err := r.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var contracts []api.FuturesContractInfo
for rows.Next() {
var c api.FuturesContractInfo
var listDate, delistDate sql.NullTime
if err := rows.Scan(
&c.SymbolID, &c.SymbolType, &c.Exchange, &c.Name, &c.Underlying,
&c.ContractMonth, &listDate, &delistDate, &c.Status); err != nil {
return nil, err
}
if listDate.Valid {
c.ListDate = &listDate.Time
}
if delistDate.Valid {
c.DelistDate = &delistDate.Time
}
contracts = append(contracts, c)
}
return &api.FuturesContractsData{
Underlying: underlying,
Count: len(contracts),
Items: contracts,
}, rows.Err()
}
// GetTradingDates 获取交易日历
func (r *FuturesRepository) GetTradingDates(ctx context.Context, start, end string) (*api.TradingDatesData, error) {
query := `
SELECT trade_date
FROM futures.trading_calendar
WHERE trade_date >= $1 AND trade_date <= $2 AND is_trading_day = true
ORDER BY trade_date ASC
`
rows, err := r.db.QueryContext(ctx, query, start, end)
if err != nil {
return nil, err
}
defer rows.Close()
var dates []string
for rows.Next() {
var date string
if err := rows.Scan(&date); err != nil {
return nil, err
}
dates = append(dates, date)
}
// 计算总天数
startDate, _ := time.Parse("20060102", start)
endDate, _ := time.Parse("20060102", end)
totalDays := int(endDate.Sub(startDate).Hours()/24) + 1
return &api.TradingDatesData{
Start: start,
End: end,
TotalDays: totalDays,
TradingDays: len(dates),
TradingDates: dates,
}, rows.Err()
}
// SaveSymbols 保存标的列表
func (r *FuturesRepository) SaveSymbols(ctx context.Context, symbols []api.Symbol) error {
if len(symbols) == 0 {
return nil
}
tx, err := r.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.PrepareContext(ctx, `
INSERT INTO futures.symbols (symbol_id, symbol_type, exchange, name, underlying, contract_month, list_date, delist_date, status)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
ON CONFLICT (symbol_id) DO UPDATE SET
name = EXCLUDED.name,
underlying = EXCLUDED.underlying,
contract_month = EXCLUDED.contract_month,
list_date = EXCLUDED.list_date,
delist_date = EXCLUDED.delist_date,
status = EXCLUDED.status,
updated_at = NOW()
`)
if err != nil {
return err
}
defer stmt.Close()
for _, s := range symbols {
var listDate, delistDate interface{}
if s.ListDate != nil {
listDate = *s.ListDate
}
if s.DelistDate != nil {
delistDate = *s.DelistDate
}
_, err := stmt.ExecContext(ctx, s.SymbolID, s.SymbolType, s.Exchange, s.Name, s.Underlying,
s.ContractMonth, listDate, delistDate, s.Status)
if err != nil {
return err
}
}
return tx.Commit()
}
// SaveTradingCalendar 保存交易日历
func (r *FuturesRepository) SaveTradingCalendar(ctx context.Context, dates []api.TradeCalData) error {
if len(dates) == 0 {
return nil
}
tx, err := r.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.PrepareContext(ctx, `
INSERT INTO futures.trading_calendar (trade_date, is_trading_day, has_night_session, week_day)
VALUES ($1, $2, $3, $4)
ON CONFLICT (trade_date) DO UPDATE SET
is_trading_day = EXCLUDED.is_trading_day,
has_night_session = EXCLUDED.has_night_session,
week_day = EXCLUDED.week_day,
updated_at = NOW()
`)
if err != nil {
return err
}
defer stmt.Close()
for _, d := range dates {
_, err := stmt.ExecContext(ctx, d.Date.Format("2006-01-02"), d.IsTradingDay, d.HasNightSession, int(d.Date.Weekday())+1)
if err != nil {
return err
}
}
return tx.Commit()
}

13
internal/repository/repository.go
Unescape View File

@ -0,0 +1,13 @@
package repository
// Repository 数据访问层接口定义
// StockRepository 股票数据仓库
type StockRepository interface {
// TODO: 定义股票相关数据访问方法
}
// FuturesRepository 期货数据仓库
type FuturesRepository interface {
// TODO: 定义期货相关数据访问方法
}

291
internal/repository/stock.go
Unescape View File

@ -0,0 +1,291 @@
package repository
import (
"context"
"database/sql"
"fmt"
"strings"
"time"
"market-data-service/api"
)
// DB PostgreSQL连接
type DB struct {
*sql.DB
}
// NewDB 创建数据库连接
func NewDB(connStr string) (*DB, error) {
db, err := sql.Open("postgres", connStr)
if err != nil {
return nil, err
}
if err := db.Ping(); err != nil {
return nil, err
}
return &DB{db}, nil
}
// ============================================
// 股票Repository
// ============================================
// StockRepository 股票数据仓库
type StockRepository struct {
db *DB
}
// NewStockRepository 创建股票Repository
func NewStockRepository(db *DB) *StockRepository {
return &StockRepository{db: db}
}
// GetKLines 获取K线数据
func (r *StockRepository) GetKLines(ctx context.Context, symbol string, freq api.Frequency, start, end time.Time, adjust api.AdjustType) ([]api.KLineItem, error) {
tableName := fmt.Sprintf("stock.klines_%s", freq)
query := fmt.Sprintf(`
SELECT ts, open, high, low, close, volume, amount
FROM %s
WHERE symbol_id = $1 AND ts >= $2 AND ts <= $3
ORDER BY ts ASC
`, tableName)
rows, err := r.db.QueryContext(ctx, query, symbol, start, end)
if err != nil {
return nil, err
}
defer rows.Close()
var items []api.KLineItem
for rows.Next() {
var item api.KLineItem
if err := rows.Scan(&item.Time, &item.Open, &item.High, &item.Low, &item.Close, &item.Volume, &item.Amount); err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
// SaveKLines 保存K线数据
func (r *StockRepository) SaveKLines(ctx context.Context, freq api.Frequency, items []api.KLineItem) error {
if len(items) == 0 {
return nil
}
tableName := fmt.Sprintf("stock.klines_%s", freq)
// 使用批量插入
valueStrs := make([]string, 0, len(items))
args := make([]interface{}, 0, len(items)*7)
argIdx := 1
for _, item := range items {
valueStrs = append(valueStrs, fmt.Sprintf("($%d, $%d, $%d, $%d, $%d, $%d, $%d)",
argIdx, argIdx+1, argIdx+2, argIdx+3, argIdx+4, argIdx+5, argIdx+6))
args = append(args, item.Symbol, item.Time, item.Open, item.High, item.Low, item.Close, item.Volume, item.Amount)
argIdx += 7
}
query := fmt.Sprintf(`
INSERT INTO %s (symbol_id, ts, open, high, low, close, volume, amount)
VALUES %s
ON CONFLICT (symbol_id, ts) DO UPDATE SET
open = EXCLUDED.open,
high = EXCLUDED.high,
low = EXCLUDED.low,
close = EXCLUDED.close,
volume = EXCLUDED.volume,
amount = EXCLUDED.amount
`, tableName, strings.Join(valueStrs, ","))
_, err := r.db.ExecContext(ctx, query, args...)
return err
}
// ListSymbols 查询标的列表
func (r *StockRepository) ListSymbols(ctx context.Context, req *api.SymbolListRequest) ([]api.Symbol, int, error) {
whereClause := "WHERE 1=1"
args := []interface{}{}
argIdx := 1
if req.Exchange != "" {
whereClause += fmt.Sprintf(" AND exchange = $%d", argIdx)
args = append(args, req.Exchange)
argIdx++
}
if req.Keyword != "" {
whereClause += fmt.Sprintf(" AND (symbol_id ILIKE $%d OR name ILIKE $%d)", argIdx, argIdx)
args = append(args, "%"+req.Keyword+"%")
argIdx++
}
// 查询总数
countQuery := "SELECT COUNT(*) FROM stock.symbols " + whereClause
var total int
if err := r.db.QueryRowContext(ctx, countQuery, args...).Scan(&total); err != nil {
return nil, 0, err
}
// 查询数据
query := fmt.Sprintf(`
SELECT symbol_id, symbol_type, exchange, name, name_en, list_date, delist_date, industry, status
FROM stock.symbols
%s
ORDER BY symbol_id
LIMIT $%d OFFSET $%d
`, whereClause, argIdx, argIdx+1)
args = append(args, req.Size, (req.Page-1)*req.Size)
rows, err := r.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var symbols []api.Symbol
for rows.Next() {
var s api.Symbol
var listDate, delistDate sql.NullTime
if err := rows.Scan(&s.SymbolID, &s.SymbolType, &s.Exchange, &s.Name, &s.NameEN,
&listDate, &delistDate, &s.Industry, &s.Status); err != nil {
return nil, 0, err
}
if listDate.Valid {
s.ListDate = &listDate.Time
}
if delistDate.Valid {
s.DelistDate = &delistDate.Time
}
symbols = append(symbols, s)
}
return symbols, total, rows.Err()
}
// GetTradingDates 获取交易日历
func (r *StockRepository) GetTradingDates(ctx context.Context, start, end string) (*api.TradingDatesData, error) {
query := `
SELECT trade_date
FROM stock.trading_calendar
WHERE trade_date >= $1 AND trade_date <= $2 AND is_trading_day = true
ORDER BY trade_date ASC
`
rows, err := r.db.QueryContext(ctx, query, start, end)
if err != nil {
return nil, err
}
defer rows.Close()
var dates []string
for rows.Next() {
var date string
if err := rows.Scan(&date); err != nil {
return nil, err
}
dates = append(dates, date)
}
// 计算总天数
startDate, _ := time.Parse("20060102", start)
endDate, _ := time.Parse("20060102", end)
totalDays := int(endDate.Sub(startDate).Hours()/24) + 1
return &api.TradingDatesData{
Start: start,
End: end,
TotalDays: totalDays,
TradingDays: len(dates),
TradingDates: dates,
}, rows.Err()
}
// SaveSymbols 保存标的列表
func (r *StockRepository) SaveSymbols(ctx context.Context, symbols []api.Symbol) error {
if len(symbols) == 0 {
return nil
}
tx, err := r.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.PrepareContext(ctx, `
INSERT INTO stock.symbols (symbol_id, symbol_type, exchange, name, name_en, list_date, delist_date, industry, status)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
ON CONFLICT (symbol_id) DO UPDATE SET
name = EXCLUDED.name,
name_en = EXCLUDED.name_en,
list_date = EXCLUDED.list_date,
delist_date = EXCLUDED.delist_date,
industry = EXCLUDED.industry,
status = EXCLUDED.status,
updated_at = NOW()
`)
if err != nil {
return err
}
defer stmt.Close()
for _, s := range symbols {
var listDate, delistDate interface{}
if s.ListDate != nil {
listDate = *s.ListDate
}
if s.DelistDate != nil {
delistDate = *s.DelistDate
}
_, err := stmt.ExecContext(ctx, s.SymbolID, s.SymbolType, s.Exchange, s.Name, s.NameEN,
listDate, delistDate, s.Industry, s.Status)
if err != nil {
return err
}
}
return tx.Commit()
}
// SaveTradingCalendar 保存交易日历
func (r *StockRepository) SaveTradingCalendar(ctx context.Context, dates []api.TradeCalData) error {
if len(dates) == 0 {
return nil
}
tx, err := r.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.PrepareContext(ctx, `
INSERT INTO stock.trading_calendar (trade_date, is_trading_day, week_day)
VALUES ($1, $2, $3)
ON CONFLICT (trade_date) DO UPDATE SET
is_trading_day = EXCLUDED.is_trading_day,
week_day = EXCLUDED.week_day,
updated_at = NOW()
`)
if err != nil {
return err
}
defer stmt.Close()
for _, d := range dates {
_, err := stmt.ExecContext(ctx, d.Date.Format("2006-01-02"), d.IsTradingDay, int(d.Date.Weekday())+1)
if err != nil {
return err
}
}
return tx.Commit()
}

305
internal/service/adapter.go
Unescape View File

@ -0,0 +1,305 @@
package service
import (
"context"
"fmt"
"sync"
"time"
"market-data-service/adapter"
"market-data-service/adapter/tushare"
"market-data-service/api"
)
// AdapterService 适配器管理服务接口
type AdapterService interface {
// GetAdapterList 获取适配器列表
GetAdapterList(ctx context.Context) (*api.AdapterListData, error)
// ToggleAdapter 启用/禁用适配器
ToggleAdapter(ctx context.Context, req *api.AdapterToggleRequest) error
// UpdateAdapterConfig 更新适配器配置
UpdateAdapterConfig(ctx context.Context, req *api.AdapterConfigUpdateRequest) error
// GetActiveAdapter 获取当前激活的适配器
GetActiveAdapter(assetClass string) (adapter.DataSourceAdapter, error)
// GetAvailableAdapters 获取所有可用的适配器名称
GetAvailableAdapters() []string
// RegisterAdapter 注册适配器
RegisterAdapter(name string, factory AdapterFactory)
}
// AdapterFactory 适配器工厂函数
type AdapterFactory func() adapter.DataSourceAdapter
// AdapterServiceImpl 适配器服务实现
type AdapterServiceImpl struct {
mu sync.RWMutex
// 已注册的适配器工厂
factories map[string]AdapterFactory
// 适配器配置
configs map[string]*adapterConfig
// 当前激活的适配器实例
activeAdapters map[string]adapter.DataSourceAdapter
// 适配器元数据
metadata map[string]*adapterMetadata
}
// adapterConfig 适配器配置
type adapterConfig struct {
Enabled bool `json:"enabled"`
Config map[string]string `json:"config"`
}
// adapterMetadata 适配器元数据
type adapterMetadata struct {
Name string `json:"name"`
Type string `json:"type"`
Version string `json:"version"`
Description string `json:"description"`
UpdatedAt time.Time `json:"updated_at"`
}
// NewAdapterService 创建适配器服务
func NewAdapterService() AdapterService {
service := &AdapterServiceImpl{
factories: make(map[string]AdapterFactory),
configs: make(map[string]*adapterConfig),
activeAdapters: make(map[string]adapter.DataSourceAdapter),
metadata: make(map[string]*adapterMetadata),
}
// 注册内置适配器
service.registerBuiltinAdapters()
return service
}
// registerBuiltinAdapters 注册内置适配器
func (s *AdapterServiceImpl) registerBuiltinAdapters() {
// 注册Tushare适配器
s.RegisterAdapter("tushare", func() adapter.DataSourceAdapter {
return tushare.NewAdapter()
})
// 设置Tushare元数据
s.metadata["tushare"] = &adapterMetadata{
Name: "tushare",
Type: "http",
Version: "1.0.0",
Description: "Tushare Pro 金融数据接口",
UpdatedAt: time.Now(),
}
// 默认配置
s.configs["tushare"] = &adapterConfig{
Enabled: true,
Config: map[string]string{
"token": "",
"base_url": "https://api.tushare.pro",
},
}
// 预留Wind适配器
s.metadata["wind"] = &adapterMetadata{
Name: "wind",
Type: "ws",
Version: "1.0.0",
Description: "Wind 金融终端接口（预留）",
UpdatedAt: time.Now(),
}
s.configs["wind"] = &adapterConfig{
Enabled: false,
Config: map[string]string{
"host": "localhost",
"port": "8081",
},
}
}
// GetAdapterList 获取适配器列表
func (s *AdapterServiceImpl) GetAdapterList(ctx context.Context) (*api.AdapterListData, error) {
s.mu.RLock()
defer s.mu.RUnlock()
adapters := make([]api.AdapterInfo, 0, len(s.metadata))
for name, meta := range s.metadata {
cfg, ok := s.configs[name]
if !ok {
cfg = &adapterConfig{Enabled: false, Config: make(map[string]string)}
}
status := api.AdapterStatusDisabled
if cfg.Enabled {
status = api.AdapterStatusStandby
// 检查是否是激活状态
if _, active := s.activeAdapters[name]; active {
status = api.AdapterStatusActive
}
}
adapters = append(adapters, api.AdapterInfo{
Name: meta.Name,
Type: meta.Type,
Version: meta.Version,
Description: meta.Description,
Status: status,
Config: cfg.Config,
UpdatedAt: meta.UpdatedAt,
})
}
return &api.AdapterListData{
Adapters: adapters,
}, nil
}
// ToggleAdapter 启用/禁用适配器
func (s *AdapterServiceImpl) ToggleAdapter(ctx context.Context, req *api.AdapterToggleRequest) error {
s.mu.Lock()
defer s.mu.Unlock()
cfg, ok := s.configs[req.Name]
if !ok {
return fmt.Errorf("adapter not found: %s", req.Name)
}
cfg.Enabled = req.Enable
// 如果禁用，关闭适配器连接
if !req.Enable {
if adapter, ok := s.activeAdapters[req.Name]; ok {
adapter.Close()
delete(s.activeAdapters, req.Name)
}
}
// 更新元数据
if meta, ok := s.metadata[req.Name]; ok {
meta.UpdatedAt = time.Now()
}
return nil
}
// UpdateAdapterConfig 更新适配器配置
func (s *AdapterServiceImpl) UpdateAdapterConfig(ctx context.Context, req *api.AdapterConfigUpdateRequest) error {
s.mu.Lock()
defer s.mu.Unlock()
cfg, ok := s.configs[req.Name]
if !ok {
return fmt.Errorf("adapter not found: %s", req.Name)
}
// 更新配置
for k, v := range req.Config {
cfg.Config[k] = v
}
// 如果适配器已激活，重新连接
if adapter, ok := s.activeAdapters[req.Name]; ok {
adapter.Close()
delete(s.activeAdapters, req.Name)
// 如果启用状态，重新连接
if cfg.Enabled {
if err := s.connectAdapter(req.Name); err != nil {
return fmt.Errorf("failed to reconnect adapter: %w", err)
}
}
}
// 更新元数据
if meta, ok := s.metadata[req.Name]; ok {
meta.UpdatedAt = time.Now()
}
return nil
}
// GetActiveAdapter 获取当前激活的适配器
func (s *AdapterServiceImpl) GetActiveAdapter(assetClass string) (adapter.DataSourceAdapter, error) {
s.mu.RLock()
defer s.mu.RUnlock()
// 根据资产类别获取配置
// 这里简化处理，实际应该从配置服务获取
adapterName := "tushare"
if assetClass == "futures" {
adapterName = "tushare"
}
// 检查是否已有激活的实例
if adapter, ok := s.activeAdapters[adapterName]; ok {
return adapter, nil
}
return nil, fmt.Errorf("no active adapter for %s", assetClass)
}
// GetAvailableAdapters 获取所有可用的适配器名称
func (s *AdapterServiceImpl) GetAvailableAdapters() []string {
s.mu.RLock()
defer s.mu.RUnlock()
names := make([]string, 0, len(s.metadata))
for name, meta := range s.metadata {
// 只返回有工厂的适配器（已实现的）
if _, ok := s.factories[name]; ok {
names = append(names, fmt.Sprintf("%s|%s", name, meta.Description))
}
}
return names
}
// RegisterAdapter 注册适配器
func (s *AdapterServiceImpl) RegisterAdapter(name string, factory AdapterFactory) {
s.mu.Lock()
defer s.mu.Unlock()
s.factories[name] = factory
}
// connectAdapter 连接适配器
func (s *AdapterServiceImpl) connectAdapter(name string) error {
factory, ok := s.factories[name]
if !ok {
return fmt.Errorf("adapter factory not found: %s", name)
}
cfg, ok := s.configs[name]
if !ok {
return fmt.Errorf("adapter config not found: %s", name)
}
adapter := factory()
if err := adapter.Connect(cfg.Config); err != nil {
return err
}
s.activeAdapters[name] = adapter
return nil
}
// HealthCheck 适配器健康检查
func (s *AdapterServiceImpl) HealthCheck(name string) error {
s.mu.RLock()
defer s.mu.RUnlock()
adapter, ok := s.activeAdapters[name]
if !ok {
return fmt.Errorf("adapter not active: %s", name)
}
return adapter.HealthCheck()
}

120
internal/service/admin.go
Unescape View File

@ -0,0 +1,120 @@
package service
import (
"context"
"fmt"
"time"
"github.com/google/uuid"
"market-data-service/api"
"market-data-service/internal/repository"
)
// AdminServiceImpl 管理服务实现
type AdminServiceImpl struct {
db *repository.DB
}
// NewAdminService 创建管理服务
func NewAdminService(db *repository.DB) AdminService {
return &AdminServiceImpl{
db: db,
}
}
// GetDataSourceStatus 获取数据源状态
func (s *AdminServiceImpl) GetDataSourceStatus(ctx context.Context) (*api.DataSourceStatusData, error) {
// 查询数据库中的数据源配置
query := `
SELECT asset_class, active_source, standby_sources, updated_at
FROM public.data_source_config
`
rows, err := s.db.QueryContext(ctx, query)
if err != nil {
return nil, err
}
defer rows.Close()
data := &api.DataSourceStatusData{}
for rows.Next() {
var assetClass, activeSource string
var standbySources []string
var updatedAt time.Time
if err := rows.Scan(&assetClass, &activeSource, &standbySources, &updatedAt); err != nil {
return nil, err
}
info := api.DataSourceInfo{
ActiveSource: activeSource,
StandbySources: standbySources,
Status: api.DataSourceHealthy,
}
switch assetClass {
case "stock":
data.Stock = info
case "futures":
data.Futures = info
}
}
return data, rows.Err()
}
// SwitchDataSource 切换数据源
func (s *AdminServiceImpl) SwitchDataSource(ctx context.Context, req *api.SourceSwitchRequest) error {
// 更新数据源配置
query := `
UPDATE public.data_source_config
SET active_source = $1, updated_at = NOW()
WHERE asset_class = $2
`
assetClasses := []string{}
if req.AssetClass == api.AssetAll {
assetClasses = []string{"stock", "futures"}
} else {
assetClasses = []string{string(req.AssetClass)}
}
for _, ac := range assetClasses {
_, err := s.db.ExecContext(ctx, query, req.Source, ac)
if err != nil {
return fmt.Errorf("failed to switch %s data source: %w", ac, err)
}
}
// 如果需要同步补录，启动后台任务
if req.SyncBackfill {
// TODO: 启动异步补录任务
go func() {
// 异步执行补录
}()
}
return nil
}
// BackfillData 历史数据补录
func (s *AdminServiceImpl) BackfillData(ctx context.Context, req *api.BackfillRequest) (string, error) {
taskID := uuid.New().String()
// TODO: 将补录任务存入数据库，启动后台Worker执行
// 这里简化处理，直接返回任务ID
return taskID, nil
}
// HealthCheck 健康检查
func (s *AdminServiceImpl) HealthCheck(ctx context.Context) (*api.HealthResponse, error) {
// 检查数据库连接
if err := s.db.PingContext(ctx); err != nil {
return nil, fmt.Errorf("database connection failed: %w", err)
}
return &api.HealthResponse{
Status: "healthy",
Timestamp: time.Now(),
}, nil
}

477
internal/service/config.go
Unescape View File

@ -0,0 +1,477 @@
package service
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"runtime"
"sync"
"time"
"market-data-service/api"
"market-data-service/pkg/config"
)
// ConfigService 配置管理服务接口
type ConfigService interface {
// GetConfigList 获取配置列表
GetConfigList(ctx context.Context, req *api.ConfigListRequest) (*api.ConfigListData, error)
// UpdateConfig 更新配置
UpdateConfig(ctx context.Context, req *api.ConfigUpdateRequest) (*api.ConfigUpdateData, error)
// ReloadConfig 热加载配置
ReloadConfig(ctx context.Context, req *api.ReloadRequest) (*api.ReloadData, error)
// GetSystemStatus 获取系统状态
GetSystemStatus(ctx context.Context) (*api.SystemStatusData, error)
// GetCurrentConfig 获取当前配置（内部使用）
GetCurrentConfig() *config.Config
}
// ConfigServiceImpl 配置服务实现
type ConfigServiceImpl struct {
configPath string
config *config.Config
mu sync.RWMutex
// 配置变更回调
callbacks map[api.ConfigType][]func()
cbMu sync.RWMutex
// 启动时间
startTime time.Time
// 配置版本
version string
}
// NewConfigService 创建配置服务
func NewConfigService(configPath string) (ConfigService, error) {
cfg, err := config.Load(configPath)
if err != nil {
// 如果加载失败，使用默认配置
cfg = getDefaultConfig()
}
return &ConfigServiceImpl{
configPath: configPath,
config: cfg,
callbacks: make(map[api.ConfigType][]func()),
startTime: time.Now(),
version: "1.0.0",
}, nil
}
// GetConfigList 获取配置列表
func (s *ConfigServiceImpl) GetConfigList(ctx context.Context, req *api.ConfigListRequest) (*api.ConfigListData, error) {
s.mu.RLock()
defer s.mu.RUnlock()
sections := []api.ConfigSection{}
// 服务器配置
if req.Type == "" || req.Type == api.ConfigTypeServer {
sections = append(sections, api.ConfigSection{
Name: "服务器配置",
Type: api.ConfigTypeServer,
Description: "HTTP服务器相关配置",
Items: []api.ConfigItem{
{
Key: "port",
Value: s.config.Server.Port,
Type: "int",
Description: "服务端口",
Editable: true,
Required: true,
},
{
Key: "mode",
Value: s.config.Server.Mode,
Type: "string",
Description: "运行模式: debug/release",
Editable: true,
Required: true,
},
{
Key: "api_key",
Value: s.config.Server.APIKey,
Type: "string",
Description: "API认证密钥",
Editable: true,
Required: true,
},
},
})
}
// 数据库配置
if req.Type == "" || req.Type == api.ConfigTypeDatabase {
sections = append(sections, api.ConfigSection{
Name: "数据库配置",
Type: api.ConfigTypeDatabase,
Description: "PostgreSQL数据库连接配置",
Items: []api.ConfigItem{
{
Key: "host",
Value: s.config.Database.Host,
Type: "string",
Description: "数据库主机地址",
Editable: true,
Required: true,
},
{
Key: "port",
Value: s.config.Database.Port,
Type: "int",
Description: "数据库端口",
Editable: true,
Required: true,
},
{
Key: "user",
Value: s.config.Database.User,
Type: "string",
Description: "数据库用户名",
Editable: true,
Required: true,
},
{
Key: "password",
Value: "********",
Type: "password",
Description: "数据库密码",
Editable: true,
Required: true,
},
{
Key: "database",
Value: s.config.Database.Database,
Type: "string",
Description: "数据库名",
Editable: true,
Required: true,
},
},
})
}
// Redis配置
if req.Type == "" || req.Type == api.ConfigTypeRedis {
sections = append(sections, api.ConfigSection{
Name: "Redis配置",
Type: api.ConfigTypeRedis,
Description: "Redis缓存配置",
Items: []api.ConfigItem{
{
Key: "host",
Value: s.config.Redis.Host,
Type: "string",
Description: "Redis主机地址",
Editable: true,
Required: false,
},
{
Key: "port",
Value: s.config.Redis.Port,
Type: "int",
Description: "Redis端口",
Editable: true,
Required: false,
},
{
Key: "password",
Value: "********",
Type: "password",
Description: "Redis密码",
Editable: true,
Required: false,
},
{
Key: "db",
Value: s.config.Redis.DB,
Type: "int",
Description: "Redis数据库编号",
Editable: true,
Required: false,
},
},
})
}
// 数据源配置
if req.Type == "" || req.Type == api.ConfigTypeSource {
sections = append(sections, api.ConfigSection{
Name: "数据源配置",
Type: api.ConfigTypeSource,
Description: "股票和期货数据源配置",
Items: []api.ConfigItem{
{
Key: "stock_active",
Value: s.config.Sources.Stock.Active,
Type: "string",
Description: "股票数据源适配器",
Editable: true,
Required: true,
},
{
Key: "futures_active",
Value: s.config.Sources.Futures.Active,
Type: "string",
Description: "期货数据源适配器",
Editable: true,
Required: true,
},
},
})
}
return &api.ConfigListData{
Sections: sections,
Version: s.version,
Updated: time.Now(),
}, nil
}
// UpdateConfig 更新配置
func (s *ConfigServiceImpl) UpdateConfig(ctx context.Context, req *api.ConfigUpdateRequest) (*api.ConfigUpdateData, error) {
s.mu.Lock()
defer s.mu.Unlock()
needRestart := false
switch req.Type {
case api.ConfigTypeServer:
if port, ok := req.Items["port"]; ok {
s.config.Server.Port = int(port.(float64))
needRestart = true
}
if mode, ok := req.Items["mode"]; ok {
s.config.Server.Mode = mode.(string)
}
if apiKey, ok := req.Items["api_key"]; ok {
s.config.Server.APIKey = apiKey.(string)
}
case api.ConfigTypeDatabase:
if host, ok := req.Items["host"]; ok {
s.config.Database.Host = host.(string)
needRestart = true
}
if port, ok := req.Items["port"]; ok {
s.config.Database.Port = int(port.(float64))
needRestart = true
}
if user, ok := req.Items["user"]; ok {
s.config.Database.User = user.(string)
needRestart = true
}
if password, ok := req.Items["password"]; ok && password.(string) != "********" {
s.config.Database.Password = password.(string)
needRestart = true
}
if database, ok := req.Items["database"]; ok {
s.config.Database.Database = database.(string)
needRestart = true
}
case api.ConfigTypeSource:
if stockActive, ok := req.Items["stock_active"]; ok {
s.config.Sources.Stock.Active = stockActive.(string)
}
if futuresActive, ok := req.Items["futures_active"]; ok {
s.config.Sources.Futures.Active = futuresActive.(string)
}
}
// 保存到文件
if err := s.saveConfig(); err != nil {
return &api.ConfigUpdateData{
Success: false,
Message: fmt.Sprintf("配置保存失败: %v", err),
}, nil
}
// 触发回调
s.triggerCallbacks(req.Type)
message := "配置更新成功"
if needRestart {
message += "，部分配置需要重启服务后生效"
}
return &api.ConfigUpdateData{
Success: true,
NeedRestart: needRestart,
Message: message,
}, nil
}
// ReloadConfig 热加载配置
func (s *ConfigServiceImpl) ReloadConfig(ctx context.Context, req *api.ReloadRequest) (*api.ReloadData, error) {
s.mu.Lock()
defer s.mu.Unlock()
// 从文件重新加载
cfg, err := config.Load(s.configPath)
if err != nil {
return &api.ReloadData{
Success: false,
Message: fmt.Sprintf("加载配置失败: %v", err),
}, nil
}
// 根据类型选择性更新
if req.ConfigType == "" {
s.config = cfg
} else {
switch req.ConfigType {
case api.ConfigTypeServer:
s.config.Server = cfg.Server
case api.ConfigTypeDatabase:
s.config.Database = cfg.Database
case api.ConfigTypeRedis:
s.config.Redis = cfg.Redis
case api.ConfigTypeSource:
s.config.Sources = cfg.Sources
}
}
// 触发回调
s.triggerCallbacks(req.ConfigType)
return &api.ReloadData{
Success: true,
Message: "配置热加载成功",
}, nil
}
// GetSystemStatus 获取系统状态
func (s *ConfigServiceImpl) GetSystemStatus(ctx context.Context) (*api.SystemStatusData, error) {
var m runtime.MemStats
runtime.ReadMemStats(&m)
uptime := time.Since(s.startTime)
uptimeStr := formatDuration(uptime)
return &api.SystemStatusData{
Status: "running",
Version: s.version,
StartTime: s.startTime,
Uptime: uptimeStr,
GoVersion: runtime.Version(),
MemoryUsage: api.MemoryInfo{
Alloc: m.Alloc,
TotalAlloc: m.TotalAlloc,
Sys: m.Sys,
NumGC: m.NumGC,
},
Goroutines: runtime.NumGoroutine(),
}, nil
}
// GetCurrentConfig 获取当前配置
func (s *ConfigServiceImpl) GetCurrentConfig() *config.Config {
s.mu.RLock()
defer s.mu.RUnlock()
return s.config
}
// RegisterCallback 注册配置变更回调
func (s *ConfigServiceImpl) RegisterCallback(configType api.ConfigType, callback func()) {
s.cbMu.Lock()
defer s.cbMu.Unlock()
s.callbacks[configType] = append(s.callbacks[configType], callback)
}
// triggerCallbacks 触发回调
func (s *ConfigServiceImpl) triggerCallbacks(configType api.ConfigType) {
s.cbMu.RLock()
defer s.cbMu.RUnlock()
// 触发特定类型的回调
if cbs, ok := s.callbacks[configType]; ok {
for _, cb := range cbs {
go cb()
}
}
// 触发通用回调
if cbs, ok := s.callbacks[""]; ok {
for _, cb := range cbs {
go cb()
}
}
}
// saveConfig 保存配置到文件
func (s *ConfigServiceImpl) saveConfig() error {
if s.configPath == "" {
return nil
}
// 确保目录存在
dir := filepath.Dir(s.configPath)
if err := os.MkdirAll(dir, 0755); err != nil {
return err
}
// 序列化为JSON
data, err := json.MarshalIndent(s.config, "", " ")
if err != nil {
return err
}
return os.WriteFile(s.configPath, data, 0644)
}
// getDefaultConfig 获取默认配置
func getDefaultConfig() *config.Config {
return &config.Config{
Server: config.ServerConfig{
Port: 8080,
Mode: "debug",
APIKey: "default-api-key",
},
Database: config.DatabaseConfig{
Host: "localhost",
Port: 5432,
User: "user",
Password: "password",
Database: "marketdata",
},
Redis: config.RedisConfig{
Host: "localhost",
Port: 6379,
Password: "",
DB: 0,
},
Sources: config.SourcesConfig{
Stock: config.SourceConfig{
Active: "tushare",
},
Futures: config.SourceConfig{
Active: "tushare",
},
},
}
}
// formatDuration 格式化持续时间
func formatDuration(d time.Duration) string {
days := int(d.Hours()) / 24
hours := int(d.Hours()) % 24
minutes := int(d.Minutes()) % 60
if days > 0 {
return fmt.Sprintf("%d天%d小时%d分钟", days, hours, minutes)
}
if hours > 0 {
return fmt.Sprintf("%d小时%d分钟", hours, minutes)
}
return fmt.Sprintf("%d分钟", minutes)
}

105
internal/service/futures.go
Unescape View File

@ -0,0 +1,105 @@
package service
import (
"context"
"fmt"
"time"
"market-data-service/api"
"market-data-service/internal/repository"
)
// FuturesServiceImpl 期货服务实现
type FuturesServiceImpl struct {
repo *repository.FuturesRepository
}
// NewFuturesService 创建期货服务
func NewFuturesService(repo *repository.FuturesRepository) FuturesService {
return &FuturesServiceImpl{
repo: repo,
}
}
// QueryKLines 查询K线数据
func (s *FuturesServiceImpl) QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.KLineData, error) {
// 解析日期
start, err := time.Parse("20060102", req.Start)
if err != nil {
return nil, fmt.Errorf("invalid start date: %w", err)
}
end, err := time.Parse("20060102", req.End)
if err != nil {
return nil, fmt.Errorf("invalid end date: %w", err)
}
end = end.Add(24 * time.Hour).Add(-time.Second)
// 获取K线数据
items, err := s.repo.GetKLines(ctx, req.Symbol, req.Freq, start, end)
if err != nil {
return nil, err
}
return &api.KLineData{
Symbol: req.Symbol,
Freq: req.Freq,
Count: len(items),
Items: items,
}, nil
}
// ListSymbols 查询标的列表
func (s *FuturesServiceImpl) ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.SymbolListData, error) {
symbols, total, err := s.repo.ListSymbols(ctx, req)
if err != nil {
return nil, err
}
return &api.SymbolListData{
Total: total,
Page: req.Page,
Size: req.Size,
Items: symbols,
}, nil
}
// BatchQueryKLines 批量查询K线
func (s *FuturesServiceImpl) BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.BatchKLineData, error) {
results := make([]api.BatchKLineResult, len(req.Symbols))
for i, symbol := range req.Symbols {
singleReq := &api.KLineQueryRequest{
Symbol: symbol,
Start: req.Start,
End: req.End,
Freq: req.Freq,
}
data, err := s.QueryKLines(ctx, singleReq)
results[i] = api.BatchKLineResult{
Symbol: symbol,
Success: err == nil,
}
if err != nil {
results[i].Error = err.Error()
} else {
results[i].Data = &api.KLineSubData{
Count: data.Count,
Items: data.Items,
}
}
}
return &api.BatchKLineData{Results: results}, nil
}
// GetTradingDates 获取交易日历
func (s *FuturesServiceImpl) GetTradingDates(ctx context.Context, req *api.TradingDatesRequest) (*api.TradingDatesData, error) {
return s.repo.GetTradingDates(ctx, req.Start, req.End)
}
// GetContractsByUnderlying 根据品种获取合约
func (s *FuturesServiceImpl) GetContractsByUnderlying(ctx context.Context, req *api.FuturesContractsRequest) (*api.FuturesContractsData, error) {
return s.repo.GetContractsByUnderlying(ctx, req.Underlying, req.Exchange)
}

56
internal/service/service.go
Unescape View File

@ -0,0 +1,56 @@
// Package service 业务逻辑层接口定义
package service
import (
"context"
"market-data-service/api"
)
// StockService 股票业务服务
type StockService interface {
// QueryKLines 查询K线数据
QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.KLineData, error)
// ListSymbols 查询标的列表
ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.SymbolListData, error)
// BatchQueryKLines 批量查询K线
BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.BatchKLineData, error)
// GetTradingDates 获取交易日历
GetTradingDates(ctx context.Context, req *api.TradingDatesRequest) (*api.TradingDatesData, error)
}
// FuturesService 期货业务服务
type FuturesService interface {
// QueryKLines 查询K线数据
QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.KLineData, error)
// ListSymbols 查询标的列表
ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.SymbolListData, error)
// BatchQueryKLines 批量查询K线
BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.BatchKLineData, error)
// GetTradingDates 获取交易日历
GetTradingDates(ctx context.Context, req *api.TradingDatesRequest) (*api.TradingDatesData, error)
// GetContractsByUnderlying 根据品种获取可交易合约
GetContractsByUnderlying(ctx context.Context, req *api.FuturesContractsRequest) (*api.FuturesContractsData, error)
}
// AdminService 管理服务
type AdminService interface {
// GetDataSourceStatus 获取数据源状态
GetDataSourceStatus(ctx context.Context) (*api.DataSourceStatusData, error)
// SwitchDataSource 切换数据源
SwitchDataSource(ctx context.Context, req *api.SourceSwitchRequest) error
// BackfillData 历史数据补录，返回任务ID
BackfillData(ctx context.Context, req *api.BackfillRequest) (string, error)
// HealthCheck 健康检查
HealthCheck(ctx context.Context) (*api.HealthResponse, error)
}

132
internal/service/stock.go
Unescape View File

@ -0,0 +1,132 @@
package service
import (
"context"
"fmt"
"strings"
"time"
"market-data-service/api"
"market-data-service/internal/repository"
)
// StockServiceImpl 股票服务实现
type StockServiceImpl struct {
repo *repository.StockRepository
config *DataSourceConfig
}
// DataSourceConfig 数据源配置
type DataSourceConfig struct {
Adapter interface{}
}
// NewStockService 创建股票服务
func NewStockService(repo *repository.StockRepository) StockService {
return &StockServiceImpl{
repo: repo,
}
}
// QueryKLines 查询K线数据
func (s *StockServiceImpl) QueryKLines(ctx context.Context, req *api.KLineQueryRequest) (*api.KLineData, error) {
// 解析日期
start, err := time.Parse("20060102", req.Start)
if err != nil {
return nil, fmt.Errorf("invalid start date: %w", err)
}
end, err := time.Parse("20060102", req.End)
if err != nil {
return nil, fmt.Errorf("invalid end date: %w", err)
}
end = end.Add(24 * time.Hour).Add(-time.Second) // 包含结束日期全天
// 获取K线数据
items, err := s.repo.GetKLines(ctx, req.Symbol, req.Freq, start, end, req.Adjust)
if err != nil {
return nil, err
}
// 处理复权
if req.Adjust != api.AdjustNone {
items = s.applyAdjust(ctx, req.Symbol, items, req.Adjust)
}
return &api.KLineData{
Symbol: req.Symbol,
Freq: req.Freq,
Adjust: req.Adjust,
Count: len(items),
Items: items,
}, nil
}
// applyAdjust 应用复权
func (s *StockServiceImpl) applyAdjust(ctx context.Context, symbol string, items []api.KLineItem, adjustType api.AdjustType) []api.KLineItem {
// TODO: 实现复权计算
// 1. 从数据库获取复权系数
// 2. 根据前复权/后复权计算价格
return items
}
// ListSymbols 查询标的列表
func (s *StockServiceImpl) ListSymbols(ctx context.Context, req *api.SymbolListRequest) (*api.SymbolListData, error) {
symbols, total, err := s.repo.ListSymbols(ctx, req)
if err != nil {
return nil, err
}
return &api.SymbolListData{
Total: total,
Page: req.Page,
Size: req.Size,
Items: symbols,
}, nil
}
// BatchQueryKLines 批量查询K线
func (s *StockServiceImpl) BatchQueryKLines(ctx context.Context, req *api.BatchKLineRequest) (*api.BatchKLineData, error) {
results := make([]api.BatchKLineResult, len(req.Symbols))
for i, symbol := range req.Symbols {
singleReq := &api.KLineQueryRequest{
Symbol: symbol,
Start: req.Start,
End: req.End,
Freq: req.Freq,
Adjust: req.Adjust,
}
data, err := s.QueryKLines(ctx, singleReq)
results[i] = api.BatchKLineResult{
Symbol: symbol,
Success: err == nil,
}
if err != nil {
results[i].Error = err.Error()
} else {
results[i].Data = &api.KLineSubData{
Count: data.Count,
Items: data.Items,
}
}
}
return &api.BatchKLineData{Results: results}, nil
}
// GetTradingDates 获取交易日历
func (s *StockServiceImpl) GetTradingDates(ctx context.Context, req *api.TradingDatesRequest) (*api.TradingDatesData, error) {
return s.repo.GetTradingDates(ctx, req.Start, req.End)
}
// SyncSymbolsFromSource 从数据源同步标的列表
func (s *StockServiceImpl) SyncSymbolsFromSource(ctx context.Context, adapter interface{ FetchSymbols(assetType string) ([]struct {
SymbolID string
Name string
Exchange string
}, error) }) error {
// TODO: 实现从Tushare同步标的列表
return nil
}

519
internal/service/test.go
Unescape View File

@ -0,0 +1,519 @@
package service
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
"market-data-service/api"
)
// TestService 测试服务接口
type TestService interface {
// GetAPITestList 获取API测试列表
GetAPITestList(ctx context.Context) (*api.APITestListData, error)
// RunAPITest 执行API测试
RunAPITest(ctx context.Context, baseURL string, req *api.APITestRequest) (*api.APITestResult, error)
// GetWSTestList 获取WebSocket测试列表
GetWSTestList(ctx context.Context) (*api.WSTestListData, error)
// RunWSTest 执行WebSocket测试
RunWSTest(ctx context.Context, wsURL string, req *api.WSTestRequest) (*api.WSTestResult, error)
// GetTestHistory 获取测试历史
GetTestHistory(ctx context.Context, req *api.TestHistoryRequest) (*api.TestHistoryData, error)
}
// TestServiceImpl 测试服务实现
type TestServiceImpl struct {
mu sync.RWMutex
apiHistory []api.APITestResult
wsHistory []api.WSTestResult
historySize int
}
// NewTestService 创建测试服务
func NewTestService() TestService {
return &TestServiceImpl{
apiHistory: make([]api.APITestResult, 0),
wsHistory: make([]api.WSTestResult, 0),
historySize: 100, // 保留最近100条记录
}
}
// GetAPITestList 获取API测试列表
func (s *TestServiceImpl) GetAPITestList(ctx context.Context) (*api.APITestListData, error) {
categories := []api.APITestCategory{
{
Name: "股票接口",
Items: []api.APITestCase{
{
ID: "stock_klines",
Name: "查询股票K线",
Method: "GET",
Path: "/v1/stock/klines/{symbol}",
Description: "查询指定股票的K线数据",
Params: map[string]string{
"symbol": "000001.SZ",
"start": time.Now().AddDate(0, 0, -30).Format("20060102"),
"end": time.Now().Format("20060102"),
"freq": "1d",
"adjust": "qfq",
},
},
{
ID: "stock_symbols",
Name: "查询股票列表",
Method: "GET",
Path: "/v1/stock/symbols",
Description: "获取所有可用股票标的",
Params: map[string]string{
"page": "1",
"size": "20",
},
},
{
ID: "stock_batch",
Name: "批量查询股票K线",
Method: "POST",
Path: "/v1/stock/klines/batch",
Description: "批量查询多只股票K线",
Body: map[string]interface{}{
"symbols": []string{"000001.SZ", "000002.SZ"},
"start": time.Now().AddDate(0, 0, -7).Format("20060102"),
"end": time.Now().Format("20060102"),
"freq": "1d",
},
},
{
ID: "stock_calendar",
Name: "查询交易日历",
Method: "GET",
Path: "/v1/stock/trading-dates",
Description: "查询股票交易日历",
Params: map[string]string{
"start": time.Now().AddDate(0, 0, -30).Format("20060102"),
"end": time.Now().AddDate(0, 0, 30).Format("20060102"),
},
},
},
},
{
Name: "期货接口",
Items: []api.APITestCase{
{
ID: "futures_klines",
Name: "查询期货K线",
Method: "GET",
Path: "/v1/futures/klines/{symbol}",
Description: "查询指定期货合约的K线数据",
Params: map[string]string{
"symbol": "CU2504.SHFE",
"start": time.Now().AddDate(0, 0, -30).Format("20060102"),
"end": time.Now().Format("20060102"),
"freq": "1d",
},
},
{
ID: "futures_symbols",
Name: "查询期货列表",
Method: "GET",
Path: "/v1/futures/symbols",
Description: "获取所有可用期货标的",
Params: map[string]string{
"page": "1",
"size": "20",
},
},
{
ID: "futures_batch",
Name: "批量查询期货K线",
Method: "POST",
Path: "/v1/futures/klines/batch",
Description: "批量查询多个期货合约K线",
Body: map[string]interface{}{
"symbols": []string{"CU2504.SHFE", "RB2505.SHFE"},
"start": time.Now().AddDate(0, 0, -7).Format("20060102"),
"end": time.Now().Format("20060102"),
"freq": "1d",
},
},
{
ID: "futures_contracts",
Name: "查询合约列表",
Method: "GET",
Path: "/v1/futures/contracts",
Description: "根据品种查询可交易合约",
Params: map[string]string{
"underlying": "CU",
"exchange": "SHFE",
},
},
{
ID: "futures_calendar",
Name: "查询期货交易日历",
Method: "GET",
Path: "/v1/futures/trading-dates",
Description: "查询期货交易日历",
Params: map[string]string{
"start": time.Now().AddDate(0, 0, -30).Format("20060102"),
"end": time.Now().AddDate(0, 0, 30).Format("20060102"),
},
},
},
},
{
Name: "管理接口",
Items: []api.APITestCase{
{
ID: "admin_health",
Name: "健康检查",
Method: "GET",
Path: "/v1/admin/health",
Description: "检查服务健康状态",
Params: map[string]string{},
},
{
ID: "admin_source_status",
Name: "数据源状态",
Method: "GET",
Path: "/v1/admin/source/status",
Description: "获取当前数据源状态",
Params: map[string]string{},
},
},
},
}
return &api.APITestListData{
Categories: categories,
BaseURL: "",
}, nil
}
// RunAPITest 执行API测试
func (s *TestServiceImpl) RunAPITest(ctx context.Context, baseURL string, req *api.APITestRequest) (*api.APITestResult, error) {
// 获取测试用例
testList, _ := s.GetAPITestList(ctx)
var testCase *api.APITestCase
for _, cat := range testList.Categories {
for _, item := range cat.Items {
if item.ID == req.ID {
testCase = &item
break
}
}
}
if testCase == nil {
return nil, fmt.Errorf("test case not found: %s", req.ID)
}
// 合并参数
params := make(map[string]string)
for k, v := range testCase.Params {
params[k] = v
}
for k, v := range req.Params {
params[k] = v
}
// 构建URL
url := baseURL + testCase.Path
for k, v := range params {
url = strings.Replace(url, "{"+k+"}", v, -1)
}
// 添加查询参数
if testCase.Method == "GET" && len(params) > 0 {
queryParts := []string{}
for k, v := range params {
if !strings.Contains(testCase.Path, "{"+k+"}") {
queryParts = append(queryParts, fmt.Sprintf("%s=%s", k, v))
}
}
if len(queryParts) > 0 {
url += "?" + strings.Join(queryParts, "&")
}
}
// 准备请求体
var body interface{}
if req.Body != nil {
body = req.Body
} else {
body = testCase.Body
}
// 创建HTTP客户端
client := &http.Client{
Timeout: 30 * time.Second,
}
// 构建请求
var httpReq *http.Request
var err error
if body != nil && testCase.Method != "GET" {
jsonBody, _ := json.Marshal(body)
httpReq, err = http.NewRequestWithContext(ctx, testCase.Method, url, bytes.NewBuffer(jsonBody))
httpReq.Header.Set("Content-Type", "application/json")
} else {
httpReq, err = http.NewRequestWithContext(ctx, testCase.Method, url, nil)
}
if err != nil {
return nil, err
}
httpReq.Header.Set("X-API-Key", "test-api-key")
// 执行请求
startTime := time.Now()
resp, err := client.Do(httpReq)
latency := time.Since(startTime).Milliseconds()
result := &api.APITestResult{
ID: int(time.Now().Unix()),
CaseID: req.ID,
Name: testCase.Name,
Latency: latency,
Timestamp: time.Now(),
Request: map[string]interface{}{
"method": testCase.Method,
"url": url,
"body": body,
},
}
if err != nil {
result.Success = false
result.Error = err.Error()
s.addAPIHistory(result)
return result, nil
}
defer resp.Body.Close()
result.StatusCode = resp.StatusCode
result.Success = resp.StatusCode >= 200 && resp.StatusCode < 300
// 解析响应
var respBody interface{}
if err := json.NewDecoder(resp.Body).Decode(&respBody); err == nil {
result.Response = respBody
} else {
result.Response = map[string]string{"raw": "非JSON响应"}
}
s.addAPIHistory(result)
return result, nil
}
// GetWSTestList 获取WebSocket测试列表
func (s *TestServiceImpl) GetWSTestList(ctx context.Context) (*api.WSTestListData, error) {
cases := []api.WSTestCase{
{
ID: "ws_subscribe_stock",
Name: "订阅股票行情",
Description: "订阅单只股票实时行情",
Action: "subscribe",
Symbols: []string{"000001.SZ"},
},
{
ID: "ws_subscribe_futures",
Name: "订阅期货行情",
Description: "订阅单个期货合约实时行情",
Action: "subscribe",
Symbols: []string{"CU2504.SHFE"},
},
{
ID: "ws_subscribe_multi",
Name: "批量订阅",
Description: "同时订阅多个标的",
Action: "subscribe",
Symbols: []string{"000001.SZ", "000002.SZ", "CU2504.SHFE"},
},
{
ID: "ws_unsubscribe",
Name: "取消订阅",
Description: "取消订阅标的",
Action: "unsubscribe",
Symbols: []string{"000001.SZ"},
},
}
return &api.WSTestListData{
Cases: cases,
WSURL: "",
}, nil
}
// RunWSTest 执行WebSocket测试
func (s *TestServiceImpl) RunWSTest(ctx context.Context, wsURL string, req *api.WSTestRequest) (*api.WSTestResult, error) {
// 获取测试用例
testList, _ := s.GetWSTestList(ctx)
var testCase *api.WSTestCase
for _, item := range testList.Cases {
if item.ID == req.ID {
testCase = &item
break
}
}
if testCase == nil {
return nil, fmt.Errorf("test case not found: %s", req.ID)
}
// 使用自定义标的
symbols := testCase.Symbols
if len(req.Symbols) > 0 {
symbols = req.Symbols
}
result := &api.WSTestResult{
ID: fmt.Sprintf("ws_%d", time.Now().Unix()),
CaseID: req.ID,
Timestamp: time.Now(),
Messages: []api.WSMessage{},
}
// 连接WebSocket
dialer := websocket.Dialer{
HandshakeTimeout: 10 * time.Second,
}
// 将 http:// 或 https:// 替换为 ws:// 或 wss://
if strings.HasPrefix(wsURL, "https://") {
wsURL = strings.Replace(wsURL, "https://", "wss://", 1)
} else if strings.HasPrefix(wsURL, "http://") {
wsURL = strings.Replace(wsURL, "http://", "ws://", 1)
}
startTime := time.Now()
conn, resp, err := dialer.Dial(wsURL, http.Header{
"X-API-Key": []string{"test-api-key"},
})
result.Latency = time.Since(startTime).Milliseconds()
if err != nil {
result.Success = false
if resp != nil {
result.Error = fmt.Sprintf("连接失败，状态码: %d", resp.StatusCode)
} else {
result.Error = fmt.Sprintf("连接失败: %v", err)
}
s.addWSHistory(result)
return result, nil
}
defer conn.Close()
result.Success = true
// 发送订阅消息
msg := map[string]interface{}{
"action": testCase.Action,
"symbols": symbols,
}
if err := conn.WriteJSON(msg); err != nil {
result.Error = fmt.Sprintf("发送消息失败: %v", err)
s.addWSHistory(result)
return result, nil
}
// 等待响应
conn.SetReadDeadline(time.Now().Add(5 * time.Second))
for i := 0; i < 3; i++ { // 最多读取3条消息
var msgData map[string]interface{}
if err := conn.ReadJSON(&msgData); err != nil {
break
}
result.Messages = append(result.Messages, api.WSMessage{
Type: "received",
Data: msgData,
Timestamp: time.Now(),
})
}
s.addWSHistory(result)
return result, nil
}
// GetTestHistory 获取测试历史
func (s *TestServiceImpl) GetTestHistory(ctx context.Context, req *api.TestHistoryRequest) (*api.TestHistoryData, error) {
s.mu.RLock()
defer s.mu.RUnlock()
limit := req.Limit
if limit <= 0 || limit > len(s.apiHistory) {
limit = len(s.apiHistory)
}
// 获取最近的数据
apiTests := make([]api.APITestResult, 0)
wsTests := make([]api.WSTestResult, 0)
if req.Type == "" || req.Type == "api" {
start := len(s.apiHistory) - limit
if start < 0 {
start = 0
}
apiTests = append(apiTests, s.apiHistory[start:]...)
}
if req.Type == "" || req.Type == "ws" {
wsLimit := req.Limit
if wsLimit <= 0 || wsLimit > len(s.wsHistory) {
wsLimit = len(s.wsHistory)
}
start := len(s.wsHistory) - wsLimit
if start < 0 {
start = 0
}
wsTests = append(wsTests, s.wsHistory[start:]...)
}
return &api.TestHistoryData{
APITests: apiTests,
WSTests: wsTests,
}, nil
}
// addAPIHistory 添加API测试历史
func (s *TestServiceImpl) addAPIHistory(result *api.APITestResult) {
s.mu.Lock()
defer s.mu.Unlock()
s.apiHistory = append(s.apiHistory, *result)
// 限制历史记录数量
if len(s.apiHistory) > s.historySize {
s.apiHistory = s.apiHistory[len(s.apiHistory)-s.historySize:]
}
}
// addWSHistory 添加WebSocket测试历史
func (s *TestServiceImpl) addWSHistory(result *api.WSTestResult) {
s.mu.Lock()
defer s.mu.Unlock()
s.wsHistory = append(s.wsHistory, *result)
// 限制历史记录数量
if len(s.wsHistory) > s.historySize {
s.wsHistory = s.wsHistory[len(s.wsHistory)-s.historySize:]
}
}

374
internal/websocket/server.go
Unescape View File

@ -0,0 +1,374 @@
package websocket
import (
"context"
"encoding/json"
"log"
"net/http"
"sync"
"time"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
"market-data-service/api"
)
var upgrader = websocket.Upgrader{
CheckOrigin: func(r *http.Request) bool {
return true // 允许所有来源，生产环境需要限制
},
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
// Hub WebSocket连接管理中心
type Hub struct {
// 已注册的客户端
clients map[*Client]bool
// 广播消息通道
broadcast chan []byte
// 注册请求通道
register chan *Client
// 注销请求通道
unregister chan *Client
// 标的订阅映射: symbol -> clients
subscriptions map[string]map[*Client]bool
// 保护subscriptions的锁
subMu sync.RWMutex
// 最大订阅标的数
maxSymbolsPerClient int
}
// NewHub 创建Hub
func NewHub() *Hub {
return &Hub{
clients: make(map[*Client]bool),
broadcast: make(chan []byte),
register: make(chan *Client),
unregister: make(chan *Client),
subscriptions: make(map[string]map[*Client]bool),
maxSymbolsPerClient: 100,
}
}
// Run 启动Hub
func (h *Hub) Run() {
for {
select {
case client := <-h.register:
h.clients[client] = true
log.Printf("Client registered, total: %d", len(h.clients))
case client := <-h.unregister:
if _, ok := h.clients[client]; ok {
delete(h.clients, client)
close(client.send)
// 清理订阅
h.removeAllSubscriptions(client)
log.Printf("Client unregistered, total: %d", len(h.clients))
}
case message := <-h.broadcast:
for client := range h.clients {
select {
case client.send <- message:
default:
// 发送缓冲满，关闭连接
close(client.send)
delete(h.clients, client)
}
}
}
}
}
// Subscribe 客户端订阅标的
func (h *Hub) Subscribe(client *Client, symbols []string) error {
if len(client.subscriptions)+len(symbols) > h.maxSymbolsPerClient {
return api.ErrRateLimit
}
h.subMu.Lock()
defer h.subMu.Unlock()
for _, symbol := range symbols {
if _, ok := h.subscriptions[symbol]; !ok {
h.subscriptions[symbol] = make(map[*Client]bool)
}
h.subscriptions[symbol][client] = true
client.subscriptions[symbol] = true
}
return nil
}
// Unsubscribe 客户端取消订阅
func (h *Hub) Unsubscribe(client *Client, symbols []string) {
h.subMu.Lock()
defer h.subMu.Unlock()
for _, symbol := range symbols {
if clients, ok := h.subscriptions[symbol]; ok {
delete(clients, client)
if len(clients) == 0 {
delete(h.subscriptions, symbol)
}
}
delete(client.subscriptions, symbol)
}
}
// removeAllSubscriptions 移除客户端所有订阅
func (h *Hub) removeAllSubscriptions(client *Client) {
h.subMu.Lock()
defer h.subMu.Unlock()
for symbol := range client.subscriptions {
if clients, ok := h.subscriptions[symbol]; ok {
delete(clients, client)
if len(clients) == 0 {
delete(h.subscriptions, symbol)
}
}
}
}
// BroadcastToSymbol 向订阅了某标的的所有客户端广播
func (h *Hub) BroadcastToSymbol(symbol string, data []byte) {
h.subMu.RLock()
clients := h.subscriptions[symbol]
h.subMu.RUnlock()
for client := range clients {
select {
case client.send <- data:
default:
// 发送缓冲满，稍后处理
}
}
}
// GetSubscriptionStats 获取订阅统计
func (h *Hub) GetSubscriptionStats() map[string]interface{} {
h.subMu.RLock()
defer h.subMu.RUnlock()
return map[string]interface{}{
"total_clients": len(h.clients),
"total_subscriptions": len(h.subscriptions),
}
}
// Client WebSocket客户端
type Client struct {
hub *Hub
conn *websocket.Conn
send chan []byte
// 已订阅的标的
subscriptions map[string]bool
subMu sync.RWMutex
}
// NewClient 创建客户端
func NewClient(hub *Hub, conn *websocket.Conn) *Client {
return &Client{
hub: hub,
conn: conn,
send: make(chan []byte, 256),
subscriptions: make(map[string]bool),
}
}
// ReadPump 读取客户端消息
func (c *Client) ReadPump() {
defer func() {
c.hub.unregister <- c
c.conn.Close()
}()
c.conn.SetReadLimit(512 * 1024) // 512KB
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
return nil
})
for {
_, message, err := c.conn.ReadMessage()
if err != nil {
if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
log.Printf("WebSocket error: %v", err)
}
break
}
// 处理客户端消息
c.handleMessage(message)
}
}
// WritePump 向客户端写入消息
func (c *Client) WritePump() {
ticker := time.NewTicker(30 * time.Second)
defer func() {
ticker.Stop()
c.conn.Close()
}()
for {
select {
case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return
}
c.conn.WriteMessage(websocket.TextMessage, message)
case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return
}
}
}
}
// handleMessage 处理客户端消息
func (c *Client) handleMessage(data []byte) {
var msg ClientMessage
if err := json.Unmarshal(data, &msg); err != nil {
c.sendError(1000, "Invalid message format")
return
}
switch msg.Action {
case "subscribe":
c.handleSubscribe(msg.Symbols)
case "unsubscribe":
c.handleUnsubscribe(msg.Symbols)
default:
c.sendError(1001, "Unknown action")
}
}
// handleSubscribe 处理订阅请求
func (c *Client) handleSubscribe(symbols []string) {
if len(symbols) == 0 {
c.sendError(1002, "Symbols cannot be empty")
return
}
if err := c.hub.Subscribe(c, symbols); err != nil {
c.sendError(1003, err.Error())
return
}
// 发送确认
ack := map[string]interface{}{
"type": "ack",
"action": "subscribe",
"symbols": symbols,
"ts": time.Now().Format(time.RFC3339),
}
data, _ := json.Marshal(ack)
c.send <- data
}
// handleUnsubscribe 处理取消订阅请求
func (c *Client) handleUnsubscribe(symbols []string) {
c.hub.Unsubscribe(c, symbols)
ack := map[string]interface{}{
"type": "ack",
"action": "unsubscribe",
"symbols": symbols,
"ts": time.Now().Format(time.RFC3339),
}
data, _ := json.Marshal(ack)
c.send <- data
}
// sendError 发送错误消息
func (c *Client) sendError(code int, message string) {
err := map[string]interface{}{
"type": "error",
"code": code,
"message": message,
"ts": time.Now().Format(time.RFC3339),
}
data, _ := json.Marshal(err)
c.send <- data
}
// ClientMessage 客户端消息结构
type ClientMessage struct {
Action string `json:"action"`
Symbols []string `json:"symbols"`
}
// Server WebSocket服务器
type Server struct {
hub *Hub
}
// NewServer 创建WebSocket服务器
func NewServer(hub *Hub) *Server {
return &Server{hub: hub}
}
// HandleWebSocket 处理WebSocket连接
func (s *Server) HandleWebSocket(c *gin.Context) {
// 认证检查
apiKey := c.GetHeader("X-API-Key")
if apiKey == "" {
c.JSON(http.StatusUnauthorized, gin.H{"error": "Missing API Key"})
return
}
conn, err := upgrader.Upgrade(c.Writer, c.Request, nil)
if err != nil {
log.Printf("WebSocket upgrade failed: %v", err)
return
}
client := NewClient(s.hub, conn)
s.hub.register <- client
go client.WritePump()
go client.ReadPump()
}
// BroadcastTick 广播Tick数据
func (s *Server) BroadcastTick(symbol string, tick map[string]interface{}) {
data, err := json.Marshal(tick)
if err != nil {
return
}
s.hub.BroadcastToSymbol(symbol, data)
}
// BroadcastKLine 广播K线闭合数据
func (s *Server) BroadcastKLine(symbol string, freq string, kline map[string]interface{}) {
msg := map[string]interface{}{
"type": "klines",
"symbol": symbol,
"freq": freq,
"data": kline,
"ts": time.Now().Format(time.RFC3339),
}
data, err := json.Marshal(msg)
if err != nil {
return
}
s.hub.BroadcastToSymbol(symbol, data)
}

73
pkg/config/config.go
Unescape View File

@ -0,0 +1,73 @@
package config
import (
"encoding/json"
"os"
)
// Config 配置管理
type Config struct {
Server ServerConfig `json:"server" yaml:"server"`
Database DatabaseConfig `json:"database" yaml:"database"`
Redis RedisConfig `json:"redis" yaml:"redis"`
Sources SourcesConfig `json:"sources" yaml:"sources"`
}
type ServerConfig struct {
Port int `json:"port" yaml:"port"`
Mode string `json:"mode" yaml:"mode"` // debug/release
APIKey string `json:"api_key" yaml:"api_key"`
}
type DatabaseConfig struct {
Host string `json:"host" yaml:"host"`
Port int `json:"port" yaml:"port"`
User string `json:"user" yaml:"user"`
Password string `json:"password" yaml:"password"`
Database string `json:"database" yaml:"database"`
}
type RedisConfig struct {
Host string `json:"host" yaml:"host"`
Port int `json:"port" yaml:"port"`
Password string `json:"password" yaml:"password"`
DB int `json:"db" yaml:"db"`
}
type SourcesConfig struct {
Stock SourceConfig `json:"stock" yaml:"stock"`
Futures SourceConfig `json:"futures" yaml:"futures"`
}
type SourceConfig struct {
Active string `json:"active" yaml:"active"`
Sources map[string]SourceInfo `json:"list" yaml:"list"`
}
type SourceInfo struct {
Type string `json:"type" yaml:"type"`
Config map[string]string `json:"config" yaml:"config"`
}
// Load 加载配置
func Load(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, err
}
// 设置默认值
if cfg.Server.Port == 0 {
cfg.Server.Port = 8080
}
if cfg.Server.Mode == "" {
cfg.Server.Mode = "debug"
}
return &cfg, nil
}

36
pkg/errors/errors.go
Unescape View File

@ -0,0 +1,36 @@
package errors
import "errors"
// 业务错误定义
var (
// 参数错误
ErrInvalidParam = errors.New("参数错误")
ErrInvalidSymbol = errors.New("无效的标的代码")
ErrInvalidDate = errors.New("无效的日期格式")
// 数据错误
ErrSymbolNotFound = errors.New("标的不存在")
ErrDataNotFound = errors.New("数据不存在")
ErrDataSourceUnavailable = errors.New("数据源不可用")
// 权限错误
ErrUnauthorized = errors.New("未授权")
ErrRateLimit = errors.New("请求过于频繁")
// 系统错误
ErrInternal = errors.New("服务器内部错误")
)
// ErrorCode 错误码
type ErrorCode int
const (
CodeOK ErrorCode = 0
CodeBadRequest ErrorCode = 400
CodeUnauthorized ErrorCode = 401
CodeNotFound ErrorCode = 404
CodeRateLimit ErrorCode = 429
CodeInternal ErrorCode = 500
)

20
pkg/logger/logger.go
Unescape View File

@ -0,0 +1,20 @@
package logger
import "log"
// Logger 日志工具
// Info 信息日志
func Info(format string, v ...interface{}) {
log.Printf("[INFO] "+format, v...)
}
// Error 错误日志
func Error(format string, v ...interface{}) {
log.Printf("[ERROR] "+format, v...)
}
// Debug 调试日志
func Debug(format string, v ...interface{}) {
log.Printf("[DEBUG] "+format, v...)
}

325
python_market_data_service/MIGRATION_GUIDE.md
Unescape View File

@ -0,0 +1,325 @@
# Go 到 Python 迁移指南
本文档详细说明了从Go项目迁移到Python项目的对应关系。
## 技术栈对照表
| Go | Python | 说明 |
|----|--------|------|
| Gin | FastAPI | Web框架 |
| Gorilla WebSocket | FastAPI原生WebSocket | WebSocket支持 |
| database/sql + pq | SQLAlchemy + psycopg2 | 数据库访问 |
| encoding/json | Pydantic + json | JSON序列化 |
| os.Getenv | python-dotenv + pydantic-settings | 环境变量 |
| log | logging | 日志 |
| time | datetime | 时间处理 |
| sync.Mutex | threading.Lock/asyncio.Lock | 并发锁 |
| context.Context | 直接使用async/await | 上下文 |
## 项目结构对照表
```
Go Python
├── adapter/ app/adapters/
│ ├── adapter.go base.py
│ └── tushare/
│ ├── adapter.go tushare_adapter.py
│ └── client.go (集成到adapter)
├── api/
│ ├── types.go app/models/types.py
│ ├── router.go app/api/routes.py
│ ├── admin_types.go app/models/admin_types.py
│ └── admin_router.go app/api/admin_routes.py
├── cmd/
│ ├── server/main.go app/main.py
│ └── sync/main.go scripts/sync_data.py
├── internal/
│ ├── handler/
│ │ ├── handler.go (合并到routes)
│ │ └── admin.go (合并到admin_routes)
│ ├── service/
│ │ ├── service.go (拆分到各service)
│ │ ├── stock.go app/services/stock_service.py
│ │ ├── futures.go app/services/futures_service.py
│ │ ├── admin.go app/services/admin_service.py
│ │ ├── config.go app/services/config_service.py
│ │ ├── adapter.go app/services/adapter_service.py
│ │ └── test.go app/services/test_service.py
│ ├── repository/
│ │ ├── repository.go (合并)
│ │ ├── stock.go app/repositories/stock_repository.py
│ │ └── futures.go app/repositories/futures_repository.py
│ ├── model/model.go app/repositories/models.py
│ ├── websocket/server.go app/websocket/server.py
│ └── monitor/monitor.go app/monitor/monitor.py
├── pkg/
│ ├── config/config.go app/core/config.py
│ ├── logger/logger.go app/core/logger.py
│ └── errors/errors.go app/core/errors.py
└── config.json config.json (相同)
```
## 类型系统对照表
### 基础类型
| Go | Python |
|----|--------|
| `type Frequency string` | `class Frequency(str, Enum)` |
| `type AdjustType string` | `class AdjustType(str, Enum)` |
| `type AssetClass string` | `class AssetClass(str, Enum)` |
| `struct KLineItem` | `class KLineItem(BaseModel)` |
| `struct KLineData` | `class KLineData(BaseModel)` |
| `interface Handler` | 通过FastAPI依赖注入实现 |
### 类型转换示例
**Go:**
```go
type KLineItem struct {
Time time.Time `json:"time"`
Open float64 `json:"open"`
Volume int64 `json:"volume"`
}
```
**Python:**
```python
class KLineItem(BaseModel):
time: datetime = Field(..., description="时间戳")
open: float = Field(..., description="开盘价")
volume: int = Field(..., description="成交量")
class Config:
json_encoders = {
datetime: lambda v: v.isoformat()
}
```
## 接口路由对照表
### 股票接口
| Go路由 | Python路由 | 方法 |
|--------|------------|------|
| `stock.GET("/klines/:symbol", r.queryStockKLines)` | `@router.get("/stock/klines/{symbol}")` | GET |
| `stock.GET("/symbols", r.listStockSymbols)` | `@router.get("/stock/symbols")` | GET |
| `stock.POST("/klines/batch", r.batchQueryStockKLines)` | `@router.post("/stock/klines/batch")` | POST |
| `stock.GET("/trading-dates", r.getStockTradingDates)` | `@router.get("/stock/trading-dates")` | GET |
### 参数绑定对比
**Go (Gin):**
```go
func (r *Router) queryStockKLines(c *gin.Context) {
var req KLineQueryRequest
if err := c.ShouldBindQuery(&req); err != nil {
c.JSON(http.StatusBadRequest, ErrorResponse{...})
return
}
req.Symbol = c.Param("symbol")
// ...
}
```
**Python (FastAPI):**
```python
@router.get("/stock/klines/{symbol}")
def query_stock_klines(
symbol: str, # 路径参数
start: str = Query(...), # 查询参数
end: str = Query(...),
db: Session = Depends(get_db) # 依赖注入
):
service = StockService(db)
req = KLineQueryRequest(symbol=symbol, start=start, end=end)
data = service.query_klines(req)
return Response(code=0, message="success", data=data)
```
## 数据库访问对照表
### 查询K线数据
**Go:**
```go
query := fmt.Sprintf(`
SELECT ts, open, high, low, close, volume, amount
FROM %s
WHERE symbol_id = $1 AND ts >= $2 AND ts <= $3
ORDER BY ts ASC
`, tableName)
rows, err := r.db.QueryContext(ctx, query, symbol, start, end)
```
**Python (SQLAlchemy):**
```python
query = self.db.query(kline_model).filter(
kline_model.symbol_id == symbol,
kline_model.ts >= start,
kline_model.ts <= end
).order_by(kline_model.ts.asc())
results = query.all()
```
### 批量插入
**Go:**
```go
query := fmt.Sprintf(`
INSERT INTO %s (symbol_id, ts, open, ...)
VALUES %s
ON CONFLICT (symbol_id, ts) DO UPDATE SET...
`, tableName, strings.Join(valueStrs, ","))
_, err := r.db.ExecContext(ctx, query, args...)
```
**Python (SQLAlchemy):**
```python
for item in items:
existing = self.db.query(kline_model).filter(...).first()
if existing:
# 更新
existing.open = item.open
...
else:
# 插入
new_record = kline_model(...)
self.db.add(new_record)
self.db.commit()
```
## WebSocket对照表
### Go (Gorilla)
```go
type Hub struct {
clients map[*Client]bool
subscriptions map[string]map[*Client]bool
}
func (h *Hub) Run() {
for {
select {
case client := <-h.register:
h.clients[client] = true
// ...
}
}
}
```
### Python (FastAPI)
```python
class WebSocketManager:
def __init__(self):
self.clients: Dict[str, WSClient] = {}
self.subscriptions: Dict[str, Set[str]] = {}
self.lock = asyncio.Lock()
async def connect(self, websocket: WebSocket, client_id: str):
await websocket.accept()
async with self.lock:
self.clients[client_id] = WSClient(id=client_id, websocket=websocket)
```
## 配置管理对照表
### Go
```go
type Config struct {
Server ServerConfig `json:"server"`
Database DatabaseConfig `json:"database"`
}
func Load(path string) (*Config, error) {
data, err := os.ReadFile(path)
// ...
json.Unmarshal(data, &cfg)
return &cfg, nil
}
```
### Python
```python
class Config(BaseModel):
server: ServerConfig = Field(default_factory=ServerConfig)
database: DatabaseConfig = Field(default_factory=DatabaseConfig)
def load_config(config_path: str = "./config.json") -> Config:
with open(config_path, 'r') as f:
data = json.load(f)
return Config.model_validate(data)
```
## 启动方式对照表
### Go
```bash
go run ./cmd/server/main.go
```
### Python
```bash
# 方式1: 直接运行
python -m app.main
# 方式2: 使用uvicorn
uvicorn app.main:app --reload --port 8080
# 方式3: 生产环境
gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker
```
## 数据同步工具对照表
### Go
```bash
go run ./cmd/sync -type stocks
go run ./cmd/sync -type klines -symbol 000001.SZ -start 20240301 -end 20240307
```
### Python
```bash
python scripts/sync_data.py --type stocks
python scripts/sync_data.py --type klines --symbol 000001.SZ --start 20240301 --end 20240307
```
## 依赖管理对照表
### Go (go.mod)
```go
require (
github.com/gin-gonic/gin v1.9.1
github.com/gorilla/websocket v1.5.0
github.com/lib/pq v1.10.9
)
```
### Python (requirements.txt)
```
fastapi==0.115.0
uvicorn[standard]==0.32.0
sqlalchemy==2.0.36
psycopg2-binary==2.9.10
```
## 测试接口对照表
| 接口 | Go调用 | Python调用 |
|------|--------|------------|
| 健康检查 | `curl http://localhost:8080/v1/admin/health` | 相同 |
| 查询股票K线 | `curl "http://localhost:8080/v1/stock/klines/000001.SZ?start=20250301&end=20250307" -H "X-API-Key: key"` | 相同 |
| 批量查询 | `curl -X POST ... -d '{"symbols":["000001.SZ"],...}'` | 相同 |
所有API接口和响应格式与Go版本完全一致，客户端无需任何修改即可切换到Python后端。

237
python_market_data_service/README.md
Unescape View File

@ -0,0 +1,237 @@
# 统一行情数据服务 - Python实现
Python版本的统一行情数据服务，所有接口和功能与Go版本保持一致。
## 特性
- **多周期K线支持**：1m/5m/15m/30m/60m/1d/1w/1month
- **股票复权支持**：前复权(qfq)/后复权(hfq)
- **数据源热切换**：支持Wind、Tushare等多个数据源动态切换
- **双轨设计**：股票和期货接口独立，数据存储隔离
- **WebSocket实时订阅**：支持实时行情推送
- **数据质量监控**：自动检测数据缺失并告警
- **交易日历**：支持查询股票和期货的交易日历
- **期货合约查询**：根据品种获取可交易合约列表
## 技术栈
- **语言**: Python 3.10+
- **Web框架**: FastAPI
- **WebSocket**: FastAPI原生WebSocket + python-socketio
- **数据库**: PostgreSQL 15+ (SQLAlchemy ORM)
- **数据源**: Tushare (首期支持)
## 项目结构
```
python_market_data_service/
├── app/
│ ├── __init__.py
│ ├── main.py # 主程序入口
│ ├── api/ # API路由
│ │ ├── __init__.py
│ │ ├── routes.py # 主要API路由
│ │ └── admin_routes.py # 管理后台路由
│ ├── core/ # 核心模块
│ │ ├── __init__.py
│ │ ├── config.py # 配置管理
│ │ ├── errors.py # 错误定义
│ │ └── logger.py # 日志工具
│ ├── models/ # 数据模型
│ │ ├── __init__.py
│ │ ├── types.py # 基础类型
│ │ └── admin_types.py # 管理后台类型
│ ├── repositories/ # 数据访问层
│ │ ├── __init__.py
│ │ ├── database.py # 数据库连接
│ │ ├── models.py # 数据库模型
│ │ ├── stock_repository.py
│ │ └── futures_repository.py
│ ├── services/ # 业务逻辑层
│ │ ├── __init__.py
│ │ ├── stock_service.py
│ │ ├── futures_service.py
│ │ ├── admin_service.py
│ │ ├── config_service.py
│ │ ├── adapter_service.py
│ │ └── test_service.py
│ ├── adapters/ # 数据源适配器
│ │ ├── __init__.py
│ │ ├── base.py # 适配器基类
│ │ └── tushare_adapter.py
│ └── websocket/ # WebSocket服务
│ ├── __init__.py
│ └── server.py
├── scripts/
│ └── sync_data.py # 数据同步工具
├── tests/ # 测试文件
├── requirements.txt # 依赖列表
├── pyproject.toml # 项目配置
└── README.md # 本文件
```
## 快速开始
### 1. 环境准备
- Python 3.10+
- PostgreSQL 15+
- Tushare Token (从 [Tushare官网](https://tushare.pro) 获取)
### 2. 安装依赖
```bash
# 创建虚拟环境
python -m venv venv
# 激活虚拟环境
# Windows:
venv\Scripts\activate
# Linux/Mac:
source venv/bin/activate
# 安装依赖
pip install -r requirements.txt
# 安装Tushare（需单独安装）
pip install tushare
```
### 3. 配置环境变量
```bash
# Windows PowerShell
$env:TUSHARE_TOKEN="your_tushare_token"
$env:DATABASE_URL="postgresql://user:password@localhost:5432/marketdata"
# Linux/Mac
export TUSHARE_TOKEN="your_tushare_token"
export DATABASE_URL="postgresql://user:password@localhost:5432/marketdata"
```
### 4. 初始化数据库
```bash
# 创建数据库（使用psql或pgAdmin）
createdb marketdata
# 启动服务时会自动创建表结构
```
### 5. 启动服务
```bash
# 开发模式
python -m app.main
# 或使用uvicorn
uvicorn app.main:app --reload --port 8080
```
服务将启动在 `http://localhost:8080`
- API文档: `http://localhost:8080/docs`
- 管理后台: `http://localhost:8080/admin`
### 6. 同步基础数据
```bash
# 同步股票列表
python scripts/sync_data.py --type stocks
# 同步期货列表
python scripts/sync_data.py --type futures
# 同步交易日历
python scripts/sync_data.py --type calendar --start 20240101 --end 20241231
# 同步K线数据
python scripts/sync_data.py --type klines --symbol 000001.SZ --start 20240301 --end 20240307 --freq 1d
```
## API接口
### 股票接口
| 接口 | 方法 | 说明 |
|------|------|------|
| `/v1/stock/klines/:symbol` | GET | 查询K线数据 |
| `/v1/stock/symbols` | GET | 查询标的列表 |
| `/v1/stock/klines/batch` | POST | 批量查询K线 |
| `/v1/stock/trading-dates` | GET | 获取交易日历 |
### 期货接口
| 接口 | 方法 | 说明 |
|------|------|------|
| `/v1/futures/klines/:symbol` | GET | 查询K线数据 |
| `/v1/futures/symbols` | GET | 查询标的列表 |
| `/v1/futures/klines/batch` | POST | 批量查询K线 |
| `/v1/futures/continuous/:underlying` | GET | 查询主力连续合约（预留） |
| `/v1/futures/trading-dates` | GET | 获取交易日历 |
| `/v1/futures/contracts` | GET | 获取品种合约列表 |
### 管理接口
| 接口 | 方法 | 说明 |
|------|------|------|
| `/v1/admin/source/status` | GET | 获取数据源状态 |
| `/v1/admin/source/switch` | POST | 切换数据源 |
| `/v1/admin/backfill` | POST | 历史数据补录 |
| `/v1/admin/health` | GET | 健康检查 |
### 管理后台
服务启动后，访问 `http://localhost:8080/admin` 进入管理后台。
### WebSocket实时订阅
**连接地址**: `ws://localhost:8080/v1/stream`
**认证**: 连接时在Header中传递 `X-API-Key`
**客户端消息**:
```json
// 订阅
{
"action": "subscribe",
"symbols": ["000001.SZ", "CU2504.SHFE"]
}
// 取消订阅
{
"action": "unsubscribe",
"symbols": ["000001.SZ"]
}
```
**服务器消息**:
```json
// 订阅确认
{
"type": "ack",
"action": "subscribe",
"symbols": ["000001.SZ", "CU2504.SHFE"],
"ts": "2025-03-07T12:30:00Z"
}
// 心跳
{
"type": "heartbeat",
"ts": "2025-03-07T12:30:30Z"
}
```
**限制**: 单连接最大订阅100个标的
## 与Go版本的主要区别
1. **Web框架**: Gin -> FastAPI
2. **ORM**: 原生SQL -> SQLAlchemy
3. **WebSocket**: Gorilla -> FastAPI原生
4. **配置**: 文件+环境变量 -> Pydantic Settings
5. **API文档**: 自动生成Swagger/ReDoc
## License
MIT

2
python_market_data_service/app/__init__.py
Unescape View File

@ -0,0 +1,2 @@
"""Market Data Service - Python实现"""
__version__ = "1.0.0"

13
python_market_data_service/app/adapters/__init__.py
Unescape View File

@ -0,0 +1,13 @@
"""数据源适配器模块"""
from .base import DataSourceAdapter, TickData, KLineData, SymbolInfo, TradeCalData, TickCallback
from .tushare_adapter import TushareAdapter
__all__ = [
"DataSourceAdapter",
"TickData",
"KLineData",
"SymbolInfo",
"TradeCalData",
"TickCallback",
"TushareAdapter",
]

102
python_market_data_service/app/adapters/base.py
Unescape View File

@ -0,0 +1,102 @@
"""数据源适配器基类 - 对应Go的adapter/adapter.go"""
from abc import ABC, abstractmethod
from dataclasses import dataclass
from datetime import datetime
from typing import Callable, List, Optional
@dataclass
class TickData:
"""Tick数据"""
symbol: str
price: float
volume: int
time: int # Unix时间戳
@dataclass
class KLineData:
"""K线数据"""
symbol: str
time: int # Unix时间戳
open: float
high: float
low: float
close: float
volume: int
amount: float
open_interest: int = 0
@dataclass
class SymbolInfo:
"""标的信息"""
symbol_id: str
name: str
exchange: str
underlying: str = "" # 期货品种代码
contract_month: str = ""
list_date: str = ""
delist_date: str = ""
@dataclass
class TradeCalData:
"""交易日历数据"""
date: datetime
is_trading_day: bool
has_night_session: bool = False
# Tick数据回调类型
TickCallback = Callable[[str, TickData], None]
class DataSourceAdapter(ABC):
"""数据源适配器接口"""
@abstractmethod
async def connect(self, config: dict) -> None:
"""建立连接"""
pass
@abstractmethod
async def subscribe_ticks(self, symbols: List[str], callback: TickCallback) -> None:
"""订阅实时Tick"""
pass
@abstractmethod
async def fetch_klines(
self,
symbol: str,
start: str,
end: str,
freq: str
) -> List[KLineData]:
"""拉取历史K线"""
pass
@abstractmethod
async def fetch_symbols(self, asset_type: str) -> List[SymbolInfo]:
"""获取标的列表"""
pass
@abstractmethod
async def fetch_trading_calendar(
self,
exchange: str,
start: str,
end: str
) -> List[TradeCalData]:
"""获取交易日历"""
pass
@abstractmethod
async def health_check(self) -> bool:
"""健康检查"""
pass
@abstractmethod
async def close(self) -> None:
"""关闭连接"""
pass

372
python_market_data_service/app/adapters/tushare_adapter.py
Unescape View File

@ -0,0 +1,372 @@
"""Tushare数据源适配器 - 对应Go的adapter/tushare/adapter.go"""
import asyncio
from datetime import datetime
from typing import List, Optional
import tushare as ts
import pandas as pd
from app.adapters.base import (
DataSourceAdapter, TickData, KLineData, SymbolInfo,
TradeCalData, TickCallback
)
from app.core.logger import info, error
class TushareAdapter(DataSourceAdapter):
"""Tushare数据源适配器"""
def __init__(self):
self.pro = None
self.token = None
self.config = {}
async def connect(self, config: dict) -> None:
"""建立连接"""
self.token = config.get("token")
if not self.token:
raise ValueError("Tushare token is required")
# 设置Tushare token
ts.set_token(self.token)
self.pro = ts.pro_api()
self.config = config
info("Tushare adapter connected")
async def subscribe_ticks(self, symbols: List[str], callback: TickCallback) -> None:
"""订阅实时Tick（Tushare不支持实时推送）"""
raise NotImplementedError("Tushare does not support real-time tick subscription")
async def fetch_klines(
self,
symbol: str,
start: str,
end: str,
freq: str
) -> List[KLineData]:
"""拉取历史K线"""
# 判断是股票还是期货
if ".SH" in symbol or ".SZ" in symbol or ".BJ" in symbol:
return await self._fetch_stock_klines(symbol, start, end, freq)
else:
return await self._fetch_futures_klines(symbol, start, end, freq)
async def _fetch_stock_klines(
self,
symbol: str,
start: str,
end: str,
freq: str
) -> List[KLineData]:
"""获取股票K线"""
# 转换日期格式: YYYYMMDD -> YYYYMMDD
start_date = start
end_date = end
if freq in ["1d", "", "day"]:
return await self._fetch_stock_daily(symbol, start_date, end_date)
elif freq in ["1m", "5m", "15m", "30m", "60m"]:
return await self._fetch_stock_minute(symbol, start_date, end_date, freq)
else:
raise ValueError(f"Unsupported frequency: {freq}")
async def _fetch_stock_daily(
self,
ts_code: str,
start_date: str,
end_date: str
) -> List[KLineData]:
"""获取股票日线"""
try:
# 使用线程池执行同步调用
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.daily(
ts_code=ts_code,
start_date=start_date,
end_date=end_date
)
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
trade_date = datetime.strptime(str(row['trade_date']), "%Y%m%d")
results.append(KLineData(
symbol=row['ts_code'],
time=int(trade_date.timestamp()),
open=float(row['open']),
high=float(row['high']),
low=float(row['low']),
close=float(row['close']),
volume=int(row['vol'] * 100), # 手 -> 股
amount=float(row['amount'] * 1000) # 千元 -> 元
))
return results
except Exception as e:
error(f"Failed to fetch stock daily: {e}")
raise
async def _fetch_stock_minute(
self,
ts_code: str,
start_date: str,
end_date: str,
freq: str
) -> List[KLineData]:
"""获取股票分钟线"""
try:
freq_map = {
"1m": "1min",
"5m": "5min",
"15m": "15min",
"30m": "30min",
"60m": "60min"
}
ts_freq = freq_map.get(freq, "1min")
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: ts.pro_bar(
ts_code=ts_code,
freq=ts_freq,
start_date=start_date,
end_date=end_date,
asset="E" # 股票
)
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
trade_time = datetime.strptime(str(row['trade_time']), "%Y-%m-%d %H:%M:%S")
results.append(KLineData(
symbol=row['ts_code'],
time=int(trade_time.timestamp()),
open=float(row['open']),
high=float(row['high']),
low=float(row['low']),
close=float(row['close']),
volume=int(row['vol'] * 100),
amount=float(row['amount'] * 1000)
))
return results
except Exception as e:
error(f"Failed to fetch stock minute: {e}")
raise
async def _fetch_futures_klines(
self,
symbol: str,
start: str,
end: str,
freq: str
) -> List[KLineData]:
"""获取期货K线"""
if freq in ["1d", "", "day"]:
return await self._fetch_futures_daily(symbol, start, end)
elif freq in ["1m", "5m", "15m", "30m", "60m"]:
return await self._fetch_futures_minute(symbol, start, end, freq)
else:
raise ValueError(f"Unsupported frequency: {freq}")
async def _fetch_futures_daily(
self,
ts_code: str,
start_date: str,
end_date: str
) -> List[KLineData]:
"""获取期货日线"""
try:
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.fut_daily(
ts_code=ts_code,
start_date=start_date,
end_date=end_date
)
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
trade_date = datetime.strptime(str(row['trade_date']), "%Y%m%d")
results.append(KLineData(
symbol=row['ts_code'],
time=int(trade_date.timestamp()),
open=float(row['open']),
high=float(row['high']),
low=float(row['low']),
close=float(row['close']),
volume=int(row['vol']),
amount=float(row['amount'] * 10000), # 万元 -> 元
open_interest=int(row.get('oi', 0))
))
return results
except Exception as e:
error(f"Failed to fetch futures daily: {e}")
raise
async def _fetch_futures_minute(
self,
ts_code: str,
start_date: str,
end_date: str,
freq: str
) -> List[KLineData]:
"""获取期货分钟线"""
# Tushare期货分钟线需要通过stk_mins接口，但需要特殊权限
# 这里简化处理，实际使用时可能需要其他数据源
raise NotImplementedError("Futures minute data requires special Tushare permission")
async def fetch_symbols(self, asset_type: str) -> List[SymbolInfo]:
"""获取标的列表"""
if asset_type == "stock":
return await self._fetch_stock_symbols()
elif asset_type == "futures":
return await self._fetch_futures_symbols()
else:
raise ValueError(f"Unsupported asset type: {asset_type}")
async def _fetch_stock_symbols(self) -> List[SymbolInfo]:
"""获取股票列表"""
try:
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.stock_basic(
exchange="",
list_status="L" # 上市状态
)
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
results.append(SymbolInfo(
symbol_id=row['ts_code'],
name=row['name'],
exchange=row['exchange'],
list_date=str(row.get('list_date', '')),
delist_date=str(row.get('delist_date', ''))
))
return results
except Exception as e:
error(f"Failed to fetch stock symbols: {e}")
raise
async def _fetch_futures_symbols(self) -> List[SymbolInfo]:
"""获取期货列表"""
try:
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.fut_basic(exchange="")
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
results.append(SymbolInfo(
symbol_id=row['ts_code'],
name=row['name'],
exchange=row['exchange'],
underlying=row.get('fut_code', ''),
contract_month=str(row['symbol'])[len(str(row.get('fut_code', ''))):],
list_date=str(row.get('list_date', '')),
delist_date=str(row.get('delist_date', ''))
))
return results
except Exception as e:
error(f"Failed to fetch futures symbols: {e}")
raise
async def fetch_trading_calendar(
self,
exchange: str,
start: str,
end: str
) -> List[TradeCalData]:
"""获取交易日历"""
# Tushare交易所代码映射
exchange_map = {
"SH": "SSE",
"SZ": "SZSE",
"SHFE": "SHFE",
"DCE": "DCE",
"CZCE": "CZCE",
"CFFEX": "CFFEX",
"INE": "INE",
}
ts_exchange = exchange_map.get(exchange, "SSE")
try:
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.trade_cal(
exchange=ts_exchange,
start_date=start,
end_date=end
)
)
if df is None or df.empty:
return []
results = []
for _, row in df.iterrows():
cal_date = datetime.strptime(str(row['cal_date']), "%Y%m%d")
results.append(TradeCalData(
date=cal_date,
is_trading_day=row['is_open'] == 1
))
return results
except Exception as e:
error(f"Failed to fetch trading calendar: {e}")
raise
async def health_check(self) -> bool:
"""健康检查"""
try:
if self.pro is None:
return False
# 尝试获取交易日历作为健康检查
loop = asyncio.get_event_loop()
df = await loop.run_in_executor(
None,
lambda: self.pro.trade_cal(
exchange="SSE",
start_date=datetime.now().strftime("%Y%m%d"),
end_date=datetime.now().strftime("%Y%m%d")
)
)
return df is not None
except Exception as e:
error(f"Health check failed: {e}")
return False
async def close(self) -> None:
"""关闭连接（Tushare是HTTP接口，无需关闭）"""
info("Tushare adapter closed")

5
python_market_data_service/app/api/__init__.py
Unescape View File

@ -0,0 +1,5 @@
"""API路由模块"""
from .routes import router
from .admin_routes import admin_router
__all__ = ["router", "admin_router"]

232
python_market_data_service/app/api/admin_routes.py
Unescape View File

@ -0,0 +1,232 @@
"""管理后台API路由 - 对应Go的api/admin_router.go"""
from fastapi import APIRouter, Depends, HTTPException, Header, Query
from typing import Optional
from app.models import (
Response, ConfigListRequest, ConfigUpdateRequest,
ReloadRequest, AdapterToggleRequest, AdapterConfigUpdateRequest,
APITestRequest, WSTestRequest, TestHistoryRequest
)
from app.services import ConfigService, AdapterService, TestService
from app.core.config import get_config
admin_router = APIRouter()
# 服务实例
config_service = ConfigService()
adapter_service = AdapterService()
test_service = TestService()
def verify_admin_token(x_admin_token: Optional[str] = Header(None)):
"""验证Admin Token"""
# TODO: 实现Token验证
return x_admin_token
# ============================================
# 系统管理接口
# ============================================
@admin_router.get("/admin/system/status", response_model=Response)
def get_system_status(
token: str = Depends(verify_admin_token)
):
"""获取系统状态"""
try:
data = config_service.get_system_status()
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/system/reload", response_model=Response)
def reload_config(
req: Optional[ReloadRequest] = None,
token: str = Depends(verify_admin_token)
):
"""热加载配置"""
try:
if req is None:
req = ReloadRequest()
data = config_service.reload_config(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/system/restart", response_model=Response)
def restart_service(
token: str = Depends(verify_admin_token)
):
"""重启服务"""
# TODO: 实现服务重启逻辑
return Response(
code=0,
message="重启命令已发送",
data={"status": "restarting"}
)
# ============================================
# 配置管理接口
# ============================================
@admin_router.get("/admin/config", response_model=Response)
def get_config_list(
type: Optional[str] = Query(None, description="配置类型筛选"),
token: str = Depends(verify_admin_token)
):
"""获取配置列表"""
try:
from app.models import ConfigType
req = ConfigListRequest()
if type:
req.type = ConfigType(type)
data = config_service.get_config_list(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.put("/admin/config", response_model=Response)
def update_config(
req: ConfigUpdateRequest,
token: str = Depends(verify_admin_token)
):
"""更新配置"""
try:
data = config_service.update_config(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/config/reload", response_model=Response)
def reload_config_endpoint(
req: Optional[ReloadRequest] = None,
token: str = Depends(verify_admin_token)
):
"""热加载配置"""
return reload_config(req, token)
# ============================================
# 适配器管理接口
# ============================================
@admin_router.get("/admin/adapters", response_model=Response)
def get_adapter_list(
token: str = Depends(verify_admin_token)
):
"""获取适配器列表"""
try:
data = adapter_service.get_adapter_list()
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/adapters/toggle", response_model=Response)
def toggle_adapter(
req: AdapterToggleRequest,
token: str = Depends(verify_admin_token)
):
"""切换适配器状态"""
try:
adapter_service.toggle_adapter(req)
return Response(code=0, message="success")
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.put("/admin/adapters/config", response_model=Response)
def update_adapter_config(
req: AdapterConfigUpdateRequest,
token: str = Depends(verify_admin_token)
):
"""更新适配器配置"""
try:
adapter_service.update_adapter_config(req)
return Response(code=0, message="success")
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
# ============================================
# 测试管理接口
# ============================================
@admin_router.get("/admin/tests/api", response_model=Response)
def get_api_test_list(
token: str = Depends(verify_admin_token)
):
"""获取API测试列表"""
try:
data = test_service.get_api_test_list()
# 设置基础URL
config = get_config()
data.base_url = f"http://localhost:{config.server.port}"
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/tests/api/run", response_model=Response)
async def run_api_test(
req: APITestRequest,
token: str = Depends(verify_admin_token)
):
"""执行API测试"""
try:
config = get_config()
base_url = f"http://localhost:{config.server.port}"
data = await test_service.run_api_test(base_url, req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.get("/admin/tests/ws", response_model=Response)
def get_ws_test_list(
token: str = Depends(verify_admin_token)
):
"""获取WebSocket测试列表"""
try:
data = test_service.get_ws_test_list()
config = get_config()
data.ws_url = f"ws://localhost:{config.server.port}/v1/stream"
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.post("/admin/tests/ws/run", response_model=Response)
async def run_ws_test(
req: WSTestRequest,
token: str = Depends(verify_admin_token)
):
"""执行WebSocket测试"""
try:
config = get_config()
ws_url = f"ws://localhost:{config.server.port}/v1/stream"
data = await test_service.run_ws_test(ws_url, req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@admin_router.get("/admin/tests/history", response_model=Response)
def get_test_history(
type: Optional[str] = Query(None, description="测试类型"),
limit: int = Query(default=20, ge=1, le=100, description="数量限制"),
token: str = Depends(verify_admin_token)
):
"""获取测试历史"""
try:
req = TestHistoryRequest(type=type, limit=limit)
data = test_service.get_test_history(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))

304
python_market_data_service/app/api/routes.py
Unescape View File

@ -0,0 +1,304 @@
"""API路由 - 对应Go的api/router.go"""
from fastapi import APIRouter, Depends, HTTPException, Header, Query
from typing import Optional
from sqlalchemy.orm import Session
from app.repositories import get_db
from app.services import StockService, FuturesService, AdminService
from app.models import (
Response, ErrorResponse, HealthResponse,
KLineQueryRequest, SymbolListRequest, BatchKLineRequest,
TradingDatesRequest, FuturesContractsRequest,
SourceSwitchRequest, BackfillRequest
)
from app.core.config import get_config
router = APIRouter()
# 获取配置
config = get_config()
# 认证依赖
def verify_api_key(x_api_key: Optional[str] = Header(None)):
"""验证API Key"""
if not x_api_key:
raise HTTPException(status_code=401, detail="Missing API Key")
# TODO: 验证API Key有效性
return x_api_key
# ============================================
# 股票接口
# ============================================
@router.get("/stock/klines/{symbol}", response_model=Response)
def query_stock_klines(
symbol: str,
start: str = Query(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8),
end: str = Query(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8),
freq: str = Query(default="1d", description="周期"),
adjust: str = Query(default="", description="复权类型"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""查询股票K线"""
try:
service = StockService(db)
req = KLineQueryRequest(
symbol=symbol,
start=start,
end=end,
freq=freq,
adjust=adjust
)
data = service.query_klines(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/stock/symbols", response_model=Response)
def list_stock_symbols(
exchange: Optional[str] = Query(None, description="交易所筛选"),
keyword: Optional[str] = Query(None, description="关键词搜索"),
page: int = Query(default=1, ge=1, description="页码"),
size: int = Query(default=20, ge=1, le=100, description="每页数量"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""查询股票标的列表"""
try:
service = StockService(db)
req = SymbolListRequest(
exchange=exchange,
keyword=keyword,
page=page,
size=size
)
data = service.list_symbols(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/stock/klines/batch", response_model=Response)
def batch_query_stock_klines(
req: BatchKLineRequest,
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""批量查询股票K线"""
try:
service = StockService(db)
data = service.batch_query_klines(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/stock/trading-dates", response_model=Response)
def get_stock_trading_dates(
start: str = Query(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8),
end: str = Query(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""获取股票交易日历"""
try:
service = StockService(db)
req = TradingDatesRequest(start=start, end=end)
data = service.get_trading_dates(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
# ============================================
# 期货接口
# ============================================
@router.get("/futures/klines/{symbol}", response_model=Response)
def query_futures_klines(
symbol: str,
start: str = Query(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8),
end: str = Query(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8),
freq: str = Query(default="1d", description="周期"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""查询期货K线"""
try:
service = FuturesService(db)
req = KLineQueryRequest(
symbol=symbol,
start=start,
end=end,
freq=freq
)
data = service.query_klines(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/futures/symbols", response_model=Response)
def list_futures_symbols(
exchange: Optional[str] = Query(None, description="交易所筛选"),
underlying: Optional[str] = Query(None, description="品种筛选"),
keyword: Optional[str] = Query(None, description="关键词搜索"),
page: int = Query(default=1, ge=1, description="页码"),
size: int = Query(default=20, ge=1, le=100, description="每页数量"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""查询期货标的列表"""
try:
service = FuturesService(db)
req = SymbolListRequest(
exchange=exchange,
underlying=underlying,
keyword=keyword,
page=page,
size=size
)
data = service.list_symbols(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/futures/klines/batch", response_model=Response)
def batch_query_futures_klines(
req: BatchKLineRequest,
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""批量查询期货K线"""
try:
service = FuturesService(db)
data = service.batch_query_klines(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/futures/continuous/{underlying}", response_model=Response)
def query_continuous_klines(
underlying: str,
start: str = Query(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8),
end: str = Query(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8),
freq: str = Query(default="1d", description="周期"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""查询主力连续合约K线（预留）"""
# TODO: 实现主力连续合约查询
from app.models import KLineData
data = KLineData(
symbol=f"{underlying}.MAIN",
name=f"{underlying}主力连续",
freq=freq,
count=0,
items=[]
)
return Response(code=0, message="success", data=data)
@router.get("/futures/trading-dates", response_model=Response)
def get_futures_trading_dates(
start: str = Query(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8),
end: str = Query(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""获取期货交易日历"""
try:
service = FuturesService(db)
req = TradingDatesRequest(start=start, end=end)
data = service.get_trading_dates(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/futures/contracts", response_model=Response)
def get_futures_contracts(
underlying: str = Query(..., description="品种代码"),
exchange: Optional[str] = Query(None, description="交易所筛选"),
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""获取品种合约列表"""
try:
service = FuturesService(db)
req = FuturesContractsRequest(underlying=underlying, exchange=exchange)
data = service.get_contracts_by_underlying(req)
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
# ============================================
# 管理接口
# ============================================
@router.get("/admin/source/status", response_model=Response)
def get_data_source_status(
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""获取数据源状态"""
try:
service = AdminService(db)
data = service.get_data_source_status()
return Response(code=0, message="success", data=data)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/admin/source/switch", response_model=Response)
def switch_data_source(
req: SourceSwitchRequest,
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""切换数据源"""
try:
service = AdminService(db)
service.switch_data_source(req)
return Response(code=0, message="数据源切换成功")
except Exception as e:
raise HTTPException(status_code=422, detail=str(e))
@router.post("/admin/backfill", response_model=Response)
def backfill_data(
req: BackfillRequest,
db: Session = Depends(get_db),
api_key: str = Depends(verify_api_key)
):
"""历史数据补录"""
try:
service = AdminService(db)
task_id = service.backfill_data(req)
return Response(
code=0,
message="补录任务已启动",
data={"task_id": task_id}
)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/admin/health", response_model=HealthResponse)
def health_check(
db: Session = Depends(get_db)
):
"""健康检查（无需认证）"""
try:
service = AdminService(db)
return service.health_check()
except Exception as e:
raise HTTPException(status_code=503, detail=str(e))

0
python_market_data_service/app/core/__init__.py
Unescape View File

131
python_market_data_service/app/core/config.py
Unescape View File

@ -0,0 +1,131 @@
"""配置管理模块"""
import json
import os
from typing import Dict, Any, Optional
from functools import lru_cache
from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings
class ServerConfig(BaseModel):
"""服务器配置"""
port: int = 8080
mode: str = "debug" # debug/release
api_key: str = "demo-api-key-2024"
class DatabaseConfig(BaseModel):
"""数据库配置"""
host: str = "localhost"
port: int = 5432
user: str = "postgres"
password: str = "postgres"
database: str = "marketdata"
@property
def database_url(self) -> str:
return f"postgresql://{self.user}:{self.password}@{self.host}:{self.port}/{self.database}"
class RedisConfig(BaseModel):
"""Redis配置"""
host: str = "localhost"
port: int = 6379
password: str = ""
db: int = 0
class SourceInfo(BaseModel):
"""数据源信息"""
type: str = "http"
config: Dict[str, str] = Field(default_factory=dict)
class SourceConfig(BaseModel):
"""源配置"""
active: str = "tushare"
list: Dict[str, SourceInfo] = Field(default_factory=dict)
class SourcesConfig(BaseModel):
"""数据源配置"""
stock: SourceConfig = Field(default_factory=lambda: SourceConfig(
active="tushare",
list={"tushare": SourceInfo(type="http", config={"base_url": "https://api.tushare.pro"})}
))
futures: SourceConfig = Field(default_factory=lambda: SourceConfig(
active="tushare",
list={"tushare": SourceInfo(type="http", config={"base_url": "https://api.tushare.pro"})}
))
class Config(BaseModel):
"""主配置类"""
server: ServerConfig = Field(default_factory=ServerConfig)
database: DatabaseConfig = Field(default_factory=DatabaseConfig)
redis: RedisConfig = Field(default_factory=RedisConfig)
sources: SourcesConfig = Field(default_factory=SourcesConfig)
class Config:
populate_by_name = True
class Settings(BaseSettings):
"""环境变量配置"""
port: int = Field(default=8080, alias="PORT")
database_url: Optional[str] = Field(default=None, alias="DATABASE_URL")
tushare_token: Optional[str] = Field(default=None, alias="TUSHARE_TOKEN")
api_key: Optional[str] = Field(default=None, alias="API_KEY")
class Config:
env_file = ".env"
case_sensitive = True
def load_config(config_path: str = "./config.json") -> Config:
"""从文件加载配置"""
if not os.path.exists(config_path):
return Config()
with open(config_path, 'r', encoding='utf-8') as f:
data = json.load(f)
return Config.model_validate(data)
def save_config(config: Config, config_path: str = "./config.json"):
"""保存配置到文件"""
os.makedirs(os.path.dirname(config_path) or '.', exist_ok=True)
with open(config_path, 'w', encoding='utf-8') as f:
json.dump(config.model_dump(), f, indent=2, ensure_ascii=False)
# 全局配置实例
_config: Optional[Config] = None
def get_config() -> Config:
"""获取当前配置"""
global _config
if _config is None:
_config = load_config()
return _config
def set_config(config: Config):
"""设置全局配置"""
global _config
_config = config
def reload_config(config_path: str = "./config.json") -> Config:
"""重新加载配置"""
global _config
_config = load_config(config_path)
return _config
@lru_cache()
def get_settings() -> Settings:
"""获取环境变量设置"""
return Settings()

78
python_market_data_service/app/core/errors.py
Unescape View File

@ -0,0 +1,78 @@
"""错误定义模块"""
from enum import IntEnum
from typing import Optional, Any, Dict
class ErrorCode(IntEnum):
"""错误码"""
OK = 0
BAD_REQUEST = 400
UNAUTHORIZED = 401
NOT_FOUND = 404
RATE_LIMIT = 429
INTERNAL = 500
class AppException(Exception):
"""应用异常基类"""
def __init__(
self,
message: str,
code: ErrorCode = ErrorCode.INTERNAL,
detail: Optional[str] = None
):
self.message = message
self.code = code
self.detail = detail
super().__init__(message)
def to_dict(self) -> Dict[str, Any]:
return {
"code": int(self.code),
"message": self.message,
"detail": self.detail
}
# 参数错误
class InvalidParamError(AppException):
def __init__(self, message: str = "参数错误", detail: Optional[str] = None):
super().__init__(message, ErrorCode.BAD_REQUEST, detail)
class InvalidSymbolError(AppException):
def __init__(self, message: str = "无效的标的代码"):
super().__init__(message, ErrorCode.BAD_REQUEST)
class InvalidDateError(AppException):
def __init__(self, message: str = "无效的日期格式"):
super().__init__(message, ErrorCode.BAD_REQUEST)
# 数据错误
class SymbolNotFoundError(AppException):
def __init__(self, message: str = "标的不存在"):
super().__init__(message, ErrorCode.NOT_FOUND)
class DataNotFoundError(AppException):
def __init__(self, message: str = "数据不存在"):
super().__init__(message, ErrorCode.NOT_FOUND)
class DataSourceUnavailableError(AppException):
def __init__(self, message: str = "数据源不可用"):
super().__init__(message, ErrorCode.INTERNAL)
# 权限错误
class UnauthorizedError(AppException):
def __init__(self, message: str = "未授权"):
super().__init__(message, ErrorCode.UNAUTHORIZED)
class RateLimitError(AppException):
def __init__(self, message: str = "请求过于频繁"):
super().__init__(message, ErrorCode.RATE_LIMIT)

47
python_market_data_service/app/core/logger.py
Unescape View File

@ -0,0 +1,47 @@
"""日志工具模块"""
import logging
import sys
from typing import Optional
def setup_logging(
level: int = logging.INFO,
format_string: Optional[str] = None
) -> logging.Logger:
"""设置日志配置"""
if format_string is None:
format_string = "%(asctime)s | %(levelname)-8s | %(message)s"
logging.basicConfig(
level=level,
format=format_string,
handlers=[
logging.StreamHandler(sys.stdout)
]
)
return logging.getLogger("market_data")
# 全局logger实例
logger = setup_logging()
def info(msg: str, *args, **kwargs):
"""信息日志"""
logger.info(msg, *args, **kwargs)
def error(msg: str, *args, **kwargs):
"""错误日志"""
logger.error(msg, *args, **kwargs)
def debug(msg: str, *args, **kwargs):
"""调试日志"""
logger.debug(msg, *args, **kwargs)
def warning(msg: str, *args, **kwargs):
"""警告日志"""
logger.warning(msg, *args, **kwargs)

346
python_market_data_service/app/main.py
Unescape View File

@ -0,0 +1,346 @@
"""主应用入口 - 对应Go的cmd/server/main.go"""
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import HTMLResponse
from fastapi.staticfiles import StaticFiles
from app.api import router, admin_router
from app.websocket import WebSocketServer
from app.core.config import get_config, get_settings
from app.core.logger import info, error, setup_logging
from app.repositories.database import init_db
# 获取配置
config = get_config()
settings = get_settings()
# 设置日志
setup_logging()
@asynccontextmanager
async def lifespan(app: FastAPI):
"""应用生命周期管理"""
# 启动时执行
info("Starting Market Data Service...")
# 初始化数据库
try:
init_db()
info("Database initialized")
except Exception as e:
error(f"Database initialization failed: {e}")
yield
# 关闭时执行
info("Shutting down Market Data Service...")
# 创建FastAPI应用
app = FastAPI(
title="统一行情数据服务",
description="提供股票和期货的标准化行情数据查询服务",
version="1.0.0",
lifespan=lifespan
)
# 添加CORS中间件
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# 注册API路由
app.include_router(router, prefix="/v1")
app.include_router(admin_router, prefix="/v1")
# WebSocket服务器
ws_server = WebSocketServer()
@app.websocket("/v1/stream")
async def websocket_endpoint(websocket):
"""WebSocket端点"""
import uuid
client_id = str(uuid.uuid4())
await ws_server.handle(websocket, client_id)
# 管理后台页面HTML（简化版）
ADMIN_HTML = """<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>行情数据服务 - 管理后台</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC', sans-serif;
background: #f0f2f5;
color: #333;
}
.layout { display: flex; min-height: 100vh; }
.sidebar {
width: 200px;
background: #001529;
color: #fff;
position: fixed;
height: 100vh;
overflow-y: auto;
}
.logo {
padding: 16px;
font-size: 18px;
font-weight: bold;
border-bottom: 1px solid rgba(255,255,255,0.1);
}
.nav-menu { padding: 16px 0; }
.nav-item {
padding: 12px 24px;
cursor: pointer;
transition: all 0.3s;
display: flex;
align-items: center;
gap: 8px;
}
.nav-item:hover { background: rgba(255,255,255,0.05); }
.nav-item.active { background: #1890ff; }
.main-content {
flex: 1;
margin-left: 200px;
padding: 24px;
}
.header {
background: #fff;
padding: 16px 24px;
margin: -24px -24px 24px -24px;
box-shadow: 0 1px 4px rgba(0,0,0,0.1);
display: flex;
justify-content: space-between;
align-items: center;
}
.page-title { font-size: 20px; font-weight: 500; }
.card {
background: #fff;
border-radius: 8px;
padding: 24px;
margin-bottom: 24px;
box-shadow: 0 1px 2px rgba(0,0,0,0.05);
}
.card-title {
font-size: 16px;
font-weight: 500;
margin-bottom: 16px;
padding-bottom: 12px;
border-bottom: 1px solid #f0f0f0;
}
.stats-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 16px;
margin-bottom: 24px;
}
.stat-card {
background: #fff;
border-radius: 8px;
padding: 20px;
box-shadow: 0 1px 2px rgba(0,0,0,0.05);
}
.stat-value {
font-size: 32px;
font-weight: bold;
color: #1890ff;
}
.stat-label { color: #666; margin-top: 4px; }
.btn {
padding: 8px 16px;
border: none;
border-radius: 4px;
font-size: 14px;
cursor: pointer;
transition: all 0.3s;
}
.btn-primary { background: #1890ff; color: #fff; }
.btn-primary:hover { background: #40a9ff; }
.btn-success { background: #52c41a; color: #fff; }
.btn-danger { background: #ff4d4f; color: #fff; }
.hidden { display: none; }
.page { display: none; }
.page.active { display: block; }
</style>
</head>
<body>
<div class="layout">
<aside class="sidebar">
<div class="logo">📊 行情数据服务</div>
<nav class="nav-menu">
<div class="nav-item active" onclick="showPage('dashboard')">
<span>📈</span> 系统概览
</div>
<div class="nav-item" onclick="showPage('config')">
<span>⚙️</span> 配置管理
</div>
<div class="nav-item" onclick="showPage('adapters')">
<span>🔌</span> 数据源适配
</div>
<div class="nav-item" onclick="showPage('tests')">
<span>🧪</span> 接口测试
</div>
</nav>
</aside>
<main class="main-content">
<div class="header">
<h1 class="page-title">系统概览</h1>
<div>
<button class="btn btn-success" onclick="reloadConfig()">🔄 热加载配置</button>
<button class="btn btn-danger" onclick="restartService()">🔁 重启服务</button>
</div>
</div>
<!-- 系统概览页面 -->
<div id="dashboard" class="page active">
<div class="stats-grid">
<div class="stat-card">
<div class="stat-value" id="stat-status">运行中</div>
<div class="stat-label">运行状态</div>
</div>
<div class="stat-card">
<div class="stat-value" id="stat-uptime">-</div>
<div class="stat-label">运行时长</div>
</div>
<div class="stat-card">
<div class="stat-value">1.0.0</div>
<div class="stat-label">系统版本</div>
</div>
<div class="stat-card">
<div class="stat-value" id="stat-threads">-</div>
<div class="stat-label">线程数量</div>
</div>
</div>
<div class="card">
<div class="card-title">API文档</div>
<p>访问 <a href="/docs">/docs</a> 查看Swagger API文档</p>
<p>访问 <a href="/redoc">/redoc</a> 查看ReDoc API文档</p>
</div>
</div>
<!-- 配置管理页面 -->
<div id="config" class="page">
<div class="card">
<div class="card-title">配置管理</div>
<p>配置管理功能开发中...</p>
</div>
</div>
<!-- 数据源适配页面 -->
<div id="adapters" class="page">
<div class="card">
<div class="card-title">数据源适配</div>
<p>适配器管理功能开发中...</p>
</div>
</div>
<!-- 接口测试页面 -->
<div id="tests" class="page">
<div class="card">
<div class="card-title">接口测试</div>
<p>接口测试功能开发中...</p>
</div>
</div>
</main>
</div>
<script>
function showPage(pageName) {
document.querySelectorAll('.page').forEach(p => p.classList.remove('active'));
document.querySelectorAll('.nav-item').forEach(n => n.classList.remove('active'));
document.getElementById(pageName).classList.add('active');
event.target.classList.add('active');
}
async function reloadConfig() {
try {
const response = await fetch('/v1/admin/system/reload', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
});
const data = await response.json();
alert(data.message || '热加载完成');
} catch (e) {
alert('热加载失败: ' + e.message);
}
}
async function restartService() {
if (confirm('确定要重启服务吗？')) {
try {
const response = await fetch('/v1/admin/system/restart', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
});
const data = await response.json();
alert(data.message || '重启命令已发送');
} catch (e) {
alert('重启失败: ' + e.message);
}
}
}
// 加载系统状态
async function loadSystemStatus() {
try {
const response = await fetch('/v1/admin/system/status');
const data = await response.json();
if (data.data) {
document.getElementById('stat-uptime').textContent = data.data.uptime;
document.getElementById('stat-threads').textContent = data.data.threads;
}
} catch (e) {
console.error('Failed to load system status:', e);
}
}
// 页面加载时获取状态
loadSystemStatus();
setInterval(loadSystemStatus, 30000); // 每30秒刷新
</script>
</body>
</html>
"""
@app.get("/admin", response_class=HTMLResponse)
async def admin_page():
"""管理后台页面"""
return ADMIN_HTML
@app.get("/")
async def root():
"""根路径重定向到管理后台"""
return {"message": "Market Data Service API", "docs": "/docs", "admin": "/admin"}
if __name__ == "__main__":
import uvicorn
# 从环境变量或配置获取端口
port = settings.port or config.server.port
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=port,
reload=config.server.mode == "debug"
)

132
python_market_data_service/app/models/__init__.py
Unescape View File

@ -0,0 +1,132 @@
"""数据模型模块"""
from .types import (
Frequency,
AdjustType,
AssetClass,
SymbolType,
Exchange,
DataSourceStatus,
KLineItem,
KLineData,
KLineQueryRequest,
BatchKLineRequest,
BatchKLineResult,
BatchKLineData,
KLineSubData,
Symbol,
SymbolListRequest,
SymbolListData,
DataSourceInfo,
DataSourceStatusData,
SourceSwitchRequest,
BackfillRequest,
TradingDatesRequest,
TradingDatesData,
FuturesContractsRequest,
FuturesContractInfo,
FuturesContractsData,
Response,
ErrorResponse,
SuccessResponse,
HealthResponse,
)
from .admin_types import (
ConfigType,
ConfigItem,
ConfigSection,
ConfigListRequest,
ConfigListData,
ConfigUpdateRequest,
ConfigUpdateData,
AdapterInfo,
AdapterStatus,
AdapterListData,
AdapterToggleRequest,
AdapterConfigUpdateRequest,
SystemStatusData,
MemoryInfo,
RestartRequest,
ReloadRequest,
ReloadData,
APITestCase,
APITestCategory,
APITestListData,
APITestRequest,
APITestResult,
WSTestCase,
WSTestListData,
WSTestRequest,
WSTestResult,
WSMessage,
TestHistoryRequest,
TestHistoryData,
)
__all__ = [
# 基础类型
"Frequency",
"AdjustType",
"AssetClass",
"SymbolType",
"Exchange",
"DataSourceStatus",
# K线数据
"KLineItem",
"KLineData",
"KLineQueryRequest",
"BatchKLineRequest",
"BatchKLineResult",
"BatchKLineData",
"KLineSubData",
# 标的
"Symbol",
"SymbolListRequest",
"SymbolListData",
# 数据源
"DataSourceInfo",
"DataSourceStatusData",
"SourceSwitchRequest",
"BackfillRequest",
# 交易日历
"TradingDatesRequest",
"TradingDatesData",
# 期货
"FuturesContractsRequest",
"FuturesContractInfo",
"FuturesContractsData",
# 响应
"Response",
"ErrorResponse",
"SuccessResponse",
"HealthResponse",
# 管理后台
"ConfigType",
"ConfigItem",
"ConfigSection",
"ConfigListRequest",
"ConfigListData",
"ConfigUpdateRequest",
"ConfigUpdateData",
"AdapterInfo",
"AdapterStatus",
"AdapterListData",
"AdapterToggleRequest",
"AdapterConfigUpdateRequest",
"SystemStatusData",
"MemoryInfo",
"RestartRequest",
"ReloadRequest",
"ReloadData",
"APITestCase",
"APITestCategory",
"APITestListData",
"APITestRequest",
"APITestResult",
"WSTestCase",
"WSTestListData",
"WSTestRequest",
"WSTestResult",
"WSMessage",
"TestHistoryRequest",
"TestHistoryData",
]

250
python_market_data_service/app/models/admin_types.py
Unescape View File

@ -0,0 +1,250 @@
"""管理后台类型定义 - 对应Go的api/admin_types.go"""
from datetime import datetime
from typing import List, Optional, Dict, Any, Literal
from enum import Enum
from pydantic import BaseModel, Field
# ============================================
# 配置管理类型
# ============================================
class ConfigType(str, Enum):
"""配置类型"""
SERVER = "server"
DATABASE = "database"
REDIS = "redis"
SOURCE = "source"
MONITOR = "monitor"
LOG = "log"
class ConfigItem(BaseModel):
"""配置项"""
key: str = Field(..., description="配置键")
value: Any = Field(..., description="配置值")
type: str = Field(..., description="值类型: string/int/bool/json")
description: str = Field(..., description="配置说明")
editable: bool = Field(default=True, description="是否可编辑")
required: bool = Field(default=True, description="是否必填")
class ConfigSection(BaseModel):
"""配置分组"""
name: str = Field(..., description="分组名称")
type: ConfigType = Field(..., description="分组类型")
description: str = Field(..., description="分组说明")
items: List[ConfigItem] = Field(default_factory=list, description="配置项列表")
class ConfigListRequest(BaseModel):
"""获取配置列表请求"""
type: Optional[ConfigType] = Field(None, description="配置类型筛选")
class ConfigListData(BaseModel):
"""配置列表响应"""
sections: List[ConfigSection] = Field(default_factory=list, description="配置分组列表")
version: str = Field(default="1.0.0", description="配置版本")
updated: datetime = Field(default_factory=datetime.now, description="最后更新时间")
class ConfigUpdateRequest(BaseModel):
"""更新配置请求"""
type: ConfigType = Field(..., description="配置类型")
items: Dict[str, Any] = Field(..., description="更新的配置项")
class ConfigUpdateData(BaseModel):
"""更新配置响应"""
success: bool = Field(..., description="是否成功")
need_restart: bool = Field(default=False, description="是否需要重启")
message: str = Field(..., description="提示信息")
# ============================================
# 适配器管理类型
# ============================================
class AdapterStatus(str, Enum):
"""适配器状态"""
ACTIVE = "active"
STANDBY = "standby"
DISABLED = "disabled"
ERROR = "error"
class AdapterInfo(BaseModel):
"""适配器信息"""
name: str = Field(..., description="适配器名称")
type: str = Field(..., description="适配器类型")
version: str = Field(..., description="版本")
description: str = Field(..., description="描述")
status: AdapterStatus = Field(..., description="状态")
config: Dict[str, str] = Field(default_factory=dict, description="当前配置")
last_error: Optional[str] = Field(None, description="最后错误")
updated_at: datetime = Field(default_factory=datetime.now, description="更新时间")
class AdapterListData(BaseModel):
"""适配器列表响应"""
adapters: List[AdapterInfo] = Field(default_factory=list, description="适配器列表")
class AdapterToggleRequest(BaseModel):
"""启用/禁用适配器请求"""
name: str = Field(..., description="适配器名称")
enable: bool = Field(..., description="是否启用")
class AdapterConfigUpdateRequest(BaseModel):
"""更新适配器配置请求"""
name: str = Field(..., description="适配器名称")
config: Dict[str, str] = Field(..., description="配置")
# ============================================
# 系统管理类型
# ============================================
class MemoryInfo(BaseModel):
"""内存信息"""
alloc: int = Field(..., description="已分配内存")
total_alloc: int = Field(..., description="累计分配")
sys: int = Field(..., description="系统内存")
num_gc: int = Field(..., description="GC次数")
class SystemStatusData(BaseModel):
"""系统状态数据"""
status: str = Field(..., description="系统状态")
version: str = Field(..., description="系统版本")
start_time: datetime = Field(..., description="启动时间")
uptime: str = Field(..., description="运行时长")
python_version: str = Field(..., description="Python版本")
memory: MemoryInfo = Field(..., description="内存使用")
threads: int = Field(..., description="线程数量")
class RestartRequest(BaseModel):
"""重启服务请求"""
force: bool = Field(default=False, description="是否强制重启")
class ReloadRequest(BaseModel):
"""热加载配置请求"""
config_type: Optional[ConfigType] = Field(None, description="指定配置类型")
class ReloadData(BaseModel):
"""热加载响应"""
success: bool = Field(..., description="是否成功")
message: str = Field(..., description="提示信息")
# ============================================
# 接口测试类型
# ============================================
class APITestCase(BaseModel):
"""接口测试用例"""
id: str = Field(..., description="用例ID")
name: str = Field(..., description="用例名称")
method: str = Field(..., description="HTTP方法")
path: str = Field(..., description="请求路径")
description: str = Field(..., description="描述")
params: Dict[str, str] = Field(default_factory=dict, description="默认参数")
body: Optional[Any] = Field(None, description="请求体")
class APITestCategory(BaseModel):
"""测试分类"""
name: str = Field(..., description="分类名称")
items: List[APITestCase] = Field(default_factory=list, description="测试用例")
class APITestListData(BaseModel):
"""接口测试列表响应"""
categories: List[APITestCategory] = Field(default_factory=list, description="分类列表")
base_url: str = Field(default="", description="基础URL")
class APITestRequest(BaseModel):
"""执行接口测试请求"""
id: str = Field(..., description="用例ID")
params: Dict[str, str] = Field(default_factory=dict, description="自定义参数")
body: Optional[Any] = Field(None, description="自定义请求体")
class APITestResult(BaseModel):
"""接口测试结果"""
id: int = Field(..., description="测试ID")
case_id: str = Field(..., description="用例ID")
name: str = Field(..., description="用例名称")
success: bool = Field(..., description="是否成功")
status_code: int = Field(0, description="HTTP状态码")
latency: int = Field(..., description="延迟(ms)")
request: Any = Field(None, description="请求信息")
response: Any = Field(None, description="响应信息")
error: Optional[str] = Field(None, description="错误信息")
timestamp: datetime = Field(default_factory=datetime.now, description="测试时间")
# ============================================
# WebSocket测试类型
# ============================================
class WSTestCase(BaseModel):
"""WebSocket测试用例"""
id: str = Field(..., description="用例ID")
name: str = Field(..., description="用例名称")
description: str = Field(..., description="描述")
action: str = Field(..., description="动作类型")
symbols: List[str] = Field(default_factory=list, description="订阅标的")
class WSTestListData(BaseModel):
"""WebSocket测试列表响应"""
cases: List[WSTestCase] = Field(default_factory=list, description="测试用例")
ws_url: str = Field(default="", description="WebSocket地址")
class WSTestRequest(BaseModel):
"""WebSocket测试请求"""
id: str = Field(..., description="用例ID")
symbols: List[str] = Field(default_factory=list, description="自定义标的")
class WSMessage(BaseModel):
"""WebSocket消息"""
type: str = Field(..., description="消息类型")
data: Any = Field(None, description="消息内容")
timestamp: datetime = Field(default_factory=datetime.now, description="时间")
class WSTestResult(BaseModel):
"""WebSocket测试结果"""
id: str = Field(..., description="测试ID")
case_id: str = Field(..., description="用例ID")
success: bool = Field(..., description="是否成功")
latency: int = Field(..., description="连接延迟(ms)")
messages: List[WSMessage] = Field(default_factory=list, description="收到的消息")
error: Optional[str] = Field(None, description="错误信息")
timestamp: datetime = Field(default_factory=datetime.now, description="测试时间")
# ============================================
# 测试历史记录类型
# ============================================
class TestHistoryRequest(BaseModel):
"""获取测试历史请求"""
type: Optional[str] = Field(None, description="测试类型: api/ws")
limit: int = Field(default=20, ge=1, le=100, description="数量限制")
class TestHistoryData(BaseModel):
"""测试历史数据"""
api_tests: List[APITestResult] = Field(default_factory=list, description="API测试历史")
ws_tests: List[WSTestResult] = Field(default_factory=list, description="WebSocket测试历史")

306
python_market_data_service/app/models/types.py
Unescape View File

@ -0,0 +1,306 @@
"""基础类型定义 - 对应Go的api/types.go"""
from datetime import datetime
from typing import List, Optional, Literal, Any
from enum import Enum
from pydantic import BaseModel, Field
# ============================================
# 基础枚举定义
# ============================================
class Frequency(str, Enum):
"""K线周期"""
FREQ_1M = "1m"
FREQ_5M = "5m"
FREQ_15M = "15m"
FREQ_30M = "30m"
FREQ_60M = "60m"
FREQ_1D = "1d"
FREQ_1W = "1w"
FREQ_1MONTH = "1month"
class AdjustType(str, Enum):
"""复权类型"""
NONE = "" # 不复权
QFQ = "qfq" # 前复权
HFQ = "hfq" # 后复权
class AssetClass(str, Enum):
"""资产类别"""
STOCK = "stock"
FUTURES = "futures"
ALL = "all"
class SymbolType(str, Enum):
"""标的类型"""
STOCK = "stock"
FUTURES = "futures"
INDEX = "index"
ETF = "etf"
CONTINUOUS = "continuous"
class Exchange(str, Enum):
"""交易所"""
# 股票交易所
SZ = "SZ" # 深交所
SH = "SH" # 上交所
BJ = "BJ" # 北交所
# 期货交易所
CFFEX = "CFFEX" # 中金所
SHFE = "SHFE" # 上期所
DCE = "DCE" # 大商所
CZCE = "CZCE" # 郑商所
INE = "INE" # 上期能源
GFEX = "GFEX" # 广期所
class DataSourceStatus(str, Enum):
"""数据源状态"""
HEALTHY = "healthy"
DEGRADED = "degraded"
DOWN = "down"
# ============================================
# K线数据模型
# ============================================
class KLineItem(BaseModel):
"""单条K线数据"""
time: datetime = Field(..., description="时间戳")
open: float = Field(..., description="开盘价")
high: float = Field(..., description="最高价")
low: float = Field(..., description="最低价")
close: float = Field(..., description="收盘价")
volume: int = Field(..., description="成交量")
amount: float = Field(..., description="成交额")
# 期货特有字段
open_interest: Optional[int] = Field(None, description="持仓量")
settlement: Optional[float] = Field(None, description="结算价")
# 股票特有字段
adj_factor: Optional[float] = Field(None, description="复权系数")
class Config:
json_encoders = {
datetime: lambda v: v.isoformat()
}
class KLineData(BaseModel):
"""K线响应数据"""
symbol: str = Field(..., description="标的代码")
name: Optional[str] = Field(None, description="标的名称")
freq: Frequency = Field(..., description="周期")
adjust: AdjustType = Field(default=AdjustType.NONE, description="复权类型")
count: int = Field(..., description="数据条数")
items: List[KLineItem] = Field(default_factory=list, description="K线数据")
class KLineQueryRequest(BaseModel):
"""K线查询请求"""
symbol: str = Field(..., description="标的代码")
start: str = Field(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8)
end: str = Field(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8)
freq: Frequency = Field(default=Frequency.FREQ_1D, description="周期")
adjust: AdjustType = Field(default=AdjustType.NONE, description="复权类型")
class BatchKLineRequest(BaseModel):
"""批量K线查询请求"""
symbols: List[str] = Field(..., description="标的列表，最多100只", max_length=100)
start: str = Field(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8)
end: str = Field(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8)
freq: Frequency = Field(default=Frequency.FREQ_1D, description="周期")
adjust: AdjustType = Field(default=AdjustType.NONE, description="复权类型")
class KLineSubData(BaseModel):
"""批量查询中的数据部分"""
count: int = Field(..., description="数据条数")
items: List[KLineItem] = Field(default_factory=list, description="K线数据")
class BatchKLineResult(BaseModel):
"""批量查询单条结果"""
symbol: str = Field(..., description="标的代码")
success: bool = Field(..., description="是否成功")
error: Optional[str] = Field(None, description="错误信息")
data: Optional[KLineSubData] = Field(None, description="数据")
class BatchKLineData(BaseModel):
"""批量K线响应数据"""
results: List[BatchKLineResult] = Field(default_factory=list, description="结果列表")
# ============================================
# 标的模型
# ============================================
class Symbol(BaseModel):
"""标的详细信息"""
symbol_id: str = Field(..., description="统一标的编码")
symbol_type: SymbolType = Field(..., description="标的类型")
exchange: Exchange = Field(..., description="交易所")
name: str = Field(..., description="名称")
name_en: Optional[str] = Field(None, description="英文名称")
list_date: Optional[datetime] = Field(None, description="上市日期")
delist_date: Optional[datetime] = Field(None, description="退市日期")
# 股票特有
industry: Optional[str] = Field(None, description="行业分类")
# 期货特有
underlying: Optional[str] = Field(None, description="标的资产")
contract_month: Optional[str] = Field(None, description="合约月份")
continuous_type: Optional[str] = Field(None, description="连续合约类型")
status: str = Field(..., description="active/suspended/delisted")
class SymbolListRequest(BaseModel):
"""标的列表请求"""
exchange: Optional[Exchange] = Field(None, description="交易所筛选")
keyword: Optional[str] = Field(None, description="关键词搜索")
underlying: Optional[str] = Field(None, description="品种筛选（期货）")
page: int = Field(default=1, ge=1, description="页码")
size: int = Field(default=20, ge=1, le=100, description="每页数量")
class SymbolListData(BaseModel):
"""标的列表响应数据"""
total: int = Field(..., description="总数")
page: int = Field(..., description="当前页")
size: int = Field(..., description="每页数量")
items: List[Symbol] = Field(default_factory=list, description="标的列表")
# ============================================
# 管理接口模型
# ============================================
class DataSourceInfo(BaseModel):
"""数据源信息"""
active_source: str = Field(..., description="当前激活源")
standby_sources: List[str] = Field(default_factory=list, description="待命源列表")
last_tick_ts: Optional[datetime] = Field(None, description="最后Tick时间")
status: DataSourceStatus = Field(..., description="数据源状态")
class DataSourceStatusData(BaseModel):
"""数据源状态响应"""
stock: DataSourceInfo = Field(default_factory=lambda: DataSourceInfo(
active_source="tushare",
status=DataSourceStatus.HEALTHY
))
futures: DataSourceInfo = Field(default_factory=lambda: DataSourceInfo(
active_source="tushare",
status=DataSourceStatus.HEALTHY
))
class SourceSwitchRequest(BaseModel):
"""数据源切换请求"""
asset_class: AssetClass = Field(..., description="资产类别")
source: str = Field(..., description="目标数据源")
sync_backfill: bool = Field(default=False, description="是否同步补录")
start_date: Optional[str] = Field(None, description="补录开始日期 YYYYMMDD")
class BackfillRequest(BaseModel):
"""历史数据补录请求"""
asset_class: AssetClass = Field(..., description="资产类别")
symbols: List[str] = Field(..., description="标的列表，空数组表示全部")
start: str = Field(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8)
end: str = Field(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8)
freqs: List[Frequency] = Field(..., description="需要补录的周期")
source: Optional[str] = Field(None, description="指定数据源")
# ============================================
# 交易日历模型
# ============================================
class TradingDatesRequest(BaseModel):
"""可交易日期查询请求"""
start: str = Field(..., description="开始日期 YYYYMMDD", min_length=8, max_length=8)
end: str = Field(..., description="结束日期 YYYYMMDD", min_length=8, max_length=8)
class TradingDatesData(BaseModel):
"""可交易日期响应数据"""
start: str = Field(..., description="查询开始日期")
end: str = Field(..., description="查询结束日期")
total_days: int = Field(..., description="总天数")
trading_days: int = Field(..., description="交易日数量")
trading_dates: List[str] = Field(default_factory=list, description="交易日列表")
# ============================================
# 期货合约模型
# ============================================
class FuturesContractsRequest(BaseModel):
"""获取期货合约请求"""
underlying: str = Field(..., description="品种代码，如 Cu, RB, M")
exchange: Optional[str] = Field(None, description="交易所筛选")
class FuturesContractInfo(BaseModel):
"""期货合约信息"""
symbol_id: str = Field(..., description="合约代码")
symbol_type: str = Field(default="futures", description="类型")
exchange: Exchange = Field(..., description="交易所")
name: str = Field(..., description="合约名称")
underlying: str = Field(..., description="品种代码")
contract_month: str = Field(..., description="合约月份")
list_date: Optional[datetime] = Field(None, description="上市日期")
delist_date: Optional[datetime] = Field(None, description="退市日期")
status: str = Field(..., description="active/expired")
class FuturesContractsData(BaseModel):
"""期货合约列表响应"""
underlying: str = Field(..., description="品种代码")
count: int = Field(..., description="合约数量")
items: List[FuturesContractInfo] = Field(default_factory=list, description="合约列表")
# ============================================
# 响应模型
# ============================================
class Response(BaseModel):
"""通用响应结构"""
code: int = Field(default=0, description="0表示成功，非0表示错误")
message: str = Field(default="success", description="提示信息")
data: Optional[Any] = Field(None, description="响应数据")
class ErrorResponse(BaseModel):
"""错误响应"""
code: int = Field(..., description="错误码")
message: str = Field(..., description="错误信息")
detail: Optional[str] = Field(None, description="详细错误")
class SuccessResponse(BaseModel):
"""成功响应"""
code: int = Field(default=0)
message: str = Field(default="success")
data: Optional[Any] = Field(None)
class HealthResponse(BaseModel):
"""健康检查响应"""
status: str = Field(..., description="状态")
timestamp: datetime = Field(default_factory=datetime.now, description="时间戳")

4
python_market_data_service/app/monitor/__init__.py
Unescape View File

@ -0,0 +1,4 @@
"""数据质量监控模块"""
from .monitor import DataQualityMonitor, AlertSender, LogAlertSender
__all__ = ["DataQualityMonitor", "AlertSender", "LogAlertSender"]

255
python_market_data_service/app/monitor/monitor.py
Unescape View File

@ -0,0 +1,255 @@
"""数据质量监控 - 对应Go的internal/monitor/monitor.go"""
import asyncio
from abc import ABC, abstractmethod
from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import List, Optional
from sqlalchemy.orm import Session
from sqlalchemy import text
from app.repositories import StockRepository, FuturesRepository
from app.models import Frequency
from app.core.logger import info, error
@dataclass
class CheckResult:
"""检查结果"""
symbol: str
freq: str
check_date: str
check_type: str
status: str # pass/fail
expect_count: int
actual_count: int
detail: str
@dataclass
class QualityReport:
"""数据质量报告"""
asset_type: str
check_date: str
total_checks: int
pass_count: int
fail_count: int
pass_rate: float
class AlertSender(ABC):
"""告警发送接口"""
@abstractmethod
def send_alert(self, title: str, content: str) -> bool:
"""发送告警"""
pass
class LogAlertSender(AlertSender):
"""日志告警发送器"""
def send_alert(self, title: str, content: str) -> bool:
info(f"[ALERT] {title}: {content}")
return True
class DataQualityMonitor:
"""数据质量监控"""
def __init__(
self,
db: Session,
stock_repo: StockRepository,
futures_repo: FuturesRepository,
sender: Optional[AlertSender] = None
):
self.db = db
self.stock_repo = stock_repo
self.futures_repo = futures_repo
self.sender = sender or LogAlertSender()
async def daily_check(self, check_date: str):
"""每日数据质量检查"""
info(f"Starting daily data quality check for {check_date}")
# 检查股票数据
try:
await self._check_stock_data(check_date)
except Exception as e:
error(f"Stock data check failed: {e}")
# 检查期货数据
try:
await self._check_futures_data(check_date)
except Exception as e:
error(f"Futures data check failed: {e}")
info("Daily data quality check completed")
async def _check_stock_data(self, check_date: str):
"""检查股票数据质量"""
# 获取所有活跃股票
from app.models import SymbolListRequest
symbols, _, _ = self.stock_repo.list_symbols(SymbolListRequest(page=1, size=10000))
# 检查1分钟线完整性（股票应有240条）
for symbol in symbols[:100]: # 限制检查数量
result = self._check_kline_completeness(
"stock", symbol.symbol_id, "1m", check_date, 240
)
self._save_check_result("stock", result)
async def _check_futures_data(self, check_date: str):
"""检查期货数据质量"""
from app.models import SymbolListRequest
symbols, _, _ = self.futures_repo.list_symbols(SymbolListRequest(page=1, size=10000))
# 检查1分钟线完整性
for symbol in symbols[:100]:
result = self._check_kline_completeness(
"futures", symbol.symbol_id, "1m", check_date, 240
)
self._save_check_result("futures", result)
def _check_kline_completeness(
self,
asset_type: str,
symbol: str,
freq: str,
check_date: str,
expect_count: int
) -> CheckResult:
"""检查K线完整性"""
result = CheckResult(
symbol=symbol,
freq=freq,
check_date=check_date,
check_type="missing",
status="pass",
expect_count=expect_count,
actual_count=0,
detail=""
)
try:
# 解析日期
start = datetime.strptime(check_date, "%Y%m%d")
end = start + timedelta(days=1) - timedelta(seconds=1)
# 查询数据
if asset_type == "stock":
items = self.stock_repo.get_klines(
symbol, Frequency(freq), start, end
)
else:
items = self.futures_repo.get_klines(
symbol, Frequency(freq), start, end
)
actual_count = len(items)
result.actual_count = actual_count
# 判断缺失情况
if actual_count < expect_count:
result.status = "fail"
result.detail = f"Data missing: expected {expect_count}, actual {actual_count}"
# 发送告警
if self.sender:
self.sender.send_alert(
f"[{asset_type}] Data Missing Alert",
f"Symbol: {symbol}, Date: {check_date}, Expected: {expect_count}, Actual: {actual_count}"
)
except Exception as e:
result.status = "fail"
result.detail = f"Error querying data: {e}"
return result
def _save_check_result(self, asset_type: str, result: CheckResult):
"""保存检查结果"""
try:
# 根据资产类型选择schema
schema = "stock" if asset_type == "stock" else "futures"
query = text(f"""
INSERT INTO {schema}.data_quality_checks
(check_date, symbol_id, freq, check_type, status, expect_count, actual_count, detail)
VALUES (:check_date, :symbol_id, :freq, :check_type, :status, :expect_count, :actual_count, :detail)
ON CONFLICT (check_date, symbol_id, freq, check_type) DO UPDATE SET
status = EXCLUDED.status,
expect_count = EXCLUDED.expect_count,
actual_count = EXCLUDED.actual_count,
detail = EXCLUDED.detail,
created_at = NOW()
""")
self.db.execute(query, {
"check_date": result.check_date,
"symbol_id": result.symbol,
"freq": result.freq,
"check_type": result.check_type,
"status": result.status,
"expect_count": result.expect_count,
"actual_count": result.actual_count,
"detail": result.detail
})
self.db.commit()
except Exception as e:
error(f"Failed to save check result: {e}")
def get_quality_report(self, asset_type: str, check_date: str) -> QualityReport:
"""获取数据质量报告"""
schema = "stock" if asset_type == "stock" else "futures"
query = text(f"""
SELECT
COUNT(*) as total_checks,
SUM(CASE WHEN status = 'pass' THEN 1 ELSE 0 END) as pass_count,
SUM(CASE WHEN status = 'fail' THEN 1 ELSE 0 END) as fail_count
FROM {schema}.data_quality_checks
WHERE check_date = :check_date
""")
result = self.db.execute(query, {"check_date": check_date}).fetchone()
total = result.total_checks or 0
pass_count = result.pass_count or 0
fail_count = result.fail_count or 0
pass_rate = (pass_count / total * 100) if total > 0 else 0
return QualityReport(
asset_type=asset_type,
check_date=check_date,
total_checks=total,
pass_count=pass_count,
fail_count=fail_count,
pass_rate=pass_rate
)
async def start_daily_check_cron(self):
"""启动每日检查定时任务"""
while True:
try:
# 计算到下一个盘后的时间（假设15:35）
now = datetime.now()
next_check = now.replace(hour=15, minute=35, second=0, microsecond=0)
if next_check <= now:
next_check = next_check + timedelta(days=1)
wait_seconds = (next_check - now).total_seconds()
info(f"Next daily check scheduled at {next_check}, waiting {wait_seconds} seconds")
await asyncio.sleep(wait_seconds)
# 执行检查
check_date = now.strftime("%Y%m%d")
await self.daily_check(check_date)
except Exception as e:
error(f"Daily check cron error: {e}")
await asyncio.sleep(3600) # 出错后等待1小时重试

13
python_market_data_service/app/repositories/__init__.py
Unescape View File

@ -0,0 +1,13 @@
"""数据访问层模块"""
from .database import get_db, SessionLocal, engine, Base
from .stock_repository import StockRepository
from .futures_repository import FuturesRepository
__all__ = [
"get_db",
"SessionLocal",
"engine",
"Base",
"StockRepository",
"FuturesRepository",
]

38
python_market_data_service/app/repositories/database.py
Unescape View File

@ -0,0 +1,38 @@
"""数据库连接管理"""
from typing import Generator
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session
from app.core.config import get_config
config = get_config()
# 创建数据库引擎
engine = create_engine(
config.database.database_url,
pool_pre_ping=True, # 自动检测断开的连接
pool_size=10,
max_overflow=20,
)
# 会话工厂
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# 声明基类
Base = declarative_base()
def get_db() -> Generator[Session, None, None]:
"""获取数据库会话（用于FastAPI依赖注入）"""
db = SessionLocal()
try:
yield db
finally:
db.close()
def init_db():
"""初始化数据库（创建所有表）"""
Base.metadata.create_all(bind=engine)

268
python_market_data_service/app/repositories/futures_repository.py
Unescape View File

@ -0,0 +1,268 @@
"""期货数据仓库"""
from datetime import datetime
from typing import List, Tuple, Optional
from sqlalchemy.orm import Session
from sqlalchemy import func, or_
from app.models import (
KLineItem, Symbol, SymbolListRequest, SymbolListData,
TradingDatesData, TradeCalData, Frequency,
FuturesContractsData, FuturesContractInfo
)
from app.repositories.models import (
FuturesSymbol, FuturesKLine1M, FuturesKLine1D,
FuturesTradingCalendar
)
class FuturesRepository:
"""期货数据仓库"""
def __init__(self, db: Session):
self.db = db
def get_klines(
self,
symbol: str,
freq: Frequency,
start: datetime,
end: datetime
) -> List[KLineItem]:
"""获取K线数据"""
kline_model = self._get_kline_model(freq)
query = self.db.query(kline_model).filter(
kline_model.symbol_id == symbol,
kline_model.ts >= start,
kline_model.ts <= end
).order_by(kline_model.ts.asc())
results = query.all()
items = []
for r in results:
item = KLineItem(
time=r.ts,
open=float(r.open),
high=float(r.high),
low=float(r.low),
close=float(r.close),
volume=r.volume,
amount=float(r.amount),
open_interest=r.open_interest
)
items.append(item)
return items
def _get_kline_model(self, freq: Frequency):
"""根据周期获取K线模型"""
mapping = {
Frequency.FREQ_1M: FuturesKLine1M,
Frequency.FREQ_1D: FuturesKLine1D,
}
return mapping.get(freq, FuturesKLine1D)
def save_klines(
self,
freq: Frequency,
symbol: str,
items: List[KLineItem]
) -> None:
"""保存K线数据"""
if not items:
return
kline_model = self._get_kline_model(freq)
for item in items:
existing = self.db.query(kline_model).filter(
kline_model.symbol_id == symbol,
kline_model.ts == item.time
).first()
if existing:
existing.open = item.open
existing.high = item.high
existing.low = item.low
existing.close = item.close
existing.volume = item.volume
existing.amount = item.amount
existing.open_interest = item.open_interest
else:
new_record = kline_model(
symbol_id=symbol,
ts=item.time,
open=item.open,
high=item.high,
low=item.low,
close=item.close,
volume=item.volume,
amount=item.amount,
open_interest=item.open_interest
)
self.db.add(new_record)
self.db.commit()
def list_symbols(
self,
req: SymbolListRequest
) -> Tuple[List[Symbol], int]:
"""查询标的列表"""
query = self.db.query(FuturesSymbol)
# 筛选条件
if req.exchange:
query = query.filter(FuturesSymbol.exchange == req.exchange.value)
if req.underlying:
query = query.filter(FuturesSymbol.underlying == req.underlying)
if req.keyword:
keyword = f"%{req.keyword}%"
query = query.filter(
or_(
FuturesSymbol.symbol_id.ilike(keyword),
FuturesSymbol.name.ilike(keyword)
)
)
# 查询总数
total = query.count()
# 分页查询
results = query.order_by(FuturesSymbol.symbol_id).offset(
(req.page - 1) * req.size
).limit(req.size).all()
symbols = []
for r in results:
s = Symbol(
symbol_id=r.symbol_id,
symbol_type=r.symbol_type,
exchange=r.exchange,
name=r.name,
underlying=r.underlying,
contract_month=r.contract_month,
list_date=r.list_date,
delist_date=r.delist_date,
status=r.status
)
symbols.append(s)
return symbols, total
def get_trading_dates(self, start: str, end: str) -> TradingDatesData:
"""获取交易日历"""
results = self.db.query(FuturesTradingCalendar).filter(
FuturesTradingCalendar.trade_date >= start,
FuturesTradingCalendar.trade_date <= end,
FuturesTradingCalendar.is_trading_day == True
).order_by(FuturesTradingCalendar.trade_date.asc()).all()
dates = [r.trade_date for r in results]
# 计算总天数
start_date = datetime.strptime(start, "%Y%m%d")
end_date = datetime.strptime(end, "%Y%m%d")
total_days = (end_date - start_date).days + 1
return TradingDatesData(
start=start,
end=end,
total_days=total_days,
trading_days=len(dates),
trading_dates=dates
)
def get_contracts_by_underlying(
self,
underlying: str,
exchange: Optional[str] = None
) -> FuturesContractsData:
"""根据品种获取合约"""
query = self.db.query(FuturesSymbol).filter(
FuturesSymbol.underlying == underlying,
FuturesSymbol.status == "active"
)
if exchange:
query = query.filter(FuturesSymbol.exchange == exchange)
results = query.order_by(FuturesSymbol.contract_month.asc()).all()
contracts = []
for r in results:
c = FuturesContractInfo(
symbol_id=r.symbol_id,
exchange=r.exchange,
name=r.name,
underlying=r.underlying,
contract_month=r.contract_month,
list_date=r.list_date,
delist_date=r.delist_date,
status=r.status
)
contracts.append(c)
return FuturesContractsData(
underlying=underlying,
count=len(contracts),
items=contracts
)
def save_symbols(self, symbols: List[Symbol]) -> None:
"""保存标的列表"""
for s in symbols:
existing = self.db.query(FuturesSymbol).filter(
FuturesSymbol.symbol_id == s.symbol_id
).first()
if existing:
existing.name = s.name
existing.underlying = s.underlying
existing.contract_month = s.contract_month
existing.list_date = s.list_date
existing.delist_date = s.delist_date
existing.status = s.status
else:
new_symbol = FuturesSymbol(
symbol_id=s.symbol_id,
symbol_type=s.symbol_type.value if s.symbol_type else "futures",
exchange=s.exchange.value if s.exchange else "",
name=s.name,
underlying=s.underlying or "",
contract_month=s.contract_month or "",
list_date=s.list_date,
delist_date=s.delist_date,
status=s.status
)
self.db.add(new_symbol)
self.db.commit()
def save_trading_calendar(self, dates: List[TradeCalData]) -> None:
"""保存交易日历"""
for d in dates:
date_str = d.date.strftime("%Y%m%d")
existing = self.db.query(FuturesTradingCalendar).filter(
FuturesTradingCalendar.trade_date == date_str
).first()
if existing:
existing.is_trading_day = d.is_trading_day
existing.has_night_session = d.has_night_session
existing.week_day = d.date.weekday() + 1
else:
new_cal = FuturesTradingCalendar(
trade_date=date_str,
is_trading_day=d.is_trading_day,
has_night_session=d.has_night_session,
week_day=d.date.weekday() + 1
)
self.db.add(new_cal)
self.db.commit()

214
python_market_data_service/app/repositories/models.py
Unescape View File

@ -0,0 +1,214 @@
"""数据库模型定义"""
from datetime import datetime
from typing import Optional
from sqlalchemy import (
Column, Integer, String, Float, DateTime,
Boolean, Numeric, BigInteger, Index
)
from sqlalchemy.dialects.postgresql import ARRAY
from app.repositories.database import Base
# ============================================
# 股票相关表
# ============================================
class StockSymbol(Base):
"""股票标的表"""
__tablename__ = "symbols"
__table_args__ = {"schema": "stock"}
symbol_id = Column(String(20), primary_key=True, index=True, comment="标的代码")
symbol_type = Column(String(20), nullable=False, comment="标的类型")
exchange = Column(String(10), nullable=False, index=True, comment="交易所")
name = Column(String(100), nullable=False, comment="名称")
name_en = Column(String(100), nullable=True, comment="英文名称")
list_date = Column(DateTime, nullable=True, comment="上市日期")
delist_date = Column(DateTime, nullable=True, comment="退市日期")
industry = Column(String(50), nullable=True, comment="行业分类")
status = Column(String(20), nullable=False, default="active", comment="状态")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
updated_at = Column(DateTime, default=datetime.now, onupdate=datetime.now, comment="更新时间")
class StockTradingCalendar(Base):
"""股票交易日历表"""
__tablename__ = "trading_calendar"
__table_args__ = {"schema": "stock"}
trade_date = Column(String(8), primary_key=True, comment="交易日期")
is_trading_day = Column(Boolean, nullable=False, comment="是否交易日")
week_day = Column(Integer, nullable=True, comment="星期几")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
updated_at = Column(DateTime, default=datetime.now, onupdate=datetime.now, comment="更新时间")
class StockKLine1M(Base):
"""股票1分钟K线"""
__tablename__ = "klines_1m"
__table_args__ = (
Index("idx_stock_1m_symbol_ts", "symbol_id", "ts"),
{"schema": "stock"}
)
id = Column(BigInteger, primary_key=True, autoincrement=True)
symbol_id = Column(String(20), nullable=False, index=True, comment="标的代码")
ts = Column(DateTime, nullable=False, comment="时间戳")
open = Column(Numeric(18, 4), nullable=False, comment="开盘价")
high = Column(Numeric(18, 4), nullable=False, comment="最高价")
low = Column(Numeric(18, 4), nullable=False, comment="最低价")
close = Column(Numeric(18, 4), nullable=False, comment="收盘价")
volume = Column(BigInteger, nullable=False, comment="成交量")
amount = Column(Numeric(20, 4), nullable=False, comment="成交额")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
class StockKLine5M(Base):
"""股票5分钟K线"""
__tablename__ = "klines_5m"
__table_args__ = (
Index("idx_stock_5m_symbol_ts", "symbol_id", "ts"),
{"schema": "stock"}
)
id = Column(BigInteger, primary_key=True, autoincrement=True)
symbol_id = Column(String(20), nullable=False, index=True, comment="标的代码")
ts = Column(DateTime, nullable=False, comment="时间戳")
open = Column(Numeric(18, 4), nullable=False, comment="开盘价")
high = Column(Numeric(18, 4), nullable=False, comment="最高价")
low = Column(Numeric(18, 4), nullable=False, comment="最低价")
close = Column(Numeric(18, 4), nullable=False, comment="收盘价")
volume = Column(BigInteger, nullable=False, comment="成交量")
amount = Column(Numeric(20, 4), nullable=False, comment="成交额")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
class StockKLine1D(Base):
"""股票日线K线"""
__tablename__ = "klines_1d"
__table_args__ = (
Index("idx_stock_1d_symbol_ts", "symbol_id", "ts"),
{"schema": "stock"}
)
id = Column(BigInteger, primary_key=True, autoincrement=True)
symbol_id = Column(String(20), nullable=False, index=True, comment="标的代码")
ts = Column(DateTime, nullable=False, comment="时间戳")
open = Column(Numeric(18, 4), nullable=False, comment="开盘价")
high = Column(Numeric(18, 4), nullable=False, comment="最高价")
low = Column(Numeric(18, 4), nullable=False, comment="最低价")
close = Column(Numeric(18, 4), nullable=False, comment="收盘价")
volume = Column(BigInteger, nullable=False, comment="成交量")
amount = Column(Numeric(20, 4), nullable=False, comment="成交额")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
# ============================================
# 期货相关表
# ============================================
class FuturesSymbol(Base):
"""期货合约表"""
__tablename__ = "symbols"
__table_args__ = {"schema": "futures"}
symbol_id = Column(String(20), primary_key=True, index=True, comment="合约代码")
symbol_type = Column(String(20), nullable=False, comment="标的类型")
exchange = Column(String(10), nullable=False, index=True, comment="交易所")
name = Column(String(100), nullable=False, comment="名称")
underlying = Column(String(10), nullable=False, index=True, comment="品种代码")
contract_month = Column(String(6), nullable=False, comment="合约月份")
list_date = Column(DateTime, nullable=True, comment="上市日期")
delist_date = Column(DateTime, nullable=True, comment="退市日期")
status = Column(String(20), nullable=False, default="active", comment="状态")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
updated_at = Column(DateTime, default=datetime.now, onupdate=datetime.now, comment="更新时间")
class FuturesTradingCalendar(Base):
"""期货交易日历表"""
__tablename__ = "trading_calendar"
__table_args__ = {"schema": "futures"}
trade_date = Column(String(8), primary_key=True, comment="交易日期")
is_trading_day = Column(Boolean, nullable=False, comment="是否交易日")
has_night_session = Column(Boolean, default=False, comment="是否有夜盘")
week_day = Column(Integer, nullable=True, comment="星期几")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
updated_at = Column(DateTime, default=datetime.now, onupdate=datetime.now, comment="更新时间")
class FuturesKLine1M(Base):
"""期货1分钟K线"""
__tablename__ = "klines_1m"
__table_args__ = (
Index("idx_futures_1m_symbol_ts", "symbol_id", "ts"),
{"schema": "futures"}
)
id = Column(BigInteger, primary_key=True, autoincrement=True)
symbol_id = Column(String(20), nullable=False, index=True, comment="合约代码")
ts = Column(DateTime, nullable=False, comment="时间戳")
open = Column(Numeric(18, 4), nullable=False, comment="开盘价")
high = Column(Numeric(18, 4), nullable=False, comment="最高价")
low = Column(Numeric(18, 4), nullable=False, comment="最低价")
close = Column(Numeric(18, 4), nullable=False, comment="收盘价")
volume = Column(BigInteger, nullable=False, comment="成交量")
amount = Column(Numeric(20, 4), nullable=False, comment="成交额")
open_interest = Column(BigInteger, nullable=True, comment="持仓量")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
class FuturesKLine1D(Base):
"""期货日线K线"""
__tablename__ = "klines_1d"
__table_args__ = (
Index("idx_futures_1d_symbol_ts", "symbol_id", "ts"),
{"schema": "futures"}
)
id = Column(BigInteger, primary_key=True, autoincrement=True)
symbol_id = Column(String(20), nullable=False, index=True, comment="合约代码")
ts = Column(DateTime, nullable=False, comment="时间戳")
open = Column(Numeric(18, 4), nullable=False, comment="开盘价")
high = Column(Numeric(18, 4), nullable=False, comment="最高价")
low = Column(Numeric(18, 4), nullable=False, comment="最低价")
close = Column(Numeric(18, 4), nullable=False, comment="收盘价")
volume = Column(BigInteger, nullable=False, comment="成交量")
amount = Column(Numeric(20, 4), nullable=False, comment="成交额")
open_interest = Column(BigInteger, nullable=True, comment="持仓量")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")
# ============================================
# 公共表
# ============================================
class DataSourceConfig(Base):
"""数据源配置表"""
__tablename__ = "data_source_config"
__table_args__ = {"schema": "public"}
asset_class = Column(String(20), primary_key=True, comment="资产类别")
active_source = Column(String(50), nullable=False, comment="当前激活源")
standby_sources = Column(ARRAY(String), nullable=True, comment="待命源列表")
updated_at = Column(DateTime, default=datetime.now, onupdate=datetime.now, comment="更新时间")
class DataQualityCheck(Base):
"""数据质量检查表"""
__tablename__ = "data_quality_checks"
__table_args__ = {"schema": "stock"} # 也可以是futures
id = Column(BigInteger, primary_key=True, autoincrement=True)
check_date = Column(String(8), nullable=False, index=True, comment="检查日期")
symbol_id = Column(String(20), nullable=False, index=True, comment="标的代码")
freq = Column(String(10), nullable=False, comment="周期")
check_type = Column(String(20), nullable=False, comment="检查类型")
status = Column(String(10), nullable=False, comment="状态 pass/fail")
expect_count = Column(Integer, nullable=True, comment="期望数量")
actual_count = Column(Integer, nullable=True, comment="实际数量")
detail = Column(String(500), nullable=True, comment="详情")
created_at = Column(DateTime, default=datetime.now, comment="创建时间")

222
python_market_data_service/app/repositories/stock_repository.py
Unescape View File

@ -0,0 +1,222 @@
"""股票数据仓库"""
from datetime import datetime, time
from typing import List, Tuple, Optional
from sqlalchemy.orm import Session
from sqlalchemy import func, or_
from app.models import (
KLineItem, Symbol, SymbolListRequest, SymbolListData,
TradingDatesData, TradeCalData, AdjustType, Frequency
)
from app.repositories.models import (
StockSymbol, StockKLine1M, StockKLine5M, StockKLine1D,
StockTradingCalendar
)
class StockRepository:
"""股票数据仓库"""
def __init__(self, db: Session):
self.db = db
def get_klines(
self,
symbol: str,
freq: Frequency,
start: datetime,
end: datetime,
adjust: AdjustType = AdjustType.NONE
) -> List[KLineItem]:
"""获取K线数据"""
# 根据周期选择表
kline_model = self._get_kline_model(freq)
query = self.db.query(kline_model).filter(
kline_model.symbol_id == symbol,
kline_model.ts >= start,
kline_model.ts <= end
).order_by(kline_model.ts.asc())
results = query.all()
items = []
for r in results:
item = KLineItem(
time=r.ts,
open=float(r.open),
high=float(r.high),
low=float(r.low),
close=float(r.close),
volume=r.volume,
amount=float(r.amount)
)
items.append(item)
return items
def _get_kline_model(self, freq: Frequency):
"""根据周期获取K线模型"""
mapping = {
Frequency.FREQ_1M: StockKLine1M,
Frequency.FREQ_5M: StockKLine5M,
Frequency.FREQ_1D: StockKLine1D,
}
return mapping.get(freq, StockKLine1D)
def save_klines(self, freq: Frequency, items: List[KLineItem]) -> None:
"""保存K线数据"""
if not items:
return
kline_model = self._get_kline_model(freq)
for item in items:
# 使用upsert逻辑
existing = self.db.query(kline_model).filter(
kline_model.symbol_id == getattr(item, 'symbol', ''),
kline_model.ts == item.time
).first()
if existing:
existing.open = item.open
existing.high = item.high
existing.low = item.low
existing.close = item.close
existing.volume = item.volume
existing.amount = item.amount
else:
new_record = kline_model(
symbol_id=getattr(item, 'symbol', ''),
ts=item.time,
open=item.open,
high=item.high,
low=item.low,
close=item.close,
volume=item.volume,
amount=item.amount
)
self.db.add(new_record)
self.db.commit()
def list_symbols(
self,
req: SymbolListRequest
) -> Tuple[List[Symbol], int]:
"""查询标的列表"""
query = self.db.query(StockSymbol)
# 筛选条件
if req.exchange:
query = query.filter(StockSymbol.exchange == req.exchange.value)
if req.keyword:
keyword = f"%{req.keyword}%"
query = query.filter(
or_(
StockSymbol.symbol_id.ilike(keyword),
StockSymbol.name.ilike(keyword)
)
)
# 查询总数
total = query.count()
# 分页查询
results = query.order_by(StockSymbol.symbol_id).offset(
(req.page - 1) * req.size
).limit(req.size).all()
symbols = []
for r in results:
s = Symbol(
symbol_id=r.symbol_id,
symbol_type=r.symbol_type,
exchange=r.exchange,
name=r.name,
name_en=r.name_en,
list_date=r.list_date,
delist_date=r.delist_date,
industry=r.industry,
status=r.status
)
symbols.append(s)
return symbols, total
def get_trading_dates(self, start: str, end: str) -> TradingDatesData:
"""获取交易日历"""
results = self.db.query(StockTradingCalendar).filter(
StockTradingCalendar.trade_date >= start,
StockTradingCalendar.trade_date <= end,
StockTradingCalendar.is_trading_day == True
).order_by(StockTradingCalendar.trade_date.asc()).all()
dates = [r.trade_date for r in results]
# 计算总天数
start_date = datetime.strptime(start, "%Y%m%d")
end_date = datetime.strptime(end, "%Y%m%d")
total_days = (end_date - start_date).days + 1
return TradingDatesData(
start=start,
end=end,
total_days=total_days,
trading_days=len(dates),
trading_dates=dates
)
def save_symbols(self, symbols: List[Symbol]) -> None:
"""保存标的列表"""
for s in symbols:
existing = self.db.query(StockSymbol).filter(
StockSymbol.symbol_id == s.symbol_id
).first()
if existing:
existing.name = s.name
existing.name_en = s.name_en
existing.list_date = s.list_date
existing.delist_date = s.delist_date
existing.industry = s.industry
existing.status = s.status
else:
new_symbol = StockSymbol(
symbol_id=s.symbol_id,
symbol_type=s.symbol_type.value if s.symbol_type else "stock",
exchange=s.exchange.value if s.exchange else "",
name=s.name,
name_en=s.name_en,
list_date=s.list_date,
delist_date=s.delist_date,
industry=s.industry,
status=s.status
)
self.db.add(new_symbol)
self.db.commit()
def save_trading_calendar(self, dates: List[TradeCalData]) -> None:
"""保存交易日历"""
for d in dates:
date_str = d.date.strftime("%Y%m%d")
existing = self.db.query(StockTradingCalendar).filter(
StockTradingCalendar.trade_date == date_str
).first()
if existing:
existing.is_trading_day = d.is_trading_day
existing.week_day = d.date.weekday() + 1
else:
new_cal = StockTradingCalendar(
trade_date=date_str,
is_trading_day=d.is_trading_day,
week_day=d.date.weekday() + 1
)
self.db.add(new_cal)
self.db.commit()

16
python_market_data_service/app/services/__init__.py
Unescape View File

@ -0,0 +1,16 @@
"""业务服务层模块"""
from .stock_service import StockService
from .futures_service import FuturesService
from .admin_service import AdminService
from .config_service import ConfigService
from .adapter_service import AdapterService
from .test_service import TestService
__all__ = [
"StockService",
"FuturesService",
"AdminService",
"ConfigService",
"AdapterService",
"TestService",
]

194
python_market_data_service/app/services/adapter_service.py
Unescape View File

@ -0,0 +1,194 @@
"""适配器管理服务 - 对应Go的internal/service/adapter.go"""
import asyncio
from datetime import datetime
from typing import Dict, List, Optional, Callable
from threading import RLock
from app.models import (
AdapterListData, AdapterInfo, AdapterStatus,
AdapterToggleRequest, AdapterConfigUpdateRequest
)
from app.adapters import DataSourceAdapter, TushareAdapter
from app.core.logger import info, error
class AdapterService:
"""适配器管理服务"""
def __init__(self):
self.lock = RLock()
# 已注册的适配器工厂
self.factories: Dict[str, Callable[[], DataSourceAdapter]] = {}
# 适配器配置
self.configs: Dict[str, dict] = {}
# 当前激活的适配器实例
self.active_adapters: Dict[str, DataSourceAdapter] = {}
# 适配器元数据
self.metadata: Dict[str, dict] = {}
# 注册内置适配器
self._register_builtin_adapters()
def _register_builtin_adapters(self):
"""注册内置适配器"""
# 注册Tushare适配器
self.register_adapter("tushare", lambda: TushareAdapter())
# 设置Tushare元数据
self.metadata["tushare"] = {
"name": "tushare",
"type": "http",
"version": "1.0.0",
"description": "Tushare Pro 金融数据接口",
"updated_at": datetime.now()
}
# 默认配置
self.configs["tushare"] = {
"enabled": True,
"config": {
"token": "",
"base_url": "https://api.tushare.pro"
}
}
# 预留Wind适配器
self.metadata["wind"] = {
"name": "wind",
"type": "ws",
"version": "1.0.0",
"description": "Wind 金融终端接口（预留）",
"updated_at": datetime.now()
}
self.configs["wind"] = {
"enabled": False,
"config": {
"host": "localhost",
"port": "8081"
}
}
def get_adapter_list(self) -> AdapterListData:
"""获取适配器列表"""
with self.lock:
adapters = []
for name, meta in self.metadata.items():
cfg = self.configs.get(name, {"enabled": False, "config": {}})
# 确定状态
if not cfg["enabled"]:
status = AdapterStatus.DISABLED
elif name in self.active_adapters:
status = AdapterStatus.ACTIVE
else:
status = AdapterStatus.STANDBY
adapters.append(AdapterInfo(
name=meta["name"],
type=meta["type"],
version=meta["version"],
description=meta["description"],
status=status,
config=cfg["config"],
updated_at=meta["updated_at"]
))
return AdapterListData(adapters=adapters)
def toggle_adapter(self, req: AdapterToggleRequest) -> None:
"""启用/禁用适配器"""
with self.lock:
if req.name not in self.configs:
raise ValueError(f"Adapter not found: {req.name}")
self.configs[req.name]["enabled"] = req.enable
# 如果禁用，关闭适配器连接
if not req.enable and req.name in self.active_adapters:
adapter = self.active_adapters.pop(req.name)
asyncio.create_task(adapter.close())
# 更新元数据
if req.name in self.metadata:
self.metadata[req.name]["updated_at"] = datetime.now()
def update_adapter_config(self, req: AdapterConfigUpdateRequest) -> None:
"""更新适配器配置"""
with self.lock:
if req.name not in self.configs:
raise ValueError(f"Adapter not found: {req.name}")
# 更新配置
self.configs[req.name]["config"].update(req.config)
# 如果适配器已激活，重新连接
if req.name in self.active_adapters:
adapter = self.active_adapters.pop(req.name)
asyncio.create_task(adapter.close())
# 如果启用状态，重新连接
if self.configs[req.name]["enabled"]:
asyncio.create_task(self._connect_adapter(req.name))
# 更新元数据
if req.name in self.metadata:
self.metadata[req.name]["updated_at"] = datetime.now()
def get_active_adapter(self, asset_class: str) -> Optional[DataSourceAdapter]:
"""获取当前激活的适配器"""
with self.lock:
# 根据资产类别获取配置（简化处理）
adapter_name = "tushare"
# 检查是否已有激活的实例
if adapter_name in self.active_adapters:
return self.active_adapters[adapter_name]
return None
def get_available_adapters(self) -> List[str]:
"""获取所有可用的适配器名称"""
with self.lock:
names = []
for name, meta in self.metadata.items():
if name in self.factories:
names.append(f"{name}|{meta['description']}")
return names
def register_adapter(self, name: str, factory: Callable[[], DataSourceAdapter]):
"""注册适配器"""
with self.lock:
self.factories[name] = factory
async def _connect_adapter(self, name: str):
"""连接适配器"""
with self.lock:
if name not in self.factories:
raise ValueError(f"Adapter factory not found: {name}")
if name not in self.configs:
raise ValueError(f"Adapter config not found: {name}")
factory = self.factories[name]
cfg = self.configs[name]
adapter = factory()
await adapter.connect(cfg["config"])
with self.lock:
self.active_adapters[name] = adapter
async def health_check(self, name: str) -> bool:
"""适配器健康检查"""
with self.lock:
if name not in self.active_adapters:
return False
adapter = self.active_adapters[name]
return await adapter.health_check()

104
python_market_data_service/app/services/admin_service.py
Unescape View File

@ -0,0 +1,104 @@
"""管理服务 - 对应Go的internal/service/admin.go"""
from datetime import datetime
from typing import Optional
import uuid
from sqlalchemy.orm import Session
from sqlalchemy import text
from app.models import (
DataSourceStatusData, DataSourceInfo, SourceSwitchRequest,
BackfillRequest, HealthResponse, DataSourceStatus
)
from app.core.logger import info
class AdminService:
"""管理服务"""
def __init__(self, db: Session):
self.db = db
def get_data_source_status(self) -> DataSourceStatusData:
"""获取数据源状态"""
try:
# 查询数据库中的数据源配置
result = self.db.execute(text("""
SELECT asset_class, active_source, standby_sources, updated_at
FROM public.data_source_config
"""))
data = DataSourceStatusData()
for row in result:
asset_class, active_source, standby_sources, updated_at = row
info_obj = DataSourceInfo(
active_source=active_source,
standby_sources=standby_sources or [],
status=DataSourceStatus.HEALTHY
)
if asset_class == "stock":
data.stock = info_obj
elif asset_class == "futures":
data.futures = info_obj
return data
except Exception as e:
info(f"Data source config not found, using defaults: {e}")
# 返回默认配置
return DataSourceStatusData()
def switch_data_source(self, req: SourceSwitchRequest) -> None:
"""切换数据源"""
asset_classes = []
if req.asset_class.value == "all":
asset_classes = ["stock", "futures"]
else:
asset_classes = [req.asset_class.value]
for ac in asset_classes:
self.db.execute(
text("""
INSERT INTO public.data_source_config
(asset_class, active_source, updated_at)
VALUES (:asset_class, :source, NOW())
ON CONFLICT (asset_class) DO UPDATE SET
active_source = EXCLUDED.active_source,
updated_at = EXCLUDED.updated_at
"""),
{"asset_class": ac, "source": req.source}
)
self.db.commit()
# 如果需要同步补录，启动后台任务
if req.sync_backfill:
info(f"Starting backfill for {req.asset_class} from {req.start_date}")
# TODO: 启动异步补录任务
def backfill_data(self, req: BackfillRequest) -> str:
"""历史数据补录"""
task_id = str(uuid.uuid4())
info(f"Starting backfill task {task_id} for {req.asset_class}")
# TODO: 将补录任务存入数据库，启动后台Worker执行
return task_id
def health_check(self) -> HealthResponse:
"""健康检查"""
try:
# 检查数据库连接
self.db.execute(text("SELECT 1"))
return HealthResponse(
status="healthy",
timestamp=datetime.now()
)
except Exception as e:
return HealthResponse(
status=f"unhealthy: {str(e)}",
timestamp=datetime.now()
)

332
python_market_data_service/app/services/config_service.py
Unescape View File

@ -0,0 +1,332 @@
"""配置管理服务 - 对应Go的internal/service/config.go"""
import platform
import psutil
import threading
from datetime import datetime, timedelta
from typing import Optional, List, Callable, Dict, Any
from app.models import (
ConfigListRequest, ConfigListData, ConfigSection, ConfigItem,
ConfigUpdateRequest, ConfigUpdateData, ConfigType,
ReloadRequest, ReloadData, SystemStatusData, MemoryInfo
)
from app.core.config import get_config, reload_config, save_config, Config
from app.core.logger import info
class ConfigService:
"""配置管理服务"""
def __init__(self):
self.config = get_config()
self.start_time = datetime.now()
self.version = "1.0.0"
self.callbacks: Dict[ConfigType, List[Callable]] = {}
self.lock = threading.RLock()
def get_config_list(self, req: ConfigListRequest) -> ConfigListData:
"""获取配置列表"""
sections = []
# 服务器配置
if not req.type or req.type == ConfigType.SERVER:
sections.append(ConfigSection(
name="服务器配置",
type=ConfigType.SERVER,
description="HTTP服务器相关配置",
items=[
ConfigItem(
key="port",
value=self.config.server.port,
type="int",
description="服务端口",
editable=True,
required=True
),
ConfigItem(
key="mode",
value=self.config.server.mode,
type="string",
description="运行模式: debug/release",
editable=True,
required=True
),
ConfigItem(
key="api_key",
value=self.config.server.api_key,
type="string",
description="API认证密钥",
editable=True,
required=True
),
]
))
# 数据库配置
if not req.type or req.type == ConfigType.DATABASE:
sections.append(ConfigSection(
name="数据库配置",
type=ConfigType.DATABASE,
description="PostgreSQL数据库连接配置",
items=[
ConfigItem(
key="host",
value=self.config.database.host,
type="string",
description="数据库主机地址",
editable=True,
required=True
),
ConfigItem(
key="port",
value=self.config.database.port,
type="int",
description="数据库端口",
editable=True,
required=True
),
ConfigItem(
key="user",
value=self.config.database.user,
type="string",
description="数据库用户名",
editable=True,
required=True
),
ConfigItem(
key="password",
value="********",
type="password",
description="数据库密码",
editable=True,
required=True
),
ConfigItem(
key="database",
value=self.config.database.database,
type="string",
description="数据库名",
editable=True,
required=True
),
]
))
# Redis配置
if not req.type or req.type == ConfigType.REDIS:
sections.append(ConfigSection(
name="Redis配置",
type=ConfigType.REDIS,
description="Redis缓存配置",
items=[
ConfigItem(
key="host",
value=self.config.redis.host,
type="string",
description="Redis主机地址",
editable=True,
required=False
),
ConfigItem(
key="port",
value=self.config.redis.port,
type="int",
description="Redis端口",
editable=True,
required=False
),
ConfigItem(
key="password",
value="********",
type="password",
description="Redis密码",
editable=True,
required=False
),
ConfigItem(
key="db",
value=self.config.redis.db,
type="int",
description="Redis数据库编号",
editable=True,
required=False
),
]
))
# 数据源配置
if not req.type or req.type == ConfigType.SOURCE:
sections.append(ConfigSection(
name="数据源配置",
type=ConfigType.SOURCE,
description="股票和期货数据源配置",
items=[
ConfigItem(
key="stock_active",
value=self.config.sources.stock.active,
type="string",
description="股票数据源适配器",
editable=True,
required=True
),
ConfigItem(
key="futures_active",
value=self.config.sources.futures.active,
type="string",
description="期货数据源适配器",
editable=True,
required=True
),
]
))
return ConfigListData(
sections=sections,
version=self.version,
updated=datetime.now()
)
def update_config(self, req: ConfigUpdateRequest) -> ConfigUpdateData:
"""更新配置"""
need_restart = False
with self.lock:
if req.type == ConfigType.SERVER:
if "port" in req.items:
self.config.server.port = int(req.items["port"])
need_restart = True
if "mode" in req.items:
self.config.server.mode = req.items["mode"]
if "api_key" in req.items:
self.config.server.api_key = req.items["api_key"]
elif req.type == ConfigType.DATABASE:
if "host" in req.items:
self.config.database.host = req.items["host"]
need_restart = True
if "port" in req.items:
self.config.database.port = int(req.items["port"])
need_restart = True
if "user" in req.items:
self.config.database.user = req.items["user"]
need_restart = True
if "password" in req.items:
password = req.items["password"]
if password != "********":
self.config.database.password = password
need_restart = True
if "database" in req.items:
self.config.database.database = req.items["database"]
need_restart = True
elif req.type == ConfigType.SOURCE:
if "stock_active" in req.items:
self.config.sources.stock.active = req.items["stock_active"]
if "futures_active" in req.items:
self.config.sources.futures.active = req.items["futures_active"]
# 保存到文件
try:
save_config(self.config)
self._trigger_callbacks(req.type)
message = "配置更新成功"
if need_restart:
message += "，部分配置需要重启服务后生效"
return ConfigUpdateData(
success=True,
need_restart=need_restart,
message=message
)
except Exception as e:
return ConfigUpdateData(
success=False,
need_restart=False,
message=f"配置保存失败: {e}"
)
def reload_config(self, req: ReloadRequest) -> ReloadData:
"""热加载配置"""
try:
with self.lock:
new_config = reload_config()
# 根据类型选择性更新
if req.config_type is None:
self.config = new_config
else:
if req.config_type == ConfigType.SERVER:
self.config.server = new_config.server
elif req.config_type == ConfigType.DATABASE:
self.config.database = new_config.database
elif req.config_type == ConfigType.REDIS:
self.config.redis = new_config.redis
elif req.config_type == ConfigType.SOURCE:
self.config.sources = new_config.sources
self._trigger_callbacks(req.config_type)
return ReloadData(
success=True,
message="配置热加载成功"
)
except Exception as e:
return ReloadData(
success=False,
message=f"加载配置失败: {e}"
)
def get_system_status(self) -> SystemStatusData:
"""获取系统状态"""
# 获取内存信息
mem = psutil.virtual_memory()
# 计算运行时长
uptime = datetime.now() - self.start_time
uptime_str = self._format_duration(uptime)
return SystemStatusData(
status="running",
version=self.version,
start_time=self.start_time,
uptime=uptime_str,
python_version=platform.python_version(),
memory=MemoryInfo(
alloc=mem.used,
total_alloc=mem.total,
sys=mem.total,
num_gc=0 # Python不需要显式GC计数
),
threads=threading.active_count()
)
def _format_duration(self, d: timedelta) -> str:
"""格式化持续时间"""
days = d.days
hours, remainder = divmod(d.seconds, 3600)
minutes, _ = divmod(remainder, 60)
if days > 0:
return f"{days}天{hours}小时{minutes}分钟"
if hours > 0:
return f"{hours}小时{minutes}分钟"
return f"{minutes}分钟"
def register_callback(self, config_type: ConfigType, callback: Callable):
"""注册配置变更回调"""
with self.lock:
if config_type not in self.callbacks:
self.callbacks[config_type] = []
self.callbacks[config_type].append(callback)
def _trigger_callbacks(self, config_type: Optional[ConfigType]):
"""触发回调"""
with self.lock:
# 触发特定类型的回调
if config_type and config_type in self.callbacks:
for cb in self.callbacks[config_type]:
try:
cb()
except Exception as e:
info(f"Callback error: {e}")

102
python_market_data_service/app/services/futures_service.py
Unescape View File

@ -0,0 +1,102 @@
"""期货业务服务 - 对应Go的internal/service/futures.go"""
from datetime import datetime, timedelta
from typing import List
from sqlalchemy.orm import Session
from app.models import (
KLineQueryRequest, KLineData, SymbolListRequest, SymbolListData,
BatchKLineRequest, BatchKLineData, BatchKLineResult, KLineSubData,
TradingDatesRequest, TradingDatesData,
FuturesContractsRequest, FuturesContractsData
)
from app.repositories import FuturesRepository
from app.core.logger import error
class FuturesService:
"""期货业务服务"""
def __init__(self, db: Session):
self.repository = FuturesRepository(db)
def query_klines(self, req: KLineQueryRequest) -> KLineData:
"""查询K线数据"""
# 解析日期
try:
start = datetime.strptime(req.start, "%Y%m%d")
end = datetime.strptime(req.end, "%Y%m%d")
end = end + timedelta(days=1) - timedelta(seconds=1)
except ValueError as e:
raise ValueError(f"Invalid date format: {e}")
# 获取K线数据
items = self.repository.get_klines(req.symbol, req.freq, start, end)
return KLineData(
symbol=req.symbol,
freq=req.freq,
count=len(items),
items=items
)
def list_symbols(self, req: SymbolListRequest) -> SymbolListData:
"""查询标的列表"""
if req.page <= 0:
req.page = 1
if req.size <= 0:
req.size = 20
if req.size > 100:
req.size = 100
symbols, total = self.repository.list_symbols(req)
return SymbolListData(
total=total,
page=req.page,
size=req.size,
items=symbols
)
def batch_query_klines(self, req: BatchKLineRequest) -> BatchKLineData:
"""批量查询K线"""
results = []
for symbol in req.symbols:
single_req = KLineQueryRequest(
symbol=symbol,
start=req.start,
end=req.end,
freq=req.freq
)
try:
data = self.query_klines(single_req)
results.append(BatchKLineResult(
symbol=symbol,
success=True,
data=KLineSubData(count=data.count, items=data.items)
))
except Exception as e:
error(f"Batch query failed for {symbol}: {e}")
results.append(BatchKLineResult(
symbol=symbol,
success=False,
error=str(e)
))
return BatchKLineData(results=results)
def get_trading_dates(self, req: TradingDatesRequest) -> TradingDatesData:
"""获取交易日历"""
return self.repository.get_trading_dates(req.start, req.end)
def get_contracts_by_underlying(
self,
req: FuturesContractsRequest
) -> FuturesContractsData:
"""根据品种获取合约"""
return self.repository.get_contracts_by_underlying(
req.underlying,
req.exchange
)

115
python_market_data_service/app/services/stock_service.py
Unescape View File

@ -0,0 +1,115 @@
"""股票业务服务 - 对应Go的internal/service/stock.go"""
from datetime import datetime, timedelta
from typing import List
from sqlalchemy.orm import Session
from app.models import (
KLineQueryRequest, KLineData, SymbolListRequest, SymbolListData,
BatchKLineRequest, BatchKLineData, BatchKLineResult, KLineSubData,
TradingDatesRequest, TradingDatesData, AdjustType, Frequency
)
from app.repositories import StockRepository
from app.core.logger import error
class StockService:
"""股票业务服务"""
def __init__(self, db: Session):
self.repository = StockRepository(db)
def query_klines(self, req: KLineQueryRequest) -> KLineData:
"""查询K线数据"""
# 解析日期
try:
start = datetime.strptime(req.start, "%Y%m%d")
end = datetime.strptime(req.end, "%Y%m%d")
end = end + timedelta(days=1) - timedelta(seconds=1) # 包含结束日期全天
except ValueError as e:
raise ValueError(f"Invalid date format: {e}")
# 获取K线数据
items = self.repository.get_klines(
req.symbol,
req.freq,
start,
end,
req.adjust
)
# 处理复权（简化实现，实际需要复权系数表）
if req.adjust != AdjustType.NONE:
items = self._apply_adjust(req.symbol, items, req.adjust)
return KLineData(
symbol=req.symbol,
freq=req.freq,
adjust=req.adjust,
count=len(items),
items=items
)
def _apply_adjust(
self,
symbol: str,
items: List,
adjust_type: AdjustType
) -> List:
"""应用复权计算（TODO: 实现复权逻辑）"""
# 复权计算需要从数据库获取复权系数
# 这里简化处理，直接返回原始数据
return items
def list_symbols(self, req: SymbolListRequest) -> SymbolListData:
"""查询标的列表"""
# 设置默认值
if req.page <= 0:
req.page = 1
if req.size <= 0:
req.size = 20
if req.size > 100:
req.size = 100
symbols, total = self.repository.list_symbols(req)
return SymbolListData(
total=total,
page=req.page,
size=req.size,
items=symbols
)
def batch_query_klines(self, req: BatchKLineRequest) -> BatchKLineData:
"""批量查询K线"""
results = []
for symbol in req.symbols:
single_req = KLineQueryRequest(
symbol=symbol,
start=req.start,
end=req.end,
freq=req.freq,
adjust=req.adjust
)
try:
data = self.query_klines(single_req)
results.append(BatchKLineResult(
symbol=symbol,
success=True,
data=KLineSubData(count=data.count, items=data.items)
))
except Exception as e:
error(f"Batch query failed for {symbol}: {e}")
results.append(BatchKLineResult(
symbol=symbol,
success=False,
error=str(e)
))
return BatchKLineData(results=results)
def get_trading_dates(self, req: TradingDatesRequest) -> TradingDatesData:
"""获取交易日历"""
return self.repository.get_trading_dates(req.start, req.end)

390
python_market_data_service/app/services/test_service.py
Unescape View File

@ -0,0 +1,390 @@
"""测试服务 - 对应Go的internal/service/test.go"""
import asyncio
import json
from datetime import datetime, timedelta
from typing import List, Optional
from threading import RLock
import httpx
import websockets
from app.models import (
APITestListData, APITestCategory, APITestCase,
APITestRequest, APITestResult,
WSTestListData, WSTestCase, WSTestRequest, WSTestResult, WSMessage,
TestHistoryRequest, TestHistoryData
)
from app.core.logger import info, error
class TestService:
"""测试服务"""
def __init__(self):
self.lock = RLock()
self.api_history: List[APITestResult] = []
self.ws_history: List[WSTestResult] = []
self.history_size = 100
def get_api_test_list(self) -> APITestListData:
"""获取API测试列表"""
today = datetime.now()
month_ago = today - timedelta(days=30)
week_ago = today - timedelta(days=7)
categories = [
APITestCategory(
name="股票接口",
items=[
APITestCase(
id="stock_klines",
name="查询股票K线",
method="GET",
path="/v1/stock/klines/{symbol}",
description="查询指定股票的K线数据",
params={
"symbol": "000001.SZ",
"start": month_ago.strftime("%Y%m%d"),
"end": today.strftime("%Y%m%d"),
"freq": "1d",
"adjust": "qfq"
}
),
APITestCase(
id="stock_symbols",
name="查询股票列表",
method="GET",
path="/v1/stock/symbols",
description="获取所有可用股票标的",
params={"page": "1", "size": "20"}
),
APITestCase(
id="stock_batch",
name="批量查询股票K线",
method="POST",
path="/v1/stock/klines/batch",
description="批量查询多只股票K线",
body={
"symbols": ["000001.SZ", "000002.SZ"],
"start": week_ago.strftime("%Y%m%d"),
"end": today.strftime("%Y%m%d"),
"freq": "1d"
}
),
APITestCase(
id="stock_calendar",
name="查询交易日历",
method="GET",
path="/v1/stock/trading-dates",
description="查询股票交易日历",
params={
"start": month_ago.strftime("%Y%m%d"),
"end": (today + timedelta(days=30)).strftime("%Y%m%d")
}
),
]
),
APITestCategory(
name="期货接口",
items=[
APITestCase(
id="futures_klines",
name="查询期货K线",
method="GET",
path="/v1/futures/klines/{symbol}",
description="查询指定期货合约的K线数据",
params={
"symbol": "CU2504.SHFE",
"start": month_ago.strftime("%Y%m%d"),
"end": today.strftime("%Y%m%d"),
"freq": "1d"
}
),
APITestCase(
id="futures_symbols",
name="查询期货列表",
method="GET",
path="/v1/futures/symbols",
description="获取所有可用期货标的",
params={"page": "1", "size": "20"}
),
APITestCase(
id="futures_batch",
name="批量查询期货K线",
method="POST",
path="/v1/futures/klines/batch",
description="批量查询多个期货合约K线",
body={
"symbols": ["CU2504.SHFE", "RB2505.SHFE"],
"start": week_ago.strftime("%Y%m%d"),
"end": today.strftime("%Y%m%d"),
"freq": "1d"
}
),
APITestCase(
id="futures_contracts",
name="查询合约列表",
method="GET",
path="/v1/futures/contracts",
description="根据品种查询可交易合约",
params={"underlying": "CU", "exchange": "SHFE"}
),
APITestCase(
id="futures_calendar",
name="查询期货交易日历",
method="GET",
path="/v1/futures/trading-dates",
description="查询期货交易日历",
params={
"start": month_ago.strftime("%Y%m%d"),
"end": (today + timedelta(days=30)).strftime("%Y%m%d")
}
),
]
),
APITestCategory(
name="管理接口",
items=[
APITestCase(
id="admin_health",
name="健康检查",
method="GET",
path="/v1/admin/health",
description="检查服务健康状态",
params={}
),
APITestCase(
id="admin_source_status",
name="数据源状态",
method="GET",
path="/v1/admin/source/status",
description="获取当前数据源状态",
params={}
),
]
),
]
return APITestListData(categories=categories, base_url="")
async def run_api_test(self, base_url: str, req: APITestRequest) -> APITestResult:
"""执行API测试"""
# 获取测试用例
test_list = self.get_api_test_list()
test_case = None
for cat in test_list.categories:
for item in cat.items:
if item.id == req.id:
test_case = item
break
if test_case:
break
if not test_case:
raise ValueError(f"Test case not found: {req.id}")
# 合并参数
params = dict(test_case.params)
if req.params:
params.update(req.params)
# 构建URL
url = base_url + test_case.path
for k, v in params.items():
url = url.replace(f"{{{k}}}", str(v))
# 添加查询参数
if test_case.method == "GET" and params:
query_parts = []
for k, v in params.items():
if f"{{{k}}}" not in test_case.path:
query_parts.append(f"{k}={v}")
if query_parts:
url += "?" + "&".join(query_parts)
# 准备请求体
body = req.body if req.body is not None else test_case.body
# 执行请求
start_time = datetime.now()
async with httpx.AsyncClient() as client:
try:
headers = {"X-API-Key": "test-api-key"}
if test_case.method == "GET":
response = await client.get(url, headers=headers, timeout=30)
elif test_case.method == "POST":
response = await client.post(
url, json=body, headers=headers, timeout=30
)
else:
raise ValueError(f"Unsupported method: {test_case.method}")
latency = int((datetime.now() - start_time).total_seconds() * 1000)
result = APITestResult(
id=int(datetime.now().timestamp()),
case_id=req.id,
name=test_case.name,
success=200 <= response.status_code < 300,
status_code=response.status_code,
latency=latency,
request={
"method": test_case.method,
"url": url,
"body": body
},
response=response.json() if response.headers.get("content-type", "").startswith("application/json") else response.text,
timestamp=datetime.now()
)
self._add_api_history(result)
return result
except Exception as e:
latency = int((datetime.now() - start_time).total_seconds() * 1000)
result = APITestResult(
id=int(datetime.now().timestamp()),
case_id=req.id,
name=test_case.name,
success=False,
latency=latency,
request={
"method": test_case.method,
"url": url,
"body": body
},
error=str(e),
timestamp=datetime.now()
)
self._add_api_history(result)
return result
def get_ws_test_list(self) -> WSTestListData:
"""获取WebSocket测试列表"""
cases = [
WSTestCase(
id="ws_subscribe_stock",
name="订阅股票行情",
description="订阅单只股票实时行情",
action="subscribe",
symbols=["000001.SZ"]
),
WSTestCase(
id="ws_subscribe_futures",
name="订阅期货行情",
description="订阅单个期货合约实时行情",
action="subscribe",
symbols=["CU2504.SHFE"]
),
WSTestCase(
id="ws_subscribe_multi",
name="批量订阅",
description="同时订阅多个标的",
action="subscribe",
symbols=["000001.SZ", "000002.SZ", "CU2504.SHFE"]
),
WSTestCase(
id="ws_unsubscribe",
name="取消订阅",
description="取消订阅标的",
action="unsubscribe",
symbols=["000001.SZ"]
),
]
return WSTestListData(cases=cases, ws_url="")
async def run_ws_test(self, ws_url: str, req: WSTestRequest) -> WSTestResult:
"""执行WebSocket测试"""
# 获取测试用例
test_list = self.get_ws_test_list()
test_case = None
for item in test_list.cases:
if item.id == req.id:
test_case = item
break
if not test_case:
raise ValueError(f"Test case not found: {req.id}")
# 使用自定义标的
symbols = req.symbols if req.symbols else test_case.symbols
result = WSTestResult(
id=f"ws_{int(datetime.now().timestamp())}",
case_id=req.id,
timestamp=datetime.now(),
messages=[]
)
# 连接WebSocket
start_time = datetime.now()
try:
async with websockets.connect(
ws_url,
extra_headers={"X-API-Key": "test-api-key"}
) as ws:
result.latency = int((datetime.now() - start_time).total_seconds() * 1000)
result.success = True
# 发送订阅消息
msg = {
"action": test_case.action,
"symbols": symbols
}
await ws.send(json.dumps(msg))
# 等待响应（最多3条消息）
for _ in range(3):
try:
msg_data = await asyncio.wait_for(ws.recv(), timeout=5)
result.messages.append(WSMessage(
type="received",
data=json.loads(msg_data),
timestamp=datetime.now()
))
except asyncio.TimeoutError:
break
except Exception as e:
result.latency = int((datetime.now() - start_time).total_seconds() * 1000)
result.success = False
result.error = str(e)
self._add_ws_history(result)
return result
def get_test_history(self, req: TestHistoryRequest) -> TestHistoryData:
"""获取测试历史"""
with self.lock:
limit = req.limit or 20
api_tests = []
ws_tests = []
if not req.type or req.type == "api":
api_tests = self.api_history[-limit:]
if not req.type or req.type == "ws":
ws_tests = self.ws_history[-limit:]
return TestHistoryData(api_tests=api_tests, ws_tests=ws_tests)
def _add_api_history(self, result: APITestResult):
"""添加API测试历史"""
with self.lock:
self.api_history.append(result)
if len(self.api_history) > self.history_size:
self.api_history = self.api_history[-self.history_size:]
def _add_ws_history(self, result: WSTestResult):
"""添加WebSocket测试历史"""
with self.lock:
self.ws_history.append(result)
if len(self.ws_history) > self.history_size:
self.ws_history = self.ws_history[-self.history_size:]

4
python_market_data_service/app/websocket/__init__.py
Unescape View File

@ -0,0 +1,4 @@
"""WebSocket服务模块"""
from .server import WebSocketServer, ws_manager
__all__ = ["WebSocketServer", "ws_manager"]

210
python_market_data_service/app/websocket/server.py
Unescape View File

@ -0,0 +1,210 @@
"""WebSocket服务 - 对应Go的internal/websocket/server.go"""
import asyncio
import json
from datetime import datetime
from typing import Dict, Set, Optional
from dataclasses import dataclass, field
from fastapi import WebSocket, WebSocketDisconnect
from app.core.logger import info, error
@dataclass
class WSClient:
"""WebSocket客户端"""
id: str
websocket: WebSocket
subscriptions: Set[str] = field(default_factory=set)
async def send(self, message: dict):
"""发送消息"""
try:
await self.websocket.send_json(message)
except Exception as e:
error(f"Failed to send message to client {self.id}: {e}")
class WebSocketManager:
"""WebSocket连接管理器"""
def __init__(self):
self.clients: Dict[str, WSClient] = {}
self.subscriptions: Dict[str, Set[str]] = {} # symbol -> set of client_ids
self.max_symbols_per_client = 100
self.lock = asyncio.Lock()
async def connect(self, websocket: WebSocket, client_id: str) -> WSClient:
"""建立连接"""
await websocket.accept()
client = WSClient(id=client_id, websocket=websocket)
async with self.lock:
self.clients[client_id] = client
info(f"WebSocket client connected: {client_id}, total: {len(self.clients)}")
return client
async def disconnect(self, client_id: str):
"""断开连接"""
async with self.lock:
if client_id in self.clients:
client = self.clients.pop(client_id)
# 清理订阅
for symbol in client.subscriptions:
if symbol in self.subscriptions:
self.subscriptions[symbol].discard(client_id)
if not self.subscriptions[symbol]:
del self.subscriptions[symbol]
info(f"WebSocket client disconnected: {client_id}, total: {len(self.clients)}")
async def subscribe(self, client_id: str, symbols: list) -> bool:
"""订阅标的"""
async with self.lock:
if client_id not in self.clients:
return False
client = self.clients[client_id]
# 检查订阅数量限制
if len(client.subscriptions) + len(symbols) > self.max_symbols_per_client:
return False
for symbol in symbols:
client.subscriptions.add(symbol)
if symbol not in self.subscriptions:
self.subscriptions[symbol] = set()
self.subscriptions[symbol].add(client_id)
return True
async def unsubscribe(self, client_id: str, symbols: list):
"""取消订阅"""
async with self.lock:
if client_id not in self.clients:
return
client = self.clients[client_id]
for symbol in symbols:
client.subscriptions.discard(symbol)
if symbol in self.subscriptions:
self.subscriptions[symbol].discard(client_id)
if not self.subscriptions[symbol]:
del self.subscriptions[symbol]
async def broadcast_to_symbol(self, symbol: str, message: dict):
"""向订阅了某标的的所有客户端广播"""
client_ids = set()
async with self.lock:
if symbol in self.subscriptions:
client_ids = self.subscriptions[symbol].copy()
# 在锁外发送消息
for client_id in client_ids:
if client_id in self.clients:
try:
await self.clients[client_id].send(message)
except Exception as e:
error(f"Failed to broadcast to {client_id}: {e}")
def get_stats(self) -> dict:
"""获取统计信息"""
return {
"total_clients": len(self.clients),
"total_subscriptions": len(self.subscriptions)
}
# 全局WebSocket管理器实例
ws_manager = WebSocketManager()
class WebSocketServer:
"""WebSocket服务器"""
def __init__(self):
self.manager = ws_manager
async def handle(self, websocket: WebSocket, client_id: str):
"""处理WebSocket连接"""
client = await self.manager.connect(websocket, client_id)
try:
while True:
# 接收消息
data = await websocket.receive_text()
try:
msg = json.loads(data)
action = msg.get("action")
symbols = msg.get("symbols", [])
if action == "subscribe":
success = await self.manager.subscribe(client_id, symbols)
if success:
await client.send({
"type": "ack",
"action": "subscribe",
"symbols": symbols,
"ts": datetime.now().isoformat()
})
else:
await client.send({
"type": "error",
"code": 1003,
"message": "Too many subscriptions or subscription failed",
"ts": datetime.now().isoformat()
})
elif action == "unsubscribe":
await self.manager.unsubscribe(client_id, symbols)
await client.send({
"type": "ack",
"action": "unsubscribe",
"symbols": symbols,
"ts": datetime.now().isoformat()
})
else:
await client.send({
"type": "error",
"code": 1001,
"message": "Unknown action",
"ts": datetime.now().isoformat()
})
except json.JSONDecodeError:
await client.send({
"type": "error",
"code": 1000,
"message": "Invalid message format",
"ts": datetime.now().isoformat()
})
except WebSocketDisconnect:
await self.manager.disconnect(client_id)
except Exception as e:
error(f"WebSocket error for client {client_id}: {e}")
await self.manager.disconnect(client_id)
async def send_heartbeat(self):
"""发送心跳（可由定时任务调用）"""
message = {
"type": "heartbeat",
"ts": datetime.now().isoformat()
}
# 向所有客户端发送心跳
clients_copy = list(self.manager.clients.values())
for client in clients_copy:
try:
await client.send(message)
except Exception:
pass

46
python_market_data_service/config.json
Unescape View File

@ -0,0 +1,46 @@
{
"server": {
"port": 8080,
"mode": "debug",
"api_key": "demo-api-key-2024"
},
"database": {
"host": "localhost",
"port": 5432,
"user": "postgres",
"password": "postgres",
"database": "marketdata"
},
"redis": {
"host": "localhost",
"port": 6379,
"password": "",
"db": 0
},
"sources": {
"stock": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "",
"base_url": "https://api.tushare.pro"
}
}
}
},
"futures": {
"active": "tushare",
"list": {
"tushare": {
"type": "http",
"config": {
"token": "",
"base_url": "https://api.tushare.pro"
}
}
}
}
}
}

44
python_market_data_service/pyproject.toml
Unescape View File

@ -0,0 +1,44 @@
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "market-data-service"
version = "1.0.0"
description = "统一行情数据服务 - Python实现"
readme = "README.md"
requires-python = ">=3.10"
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
dependencies = [
"fastapi>=0.115.0",
"uvicorn[standard]>=0.32.0",
"python-socketio>=5.12.1",
"websockets>=14.1",
"sqlalchemy>=2.0.36",
"psycopg2-binary>=2.9.10",
"pandas>=2.2.3",
"numpy>=2.1.3",
"pydantic>=2.10.0",
"pydantic-settings>=2.6.1",
"python-dotenv>=1.0.1",
"PyYAML>=6.0.2",
"httpx>=0.28.0",
"apscheduler>=3.11.0",
]
[project.optional-dependencies]
dev = [
"pytest>=8.3.4",
"pytest-asyncio>=0.24.0",
]
[tool.setuptools.packages.find]
where = ["."]
include = ["app*"]

38
python_market_data_service/requirements.txt
Unescape View File

@ -0,0 +1,38 @@
# Web Framework
fastapi==0.115.0
uvicorn[standard]==0.32.0
python-socketio==5.12.1
websockets==14.1
# Database
sqlalchemy==2.0.36
psycopg2-binary==2.9.10
alembic==1.14.0
# Data Processing
pandas==2.2.3
numpy==2.1.3
# Data Source
# Note: tushare needs to be installed separately with: pip install tushare
tushare==1.4.14
# Configuration
pydantic==2.10.0
pydantic-settings==2.6.1
python-dotenv==1.0.1
PyYAML==6.0.2
# Utilities
python-multipart==0.0.19
httpx==0.28.0
aiohttp==3.11.10
aioredis==2.0.1
# Monitoring
apscheduler==3.11.0
# Testing
pytest==8.3.4
pytest-asyncio==0.24.0
httpx==0.28.0

236
python_market_data_service/scripts/sync_data.py
Unescape View File

@ -0,0 +1,236 @@
"""数据同步工具 - 对应Go的cmd/sync/main.go"""
import asyncio
import os
import sys
from datetime import datetime, timedelta
from argparse import ArgumentParser
# 添加项目根目录到路径
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from app.adapters import TushareAdapter
from app.repositories import SessionLocal
from app.repositories.stock_repository import StockRepository
from app.repositories.futures_repository import FuturesRepository
from app.models import Symbol, SymbolType, TradeCalData
from app.core.logger import info, error
def parse_date(date_str: str) -> datetime:
"""解析日期字符串"""
return datetime.strptime(date_str, "%Y%m%d")
def format_date(date: datetime) -> str:
"""格式化日期为字符串"""
return date.strftime("%Y%m%d")
def is_stock(symbol: str) -> bool:
"""判断是否为股票代码"""
return symbol.endswith(".SH") or symbol.endswith(".SZ") or symbol.endswith(".BJ")
async def sync_stocks(adapter: TushareAdapter, db):
"""同步股票基础信息"""
info("Syncing stock basic info...")
try:
symbols_data = await adapter.fetch_symbols("stock")
repo = StockRepository(db)
symbols = []
for d in symbols_data:
list_date = None
if d.list_date:
try:
list_date = datetime.strptime(d.list_date, "%Y%m%d")
except:
pass
symbols.append(Symbol(
symbol_id=d.symbol_id,
symbol_type=SymbolType.STOCK,
exchange=d.exchange,
name=d.name,
list_date=list_date,
status="active"
))
repo.save_symbols(symbols)
info(f"Synced {len(symbols)} stocks")
except Exception as e:
error(f"Failed to sync stocks: {e}")
raise
async def sync_futures(adapter: TushareAdapter, db):
"""同步期货基础信息"""
info("Syncing futures basic info...")
try:
symbols_data = await adapter.fetch_symbols("futures")
repo = FuturesRepository(db)
symbols = []
for d in symbols_data:
list_date = None
delist_date = None
if d.list_date:
try:
list_date = datetime.strptime(d.list_date, "%Y%m%d")
except:
pass
if d.delist_date:
try:
delist_date = datetime.strptime(d.delist_date, "%Y%m%d")
except:
pass
status = "active"
if delist_date and datetime.now() > delist_date:
status = "expired"
symbols.append(Symbol(
symbol_id=d.symbol_id,
symbol_type=SymbolType.FUTURES,
exchange=d.exchange,
name=d.name,
underlying=d.underlying,
contract_month=d.contract_month,
list_date=list_date,
delist_date=delist_date,
status=status
))
repo.save_symbols(symbols)
info(f"Synced {len(symbols)} futures")
except Exception as e:
error(f"Failed to sync futures: {e}")
raise
async def sync_calendar(adapter: TushareAdapter, db, start: str, end: str):
"""同步交易日历"""
info(f"Syncing trading calendar from {start} to {end}...")
try:
# 同步股票交易日历（上交所）
stock_data = await adapter.fetch_trading_calendar("SH", start, end)
stock_repo = StockRepository(db)
stock_dates = [
TradeCalData(date=d.date, is_trading_day=d.is_trading_day)
for d in stock_data
]
stock_repo.save_trading_calendar(stock_dates)
# 同步期货交易日历
futures_repo = FuturesRepository(db)
futures_repo.save_trading_calendar(stock_dates)
info(f"Synced {len(stock_dates)} calendar days")
except Exception as e:
error(f"Failed to sync calendar: {e}")
raise
async def sync_klines(adapter: TushareAdapter, db, symbol: str, start: str, end: str, freq: str):
"""同步K线数据"""
info(f"Syncing {freq} klines for {symbol} from {start} to {end}...")
try:
# 获取K线数据
klines_data = await adapter.fetch_klines(symbol, start, end, freq)
# 转换为KLineItem并保存
from app.models import KLineItem
items = [
KLineItem(
time=datetime.fromtimestamp(d.time),
open=d.open,
high=d.high,
low=d.low,
close=d.close,
volume=d.volume,
amount=d.amount,
open_interest=d.open_interest if d.open_interest > 0 else None
)
for d in klines_data
]
# 判断股票还是期货并保存
from app.models import Frequency
if is_stock(symbol):
repo = StockRepository(db)
# 为每个item设置symbol
for item in items:
item.symbol = symbol
repo.save_klines(Frequency(freq), items)
else:
repo = FuturesRepository(db)
repo.save_klines(Frequency(freq), symbol, items)
info(f"Synced {len(items)} klines")
except Exception as e:
error(f"Failed to sync klines: {e}")
raise
async def main():
"""主函数"""
parser = ArgumentParser(description="Market Data Sync Tool")
parser.add_argument(
"--type", "-t",
required=True,
choices=["stocks", "futures", "calendar", "klines"],
help="同步类型"
)
parser.add_argument("--start", "-s", help="开始日期 YYYYMMDD")
parser.add_argument("--end", "-e", help="结束日期 YYYYMMDD")
parser.add_argument("--symbol", help="标的代码（klines类型需要）")
parser.add_argument("--freq", "-f", default="1d", help="K线周期")
args = parser.parse_args()
# 配置
tushare_token = os.environ.get("TUSHARE_TOKEN")
if not tushare_token:
error("TUSHARE_TOKEN environment variable is required")
sys.exit(1)
# 初始化适配器
adapter = TushareAdapter()
await adapter.connect({"token": tushare_token})
# 创建数据库会话
db = SessionLocal()
try:
if args.type == "stocks":
await sync_stocks(adapter, db)
elif args.type == "futures":
await sync_futures(adapter, db)
elif args.type == "calendar":
# 设置默认日期范围
start = args.start or (datetime.now() - timedelta(days=30)).strftime("%Y%m%d")
end = args.end or (datetime.now() + timedelta(days=180)).strftime("%Y%m%d")
await sync_calendar(adapter, db, start, end)
elif args.type == "klines":
if not args.symbol:
error("symbol is required for klines sync")
sys.exit(1)
start = args.start or (datetime.now() - timedelta(days=7)).strftime("%Y%m%d")
end = args.end or datetime.now().strftime("%Y%m%d")
await sync_klines(adapter, db, args.symbol, start, end, args.freq)
finally:
db.close()
await adapter.close()
if __name__ == "__main__":
asyncio.run(main())

78
scripts/README.md
Unescape View File

@ -0,0 +1,78 @@
# 安装脚本使用说明
本目录包含 Go 环境的自动安装脚本。
## 脚本列表
| 脚本 | 适用系统 | 说明 |
|------|----------|------|
| `install-go-windows.ps1` | Windows 10/11 | PowerShell 安装脚本 |
| `install-go-linux.sh` | Linux/macOS | Bash 安装脚本 |
## 使用方法
### Windows
1. **以管理员身份打开 PowerShell**
2. **执行安装脚本**
```powershell
cd d:\fs_workspace\market-data-service\scripts
.\install-go-windows.ps1
```
3. **等待安装完成**
脚本会自动：
- 下载 Go 1.21.6
- 执行安装
- 配置环境变量
- 设置国内镜像
4. **重新打开 PowerShell**，验证安装
```powershell
go version
```
### Linux / macOS
1. **打开终端**
2. **执行安装脚本**
```bash
cd /path/to/market-data-service/scripts
chmod +x install-go-linux.sh
./install-go-linux.sh
```
3. **使环境变量生效**
```bash
source ~/.bashrc # Linux Bash
source ~/.zshrc # macOS Zsh
```
4. **验证安装**
```bash
go version
```
## 注意事项
- 脚本需要管理员/Root 权限
- 安装过程中需要联网下载
- 安装完成后需要重新打开终端
## 安装后步骤
安装完成后，返回项目目录启动服务：
```bash
cd d:\fs_workspace\market-data-service
go mod download
go run ./cmd/server
```
然后访问管理后台：
```
http://localhost:8080/admin
```

116
scripts/fix-dependencies.ps1
Unescape View File

@ -0,0 +1,116 @@
# 修复 Go 依赖脚本
param()
$ErrorActionPreference = "Stop"
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host " 修复 Go 依赖问题 " -ForegroundColor Cyan
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host ""
# 检查 Go 是否安装
Write-Host "[1/5] 检查 Go 环境..." -ForegroundColor Yellow
$GoCmd = Get-Command go -ErrorAction SilentlyContinue
if (-not $GoCmd) {
Write-Error "未找到 Go 命令，请先安装 Go"
exit 1
}
$GoVersion = & go version
Write-Host " Go 版本: $GoVersion" -ForegroundColor Green
# 检查 GOPROXY
Write-Host "[2/5] 检查 GOPROXY 设置..." -ForegroundColor Yellow
$GoProxy = & go env GOPROXY
Write-Host " 当前 GOPROXY: $GoProxy" -ForegroundColor Gray
if ($GoProxy -ne "https://goproxy.cn,direct") {
Write-Host " 设置国内镜像..." -ForegroundColor Yellow
& go env -w GOPROXY="https://goproxy.cn,direct"
Write-Host " GOPROXY 已设置为 https://goproxy.cn,direct" -ForegroundColor Green
}
# 检查 GOPATH
Write-Host "[3/5] 检查 GOPATH..." -ForegroundColor Yellow
$GoPath = & go env GOPATH
Write-Host " GOPATH: $GoPath" -ForegroundColor Gray
# 进入项目目录
Write-Host "[4/5] 进入项目目录..." -ForegroundColor Yellow
$ProjectDir = "d:\fs_workspace\market-data-service"
if (-not (Test-Path $ProjectDir)) {
Write-Error "项目目录不存在: $ProjectDir"
exit 1
}
Set-Location $ProjectDir
Write-Host " 当前目录: $(Get-Location)" -ForegroundColor Green
# 清理缓存
Write-Host "[5/5] 修复依赖..." -ForegroundColor Yellow
# 删除旧的模块缓存
Write-Host " 清理模块缓存..." -ForegroundColor Gray
Remove-Item -Path "go.sum" -ErrorAction SilentlyContinue
Remove-Item -Path "$GoPath\pkg\mod\cache" -Recurse -Force -ErrorAction SilentlyContinue
# 设置环境变量
$env:GOPROXY = "https://goproxy.cn,direct"
# 运行 go mod tidy
Write-Host " 运行 go mod tidy..." -ForegroundColor Yellow
try {
& go mod tidy -v
if ($LASTEXITCODE -eq 0) {
Write-Host " go mod tidy 成功" -ForegroundColor Green
} else {
Write-Warning "go mod tidy 返回非零退出码: $LASTEXITCODE"
}
} catch {
Write-Warning "go mod tidy 执行出错: $_"
}
# 下载依赖
Write-Host " 运行 go mod download..." -ForegroundColor Yellow
try {
& go mod download -x
if ($LASTEXITCODE -eq 0) {
Write-Host " go mod download 成功" -ForegroundColor Green
} else {
Write-Warning "go mod download 返回非零退出码: $LASTEXITCODE"
}
} catch {
Write-Warning "go mod download 执行出错: $_"
}
# 验证
Write-Host " 验证依赖..." -ForegroundColor Yellow
try {
& go list -m all | Out-Null
Write-Host " 依赖验证成功" -ForegroundColor Green
} catch {
Write-Warning "依赖验证失败: $_"
}
Write-Host ""
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host " 依赖修复完成 " -ForegroundColor Cyan
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host ""
# 检查 go.sum 是否存在
if (Test-Path "go.sum") {
Write-Host "✓ go.sum 文件已生成" -ForegroundColor Green
} else {
Write-Warning "✗ go.sum 文件未生成"
}
Write-Host ""
Write-Host "接下来可以尝试编译项目:" -ForegroundColor Yellow
Write-Host " go build ./cmd/server" -ForegroundColor Gray
Write-Host ""
Write-Host "或者直接运行:" -ForegroundColor Yellow
Write-Host " go run ./cmd/server" -ForegroundColor Gray
Write-Host ""
Read-Host "按 Enter 键退出"

173
scripts/install-go-linux.sh
Unescape View File

@ -0,0 +1,173 @@
#!/bin/bash
# Go 环境安装脚本 (Linux/macOS)
# 适用于 Ubuntu/CentOS/macOS
set -e
GO_VERSION="1.21.6"
INSTALL_DIR="/usr/local"
# 颜色定义
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
CYAN='\033[0;36m'
NC='\033[0m' # No Color
# 检测操作系统
OS=""
ARCH=$(uname -m)
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
OS="linux"
if [[ "$ARCH" == "x86_64" ]]; then
ARCH="amd64"
elif [[ "$ARCH" == "aarch64" ]]; then
ARCH="arm64"
fi
elif [[ "$OSTYPE" == "darwin"* ]]; then
OS="darwin"
if [[ "$ARCH" == "x86_64" ]]; then
ARCH="amd64"
elif [[ "$ARCH" == "arm64" ]]; then
ARCH="arm64"
fi
else
echo -e "${RED}不支持的操作系统: $OSTYPE${NC}"
exit 1
fi
echo -e "${CYAN}==============================================${NC}"
echo -e "${CYAN} Go ${GO_VERSION} 环境安装脚本 (${OS}/${ARCH}) ${NC}"
echo -e "${CYAN}==============================================${NC}"
echo ""
# 检查当前 Go 版本
echo -e "${YELLOW}[1/6] 检查当前 Go 版本...${NC}"
if command -v go &> /dev/null; then
CURRENT_VERSION=$(go version)
echo -e "${GREEN} 检测到已安装: $CURRENT_VERSION${NC}"
read -p " 是否重新安装? (y/N): " response
if [[ ! "$response" =~ ^[Yy]$ ]]; then
echo -e "${GRAY} 跳过安装${NC}"
exit 0
fi
else
echo -e " 未检测到 Go"
fi
# 下载安装包
echo -e "${YELLOW}[2/6] 下载 Go ${GO_VERSION} ...${NC}"
DOWNLOAD_URL="https://go.dev/dl/go${GO_VERSION}.${OS}-${ARCH}.tar.gz"
TEMP_FILE="/tmp/go${GO_VERSION}.${OS}-${ARCH}.tar.gz"
echo " 下载地址: $DOWNLOAD_URL"
if command -v wget &> /dev/null; then
wget -q --show-progress "$DOWNLOAD_URL" -O "$TEMP_FILE"
elif command -v curl &> /dev/null; then
curl -L --progress-bar "$DOWNLOAD_URL" -o "$TEMP_FILE"
else
echo -e "${RED}错误: 需要 wget 或 curl${NC}"
exit 1
fi
if [ ! -f "$TEMP_FILE" ]; then
echo -e "${RED}下载失败${NC}"
exit 1
fi
FILE_SIZE=$(du -h "$TEMP_FILE" | cut -f1)
echo -e "${GREEN} 下载完成: $FILE_SIZE${NC}"
# 删除旧版本
echo -e "${YELLOW}[3/6] 清理旧版本...${NC}"
if [ -d "$INSTALL_DIR/go" ]; then
echo " 删除旧版本..."
sudo rm -rf "$INSTALL_DIR/go"
fi
# 解压安装
echo -e "${YELLOW}[4/6] 安装 Go...${NC}"
echo " 解压到 $INSTALL_DIR ..."
sudo tar -C "$INSTALL_DIR" -xzf "$TEMP_FILE"
if [ ! -d "$INSTALL_DIR/go/bin" ]; then
echo -e "${RED}安装失败${NC}"
exit 1
fi
echo -e "${GREEN} 安装完成${NC}"
# 配置环境变量
echo -e "${YELLOW}[5/6] 配置环境变量...${NC}"
# 检测 shell
SHELL_NAME=$(basename "$SHELL")
RC_FILE=""
if [[ "$SHELL_NAME" == "bash" ]]; then
RC_FILE="$HOME/.bashrc"
elif [[ "$SHELL_NAME" == "zsh" ]]; then
RC_FILE="$HOME/.zshrc"
else
RC_FILE="$HOME/.profile"
fi
# 检查是否已配置
if ! grep -q "export PATH=.*go/bin" "$RC_FILE" 2>/dev/null; then
echo "" >> "$RC_FILE"
echo "# Go 环境配置" >> "$RC_FILE"
echo "export PATH=\$PATH:$INSTALL_DIR/go/bin" >> "$RC_FILE"
echo "export GOPATH=\$HOME/go" >> "$RC_FILE"
echo "export PATH=\$PATH:\$GOPATH/bin" >> "$RC_FILE"
echo "export GOPROXY=https://goproxy.cn,direct" >> "$RC_FILE"
echo -e "${GREEN} 环境变量已添加到 $RC_FILE${NC}"
else
echo -e " 环境变量已存在"
fi
# 创建 GOPATH 目录
mkdir -p "$HOME/go/bin"
mkdir -p "$HOME/go/pkg"
mkdir -p "$HOME/go/src"
# 验证安装
echo -e "${YELLOW}[6/6] 验证安装...${NC}"
export PATH=$PATH:$INSTALL_DIR/go/bin
export GOPATH=$HOME/go
if command -v go &> /dev/null; then
VERSION=$(go version)
echo -e "${GREEN} Go 版本: $VERSION${NC}"
GOPATH_VAL=$(go env GOPATH)
echo -e "${GREEN} GOPATH: $GOPATH_VAL${NC}"
GOPROXY_VAL=$(go env GOPROXY)
echo -e "${GREEN} GOPROXY: $GOPROXY_VAL${NC}"
else
echo -e "${RED} 验证失败${NC}"
fi
# 清理
echo "[清理] 删除安装包..."
rm -f "$TEMP_FILE"
echo ""
echo -e "${CYAN}==============================================${NC}"
echo -e "${CYAN} Go 安装完成！ ${NC}"
echo -e "${CYAN}==============================================${NC}"
echo ""
echo -e "${YELLOW}请运行以下命令使环境变量生效:${NC}"
echo " source $RC_FILE"
echo ""
echo -e "${YELLOW}然后验证安装:${NC}"
echo " go version"
echo ""
echo -e "${YELLOW}接下来可以启动行情数据服务:${NC}"
echo " cd d:\fs_workspace\market-data-service"
echo " go run ./cmd/server"
echo ""

157
scripts/install-go-windows.ps1
Unescape View File

@ -0,0 +1,157 @@
# Go 1.21 环境安装脚本 (Windows PowerShell)
# 适用于 Windows 10/11
# 以管理员身份运行
param()
$ErrorActionPreference = "Stop"
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host " Go 1.21 环境安装脚本 (Windows) " -ForegroundColor Cyan
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host ""
# 配置参数
$GoVersion = "1.21.6"
$DownloadUrl = "https://go.dev/dl/go$GoVersion.windows-amd64.msi"
$InstallerPath = "$env:TEMP\go$GoVersion.windows-amd64.msi"
# 检查当前是否已安装 Go
Write-Host "[1/5] 检查当前 Go 版本..." -ForegroundColor Yellow
$CurrentGo = Get-Command go -ErrorAction SilentlyContinue
if ($CurrentGo) {
$Version = & go version
Write-Host " 检测到已安装: $Version" -ForegroundColor Green
$response = Read-Host " 是否重新安装? (y/N)"
if ($response -ne 'y' -and $response -ne 'Y') {
Write-Host " 跳过安装" -ForegroundColor Gray
exit 0
}
} else {
Write-Host " 未检测到 Go" -ForegroundColor Gray
}
# 下载安装包
Write-Host "[2/5] 下载 Go $GoVersion ..." -ForegroundColor Yellow
Write-Host " 下载地址: $DownloadUrl" -ForegroundColor Gray
try {
$webClient = New-Object System.Net.WebClient
$webClient.DownloadFile($DownloadUrl, $InstallerPath)
Write-Host " 下载完成: $InstallerPath" -ForegroundColor Green
} catch {
Write-Error "下载失败: $_"
exit 1
}
# 验证下载文件
if (-not (Test-Path $InstallerPath)) {
Write-Error "下载失败，文件不存在"
exit 1
}
$fileSize = (Get-Item $InstallerPath).Length / 1MB
Write-Host " 文件大小: $([math]::Round($fileSize, 2)) MB" -ForegroundColor Gray
# 执行安装
Write-Host "[3/5] 开始安装 Go..." -ForegroundColor Yellow
Write-Host " 正在运行安装程序..." -ForegroundColor Gray
try {
$arguments = "/i `"$InstallerPath`" /passive /norestart"
$process = Start-Process -FilePath "msiexec.exe" -ArgumentList $arguments -Wait -PassThru
if ($process.ExitCode -ne 0) {
Write-Error "安装失败，退出码: $($process.ExitCode)"
exit 1
}
Write-Host " 安装完成" -ForegroundColor Green
} catch {
Write-Error "安装过程出错: $_"
exit 1
}
# 设置环境变量
Write-Host "[4/5] 配置环境变量..." -ForegroundColor Yellow
$GoPath = "$env:USERPROFILE\go"
$GoBin = "$GoPath\bin"
# 检查 GOPATH
$CurrentGoPath = [Environment]::GetEnvironmentVariable("GOPATH", "User")
if (-not $CurrentGoPath) {
[Environment]::SetEnvironmentVariable("GOPATH", $GoPath, "User")
Write-Host " 设置 GOPATH: $GoPath" -ForegroundColor Green
}
# 更新 PATH - 添加 Goin
$UserPath = [Environment]::GetEnvironmentVariable("Path", "User")
if ($UserPath -notlike "*C:\Program Files\Go\bin*") {
$NewPath = $UserPath + ";C:\Program Files\Go\bin"
[Environment]::SetEnvironmentVariable("Path", $NewPath, "User")
Write-Host " 添加 Go 到 PATH" -ForegroundColor Green
}
# 添加 GOPATH\bin 到 PATH
if ($UserPath -notlike "*$GoBin*") {
$UserPath = [Environment]::GetEnvironmentVariable("Path", "User")
$NewPath = $UserPath + ";" + $GoBin
[Environment]::SetEnvironmentVariable("Path", $NewPath, "User")
Write-Host " 添加 GOPATH/bin 到 PATH" -ForegroundColor Green
}
# 设置 GOPROXY
[Environment]::SetEnvironmentVariable("GOPROXY", "https://goproxy.cn,direct", "User")
Write-Host " 设置 GOPROXY: https://goproxy.cn,direct" -ForegroundColor Green
Write-Host " 环境变量配置完成" -ForegroundColor Green
# 验证安装
Write-Host "[5/5] 验证安装..." -ForegroundColor Yellow
# 刷新环境变量
$env:Path = [Environment]::GetEnvironmentVariable("Path", "Machine") + ";" + [Environment]::GetEnvironmentVariable("Path", "User")
# 验证
try {
$GoExePath = "C:\Program Files\Go\bin\go.exe"
if (Test-Path $GoExePath) {
$GoVersionOutput = & $GoExePath version
Write-Host " Go 版本: $GoVersionOutput" -ForegroundColor Green
$GoEnvPath = & $GoExePath env GOPATH
Write-Host " GOPATH: $GoEnvPath" -ForegroundColor Green
$GoProxyVal = & $GoExePath env GOPROXY
Write-Host " GOPROXY: $GoProxyVal" -ForegroundColor Green
} else {
Write-Warning " 未找到 go.exe，可能安装未完成"
}
} catch {
Write-Warning " 验证失败，请重新打开命令提示符后再次运行 'go version'"
}
# 清理
Write-Host "[清理] 删除安装包..." -ForegroundColor Gray
Remove-Item $InstallerPath -ErrorAction SilentlyContinue
Write-Host ""
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host " Go 安装完成！ " -ForegroundColor Cyan
Write-Host "==============================================" -ForegroundColor Cyan
Write-Host ""
Write-Host "请重新打开 PowerShell 或命令提示符，然后运行:" -ForegroundColor Yellow
Write-Host " go version"
Write-Host ""
Write-Host "接下来可以启动行情数据服务:" -ForegroundColor Yellow
Write-Host " cd d:\fs_workspace\market-data-service"
Write-Host " go mod download"
Write-Host " go run ./cmd/server"
Write-Host ""
Write-Host "然后访问管理后台:" -ForegroundColor Yellow
Write-Host " http://localhost:8080/admin"
Write-Host ""
Read-Host "按 Enter 键退出"
Write Preview
Loading…
Cancel
Save