feat: 数据库层方言可插拔(SQLite → 高斯/MySQL/达梦易扩展) - #39
Open
tianling536 wants to merge 6 commits into
Open
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+8
to
+109
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Start openGauss | ||
| run: | | ||
| docker run -d --name opengauss \ | ||
| -e GS_PASSWORD='Staffdeck@123' \ | ||
| -p 5432:5432 \ | ||
| enmotech/opengauss:5.0.0 | ||
| for i in $(seq 1 120); do | ||
| if docker exec opengauss su - omm -c "gsql -d postgres -c 'select 1'" >/dev/null 2>&1; then | ||
| echo "openGauss ready after ${i}s" | ||
| break | ||
| fi | ||
| sleep 2 | ||
| if [ "$i" -eq 120 ]; then | ||
| echo "openGauss failed to become ready" | ||
| docker logs opengauss | ||
| exit 1 | ||
| fi | ||
| done | ||
| docker exec opengauss su - omm -c "gsql -d postgres -c 'select version()'" | ||
|
|
||
| - name: Setup Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Install backend deps | ||
| run: | | ||
| python -m venv .venv | ||
| .venv/bin/pip install -q -e backend[dev] 2>/dev/null || (cd backend && ../.venv/bin/pip install -q -e . && ../.venv/bin/pip install -q pytest pytest-asyncio ruff httpx) | ||
|
|
||
| - name: create_all + dialect smoke on openGauss | ||
| env: | ||
| DATABASE_URL: postgresql+psycopg://postgres:Staffdeck%40123@127.0.0.1:5432/postgres | ||
| STAFFDECK_ROLE: all | ||
| working-directory: backend | ||
| run: | | ||
| set -e | ||
| ../.venv/bin/python - <<'PY' | ||
| from app.db.database import init_db | ||
| init_db() | ||
| print("create_all on openGauss: OK") | ||
| from sqlalchemy import inspect | ||
| from app.db import engine | ||
| idx = {i["name"] for t in ("sessions", "model_configs") for i in inspect(engine).get_indexes(t)} | ||
| assert "uq_sessions_agent_channel_extconv" in idx, "missing session index" | ||
| assert "uq_model_configs_tenant_default" in idx, "missing default-model partial index" | ||
| print("indexes: OK") | ||
| from app.db.dialect import get_dialect | ||
| from sqlmodel import Session | ||
| d = get_dialect(engine.url.get_backend_name()) | ||
| print("dialect:", d.name) | ||
| assert d.acquire_advisory_lock("staffdeck-connector") | ||
| print("advisory lock acquire: OK") | ||
| d.release_advisory_lock("staffdeck-connector") | ||
| print("advisory lock release: OK") | ||
| with Session(engine) as db: | ||
| from app.db.models import User | ||
| db.exec(__import__("sqlmodel").select(User)).all() | ||
| print("basic query: OK") | ||
| PY | ||
|
|
||
| - name: Boot app and API smoke on openGauss | ||
| env: | ||
| DATABASE_URL: postgresql+psycopg://postgres:Staffdeck%40123@127.0.0.1:5432/postgres | ||
| STAFFDECK_ROLE: all | ||
| ULTRARAG_PORT: 5199 | ||
| working-directory: backend | ||
| run: | | ||
| set -e | ||
| ../.venv/bin/python -m uvicorn app.main:app --host 127.0.0.1 --port 5199 & | ||
| APP_PID=$! | ||
| for i in $(seq 1 60); do | ||
| if curl -sf http://127.0.0.1:5199/api/health >/dev/null 2>&1; then echo "healthy after ${i}s"; break; fi | ||
| sleep 1 | ||
| if [ "$i" -eq 60 ]; then echo "app failed to start"; kill $APP_PID; exit 1; fi | ||
| done | ||
| curl -sf http://127.0.0.1:5199/api/health | ||
| TOKEN=$(curl -sf -X POST http://127.0.0.1:5199/api/auth/login \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d '{"tenant_id":"tenant_demo","username":"admin","password":"admin"}' | python3 -c 'import sys,json;print(json.load(sys.stdin)["token"])') | ||
| echo "login: OK" | ||
| AGENTS=$(curl -sf "http://127.0.0.1:5199/api/chat/agents?tenant_id=tenant_demo" -H "Authorization: Bearer $TOKEN") | ||
| echo "agents: $(echo "$AGENTS" | python3 -c 'import sys,json;print(len(json.load(sys.stdin)))')" | ||
| BID=$(curl -sf -X POST http://127.0.0.1:5199/api/enterprise/channels \ | ||
| -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \ | ||
| -d '{"tenant_id":"tenant_demo","agent_id":"agent_30b8f623c6fe445b","channel":"wecom"}' | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])') | ||
| echo "binding: $BID" | ||
| curl -sf -X POST "http://127.0.0.1:5199/api/enterprise/channels/$BID/wecom/credentials" \ | ||
| -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \ | ||
| -d '{"tenant_id":"tenant_demo","bot_id":"smoke_bot","secret":"smoke_secret","corp_id":"smoke_corp"}' -o /dev/null | ||
| echo "credentials(JSON config patch): OK" | ||
| curl -sf "http://127.0.0.1:5199/api/enterprise/channels/$BID/deliveries/days?tenant_id=tenant_demo" \ | ||
| -H "Authorization: Bearer $TOKEN" | python3 -c 'import sys,json;print("day-bucket endpoint:", json.load(sys.stdin)["total_days"], "days")' | ||
| curl -sf "http://127.0.0.1:5199/api/enterprise/knowledge-bases?tenant_id=tenant_demo" \ | ||
| -H "Authorization: Bearer $TOKEN" | python3 -c 'import sys,json;print("knowledge-bases:", len(json.load(sys.stdin)))' | ||
| kill $APP_PID || true |
added 6 commits
July 27, 2026 23:19
- 新增 app/db/dialect.py 方言提供者(协议+注册表+ SQLite/Postgres/Generic 实现),引擎创建走方言 kwargs - 收编 4 处生产方言 SQL:渠道配置补丁与微信运行时补丁 改 ORM 读改写+revision CAS,知识库 BLOB 查询方言分支, 日分桶方言助手(PG 支持 app_timezone) - 两个索引并入 models.py,新库 create_all 即完整 schema - 加 psycopg[binary] 驱动与 PG 冒烟脚本 全量 1006 passed,ruff 零告警
- connector 进程锁走方言提供器:PG 用 advisory lock (常驻会话持锁),SQLite 等保持数据目录文件锁 - 方言文件锁内置 fork 防护(pid 判别,子进程不误解父锁) - 飞书链路传完整 SQLAlchemy URL 替代 DB 文件路径, 放开文件 SQLite 守卫(仅拒绝内存库) 全量 1013 passed,ruff 零告警
main 合入的渠道加固在企微重连清理 disconnect_alerted_at 时用了 SQLite 专有 json_remove;改为读-改-写 + dialect.json_config_remove,与微信 _patch_runtime_config 同一模式,PG/openGauss 可用
tianling536
force-pushed
the
feat/multi-db-clean
branch
from
July 27, 2026 15:27
dd9add0 to
7945867
Compare
Contributor
Author
|
已 rebase 到最新 main(cd64f92，含渠道加固合并），冲突已解决，当前 MERGEABLE。 本次 rebase 要点：
验证：后端全量 1201 passed、ruff 通过。 另注： |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
将数据库层从"SQLite 实现"升级为方言可插拔：默认走 ORM/标准 SQL(所有数据库通用),无法通用的少数点收口到方言提供者接口。新增一种数据库 = 注册一个小适配器(锁/日期/能力声明),业务代码零改动。高斯(openGauss)经 PostgreSQL 协议直接可用,MySQL/达梦按 docstring 中的扩展方式接入。
设计
新模块
backend/app/db/dialect.pyDatabaseDialect协议:engine_kwargs(引擎/连接池配置)、day_bucket(自然日分桶)、JSON 配置读改写、acquire/release_advisory_lock、supports_partial_index能力声明;sqlite → SQLiteDialect、postgresql → PostgresDialect(高斯直接复用,pg_try_advisory_lock)、未知名 →GenericDialect(全部默认实现);cast(..., Date)、数据目录文件锁),因此即使无适配器的新数据库也能先跑起来,适配器只补特性。改造点
get_dialect(backend).engine_kwargs(url)(SQLite 行为逐字不变);api/channels.py配置补丁(SQLite JSON1)→ ORM 读-改-写;adapters/wechat.py::_patch_runtime_config(JSON1+CAS)→ ORM 读-改-写 +config_revision乐观锁 CAS,并发语义三层保住(进程锁+revision CAS+Python 校验),微信全链路测试零回归并加 2 个 CAS 补强测试;api/knowledge.py的CAST AS BLOB→ 方言分支(SQLite 保留,其它走 ORM);app_timezone配置,空值回退服务器本地偏移对齐 SQLite localtime 语义);uq_sessions_agent_channel_extconv、uq_model_configs_tenant_default部分索引sqlite_where="is_default = 1"/postgresql_where="is_default"),新库create_all即完整 schema,与 SQLite 迁移里的幂等CREATE INDEX IF NOT EXISTS共存;psycopg[binary](GaussDB 标准 PG 协议);MySQL/DM 不预装,docstring 注明扩展方式。验证
.github/workflows/gaussdb-smoke.yml(在 x86 CI 一键起 openGauss 容器跑全套方言冒烟),作为后续验证跟进项——openGauss 走 PG 协议,代码内对 advisory lock/JSON 函数均有回退路径,风险集中且可控。后续(登记项,不在本 PR)