Skip to content

Commit 9228f87

Browse files
claudeclaude
committed
goldeneye: add MySQL, generating relations.jsonl and checking the analyze cases
Add a mysql package to goldeneye that reads a live server named by MYSQL_SERVER_URI. MySQL keeps no catalog of its types, functions or operators, so those files stay hand-written; what the server does describe is its data dictionary, so relations.jsonl is generated from information_schema, with names in lower case since MySQL matches them in any case and sqlc's parser lowercases every identifier. The other system schemas are tables rather than views, which the analysis core would hand codegen as models, so they are left out for now. The dialect lives under internal/engine/dolphin, so the command gains a dialect directory distinct from the engine name. The package also checks the analyze_*/mysql cases against the server. Result columns come from the result set's metadata as go-sql-driver reports it; provenance and parameters come from the optimizer trace's expanded_query, which prints each block after resolution and before optimisation, and from the note EXPLAIN leaves for the statements the trace does not expand. Views and derived tables are kept unmerged so a column read through one is still its column. A column read from a table is spelled by its declaration, since the driver hides the length that tells the sizes of TEXT apart. Making sqlc agree: MySQL reports its aggregates as nullable, so the aggregates that are NULL over no rows are marked so in functions.jsonl, which changes the committed analyze_select output; "bigint unsigned" and the other unsigned spellings become aliases of their types so the generated relations resolve; and an analyze_system_catalog case covers querying information_schema. The shared placeholder rewriter learns sqlc.slice and the second count of LIMIT ?, ?. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DTeySVan5NcA3RS6xxdUn
1 parent 7e46437 commit 9228f87

21 files changed

Lines changed: 2046 additions & 48 deletions

File tree

‎.github/workflows/gen.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,35 @@ jobs:
3434
- name: Fail if the committed dialect differs
3535
run: git add -N internal/engine/postgresql && git diff --exit-code --stat -- internal/engine/postgresql
3636

37+
mysql:
38+
name: generate mysql dialect
39+
runs-on: ubuntu-24.04
40+
services:
41+
mysql:
42+
image: mysql:9.1
43+
env:
44+
MYSQL_ROOT_PASSWORD: mysecretpassword
45+
ports:
46+
- 3306:3306
47+
options: --health-cmd "mysqladmin ping -h 127.0.0.1 -pmysecretpassword" --health-interval 10s --health-timeout 5s --health-retries 5
48+
steps:
49+
- uses: actions/checkout@v7
50+
- uses: actions/setup-go@v7
51+
with:
52+
go-version-file: internal/goldeneye/go.mod
53+
check-latest: true
54+
- run: go run ./cmd/goldeneye generate mysql
55+
working-directory: internal/goldeneye
56+
env:
57+
MYSQL_SERVER_URI: root:mysecretpassword@tcp(localhost:${{ job.services.mysql.ports['3306'] }})/mysql
58+
- name: Save results
59+
uses: actions/upload-artifact@v7
60+
with:
61+
name: dialect-mysql
62+
path: internal/engine/dolphin/dialect
63+
- name: Fail if the committed dialect differs
64+
run: git add -N internal/engine/dolphin && git diff --exit-code --stat -- internal/engine/dolphin
65+
3766
clickhouse:
3867
name: generate clickhouse dialect
3968
runs-on: ubuntu-24.04

‎CLAUDE.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -150,14 +150,15 @@ from a live database by `/internal/goldeneye`, a nested module, and its tests
150150
verify the committed files against one byte for byte. The same module checks
151151
the `analyze_*` cases under `/internal/endtoend/testdata/` against what the
152152
database itself reports for them, so a `fixture.sql` next to a case's schema
153-
gives the queries rows to run against. ClickHouse and SQLite have the check
154-
today; engines whose database is not available skip.
153+
gives the queries rows to run against. ClickHouse, MySQL and SQLite have the
154+
check today; engines whose database is not available skip.
155155

156156
```bash
157157
cd internal/goldeneye
158158
go run ./cmd/goldeneye install clickhouse # download the pinned clickhouse binary once
159159
go run ./cmd/goldeneye install sqlite # build the pinned sqlite3 shells once; needs a C compiler
160-
POSTGRESQL_SERVER_URI="postgres://postgres:postgres@127.0.0.1:5432/postgres?sslmode=disable" go test ./...
160+
POSTGRESQL_SERVER_URI="postgres://postgres:postgres@127.0.0.1:5432/postgres?sslmode=disable" \
161+
MYSQL_SERVER_URI="root:mysecretpassword@tcp(127.0.0.1:3306)/mysql" go test ./...
161162
go run ./cmd/goldeneye generate postgresql # rewrite the files after a change
162163
```
163164

‎internal/endtoend/testdata/analyze_select/mysql/stdout.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,8 @@
153153
{
154154
"name": "created",
155155
"type": {
156-
"name": "datetime"
156+
"name": "datetime",
157+
"nullable": true
157158
}
158159
}
159160
],

‎internal/endtoend/testdata/analyze_system_catalog/mysql/exec.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"command": "analyze",
3+
"args": ["--dialect", "mysql", "--schema", "schema.sql", "query.sql"],
4+
"contexts": ["base"]
5+
}

‎internal/endtoend/testdata/analyze_system_catalog/mysql/query.sql‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-- name: ListColumns :many
2+
SELECT table_name, column_name, data_type
3+
FROM information_schema.columns
4+
WHERE table_schema = ?;
5+
6+
-- name: CountTables :one
7+
SELECT count(*) AS total FROM information_schema.tables WHERE table_type = ?;

‎internal/endtoend/testdata/analyze_system_catalog/mysql/schema.sql‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
CREATE TABLE authors (
2+
id BIGINT PRIMARY KEY,
3+
name VARCHAR(255) NOT NULL
4+
);

‎internal/endtoend/testdata/analyze_system_catalog/mysql/stdout.json‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
[
2+
{
3+
"name": "ListColumns",
4+
"cmd": ":many",
5+
"columns": [
6+
{
7+
"name": "table_name",
8+
"type": {
9+
"name": "varchar"
10+
},
11+
"table": "columns"
12+
},
13+
{
14+
"name": "column_name",
15+
"type": {
16+
"name": "varchar",
17+
"nullable": true
18+
},
19+
"table": "columns"
20+
},
21+
{
22+
"name": "data_type",
23+
"type": {
24+
"name": "longtext",
25+
"nullable": true
26+
},
27+
"table": "columns"
28+
}
29+
],
30+
"params": [
31+
{
32+
"number": 1,
33+
"column": {
34+
"name": "table_schema",
35+
"type": {
36+
"name": "varchar"
37+
},
38+
"table": "columns"
39+
}
40+
}
41+
]
42+
},
43+
{
44+
"name": "CountTables",
45+
"cmd": ":one",
46+
"columns": [
47+
{
48+
"name": "total",
49+
"type": {
50+
"name": "bigint"
51+
}
52+
}
53+
],
54+
"params": [
55+
{
56+
"number": 1,
57+
"column": {
58+
"name": "table_type",
59+
"type": {
60+
"name": "enum"
61+
},
62+
"table": "tables"
63+
}
64+
}
65+
]
66+
}
67+
]

‎internal/engine/dolphin/dialect/functions.jsonl‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
{"name":"ASIN","args":[{"type":"double precision"}],"returns":"double precision"}
1818
{"name":"ATAN","args":[{"type":"double precision"}],"returns":"double precision"}
1919
{"name":"ATAN2","args":[{"type":"double precision"}],"returns":"double precision"}
20-
{"name":"AVG","args":[{"type":"any"}],"returns":"any"}
20+
{"name":"AVG","args":[{"type":"any"}],"returns":"any","nullable":true}
2121
{"name":"BENCHMARK","args":[{"type":"int"},{"type":"any"}],"returns":"int"}
2222
{"name":"BIN","args":[{"type":"int"}],"returns":"text"}
2323
{"name":"BIN_TO_UUID","args":[{"type":"binary"}],"returns":"text"}
@@ -130,7 +130,7 @@
130130
{"name":"IS_USED_LOCK","args":[{"type":"text"}],"returns":"bool"}
131131
{"name":"IS_UUID","args":[{"type":"text"}],"returns":"bool"}
132132
{"name":"JSON_ARRAY","args":[{"type":"any","mode":"v"}],"returns":"json"}
133-
{"name":"JSON_ARRAYAGG","args":[{"type":"any"}],"returns":"json"}
133+
{"name":"JSON_ARRAYAGG","args":[{"type":"any"}],"returns":"json","nullable":true}
134134
{"name":"JSON_ARRAY_APPEND","args":[{"type":"text"},{"type":"text"},{"type":"any"},{"type":"any","mode":"v"}],"returns":"json"}
135135
{"name":"JSON_ARRAY_INSERT","args":[{"type":"text"},{"type":"text"},{"type":"any"},{"type":"any","mode":"v"}],"returns":"json"}
136136
{"name":"JSON_CONTAINS","args":[{"type":"text"},{"type":"text"}],"returns":"bool"}
@@ -147,7 +147,7 @@
147147
{"name":"JSON_MERGE_PATCH","args":[{"type":"text"},{"type":"text"},{"type":"text","mode":"v"}],"returns":"json"}
148148
{"name":"JSON_MERGE_PRESERVE","args":[{"type":"text"},{"type":"text"},{"type":"text","mode":"v"}],"returns":"json"}
149149
{"name":"JSON_OBJECT","args":[{"type":"any","mode":"v"}],"returns":"json"}
150-
{"name":"JSON_OBJECTAGG","args":[{"type":"any"},{"type":"any"}],"returns":"json"}
150+
{"name":"JSON_OBJECTAGG","args":[{"type":"any"},{"type":"any"}],"returns":"json","nullable":true}
151151
{"name":"JSON_OVERLAPS","args":[{"type":"text"},{"type":"text"}],"returns":"bool"}
152152
{"name":"JSON_PRETTY","args":[{"type":"text"}],"returns":"any"}
153153
{"name":"JSON_QUOTE","args":[{"type":"text"}],"returns":"text"}
@@ -196,7 +196,7 @@
196196
{"name":"MASTER_POS_WAIT","args":[{"type":"text"},{"type":"int"}],"returns":"int"}
197197
{"name":"MASTER_POS_WAIT","args":[{"type":"text"},{"type":"int"},{"type":"int"}],"returns":"int"}
198198
{"name":"MASTER_POS_WAIT","args":[{"type":"text"},{"type":"int"},{"type":"int"},{"type":"text"}],"returns":"int"}
199-
{"name":"MAX","args":[{"type":"any"}],"returns":"any"}
199+
{"name":"MAX","args":[{"type":"any"}],"returns":"any","nullable":true}
200200
{"name":"MBRCONTAINS","args":[{"type":"any"},{"type":"any"}],"returns":"bool"}
201201
{"name":"MBRCOVEREDBY","args":[{"type":"any"},{"type":"any"}],"returns":"bool"}
202202
{"name":"MBRCOVERS","args":[{"type":"any"},{"type":"any"}],"returns":"bool"}
@@ -209,7 +209,7 @@
209209
{"name":"MD5","args":[{"type":"text"}],"returns":"text"}
210210
{"name":"MICROSECOND","args":[{"type":"time"}],"returns":"int"}
211211
{"name":"MID","args":[{"type":"text"},{"type":"int"},{"type":"int"}],"returns":"text"}
212-
{"name":"MIN","args":[{"type":"any"}],"returns":"any"}
212+
{"name":"MIN","args":[{"type":"any"}],"returns":"any","nullable":true}
213213
{"name":"MINUTE","args":[{"type":"time"}],"returns":"int"}
214214
{"name":"MOD","args":[{"type":"int"},{"type":"int"}],"returns":"int"}
215215
{"name":"MONTH","args":[{"type":"date"}],"returns":"int"}
@@ -277,10 +277,10 @@
277277
{"name":"SQRT","args":[{"type":"double precision"}],"returns":"double precision"}
278278
{"name":"STATEMENT_DIGEST","args":[{"type":"text"}],"returns":"text"}
279279
{"name":"STATEMENT_DIGEST_TEXT","args":[{"type":"text"}],"returns":"text"}
280-
{"name":"STD","args":[{"type":"any"}],"returns":"any"}
281-
{"name":"STDDEV","args":[{"type":"any"}],"returns":"any"}
282-
{"name":"STDDEV_POP","args":[{"type":"any"}],"returns":"any"}
283-
{"name":"STDDEV_SAMP","args":[{"type":"any"}],"returns":"any"}
280+
{"name":"STD","args":[{"type":"any"}],"returns":"any","nullable":true}
281+
{"name":"STDDEV","args":[{"type":"any"}],"returns":"any","nullable":true}
282+
{"name":"STDDEV_POP","args":[{"type":"any"}],"returns":"any","nullable":true}
283+
{"name":"STDDEV_SAMP","args":[{"type":"any"}],"returns":"any","nullable":true}
284284
{"name":"STRCMP","args":[{"type":"text"},{"type":"text"}],"returns":"tinyint"}
285285
{"name":"STR_TO_DATE","args":[{"type":"text"},{"type":"text"}],"returns":"datetime"}
286286
{"name":"ST_AREA","args":[{"type":"any"}],"returns":"double precision"}
@@ -468,7 +468,7 @@
468468
{"name":"SUBSTRING","args":[{"type":"text"},{"type":"int"},{"type":"int"}],"returns":"text"}
469469
{"name":"SUBSTRING_INDEX","args":[{"type":"text"},{"type":"int"},{"type":"int"}],"returns":"text"}
470470
{"name":"SUBTIME","args":[{"type":"time"},{"type":"time"}],"returns":"time"}
471-
{"name":"SUM","args":[{"type":"any"}],"returns":"any"}
471+
{"name":"SUM","args":[{"type":"any"}],"returns":"any","nullable":true}
472472
{"name":"SYSDATE","returns":"datetime"}
473473
{"name":"SYSDATE","args":[{"type":"int"}],"returns":"datetime"}
474474
{"name":"SYSTEM_USER","returns":"text"}
@@ -505,9 +505,9 @@
505505
{"name":"UUID_TO_BIN","args":[{"type":"text"},{"type":"tinyint"}],"returns":"binary"}
506506
{"name":"VALIDATE_PASSWORD_STRENGTH","args":[{"type":"text"}],"returns":"int"}
507507
{"name":"VALUES","args":[{"type":"any"}],"returns":"any"}
508-
{"name":"VARIANCE","args":[{"type":"any"}],"returns":"any"}
509-
{"name":"VAR_POP","args":[{"type":"any"}],"returns":"any"}
510-
{"name":"VAR_SAMP","args":[{"type":"any"}],"returns":"any"}
508+
{"name":"VARIANCE","args":[{"type":"any"}],"returns":"any","nullable":true}
509+
{"name":"VAR_POP","args":[{"type":"any"}],"returns":"any","nullable":true}
510+
{"name":"VAR_SAMP","args":[{"type":"any"}],"returns":"any","nullable":true}
511511
{"name":"VERSION","returns":"text"}
512512
{"name":"WAIT_FOR_EXECUTED_GTID_SET","args":[{"type":"text"},{"type":"int","has_default":true}],"returns":"bool"}
513513
{"name":"WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS","args":[{"type":"text"},{"type":"int","has_default":true},{"type":"text","has_default":true}],"returns":"bool"}

0 commit comments

Comments
 (0)