Run the SQL dialect over the client/server protocol and in the TTY#42
Merged
Conversation
jnidzwetzki
force-pushed
the
jan/integrate_optimizer
branch
2 times, most recently
from
July 21, 2026 17:39
d0b3d08 to
ed0dd04
Compare
jnidzwetzki
force-pushed
the
jan/integrate_optimizer
branch
3 times, most recently
from
July 23, 2026 14:55
fe1a1f0 to
89af84b
Compare
jnidzwetzki
marked this pull request as ready for review
July 24, 2026 19:12
The kernel could only execute plans; the SQL dialect had to be optimized either in the standalone Prolog-embedded binaries (SecondoPL*TTY*) or via the separate Java OptServer. This teaches the server and the TTY to run SQL directly through the embedded optimizer. Protocol: add command level 2 (SQL dialect) to the existing <Secondo> framing. SecondoServer::CallSecondo routes level 2 to the embedded optimizer (before any si->Secondo call, so the optimizer's secondo/2 catalog callbacks are not reentrant), turns the SQL into an executable plan, executes it, and returns the two-element list (plan result) -- a new, backward-compatible shape only level-2 clients see. SecondoInterfaceCS learns to send level 2 and to ask an <OptimizerAvailable/> capability query. Runtime toggle: the server reads Environment/EnableOptimizer (default on; override with SECONDO_PARAM_EnableOptimizer). SecondoMonitor gains a --no-optimizer flag (optimizer builds only) that sets that for the servers it forks. When disabled, level-2 returns ERR_OPTIMIZER_NOT_AVAILABLE and the capability query reports unavailable. Optimizer embedding: initEmbeddedOptimizer/embeddedSqlToPlan/ embeddedOptimizerUseDatabase (UserInterfaces/SecondoPL.cpp) start SWI-Prolog and load the optimizer in-process, sharing the host interface (no second SMI environment) and loading the open database's schema on demand. Preload library(error) so restricted autoloading no longer spams must_be/2 errors while the optimizer loads. TTY: a leading "sql" or "select" token (case-insensitive, prefix optional) is optimized -- over the network in SecondoCS, in-process in SecondoTTYBDB -- and the generated plan is shown. Both report optimizer availability at startup. Consolidation/cleanup: the regular TTY now runs SQL, so retire the hybrids -- remove SecondoPLCS/SecondoPLTTYCS and the -pl/-pltty logic from SecondoCS, and remove SecondoPLTTY/SecondoPLTTYNT and SecondoPLTTYMode (SecondoTTYBDB replaces -pltty). Keep the -pl raw Prolog shell. Build/CI: the default optimizer build now also generates the Optimizer/optimizer program the embedded optimizer consults at runtime (gitignored). A new CI step optimizes and executes an SQL query over the network deterministically.
Since the SECONDO kernel is now able to drive the Optimizer directly, and the JavaGUI is using this code path, this commit cleans up the unneeded data and removes the custom SWI prolog build on OS X.
So far, the client had to determine which language level should be used. This leads to different language detection implementations in C and Java. This commit introduces server-side auto-detection and unifies the existing approaches in one central place.
jnidzwetzki
force-pushed
the
jan/integrate_optimizer
branch
from
July 25, 2026 14:40
9ccd4d9 to
e25a390
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The kernel could only execute plans; the SQL dialect had to be optimized either in the standalone Prolog-embedded binaries (SecondoPLTTY) or via the separate Java OptServer. This teaches the server and the TTY to run SQL directly through the embedded optimizer.
Protocol: add command level 2 (SQL dialect) to the existing framing. SecondoServer::CallSecondo routes level 2 to the embedded optimizer (before any si->Secondo call, so the optimizer's secondo/2 catalog callbacks are not reentrant), turns the SQL into an executable plan, executes it, and returns the two-element list (plan result) -- a new, backward-compatible shape only level-2 clients see. SecondoInterfaceCS learns to send level 2 and to ask an capability query.
Runtime toggle: the server reads Environment/EnableOptimizer (default on; override with SECONDO_PARAM_EnableOptimizer). SecondoMonitor gains a --no-optimizer flag (optimizer builds only) that sets that for the servers it forks. When disabled, level-2 returns ERR_OPTIMIZER_NOT_AVAILABLE and the capability query reports unavailable.
Optimizer embedding: initEmbeddedOptimizer/embeddedSqlToPlan/ embeddedOptimizerUseDatabase (UserInterfaces/SecondoPL.cpp) start SWI-Prolog and load the optimizer in-process, sharing the host interface (no second SMI environment) and loading the open database's schema on demand. Preload library(error) so restricted autoloading no longer spams must_be/2 errors while the optimizer loads.
TTY: a leading "sql" or "select" token (case-insensitive, prefix optional) is optimized -- over the network in SecondoCS, in-process in SecondoTTYBDB -- and the generated plan is shown. Both report optimizer availability at startup.
Consolidation/cleanup: the regular TTY now runs SQL, so retire the hybrids -- remove SecondoPLCS/SecondoPLTTYCS and the -pl/-pltty logic from SecondoCS, and remove SecondoPLTTY/SecondoPLTTYNT and SecondoPLTTYMode (SecondoTTYBDB replaces -pltty). Keep the -pl raw Prolog shell.
Build/CI: the default optimizer build now also generates the Optimizer/optimizer program the embedded optimizer consults at runtime (gitignored). A new CI step optimizes and executes an SQL query over the network deterministically.