qBERT

qBERT is a text generation system built around BERT, with a second model helping choose continuations that stay on topic. BERT suggests tokens that fit their surroundings; a SentenceTransformer compares the meaning of possible continuations with the prompt and recent output. Sampling rules combine these signals into a streaming response.

BERT normally fills gaps such as The cat sat on the [MASK]. qBERT turns that ability into continuation generation by repeatedly masking the next output position. It also drafts possible future text so BERT can use both left and right context when reconsidering that position. The custom work in qBERT.py is primarily this inference algorithm; model training lives in separate scripts.

Component	ModelConfig default	Role
Masked language model	bert-base-cased	Propose tokens from surrounding text
Sentence encoder	all-MiniLM-L6-v2	Score semantic fit of candidate continuations

The chat applications override these defaults through config/model_config.yaml, which currently points to local dialogue-trained checkpoints under models/. Calling ModelConfig() directly uses the pretrained defaults above.

The configured generator and tokenizer are models/qbert-hermes-v4; the semantic scorer is models/qbert-st-hermes-v4. These local checkpoints must be present to use the configured chat applications.

Generation advances along a token frontier. At each position, qBERT places a mask, asks BERT for candidates, evaluates their semantic and local fit, samples from the fused distribution, commits the selected token, and advances the frontier. A speculative masked-future canvas supplies provisional right context, while iterative denoising can revisit the completed draft with full left/right context.

Architecture in one sentence: qBERT is an iterative masked-token generator that combines BERT proposals, speculative bidirectional lookahead, semantic energy scoring, local fluency priors, adaptive sampling, memory, and revision.

Repository Map

qBERT.py                    Core qBERT generation framework
CHATbert.py                 Manual terminal chat interface
autoCHATbert.py             Autonomous chat with a tool-calling API teacher
discord_qbert.py            Discord mention/reply and DM chatbot
run_discord.cmd             Windows Discord launcher
api_client.py               Provider-neutral API transport and tool loop
api_schema.py               Typed API requests, responses, and tools
gui.py                      Local browser interface
gui/                        Browser interface assets
reasoning.py                Shared assistant reasoning parser
trainer.py                  Structure-aware BERT MLM fine-tuning
trainer_st.py               SentenceTransformer dialogue-pair fine-tuning
trainer_fusion.py           Candidate fusion-ranker training
validate_trainer.py         Checkpoint validation helpers
tests/benchmark_qbert.py    Generation and ablation benchmark
tests/audit_dataset.py      Dataset structure and length auditor
tests/test_*.py             Architecture, Discord, stopping, teacher, and GUI tests
config/                     Runtime, trainer, and prompt configuration
requirements.txt            Shared pinned dependencies, including API and Discord SDKs
.env.example                Discord token configuration example
assets/                     Static assets

Generated checkpoints, tokenizer files, TensorBoard runs, logs, caches, and chat history are expected to live under directories such as qbert-hermes*/, qbert-st-hermes-v4/, checkpoints/, logs/, cache/, and data/.

Architecture

qBERT.py exposes three main config/model layers:

• GenerationConfig: sampling, coherence, memory, denoise, and device settings.
• ModelConfig: BERT model, tokenizer, SentenceTransformer, and attention backend.
• ParallelBERTGenerator: the generation system that coordinates proposal, lookahead, semantic scoring, priors, memory, sampling, and revision.

Component topology:

prompt + role framing
        |
        v
tokenized generation canvas
        |
        +--> BERT masked-LM proposals
        +--> speculative masked-future draft
        +--> SentenceTransformer semantic critic
        +--> sequence-memory boosts / repetition / entity / style priors
        +--> optional Markov transition prior
        |
        v
normalized score fusion
        |
        v
coherence homeostasis + adaptive temperature + nucleus sampling
        |
        v
commit token -> stream text -> advance frontier
        |
        v
optional masked-token revision passes -> final response

For each generated position, qBERT runs this lifecycle:

1. Attend to the committed prefix and a bounded window of future positions.
2. When needed, draft a future block of 12 tokens using two rounds of confidence-ordered mask filling (the defaults).
3. Mask the current position and obtain BERT's strongest valid candidates. The default shortlist starts at 32 tokens and can expand when coherence is low.
4. Expand the leading eight candidates into distinct continuations of up to eight tokens, then score their semantic fit. Other candidates use the shared speculative future when available.
5. Apply sequence-memory boosts, repetition penalties, entity and style biases, and the optional Markov transition prior.
6. Normalize and fuse the proposal and semantic scores, then apply adaptive temperature and nucleus filtering.
7. Sample one token. If it differs enough from the speculative token in BERT's input-embedding space, discard the remaining draft and regenerate it next step.
8. Commit, format, and stream the selected token, then advance one position.

Despite the class name ParallelBERTGenerator, the main output is committed one token at a time. Draft positions and candidate branches provide the parallel work. The coherence controller also updates during this loop and can stop a response when candidate scores stay low.

The public methods are:

• generate_stream(initial_text, num_tokens): yields formatted draft token text.
• generate_with_denoise_stream(initial_text, num_tokens): yields structured events for draft, denoise start, denoise edits, and denoise completion.
• generate(initial_text, num_tokens): returns the final string.

Architectural Components

BERT Proposal Engine

The generator loads AutoModelForMaskedLM and uses BERT's MLM logits at the active frontier. Dialogue structure tokens provide turn landmarks; the raw probability of the configured [EOT] target supplies the end-of-turn signal. Sampling candidates exclude structural tokens and receive a token-class prior before entering the fusion stage.

Masked-Future Self-Speculation

The generation canvas keeps a bounded attended fringe of future [MASK] positions. qBERT samples a short block through confidence-ordered refinement, committing confident positions first and filling the remaining positions in later rounds. The resulting token IDs supply provisional right context to BERT for the careful next-token decision. A disagreement with the draft triggers a refresh when token embedding similarity falls below spec_keep_sim.

Semantic Critic

qBERT builds short hypothetical continuations and encodes them with the loaded SentenceTransformer, which may be a base or dialogue-trained model. Candidate embeddings are evaluated against a context representation composed from the recent tail, the original prompt anchor, and an extrapolated semantic waypoint. The waypoint projects the anchor-to-tail trajectory forward to bias selection toward that direction. This is an embedding-space heuristic, not a guarantee of meaningful progress, factual correctness, or a complete answer to the prompt.

Important controls:

• tau_coherence: semantic score sharpness.
• coherence_anchor_alpha: how strongly the user prompt anchors coherence.
• coherence_anchor_taper: starts anchor pressure stronger, then eases it.
• use_coherence_homeostasis: dynamically tightens or relaxes semantic pressure based on a running coherence EMA.

Candidate Score Fusion

Candidate scores are shaped by:

• BERT logits.
• Semantic coherence scores.
• An optional fusion gate (use_learned_gate; requires trained weights supplied externally to be a learned policy).
• A trained per-candidate residual ranker (use_fusion_ranker).
• Markov transition scores from the committed left token and speculative right token.
• Sequence-memory boosts.
• Left-to-right tokenization/punctuation bias.
• Entity reuse bias.
• Early-token style penalties for punctuation-heavy starts.
• Sliding-window repetition penalty.
• Adaptive temperature derived from distribution entropy.
• Nucleus sampling governed by p_nucleus.

BERT logits and semantic scores are normalized across the active candidate set before fusion. The remaining signals enter as multiplicative or log-space preferences, producing a single distribution for token selection.

The residual fusion ranker consumes six aligned candidate features: BERT score, semantic score, Markov score, repetition count, entity/style score, and response position. trainer_fusion.py learns this policy from human assistant tokens and stores it as a compact independent checkpoint.

Coherence Homeostasis

qBERT maintains an exponential moving average of the mean candidate coherence score, rather than a correctness score for the selected output. This state continuously regulates semantic sharpness, prompt-anchor strength, nucleus breadth, and quality-based stopping. Higher candidate coherence lets sampling return toward its configured breadth; lower coherence tightens guidance.

Memory Systems

qBERT combines two generator-local memories:

• SemanticMemoryCache: compresses the 4D hidden-state memory matrix and can blend prior state back into a future generation with memory_alpha and memory_beta. In the current implementation, this matrix does not feed back into token logits or candidate selection; see the limitations below.
• SequenceMemoryCache: stores recent generated sequences per prompt key and boosts candidates that resemble similar prior continuations.

Autonomous mode extends this layer with persistent vector and BM25 retrieval for reflections, teacher attempts, and post-change outcomes.

Revision and Denoising

The draft generator can be followed by a denoise pass:

• use_denoise: enables post-draft denoising.
• denoise_passes: number of revision passes.
• denoise_pattern: sequential, random, entropy, span, or checkerboard.
• denoise_batch_size: number of positions masked per BERT batch (0 means all).
• denoise_accept_margin: incumbent bonus that discourages replacing a draft token; it is a strict improvement threshold only in greedy mode.
• stream_denoise: whether denoise edits are emitted as stream events.

The revision layer supports entropy-prioritized, sequential, random, span, and checkerboard schedules. With positive denoise_mh_temperature, the implementation samples a proposal and accepts it probabilistically based on its score relative to the incumbent, the margin, and temperature. Otherwise, it uses greedy selection with an improvement threshold. Each batched row masks one position, so BERT can inspect both sides of the completed draft. Accepted edits are applied at batch boundaries and can be emitted as replacement-text events. The Gibbs backedit path supplies an additional configurable revision schedule.

Current Implementation Limits

• Custom hidden-state memory has no selection path. _bidir() applies additional attention projections to the matrix M, and its result is stored back in M. Neither that result nor the compressed semantic memory enters BERT's logits or the candidate scorer. Right-context influence comes from the speculative token IDs passed into BERT. Sequence-memory text boosts do affect scoring.
• Some added layers start untrained. The custom attention projections, memory compressor/decompressor, optional fusion gate, and optional semantic alignment projection are randomly initialized here, with no training or checkpoint loading for those layers in this file. The optional residual fusion ranker has an explicit checkpoint loader.
• Context is bounded. The generation canvas is capped at 510 token positions, including the tokenized prompt. Requested output is reduced to the remaining capacity. The code does not continually slide the context to generate beyond this limit.
• Semantic similarity is a proxy. A continuation can remain on topic while being repetitive, incorrect, or unhelpful. The coherence stop is also based on candidate scores, not an independent assessment of answer quality.
• Lookahead costs inference time. Drafting, candidate branching, sentence encoding, and optional revision add work beyond the main BERT pass for each committed token. Quality and throughput need benchmarking with the actual checkpoints and settings in use.

Installation

Create a virtual environment, then install the pinned requirements:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Run all commands below from the repository root, using the activated virtual environment or .venv\Scripts\python.exe explicitly. API provider SDKs and discord.py are included in requirements.txt; there are no separate provider or Discord requirements files.

The checked-in requirements.txt targets Python 3.12 and CUDA 12.1 PyTorch wheels. If you need CPU-only PyTorch or a different CUDA version, install the matching PyTorch build first, then install the remaining packages.

Browser Interface

Start the local browser interface:

python gui.py

It serves gui/ at http://127.0.0.1:8642/ and opens the browser. Use python gui.py --port 8643 --no-browser to choose another port and open the page yourself. The interface exposes chat, live generation settings, model selection, and training/evaluation workflows. It shares the local model configuration with the terminal applications.

The GUI's benchmark launcher (including the pipeline's final benchmark stage) still references the former root-level benchmark_qbert.py. Until that launcher is updated, run benchmarks from the repository root with python -m tests.benchmark_qbert as shown below.

Manual Chat

Start the qBERT terminal interface:

python CHATbert.py

CHATbert.py wraps each prompt as:

[USER] <your input>
[ASSISTANT]

Then it calls generate_with_denoise_stream().

Useful commands:

/help                         show commands
/quit                         exit
/config                       print GenerationConfig
/clear                        clear the terminal
/device cpu|cuda              rebuild on CPU or CUDA
/tokens <n>                   set generated token count
/stream                       toggle token streaming
/denoise                      toggle post-generation denoise
/denoise_passes <n>           set denoise passes
/denoise_pattern <pattern>    sequential|random|entropy|span|checkerboard
/denoise_batch_size <n>       denoise batch size, 0 = all positions
/bert_model <name_or_path>    swap BERT/tokenizer
/sentence_model <name_or_path> swap SentenceTransformer
/attn_impl <type>             eager|sdpa|flash_attention_2
/<config_field> <value>       update any GenerationConfig field

Conversation and system updates are appended to logs/conversation_history.jsonl and logs/system_updates.jsonl.

Discord Chat

discord_qbert.py runs the local qBERT model as a dedicated Discord bot. It responds to an explicit @qBERT mention or a reply to its own message, including replies with Discord's reply ping turned off. Ordinary channel messages, webhooks, and role/everyone mentions do not trigger it. Other bots can mention or reply to qBERT; its own messages are always ignored. Set allow_bot_messages: false in config/discord.yaml to disable other bots. Direct messages are enabled by default and respond to ordinary text without a mention or reply. Set allow_dms: false in config/discord.yaml to disable them.

The lightweight adapter ports mention cleanup, referenced-message lookup, typing indicators, and chunked delivery from defaultmodeAGENT's discord_bot.py, discord_utils.py, and adapters/discord_adapter.py. It runs qBERT directly; it does not require Ollama or the larger agent runtime.

1. Install the shared dependencies into the existing virtual environment:

   .venv\Scripts\python.exe -m pip install -r requirements.txt

2. Create a dedicated application and bot in the Discord Developer Portal. Enable Message Content Intent on its Bot page so it can read unpinged replies. Use the OAuth2 URL Generator with the bot scope to invite it with View Channels, Send Messages, and Read Message History permissions. Add Send Messages in Threads if it will chat in threads. See Discord's gateway intents and channel permissions documentation.

3. Copy .env.example to .env (or add this entry to an existing .env) and set the dedicated bot token:

   DISCORD_TOKEN_QBERT=your_bot_token

   The launcher reads only this repository's .env or the configured environment variable. It does not copy credentials from the other project. To use another variable, set token_env in config/discord.yaml or pass --token-env NAME.

4. Double-click run_discord.cmd, or run:

   .venv\Scripts\python.exe discord_qbert.py

   The console reports model loading and the bot identity when connected. Close it or press Ctrl+C to stop. A generation already running may need to finish before the Python process exits.

Mention the bot with a text question, then reply to its answer to continue. config/discord.yaml controls output length (96 tokens by default), optional guild/channel allowlists, DMs, and reply-history depth. For thread allowlists, use the thread's channel ID. Empty allowlists allow any installed guild/visible channel; they do not cause unsolicited responses.

Models and generation parameters come from config/model_config.yaml. A single inference worker loads the models once and serializes generation; the Discord gateway stays responsive and at most four requests are active/queued by default. In guild channels the adapter follows up to six messages in the explicit reply chain; it does not read surrounding channel chat. In DMs it reads up to six recent messages strictly before the current message from that DM's history, including qBERT's replies, so normal follow-ups retain context without Discord reply references. DM history is scoped to that channel and its participant; guild/channel allowlists apply to guild traffic, while allow_dms independently controls direct messages. Discord supplies the DM history after bot restarts. history_messages: 0 disables history in both modes. Oldest turns are dropped to fit, reserving output/hidden-reasoning space inside qBERT's 510-token canvas. With the default 96-token output budget, up to 414 tokens remain for system framing, the current message, and retained history combined. An oversized current message keeps its most recent tokens. Attachments are not downloaded or interpreted. Empty mentions receive a usage hint. Replies are split below Discord's message length limit with all mention pings disabled.

Shared sequence/semantic memory and the prompt-learning Markov prior are disabled for this adapter so unrelated channels do not supply each other's generated context. The shared model configuration file is unchanged. Hidden reasoning is not posted; when denoising is enabled the final revised answer is sent. The wrapper logs connection/message IDs and errors, without adding a chat transcript store or running autonomous teacher tuning.

Responses use the trained [EOT] target to end naturally. Discord system, history, and user turns include [EOT] boundaries matching the training format; the current assistant turn is left open. Registered role-framed prompts are encoded without inserting BERT's [SEP] between the prompt and continuation.

Stopping settings live in the qbert section of config/model_config.yaml:

eot_token: "[EOT]"                 # match trainer.eot_token / the checkpoint tokenizer
eot_stop_threshold: 0.5
eot_min_tokens: 1                  # permits brief replies
eot_probe_at_sentence_end: true

The normal next-position BERT distribution is checked for EOT probability before special-token filtering and candidate scoring. Only the configured stop target counts; other role tokens do not end the answer. A missing stop token does not fall back to UNK or all role tokens, and Discord startup warns about it. Separate reasoning uses [ANSWER] as its own phase-transition target.

Speculative right context can suppress EOT by implying that more text follows. With eot_probe_at_sentence_end, generation also checks the committed prefix plus one mask, without the speculative future, after ., !, or ? (including closing quotes/brackets). It stops only if EOT meets the configured probability threshold and the minimum length is satisfied. This probe is punctuation-gated because the current checkpoint also assigns high EOT probability to some unfinished prefixes without right context. It can favor short, single-sentence answers; it is a stopping heuristic, not a measure of answer completeness. Disable the probe to retain only the normal canvas-based check; a zero eot_stop_threshold disables both token-based checks.

The token budget remains an upper bound. Coherence-based stopping and context capacity remain fallback termination paths. Discord's console logs distinguish eot, quality, length, and capacity, plus the signal source, generated token count, and stop probability when available. Stopping tokens are never posted.

Offline checks do not connect to Discord or require a bot token:

.venv\Scripts\python.exe discord_qbert.py --check
.venv\Scripts\python.exe discord_qbert.py --prompt "Hello, who are you?"
.venv\Scripts\python.exe -m unittest discover -s tests -p "test_discord_qbert.py" -v
.venv\Scripts\python.exe -m unittest discover -s tests -p "test_qbert_stopping.py" -v

--check and --prompt load the same local checkpoints as the bot; the unit tests mock model/Discord I/O and require no GPU or Discord credentials.

Autonomous Chat

Start the autonomous qBERT/Ollama loop:

python autoCHATbert.py

The loop is:

seed input
  -> qBERT generates a response
  -> a configurable API teacher retrieves related attempts from persistent BM25 memory
  -> the teacher calls adjust_hyperparameters with a validated sparse patch
  -> confident tool calls update GenerationConfig and return an application receipt
  -> Ollama generates a reflection and stores it in hybrid memory
  -> Ollama responds to qBERT using recent history and relevant reflections
  -> the teacher records the post-change reflection/outcome
  -> qBERT receives that response as the next turn
  -> repeat

Runtime pieces:

• OllamaChat: owns the Ollama client, prompt templates, message history, and hybrid reflection memory.
• TeacherAgent: uses the provider-neutral APIClient tool loop for tuning. Current tunable fields include base_temperature, top_k, context_window, tau_coherence, p_nucleus, repetition_penalty, repetition_window, coherence_anchor_alpha, and the denoise controls.
• HybridMemoryDB: combines vector retrieval from VectorIndex with persistent BM25 lexical search from PersistentBM25Index.
• VectorIndex: stores reflection embeddings in data/vector_index.pkl.
• PersistentBM25Index: stores reflection, teacher-attempt, and post-change records in data/hybrid_memory_bm25_index.json.

autoCHATbert.py expects a local Ollama server and the configured models. Current defaults are loaded from config/model_config.yaml: gemma4:12b for chat, nomic-embed-text for embeddings, and http://127.0.0.1:11434 for the Ollama host when no host is configured. The teacher follows the chat model when teacher.provider is ollama and teacher.use_ollama_model is true. Prompt templates live in config/prompts.yaml.

The teacher client and typed schemas are ported from defaultmodeAGENT/agent/api_client.py and api_schema.py. The chat transports, media support, and bounded tool loop are retained. Source-project SQL logging, embedding preparation/compression, and agent runtime dependencies are excluded; API observations go to logs/api_calls.jsonl, while teacher attempts and applied changes retain the existing history/BM25 records. This observation log contains prompts and responses; it does not serialize API credentials.

Install the shared dependencies in the project virtual environment:

.venv\Scripts\python.exe -m pip install -r requirements.txt

openai supplies the transport for OpenAI, OpenRouter, Ollama, llama-server, vLLM, and Unsloth. The Anthropic and Google GenAI SDKs for native Anthropic and Gemini transport are also included in requirements.txt. The rest of autonomous chat, reflections, and embeddings still use Ollama.

For a separate teacher endpoint, edit config/model_config.yaml, for example:

teacher:
  provider: "llama-server"
  api_base: "http://127.0.0.1:8888"
  model_name: "your-tool-capable-model"
  use_ollama_model: false
  min_confidence: 0.6
  max_rounds: 4
  max_output_tokens: 2048

Providers are openai, openrouter, ollama, llama-server, vllm, unsloth, anthropic, and gemini. Credentials come from <PROVIDER>_API_KEY; endpoint and model defaults can use <PROVIDER>_API_BASE and <PROVIDER>_MODEL_NAME (uppercase, replacing hyphens with underscores). .env is loaded locally and ignored by Git. Hosted providers require their key; Ollama and llama-server use local placeholder keys. Explicit teacher YAML endpoint/model values take precedence; when no Ollama teacher endpoint is set, it inherits the chat host.

The selected model must support tool calling. The teacher submits at most one validated adjustment per evaluation. Unknown fields, incorrect types, and out-of-range values are rejected before mutation; the model can correct a rejected call within the round budget. Low-confidence calls and empty patches leave configuration unchanged. Plain JSON in response text is never applied. The tool receipt reports actual changes and current adjustable configuration. Updates affect the running generator and do not rewrite YAML or model weights.

Run the offline transport/tool integration checks without loading checkpoints:

.venv\Scripts\python.exe -m unittest discover -s tests -p "test_teacher_tools.py" -v

During the loop:

• Ctrl+C toggles pause/resume.
• Ctrl+D exits.
• At the seed prompt, /help, /models, and /quit are available before starting the loop.
• While paused, commands include /help, /models, /ollama <model>, /ollama_host <host>, /device <cpu|cuda>, /tokens <n>, /stream, /denoise, /denoise_passes <n>, /bert_model <name>, /sentence_model <name>, and /<config_field> <value>.

Teacher updates shape the live generator throughout the session. Attempts, reflections, and post-change outcomes are indexed into persistent BM25 memory.

Configuration

Main config files:

• config/model_config.yaml: model/generation defaults used by terminal chat, autonomous chat, the browser interface, and the Discord adapter, plus Ollama, teacher, and memory defaults.
• config/discord.yaml: Discord token environment variable, model-config path, response length, reply/DM history, bot-message and DM toggles, queue limit, guild/channel allowlists, and system prompt.
• config/prompts.yaml: Ollama system, reflection, response, and teacher prompts.
• config/trainer.yaml: shared dataset/training defaults for trainer.py and trainer_st.py.

config/model_config.yaml is split into these runtime sections:

• model_defaults: BERT checkpoint, tokenizer, SentenceTransformer, and attention implementation.
• qbert: GenerationConfig fields. device: null means use qBERT's automatic CUDA/CPU default, and phrase_window: null derives from context_window.
• autochat and autochat_session: loop timing, context limits, token count, streaming, and model type.
• ollama: chat model, embedding model, host, history length, and history path.
• teacher: provider, endpoint/model policy, tool-round and output limits, confidence threshold, short in-process history length, and related-attempt retrieval count.
• memory: vector/BM25 file paths, embedding dimension, backfill limit, and hybrid retrieval weights.

Key GenerationConfig fields (Python defaults; the YAML runtime configuration can override them, including the denoise pass count and pattern):

Field	Default	Purpose
max_length	512	Extra capacity allocated for the hidden-state matrix, capped at 512 total positions
top_k	32	Candidate tokens kept from MLM logits
base_temperature	0.7	Sampling temperature
p_nucleus	0.9	Top-p sampling threshold
context_window	256	Window used by bidirectional memory attention
phrase_window	max(32, context_window // 2)	Context tail for semantic scoring
tau_coherence	1.7	Coherence score shaping
repetition_penalty	0.8	Downweight repeated recent tokens
repetition_window	64	Repetition lookback
coherence_anchor_alpha	0.5	Prompt-anchor strength
use_coherence_homeostasis	true	Dynamic coherence pressure
use_semantic_memory	true	Enable compressed hidden-state memory
use_sequence_memory	true	Enable recent-sequence retrieval boosts
use_mask_future	true	Attend a bounded fringe of future mask positions
mask_future_window	16	Attended mask fringe beyond the active draft block
use_self_speculation	true	Draft provisional right context
spec_block_n	12	Number of tokens in each speculative draft block
spec_refine_rounds	2	Confidence-ordered draft filling rounds
use_semantic_blocks	true	Score candidate-specific continuation blocks
semantic_block_size	8	WordPiece horizon for semantic trajectories
semantic_block_candidates	8	Leading candidates expanded into distinct branches
coherence_score_mode	trajectory	Query/continuation semantic energy mode
use_fusion_ranker	false	Apply the trained residual candidate ranker
use_learned_gate	false	Enable the optional fusion gate
use_markov_prior	false	Apply token transition counts during selection
use_denoise	false	Enable post-draft denoise
denoise_pattern	sequential	Denoise position order
denoise_batch_size	32	Denoise BERT batch size
gibbs_every_M	0	Backedit scheduling interval
gibbs_span_L	0	Backedit span length
eot_token	[EOT]	Trained end-of-turn target; match the checkpoint tokenizer
eot_stop_threshold	0.5	Raw stop-token probability needed to end a turn; 0 disables token-based stopping
eot_min_tokens	1	Minimum generated tokens before stopping
eot_probe_at_sentence_end	true	Also check EOT without speculative right context after sentence punctuation
quality_stop_patience	12	Consecutive low-coherence steps before stopping; 0 disables
reasoning_mode	answer_only	Assistant trace policy: answer-only, separate, or inline
max_reasoning_tokens	96	Hidden reasoning budget in separate mode
reasoning_stop_threshold	0.5	Probability threshold for the [ANSWER] transition

num_tokens, passed to a generation method, sets the maximum output length; natural EOT or coherence stopping can end a response earlier. The generator also limits that request to 510 - tokenized_prompt_length. max_length controls matrix allocation rather than directly setting the output length; keep it large enough for the requested generation.

Training Workflow

Audit a dataset before training:

python -m tests.audit_dataset `
  --dataset_name json `
  --data_files "hf://datasets/NousResearch/Hermes-3-Dataset/hermes-3-dataset.jsonl" `
  --format sharegpt `
  --model_name_or_path bert-base-cased

Fine-tune the BERT masked LM:

python trainer.py --config config/trainer.yaml

The trainer supports sharegpt, qa, and text formats. It inserts structure tokens such as [SYSTEM], [USER], [ASSISTANT], and [EOT], then applies a structure-aware MLM collator with:

• separate assistant/context masking rates,
• configurable assistant-focused masking,
• boundary masking around [ASSISTANT] and [EOT],
• [EOT] prediction as a learned turn-completion signal,
• contiguous fringe spans aligned with the masked-future inference canvas.

Reasoning phases

All training paths share the same reasoning parser. Tagged <thinking>, <think>, and <reasoning> spans are normalized before they reach the model:

• answer_only removes the tagged span and trains the visible answer.
• separate renders [THINK] reasoning [ANSWER] answer and records reasoning positions independently from answer positions.
• inline preserves the original assistant text.

In separate mode the masked LM learns [ANSWER] as a phase boundary and uses an independent reasoning masking rate. At inference qBERT first generates a hidden [THINK] phase, transitions when [ANSWER] probability crosses the configured threshold, then streams only the visible response until [EOT].

The SentenceTransformer trains on visible answer text, while the fusion ranker selects gold candidates from answer positions. The dataset auditor reports tag counts, malformed traces, and reasoning records without visible answers.

Fine-tune the SentenceTransformer coherence model:

python trainer_st.py --config config/trainer.yaml

The semantic trainer extracts prompt/response and partial-tail/continuation pairs, optimizes them with MultipleNegativesRankingLoss, and evaluates the result against balanced genuine and mismatched dialogue pairs. The resulting embedding model supplies qBERT's semantic energy scores during generation.

Train the candidate fusion ranker:

python trainer_fusion.py --max_examples 2000 --positions_per_example 2

The ranker builds candidate sets around human assistant tokens, extracts the same features used by _fuse_sample(), and learns a residual 32-way selection policy. Its checkpoint is written to models/qbert-fusion-v1.pt.

Validate a fine-tuned checkpoint:

python validate_trainer.py --model_dir ./models/qbert-hermes-v4

Then load trained artifacts in CHATbert.py:

/bert_model ./models/qbert-hermes-v4
/sentence_model ./models/qbert-st-hermes-v4

Benchmark and Ablations

Run the standard 20-prompt architecture comparison:

python -m tests.benchmark_qbert `
  --variants bert_only,token_semantic,legacy_semantic,with_markov,full,fusion_ranker `
  --num_tokens 32

The benchmark uses identical per-prompt random seeds across variants and writes JSONL generations plus a JSON summary under benchmarks/. It records semantic similarity, expected-keyword recall, distinct-1/2, repeated-token runs, punctuation ratio, latency, throughput, peak VRAM, and stop reasons. The prompt suite lives in config/benchmark_prompts.jsonl and can be replaced with another JSONL file through --prompts.

Run all focused tests, including architecture, stopping, Discord routing, teacher tool calls, and the GUI backend:

python -m unittest discover -s tests -v

The auditor and benchmark live under tests/; invoke them with python -m from the repository root so imports of the root-level model/trainer modules resolve correctly. Unit tests mock model or Discord I/O as needed; the dataset auditor, benchmark, and checkpoint validator require their actual data/models.

Programmatic Use

from qBERT import GenerationConfig, ModelConfig, ParallelBERTGenerator

config = GenerationConfig(max_length=128, top_k=32, use_denoise=False)
model_config = ModelConfig(
    bert_model_name="./models/qbert-hermes-v4",
    tokenizer_name="./models/qbert-hermes-v4",
    sentence_transformer_name="./models/qbert-st-hermes-v4",
)

generator = ParallelBERTGenerator(config, model_config=model_config)
prompt = "[USER] hello [EOT]\n[ASSISTANT] "
text = generator.generate(prompt, num_tokens=64)
print(text)

For live streaming:

for event in generator.generate_with_denoise_stream(prompt, num_tokens=64):
    if event["phase"] == "draft":
        print(event["token"], end="", flush=True)