OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning (ISSRE 2026)

💡 What is OpsLLM?

OpsLLM is a domain-specialized LLM for software operations with core capabilities in QA and RCA. We design modality-specific data pipelines to unify fragmented operational knowledge into QA pairs, and introduce a human-in-the-loop (HITL) mechanism for automated quality control, enabling scalable data construction. We propose a multi-stage learning framework with a domain process reward model (DPRM) based on GRPO to provide process-level supervision. Combined with curriculum learning and strict gating, this ensures both accurate Root Cause Analysis (RCA) outcomes and reliable reasoning. We also design a multi-expert benchmark generation method for objective evaluation of QA capability.

Experimental results on the tasks with diverse difficulties demonstrate that OpsLLMs effectively learns and aligns with the operational domain knowledge infused, outperforming existing open-source and closed-source LLMs in accuracy with improvements of 0.2%~11.9% on QA tasks and 8.5%~70.3% on RCA tasks.

OpsLLM demonstrates strong transferability across different test sets and model scales.

🚀 Performance snapshot

We evaluate on QA and RCA tasks at three difficulty levels. Easy samples are drawn from SFT training data (seen during training), testing whether the model retains learned knowledge. Mid samples are held-out data not used in any training stage (unseen), testing generalization to new but in-distribution queries. Hard samples come from external public benchmarks, testing transferability to out-of-distribution scenarios.

For QA, we use accuracy (ACC): the proportion of correctly answered MCQs. For RCA, we also use ACC. For each case, the model is restricted to output exactly one root-cause component, one fault type, and the corresponding diagnostic process. A case is considered correct only if the predicted component and fault type exactly match the ground-truth strings.

From the table below, it can be seen that OpsLLM outperforms existing LLMs in software operations and demonstrates strong transferability.

💻 Quickstart

Set Up Python Environment: Use the following commands to create and activate the python environment:

conda create -n OpsLLM python=3.10
conda activate OpsLLM

Install Dependencies: install the necessary dependencies by running:

pip install -r requirements.txt

After completing the above steps, follow the steps below to run the OpsLLM project.

Construction of the Operational Fine-tuning Dataset

LLM-Based QA Pair Generation

1. Set API KEY

export DASHSCOPE_API_KEY="your_api_key"

2. Generate reasoning Trace

python reasoning_trace.py generate \
  -i ./anomalies \
  -o ./traces \
  --apikey YOUR_API_KEY

HITL-Guided Knowledge Calibration

1. Set API KEY

export API_KEY="your_api_key"

2. Create input files in the allQA directory (default) following the Alpaca format:

[
    {
        "instruction": "Your question here",
        "output": "Your answer here"
    }
]

3. Define Evaluation Rules Create a rule file containing your evaluation criteria. The content will be used as the system prompt. Example:

    Evaluate the following QA pair based on these criteria:
    1. Accuracy
    2. Completeness
    3. Clarity

    Return results in this format:
    <meet>111000</meet>
    <confidence>high</confidence>

4. Run Judge

python judge.py \
    --input_dir /path/to/input_files \
    --output_dir /path/to/output_results \
    --api_key your_api_key \
    --rule_file /path/to/custom_rules.txt \
    --model qwen-max-latest \
    --max_concurrent 5 \
    --delay_seconds 30

5. Example output Format:

[
    {
        "quality": 1,
        "confidence": "high"
    },
    {
        "quality": 0,
        "confidence": "low"
    },
    ...
]

and then run filter

python filter.py -i /path/to/input_files/original_data -j /path/to/judgment_results -o /path/to/output_results

where /path/to/judgment_results is the above json, which results from /path/to/output_results in step 4

Post-training Pipeline of OpsLLM

Supervised Fine-Tuning (SFT)

Based on LLaMA-Factory, place the fine-tuning data into the data directory of LLaMA-Factory.

✨️ Note: Perform the following RL training based on the fine-tuned model.

Reinforcement Learning (RL)

Based on Verl, run the following command to perform RL training:

✨️ Switch Verl to version v0.5.0:

git checkout v0.5.0

✨️ Domain Process Reward Model (DPRM): (i) The detailed RCA-specific scoring rubrics can be found in RL/DPRM/rule.txt. (ii) The code for generating diagnostic reasoning chains can be found in RL/DPRM/candidate_answer_generation.py. (iii) The code for invoking the LLM to perform scoring can be found in RL/DPRM/score.py.

✨️ Multi-dimensional Reward: The implementation details of our multi-dimensional reward can be found in RL/compute_score.py.

✨️ The base model for Stage 1 is the fine-tuned model, and the base model for Stage 2 is the model obtained from Stage 1 training.

Multi-Expert Automated Benchmark Generation

1. Each input JSON file must contain a list of objects, each object having exactly two keys(a.k.a in alpaca format):

• "instruction": the question text (string)
• "output": the correct answer text (string)

2. Set API KEY

export API_KEY="your_api_key"

3. Run the code

python generate_mcq.py [-h] [-o OUTPUT_DIR] [-c CONCURRENCY] [-m MODEL] input_files [input_files ...]

4. For each input file, the script creates a corresponding JSON file in the output directory with the same filename. The content is a list of results, one per input Q&A pair. Each result is either:

An MCQ object (if generation succeeded), or null (if generation failed after all retries).