mehul79/Text2SQL-4B-Model

Qwen3.5-4B Spider text-to-SQL LoRA

A LoRA adapter for Qwen/Qwen3.5-4B that writes SQLite queries from English questions. It was trained on tool-calling traces built from the Spider training set.

Code to rebuild the data, retrain, and run the benchmarks: Self-healing-Text2SQL-Agent (training/ and backend/eval/).

Results

Spider dev, execution accuracy (%), all 1,034 questions, official Spider evaluator. Both models run inside the Self-healing Text2SQL harness (LangGraph: generate, validate, execute, repair), temperature 0.

Model	Easy	Medium	Hard	Extra	All
Qwen3.5-4B base, first attempt	87.1	78.0	56.9	45.2	71.4
Qwen3.5-4B base, with repair	86.7	80.5	57.5	50.0	73.2
This adapter, first attempt	89.9	77.4	74.1	51.8	75.7
This adapter, with repair	90.3	80.0	74.7	53.0	77.3

Most of the gain is on hard questions (+17 points).

Training

Every training example has the same three-step tool-calling shape:

1. get_schema returns the database's CREATE TABLE statements.
2. execute_sql runs the gold query and returns up to 10 rows.
3. submit_answer submits the gold query.

The traces come from Spider train. Gold queries that failed to run were dropped, leaving 6,797 traces. 200 were held out for validation (seed 42). Loss is computed only on the assistant turns, after an empty <think></think> block, so the model is trained with thinking off.

Setting	Value
Method	LoRA, bf16 (Unsloth)
Rank / alpha / dropout	64 / 128 / 0
Target modules	q, k, v, o, gate, up, down projections
Learning rate	1e-4, cosine schedule, 3% warmup
Batch	4 per device x 4 accumulation = 16
Epochs	2 (850 steps)
Max sequence length	4096
Hardware	1 GPU, 13.7 GB peak, about 75 minutes
Final losses	train 0.009, validation 0.0165

Usage

Serve the base model with the adapter in vLLM, with thinking off:

vllm serve Qwen/Qwen3.5-4B --enable-lora \
  --lora-modules text2sql-sft=mehul79/Text2SQL-4B-Model \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --language-model-only --max-model-len 16384

Or merge it into the base model with PEFT (PeftModel.from_pretrained(base, "mehul79/Text2SQL-4B-Model").merge_and_unload()) and serve the merged weights.

Limitations

• Trained only on Spider and SQLite. Other SQL dialects and real-world schemas are untested.
• The validation split comes from Spider train, so the validation loss says little about new databases; the Spider dev results above are the real measure.
• Every training trace has one fixed shape and contains no errors, so the model never saw a failed query being fixed during training.
• It sometimes writes Spider-style double-quoted string literals ("value"), which strict SQL parsers reject.