AI4SE T5 code-review checkpoints
Saved models from the completed William & Mary AI4SE experiment. All four use the same T5-small architecture (60,441,088 parameters) and the original, independently trained 32,000-token SentencePiece Unigram tokenizer.
| Folder | Checkpoint | Starting weights | Fine-tuning data |
|---|---|---|---|
part2-pretrained/ |
Fixed final pre-training step 15,000 | Random initialization | Span-corruption pre-training; no fine-tuning |
condition-a/ |
Selected step 10,000 | Random initialization | Both tasks, 50/50 |
condition-b/ |
Selected step 10,000 | Part 2 final checkpoint | Both tasks, 50/50 |
condition-c/ |
Selected step 10,000 | Part 2 final checkpoint | Code-and-comment-to-code only |
A and B were selected by the minimum mean validation loss across both tasks; C was selected on the edit task's validation loss. Uploading preserves the selected weights; it does not retrain or reselect checkpoints. MANIFEST.json records the source-file SHA-256 hashes and byte sizes.
The tokenizer/ directory contains the original SentencePiece files and project-specific configuration. It is not an automatically converted Hugging Face AutoTokenizer package. Load the .model using SentencePiece and retain the recorded token IDs, including EOS, the task markers and T5 sentinels.
Example weight loading after authenticating to this private repository:
from transformers import T5ForConditionalGeneration
from huggingface_hub import hf_hub_download
import sentencepiece as spm
repo_id = "Shemar12/ai4se-t5-code-review-checkpoints"
model = T5ForConditionalGeneration.from_pretrained(
repo_id, subfolder="condition-b"
)
sp = spm.SentencePieceProcessor(model_file=hf_hub_download(
repo_id, filename="tokenizer/tokenizer.model"
))
The exact input task prefixes are code&comment2code: and code2comment: (trailing space included). Model outputs remain experimental: all A/B/C reported test exact-match counts are zero. These are research artifacts, not validated automated review tools.
Optimizer, scheduler and random-state recovery archives are retained on the server and are not included in this selected-model upload. The corpus, datasets, predictions and LaTeX report are not uploaded here.