Instructions to use tzwilliam0/Llama-3.2-1B-GRPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tzwilliam0/Llama-3.2-1B-GRPD with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tzwilliam0/Llama-3.2-1B-GRPD") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("tzwilliam0/Llama-3.2-1B-GRPD") model = AutoModelForCausalLM.from_pretrained("tzwilliam0/Llama-3.2-1B-GRPD", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tzwilliam0/Llama-3.2-1B-GRPD with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tzwilliam0/Llama-3.2-1B-GRPD" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzwilliam0/Llama-3.2-1B-GRPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/tzwilliam0/Llama-3.2-1B-GRPD
- SGLang
How to use tzwilliam0/Llama-3.2-1B-GRPD with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tzwilliam0/Llama-3.2-1B-GRPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzwilliam0/Llama-3.2-1B-GRPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tzwilliam0/Llama-3.2-1B-GRPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzwilliam0/Llama-3.2-1B-GRPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use tzwilliam0/Llama-3.2-1B-GRPD with Docker Model Runner:
docker model run hf.co/tzwilliam0/Llama-3.2-1B-GRPD
Llama-3.2-1B-GRPD
Student checkpoint produced by GRPD in the E3 setting of our paper on reward aware on policy distillation.
Built with Llama.
Setup
| student (initialization) | meta-llama/Llama-3.2-1B-Instruct |
| teacher (frozen) | meta-llama/Llama-3.2-3B-Instruct |
| method | GRPD |
| training data | UltraChat, 9000 prompts |
| steps | 50, with 128 prompts per step and 8 samples per prompt |
| learning rate | 1e-6 |
| max response length | 2048 |
| rollout seed | 0 |
| selected step | 30 |
All five methods in this setting share one hyperparameter configuration, the same training data, the same data seed and paired rollout seeds. No method received per method tuning.
Checkpoint selection
The checkpoint was selected on a held out set of 1000 prompts scored by Skywork-Reward-V2-Llama-3.2-3B. That selection set shares no prompts with any reported benchmark, and the benchmark numbers below come from a different reward model, Skywork-Reward-V2-Llama-3.1-8B. Candidate steps were 10, 20, 30, 40 and 50.
Results
External reward model scalar from Skywork-Reward-V2-Llama-3.1-8B, 8 samples per prompt averaged per prompt then averaged over prompts. Values are the mean over 3 rollout seeds with the across seed sample standard deviation; this repository holds the seed 0 checkpoint.
| benchmark | prompts | score |
|---|---|---|
| AlpacaEval | 805 | 12.547 ± 0.083 |
| WritingBench | 1000 | 3.789 ± 0.055 |
| WildBench | 1024 | 3.245 ± 0.073 |
For reference, under the same protocol the untrained 1B student scores 12.991 / 5.043 / 4.295 and the 3B teacher scores 21.085 / 12.992 / 14.019.
Decoding for evaluation: AlpacaEval and WildBench at temperature 0.7 and top_p 0.95; WritingBench at temperature 0.7, top_p 0.8 and top_k 20. Identical for every method and for both reference models.
Files
training_logs/ holds the full record for this run: selected.json with every candidate step and its validation score, metrics.jsonl with the per step training dynamics, and launch.json with the complete hyperparameter configuration and switches.
License
This model is a derivative of Llama 3.2 and is governed by the Llama 3.2 Community License. Use is subject to the Llama 3.2 Acceptable Use Policy.
- Downloads last month
- 39
Model tree for tzwilliam0/Llama-3.2-1B-GRPD
Base model
meta-llama/Llama-3.2-1B-Instruct