plasmova/Nova-v2

Nova v2

Nova v2 is a decoder-only causal language model with 176.2 million parameters. This repository contains its Transformers-compatible checkpoint, tokenizer, and model code.

Model specifications

Specification	Value
Parameters	176.2M
Vocabulary	32,768 tokens
Hidden size	768
Transformer layers	20
Attention	12 query heads, 4 key/value heads
Feed-forward size	2,048
Maximum context	2,048 tokens
Weights	Float32 (safetensors)

Nova v2 uses grouped-query attention, rotary position embeddings, RMSNorm, and a SwiGLU feed-forward network. Its embeddings and output head are untied. The included custom modeling code uses the Transformers causal language model interface and supports KV caching during generation. Review the code before loading it with trust_remote_code=True.

Load with Transformers

Install PyTorch and Transformers:

pip install torch transformers safetensors

Load the tokenizer and model from the Hub:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "plasmova/Nova-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

system_prompt = (
    "You are Nova, a helpful and accurate assistant. Answer directly and concisely. "
    "For simple arithmetic, calculate the result. Do not invent names, scenarios, or equations."
)
prompt = f"<|user|>System instruction: {system_prompt}\n\nUser request: Explain why the sky appears blue in one sentence.<|assistant|>"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
output = model.generate(
    **inputs,
    max_new_tokens=96,
    do_sample=True,
    temperature=0.2,
    top_p=0.85,
    top_k=12,
    repetition_penalty=1.12,
    no_repeat_ngram_size=3,
    eos_token_id=[tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|endoftext|>")],
    pad_token_id=tokenizer.pad_token_id,
    use_cache=True,
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

These recommended starting settings match the local inference defaults: temperature 0.2, top-p 0.85, top-k 12, repetition penalty 1.12, and up to 96 new tokens. Nova v2 was trained without a dedicated system role, so the example places its concise instruction in the user text. Keep the prompt and generated text within the 2,048-token context.

Training and evaluation

The checkpoint contains 176,192,256 learned parameters. The training code targets a 3-billion-token run. No benchmark results are included with this release; evaluate the model for your intended use before deployment.

License

The model weights and accompanying custom model code are released under the Apache License 2.0. See LICENSE. Upstream dataset terms continue to apply to training data.