OneJev-0.8B GGUF

GGUF conversion of OmniJev/OneJev-0.8B, a 0.8B decision model (Qwen3.5-0.8B base), for llama.cpp and the Observer desktop app.

File Size Same top token as F16 Use
OneJev-0.8B-F16.gguf 2.03 GB โ€” Reference precision
OneJev-0.8B-Q8_0.gguf 1.08 GB 98.8% Default
OneJev-0.8B-Q6_K.gguf 0.83 GB 95.9% Recommended smaller option (mobile)
OneJev-0.8B-Q5_K_M.gguf 0.75 GB 93.1% Smaller, some quality loss
OneJev-0.8B-Q4_K_M.gguf 0.67 GB 86.7% Smallest K-quant; noticeable drift
OneJev-0.8B-Q4_0.gguf 0.64 GB 82.3% Fast ARM kernels in llama.cpp; largest drift
mmproj-OneJev-0.8B-F16.gguf 0.20 GB โ€” Vision projector, needed for image (screen/camera) input

The "same top token" column is from llama-perplexity --kl-divergence against F16 on general English text (20 ร— 512-token chunks). Mean KLD: Q8_0 0.0004, Q6_K 0.0044, Q5_K_M 0.015, Q4_K_M 0.059, Q4_0 0.120. It is a proxy, not a measurement on OneJev decision prompts.

Note on speed: OneJev scores a single forward pass over the prompt and does not generate text, so it is prompt-processing bound. Lower-bit quants mainly reduce download size and memory. They do not reliably make decisions faster, except where llama.cpp has quant-specific kernels (e.g. Q4_0 on ARM). For image input, the vision projector and image tokens dominate the cost.

How it's used

OneJev is not a chat model. It answers a typed question about a state (text and/or images) by putting probability on the option letters A..Z. Read the logits of those letter tokens at the answer position after a single forward pass; do not sample text.

  • The prompt format follows the OneJev model card: a system turn, then a <state>โ€ฆ</state> block plus the question and lettered options in the user turn, rendered with the embedded chat template and enable_thinking=false.
  • Observer builds these prompts in app/src/utils/localLlm/systemOne.ts and scores them natively in app/plugins/llm_engine (LlmEngine::decide).

Conversion

  • Converted with convert_hf_to_gguf.py from llama.cpp commit e21cdc11a (the build vendored by Observer's llama-cpp-rs). Text model at F16 and Q8_0; vision projector at F16.
  • Q6_K, Q5_K_M, Q4_K_M and Q4_0 were quantized from the F16 file with llama-quantize (llama.cpp build 8880, 2799d933b), without an importance matrix.
  • The rendered prompt was checked to be byte-identical to transformers.js apply_chat_template for the onnx-community/OneJev-0.8B-ONNX processor.
  • Smoke-tested in Observer's llama.cpp engine. Q8_0 and F16 agree within 0.004 on a text yes/no probability.

License and intended use follow the base model.

Downloads last month
452
GGUF
Model size
1B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Roy3838/OneJev-0.8B-GGUF

Quantized
(6)
this model