huggingface
A 325,600,000-parameter language model trained from scratch on an inference-only NPU
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
VKORE-325M-NPU-Base is a base language model trained from random initialization on an inference-only NPU. Its loss on unseen text was compared with that of a GPU model trained with the same initialization, data, batches, and learning rate schedule. Weights and evaluation scripts for both models are available.
KEY FEATURES
Key features
- Trained all model weights by running backpropagation on an inference-only NPU.
- Trained on 5,000,000,000 byte tokens from FineWeb-Edu over 76,292 steps.
- Compared the NPU and GPU models' loss across 1,024 segments of unseen text.
- Released both models' weights and evaluation scripts.