Model · opt-125m
opt-125m
Developer: Meta AI
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/opt-125m.json
Fields
- id
- opt-125m
- identifiers
- huggingface
- facebook/opt-125m
- developer
- Meta AI
- release_date
- 2022-05-11partial · sourcenote: HF repo creation date; may predate or postdate public release.
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- otherpartial · sourcenote: From HF card metadata (uploader-declared); confirm against the license text.
- architecture
- family
- decoder_only
- note
- family read from config 'architectures': ['OPTForCausalLM'].
- n_layers
- 12recorded · source
- hidden_size
- 768recorded · source
- n_heads
- 12recorded · source
- vocab_size
- 50272recorded · source
- context_length
- 2048recorded · source
- positional_encoding
- learned_absoluterecorded · sourcenote: Code: OPTLearnedPositionalEmbedding. Same as opt-6-7b.
- training_data
- ~180B tokens: RoBERTa corpus subsets, a subset of the Pile, and PushShift.io Reddit. Same recipe as the 6.7B model.recorded · source
- techniques
- primary_sources
- record_history
- date:2026-09-24 · change:ingested as candidate from HF (facebook/opt-125m@27dcfa74d334bc871f3234de431e71c6eeba5dd6) · by:ingest_hf.py ·date:2026-09-24 · change:preparer: developer, positional encoding, trained_on set (from paper, matches opt-6-7b) · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:preparer cleanup: removed 2 stale flag(s), filled training_data · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:reviewed and promoted from staging (1 edge(s) accepted) · by:Wilson Pruitt ·
Parents
Training data
- trained_on → The Pile declared source Paper: trained on ~180B tokens incl. 'a subset of the Pile'; also RoBERTa corpus subsets and PushShift.io Reddit (not recorded as datasets). Same mix as opt-6-7b -- not stated per-size in the card, but the paper describes one training recipe for the whole OPT suite.
Children
No edges recorded.
Read in
No station on the reading path has touched this record yet.