Model · mpt-7b
MPT-7B
Developer: MosaicML
Availability: removed · checked 2026-09-24 · source
HF API returned HTTP 401 for this repo and for other mosaicml/ repos; only third-party re-uploads were found. 'removed' means unreachable at its original address, not a confirmed deletion.
Raw record: /data/models/mpt-7b.json
Fields
- id
- mpt-7b
- identifiers
- huggingface
- mosaicml/mpt-7b
- developer
- MosaicML
- release_date
- 2023-05-05recorded · source
- weights_status
- open
- availability
- removed · checked 2026-09-24 · sourcenote: HF API returned HTTP 401 for this repo and for other mosaicml/ repos; only third-party re-uploads were found. 'removed' means unreachable at its original address, not a confirmed deletion.
- license
- Apache 2.0recorded · source
- architecture
- family
- decoder_only
- note
- Post: 'a transformer trained from scratch'; 'GPT-style decoder-only transformers' with ALiBi replacing positional embeddings, FlashAttention, and stability changes; 6.7B parameters. Dimensions are not stated in the post and the developer's config could not be retrieved.
- n_layers
- nullnot_recorded
- hidden_size
- nullnot_recorded
- n_heads
- nullnot_recorded
- vocab_size
- nullnot_recorded
- positional_encoding
- ALiBi (no position embeddings)recorded · source
- training_data
- 1T tokens of text and code 'curated by MosaicML's data team'.partial · sourcenote: Mix components not stated in the release post.
- techniques
- transformer-decoderalibi
- primary_sources
- record_history
- date:2026-09-24 · change:created from primary sources (Phase 1 seed, batch 2) · by:wilson-pruitt + claude ·date:2026-09-24 · change:availability checked and recorded · by:wilson-pruitt + claude ·
Parents
No edges recorded.
Children
Weights descend
- ← fine_tuned_from mpt-7b-instruct declared source Post: 'Built by finetuning MPT-7B on a dataset we also release, derived from Databricks Dolly-15k and Anthropic's Helpful and Harmless datasets.'
- ← fine_tuned_from MPT-7B-Chat declared source Blog: MPT-7B-Chat 'built by finetuning MPT-7B'; archived card: 'built by finetuning MPT-7B on ...'.
Read in
No station on the reading path has touched this record yet.