Model · pythia-6-9b-deduped
pythia-6.9b-deduped
Developer: EleutherAI
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/pythia-6-9b-deduped.json
Fields
- id
- pythia-6-9b-deduped
- identifiers
- huggingface
- EleutherAI/pythia-6.9b-deduped
- developer
- EleutherAI
- release_date
- 2023-02-25partial · sourcenote: HF repo creation date; may predate or postdate public release.
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- apache-2.0partial · sourcenote: From HF card metadata (uploader-declared); confirm against the license text.
- architecture
- family
- decoder_only
- note
- family read from config 'architectures': ['GPTNeoXForCausalLM'].
- n_layers
- 32recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 50432recorded · source
- context_length
- 2048recorded · source
- positional_encoding
- rotary (RoPE), partial: 25% of dimsrecorded · sourcenote: Same as pythia-6-9b.
- training_data
- Near-deduplicated Pile (~207B tokens); the deduplicated sibling of the pythia-6.9b record. Dataset has no Stemma record yet (see flags).recorded · source
- techniques
- primary_sources
- record_history
- date:2026-09-24 · change:ingested as candidate from HF (EleutherAI/pythia-6.9b-deduped@372b1c08d9b5b0fc18ce86bbf294930e26e66ed5) · by:ingest_hf.py ·date:2026-09-24 · change:preparer: developer, positional encoding set; dataset edge rejected pending a the-pile-deduplicated record · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:preparer cleanup: removed 3 stale flag(s), filled training_data · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:reviewed and promoted from staging (0 edge(s) accepted) · by:Wilson Pruitt ·
Parents
No edges recorded.
Children
Weights descend
- ← fine_tuned_from Pythia-Chat-Base-7B-v0.16 declared source OpenChatKit README: 'Pythia-Chat-Base-7B is a 7B-parameter fine-tuned variant of Pythia-6.9B-deduped from Eleuther AI'; the HF card says 'fine-tuned from EleutherAI's Pythia 7B' (no variant named). The README is what pins the deduped 6.9B.
Read in
No station on the reading path has touched this record yet.