Model · code-llama-7b
Code Llama 7B
Developer: Meta
Availability: gated · checked 2026-09-24 · source
Meta's repo is gated (manual approval); the codellama/ transformers conversion used for the config is not gated.
Raw record: /data/models/code-llama-7b.json
Fields
- id
- code-llama-7b
- identifiers
- huggingface
- meta-llama/CodeLlama-7b-hfcodellama/CodeLlama-7b-hf
- developer
- Meta
- release_date
- 2023-08-24recorded · source
- weights_status
- open
- availability
- gated · checked 2026-09-24 · sourcenote: Meta's repo is gated (manual approval); the codellama/ transformers conversion used for the config is not gated.
- license
- Released under the same custom license as Llama 2 (paper: 'released under the same custom permissive license').recorded · source
- architecture
- family
- decoder_only
- n_layers
- 32recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 32016recorded · source
- positional_encoding
- rotary (RoPE)recorded · source
- context_length
- 16384recorded · source
- n_kv_heads
- 32recorded · source
- note
- Config is from the codellama/ HF org's transformers conversion, pinned by commit. Paper: RoPE base period raised from 10,000 to 1,000,000 for long-context fine-tuning (config rope_theta = 1000000). Vocab 32016 vs Llama 2's 32000; the extra tokens are not explained in the sources read.
- training_data
- Initialized from Llama 2 7B, then 500B tokens of 'a near-deduplicated dataset of publicly available code' (8% natural language about code).partial · source
- techniques
- transformer-decoderrotary-position-embeddingrmsnormswiglu
- primary_sources
- record_history
- date:2026-09-24 · change:created from primary sources (Phase 1 seed, batch 1) · by:wilson-pruitt + claude ·date:2026-09-24 · change:availability checked and recorded · by:wilson-pruitt + claude ·
Parents
Weights descend
- fine_tuned_from → Llama 2 7B declared source Paper sec. 2: 'We train Code Llama on 500B tokens during the initial phase, starting from the 7B, 13B, and 34B versions of Llama 2.' Continued pretraining rather than instruction tuning.
Children
Weights descend
- ← fine_tuned_from CodeLlama-7b-Instruct-hf declared source Paper describes Code Llama - Instruct as a further fine-tuned stage of the base Code Llama models for instruction following.
- ← fine_tuned_from CodeLlama-7b-Python-hf declared source Paper: Code Llama - Python models are 'further specialized on 100B tokens using a Python-heavy dataset', starting from the base Code Llama checkpoints.
Read in
No station on the reading path has touched this record yet.