Model · llama-2-7b
Llama 2 7B
Developer: Meta
Availability: gated · checked 2026-09-24 · source
HF gate: manual approval.
Raw record: /data/models/llama-2-7b.json
Fields
- id
- llama-2-7b
- identifiers
- huggingface
- meta-llama/Llama-2-7b-hfmeta-llama/Llama-2-7b
- developer
- Meta
- release_date
- 2023-07-18recorded · source
- weights_status
- open
- availability
- gated · checked 2026-09-24 · sourcenote: HF gate: manual approval.
- license
- Llama 2 Community License Agreement (research and commercial use; licensees with >700M monthly active users on the release date must request a separate license from Meta).recorded · source
- architecture
- family
- decoder_only
- note
- Paper (sec. 2.2): 'We adopt most of the pretraining setting and model architecture from Llama 1'; primary differences are longer context and GQA, and GQA is used only at 34B/70B (Table 1 marks 7B without GQA). Per-size dimensions are not given in the paper, and Meta's HF config is gated, so layers/hidden/heads stay not_recorded rather than copied from Llama 1.
- n_layers
- nullnot_recorded
- hidden_size
- nullnot_recorded
- n_heads
- nullnot_recorded
- vocab_size
- 32000recorded · sourcenote: Paper says 'The total vocabulary size is 32k tokens' and 'the same tokenizer as Llama 1'.
- positional_encoding
- rotary (RoPE)recorded · source
- context_length
- 4096recorded · sourcenote: Table 1: '4k'.
- training_data
- 2.0T tokens, 'a new mix of data from publicly available sources, which does not include data from Meta's products or services.' Per-source breakdown not disclosed (unlike Llama 1).partial · source
- techniques
- transformer-decoderrotary-position-embeddingrmsnormswiglu
- primary_sources
- record_history
- date:2026-09-24 · change:created from primary sources (Phase 1 seed, batch 1) · by:wilson-pruitt + claude ·date:2026-09-24 · change:availability checked and recorded · by:wilson-pruitt + claude ·
Parents
Design
- successor_in_series → LLaMA 7B declared source Paper: 'Llama 2, an updated version of Llama 1'. Trained from scratch on a new data mix, NOT initialized from Llama 1 weights.
Children
Weights descend
- ← fine_tuned_from Llama 2-Chat 7B declared source Paper sec. 1: 'Llama 2-Chat, a fine-tuned version of Llama 2 that is optimized for dialogue use cases.'
- ← fine_tuned_from Code Llama 7B declared source Paper sec. 2: 'We train Code Llama on 500B tokens during the initial phase, starting from the 7B, 13B, and 34B versions of Llama 2.' Continued pretraining rather than instruction tuning.
- ← fine_tuned_from Nous-Hermes-llama-2-7b declared_by_uploader source Card: fine-tuned version of Llama-2-7b.
- ← fine_tuned_from vicuna-7b-v1.5 declared source Version doc: v1.5 base model is Llama 2. Card doesn't carry base_model metadata; sourced to FastChat's own release notes, same as v1.3.
- ← fine_tuned_from Orca 2 7B declared source Paper 4.2: 'We start with LLaMA-2-7B or LLaMA-2-13B checkpoint and finetune it'; card: 'Orca 2 is a finetuned version of LLAMA-2'.
- ← fine_tuned_from Tulu 2 7B declared source Card: 'Tulu 2 7B is a fine-tuned version of Llama 2'; 'Finetuned from model: meta-llama/Llama-2-7b-hf'. Paper abstract: models 'finetuned on the V2 mixture'. Uploader allenai (Ai2) is the developer.
Design
- ← same_architecture_retrained TinyLlama-1.1B (intermediate-step-1431k-3T) declared source Card: 'We adopted exactly the same architecture and tokenizer as Llama 2.' Paper: 'Following the same architecture and tokenizer as Llama 2, we name our model TinyLlama.' Relation chosen from those exact words (method.md: 'same' -> same_architecture_retrained). Parent id llama-2-7b stands for the Llama 2 design; the developer names Llama 2 generally, not a size. TENSION: the paper's own hyperparameters use grouped-query attention, which the llama-2-7b record notes is not used at 7B; and the size is 1.1B. If the reviewer reads 'exactly the same' as too strong, design_follows is the alternative.
Read in
No station on the reading path has touched this record yet.