Model · tulu-2-7b
Tulu 2 7B
Developer: Allen Institute for AI (Ai2) and University of Washington (paper authors Ivison, Wang, et al.)
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/tulu-2-7b.json
Fields
- id
- tulu-2-7b
- identifiers
- huggingface
- allenai/tulu-2-7b
- developer
- Allen Institute for AI (Ai2) and University of Washington (paper authors Ivison, Wang, et al.)
- release_date
- 2023-11-13partial · sourcenote: HF repo creation date (2023-11-13). The paper (arXiv 2311.10702) is dated 2023-11-17 to 2023-11-20 (v2 read: 20 Nov 2023).
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- AI2 ImpACT Low-risk license (card text); base weights also subject to the Llama 2 Community Licensepartial · sourcenote: Card text: 'License: AI2 ImpACT Low-risk license' (link allenai.org/impact-license). The card metadata has no license field, and the ImpACT page renders by JavaScript so its text could not be read this session. The card does not mention the Llama 2 license; the base is Llama 2.
- architecture
- family
- decoder_only
- note
- family read from config 'architectures': ['LlamaForCausalLM'].
- n_layers
- 32recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 32000recorded · source
- context_length
- 8192recorded · sourcenote: Config value. The Tulu 2 paper (arXiv 2311.10702) trains at max sequence length 8,192; Llama 2 base is 4,096.
- n_kv_heads
- 32recorded · source
- positional_encoding
- rotary (RoPE)recorded · sourcenote: Architecture unchanged from llama-2-7b (fine-tune, not a structural change).
- training_data
- Tulu V2 mix (allenai/tulu-v2-sft-mixture): a blend of human-written and synthetic instruction data, including distilled GPT-4 and ChatGPT-derived subsets; SFT only.recorded · sourcenote: Card: 'fine-tuned on a filtered and preprocessed of the Tulu V2 mix dataset, which contains a diverse range of human created instructions and synthetic dialogues generated primarily by other LLMs.' Composition is in the tulu-v2-sft-mixture record.
- techniques
- primary_sources
- record_history
- date:2026-09-24 · change:ingested as candidate from HF (allenai/tulu-2-7b@3c6e328ae91fabdd0daf09de16887de9615c1f66) · by:ingest_hf.py ·date:2026-09-24 · change:preparer: developer, license (partial), training data, positional encoding (propagated), 2 edges; sources: card, arXiv 2311.10702 · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·
Parents
Weights descend
- fine_tuned_from → Llama 2 7B declared source Card: 'Tulu 2 7B is a fine-tuned version of Llama 2'; 'Finetuned from model: meta-llama/Llama-2-7b-hf'. Paper abstract: models 'finetuned on the V2 mixture'. Uploader allenai (Ai2) is the developer.
Training data
- trained_on → Tulu V2 SFT mixture declared source Card metadata `datasets: allenai/tulu-v2-sft-mixture`; paper: finetuned on Tulu-V2-mix. Ai2 built the mixture. The distillation inside it (GPT4-Alpaca, OpenOrca GPT-4 subset) is recorded on the dataset.
Children
Weights descend
- ← fine_tuned_from Tulu 2 DPO 7B declared source Paper abstract: 'a LLaMA-2 70B model finetuned on Tulu-V2-mix and further trained using direct preference optimization (DPO)'; Table 3 compares Tulu V2 models 'with and without DPO finetuning' at 7B, 13B, 70B. The paper states this for the family, not in a 7B-specific sentence, so the 7B DPO checkpoint's start point is read from that framing. Card metadata instead names only Llama-2-7b-hf as base_model (the original base, not the SFT stage).
Read in
No station on the reading path has touched this record yet.