Model · redpajama-incite-7b-instruct

RedPajama-INCITE-7B-Instruct

Developer: Together Computer (with Ontocord.ai, ETH DS3Lab, AAI CERC, Université de Montréal, MILA, Stanford CRFM, Stanford Hazy Research, LAION)
Availability: available · checked 2026-09-24 · source

Raw record: /data/models/redpajama-incite-7b-instruct.json

Fields

id

redpajama-incite-7b-instruct

identifiers

huggingface

togethercomputer/RedPajama-INCITE-7B-Instruct

developer

Together Computer (with Ontocord.ai, ETH DS3Lab, AAI CERC, Université de Montréal, MILA, Stanford CRFM, Stanford Hazy Research, LAION)

release_date

2023-05-05recorded · source

note:  Together post dated May 5, 2023: the release included '3B and 7B' models, the 7B labelled an early preview (800B of a planned 1T tokens at the time). The repo's first commit is the same day.

weights_status

open

availability

available · checked 2026-09-24 · source

license

apache-2.0recorded · source

note:  Card: 'License: Apache 2.0'. Uploader is the developer.

architecture

family

decoder_only

note

family read from config 'architectures': ['GPTNeoXForCausalLM'].

n_layers

32recorded · source

hidden_size

4096recorded · source

n_heads

32recorded · source

vocab_size

50432recorded · source

context_length

2048recorded · source

positional_encoding

nullnot_recorded

note:  Card does not state it; config hints are not evidence. Also not_recorded on the base record.

training_data

Fine-tuned 'for few-shot applications on the data of GPT-JT, with exclusion of tasks that overlap with the HELM core scenarios'; 1B tokens, learning rate 1e-5 (card). The card's 'Training Data' line points to RedPajama-Data-1T, which is the base model's pretraining corpus, not this fine-tuning set.recorded · source

techniques

primary_sources

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct/raw/7f36397b9985a3f981cdb618f8fec1c565ca5927/config.json

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct/blob/7f36397b9985a3f981cdb618f8fec1c565ca5927/README.md

https://www.together.ai/blog/redpajama-models-v1

record_history

date:2026-09-24 · change:ingested as candidate from HF (togethercomputer/RedPajama-INCITE-7B-Instruct@7f36397b9985a3f981cdb618f8fec1c565ca5927) · by:ingest_hf.py ·

date:2026-09-24 · change:preparer: developer, release date (blog), license, training data, 2 edges accepted + 1 rejected; sources: card, Together blog · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·

Parents

Weights descend

• fine_tuned_from → RedPajama-INCITE-7B-Base declared source Card lists 'Base Model: RedPajama-INCITE-7B-Base' and says the model 'was fine-tuned for few-shot applications on the data of GPT-JT'. No `base_model` metadata; the relation rests on the card text.

Training data

• trained_on → RedPajama-Data-Instruct declared source Card metadata lists togethercomputer/RedPajama-Data-Instruct; the dataset card describes it as P3 plus Natural Instruction, decontaminated against HELM, consistent with the model card's 'GPT-JT data minus HELM overlap'.

Children

No edges recorded.

Read in

No station on the reading path has touched this record yet.