Dataset · slimpajama

SlimPajama

Builder: Cerebras
Availability: removed · checked 2026-09-24 · source
HF API returned HTTP 401 for cerebras/SlimPajama-627B when checked. 'removed' means unreachable at its original address, not a confirmed deletion; DKYoon/SlimPajama-6B (a third-party 6B sample) was reachable.

Raw record: /data/datasets/slimpajama.json

Fields

id

slimpajama

identifiers

huggingface

cerebras/SlimPajama-627B

builder

Cerebras

release_date

nullnot_recorded

availability

removed · checked 2026-09-24 · source

note:  HF API returned HTTP 401 for cerebras/SlimPajama-627B when checked. 'removed' means unreachable at its original address, not a confirmed deletion; DKYoon/SlimPajama-6B (a third-party 6B sample) was reachable.

content

Derived from RedPajama by cleaning and deduplication, recorded in prose because Stemma has no dataset-to-dataset edge (parent: `redpajama-data-1t`). TinyLlama paper: 'derived from RedPajama ... underwent additional cleaning and deduplication ... SlimPajama retains only 50% of the original tokens from RedPajama.'partial · source

note:  Secondary description: the Cerebras announcement text could not be extracted this session, so the builder's own wording and date are not recorded.

primary_sources

https://arxiv.org/abs/2401.02385

https://www.cerebras.ai/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama

record_history

date:2026-09-24 · change:created from primary sources (tranche 2 dataset records) · by:wilson-pruitt + claude ·

Parents

No edges recorded.

Children

Training data

• ← trained_on TinyLlama-1.1B (intermediate-step-1431k-3T) declared source Card metadata `datasets: cerebras/SlimPajama-627B`; paper names SlimPajama as a primary source (GitHub subset removed). The `slimpajama` record has availability `removed` (HF API 401) — the corpus is recorded, not necessarily obtainable.

Read in

No station on the reading path has touched this record yet.