Model · falcon-40b

falcon-40b

Developer: Technology Innovation Institute (TII)
Availability: available · checked 2026-09-24 · source

Raw record: /data/models/falcon-40b.json

Fields

id

falcon-40b

identifiers

huggingface

tiiuae/falcon-40b

developer

Technology Innovation Institute (TII)

release_date

2023-05-24partial · source

note:  HF repo creation date; may predate or postdate public release.

weights_status

open

availability

available · checked 2026-09-24 · source

license

apache-2.0partial · source

note:  From HF card metadata (uploader-declared); confirm against the license text.

architecture

family

decoder_only

note

family read from config 'architectures': ['FalconForCausalLM'].

n_layers

60recorded · source

hidden_size

8192recorded · source

n_heads

128recorded · source

vocab_size

65024recorded · source

context_length

nullnot_recorded

positional_encoding

rotary (RoPE)recorded · source

note:  Same design as falcon-7b (card: 'broadly adapted from the GPT-3 paper').

training_data

1,500B tokens: RefinedWeb (filtered, deduplicated web crawl) plus curated corpora. Same recipe as the 7B model.recorded · source

techniques

primary_sources

https://huggingface.co/tiiuae/falcon-40b

https://huggingface.co/tiiuae/falcon-40b/raw/05ab2ee8d6b593bdbab17d728de5c028a7a94d83/config.json

record_history

date:2026-09-24 · change:ingested as candidate from HF (tiiuae/falcon-40b@05ab2ee8d6b593bdbab17d728de5c028a7a94d83) · by:ingest_hf.py ·

date:2026-09-24 · change:preparer: developer, positional encoding, trained_on accepted, design_follows proposed (needs confirmation) · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:preparer: design_follows edge independently verified against the 40B card text · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:preparer cleanup: removed 3 stale flag(s), filled training_data · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·

Parents

Training data

• trained_on → RefinedWeb declared_by_uploader source HF card metadata datasets: tiiuae/falcon-refinedweb. Often incomplete; confirm which training stage used it.

Design

• design_follows → GPT-3 (175B, 2020 paper) declared_by_uploader source Card verified this session: 'The architecture is broadly adapted from the GPT-3 paper (Brown et al., 2020), with the following differences...' -- same wording as falcon-7b.

Children

Weights descend

• ← fine_tuned_from falcon-40b-instruct declared source Card: 'Finetuned from model: Falcon-40B'.

Read in

No station on the reading path has touched this record yet.