#

fineweb

Here are 8 public repositories matching this topic...

kmkrofficial / LiteGPT

LiteGPT: A 124M Small Language Model (SLM) pre-trained on FineWeb and fine-tuned on Alpaca.

pytorch alpaca gpt2 llm nanogpt fineweb

Updated Dec 16, 2025
Python

evanatyourservice / llm-jax

Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.

optimization flax gpt jax huggingface large-language-models llm fineweb

Updated Jul 24, 2025
Python

lmmx / bbcfw

Exploring the BBC News subset of the FineWeb dataset (via HuggingFaceFW/fineweb's dated subsets on HF), originally a Common Crawl dataset.

news bbc-news fineweb

Updated Feb 17, 2025
Python

MasterSkepticista / gpt2

Training GPT-2 on FineWeb-Edu in JAX/Flax

flax jax gpt2 flash-attention fineweb

Updated Dec 28, 2024
Python

aidendorian / Marcella-60M-SLM

A 66M parameter decoder-only transformer language model implemented from scratch in PyTorch. Features a custom SentencePiece tokenizer, RoPE positional embeddings, SwiGLU feed-forward network, per-layer KV cache for efficient autoregressive inference, and a Svelte-based streaming chat interface.

transformers torch pytorch pretrained-models slm language-model rope alpaca sdpa finetuning sentencepiece transformer-models kv-cache small-language-models fineweb flashattention marcella

Updated Apr 5, 2026
Python

elevonix-labs / finest-data-ui

subnet bittensor fineweb elevonix

Updated Mar 6, 2025
TypeScript

aditaa / llm

Decoder-only LLM from scratch with reproducible data pipelines, tokenizer/sharding workflows, and GPU training.

nlp machine-learning pytorch zim training-pipeline llm fineweb

Updated Apr 13, 2026
Python

mn-cs / fineweb-spark

FineWeb-Edu dataset analysis using Apache Spark - DSC 232R group project

python nlp machine-learning big-data apache-spark text-classification word2vec distributed-computing pyspark fineweb

Updated Mar 24, 2026
Jupyter Notebook

Improve this page

Add a description, image, and links to the fineweb topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the fineweb topic, visit your repo's landing page and select "manage topics."