Skip to content
#

marcella

Star

Here is 1 public repository matching this topic...

aidendorian / Marcella-60M-SLM

Star 2

A 66M parameter decoder-only transformer language model implemented from scratch in PyTorch. Features a custom SentencePiece tokenizer, RoPE positional embeddings, SwiGLU feed-forward network, per-layer KV cache for efficient autoregressive inference, and a Svelte-based streaming chat interface.

transformers torch pytorch pretrained-models slm language-model rope alpaca sdpa finetuning sentencepiece transformer-models kv-cache small-language-models fineweb flashattention marcella
  • Updated Apr 5, 2026
  • Python

Improve this page

Add a description, image, and links to the marcella topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the marcella topic, visit your repo's landing page and select "manage topics."

Learn more