A comprehensive, accessible Streamlit application for teaching AI language models in K-12 and higher education classrooms.
Token Explorer for Educators is an interactive web application designed to make AI language models accessible to non-technical educators and students. It demonstrates how AI predicts text, helps explore tokenization, and provides ready-to-use classroom activities.
✅ 25 Curated Example Prompts across 5 categories
✅ Interactive Glossary & Help Panels with classroom-friendly language
✅ 5 AI Model Options including multilingual support
✅ Advanced Parameter Controls (Temperature, Top-k, Top-p)
✅ Class Poll Mode for live student engagement
✅ Streamlined Visualizations (probability chart, metrics, confidence tracking)
✅ Lightweight CSV & Text Exports for sharing results
✅ 2 Ready-to-Use Classroom Activities with summaries
✅ Full Accessibility Support (WCAG 2.1 compliant)
✅ Standards Alignment (CSTA, ISTE, Common Core)
# Clone the repository
git clone https://github.com/your-username/token-explorer.git
cd token-explorer
# Install dependencies
pip install -r requirements.txt
# Run the application
streamlit run app.pyThe app will open automatically at http://localhost:8501
- Push code to GitHub
- Go to streamlit.io/cloud
- Connect your repository
- Deploy with one click!
See DEPLOYMENT_GUIDE.md for detailed instructions.
- Create a free Hugging Face access token with “Inference” scope
- Set the token as
HF_API_TOKEN(Streamlit secrets,.streamlit/secrets.toml, or terminal export) - The app now calls the hosted GPT-2 model by default for live probabilities
- If the token is missing or a model is unsupported, the classroom simulator automatically fills in fake probabilities
- Famous Quotes: "To be or not to be," "I have a dream..."
- Story Starters: "Once upon a time in a..."
- Science Facts: "Water boils at..." "DNA stands for..."
- Simple Sentences: "The cat sat on the..."
- Math & Logic: "Two plus two equals..."
- GPT-2 (English) - General-purpose, creative text
- BERT Base (English) - Context understanding
- BERT Multilingual - 104 languages supported
- GPT-2 Spanish - Spanish language generation
- DistilGPT-2 (Fast) - Optimized for speed
- Predict the Next Word Game (Grades 3-8, 15-20 min)
- Temperature Experiment (Grades 6-12, 25-30 min)
Each activity includes:
- Step-by-step instructions
- Learning goals
- Discussion questions
- Downloadable summaries
- 📊 Probability Distribution - Interactive bar chart of top tokens
- 📋 Token Table - Ranked probabilities with CSV/text export
- 📈 Metrics Snapshot - Entropy, perplexity, and top-token confidence
- 📉 Confidence Tracking - Probability and entropy over time
Lesson Planning:
- Browse 5 standards-aligned activities
- Load example prompts relevant to curriculum
- Export materials for student handouts
Live Demonstrations:
- Project app on classroom screen
- Enable Class Poll Mode
- Compare student predictions with AI
Assessment:
- Generate knowledge check quizzes
- Export results as CSV
- Track conceptual understanding
Self-Paced Exploration:
- Try different example prompts
- Adjust parameters to see effects
- Read glossary definitions
Collaborative Learning:
- Submit predictions in poll mode
- Compare with AI and peers
- Discuss results in groups
Assessment:
- Take built-in quizzes
- Get instant feedback
- Review explanations
✅ High Contrast Mode - 7:1 color ratio
✅ Font Size Controls - 14px, 16px, 20px options
✅ Keyboard Navigation - Full tab/arrow key support
✅ Screen Reader Support - ARIA labels throughout
✅ Touch-Friendly - 44×44px minimum tap targets
✅ Responsive Design - Works on tablets and phones
- 1B-AP-15: Test and debug algorithms
- 3A-IC-24: Evaluate computational artifacts for bias
- 3B-AP-08: Describe how AI and ML algorithms work
- 1.6.d: Students understand how AI makes decisions
- 1.1.c: Students use technology for creative expression
- HSS-IC.A.2: Analyze decisions using probability
- 7.SP.C.7: Develop probability models
streamlit >= 1.28.0
pandas >= 2.0.0
numpy >= 1.24.0
plotly >= 5.17.0
Frontend: Streamlit web framework
Visualizations: Plotly interactive charts
Data: Pandas DataFrames
AI Simulation: Context-aware probability generation
- ⚡ Fast loading with session state
- 💾 Efficient memory usage
- 📱 Optimized for mobile devices
- 🔄 Real-time updates in poll mode
(Coming soon - link to walkthrough video)
- DEPLOYMENT_GUIDE.md - Detailed deployment instructions
- app.py - Main application code
- requirements.txt - Python dependencies
Educators are encouraged to:
- Adapt activities for their grade levels
- Add example prompts in different languages
- Suggest new visualizations
- Report bugs or usability issues
This project is released for educational use. Feel free to adapt, modify, and share with educators and students.
This project builds upon code and ideas from TKBEN Tokenizers Benchmarker. Original author: CTCycle.
Certain components, logic, or inspiration were directly adapted or modified from this open-source repository.
Built with:
- Streamlit - Web framework
- Plotly - Interactive visualizations
- Educational content aligned with CSTA, ISTE, and Common Core standards
- Click the "📖 Glossary" button for term definitions
- Use "❓ Help & Tutorial" for guided walkthrough
- Check "🏫 Classroom Activities" for lesson ideas
App won't start?
pip install --upgrade pip
pip install -r requirements.txt
streamlit run app.pyPort already in use?
streamlit run app.py --server.port 8502Charts not displaying?
streamlit cache clearFuture enhancements:
- Real API integration (Hugging Face)
- Student account system
- Teacher dashboard analytics
- Custom activity builder
- Video tutorial library
- Offline mode support
If you find Token Explorer helpful for your classroom, please ⭐ star this repository to help other educators discover it!
For questions, feedback, or collaboration opportunities, reach out through GitHub Issues.
Token Explorer for Educators
Making AI Accessible to All Learners
Version 2.0 | Streamlit Edition | November 2025