Lordog/dive-into-llmsPublic

《动手学大模型Dive into LLMs》系列编程实践教程

AI summary: An educational repository teaching developers to build, train, and deploy large language models from scratch.

Stars
60.3K
+192 today
Forks
7.2K
Watchers
368
Open issues
12
Open PRs
7
Contributors
~14
Commits
73
Branches
1
Views (30d)
10

Jupyter NotebookNo licenseCreated Apr 8, 2024Last push 1y agoLatest release v1+366 stars this week+3.5K this month

Quick answers

What is dive-into-llms?
An educational repository teaching developers to build, train, and deploy large language models from scratch.
What does dive-into-llms do?
Dive Into LLMs functions as a comprehensive educational repository designed to bridge the gap between high-level AI concepts and practical implementation details. It systematically breaks down complex machine learning architectures into digestible, runnable code examples utilizing modern frameworks like PyTorch. The curriculum guides learners through the entire lifecycle of model creation, from basic tokenization and transformer mechanics to advanced fine-tuning techniques like LoRA. It emphasizes building intuition by providing interactive Jupyter notebooks that allow users to inspect tensor transformations at every layer. The project effectively demystifies the perceived black box of modern AI systems through transparent, step-by-step implementations.
Who is dive-into-llms for?
Machine learning practitioners, software engineers, and researchers seeking a rigorous, code-first understanding of modern language models. It provides a structured learning path for practitioners who want to move beyond superficial API usage.
How do I get started with dive-into-llms?
git clone https://github.com/Lordog/dive-into-llms.git && pip install -r requirements.txt
How popular is dive-into-llms on GitHub?
Lordog/dive-into-llms has 60,313 stars and 7,217 forks on GitHub, and gained 366 stars in the last 7 days.

Star history

since Jul 28, 2026
020K40K60KJul 2026Sep 2026Oct 2026Nov 2026
60.3K stars as of Nov 17, 2026. Measured daily since Jul 28, 2026; GitHub no longer exposes earlier star timestamps.

Contribution activity

commits per day, last 52 weeks

Signals and awards

derived from tracked data
  • Landmark project

    60,313 stars

  • Repeat trending

    9 trending appearances

What dive-into-llms does

Dive Into LLMs functions as a comprehensive educational repository designed to bridge the gap between high-level AI concepts and practical implementation details. It systematically breaks down complex machine learning architectures into digestible, runnable code examples utilizing modern frameworks like PyTorch. The curriculum guides learners through the entire lifecycle of model creation, from basic tokenization and transformer mechanics to advanced fine-tuning techniques like LoRA. It emphasizes building intuition by providing interactive Jupyter notebooks that allow users to inspect tensor transformations at every layer. The project effectively demystifies the perceived black box of modern AI systems through transparent, step-by-step implementations.

Machine learning practitioners, software engineers, and researchers seeking a rigorous, code-first understanding of modern language models. It provides a structured learning path for practitioners who want to move beyond superficial API usage.

  • Interactive notebooks: Provides runnable Jupyter environments that demonstrate core concepts with executable code rather than static theory.
  • Architecture breakdowns: Details the internal mechanics of popular models including LLaMA, GPT, and Claude at the tensor level.
  • Training pipelines: Demonstrates how to build robust data ingestion and processing workflows necessary for pre-training large models.
  • Optimization techniques: Covers practical memory management and hardware acceleration methods like quantization and mixed precision.
  • Fine-tuning examples: Includes complete scripts for adapting base models to specific tasks using parameter-efficient techniques.

Where teams use it

Skill development

Transition from utilizing high-level APIs to engineering custom model architectures from first principles.

Academic research

Establish a foundational understanding of current transformer designs to inform novel architectural experiments.

Technical onboarding

Accelerate the ramp-up process for new engineers joining specialized AI infrastructure teams.

Performance optimization

Analyze the computational bottlenecks within training loops to improve hardware utilization and reduce costs.

Getting started: git clone https://github.com/Lordog/dive-into-llms.git && pip install -r requirements.txt

README

main branch

《动手学大模型》系列编程实践教程

version Status-building PRs-Welcome stars FORK Issues

💡 Updates

2025/06/06 感谢各位朋友们的关注和积极反馈!我们从以下两个方面对本教程进行了更新:

  • 上线国产化《大模型开发全流程》公益教程(含PPT、实验手册和视频),此处特别感谢华为昇腾社区的支持!
  • 在原系列编程实践教程的基础上进行内容更新,并增加了新的主题(数学推理、GUI Agent、大模型对齐、隐写术等)!

🎯 项目动机

《动手学大模型》系列编程实践教程,由上海交通大学《自然语言处理前沿技术》(NIS8021)、《人工智能安全技术》课程(NIS3353)讲义拓展而来(教师:张倬胜),旨在提供大模型相关的入门编程参考。本教程属公益性质、完全免费。通过简单实践,帮助同学们快速入门大模型,更好地开展课程设计或学术研究。

📚 教程目录

教程内容 简介 地址
微调与部署 预训练模型微调与部署指南:想提升预训练模型在指定任务上的性能?让我们选择合适的预训练模型,在特定任务上进行微调,并将微调后的模型部署成方便使用的Demo! [课件] [教程] [脚本]
提示学习与思维链 大模型的API调用与推理指南:“AI在线求鼓励?大模型对一些问题的回答令人大跌眼镜,但它可能只是想要一句「鼓励」” [课件] [教程] [脚本]
知识编辑 语言模型的编辑方法和工具:想操控语言模型在对指定知识的记忆?让我们选择合适的编辑方法,对特定知识进行编辑,并将对编辑后的模型进行验证! [课件] [教程] [脚本]
数学推理 如何让大模型学会数学推理?让我们快速蒸馏一个迷你R1! [课件] [教程] [脚本]
模型水印 语言模型的文本水印:在语言模型生成的内容中嵌入人类不可见的水印 [课件] [教程] [脚本]
越狱攻击 想要得到更好的安全,要先从学会攻击开始。让我们了解越狱攻击如何撬开大模型的嘴! [课件] [教程] [脚本]
大模型隐写 “看不见的墨水”!想让大模型在流畅回答的同时,悄悄携带只有“自己人”能识别的信息吗?大模型隐写告诉你! [课件] [教程] [脚本]
多模态模型 作为能够更充分模拟真实世界的多模态大语言模型,其如何实现更强大的多模态理解和生成能力?多模态大语言模型是否能够帮助实现AGI? [课件] [教程] [脚本]
GUI智能体 想要饭来张口、解放双手?那么让我们一起来让AI Agent替你点外卖、回消息、购物比价吧! [课件] [教程] [脚本]
智能体安全 大模型智能体迈向了未来操作系统之旅。然而,大模型在开放智能体场景中能意识到风险威胁吗? [课件] [教程] [脚本]
RLHF安全对齐 基于PPO的RLHF实验指南:本教程”十分危险“,阅读后请检查你的大模型是否在冷笑。 [课件] [教程] [脚本]

🔥 新上线:国产化《大模型开发全流程》

  • ✨ 我们联合华为昇腾推出的《大模型开发全流程》公益教程正式上线!前沿技术+代码实践,手把手带你玩转AI大模型 ✨:

    在《动手学大模型》原系列教程的基础上,我们联合华为开发了《大模型开发全流程》系列课程。本系列教程基于昇腾基础软硬件开发,覆盖PPT、实验手册、视频等教程形式。该教程分为初级、中级、高级系列,面向不同的大模型实践需求,旨在将前沿技术通过代码实践的方式,为相关研究者、开发者由浅入深地提供快速上手、应用昇腾已支持模型和全新模型迁移调优的全流程开发指南。

  • 🚀 前往昇腾社区探索《大模型开发全流程》系列课程:

    👉《大模型开发学习专区》@ 昇腾社区 👈

  • ✨ 课程内容展示 ✨

🙏 免责声明

本教程所有内容仅仅来自于贡献者的个人经验、互联网数据、日常科研工作中的相关积累。所有技巧仅供参考,不保证百分百正确。若有任何问题,欢迎提交 Issue 或 PR。另本项目所用徽章来自互联网,如侵犯了您的图片版权请联系我们删除,谢谢。

🤝 欢迎贡献

本教程目前是一个正在进行中的项目,如有疏漏在所难免,欢迎任何的PR及issue讨论。

❤️ 贡献者列表

感谢以下老师和同学对本项目的支持与贡献:

《动手学大模型》系列教程开发团队:

《大模型开发全流程》系列教程开发团队:

🌟 Star History

Star History Chart

View on GitHub

Recent activity

commits and pull requests

Releases and announcements

1 total
  1. dive-into-llmsv1Jun 12, 2025

Code frequency

additions and deletions

Commits per week

last 52 weeks

When work happens

weekday and hour
SunMonTueWedThuFriSat036912151821Sun 0:00 — 0 commitsSun 1:00 — 0 commitsSun 2:00 — 0 commitsSun 3:00 — 0 commitsSun 4:00 — 0 commitsSun 5:00 — 0 commitsSun 6:00 — 0 commitsSun 7:00 — 0 commitsSun 8:00 — 0 commitsSun 9:00 — 0 commitsSun 10:00 — 0 commitsSun 11:00 — 0 commitsSun 12:00 — 1 commitsSun 13:00 — 0 commitsSun 14:00 — 0 commitsSun 15:00 — 0 commitsSun 16:00 — 1 commitsSun 17:00 — 0 commitsSun 18:00 — 0 commitsSun 19:00 — 0 commitsSun 20:00 — 0 commitsSun 21:00 — 0 commitsSun 22:00 — 0 commitsSun 23:00 — 0 commitsMon 0:00 — 0 commitsMon 1:00 — 0 commitsMon 2:00 — 0 commitsMon 3:00 — 0 commitsMon 4:00 — 0 commitsMon 5:00 — 0 commitsMon 6:00 — 0 commitsMon 7:00 — 0 commitsMon 8:00 — 0 commitsMon 9:00 — 0 commitsMon 10:00 — 0 commitsMon 11:00 — 0 commitsMon 12:00 — 0 commitsMon 13:00 — 0 commitsMon 14:00 — 0 commitsMon 15:00 — 0 commitsMon 16:00 — 0 commitsMon 17:00 — 6 commitsMon 18:00 — 3 commitsMon 19:00 — 1 commitsMon 20:00 — 2 commitsMon 21:00 — 4 commitsMon 22:00 — 0 commitsMon 23:00 — 0 commitsTue 0:00 — 0 commitsTue 1:00 — 0 commitsTue 2:00 — 0 commitsTue 3:00 — 0 commitsTue 4:00 — 0 commitsTue 5:00 — 0 commitsTue 6:00 — 0 commitsTue 7:00 — 0 commitsTue 8:00 — 0 commitsTue 9:00 — 0 commitsTue 10:00 — 0 commitsTue 11:00 — 1 commitsTue 12:00 — 3 commitsTue 13:00 — 0 commitsTue 14:00 — 1 commitsTue 15:00 — 2 commitsTue 16:00 — 4 commitsTue 17:00 — 0 commitsTue 18:00 — 3 commitsTue 19:00 — 0 commitsTue 20:00 — 3 commitsTue 21:00 — 0 commitsTue 22:00 — 0 commitsTue 23:00 — 0 commitsWed 0:00 — 0 commitsWed 1:00 — 0 commitsWed 2:00 — 0 commitsWed 3:00 — 0 commitsWed 4:00 — 0 commitsWed 5:00 — 1 commitsWed 6:00 — 0 commitsWed 7:00 — 0 commitsWed 8:00 — 0 commitsWed 9:00 — 0 commitsWed 10:00 — 1 commitsWed 11:00 — 1 commitsWed 12:00 — 0 commitsWed 13:00 — 0 commitsWed 14:00 — 0 commitsWed 15:00 — 0 commitsWed 16:00 — 0 commitsWed 17:00 — 1 commitsWed 18:00 — 1 commitsWed 19:00 — 0 commitsWed 20:00 — 0 commitsWed 21:00 — 2 commitsWed 22:00 — 0 commitsWed 23:00 — 0 commitsThu 0:00 — 0 commitsThu 1:00 — 0 commitsThu 2:00 — 0 commitsThu 3:00 — 0 commitsThu 4:00 — 0 commitsThu 5:00 — 0 commitsThu 6:00 — 0 commitsThu 7:00 — 0 commitsThu 8:00 — 2 commitsThu 9:00 — 0 commitsThu 10:00 — 0 commitsThu 11:00 — 1 commitsThu 12:00 — 4 commitsThu 13:00 — 1 commitsThu 14:00 — 0 commitsThu 15:00 — 1 commitsThu 16:00 — 0 commitsThu 17:00 — 0 commitsThu 18:00 — 2 commitsThu 19:00 — 0 commitsThu 20:00 — 0 commitsThu 21:00 — 0 commitsThu 22:00 — 0 commitsThu 23:00 — 1 commitsFri 0:00 — 0 commitsFri 1:00 — 1 commitsFri 2:00 — 0 commitsFri 3:00 — 0 commitsFri 4:00 — 0 commitsFri 5:00 — 0 commitsFri 6:00 — 0 commitsFri 7:00 — 0 commitsFri 8:00 — 0 commitsFri 9:00 — 0 commitsFri 10:00 — 1 commitsFri 11:00 — 0 commitsFri 12:00 — 2 commitsFri 13:00 — 0 commitsFri 14:00 — 1 commitsFri 15:00 — 0 commitsFri 16:00 — 0 commitsFri 17:00 — 0 commitsFri 18:00 — 0 commitsFri 19:00 — 0 commitsFri 20:00 — 0 commitsFri 21:00 — 0 commitsFri 22:00 — 0 commitsFri 23:00 — 0 commitsSat 0:00 — 0 commitsSat 1:00 — 0 commitsSat 2:00 — 0 commitsSat 3:00 — 0 commitsSat 4:00 — 0 commitsSat 5:00 — 0 commitsSat 6:00 — 0 commitsSat 7:00 — 0 commitsSat 8:00 — 0 commitsSat 9:00 — 0 commitsSat 10:00 — 0 commitsSat 11:00 — 0 commitsSat 12:00 — 0 commitsSat 13:00 — 0 commitsSat 14:00 — 0 commitsSat 15:00 — 0 commitsSat 16:00 — 0 commitsSat 17:00 — 0 commitsSat 18:00 — 0 commitsSat 19:00 — 0 commitsSat 20:00 — 0 commitsSat 21:00 — 0 commitsSat 22:00 — 0 commitsSat 23:00 — 0 commits
Commit volume by weekday and hour (UTC). Larger dots mean more commits.
DateListRankStars gained
Jul 23, 2026daily#16+5
Apr 17, 2026daily#11+144
Apr 16, 2026daily#9+162
Apr 15, 2026daily#18+123
Apr 14, 2026daily#22+107
Jan 26, 2026daily#24+118
Jan 24, 2026daily#8+350
Jan 23, 2026daily#7+410
Jan 22, 2026daily#4+467