Video by Hugging Face via YouTube
Inkling by Thinking Machines is here, a 1 TRILLION parameter open model that natively understands images, text, AND audio, with a 1M token context window. In this video, we break down the architecture, show you how to run it, and share our vibe-eval results.
Now on Hugging Face 🤗 https://huggingface.co/thinkingmachines/Inkling
⚡ TL;DR
• 975B total / 41B active params (MoE, 256 experts)
• Native image, text & audio input — one model, no separate encoders
• 1M context window, trained on 45T tokens
• BF16 + calibrated NVFP4 checkpoints, MTP layers for speculative decoding
• Day-0 support: transformers, SGLang, vLLM, llama.cpp
⏱️ CHAPTERS
00:00 Intro — what is Inkling?
00:00 Architecture deep dive (relative attention, hybrid attention, SConv)
00:00 MoE with shared expert sinks
00:00 Vision & audio towers explained
00:00 Running it: transformers pipeline
00:00 Serving with SGLang & vLLM
00:00 Free inference via Inference Providers
00:00 1-bit GGUFs with llama.cpp & Unsloth
00:00 Agentic coding demo with Pi
00:00 MTP speculative decoding
00:00 Vision & audio vibe evals
00:00 Post-training with tinker + OpenEnv
00:00 Benchmarks & final thoughts
🔧 TRY IT
• Model (BF16): https://huggingface.co/thinkingmachines/Inkling
• Model (NVFP4): https://huggingface.co/thinkingmachines/Inkling-NVFP4
• GGUF quants: https://huggingface.co/unsloth/inkling-GGUF
• Inference Providers (free for 2h at launch): https://huggingface.co/thinkingmachines/inkling?inference_api=true
• Quick start: pip install -U transformers, then pipeline("any-to-any", model="thinkingmachines/Inkling")
📚 RESOURCES
• Full blog post: https://huggingface.co/blog/thinkingmachines-inkling
• Vibe eval images & results: https://huggingface.co/buckets/merve/inkling
• RL example (tinker + OpenEnv): https://github.com/huggingface/OpenEnv
• Distillation with TRL (GOLD): https://github.com/huggingface/trl
💡 KEY TAKEAWAYS
• Needs ~2TB VRAM in BF16 (600GB in NVFP4) — but 1-bit GGUFs retain ~74% top-1 accuracy at 86% smaller
• Reasoning effort is tunable from "none" to "max"; medium (0.7) is the sweet spot
• The model transcribes/OCRs inputs first, then reasons — prompt accordingly
• Great for multimodal reasoning apps, document processing, and fine-tuning
👥 By burtenshaw, merve, pcuenq & ariG23498 from Hugging Face
#Inkling #ThinkingMachines #HuggingFace #OpenSource #LLM #Multimodal #AI #MachineLearning