This report details the creation of Bielik-Minitron-7B, a compressed 7.35B parameter version of the Bielik-11B-v3.0 model, specifically optimized for European languages. By leveraging a two-stage compression methodology inspired by the NVIDIA Minitron approach, we combined structured hybrid pruning and knowledge distillation to reduce the model's parameter count by 33.4%, from 11.04B to 7.35B. We utilized the NVIDIA Model Optimizer for structural pruning and the NVIDIA NeMo Framework for logit-based distillation for quality recovery. Following distillation, the model underwent a rigorous alignment pipeline consisting of Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO-P), and Reinforcement Learning (GRPO). Our final model successfully recovered approximately 90% of the baseline model's performance while providing up to 50% inference speedup. This approach demonstrates an efficient pathway to create language models for less-represented languages, preserving the original model quality while reducing inference deployment costs.
Alex: Welcome to another episode of ResearchPod. Sam, I've been reading about efforts to make powerful language AI work better for languages beyond English, like Polish. What's this particular study tackling?
Sam: This report describes creating Bielik-Minitron-7B, a smaller version of a large Polish language model called Bielik-11B-v3.0. The core achievement is shrinking the model by removing about a third of its internal parts—from 11 billion to 7.35 billion parameters—while keeping most of its abilities intact through careful trimming and retraining. It ends up running 50% faster on everyday computer hardware, which matters for places like Polish clinics needing quick AI answers to medical questions without massive servers.
Alex: So this is basically solving the puzzle of how to make a top-tier Polish AI model fit on normal GPUs like an RTX 4090, without losing much smarts? The big ones overload them, right?
Sam: Exactly. Large language models excel at understanding and generating text in specific languages, but their size demands huge amounts of memory and processing power—think billions of tiny adjustable numbers inside that store what they've learned from training on vast text data. For less common languages like Polish, building these from scratch is costly, and running the full versions slows down real-world use, such as in healthcare apps. This work shows a path to compress them efficiently, recovering 90% of the original performance.
Alex: Right, so the challenge isn't just making AI good at Polish—it's making it fast and cheap enough to actually use every day. That sets up why they'd go for this compression approach.
Sam: Yes, and the key is a two-step method: first, structured pruning to cut out less useful sections of the model's structure—like removing redundant layers or inactive neuron groups in a targeted way for instant speed gains. Think of it like trimming extra branches from a tree to make it lighter without losing its shape. Then, knowledge distillation, where the smaller model learns directly from the original's outputs on Polish text, mimicking its behavior without starting over.
Alex: So it's like taking a bulky bodybuilder and trimming fat surgically, then coaching a leaner athlete to match the strength. But doesn't cutting parts risk breaking the model's reasoning?
Sam: A fair concern. The pruning uses activation patterns from sample data to identify low-impact components—those that barely change the model's internal signals—and removes them in hardware-friendly chunks. Parts that stayed quiet or barely shifted the data flow got low scores and were targeted first, since they added little value. Distillation then fine-tunes recovery, plus alignment steps like supervised fine-tuning to polish responses.
Alex: Interesting that it holds up on benchmarks. How did they decide exactly which parts to cut—like, what made a layer or neuron low-impact enough to remove safely?
Sam: They ran the model on a small set of sample Polish texts to watch its internal activity—measuring how much each part lit up or changed signals during normal use. For whole layers, they compared how much each one transformed the incoming data versus passing it through mostly unchanged—like checking if a filter in a photo app actually alters the picture or just copies it. Layers with the least change got removed. They tested 10 different combos of cuts, picking the one that hit 7.35 billion parameters with the least performance drop.
Alex: And after those cuts, how does the distillation step glue it back together without labeled data?
Sam: The smaller model starts with the kept weights from the big one, then trains only on matching the teacher's full predictions across the whole vocabulary—not just right answers, but probabilities for every possible word. It's like the student copies the teacher's thought process in full detail, not just the final answers. This captures relationships between words and confidence levels that labels alone miss, done on unlabeled Polish text.
Alex: That explains holding onto Polish nuances while running twice as fast on clinic hardware. But after distillation, what turns this base model into something safe and helpful for real questions, like medical ones?
Sam: They add a multi-stage alignment process. First, supervised fine-tuning: the model practices on millions of example questions and answers in Polish and English, learning to respond helpfully—like drilling a student on sample tests until patterns stick. Next comes preference alignment using Direct Preference Optimization, or DPO. They show the model pairs of answers to the same question—one good, one bad—and train it to favor the preferred one.
Alex: And the final step?
Sam: Last is reinforcement learning via Group Relative Policy Optimization, or GRPO. For logic and math tasks, it compares groups of its own answers to pick better reasoning paths—like a team reviewing plays to refine strategy. The paper reports the student model retains about 90% of the teacher's overall performance after these steps.
Alex: A meaningful recovery for deployment. But on actual Polish medical tests—like board certification questions—did the smaller model hold up as well as the original?
Sam: It did, retaining over 94% of the teacher's score on those exams. The v3.0 teacher had improved through better multilingual training data, passing that refined medical knowledge base intact via distillation. On a benchmark testing complex Polish reading—spotting implications, tricky phrases, and feelings—it matched an earlier 11-billion version while using 35% fewer parts.
Alex: Huh, so the teacher's upgrades carried over without much loss. What about where recovery wasn't as strong?
Sam: The analysis shows bigger drops in areas like common-sense reasoning and factual knowledge, with recoveries around 75 to 89 percent. Dense details and multi-step logic proved most sensitive to the cuts, as the paper notes—pruning inevitably trims some raw storage capacity. Still, linguistic tasks and summarization held over 95 percent.
Alex: So not everything recovered equally—knowledge took the biggest hit. What about squeezing it even smaller for everyday hardware?
Sam: They tested quantization, which rounds the model's numbers to lower precision—like simplifying a detailed recipe to quicker measurements—to cut memory use further. A 4-bit version lost just under one percent on Polish benchmarks. On an NVIDIA RTX PRO 6000, the smaller model generated text about 50 percent faster overall.
Alex: A balanced trade-off, then.
Sam: Yes—the paper highlights depth as trickier than width pruning, and distillation as essential for fidelity. This blueprint scales high-quality models for under-resourced languages like Polish to consumer GPUs, enabling reliable medical Q&A at double the speed.
Alex: A practical path forward without enterprise gear. That's a meaningful contribution for localized AI. Thanks for breaking it down, Sam. Thanks for listening to ResearchPod.