https://ndthuan.com/{ndt.write();}Software engineer exploring AI, software systems, and how technology changes the way we work. 2026-10-12T14:20:42+07:00 Thuan Nguyen https://ndthuan.com/ Jekyll © 2026 Thuan Nguyen /assets/img/favicons/favicon.ico /assets/img/favicons/favicon-96x96.png EmbeddingGemma 2 and Qwen3-Embedding vs bge-m3: Why I Kept bge-m32026-10-08T20:00:00+07:00 2026-10-08T20:00:00+07:00 https://ndthuan.com/embeddinggemma-2-qwen3-embedding-vs-bge-m3/ Thuan Nguyen EmbeddingGemma 2 and Qwen3-Embedding 0.6B against bge-m3 on one workload, clustering news articles from Vietnamese newspapers into stories. Quality was level, each model has its own cosine scale, and the one real gap covers under one percent of pairs. Why I Kept My Own LLM Gateway over LiteLLM and 9Router2026-10-06T15:25:00+07:00 2026-10-06T15:25:00+07:00 https://ndthuan.com/own-llm-gateway-over-litellm-and-9router/ Thuan Nguyen I put four model profiles on LiteLLM, sent back two comments and two issues, read how 9Router handles the same cases, and still run 3,000 lines of my own. Here is what worked, what did not, and why. Retry Storms: How Client Retries Turn a Short Dip into a Long Outage2026-10-04T08:00:00+07:00 2026-10-04T08:00:00+07:00 https://ndthuan.com/retry-storms-backoff-jitter-retry-budgets/ Thuan Nguyen A two-second capacity dip, four retry policies and one retry budget, simulated in about a hundred lines of Go: how retries multiply load, why jitter helps, and what a budget changes. The GIL Bottleneck Hiding in Self-Hosted LLM Serving2026-09-25T12:00:00+07:00 2026-09-25T12:00:00+07:00 https://ndthuan.com/gil-bottleneck-self-hosted-llm-serving/ Thuan Nguyen Why a custom logits processor that runs once per request can dominate tail latency even when the GPU batches perfectly, and the batch-level rewrite that fixes it. Open-weight LLMs are winning on cost, not benchmarks2026-09-13T10:00:00+07:00 2026-09-13T10:00:00+07:00 https://ndthuan.com/open-weight-models-winning-on-cost/ Thuan Nguyen Open-weight models are not catching up to frontier APIs on benchmarks. They are already ahead on the number that decides most production budgets: cost per request at scale.