Tong Niu
AI researcher. I build large language models, retrieval systems, and speech models — and ship them.
I'm Tong (I also go by Ben; Niu sounds like “new”). Over the past nine years I have worked on LLMs, retrieval, and natural language generation: most recently an on-device speech model at Archer Aviation, before that vision-language models at Meta Superintelligence Labs, and for six years at Salesforce AI Research, where I built JudgeRank (#1 on BRIGHT), co-developed the SFR-Embedding model (#1 on MTEB), and generated a regression test suite with LLMs that saves millions of dollars per release.
Lately I also prove small theorems in combinatorics for fun. Earlier I wrote two books and founded an English school. This site is where I think out loud.
Latest thoughts
All thoughts →- Neural networks need sleep
We borrowed neurons from the brain and left out sleep. In a toy test, a network that prunes and dreams every night keeps its old task at 96%. One that studies all night falls to 68%.
- Quantization is just a histogram
A 4-bit model is a 16-bar histogram of its own weights. Bit-width, calibration, and NF4 all fall out of that one picture.
News
All news →This site is rebuilt from scratch
New home for my thoughts, publications and patents, and CV. First post: Quantization is just a histogram.