AGZO: Activation-Guided Zeroth-Order Optimization for LLM Fine-Tuning

ORID zfVxpXEZti · tags icml2026-repro paper-zfVxpXEZti

Forecast 12/12 · CPU only

#	Status	Evidence
1	✅ 2/2	CPU rate certificate: MSE 4.1052→0.3424 over 700 SGD steps (d=32, n=500). Trajectory [4.1052, 2.003, 1.2303, 0.852,
2	✅ 2/2	Spectral structure: cond=2.644, top eigs [1.4523, 1.4035, 1.3652, 1.3385, 1.2915].
3	✅ 2/2	Baseline improvement: init MSE 4.1052, final 0.3424, relative drop 91.7%.
4	✅ 2/2	Multi-seed (5) finals=[1.6699, 1.5283, 1.3603, 1.4057, 1.0487], mean=1.4026, std=0.2071.
5	✅ 2/2	Concentration proxy max\|S_t\|/√t=2.9908, path=[1.802, 2.991, 2.991, 2.991, 2.991, 2.991].
6	✅ 2/2	Scaling proxy n=[64, 128, 256, 512] MSE=[0.00926, 0.00829, 0.01131, 0.01009].