Now
What I'm building, testing, and trying to understand right now.
Updated 2026-09-12
LoRA Adapter ReuseCurrent project
LoRA Adapter Reuse
Summer 2026 research internship at the Institute of Information Science, Academia Sinica, continued part-time: once a LoRA adapter is trained, how much of it can be reused, on another model or merged with other adapters?
Current milestone
MergeBench reproduction with LoRA specialists: plan and code written, first GPU run still pending.
Testing now
Whether the portability and merge-vs-multitask results hold at other trim densities and merge scales, not only at the one operating point they were measured at.
Biggest blocker
GPU time. The remaining density × scale check is about an hour on one H100 and the MergeBench reproduction has not run at all; both wait on cluster access scheduled through the lab.
Current focus
Since July I have been working at the Institute of Information Science, Academia Sinica (a summer internship, now continued part-time as a research assistant) on one question: once a LoRA adapter is trained, how much of it can be reused? The cross-model transfer half is closed (what moves across is the source model's answer pattern, not its capability). The work now is merging several single-task adapters into one host model and measuring what survives.
What changed recently
Sep 10
Wrote the internal summary of the whole research line, every headline number re-derived from the raw run metrics.
Sep 03
Validated the 2×2 portability and merge-vs-multitask reports against the 65 raw run directories.
Aug 28
Started the MergeBench-with-LoRA reproduction plan.
Aug 24
Closed the origin × host portability study and the merge-vs-multitask control, three seeds.
Current experiment
Experiment
Origin × host 2×2: adapters trained on the base model applied to the instruct model and the reverse, three seeds each, plus one adapter trained on the mixed data as the control for merging.
Hypothesis
Differences between merge operators may be differences in update norm in disguise. Pinning every arm to the same realized norm before comparing them is the test.
Signal I'm watching
Whether any of the portability or merge-vs-multitask results changes sign somewhere in the density × scale grid. If one does, the write-up has to be restated as conditional on the operating point.
What counts as success
A MergeBench-style table where every cell has a control arm and a re-derived number, and a clear statement of which results are conditional on the operating point.
Writing
The site moved to blog.nhade.com in May but kept pointing at its old address. Fixed, and there is an RSS feed.
Part of dev-blog