Hypernet
Hypernet起源于2017年 iclr的一篇文章 hypernetworks
In this work, we consider an approach of using a small network (called a “hypernetwork") to generate the weights for a larger network (called a main network). The behavior of the main network is the same as with any usual neural network: it learns to map some raw inputs to their desired targets; whereas the hypernetwork takes a set of inputs that contain information about the structure of the weights and generates the weights for that layer.
Hypernet 在 Module Parameter Generation中起到了重要的作用
需要去阅读的文章：
- https://aclanthology.org/2021.tacl-1.25/
- https://proceedings.mlr.press/v162/he22f/he22f.pdf
- https://aclanthology.org/2021.acl-long.47/
- https://arxiv.org/abs/2205.12148
