Yong-Lu Li, Yue Xu, Xiaohan Mao, Cewu Lu
Attributes and objects can compose diverse compositions. To model the compositional nature of these general concepts, it is a good choice to learn them through transformations, such as coupling and decoupling. However, complex transformations need to satisfy specific principles to guarantee the rationality. In this paper, we first propose a previously ignored principle of attribute-object transformation: Symmetry. For example, coupling peeled-apple with attribute peeled should result in peeled-apple, and decoupling peeled from apple should still output apple. Incorporating the symmetry principle, a transformation framework inspired by group theory is built, i.e. SymNet. SymNet consists of two modules, Coupling Network and Decoupling Network. With the group axioms and symmetry property as objectives, we adopt Deep Neural Networks to implement SymNet and train it in an end-to-end paradigm. Moreover, we propose a Relative Moving Distance (RMD) based recognition method to utilize the attribute change instead of the attribute pattern itself to classify attributes. Our symmetry learning can be utilized for the Compositional Zero-Shot Learning task and outperforms the state-of-the-art on widely-used benchmarks. Code is available at https://github.com/DirtyHarryLYL/SymNet.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Zero-Shot Learning | MIT-States, generalized split | H-Mean | 16.1 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Seen accuracy | 24.4 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Test AUC top 1 | 3 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Test AUC top 2 | 7.6 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Test AUC top 3 | 12.3 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Unseen accuracy | 25.2 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Val AUC top 1 | 4.3 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Val AUC top 2 | 9.8 | SymNet |
| Zero-Shot Learning | MIT-States, generalized split | Val AUC top 3 | 14.8 | SymNet |
| Zero-Shot Learning | MIT-States | Top-1 accuracy % | 19.9 | SymNet |
| Zero-Shot Learning | MIT-States | Top-2 accuracy % | 28.2 | SymNet |
| Zero-Shot Learning | MIT-States | Top-3 accuracy % | 33.8 | SymNet |
| Zero-Shot Learning | UT-Zappos | Top-1 accuracy % | 52.1 | SymNet |
| Zero-Shot Learning | UT-Zappos | Top-2 accuracy % | 67.8 | SymNet |
| Zero-Shot Learning | UT-Zappos | Top-3 accuracy % | 76 | SymNet |