Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy
We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space. Direct comparison under the same search space shows that our method is up to 5 times more efficient than the RL method of Zoph et al. (2018) in terms of number of models evaluated, and 8 times faster in terms of total compute. The structures we discover in this way achieve state of the art classification accuracies on CIFAR-10 and ImageNet.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Neural Architecture Search | NAS-Bench-201, ImageNet-16-120 | Accuracy (Val) | 44.75 | PNAS + |
| Neural Architecture Search | ImageNet | Params | 5.1 | PNAS |
| Image Classification | ImageNet | GFLOPs | 50 | PNASNet-5 |
| Image Classification | ImageNet | Top 5 Accuracy | 96.2 | PNASNet-5 |
| AutoML | NAS-Bench-201, ImageNet-16-120 | Accuracy (Val) | 44.75 | PNAS + |
| AutoML | ImageNet | Params | 5.1 | PNAS |