Bayesian Optimization for Neural Architecture Search using Graph Kernels

Neural architecture search is a popular method for automating architecture design. Bayesian optimization is a widely used approach for hyper-parameter optimization and can estimate a function with limited samples. However, Bayesian optimization methods are not preferred for architecture search as it expects vector inputs while graphs are high dimensional data. This thesis presents a Bayesian approach with Gaussian priors that use graph kernels specifically targeted to work in the higher-dimensional graph space. We implemented three different graph kernels and show that on the NAS-Bench-101 dataset, even an untrained GCN kernel outperforms previous methods significantly in terms of the best network found and the number of samples required to find it. We follow the NAS guidelines to make this work reproducible.