Abstract:A novel neural network for object recognition, CapsNet, uses dynamic routing and capsules to recognize novel state of a known object, while the input layer of CapsNet decoder increases when the number of categories increases, which means a relatively limited scalability. To overcome this weakness, we propose the Multi-branches Auto-Encoder (MAE) which gives coding vectors of every class to the decoder respectively letting the scale of decoder independent from the number of categories enhancing the representation capability of the proposed model. The experiment on MNIST shows that MAE is competitive in recognition and more powerful in reconstruction which means a more complete capability on representation.