You are here

Home » Content

Neuroevolution of Autoencoders by Genetic Algorithm

Journal Name:

  • International Journal of Science and Engineering Investigations

Publication Year:

  • 2017

Key Words:

Author NameUniversity of AuthorFaculty of Author
Abstract (2. Language): 
In this paper, the author experimentally evaluates the ability of a genetic algorithm in evolutionary training of autoencoders. An autoencoder is a component of a deep neural network known as a stacked autoencoder. Optimization of neural networks by means of evolutionary algorithms is called neuroevolution. Weights and biases in an autoencoder are optimized by the genetic algorithm so that the autoencoder can precisely reproduce its input data. A dataset of handwritten digits is used in the experiment. Results showed that the genetic algorithm could evolve autoencoders that reproduced the training and test data better as the autoencoders included more hidden units. A clear trade-off was observed between the reproduction accuracy and the encoding efficiency.
Bookmark/Search this post with
FULL TEXT (PDF): 
PDF icon arastirmax-neuroevolution-autoencoders-genetic-algorithm.pdf
  • 6
127
131
  • English

REFERENCES

References: 

[1] Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504-507.
[2] Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural computation, 18(7), 1527-1554.
[3] Boureau, Y. L., & Cun, Y. L. (2008). Sparse feature learning for deep belief networks. In Advances in neural information processing systems (pp. 1185-1192).
[4] Sutskever, I., & Hinton, G. E. (2008). Deep, narrow sigmoid belief networks are universal approximators. Neural Computation, 20(11), 2629-2636.
[5] Bengio, Y. (2009). Learning deep architectures for AI. Foundations and trends in Machine Learning, 2(1), 1-127.
[6] Larochelle, H., Bengio, Y., Louradour, J., & Lamblin, P. (2009). Exploring strategies for training deep neural networks. Journal of Machine Learning Research, 10(Jan), 1-40.
[7] Tan, C. C., & Eswaran, C. (2010). Autoencoder Neural Networks: A Performance Study Based on Image Reconstruction, Recognition and Compression. LAP Lambert Academic Publishing.
[8] Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., & Manzagol, P. A. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11(Dec), 3371-3408.
[9] Salakhutdinov, R., & Hinton, G. (2012). An efficient learning procedure for deep Boltzmann machines. Neural computation, 24(8), 1967-2006.
[10] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
[11] Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828.
[12] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
[13] Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.
[14] Zhang, S., Choromanska, A. E., & LeCun, Y. (2015). Deep learning with elastic averaging SGD. In Advances in Neural Information Processing Systems, 685-693.
[15] Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1988). Learning representations by back-propagating errors. In Neurocomputing: foundations of research, James A. Anderson and Edward Rosenfeld (Eds.). MIT Press, Cambridge, MA, USA, 696-699.
[16] Williams, D. R. G. H. R., & Hinton, G. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-538.
[17] Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423-1447.
[18] Floreano, D., Dürr, P., & Mattiussi, C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1(1), 47-62.
[19] Montana, D. J., & Davis, L. (1989). Training feedforward neural networks using genetic algorithms. IJCAI, 89, 762-767.
[20] Sexton, R. S., Dorsey, R. E., & Johnson, J. D. (1998). Toward global optimization of neural networks: a comparison of the genetic algorithm and backpropagation. Decision Support Systems, 22(2), 171-185.
[21] Sexton, R. S., & Gupta, J. N. (2000). Comparative evaluation of genetic algorithm and backpropagation for training neural networks. Information Sciences, 129(1), 45-59.
[22] Örkcü, H. H., & Bal, H. (2011). Comparing performances of backpropagation and genetic algorithms in the data classification. Expert systems with applications, 38(4), 3703-3709.
[23] Joy, C. U. (2011). Comparing the Performance of Backpropagation Algorithm and Genetic Algorithms in Pattern Recognition Problems. International Journal of Computer Information Systems, 2(5), 7-12.
[24] Che, Z. G., Chiang, T. A., & Che, Z. H. (2011). Feed-forward neural networks training: A comparison between genetic algorithm and back-propagation learning algorithm. International Journal of Innovative Computing, Information and Control, 7(10), 5839-5850.
[25] Goldberg, D. E., & Holland, J. H. (1988). Genetic algorithms and machine learning. Machine learning, 3(2), 95-99.
[26] Goldberg, D. E., & Holland, J. H. (1988). Genetic algorithms and machine learning. Machine learning, 3(2), 95-99.
[27] Eshelman Larry, J., & Schaffer David, J. Real-coded Genetic Algorithms and Interval-Schemata. Foundations of Genetic Algorithms 2, 187-202, 1993
[28] Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013, May). Improving deep neural networks for LVCSR using rectified linear units and dropout. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on (pp. 8609-8613). IEEE.
[29] Gal, Y., & Ghahramani, Z. (2016, June). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning (pp. 1050-1059).
[30] Ba, J., & Frey, B. (2013). Adaptive dropout for training deep neural networks. In Advances in Neural Information Processing Systems (pp. 3084-3092).
[31] Boureau, Y. L., & Cun, Y. L. (2008). Sparse feature learning for deep belief networks. In Advances in neural information processing systems (pp. 1185-1192).
[32] Ng, A. (2011). Sparse autoencoder. CS294A Lecture notes, 72(2011), 1-19.

Thank you for copying data from http://www.arastirmax.com