On Analysing Similarity Knowledge Transfer by Ensembles

• Danilo Pereira ^[1] ; Flávio Santos ^[1] ; Matos, Leonardo N. ^[2] ; Paulo Novais ^[3] ; Cleber Zanchettin ^[1] ; Ludermir, Teresa B. ^[1]
  1. [1] Universidade Federal de Pernambuco

     Universidade Federal de Pernambuco

     Brasil

     
  2. [2] Universidade Federal de Sergipe

     Universidade Federal de Sergipe

     Brasil

     
  3. [3] Universidade do Minho

     Universidade do Minho

     Braga (São José de São Lázaro), Portugal

     

  Mostrar afiliaciones +
• Localización: Intelligent Data Engineering and Automated Learning – IDEAL 2020. 21st International Conference: Guimarães, Portugal; November 4–6, 2020. Proceedings / Cesar Analide (ed. lit.), Paulo Novais (ed. lit.), David Camacho Fernández (ed. lit.), Hujun Yin (ed. lit.), Vol. 2, 2020 (Part II), ISBN 978-3-030-62365-4, págs. 202-210
• Idioma: inglés
• Enlaces
  • Texto completo
• Resumen

  • Knowledge transfer is the task of transferring the knowledge learned by a model A to a new model B. This task is essential in Deep Learning, since there are complex models with excellent results, but computationally costly to be executed. The Similarity Knowledge Transfer (SKT) method proposes an approach to transfer the knowledge layer-bylayer between a donor model and a receiver model. This transfer is carried out through the representations learned by the layers from the teacher model. Despite presenting good results, the SKT method proposes just a way to transfer knowledge between two models. Therefore, this work presents the Similarity Knowledge Transfer Ensemble (SKTE) method, a generic form of SKT that allows the transfer from several teachers to a single student model. We carried out experiments with the CIFAR10 benchmark, where the results obtained showed promising results in this activity.