Drawing Music: Using Neural Networks to Compose Descriptive Music from Illustrations

• Autores: Lucía Martín Gómez, Javier Pérez Marcos, Alfonso José López Rivero, Giovanny Mauricio Tarazona Bermúdez
• Localización: New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence: The DITTET 2022 Collection / Daniel Hernández de la Iglesia (ed. lit.), Juan Francisco de Paz Santana (ed. lit.), Alfonso José López Rivero (ed. lit.), 2023, ISBN 978-3-031-14858-3, págs. 30-42
• Idioma: español
• Texto completo no disponible (Saber más ...)
• Resumen

  • The creative capacity of machines is still questioned by researchers and users alike. For this reason, computational creativity does not only focus on the development of machines for the creation of artistic content but also on the evaluation of the generated content. This works presents a system that composes polyphonic music from the drawings of a user in real time. Our proposal provides an analysis of the Fantasia film, produced in 1940 by Walt Disney and deduces the relationship between its audio and images. As part of system development, an LSTM-based Recurrent Neural Network was trained with MIDI music files and a model was obtained. As a result, the proposed system generates polyphonic music with expressive timing and dynamics by inferring chords from the user’s drawings. To assess the creative ability of the machine a Turing test was conducted and the quality of the interconnection between drawings and music was measured by another user test. Additionally, the performance of the considered classifiers is discussed.