19th December 2017

This paper presents the results of a psychological experiment in which we measured the perceived level of immersion and time spent in the experience in a virtual reality experience under three music conditions: no music, linear music and machine-generated deep adaptive music. We found that deep adaptive music increased both the time spent in the interactive experience and significantly amplified the immersion of participants.-

2011

Proceedings of ISMIR 2011

Creating an aesthetically pleasing melody is no simple task; indeed, the process has been studied and scrutinized for centuries. Even today, it is not easy for someone with little musical background to create a melody that sounds harmonious with a given chord progression, even if that melody is in the correct key signature. JS Bach is considered one of the greatest melody writers of all time, so in researching melodies, it is natural to look to the rules by which he wrote melodies: Counterpoint. The goal of this research is to derive the basic rules by which Counterpoint was founded, and to apply them to contemporary melody creation. Specifically, the focus is to correct a melody so that it consonates with a given harmony (eg a chord progression) after the notes are already in the correct key signature. The research presented here is the foundation of algorithms that have been incorporated in a new genre of commercially available music software and has found to be effective in bringing melodic composition more accessible to the musically un-trained mainstream.

2016

Proceedings of the Fourth International Workshop on Musical Metacreation

This research explores the generation of melodies through the creation of a probabilistic analytical model of melodies. Using a Natural Language Processing technique utilized for the automatic reduction of melodies: the Probabilistic Context-Free Grammar (PCFG), it is possible to reduce new incipit melodies, or to generate and embellish melodies. Automatic melodic reduction has been previously explored by means of a probabilistic grammar (Gilbert and Conklin 2007) (Abdallah and Gold 2014). However, these methods used unsupervised learning to estimate the probabilities for the grammar rules rather than a corpus-based evaluation. A treebank of analyses using the Generative Theory of Tonal Music (GTTM) exists (Hamanaka, Hirata, and Tojo 2007), which contains 300 Western tonal melodies and their corresponding melodic reductions in tree format. In this work, a new representation of the GTTM grammar is created using a higher-level representation based on intervals. Then, supervised learning is used to train a PCFG on the treebanks with different versions of the new data representation. The resulting model is evaluated on its ability to create accurate reduction trees, and its ability to generate melodies is subjectively explored. With this approach, each generated melody will not only be a sequence of pitches, but also a hierarchical structure containing the melodic reduction of the generated melody. Multiple data representations are tested, and example output reductions—and embellishments—are shown.

2015

This paper presents a computational approach to a partic- ular theory in the work of Julian Hook—Uniform Triadic Transforma- tions (UTTs). A UTT defines a function for transforming one chord into another, and is useful for explaining triadic transitions that circumvent traditional harmonic theory. By combining two UTTs and extrapolating, it is possible to create a two-dimensional chord graph. Meanwhile, graph theory has long been studied in the field of Computer Science. Dijkstra’s shortest-path algorithm, for example, provides a method for finding the shortest path between two points on a weighted graph. This work de- scribes a software tool which can compute the shortest path between two points in a two-dimensional transformational chord space. Utilizing computational techniques, it is then possible to find the optimal chord space for a given musical piece. The musical work of Michael Nyman is analyzed computationally, and the implications of a weighted chord graph are explored.

2016

This research explores a Natural Language Processing technique utilized for the automatic reduction of melodies: the Probabilistic Context-Free Grammar (PCFG). Au-tomatic melodic reduction was previously explored by means of a probabilistic grammar.However, each of these methods used unsupervised learning to estimate the probabilities for the grammar rules, and thus a corpus-based evaluation was not performed.

2013

2015

McGill University Libraries

The process of melodic reduction can also involve auxiliary melodic analysis methods such as melodic segmentation (or grouping), metrical structure analysis, and melodic parallelism. This thesis investigatesthe use of a technique originally developed for Natural Language Processing (NLP) to identify hierarchies in sequential data—the Probabilistic Context-Free Grammar(PCFG).

2013/11/13

Ninth Artificial Intelligence and Interactive Digital Entertainment Conference

Hidden Markov Models have been used frequently in the audio domain to identify underlying musical structure. Much less work has been done in the purely symbolic realm. Recently, a substantial amount of expertlabelled symbolic musical data has been injected into the research community. The new availability of data allows for the application of machine learning models to purely symbolic tasks. Similarly, the continued expansion of the field of machine learning provides new perspectives and implementations of machine learning methods, which are powerful tools when approaching complex musical challenges. This research explores the use of an extended probabilistic model such as the Hidden Semi-Markov Model (HSMM) to approach the task of automatic harmonization. One distinct advantage of the HSMM is that it is able to automatically differentiate harmonic boundaries, through its inclusion of an extra parameter: duration. In this way, a melody can be harmonized automatically in the style of a particular corpus. In the case of this research, the corpus was in the style of Rock’n’Roll.

2014/7/15

US

8779268

13194819

A system and process for producing a more harmonious musical accompaniment for a musical compilation, the process comprising determining a plurality of probable key signatures for the musical compilation, creating an interval profiling matrix for each of the probable key signatures, finding products of a major key interval profile matrix with each of the interval profiling matrices, summing each of the major key interval products into a running major key sum, finding a product of a minor key interval profile with each of the interval profiling matrices, summing each of the minor key interval products into a running minor key sum, and selecting the most probable key signature from the plurality of probable key signatures by comparing the minor key sum and the major key sum.

2016/2/2

US

8779268

13194819

A system and method of creating harmonizing tracks for an audio input. Audio input is received and a plurality of harmonizing tracks are created based on the received audio input. Each of the plurality of harmonizing tracks are transposed based on a transposition value for each respective track. Individual notes of the harmonizing tracks are manipulated based on a chord strictness threshold and an audio output is provided based on the audio input and the manipulated plurality of harmonizing tracks.

2018/9/20

US

15/986,589

15986589

A computer implemented method for automatically generating musical works including receiving a lyrical input and receiving a musical input. The method includes analyzing the lyrical input to determine at least one lyrical characteristic and analyzing the musical input to determine at least one musical characteristic. Based on the at least one lyrical characteristic, the method includes correlating the lyrical input with the musical input to generate a synthesizer input. The method includes sending the synthesizer input and the at least one voice characteristic to a voice synthesizer. The method may also include receiving, from the voice synthesizer, a vocal rendering of the lyrical input. The method includes receiving a singer selection corresponding to at least one voice characteristic, and generating a musical work from the vocal rendering based on the lyrical input, the musical input, and the at least one voice characteristic.

2015/11/3

US

9177540

14067942

A system and method for conforming an audio input to a musical key. Audio input is received and the musical key is determined. Sequentially for each consecutive note, a pitch value for each note and an interval between each preceding and subsequent note is determined; alternative subsequent notes based on the musical key and the pitch value of the subsequent note are determined; each interval between each alternative subsequent note and each respective note of the alternative subsequent notes corresponding to the preceding note is scored based on the interval between the preceding note and the subsequent note of the audio input; and a best interval is selected for each alternative subsequent note. A best-match note is selected for each note of the audio input based on the best intervals of all the notes, and each audio input note is conformed to a frequency of the best-match note.

2016/4/12

US

9310959

14067934

A system and method for enhancing audio, the method including receiving audio input tracks, with at least one audio input track including a restriction parameter, determining a restricted audio input track, where the restricted audio input track is the audio input track including the restriction parameter, manipulating another audio input track based on musical properties of the restricted audio input track, and combining the restricted audio input track and the manipulated audio input track into a single output audio track.

2018/12/27

US

16/016,260

16016260

A computer implemented method for automatically generating lyric videos comprising receiving an audio selection, determining timing information of the audio selection, and determining lyric information of the audio selection. The method includes receiving tone information of the audio selection and generating video content based on at least one of the timing information, the lyric information, and the tone information of the audio selection. The method also includes rendering a lyric video based on the video content and the audio selection.

2020/1/7

US

10529310

15431521

A method for converting textual messages to musical messages comprising receiving a text input and receiving a musical input selection. The method includes analyzing the text input to determine text characteristics and analyzing a musical input corresponding to the musical input selection to determine musical characteristics. Based on the text characteristic and the musical characteristic, the method includes correlating the text input with the musical input to generate a synthesizer input, and sending the synthesizer input to a voice synthesizer. The method includes receiving a vocal rendering of the text input from the voice synthesizer and generating a musical message from the vocal rendering and the musical input. The method includes generating a video element based on a video input, incorporating the video element into the musical message, and outputting the musical message including the video element.

2020/1/7

US

10529310

15431521

A method for converting textual messages to musical messages comprising receiving a text input and receiving a musical input selection. The method includes analyzing the text input to determine text characteristics and analyzing a musical input corresponding to the musical input selection to determine musical characteristics. Based on the text characteristic and the musical characteristic, the method includes correlating the text input with the musical input to generate a synthesizer input, and sending the synthesizer input to a voice synthesizer. The method includes receiving a vocal rendering of the text input from the voice synthesizer and generating a musical message from the vocal rendering and the musical input. The method includes generating a video element based on a video input, incorporating the video element into the musical message, and outputting the musical message including the video element.