Native integration of vocal synthesis in LMMS

SeleDreams

Aspiring Fan
Jul 31, 2019
86
19
Hi, I'm posting this because I have a project of integrating natively in LMMS a vocal synthesis tool and I felt like it would be interesting to get some feedback and maybe some other devs interested in the project.

at the moment there are two main way to integrate it that were imagined, the first is to integrate it in an UTAU/VOCALOID kind of manner so by directly modifying lyrics in the midi editor, the second is to do it the alter/ego way

one of the issues I thought about in the alter/ego way to do things is the issue of synchronization to the song as the way it's done makes it hard for the engine to know what note corresponds to what lyric.

but one of the other issues with the vocaloid/utau way to do things is that it will require some modification of core lmms UI elements, and LMMS is planning a complete UI rewrite so it will require to rewrite the UI changes after the complete overhaul.

what do you think of this ? and if you have ideas about how you'd like the workflow to be, some sketches showing how it would look like would be very helpful !

the vocal synth would be new but would base its engine on existing open source technology, the first idea is to base the engine on WORLD as it is freely available under a GPL compatible licence, but more open source engines could get added in the future.
 
  • Like
Reactions: Kazumimi and kozet

kozet

Conlanger
Jul 5, 2019
53
kyarei.gitlab.io
Seems like an interesting idea. I wouldn't hardcode the vocal synth functionality into LMMS's code but rather create a plugin. Perhaps you could propose the ability for LMMS's own plugin format to read lyrics to the devs. I wouldn't count on such a proposal going through to the mainstream plugin formats such as VST or LV2, but it seems more likely to be considered for one that's used by only one program.
 

SeleDreams

Aspiring Fan
Jul 31, 2019
86
19
Seems like an interesting idea. I wouldn't hardcode the vocal synth functionality into LMMS's code but rather create a plugin. Perhaps you could propose the ability for LMMS's own plugin format to read lyrics to the devs. I wouldn't count on such a proposal going through to the mainstream plugin formats such as VST or LV2, but it seems more likely to be considered for one that's used by only one program.
Yeah I planned for the audio part to use the LMMS plugin format,the main thing that would get hard coded would mostly be having a midi editor that would inherit from the normal one but override some code to add some lyrics editing abilities to it on top and the audio plugins would be able to receive lyrics and stuff like this from the midi
by using overrides through inheritance it would allow to add functionality without modifying the core itself
 

Users Who Are Viewing This Thread (Users: 0, Guests: 0)