CeVIO New singing synthesis demo from CeVIO developer Techno-Speech

[Ulysses]

Reproducing high-quality singing voice

Techno-Speech - the company which developed CeVIO - released a new article to show how their singing synthesis engine (i.e. CeVIO SVSS engine) sounds like after update (I guess the update is from HMM to DNN). There are 2 Japanese demo (by Sato Sasara), 2 English demo (by IA) and 1 Mandarin Chinese demo (the song is Love Transfer by Eason Chan ).

Maybe we can expect a better CeVIO next year!

 

[Retrocat]

Oh wow! A little shaky, but definitely a big difference, especially IA!

 

[Exemplar]

Wow. They finally got the engine noise under control

 

[Twillby]

Ahhhhhhhhhhhhhhhhh

Years of hoping may be paying off? please please please tbh I'm anxious they'll pull a Voctro and leave CeVIO like it is and move onto other programs/projects PLEASE USE THIS RESEARCH TO UPGRADE CEVIO

 

[Mika]

If they actually use this to update CeVIO I'm so gonny buy IA english. I hated the super obvious engine noise so much but without? Damn.

 

[YOYo_MAMA]

Is there a name for the Chinese voicebank? or maybe the voice provider?

 

[lIlI]

Amazing improvement! If I didn't know better, I would have thought the Japanese voicebank was a human. As a native speaker, the English voicebank still has some pronunciation quirks to be ironed out, but it's so much nicer to listen to.

 

[BenHo]

Jesus, that is quite the difference. given that this fixes the one real issue people have had with Cevio, I think this would greatly increase the number of people who use the software when it gets fully implemented.

 

[Kona]

This so so wonderful!! I didn’t listen to the Chinese, but Sasara and IA sound so great! I’ve grown to like IA woth her engine noise (and choppiness) but aahh if I can get my hands on that updated IA, that’s all I need in life really

 

[Ulysses]

Is there a name for the Chinese voicebank? or maybe the voice provider?

Currently there is no name for the Chinese voice but I guess she (at least part of she) is Xiang Ling (香铃) from Sinsy.
I doubt they will reveal the voice provider since a HMM/DNN voice model can be extracted from multiple people rather than only single one.

Anyway I can tell the new Chinese voice is good - not perfect, but should be better than most V3 Chinese voices, not to say the half-finished MUTA.

 

[xuu]

FINALLY. It's been so long waiting for this Chinese voicebank. It sounds gorgeous. Can't wait to use the new engine! Still a bit shaky but with pitch correction can probably sound very realistic. Yes I did come back just for this. Thank you @Ulysses <3

 

[uncreepy]

I follow a person who's obsessed with CeVIO, they have some juicy insight about the news. I am introducing otamakazaha's translated tweets to the English-speaking tinfoil hat brigade.
(Er, warning, my translation might be in a bit sloppy English cause I'm just quickly translating it as I read it and not obsessing over every word like I would a Crypton-related tweet.)

"Theoretically, according to the study method, it's possible to introduce a voice type parameter, singing style parameter, etc, so I'd like to try those kinds of parts after."
This! I'll be happy if there is even a CeVIO falsetto function that's effective.

"Also, it's not just singing, next is even concerning speaking, we will continue researching the technical application potential."
Wha, this is vocal synthesis!? Furthermore, inputting notes!? Techno Speech and Nagoya Institute of Technology, with AI technology ultra high precision vocal synthesis implementation

On Techno Speech's official website, Japanese, English, and Chinese singing can even be heard. The current technology and even the new technology's voice source is the same, but if we assume the new Japanese technology is Sasara-chan's voice, but the quality is quite different, so I wonder if I can even think of it as the same character...

I though it's similar, HOYA's VoiceText singing. I wonder if both are using deep neural network (DNN)?

The people developing CeVIO seem to be developing a new AI vocal synthesis system. They seem to be thinking about making it a product, but it requires a lot of machine power, so they are investigating a method (Will it use a cloud model like Sinsy?)

 

[Blue Of Mind]

I couldn't tell the difference between the new engine and V5. Yamaha better watch their backs! ;)

 

[Infoholic]

Currently there is no name for the Chinese voice but I guess she (at least part of she) is Xiang Ling (香铃) from Sinsy.
I doubt they will reveal the voice provider since a HMM/DNN voice model can be extracted from multiple people rather than only single one.

Anyway I can tell the new Chinese voice is good - not perfect, but should be better than most V3 Chinese voices, not to say the half-finished MUTA.

Not many HMM/DNN voice models use multiple providers, the end result can be quite the pickle.

 

[Exemplar]

Kanru pointed out on Twitter that Techno Speech published a paper about their recent work

 

[uncreepy]

The article Kanru linked to was submitted by Kazuhiro Nakamura , who I noticed commented on that tweet. So, I looked at Kazuhiro's Twitter and found a link to a DTM Station article by Ken Fujimoto (he always writes the juicy articles) released 8 hours ago for "The world's first CD released with a AI vocal synth"! The CD is called "Sing truly" and is coming out on the 28th and features IA and Satou Sasara:


The title is "いつかかならず / Itsuka Kanarazu / Surely some day". Satou sings alone for about 55 seconds before IA starts singing in English (she's really quiet and sings very slowly, keep your ears open to try to hear her words). I want to make it clear that BOTH characters are singing from deep learning, IA is singing about deep learning and Satou is singing about love/confidence.
The CD is being released under DTM Station's music label, DTM Station Creative.

Unfortunately, the article reveals that the technology isn't quite ready and is not ready for sale. ( @xuu cause we talked about how soon is "soon" on Twitter.) But it's at a point where it's good enough for DTM Station to be allowed permission to make a song. They also said it takes a whopping 10 hours to synthesize a 5 minute song and that making a neural network takes a lot of effort, cost, and time. (It seemed like the author, Ken, who also wrote the article I translated before saying CeVIO's deep learning was coming "soon" didn't truly realize how long it took to render a voice until after doing this collab and ended up saying it wasn't really soon, but instead implied in "the near future".)

They also had to correct the pitch of Satou's singing in Melodyne because it didn't have perfect results. (They did NOT have to use Melodyne on IA, but her chorus is 3 of her singing overlapped.)
Because people would wonder how much the vocals changed after being edited in Melodyne, the CD actually is split up so listeners can compare:
Track 1 = finished version
Track 2 = Sasara without Melodyne
Track 3 = karaoke version that removes Satou's singing, but keeps IA's chorus
Track 4 = pure instrumental
Track 5 = Satou's singing only without EQ or reverb
Track 6 = IA's chorus only without any effects

I dunno about you guys, but I think even if it needs Melodyne tweaking, it sounds REALLY good.

 

[xuu]

Yeah, this sounds absolutely gorgeous. It's a shame it's not quite ready for public use but it seems to be a typical techno-speech thing for literally everything to do with CeVIO to take way longer than they say it will. Hopefully it'll be ready within the coming year? I don't quite see how they're going to capture the vtuber market of potential voicers like they said they would if it takes them this long, though.

 

[DefiantKitsune]

If it takes ten hours with current, high end PCs, we aren't getting this for a while. If a good PC takes 10 hours, imagine trying to run it on a fun of the mill PC.
Which is kinda a shame.


EDIT: I see the typo, but I'm not fixing it because it's funny rip