umjammer/vavi-speech

🗣 Java Text to Speech (JSAPI) engines (google cloud, cocoa, aquestalk(ゆっくり))

vavi-speech

Text to Speech (JSAPI) engines for Java and Japanese phonemeizers

Staus

Type	Description	Staus	SPI Status	Quality	Comment
AquesTalk	JNI	✅	✅	😐	32bit only
AquesTalk	JNA	✅	✅	😐	32bit only
AquesTalk2	JNI	-	-		mac: intel only
AquesTalk10	JNA	✅	✅	😐	mac: intel only
Google Cloud Text To Speech	Google Cloud Text To Speech	✅	✅	👑
Rococoa	Rococoa, JNA	✅	✅	😃
FreeTTS	FreeTTS	-	✅	💩	not included (test only)

Kanji to Kana converter

Type	Description	Staus	Quality	Comment
Kromoji		✅	😐	number, alphabet
Sen		✅	😐	number, alphabet
Sudachi		✅	😐	助詞(は,へ), number, alphabet
YahooJapan Furigana	Web	✅	😐	助詞(は,へ)
YahooJapan MA	Web	-
Goo Furigana	Web	✅	😐	助詞(は,へ)
Rococoa		✅	😃	数字+助数詞

selected by aquestalk10.properties

Yakuwarigo Converter

name	type	status	qyality	comment
simple	SimpleConverter	✅	💩
ojosama	YakuearigoConverter	✅	😐
zundamon	YakuearigoConverter	🚧

Install

meven

• https://jitpack.io/#umjammer/vavi-speech

AquesTalk10

• place AquesTalk10.framework into ~/Library/Frameworks
• create symbolic link AquesTalk10.framework/AquesTalk as AquesTalk10.framework/AquesTalk10
• write aquesTalk10DevKey into local.properties

Google Cloud Text To Speech

• get token as json
• set environment variable "GOOGLE_APPLICATION_CREDENTIALS" your_json_path

Sen

$ git clone https://gitlab.com/umjammer/sen.git
$ mvn install

• set $SEN/src/main/home as sen.home in local.properties
• edit aquestalk10.properties like below

$ cat src/test/resources/aquestalk10.properties
phonemer=vavi.speech.phoneme.SenJaPhonemer

Sudachi

$ cd vavi-speech
$ curl -o sudachi-dictionary-20210802-full.zip http://sudachi.s3-website-ap-northeast-1.amazonaws.com/sudachidict/sudachi-dictionary-20210802-full.zip
$ tar zxvf sudachi-dictionary-20210802-full.zip src/test/resources/sudachi/

• edit aquestalk10.properties like below

$ cat src/test/resources/aquestalk10.properties
phonemer=vavi.speech.phoneme.SudachiJaPhonemer

goo to hiragana

• register by github account
  • https://labs.goo.ne.jp/jp/apiregister/
• set key to 'local.properties'
• edit aquestalk10.properties like below

$ cat src/test/resources/aquestalk10.properties
phonemer=vavi.speech.phoneme.GooFuriganaJaPhonemer

yahoo! japan

• https://e.developer.yahoo.co.jp/dashboard/
• set client id to 'local.properties'
• edit aquestalk10.properties like below

$ cat src/test/resources/aquestalk10.properties
phonemer=vavi.speech.phoneme.YahooJapanJaPhonemer

Kuromoji

• default
• edit aquestalk10.properties like below

$ cat src/test/resources/aquestalk10.properties
phonemer=vavi.speech.phoneme.KuromojiJaPhonemer

a3rt

• https://a3rt.recruit.co.jp/product/proofreadingAPI/registered/
• set client id by email into 'local.properties'

References

• https://www.a-quest.com/archive/manual/siyo_onseikigou.pdf
• https://github.com/Luigi-Pizzolito/English2KanaTransliteration

TODO

• implement parameters
• implement voices
• implement audio queues (using org.jvoicexml in FreeTTS, see freetts package also)
• speech.properties
• engine
  • watson
  • https://github.com/festvox/festival
  • voicevox -> vavi-speech2
• jsapi source -> independent project
• auto load aques framework (jna)
• https://www.google.co.jp/ime/cgiapi.html
• text analytics + nicotalk character emotion -> vavi-speech2 (wip)
• tacotron2
• yakuwarigo modifier
  • お嬢様言葉 done
    • remaining some unit tests
    • extract rules as json (done)
  • ずんだもん (wip)
• unify japanese tokenizer (lucene-gosen?)
• AVSpeechSynthesizer needs obj-c block
• access private
  • reflection -> JEP 193: Variable Handles https://qiita.com/yoshioterada/items/319ed0dec4b847d2b1ed <= 17
  • https://github.com/FabricMC/access-widener

Morphological Analyzer

• https://mocobeta.github.io/janome/
• https://developer.yahoo.co.jp/webapi/jlp/ma/v1/parse.html
• https://github.com/neubig/kytea
• https://github.com/google/sentencepiece
  • https://github.com/levyfan/sentencepiece-jni
  • https://github.com/keigohtr/apitore-sdk-java
• https://github.com/oyahiroki/nlp4j
• https://github.com/lucene-gosen/lucene-gosen
• https://github.com/ikawaha/kagome

NLP

• https://emorynlp.github.io/nlp4j/
• https://opennlp.apache.org/

Alphabet -> Kana dictionary

• https://fastapi.metacpan.org/source/MASH/Lingua-JA-Yomi-0.01/lib/Lingua/JA
• https://github.com/KEINOS/google-ime-user-dictionary-ja-en

Kanji -> Kana

• chawan (faded away from the internet)
• open jtalk njd* -> Gyutan
• http://kakasi.namazu.org/index.html.ja
• https://github.com/nicolas-raoul/kakasi-java (predecessor of jakaroma)
• https://github.com/nicolas-raoul/jakaroma (engine is kuromoji)
• mac cocoa (done)
• unidic

Romaji -> Kana

• https://icu.unicode.org/home
• https://github.com/andree-surya/moji4j

Proofreading

• https://developer.yahoo.co.jp/webapi/jlp/kousei/v2/kousei.html

vavi-speech's Issues

[yakuwarigo-modifier] update zundamon rules

yakuwarigo (役割語) modifier for ずんだもん is not completed.
it's just a copy of サロメ's rules and replaced "ですわ" into "なのだ" only.
so output text is not natural.

https://github.com/umjammer/vavi-speech/blob/master/src/main/resources/vavi/speech/modifier/yakuwarigo/zundamon.json

[maven] why do we need to specify a dependency's repository setting???

this is the significance of maven's existence. what happened???

46e73e8 ... this is the vavi-sound-nda (a vavi-sound's dependency)'s repository.

maybe a java bug

old code (after the patch below)

diff --git a/src/main/java/vavi/speech/phonemizer/SudachiJaPhonemizer.java b/src/main/java/vavi/speech/phonemizer/SudachiJaPhonemizer.java
index c0045e2..8314106 100644
--- a/src/main/java/vavi/speech/phonemizer/SudachiJaPhonemizer.java
+++ b/src/main/java/vavi/speech/phonemizer/SudachiJaPhonemizer.java
@@ -38,12 +38,14 @@ public class SudachiJaPhonemizer implements JaPhonemizer {
 
     public SudachiJaPhonemizer() {
         try {
-            Config config = Config.fromClasspath("sudachi.json");
-            Dictionary dict = new DictionaryFactory().create(config);
+        Config config = Config.fromClasspath("sudachi.json");
+        try (
+            Dictionary dict = new DictionaryFactory().create(config)) {
             tokenizer = dict.create();
-            Runtime.getRuntime().addShutdownHook(new Thread(() -> {
-                try { dict.close(); } catch (IOException e) { throw new UncheckedIOException(e); }
-            }));
+//            Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+//                try { dict.close(); } catch (IOException e) { throw new UncheckedIOException(e); }
+//            }));
+            }
         } catch (IOException e) {
             throw new UncheckedIOException(e);
         }

causes crash around java.nio.DirectIntBufferU#get(int).

hs_err_pid92309.log

after the patch above, this program will have a bug that closing dict before using that.

but on catalina the program after the patch works, it contains a bug though.

$ java -version                                                                                                                                                                                                master
java version "1.8.0_301"
Java(TM) SE Runtime Environment (build 1.8.0_301-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode)
nsano@nsanomac3 02:22:08 ~/src/java/vavi-speech2 
$ uname -a                                                                                                                                                                                                     master
Darwin nsanomac3 19.6.0 Darwin Kernel Version 19.6.0: Tue Jun 21 21:18:39 PDT 2022; root:xnu-6153.141.66~1/RELEASE_X86_64 x86_64 i386 Darwin

on the other hand, on monterey the program after patch cause crash. java should throw an exception.

$ java -version                                                                                                                                                                                                   0.1.7
java version "1.8.0_341"
Java(TM) SE Runtime Environment (build 1.8.0_341-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.341-b10, mixed mode)
$ uname -a                                                                                                                                                                                                        0.1.7
Darwin nsanomac4 21.6.0 Darwin Kernel Version 21.6.0: Thu Sep 29 20:12:57 PDT 2022; root:xnu-8020.240.7~1/RELEASE_X86_64 x86_64

i couldn't create a small sample bug reproducer. because there is no way to create a java.nio.DirectIntBufferU instance.