patrick/Vosklet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

switch to makefile instead, reuse threads

Browse Source
This commit is contained in:
msqr1
2024-02-03 22:19:17 -08:00
parent 076e9f65a2
commit 70b7d2c21b
13 changed files with 167 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -1,13 +1,12 @@
# Browser-recognizer # Browser-recognizer
- A speech recognizer built on Vosk that can be run on the browser, inspired by [vosk-browser](https://github.com/ccoreilly/vosk-browser), but built from scratch and no code taken! - A speech recognizer built on Vosk that can be run on the browser, inspired by [vosk-browser](https://github.com/ccoreilly/vosk-browser), but built from scratch and no code taken!
- Browser-recognizer can run both in the browser main thread and web workers
- The API is also designed with strong exception safety - The API is also designed with strong exception safety
## Global and all objects' common interface ## Global and all objects' common interface
| Function signature (global) | Description | | Function signature (global) | Description |
|---|---| |---|---|
| ```Promise<Model> makeModel(path: string, url: string, id: string)```<br><br>```Promise<SpkModel> makeSpkModel(path: string, url: string, id: string)``` | Make a ```Model``` or ```SpkModel```<br>- If **path** contains valid model files and **id** is the same, there will not be a fetch from **url**.<br>- If **path** doesn't contain valid model files, or if it contains valid model files but **id** is different, there will be a fetch from **url**, and the model is stored with **id**. | | ```Promise<Model> makeModel(path: string, url: string, id: string)```<br><br>```Promise<SpkModel> makeSpkModel(path: string, url: string, id: string)``` | Make a ```Model``` or ```SpkModel```<br>- If **path** contains valid model files and **id** is the same, there will not be a fetch from **url**.<br>- If **path** doesn't contain valid model files, or if it contains valid model files but **id** is different, there will be a fetch from **url**, and the model is stored with **id**. |
| ```Promise<Recognizer> makeRecognizer(model: Model, sampleRate: float)``` | Make a ```Recognizer```, it will use **model**'s thread if it's the first to use **model**, else it will use a new thread. | ```Promise<Recognizer> makeRecognizer(model: Model, sampleRate: float)``` | Make a ```Recognizer```, it will use **model**'s thread if it's the first user of **model**, else it will use a new thread.
| ```setLogLevel(lvl: int)``` | Set Vosk's log level (default: -1) <br>- 2: Error<br>- 1: Warning<br>- 0: Info <br>- 1: Verbose<br>- 2: More verbose<br>- 3: Debug | | ```setLogLevel(lvl: int)``` | Set Vosk's log level (default: ```0```: Info) <br>```-2```: Error<br>```-1```: Warning<br>```1```: Verbose<br>```2```: More verbose<br>```3```: Debug |
| ```cleanUp()``` | Call ```delete()``` on all objects and revoke all Blob URLs. | | ```cleanUp()``` | Call ```delete()``` on all objects and revoke all Blob URLs. |
| Function signature (all objects) | Description | Function signature (all objects) | Description
@@ -41,7 +40,7 @@ cd Browser-recognizer &&
| MAX_MEMORY | Set max memory, valid suffixes: kb, mb, gb, tb or none (bytes) | ```300mb```, as [recommended](https://alphacephei.com/vosk/models) | | MAX_MEMORY | Set max memory, valid suffixes: kb, mb, gb, tb or none (bytes) | ```300mb```, as [recommended](https://alphacephei.com/vosk/models) |
| MAX_THREADS | Set the max number of thread (2 min) | ```2``` (1 OPFS thread + 1 model/recognizer thread) | | MAX_THREADS | Set the max number of thread (2 min) | ```2``` (1 OPFS thread + 1 model/recognizer thread) |
| COMPILE_JOBS | Set the number of jobs (threads) when compiling | ```$(nproc)``` | | COMPILE_JOBS | Set the number of jobs (threads) when compiling | ```$(nproc)``` |
| EMSDK | Set EMSDK's path (will install EMSDK in root folder if unset) | ```emsdk``` | | EMSDK | Set EMSDK's path (will install EMSDK in root folder if unset) | ```../emsdk``` |
## Response headers ## Response headers
Browser-recognizer require SharedArrayBuffer, so these response headers must be set: Browser-recognizer require SharedArrayBuffer, so these response headers must be set:
- ***Cross-Origin-Embedder-Policy*** ---> ***require-corp*** - ***Cross-Origin-Embedder-Policy*** ---> ***require-corp***
@@ -57,7 +56,6 @@ If you can't set them, you may use a VERY HACKY workaround at *src/addCOI.js*.
``` ```
<!--Load this from a script tag--> <!--Load this from a script tag-->
<script src="BrowserRecognizer.js"></script> <script src="BrowserRecognizer.js"></script>
<!-->
<script> <script>
// Select name // Select name
const BrRec = await loadBR() const BrRec = await loadBR()
@@ -72,7 +70,7 @@ If you can't set them, you may use a VERY HACKY workaround at *src/addCOI.js*.
console.log("Partial result: ",e.details) console.log("Partial result: ",e.details)
}) })
// Process audio // Microphone setup
media = await navigator.mediaDevices.getUserMedia({ media = await navigator.mediaDevices.getUserMedia({
video: false, video: false,
audio: { audio: {
@@ -81,7 +79,7 @@ If you can't set them, you may use a VERY HACKY workaround at *src/addCOI.js*.
channelCount: 1, channelCount: 1,
sampleRate: 16000 sampleRate: 16000
}, },
}); })
</script> </script>
``` ```

1
emsdk Submodule

Submodule emsdk added at 2aa7490715

166
src/Makefile
View File

@@ -5,97 +5,95 @@
# #### # # ### #### # #### ##### #### # #### # # #### # #### # # ### #### # #### ##### #### # #### # # ####
# 1 hour build time # 1 hour build time
sudo apt install shtool libtool autogen autotools-dev pkg-config make && SHELL=/bin/bash
MAX_MEMORY?=300mb
MAX_THREADS?=2
EMSDK?=../emsdk
COMPILE_JOBS?:=$(nproc)
MAX_MEMORY?=300mb && SRC:=$(realpath src)
MAX_THREADS?=2 && KALDI:=$(realpath kaldi)
EMSDK?=$(realpath emsdk) && VOSK:=$(realpath vosk-api)
COMPILE_JOBS?=$(nproc) && OPENFST:=$(KALDI)/tools/openfst
LIBARCHIVE:=$(realpath libarchive)
ZSTD:=$(realpath zstd)
CLAPACK_WASM:=$(realpath clapack-wasm)
SRC=$(realpath src) && browser-recognizer: vosk libarchive
KALDI=$(realpath kaldi) && cd $(SRC) && \
VOSK=$(realpath vosk-api) && em++ -O3 global.cc genericModel.cc model.cc spkModel.cc recognizer.cc bindings.cc -sWASMFS -sWASM_BIGINT -sSINGLE_FILE -sMODULARIZE -sTRUSTED_TYPES -sEMBIND_STD_STRING_IS_UTF8 -sPTHREAD_POOL_DELAY_LOAD -sTEXTDECODER=2 -sPTHREAD_POOL_SIZE_STRICT=2 -sINITIAL_MEMORY=$(MAX_MEMORY) -sPTHREAD_POOL_SIZE=$(MAX_THREADS) -sPOLYFILL=0 -sSUPPORT_LONGJMP=0 -sEXPORT_NAME=loadBR -sEXPORTED_RUNTIME_METHODS=UTF8ToString,stringToUTF8OnStack -sMALLOC=emmaloc -sEXPORTED_FUNCTIONS=_malloc,_main -sENVIRONMENT=web -I. -I$(LIBARCHIVE)/include -I$(VOSK)/src -L$(LIBARCHIVE)/lib -larchive -L$(ZSTD)/lib -lzstd -L$(KALDI)/src -l:online2/kaldi-online2.a -l:decoder/kaldi-decoder.a -l:ivector/kaldi-ivector.a -l:gmm/kaldi-gmm.a -l:tree/kaldi-tree.a -l:feat/kaldi-feat.a -l:cudamatrix/kaldi-cudamatrix.a -l:lat/kaldi-lat.a -l:lm/kaldi-lm.a -l:rnnlm/kaldi-rnnlm.a -l:hmm/kaldi-hmm.a -l:nnet3/kaldi-nnet3.a -l:transform/kaldi-transform.a -l:matrix/kaldi-matrix.a -l:fstext/kaldi-fstext.a -l:util/kaldi-util.a -l:base/kaldi-base.a -L$(OPENFST)/lib -l:libfst.a -l:libfstngram.a -L$(CLAPACK_WASM) -l:CBLAS/lib/cblas.a -l:CLAPACK-3.2.1/lapack.a -l:CLAPACK-3.2.1/libcblaswr.a -l:f2c_BLAS-3.8.0/blas.a -l:libf2c/libf2c.a -L$(VOSK)/src -l:vosk.a -lopfs.js -lembind -pthread -flto --pre-js pre.js ../BrowserRecognizer.js && \
OPENFST=$KALDI/tools/openfst && cd .. && \
LIBARCHIVE=$(realpath libarchive) && rm -f BrowserRecognizer.worker.js && \
ZSTD=$(realpath zstd) && sed -i 's/locateFile("BrowserRecognizer.worker.js")/pthreadUrl/g' BrowserRecognizer.js && \
CLAPACK_WASM=$(realpath clapack-wasm) && sed -i 's/let root/var root/g' BrowserRecognizer.js
if [ ! -d $EMSDK_PATH ]; then prepare:
echo "Invalid EMSDK path" && sudo apt install shtool libtool autogen autotools-dev pkg-config make && \
exit 1 [ $(EMSDK) != emsdk -a ! -d $(EMSDK) ] && \
fi echo "Invalid emsdk path"; \
if [ $MAX_THREAD -lt 2 ]; then exit 1; \
echo "MAX_THREADS must be greater than or equal to 2" && [ $(MAX_THREAD) -lt 2 ] && \
exit 1 echo "MAX_THREADS must be greater than or equal to 2"; \
fi exit 1; \
if [ $COMPILE_JOBS -lt 1 ]; then [ $(COMPILE_JOBS) -lt 1 ] && \
echo "COMPILE_JOBS must be greater than or equal to 1" && echo "COMPILE_JOBS must be greater than or equal to 1"; \
exit 1 exit 1; \
fi [[ ! $(MAX_MEMORY) =~ "$(^[0-9]+([kmgt]b)?$$)" ]] && \
if ! [[ $MAX_MEMORY =~ ^[0-9]+([kmgt]b)?$ ]]; then echo "MAX_MEMORY valid suffixes are kb, mb, gb, tb, none (bytes)"; \
echo "MAX_MEMORY valid suffixes are kb, mb, gb, tb, none (bytes)" && exit 1; \
exit 1 [ $(EMSDK) = emsdk ] && \
fi echo "Installing emsdk + Emscripten..."; \
if [ $(realpath $EMSDK) == $(realpath emsdk) ]; then git clone --depth=1 https://github.com/emscripten-core/emsdk.git && \
echo "EMSDK is current directory, installing emsdk and Emscripten..." && cd emsdk && \
git clone --depth=1 https://github.com/emscripten-core/emsdk.git && ./emsdk install 3.1.53 && \
cd emsdk && ./emsdk activate 3.1.53; \
./emsdk install 3.1.53 && . $(EMSDK)/emsdk_env.sh && \
./emsdk activate 3.1.53 export PATH=:$$PATH:$(EMSDK)/upstream/bin
fi
. $EMSDK/emsdk_env.sh && zstd: prepare
export PATH=:$PATH:$EMSDK/upstream/bin && rm -rf /tmp/zstd && \
git clone -b v1.5.5 --depth=1 https://github.com/facebook/zstd /tmp/zstd && \
cd /tmp/zstd && \
HAVE_THREAD=0 ZSTD_LEGACY_SUPPORT=0 HAVE_ZLIB=0 HAVE_LZMA=0 HAVE_LZ4=0 ZSTD_NOBENCH=1 ZSTD_NODICT=1 ZSTD_NOCOMPRESS=1 BACKTRACE=0 PREFIX=$(ZSTD) CPPFLAGS="-O3 -flto" LDFLAGS="-O3 -flto" emmake make -j$(COMPILE_JOBS) install && \
rm -rf /tmp/zstd
zstd: libarchive: zstd
rm -rf /tmp/zstd && rm -rf /tmp/libarchive && \
git clone -b v1.5.5 --depth=1 https://github.com/facebook/zstd /tmp/zstd && git clone -b v3.7.2 --depth=1 https://github.com/libarchive/libarchive /tmp/libarchive && \
cd /tmp/zstd && cd /tmp/libarchive && \
HAVE_THREAD=0 ZSTD_LEGACY_SUPPORT=0 HAVE_ZLIB=0 HAVE_LZMA=0 HAVE_LZ4=0 ZSTD_NOBENCH=1 ZSTD_NODICT=1 ZSTD_NOCOMPRESS=1 BACKTRACE=0 PREFIX=$ZSTD CPPFLAGS="-O3 -flto" LDFLAGS="-O3 -flto" emmake make -j$COMPILE_JOBS install && build/autogen.sh && \
rm -rf /tmp/zstd && CPPFLAGS="-I$(ZSTD)/include -flto" LDFLAGS="-L$(ZSTD)/lib -flto" emconfigure ./configure --prefix=$(LIBARCHIVE) --without-lz4 --without-lzma --without-zlib --without-bz2lib --without-xml2 --without-expat --without-cng --without-openssl --without-libb2 --disable-bsdunzip --disable-xattr --disable-acl --disable-bsdcpio --disable-bsdcat --disable-rpath --disable-maintainer-mode --disable-dependency-tracking --enable-static --disable-shared && \
emmake make -j$(COMPILE_JOBS) install && \
rm -rf /tmp/libarchive
libarchive: clapack-wasm: prepare
rm -rf /tmp/libarchive && git clone --depth=1 https://gitlab.inria.fr/multispeech/kaldi.web/clapack-wasm.git $(CLAPACK_WASM) && \
git clone -b v3.7.2 --depth=1 https://github.com/libarchive/libarchive /tmp/libarchive && cd $(CLAPACK_WASM) && \
cd /tmp/libarchive && bash ./install_repo.sh emcc
build/autogen.sh &&
CPPFLAGS="-I$ZSTD/include -flto" LDFLAGS="-L$ZSTD/lib -flto" emconfigure ./configure --prefix=$LIBARCHIVE --without-lz4 --without-lzma --without-zlib --without-bz2lib --without-xml2 --without-expat --without-cng --without-openssl --without-libb2 --disable-bsdunzip --disable-xattr --disable-acl --disable-bsdcpio --disable-bsdcat --disable-rpath --disable-maintainer-mode --disable-dependency-tracking --enable-static --disable-shared &&
emmake make -j$COMPILE_JOBS install &&
rm -rf /tmp/libarchive &&
clapack-wasm: openfst: prepare
git clone --depth=1 https://gitlab.inria.fr/multispeech/kaldi.web/clapack-wasm.git $CLAPACK_WASM && rm -rf /tmp/openfst && \
cd $CLAPACK_WASM && git clone --depth=1 https://github.com/alphacep/openfst /tmp/openfst && \
bash ./install_repo.sh emcc && cd /tmp/openfst && \
autoreconf -i && \
CXXFLAGS="-pthread -r -O3 -flto" LDFLAGS="-O3 -pthread -flto" emconfigure ./configure --prefix=$(OPENFST) --enable-static --disable-shared --enable-ngram-fsts --enable-lookahead-fsts --disable-bin --with-pic && \
emmake make -j$(COMPILE_JOBS) install && \
echo "PACKAGE_VERSION = 1.8.0" >> $(OPENFST)/Makefile && \
rm -rf /tmp/openfst
openfst: kaldi: openfst clapack-wasm
rm -rf /tmp/openfst && git clone -b vosk --depth=1 https://github.com/alphacep/kaldi $(KALDI) && \
git clone --depth=1 https://github.com/alphacep/openfst /tmp/openfst && cd $(KALDI)/src && \
cd /tmp/openfst && git apply $(SRC)/kaldi.patch && \
autoreconf -i && CXXFLAGS="-O3 -msimd128 -UHAVE_EXECINFO_H -pthread -flto" LDFLAGS="-O3 -sERROR_ON_UNDEFINED_SYMBOLS=0 -lembind -pthread -flto" emconfigure ./configure --use-cuda=no --with-cudadecoder=no --static --static-math=yes --static-fst=yes --debug-level=0 --double-precision=yes --clapack-root=$(CLAPACK_WASM) --host=WASM && \
CXXFLAGS="-pthread -r -O3 -flto" LDFLAGS="-O3 -pthread -flto" emconfigure ./configure --prefix=$OPENFST --enable-static --disable-shared --enable-ngram-fsts --enable-lookahead-fsts --disable-bin --with-pic && emmake make -j$(COMPILE_JOBS) online2 lm rnnlm
emmake make -j$COMPILE_JOBS install &&
echo "PACKAGE_VERSION = 1.8.0" >> $OPENFST/Makefile &&
rm -rf /tmp/openfst &&
kaldi: vosk: kaldi
git clone -b vosk --depth=1 https://github.com/alphacep/kaldi $KALDI && git clone -b go/v0.3.46 --depth=1 https://github.com/alphacep/vosk-api $(VOSK) && \
cd $KALDI/src && cd $(VOSK)/src && \
git apply $SRC/kaldi.patch && git apply $(SRC)/vosk.patch && \
CXXFLAGS="-O3 -msimd128 -UHAVE_EXECINFO_H -pthread -flto" LDFLAGS="-O3 -sERROR_ON_UNDEFINED_SYMBOLS=0 -lembind -pthread -flto" emconfigure ./configure --use-cuda=no --with-cudadecoder=no --static --static-math=yes --static-fst=yes --debug-level=0 --double-precision=yes --clapack-root=$CLAPACK_WASM --host=WASM && VOSK_FILES="recognizer.cc language_model.cc model.cc spk_model.cc vosk_api.cc" && \
emmake make -j$COMPILE_JOBS online2 lm rnnlm em++ -pthread -O3 -flto -Wno-deprecated -I. -I$(KALDI)/src -I$(OPENFST)/include $(VOSK_FILES) -c && \
emar -rcs vosk.a $(VOSK_FILES:.cc=.o)
vosk: .PHONY: prepare
git clone -b go/v0.3.46 --depth=1 https://github.com/alphacep/vosk-api $VOSK &&
cd $VOSK/src &&
git apply $SRC/vosk.patch &&
VOSK_FILES="recognizer.cc language_model.cc model.cc spk_model.cc vosk_api.cc" &&
em++ -pthread -O3 -flto -Wno-deprecated -I. -I$KALDI/src -I$OPENFST/include $VOSK_FILES -c &&
emar -rcs vosk.a ${VOSK_FILES//.cc/.o}
browser-recognizer:
cd $SRC &&
em++ -O3 global.cc genericModel.cc model.cc spkModel.cc recognizer.cc bindings.cc -sWASMFS -sWASM_BIGINT -sSINGLE_FILE -sMODULARIZE -sTRUSTED_TYPES -sEMBIND_STD_STRING_IS_UTF8 -sPTHREAD_POOL_DELAY_LOAD -sALLOW_BLOCKING_ON_MAIN_THREAD -sINITIAL_MEMORY=$MAX_MEMORY -sPTHREAD_POOL_SIZE=$MAX_THREADS -sPOLYFILL=0 -sPTHREAD_POOL_SIZE_STRICT=0 -sSUPPORT_LONGJMP=0 -sEXPORTED_RUNTIME_METHODS=UTF8ToString,stringToUTF8OnStack -sEXPORTED_FUNCTIONS=_malloc,_main -sEXPORT_NAME=loadBR -sENVIRONMENT=web,worker -I. -I$LIBARCHIVE/include -I$VOSK/src -L$LIBARCHIVE/lib -larchive -L$ZSTD/lib -lzstd -L$KALDI/src -l:online2/kaldi-online2.a -l:decoder/kaldi-decoder.a -l:ivector/kaldi-ivector.a -l:gmm/kaldi-gmm.a -l:tree/kaldi-tree.a -l:feat/kaldi-feat.a -l:cudamatrix/kaldi-cudamatrix.a -l:lat/kaldi-lat.a -l:lm/kaldi-lm.a -l:rnnlm/kaldi-rnnlm.a -l:hmm/kaldi-hmm.a -l:nnet3/kaldi-nnet3.a -l:transform/kaldi-transform.a -l:matrix/kaldi-matrix.a -l:fstext/kaldi-fstext.a -l:util/kaldi-util.a -l:base/kaldi-base.a -L$OPENFST/lib -l:libfst.a -l:libfstngram.a -L$CLAPACK_WASM -l:CBLAS/lib/cblas.a -l:CLAPACK-3.2.1/lapack.a -l:CLAPACK-3.2.1/libcblaswr.a -l:f2c_BLAS-3.8.0/blas.a -l:libf2c/libf2c.a -L$VOSK/src -l:vosk.a -lopfs.js -lembind -pthread -flto --pre-js pre.js ../BrowserRecognizer.js &&
cd .. &&
rm -f BrowserRecognizer.worker.js &&
sed -i 's/locateFile("BrowserRecognizer.worker.js")/pthreadUrl/g' BrowserRecognizer.js

18
src/genericModel.cc
View File

@@ -1,4 +1,5 @@
#include "genericModel.h" #include "genericModel.h"
genericModel::genericModel(const std::string& storepath, const std::string &id, int index) : storepath(storepath), id(id), index(index) { genericModel::genericModel(const std::string& storepath, const std::string &id, int index) : storepath(storepath), id(id), index(index) {
fs::current_path("/opfs", tank); fs::current_path("/opfs", tank);
if(tank.value() != 0) { if(tank.value() != 0) {
@@ -25,16 +26,12 @@ bool genericModel::checkModel() {
file.read(&oldid[0], size); file.read(&oldid[0], size);
return id.compare(oldid) == 0 ? true : false; return id.compare(oldid) == 0 ? true : false;
} }
void genericModel::afterFetch(int memAddr, size_t size) { void genericModel::afterFetch() {
// FIXME: Recognizer reuse this thread if possible thrd.setTask1([this](){
std::thread t{[this, memAddr, size](){ if(!extractModel()) {
char* modelData = reinterpret_cast<char*>(memAddr);
if(!extractModel(modelData, size)) {
free(modelData);
fireEv("_continue", "Unable to extract model", index); fireEv("_continue", "Unable to extract model", index);
return; return;
} }
free(modelData);
std::ofstream idFile("id"); std::ofstream idFile("id");
if(!idFile.is_open()) { if(!idFile.is_open()) {
fs::current_path("/opfs"); fs::current_path("/opfs");
@@ -45,16 +42,15 @@ void genericModel::afterFetch(int memAddr, size_t size) {
idFile << id; idFile << id;
idFile.close(); idFile.close();
load(false); load(false);
}}; });
t.detach();
} }
bool genericModel::extractModel(const char* fileBuf, size_t size) { bool genericModel::extractModel() {
std::string path{}; std::string path{};
archive* src {archive_read_new()}; archive* src {archive_read_new()};
archive_entry* entry {}; archive_entry* entry {};
archive_read_support_filter_all(src); archive_read_support_filter_all(src);
archive_read_support_format_all(src); archive_read_support_format_all(src);
archive_read_open_memory(src, fileBuf, size); archive_read_open_filename(src, "m0dEl.tzst", 10240);
if(archive_errno(src) != 0) return false; if(archive_errno(src) != 0) return false;
while (archive_read_next_header(src, &entry) == ARCHIVE_OK) { while (archive_read_next_header(src, &entry) == ARCHIVE_OK) {
path = archive_entry_pathname(entry); path = archive_entry_pathname(entry);

5
src/genericModel.h
View File

@@ -14,12 +14,13 @@ namespace fs = std::filesystem;
struct genericModel { struct genericModel {
const std::string storepath{}; const std::string storepath{};
const std::string id{}; const std::string id{};
twiceThrd thrd{};
int index{}; int index{};
static bool extractModel(const char* fileBuf, size_t size); static bool extractModel();
virtual bool checkModelFiles() = 0; virtual bool checkModelFiles() = 0;
virtual void load(bool) = 0; virtual void load(bool) = 0;
bool checkModel(); bool checkModel();
void afterFetch(int memAddr, size_t size); void afterFetch();
genericModel(const std::string &storepath, const std::string &id, int index); genericModel(const std::string &storepath, const std::string &id, int index);
}; };

17
src/global.cc
View File

@@ -1,5 +1,4 @@
#include "global.h" #include "global.h"
void throwJS(const char* msg, bool err) { void throwJS(const char* msg, bool err) {
EM_ASM({ EM_ASM({
if($1) { if($1) {
@@ -22,12 +21,24 @@ void fireEv(const char *type, const char *content, int index) {
} }
pq.proxySync(selfTID, proxy); pq.proxySync(selfTID, proxy);
} }
int main() { int main() {
//vosk_set_log_level(-1);
std::thread t{[](){ std::thread t{[](){
wasmfs_create_directory("/opfs", 0777, wasmfs_create_opfs_backend()); wasmfs_create_directory("/opfs", 0777, wasmfs_create_opfs_backend());
}}; }};
t.detach(); t.detach();
emscripten_exit_with_live_runtime(); emscripten_exit_with_live_runtime();
} }
void twiceThrd::setTask1(std::function<void()> task1) {
blocker.lock();
std::thread t{[this, task1](){
task1();
blocker.lock();
task2();
}};
t.detach();
}
void twiceThrd::setTask2(std::function<void()> task2) {
this->task2 = task2;
blocker.unlock();
reusable = false;
}

8
src/global.h
View File

@@ -13,3 +13,11 @@ static std::error_code tank{};
void throwJS(const char* msg, bool err = false); void throwJS(const char* msg, bool err = false);
void fireEv(const char *type, const char *content, int index); void fireEv(const char *type, const char *content, int index);
int main(); int main();
struct twiceThrd { // A minimal std::thread wrapper to run exactly 2 tasks
bool reusable{true};
std::mutex blocker{};
std::function<void()> task2{};
void setTask1(std::function<void()> task1);
void setTask2(std::function<void()> task2);
};

8
src/model.cc
View File

@@ -4,8 +4,8 @@ model::model(const std::string& storepath, const std::string& id, int index) : g
model::~model() { model::~model() {
vosk_model_free(mdl); vosk_model_free(mdl);
} }
void model::afterFetch(int addr, size_t size) { void model::afterFetch() {
genericModel::afterFetch(addr,size); genericModel::afterFetch();
} }
bool model::checkModel() { bool model::checkModel() {
return genericModel::checkModel(); return genericModel::checkModel();
@@ -23,9 +23,7 @@ void model::load(bool newThrd) {
main(); main();
return; return;
} }
// FIXME: Recognizer reuse this thread if possible thrd.setTask1(main);
std::thread t{main};
t.detach();
} }
bool model::checkModelFiles() { bool model::checkModelFiles() {
return fs::exists("am/final.mdl", tank) && return fs::exists("am/final.mdl", tank) &&

2
src/model.h
View File

@@ -6,7 +6,7 @@ struct model : genericModel {
VoskModel* mdl{}; VoskModel* mdl{};
model(const std::string& storepath, const std::string& id, int index); model(const std::string& storepath, const std::string& id, int index);
bool checkModel(); bool checkModel();
void afterFetch(int addr, size_t size); void afterFetch();
void load(bool newThrd); void load(bool newThrd);
~model(); ~model();
}; };

56
src/pre.js
View File

@@ -1,25 +1,9 @@
let objs = [] let objs = []
let processorUrl = URL.createObjectURL(new Blob([ Module.cleanUp = () => {
(() => { objs.forEach(obj => obj.delete())
registerProcessor("BRProcessor", class extends AudioWorkletProcessor { URL.revokeObjectURL(pthreadUrl)
constructor(options) { URL.revokeObjectURL(processorUrl)
super(options)
this.done = false
this.port.onmessage = (ev) => {
if(ev.cmd === "deinit") this.done = false
} }
this.ptr = options.processorOptions.ptr
}
process(inputs, outputs, params) {
if(this.done) return false;
this.wasmMem.set(inputs[0].getChannelData(this.channel));
this.recognizerPort.postMessage(".")
outputs = inputs
return true
}
})
}).toString()
], {type : "text/javascript"}))
class Recognizer extends EventTarget { class Recognizer extends EventTarget {
constructor() { constructor() {
super() super()
@@ -114,10 +98,8 @@ Module.makeModel = async (url, storepath, id) => {
if(!res.ok) { if(!res.ok) {
return reject("Unable to download model") return reject("Unable to download model")
} }
let arr = await res.arrayBuffer() await (await (await root.getFileHandle("m0dEl.tzst", {create : true})).createWritable()).write(await res.arrayBuffer())
let mdlMem = Module._malloc(arr.byteLength) // Will free in C++ mdl.obj.afterFetch()
Module.HEAP8.set(new Int8Array(arr), mdlMem)
mdl.obj.afterFetch(mdlMem, arr.byteLength)
})() })()
}) })
} }
@@ -167,6 +149,27 @@ Module.makeRecognizer = (model, sampleRate) => {
rec._init(model.obj, sampleRate) rec._init(model.obj, sampleRate)
return retval return retval
} }
let processorUrl = URL.createObjectURL(new Blob([
(() => {
registerProcessor("BRProcessor", class extends AudioWorkletProcessor {
constructor(options) {
super(options)
this.done = false
this.port.onmessage = (ev) => {
if(ev.cmd === "deinit") this.done = false
}
this.ptr = options.processorOptions.ptr
}
process(inputs, outputs, params) {
if(this.done) return false;
this.wasmMem.set(inputs[0].getChannelData(this.channel));
this.recognizerPort.postMessage(".")
outputs = inputs
return true
}
})
}).toString()
], {type : "text/javascript"}))
// Taken from the worker.js file // Taken from the worker.js file
let pthreadUrl = URL.createObjectURL(new Blob([ let pthreadUrl = URL.createObjectURL(new Blob([
(() => { (() => {
@@ -340,8 +343,3 @@ let pthreadUrl = URL.createObjectURL(new Blob([
self.onmessage = handleMessage; self.onmessage = handleMessage;
}).toString() }).toString()
], {type : "text/javascript"})) ], {type : "text/javascript"}))
Module.cleanUp = () => {
objs.forEach(obj => obj.delete())
URL.revokeObjectURL(pthreadUrl)
URL.revokeObjectURL(processorUrl)
}

11
src/recognizer.cc
View File

@@ -1,7 +1,7 @@
#include "recognizer.h" #include "recognizer.h"
recognizer::recognizer(model* mdl, float sampleRate, int index) : index(index) { recognizer::recognizer(model* mdl, float sampleRate, int index) : index(index) {
std::thread t{[this](VoskModel* mdl, VoskRecognizer* rec, float sampleRate){ auto main{[this, mdl, sampleRate](){
rec = vosk_recognizer_new(mdl,sampleRate); rec = vosk_recognizer_new(mdl->mdl,sampleRate);
if(rec == nullptr) { if(rec == nullptr) {
fireEv("_continue", "Unable to initialize recognizer", this->index); fireEv("_continue", "Unable to initialize recognizer", this->index);
return; return;
@@ -20,7 +20,12 @@ recognizer::recognizer(model* mdl, float sampleRate, int index) : index(index) {
} }
} }
} }
},mdl->mdl, rec, sampleRate}; }};
if(mdl->thrd.reusable) {
mdl->thrd.setTask2(main);
return;
}
std::thread t{main};
t.detach(); t.detach();
} }
recognizer::~recognizer() { recognizer::~recognizer() {

8
src/spkModel.cc
View File

@@ -11,8 +11,8 @@ spkModel::~spkModel() {
bool spkModel::checkModel() { bool spkModel::checkModel() {
return genericModel::checkModel(); return genericModel::checkModel();
} }
void spkModel::afterFetch(int addr, size_t size) { void spkModel::afterFetch() {
genericModel::afterFetch(addr,size); genericModel::afterFetch();
} }
void spkModel::load(bool newThrd) { void spkModel::load(bool newThrd) {
auto main{[this](){ auto main{[this](){
@@ -27,9 +27,7 @@ void spkModel::load(bool newThrd) {
main(); main();
return; return;
} }
// FIXME: Recognizer reuse this thread if possible thrd.setTask1(main);
std::thread t{main};
t.detach();
} }
bool spkModel::checkModelFiles() { bool spkModel::checkModelFiles() {
return fs::exists("mfcc.conf", tank) && return fs::exists("mfcc.conf", tank) &&

2
src/spkModel.h
View File

@@ -6,7 +6,7 @@ struct spkModel : genericModel {
VoskSpkModel* mdl{}; VoskSpkModel* mdl{};
spkModel(const std::string& storepath, const std::string& id, int index); spkModel(const std::string& storepath, const std::string& id, int index);
bool checkModel(); bool checkModel();
void afterFetch(int addr, size_t size); void afterFetch();
void load(bool newThrd); void load(bool newThrd);
~spkModel(); ~spkModel();
}; };