patrick/Vosklet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Recognizer design change

Browse Source
This commit is contained in:
msqr1
2024-01-25 23:30:45 -08:00
parent 48b54295f3
commit efb56e19f9
7 changed files with 67 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@@ -1,27 +1,29 @@
# Browser-recognizer
- A speech recognizer built on Vosk that can be run on the browser, inspired by [vosk-browser](https://github.com/ccoreilly/vosk-browser), but built from scratch and no code taken!
- Browser-recognizer can run both in the browser main thread and web workers
- The API also designed to have strong exception safety
- The API is also designed with strong exception safety
## Global and all objects' common interface
| Function signature (global) | Description |
|---|---|
| ```Promise makeModel(path: string, url: string, id: string)```<br><br>```Promise makeSpkModel(path: string, url: string, id: string)``` | - If **path** contains valid model files and **id** is the same, there will not be a fetch from **url**.<br>- If **path** doesn't contain valid model files, or if it contains valid model files but **id** is different, there will be a fetch from **url**, and the model is stored with **id**. |
| ```Promise<Model> makeModel(path: string, url: string, id: string)```<br><br>```Promise<SpkModel> makeSpkModel(path: string, url: string, id: string)``` | Make a ```Model``` or ```SpkModel```<br>- If **path** contains valid model files and **id** is the same, there will not be a fetch from **url**.<br>- If **path** doesn't contain valid model files, or if it contains valid model files but **id** is different, there will be a fetch from **url**, and the model is stored with **id**. |
| ```Promise<Recognizer> makeRecognizer(model: Model)``` | Make a ```Recognizer```, it will use a separate thread for recognition
| ```setLogLevel(lvl: int)``` | Set Vosk's log level (default: -1) <br>- 2: Error<br>- 1: Warning<br>- 0: Info <br>- 1: Verbose<br>- 2: More verbose<br>- 3: Debug |
| ```deleteAll()``` | Call ```delete()``` on all objects, it is recommended to put this at the end of the program to automatically clean up. See [why](https://emscripten.org/docs/getting_started/FAQ.html#what-does-exiting-the-runtime-mean-why-don-t-atexit-s-run).|
| ```deleteAll()``` | Call ```delete()``` on all objects, it is recommended to run this at the API usage end to automatically clean up everything. See [why](https://emscripten.org/docs/getting_started/FAQ.html#what-does-exiting-the-runtime-mean-why-don-t-atexit-s-run).|
| Function signature (all objects) | Description
|---|---|
| ```delete()``` | Delete (call C++ destructor on) this object
| ```delete()``` | Delete this object
## ```Recognizer``` object
| Function signature | Description |
|---|---|
| ```processAudio(ctx: AudioContext)``` | Recognize an audio chunk
| ```Promise<AudioWorkletNode> getNode(ctx: AudioContext)``` | Get a pass-through node that recognize audio and is connectable to a processing graph |
| ```recognize(buf: AudioBuffer)``` | Recognize an AudioBuffer, usually from something like ```BaseAudioContext.decodeAudioData()```
| ```setPartialWords(partialWords: bool)``` | Return words' information in a partialResult event (default: false) |
| ```setWords(words: bool)``` | Return words' information in a result event (default: false) |
| ```setNLSML(nlsml: bool)``` | Return result and partialResult in NLSML form (default: false) |
| ```setMaxAlternatives(alts: int)``` | Set the max number of alternatives for result event (default: false) |
| ```setGrm(grm: string)``` | Add grammar to the recognizer (default: none) |
| ```setSpkModel(mdl: spkmodel)``` | Set the speaker model of the recognizer (default: none) |
| ```setSpkModel(mdl: SpkModel)``` | Set the speaker model of the recognizer (default: none) |
| Event | Description |
|---|---|
@@ -32,13 +34,14 @@ Changing any setting to non-default values requires recompilation
```
git clone --depth=1 https://github.com/msqr1/Browser-recognizer &&
cd Browser-recognizer &&
[Name]=[Value]... ./compile.sh
[Options] ./compile.sh
```
| Name | Description | Default value |
| Option | Description | Default value |
|---|---|---|
| MAX_MEMORY | Set max memory (suffix mb, gb, or none for bytes) | 300mb, as [recommended](https://alphacephei.com/vosk/models) |
| MAX_THREAD | Set the max number of thread (min: 2) | 2 (1 OPFS thread + 1 recognizer thread) |
| EMSDK | Set EMSDK's path (will install EMSDK in root folder if unset) | **.** |
| MAX_MEMORY | Set max memory, valid suffixes: kb, mb, gb, tb or none (bytes) | ```300mb```, as [recommended](https://alphacephei.com/vosk/models) |
| MAX_THREADS | Set the max number of thread (2 min) | ```2``` (1 OPFS thread + 1 recognizer thread) |
| COMPILE_JOBS | Set the number of jobs (threads) when compiling | ```$(nproc)``` |
| EMSDK | Set EMSDK's path (will install EMSDK in root folder if unset) | ```.``` |
## Response headers
Browser-recognizer require SharedArrayBuffer, so these response headers must be set:
- ***Cross-Origin-Embedder-Policy*** ---> ***require-corp***
@@ -46,7 +49,7 @@ Browser-recognizer require SharedArrayBuffer, so these response headers must be
If you can't set them, you may use a VERY HACKY workaround at *src/addCOI.js*.
## Additions to vosk-browser:
- Can download multiple models
- Download multiple models
- Model storage path management (when many models are required)
- Model ID management (when model updates are required)