Usage in ASR->LLM->TTS stack

[acatovic]

Hi! This is not an issue but first kudos on building this, and second I just have a question :-)

I have put together a local voice assistant (see https://github.com/acatovic/ova) and I use Parakeet as the ASR component (currently supports both CUDA and MLX backends).

My question is - what would be your recommended integration guide in this kind of setup? Since parakeet.cpp is in C++ and my other bits are in Python, should I just dump the ASR text output in temp files and then proceed with the rest of the Python stack. Or should I create a Python binding (which feels kinda weird since we are going Python -> C++ ->Python.

Just wanted to see your thoughts! Thanks!

[Frikallo]

I don't think this library would be best used in your pipeline; parakeet.cpp and axiom were built to be embeddable and portable for compiled applications that need on-device inference (Similar to whisper.cpp and ggml). However, parakeet.cpp now has a high-level C API, and you're more than welcome to create Python bindings from the C FFI. Feel free to PR if you ever do too; otherwise, I'm sure I will get to Python, Swift, and other bindings once the core library is solid. Thanks for your question and interest in the project!

[m13v]

ran into the same integration question. tried three approaches: temp files (added ~180ms latency per turn), python bindings via pybind11 (works but painful to maintain), and unix domain sockets with streaming (settled on this). the socket approach cut our voice-to-response latency from 2.1s to 0.8s on an M2 because we could start LLM inference on partial ASR output before transcription finished. main tradeoff is complexity in the framing protocol, but for anything conversational it's worth it.