请问HN:你们使用哪个推理服务器来托管文本转语音(TTS)模型?

1作者: samagra1410 天前原帖
我手头的所有示例都非常不优化——例如,Modal Labs 使用 FastAPI - [Modal Labs 示例](https://modal.com/docs/examples/chatterbox_tts)。BentoML 也使用类似 FastAPI 的服务 - [BentoML 部署文本转语音应用](https://www.bentoml.com/blog/deploying-a-text-to-speech-application-with-bentoml)。甚至 Chatterbox TTS 也有一个非常简单的示例 - [Chatterbox TTS 示例](https://github.com/resemble-ai/chatterbox)。 Tritonserver 的文档中没有文本转语音的示例。我百分之百确定,可以使用 TritonServer 编写一个高度优化的变体,利用模型的并发和批处理功能。 如果有人使用 Tritonserver 实现了文本转语音服务,或者有更好的推理服务器替代方案可以部署,请帮帮我。我不想重新发明轮子。
查看原文
All the examples I have are highly unoptimized - For eg, Modal Labs uses FastAPI - https:&#x2F;&#x2F;modal.com&#x2F;docs&#x2F;examples&#x2F;chatterbox_tts\ BentoML also uses FastAPI like service - https:&#x2F;&#x2F;www.bentoml.com&#x2F;blog&#x2F;deploying-a-text-to-speech-application-with-bentoml\<p>Even Chatterbox TTS has a very naive example - https:&#x2F;&#x2F;github.com&#x2F;resemble-ai&#x2F;chatterbox\<p>Tritonserver docs don’t have a TTS example.<p>I am 100% certain that a highly optimized variant can be written with TritonServer, utilizing model concurrency and batching.<p>If someone has implemented a TTS service with Tritonserver or has a better inference server alternative to deploy, please help me out here. I don’t want to reinvent the wheel.