#tts | Anand S - Things I Learned

Tue, Mar 25, 2025. The new GPT-4o mini Transcribe model is a bit better than Whisper and costs half: ~18 cents per hour. It includes background noise cancellation and semantic chunking, which is useful. #speech-to-text #tts #voice-cloning

Tue, Mar 25, 2025. The new GPT-4o mini TTS is about 3-4 times cheaper than TTS-1 since it's ~$12/MTok instead of $15/Mchar. It supports emotions with streaming. #speech-to-text #tts #voice-cloning

Sun, Mar 23, 2025. Phi-4 multimodal procehttps://huggingface.co/microsoft/Phi-4-multimodal-instructsses speech better than Whisper V3 on HuggingFace OpenASR, and images better than Gemini Flash Lite #future #huggingface #image-generation #speech-to-text #tts #voice-cloning

Fri, Mar 14, 2025. Twine (docs) is an open source interactive fiction / story writing tool. #ai-coding-tools #github #markdown #tts
- Snowman is a browser-based Twine 2 story template format.
- These enable behavioural experimentation. Cheaper than using tools like Gorilla.sc and Pavlovia for behavioral experiments
- For example, you can present a social or political issue and see if people change their opinions more or less depending on the content/path they see. Or, if it varies by demographics. Or, check if repeated mentions or emotional hooks improve memory / retention. More research ideas

Wed, Mar 5, 2025. Reliably creating interactive tutorials is hard today. Claude 3.7 Sonnet ran out of tokens when I tried creating an interactive tutorial on diffraction. Cursor got the tokens but failed to get the application right after 3 attempts. This is not yet reliable, and when it does become reliable, education will change a fair bit. #IMPOSSIBLE #tts

Sun, Mar 2, 2025. YayText converts text to Unicode that has strikethrough, bold, italics, alternate fonts, and other interesting features. So does #ai-coding-tools #speech-to-text #tts Unitextify, ConvertCase, and LingoJam.

Sun, Jan 12, 2025. TTS Arena is a benchmark of text-to-speech models. Kokoro-TTS is the current leader. It's just 82M, runs on Google Colab, and sounds slightly better than OpenAI TTS. #speech-to-text #tts #voice-cloning

Wed, Jan 8, 2025. whisper-flow does real-time speech transcription! #future #speech-to-text #tts #voice-cloning

Fri, Jan 3, 2025. Assembly AI offers speech to text with diarization at 12c/hour. Good diarization, average transcription quality. #speech-to-text #tts #voice-cloning In comparison, WhisperX (with GPU) was much slower, had slightly poorer diarization, and slightly better transcription.
```
uvx --python 3.9 --index https://download.pytorch.org/whl/cu121 whisperx --diarize --lang en --hf_token $HUGGINGFACE_TOKEN`
```

Wed, Dec 18, 2024. GPT 4o Mini Realtime was released. A realtime conversation will cost ~50c/hr. About 36c for input, 72c for output. (I extrapolated from the 6c/min audio input cost for GPT 4o Realtime when it was $100/MTok. GPT 4o Mini Realtime is $10/MTok input and $20/MTok output.) #chatgpt #pricing #speech-to-text #tts

Fri, Nov 29, 2024. GPT-4o Audio supports tone control via XML tags like <cough>..., <laugh>..., etc. But at ~$15/hr of output, it's too expensive. Ref #speech-to-text #tts #voice-cloning

Mon, Nov 11, 2024. Gemini transcription does not give accurate timestamps. Whisper does. But the quality of transcription is similar. #speech-to-text #tts

Mon, Nov 4, 2024. Artificial Analysis has a bunch of new leaderboards and arenas. #speech-to-text #tts
- Open AI TTS leads the TTS Leaderboard. ElevenLabs is a bit behind.
- Recraft V3 > Flux 1.1 leads Text to Image Leaderboard

Mon, Nov 4, 2024. Hertz-Dev is an open source realtime voice chat model. But it doesn't fit in Google Colab T4's RAM #github #tts #voice-cloning

Tue, Oct 29, 2024. F5-TTS clones voices with just 15-second samples. #future #speech-to-text #tts #voice-cloning

Sun, Oct 27, 2024. Elevenlabs lets you create voices with a prompt. No need to even clone one! #speech-to-text #tts #voice-cloning

Wed, Oct 9, 2024. Reverb ASR does diarration as well as transcription. It seems the state of art right now. #speech-to-text #tts #voice-cloning

Tue, Oct 8, 2024. Revisiting text to speech models. Nothing much has changed since July 2024. #speech-to-text #tts #voice-cloning
- OpenAI TTS: $15/1M chars Ref
- Deepgram Aura: $15/1M chars Ref
- Azure AI Speech: $15/1M chars Ref
- Google TTS Neural2: $16/1M chars Ref
- AWS Polly Neural TTS: $16/1M chars Ref
- Cartesia Pro: $50/1M chars Ref
- Elevenlabs Scale: $300/1M chars Ref

Thu, Oct 3, 2024. OpenAI's Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref #speech-to-text #tts

Thu, Oct 3, 2024. ChatGPT's advanced mode includes: "...you can use various regional accents and dialects." Ref Source #chatgpt #speech-to-text #tts #voice-cloning
- But the API can "laugh, whisper, and adhere to tone direction." Ref

Sat, Sep 21, 2024. Sarvam.ai offers Indic text to speech #speech-to-text #tts #voice-cloning

Mon, Jul 8, 2024. A quick check on the pricing of text to speech models #speech-to-text #tts #voice-cloning
- OpenAI TTS: $15/1M chars Ref
- Deepgram Aura: $15/1M chars Ref
- Elevenlabs Scale: $165/1M chars Ref
- Google TTS Neural2: $16/1M chars Ref
- Azure AI Speech: $15/1M chars Ref
- AWS Polly Neural TTS: $16/1M chars Ref

Fri, Feb 23, 2024. Teknoturf is using Gen AI to #ai-coding-tools #automation #prompt-engineering #speech-to-text #tts
- Improve prompts when teaching prompt engineering.
- Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce.

Wed, Jan 31, 2024. You can @mention GPTs to ask a specific GPT a question in ChatGPT. This is really powerful. #chatgpt #tts

Thu, Dec 21, 2023. I tried running Mixtral-8x7b locally (via Llamafile) and on together.ai. It's good, but far from GPT 4. #chatgpt #tts

Fri, Dec 15, 2023. Mixtral-8x7b-Instruct "... really does seem to be equivalent in quality to ChatGPT 3.5." Ref #chatgpt #speech-to-text #tts

Sun, Dec 3, 2023. Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations #embeddings #future #speech-to-text #tts #voice-cloning

Sat, Nov 4, 2023. 04 Nov 2023: Seamless4MT does language and speech inter-conversion. CC-NC-BY license #document-conversion #speech-to-text #tts #voice-cloning

Thu, Oct 26, 2023. 26 Oct 2023: ChatGPT AutoExpert is a great prompt mechanism for higher accuracy, context and control. Example #ai-coding-tools #automation #chatgpt #code-agents #prompt-engineering #speech-to-text #tts