#speech-to-text | Anand S - Things I Learned

Fri, Jan 17, 2025. Audio diaries are a thing. Monash University asks students to voice their learnings, share it with each other and have them give feedback. I wonder if ChatGPT diaries could become a thing, too, and LLM journalling starts helping with therapy. #future #speech-to-text #voice-cloning

Tue, Jan 14, 2025. I switched back from Brave to Edge, mainly because Edge's native text-to-speech and speech recognition is far better. I can use it better on my mobile. #speech-to-text

Sun, Jan 12, 2025. TTS Arena is a benchmark of text-to-speech models. Kokoro-TTS is the current leader. It's just 82M, runs on Google Colab, and sounds slightly better than OpenAI TTS. #speech-to-text #tts #voice-cloning

Wed, Jan 8, 2025. whisper-flow does real-time speech transcription! #future #speech-to-text #tts #voice-cloning

Wed, Jan 8, 2025. Switchboard-1 is a labelled audio corpus with ~260 hours of speech. It has ~2,400 calls among 500+ speakers in the US. #speech-to-text #voice-cloning

Fri, Jan 3, 2025. Assembly AI offers speech to text with diarization at 12c/hour. Good diarization, average transcription quality. #speech-to-text #tts #voice-cloning In comparison, WhisperX (with GPU) was much slower, had slightly poorer diarization, and slightly better transcription.
```
uvx --python 3.9 --index https://download.pytorch.org/whl/cu121 whisperx --diarize --lang en --hf_token $HUGGINGFACE_TOKEN`
```

Wed, Dec 18, 2024. #HARD prompt. Ask video generators like SORA to generate text in videos. It is of average quality. #future #speech-to-text

Wed, Dec 18, 2024. GPT 4o Mini Realtime was released. A realtime conversation will cost ~50c/hr. About 36c for input, 72c for output. (I extrapolated from the 6c/min audio input cost for GPT 4o Realtime when it was $100/MTok. GPT 4o Mini Realtime is $10/MTok input and $20/MTok output.) #chatgpt #pricing #speech-to-text #tts

Wed, Dec 4, 2024. Fish eye text summary is a great way to read text while summarizing context. Amelia Wattenberger #speech-to-text

Fri, Nov 29, 2024. GPT-4o Audio supports tone control via XML tags like <cough>..., <laugh>..., etc. But at ~$15/hr of output, it's too expensive. Ref #speech-to-text #tts #voice-cloning

Tue, Nov 26, 2024. Ultravox lets you build voice agents at 5c/min = $3/hr (OpenAI is 6c input, 24c output). Or clone their repo. #speech-to-text #voice-cloning
- Idle call time is counted towards cost. So cost may be higher than OpenAI.
- Voice cloning quality is average. Very distinctive voices are just partly identifiable.
- Supports tool calls (from their server).
- Their API is simple but the docs have minor errors (e.g. a trailing comma in the JSON, which leads to an error) reducing confidence.

Wed, Nov 20, 2024. Alt Text will very likely be a browser feature. It's important for the Alt text to flow as part of the content when listening to the page. Perhaps even become a part of the browser APIs like speechRecognition. #future #html #speech-to-text

Mon, Nov 11, 2024. Gemini transcription does not give accurate timestamps. Whisper does. But the quality of transcription is similar. #speech-to-text #tts

Fri, Nov 8, 2024. Here is a prompt for audio transcription using Gemini. Ref #speech-to-text #voice-cloning
- Transcription: Accurately transcribe the audio clip in the original language. Include all spoken words, fillers, slang, colloquialisms, and any code-switching instances. Pay attention to dialects and regional variations common among immigrant communities. Do your best to capture the speech accurately, and flag any unintelligible portions with [inaudible].
- Translation: Translate the transcription into English. Preserve the original meaning, context, idiomatic expressions, and cultural references. Ensure that nuances and subtleties are accurately conveyed.
- Capture Vocal Nuances: Note vocal cues such as tone, pitch, pacing, emphasis, and emotional expressions that may influence the message. These cues are critical for understanding intent and potential impact.

Fri, Nov 8, 2024. ChatGPT for Windows desktop supports real-time voice and a global shortcut (Alt Space). #chatgpt #speech-to-text #voice-cloning

Mon, Nov 4, 2024. Recraft.ai is currently SOTA in text to image. It's fairly impressive and could be a good alternative to Figma. #ai-art #future #image-generation #speech-to-text

Mon, Nov 4, 2024. Artificial Analysis has a bunch of new leaderboards and arenas. #speech-to-text #tts
- Open AI TTS leads the TTS Leaderboard. ElevenLabs is a bit behind.
- Recraft V3 > Flux 1.1 leads Text to Image Leaderboard

Tue, Oct 29, 2024. F5-TTS clones voices with just 15-second samples. #future #speech-to-text #tts #voice-cloning

Sun, Oct 27, 2024. Elevenlabs lets you create voices with a prompt. No need to even clone one! #speech-to-text #tts #voice-cloning

Wed, Oct 9, 2024. Reverb ASR does diarration as well as transcription. It seems the state of art right now. #speech-to-text #tts #voice-cloning

Tue, Oct 8, 2024. Revisiting text to speech models. Nothing much has changed since July 2024. #speech-to-text #tts #voice-cloning
- OpenAI TTS: $15/1M chars Ref
- Deepgram Aura: $15/1M chars Ref
- Azure AI Speech: $15/1M chars Ref
- Google TTS Neural2: $16/1M chars Ref
- AWS Polly Neural TTS: $16/1M chars Ref
- Cartesia Pro: $50/1M chars Ref
- Elevenlabs Scale: $300/1M chars Ref

Thu, Oct 3, 2024. Speak is a language learning app based on OpenAI's Realtime API. #future #speech-to-text #voice-cloning

Thu, Oct 3, 2024. OpenAI's Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref #speech-to-text #tts

Thu, Oct 3, 2024. ChatGPT's advanced mode includes: "...you can use various regional accents and dialects." Ref Source #chatgpt #speech-to-text #tts #voice-cloning
- But the API can "laugh, whisper, and adhere to tone direction." Ref

Sun, Sep 22, 2024. Sentient lets you control the browser via Python in natural language #prompt-engineering #python #speech-to-text

Sat, Sep 21, 2024. Sarvam.ai offers Indic text to speech #speech-to-text #tts #voice-cloning

Tue, Sep 17, 2024. Segmind's Hallo lets you animate a face to an audio clip #speech-to-text #voice-cloning

Fri, Sep 13, 2024. Hume provides a voice-to-voice model (EVI 2) that handles emotions at 7 cents/minute. #speech-to-text #voice-cloning

Thu, Sep 12, 2024. Pixtral seems quite good at OCR #document-conversion #future #speech-to-text

Tue, Aug 20, 2024. Lumentis creates docs from transcripts and text #document-conversion #future #github #speech-to-text

Thu, Jul 25, 2024. Speech editing in audio files is a thing. Speech Editing Toolkit and Descript #ai-coding-tools #speech-to-text #voice-cloning

Mon, Jul 8, 2024. A quick check on the pricing of text to speech models #speech-to-text #tts #voice-cloning
- OpenAI TTS: $15/1M chars Ref
- Deepgram Aura: $15/1M chars Ref
- Elevenlabs Scale: $165/1M chars Ref
- Google TTS Neural2: $16/1M chars Ref
- Azure AI Speech: $15/1M chars Ref
- AWS Polly Neural TTS: $16/1M chars Ref

Wed, May 29, 2024. Some audio embedding models: #embeddings #speech-to-text #voice-cloning unoti/voice-embeddings, retkowsky/audio_embeddings, pyannote/embedding (for speaker similarity), and more.

Sun, Apr 14, 2024. #WRITE fal.ai "animates" pictures, creating videos. It made one from my talk. I morphed into various somewhat similar people rapidly in a 2-second span. Very promising, and far from good. #future #speech-to-text #voice-cloning

Sun, Apr 14, 2024. Lemur from Assembly.ai does real time call transcription and summary #future #speech-to-text #voice-cloning

Sat, Mar 30, 2024. Hume.ai offers voice emotion API and emotion-based conversational responses. An empathic AI. #future #huggingface #speech-to-text #voice-cloning

Fri, Mar 29, 2024. pyannote-audio does speaker diarization #speech-to-text

Mon, Feb 26, 2024. MetaVoice 1B offers voice cloning on American & British accents with 30s training #speech-to-text #voice-cloning

Mon, Feb 26, 2024. Buildspace's demo is a great demo of how voice and actions can be used effectively. #speech-to-text #voice-cloning

Fri, Feb 23, 2024. Teknoturf is using Gen AI to #ai-coding-tools #automation #prompt-engineering #speech-to-text #tts
- Improve prompts when teaching prompt engineering.
- Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce.

Mon, Feb 19, 2024. All image-to-text models on HuggingFace #future #huggingface #speech-to-text

Sun, Feb 18, 2024. HuggingFace Chat Assistants has open source system prompts!! #future #huggingface #speech-to-text

Mon, Jan 15, 2024. Eleven-labs speech synthesis with voice cloning is at the uncanny valley. With two 5-minute samples, my voice sounds a fair bit like my voice but is very clearly not my voice. I find stability ~ 30%, similarity ~ 80% and style ~50% gives a reasonable outcome. But the default voices (e.g. Joseph, George, Charlie) are excellent. #speech-to-text #voice-cloning

Sun, Dec 17, 2023. ⭐ whisper-standalone-win provides a Windows binary for Faster-Whisper. It just needs CUDA and cuDNN installed. Then whisper-faster.exe video.mkv --language=English --model=medium generates the transcript. #speech-to-text

Fri, Dec 15, 2023. Mixtral-8x7b-Instruct "... really does seem to be equivalent in quality to ChatGPT 3.5." Ref #chatgpt #speech-to-text #tts

Sun, Dec 3, 2023. Tools to explore #ai-coding-tools #speech-to-text #voice-cloning
- ElevenLabs speaks in your voice
- Cutout Pro removes backgrounds and parts of images
- Vocal Remover removes vocals from songs
- CapCut video editor

Sun, Dec 3, 2023. Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations #embeddings #future #speech-to-text #tts #voice-cloning

Mon, Nov 6, 2023. 06 Nov 2023: ChatGPT is slightly better that Github Copilot or CodeWhisperer #ai-coding-tools #chatgpt #code-agents #github #speech-to-text

Sat, Nov 4, 2023. 04 Nov 2023: Seamless4MT does language and speech inter-conversion. CC-NC-BY license #document-conversion #speech-to-text #tts #voice-cloning

Thu, Oct 26, 2023. 26 Oct 2023: Auxi pro is a VS Code + Github Copilot Chat for PowerPoint #ai-coding-tools #github #speech-to-text

Thu, Oct 26, 2023. 26 Oct 2023: ChatGPT AutoExpert is a great prompt mechanism for higher accuracy, context and control. Example #ai-coding-tools #automation #chatgpt #code-agents #prompt-engineering #speech-to-text #tts