#voice-cloning

• Stickers
  • Sending your wife AI-generated family photos, stickers, etc. is now a thing. Both an AI use case and a ... um... "family media" (?) use case. For example, ask ChatGPT to "Create a transparent comic-style sticker of a lady chef featuring this person happily cooking salad" with a photo. Then send it as a custom sticker. Image
  • Vadivelu stickers work well but the Tamil script generation is poor. Image
• Asking ChatGPT to generate 25-year younger pictures of people produces pretty poor results if you really knew what they looked like then. If you didn't, it's fairly convincing. Yet another example of "hallucinations" - except, it does have its uses.

• Personal writing with connection won't go away. AI can't give you heartbreak. But the rest of non fiction writing will vanish.
• What AI is extraordinary at is personalizing to each audience member's interest
• Outlier opinions will thrive among humans - since AI is trained on consensus.
• Managers tend to be good at working with LLMs because it's mostly about delegation.
• LLMs are perfect for things that don't have a wrong answer! -- Benedict Evans.
• 💡 Explore arguing with AI. It's a safe way to get into a confrontational emotional state (which has its own benefits.)
• 💡 Keep an LLM on in voice mode while reading and ask it any questions you have.
• What models are good for what?
  • GPT 4.5 is great for creation - has a great sense of humor but a corporate style. Still, way better than GPT 4o.
  • ChatGPT is good for voice transcription and note taking. (Increasingly we take notes for AI rather than ourselves.)
  • Claude 3.7 has the best style of writing. It's also great for drawing charts.
  • O1 Pro and Deep Research is great for consumption - research.
  • Grok is the least corporate, able to argue with you, and the latest knowledge cutoff.
  • ElevenLabs for editing podcasts in your voice, making corrections.

• Google's AI overviews now use Gemini 2.0. They've introduced an AI mode that functions like a mini deep research tool, incorporating planning and search. (A Perplexity-killer). It's a fine-tuned model that is extra cautious with topics like healthcare and always verifies information.
• QWQ from Quen competes with DeepSeq R1, but with only 32b parameters compared to R1's several hundred billion.
• AI models are becoming less restrictive. Gemini and GPT-4.5 have relaxed some constraints, shifting more responsibility onto users, similar to Grok.
• What's GPT-4.5 good for? It seems to excel in creativity, humor, education, emotional intelligence, and teaching. It follows instructions better and understands intent better. However, it's not a major leap in coding or math.
• OpenAI's Deep Research mode always uses O3, regardless of the model selected in the UI.
• Tencent has released a new video model available at https://aivideo.hunyuan.tencent.com/ and it appears to be quite good.
• Many clients now support Model Context Protocol (MCP), including Cursor, Claude Code, and Claude Desktop. The clients list is long. Some MCP uses include:
  • Interact with GitHub using the GitHub API.
  • Using Knowledge Graph memory to premember previous conversations
  • Using the Cloudflare MCP server to perform Cloudflare actions.
  • File retrieval and custom prompts -- which MCP supports in addition to tools.
  • Calling other MCPs or LLMs (conditionally) from an MCP, enabling the creation of full-fledged workflows.
• Composio offers a Hosted MCP service. CloudFlare lets you build remote MCP servers.
• Notagen is an open-source note generation engine that produces high-quality classical sheet music.
• Sesame has an open-source voice model worth exploring.
• DiffRhythm is a music generation model that appears to be quite good.

• Deepgram has a MediaRecorder API, which is perfect.
• Whisper Streaming Web is a web app that can transcribe audio real-time from the browser. A good approach, but I wouldn't use it for meeting transcription on my mid-end laptop. Streaming takes up the bulk of my GPU, leaving little for transcription.
• whisper-live runs as a Python console app and does something similar.
• Whisper WebGPU runs on the browser (only 200MB). Cool! But slow and still takes up GPU.

uvx --python 3.9 --index https://download.pytorch.org/whl/cu121 whisperx --diarize --lang en --hf_token $HUGGINGFACE_TOKEN`

• Idle call time is counted towards cost. So cost may be higher than OpenAI.
• Voice cloning quality is average. Very distinctive voices are just partly identifiable.
• Supports tool calls (from their server).
• Their API is simple but the docs have minor errors (e.g. a trailing comma in the JSON, which leads to an error) reducing confidence.

• Transcription: Accurately transcribe the audio clip in the original language. Include all spoken words, fillers, slang, colloquialisms, and any code-switching instances. Pay attention to dialects and regional variations common among immigrant communities. Do your best to capture the speech accurately, and flag any unintelligible portions with [inaudible].
• Translation: Translate the transcription into English. Preserve the original meaning, context, idiomatic expressions, and cultural references. Ensure that nuances and subtleties are accurately conveyed.
• Capture Vocal Nuances: Note vocal cues such as tone, pitch, pacing, emphasis, and emotional expressions that may influence the message. These cues are critical for understanding intent and potential impact.

• OpenAI TTS: $15/1M chars Ref
• Deepgram Aura: $15/1M chars Ref
• Azure AI Speech: $15/1M chars Ref
• Google TTS Neural2: $16/1M chars Ref
• AWS Polly Neural TTS: $16/1M chars Ref
• Cartesia Pro: $50/1M chars Ref
• Elevenlabs Scale: $300/1M chars Ref

• But the API can "laugh, whisper, and adhere to tone direction." Ref

• OpenAI TTS: $15/1M chars Ref
• Deepgram Aura: $15/1M chars Ref
• Elevenlabs Scale: $165/1M chars Ref
• Google TTS Neural2: $16/1M chars Ref
• Azure AI Speech: $15/1M chars Ref
• AWS Polly Neural TTS: $16/1M chars Ref

• ElevenLabs speaks in your voice
• Cutout Pro removes backgrounds and parts of images
• Vocal Remover removes vocals from songs
• CapCut video editor