Sun, Dec 22, 2024 | Anand S - Things I Learned

Anthropic defines agents. Building effective agents + Cookbook #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
- Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
- Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
  - Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
  - Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
  - Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
  - Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
  - Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
  - Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
  - Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
- Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

o1-preview diagnoses better than doctors. Harvard #future

OpenAI's release of ephemeral tokens via sessions (valid for 1 minute) are a useful way of exposing apps for public demos. Currently it works only for the Realtime API, though.

SpreadsheetLLM is a way of encoding spreadsheets in an LLM friendly format. #llm-ops

#HARD prompt. Ask video generators like SORA to generate text in videos. It is of average quality. #future

GPT 4o Mini Realtime was released. A realtime conversation will cost ~50c/hr. About 36c for input, 72c for output. (I extrapolated from the 6c/min audio input cost for GPT 4o Realtime when it was $100/MTok. GPT 4o Mini Realtime is $10/MTok input and $20/MTok output.) #chatgpt #pricing #tts

The King James Bible and all Harry Potters, each, are about $1M tokens (rounded off).

Claude 3.5 Sonnet is way ahead of competition on the LMSYS Webdev Arena

What to use for hosting: ChatGPT #chatgpt #cloud #github #hosting #html #server #web-dev
- GitHub Pages: Static websites, medium files
- Cloudflare Pages: Static websites, global delivery
- Vercel: Frontend frameworks (e.g. Next.js) with high DX and ISR, small files
- Netlify: JAMstack projects, minimal back-end, moderate files
- Glitch: Small static projects
- Render: Full-stack apps requiring databases and server-side compute
- Firebase Hosting: Small sites, limited large files
- Archive.org: Public archival, large files
- Google Drive: File sharing, large files
- Dropbox: File sharing, moderate files
- Cloudflare R2: Static assets, large file delivery

To download YouTube subtitles, use: yt-dlp -q --skip-download --convert-subs srt --write-sub --sub-langs "en" --write-auto-sub --print "requested_subtitles.en.url" "$url" Simon Willison

markdown2 is the new de facto Markdown library for Python. #markdown

Raspberry Pi 5 has a faster CPU, more RAM and GPU, 4K support, multiple USB 3 ports

Government websites like the official press releases cannot be crawled from outside India. Hence the need for server farms in India!