Sun, Oct 13, 2024 | Anand S - Things I Learned

O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution #future #llm-ops #optimization #prompt-engineering

GitHub co-pilot workspaces let you code using your mobile with AI and deploy it at one shot #ai-coding-tools #future #github

If you need an Ubuntu Docker container with Python, install it via uv rather than compiling from source. via

VTracer is an open source library (and tool) to convert raster images to SVGs. via

If you want to create a console.llm() function, a browser extension is the best way, because some pages have Content-Security-Policy that block eval, form submission, fetch from other domains, and script execution. #html #llm-ops

PyPi lets you publish from GitHub Actions without a token. Also from Gitlab.com CI/CD and Google Cloud. #github

ActiveState which made ActivePython, ActivePerl, etc. made these products paid for commercial use around 2013 after a series of acquisitions.

Marimo supports: #markdown
- Publishing any notebook to static.marimo.app as a static app
- Creating a SINGLE link that embeds the ENTIRE notebook in the URL!
- Runnable via uvx marimo edit

Parables on the Power of Planning in AI: Giving models about 30 seconds of thinking time consistently improves results - as much as increasing parameter size by a factor of 1,000 to 100,000! #best-practices #optimization #prompt-engineering
- This works particularly well for verifiable results (code, math, etc.)
- Technique: Ask an LLM hundreds of times at low temperature and pick the most common one. (Google's Minerva used this on the MATH dataset.)
- Better Technique: Ask an LLM hundreds of times. Pick the best solution based on an evaluation metric (reward model)
- Better Technique: Apply a reward model at EACH step of the process. OpenAI's "Let's Verify Step by Step"

Reverb ASR does diarration as well as transcription. It seems the state of art right now. #speech-to-text #tts #voice-cloning

Gemini Flash and Gemini Flash 8b can be fine-tuned at zero cost. Inference is at the same price! Ref

Flux 1.1 Pro is released. I tried my Calvin & Hobbes test on it. Not great. ImageGen3 is better, ChatGPT is the best. Ref #image-generation

Late chunking is an interesting approach to adding context to embeddings. (I don't understand it, but it's cheap and effective.) #embeddings #markdown

DeepInfra offers embedding models as APIs at about 0.5 to 1 cent per MTok in an OpenAI compatible API. #embeddings It also supports text-to-image models like flux.dev and speech recognition models like Whisper.

Jake Heller: #future #llm-ops
- "One of the things we learned is (an LLM app) after it passes passes frankly even 100 tests, the odds that it will do, on any random distribution of user inputs, the next 100,000 100% accurately is very high."

OpenAI's O1 is like Daniel Kahneman's System 2 thinking - as against other LLMs' System 1 thinking. #llm-ops