#optimization

• Sun, Aug 31, 2025. Notes from Anthropic's How we built our multi-agent research system #automation #code-agents #optimization #prompt-engineering
  • Multi-agent systems are like organizations that can do more than a single human.
  • Multi-agent systems conserve the context window.
  • The top 3 drivers of performance variance: spending more tokens, more tool calls, better models
  • You need to teach (prompt) the orchestrator how to delegate to sub-agents
    • How to avoid task duplication among agents
    • How many sub-agents to spin up for different kinds of tasks
    • Which tools to use for what
    • Provide sub-agents objective, output format, tools/sources, clear task boundaries
  • ⭐ Self-improving agents, e.g. prompt optimizers or tool-testing agents that run and rewrite tool descriptions, are powerful
  • Since agents are stateful, resuming from failure is important.
  • Agent prompts are public

• Sat, Aug 30, 2025. This CodeMonkeys paper suggests running a mixture of agents in parallel for multiple code + test tasks and auto-pick the best by running and LLM-rewriting tests. #ai-coding #automation #code-agents #optimization #prompt-engineering

• Tue, Aug 26, 2025. When seeking advice, people sometimes think aloud, become repetitive, and introduce detail before clarifying intent. Kind candor helps. You can: #best-practices #optimization #prompt-engineering
  • State time boundaries. "We have 20 min. If we spend 5 min on your question, we'll have 15 for solutions."
  • Clarify intent upfront. "Before we dive in: What can I help with?"
  • Interrupt, summarize, clarify early. "Cooperative interruptions" are seen as supportive. E.g. "I get this: six accelerators, two done. Great! What can I help with? To accelerate?"

• Mon, Aug 25, 2025. With AI coding, task breakdown, context right-sizing, and automated testing are key levers. #ai-coding #automation #best-practices #code-agents #future #optimization #prompt-engineering

• Fri, Aug 22, 2025. ⭐ Creating systems that rapidly improve from feedback is the key to success. Rapidity, quality of improvement, quantity of feedback are all enablers. #best-practices #future #optimization #prompt-engineering

• Fri, Aug 22, 2025. DSPy auto-optimizes prompts based on input-output pairs or evals. Typical improvements are ~10-20%. My opinion: avoid. It's a good idea, but has too much abstraction that hides the implementation. Worth learning from but not implementing. #automation #dev #gpu #markdown #optimization #prompt-engineering #prediction

• Wed, Aug 20, 2025. Allow humans to intervene while agent loops execute, even unsolicited, to improve collaboration. Deepak Sharma #automation #code-agents #future #optimization #prompt-engineering

• Wed, Aug 20, 2025. Given the early, experimental days of AI, the better KPIs might be more about experimentation (e.g. number of prototypes) than operational (e.g. cost reduction). Krishnakumar Menon #best-practices #future #optimization #prompt-engineering

• Wed, Aug 20, 2025. ⭐ "Potentially nitpicky but competitive advantage in AI goes not so much to those with data but those with a data engine: iterated data aquisition, re-training, evaluation, deployment, telemetry. And whoever can spin it fastest. Slide from Tesla to ~illustrate but concept is general." Andrej Karpathy, Dec 2022 #automation #future #optimization #prompt-engineering

• Wed, Aug 20, 2025. The ROI of prompting carefully and using meta-prompts is high. Prompt clarity reduces iterations & dead-ends. The initial time spent (10-15 min) pays off with just a single reduced iteration (time to generate + review). Tanika Gupta #best-practices #optimization #prompt-engineering

• Sat, Aug 16, 2025. From the GPT 5 coding cheatsheet: #ai-coding #best-practices #code-agents #optimization #prompt-engineering
  1. Be precise and avoid conflicting information. Use a prompt optimizer to check for inconsistencies.
  2. Use the right reasoning effort. Prefer medium or low reasoning to avoid overthinking simple problems.
  3. Use XML-like syntax to help structure instructions
  4. Avoid overly firm language, e.g. "You MUST be THOROUGH" vs "Thoroughly".
  5. Give room for planning and self-reflection. Explain what to do in steps, asking it to think deeply
  6. Control the eagerness of your coding agent, e.g. do not ask for confirmation, parallelize tool calls, use more tools, etc.

• Mon, Aug 11, 2025. Notes from "Scaffolding the Problem-Solving Process for Introductory Computing Students" by Ashish Dandekar, NUS, at PyConSG Edu Summit 2025 #best-practices #optimization
  • Built an intelligent tutoring system
  • Encourage students to create their own pattern banks / cheat sheets. "Find 2 more problems that can be solved in the same way."
  • Focusing on the problem-solving process shrinks the gap. Students above the 50th percentile of pre-assessment did not improve much. The lowest percentile improved the most.
  • "At NUS, I know that even if I give 0.5% weightage for students attending tutorials, everyone will attend it for those 'free marks'."

• Tue, Aug 5, 2025. Concise explanations speed up decisions because they're faster to read and understand (obvious). They're also easier to combine with other ideas (less obvious). # #markdown #optimization #easy

• Mon, Aug 4, 2025. ⭐ In "kind" environments (stable rules, rapid and accurate feedback), specialize. In "wicked" environments (rules shift, feedback is noisy/late), generalize. ChatGPT #best-practices #chatgpt #code-agents #future #github #markdown #optimization #prompt-engineering

• Mon, Jul 28, 2025. LLMs have an internal "thinking progress" bar in its hidden states (a "Thinking Progress Vector"). By moving the bar forward ("overclocking") you can make them conclude faster without hurting accuracy! Can't do this with APIs, but is a way by which LLMs might start speeding up. Overclocking LLM Reasoning #llm-ops #optimization #learning

• Mon, Jul 28, 2025. Since coding is fast, deciding the next feature is a bottleneck. The Batch. #optimization #prompt-engineering #ai-coding
  • Ask PMs who know what users want
  • Ask PMs again after sharing log analysis and survey analysis with them
  • Automate via LLMs to scale backlogs

• Sun, Jul 27, 2025. AlphaEvolve implemented an evolutionary loop: Given initial code and evaluation code, Gemini 2.0 Flash and Gemini 2.0 Pro suggested changes, stored the revised program in a database, evaluated it, suggested further changes, and repeated the process. With automated evaluation this is a very powerful approach. The Batch #ai-coding #automation #code-agents #dev #future #markdown #optimization #prompt-engineering

• Wed, Jul 23, 2025. Agents face 3 problems: compounding errors, quadratic costs, and poorly designed tools. Start with small scope & strong reviews while you solve these problems. Betting Against Agents #code-agents #optimization

• Tue, Jul 22, 2025. CLI optimization for LLMs will likely emerge. More CLIs (and wrappers / hooks in the shell) will improve output and error contexts for LLMs, e.g. printing current directory, caching slow outputs, suggesting alternate commands, etc. Ref #llm-ops #optimization

• Mon, Jun 30, 2025. ⭐ When bringing in humans-in-the-loop, applications must make it easier to review and to edit the work. #automation #best-practices #document-conversion #future #optimization #prompt-engineering #write

• Wed, Jun 25, 2025. When AI automates a task, the bottleneck shifts. AI process re-design is about reworking the process around the new bottleneck, and iterating quickly. #ai-coding #automation #optimization #prompt-engineering
  • With coding, it's testing, reviewing, deploying, use-case identification.

• Sat, Jun 21, 2025. Metaprompting. Have frontier LLMs revise prompts. They're GOOD! Ref #best-practices #future #llm-ops #markdown #optimization #prompt-engineering
  • Increase clarity, providing step-by-step instructions.
  • Resolve conflicting instructions.
  • Expand instructions to cover all scenarios and edge cases.

• Thu, Jun 19, 2025. For all AI code, review it, learn from it and share learnings. That prevents bugs AND we learn in the process. Ref #ai-coding #best-practices #code-agents #future #github #optimization #learning

• Sat, May 31, 2025. LLMs optimize for compression. Humans optimize for adaptive flexibility. Ref arXiv #llm-ops #optimization

• Mon, May 12, 2025. We could "fine-tune" system prompts automatically with evals, creating a "system prompt learning" paradim -- like my promptevals. Andrej Karpathy #automation #future #markdown #optimization #prompt-engineering

• Mon, May 12, 2025. I was asked how to improve speed when building an enterprise ChatGPT clone using an API. Here's what I'd suggest, in order: #best-practices #chatgpt #optimization #prompt-engineering
  • Streaming. High impact, low effort.
  • Caching RAG retrieval as well as generation. High impact, low effort.
  • UI tweaks. Loading / streaming icons and progress hints ()"Retrieving context", "Generating answer", etc.)
  • Parallelize, if possible
  • Use model options where available, e.g. speculative decoding, models with higher speed, models with closer CDN, etc.
  • Shorten prompts
  • Persistent HTTP/2 Keep-Alive. Low impact, low effort (tweak server settings).

• Tue, Apr 22, 2025. "Evaluation in the loop" or "Evals-in-the-loop" is a new term I learnt. SignalBloom's Hallucination Bechmark #ai-coding #embeddings #future #optimization #prompt-engineering

• Fri, Mar 28, 2025. 13 things I would have told myself before building an autorouter has a few interesting points: #optimization
  • The A* algorithm finds the shortest path in a graph much quicker than others like Dijkstra's algorithm by preferring nodes closer to the goal.
  • Spatial Hash Indexing are O(1) and beat Tree Data Structures which are O(log n). Always prefer hashes when possible.

• Sat, Mar 22, 2025. If we can DESCRIBE what good looks like, training data is no gap. We can auto optimise models towards that. That's RLF. DeepSeek R1 side stepped the need for training data by creating reward functions and prompts. This tells the fine tuning process how to go correct as it goes along. This video is the first one that really help me understand what's going on. #optimization

• Fri, Mar 14, 2025. Unsloth might be one of the simplest ways of fine-tuning. #automation #markdown #optimization #prompt-engineering

• Fri, Mar 14, 2025. Apify is a cloud scraper platform. Here's how they optimize their AI Web agent - Source: #ai-coding #automation #code-agents #html #optimization #prompt-engineering #web-dev
  • Remove redundant tags and attributes (e.g. accessibility, etc.). Explore readability.
  • Add a unique gid to each element.
  • Add the screenshot WITH a "Set of Marks" - "SoM" (read research paper) highlighting important clickable elements.
  • Code output is brittle. Use tools / DSL - e.g. visit_url(url), click_element(text, gid, tagName), etc.

• Mon, Mar 3, 2025. Notes from AI Engineering Summit, NY. Session by Lux Capital. #automation #code-agents #llm-ops #optimization #prompt-engineering
  • Agents make multiple LLM calls. Errors accumulate. So the quality of the model is key
  • What's really critical: data + context + user preference
  • Set up evals for subjective responses by collecting signals continuously.
  • Create scaffolding for agents where errors don't accumulate. Better yet, make it FIX errors
  • UX is critical. We need lots more UX styles

• Mon, Jan 20, 2025. If you put LLMs in a feedback loop, it can optimize for its reward function by emotionally pushing people, generating misinformation, nudging towards a narrow definition of creativity, etc.: https://bsky.app/profile/emollick.bsky.social/post/3lg4darqwfc2d #llm-ops #optimization #learning

• Fri, Jan 17, 2025. Doctors can only do so much. Air quality, where you live, etc can do more for the patient than medicines or the doctor. If doctors keep this in mind, they can be more effective. #best-practices #future #optimization
  • Extending that thought, ANYONE who leverages assets through holistic thinking, becomes FAR more effective.

• Sat, Jan 11, 2025. Can LLMs write better code if you keep asking them to “write better code? A delightful exploration of how Claude 3.5 Sonnet keeps optimizing and adding features to improve code. My takeaway: repeatedly applying a prompt gives us interesting new directions to explore. #ai-coding #code-agents #llm-ops #markdown #optimization #prompt-engineering #write

• Thu, Dec 12, 2024. I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: #automation #best-practices #llm-ops #optimization #prompt-engineering #ask
  • Ask better questions. (Prompt engineering.)
  • Create templates or step-by-step instructions. (Chain of Thought.)
  • Ask for multiple options and pick from the best options. (Agentic approach?)
  • Training. (Fine tuning.)
  • Price weaker responses lower. (Stratified model pricing?)

• Wed, Nov 27, 2024. With LLMs, fostering innovation is a new path to profitability. Companies are increasing innovation team sizes. Productionizing that is the next. Some initiatives are: #best-practices #llm-ops #optimization
  • Convert popular demos into starter kits
  • Create and evangelize trainings on solutions and solution techniques
  • Create larger pools of capacity to build innovation and productionize it

• Thu, Oct 17, 2024. SQL optimizations for multi-threaded web applications. Ref #optimization
  • PRAGMA journal_mode = WAL. Improves performance for frequent writes. It allows concurrent reads and writes.
  • PRAGMA synchronous = NORMAL. Improves performance. We might lose a few transactions but won't corrupt the database.
  • PRAGMA mmap_size = 128000000. Set global memory map for processes to share data
  • PRAGMA journal_size_limit = 64000000. Limit WAL file to prevent unlimited growth
  • BEGIN IMMEDIATE instead of BEGIN. Prevents writes to the journal file until the transaction is complete. Improves concurrency.

• Wed, Oct 9, 2024. O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution #future #llm-ops #optimization #prompt-engineering

• Sun, Oct 6, 2024. Parables on the Power of Planning in AI: Giving models about 30 seconds of thinking time consistently improves results - as much as increasing parameter size by a factor of 1,000 to 100,000! #best-practices #optimization #prompt-engineering #learning
  • This works particularly well for verifiable results (code, math, etc.)
  • Technique: Ask an LLM hundreds of times at low temperature and pick the most common one. (Google's Minerva used this on the MATH dataset.)
  • Better Technique: Ask an LLM hundreds of times. Pick the best solution based on an evaluation metric (reward model)
  • Better Technique: Apply a reward model at EACH step of the process. OpenAI's "Let's Verify Step by Step"

• Mon, Aug 19, 2024. Anthropic Prompt Caching is 90% cheaper to use and 25% costlier to create. So if there's a 27% chance it'll be re-used, cache it. #optimization #prompt-engineering

• Sat, Aug 10, 2024. Embedding models can be fine-tuned. Example: #embeddings #future #gpu #markdown #models #optimization #todo

• Wed, Aug 7, 2024. Notes on LLM Fine-Tuning #gpu #llm-ops #optimization
  • Rouge 2 and Bleu and such metrics are NOT good. Create you own benchmarks
  • Non-PEFT fine tuning needs 6X GPU RAM. Optimizer states, Gradient, Activations are the overhead. PEFT is about tuning a subset of parameters.
  • LORA adds additional weights without updating the model. It's a low rank matrix multiplication. You can change these adapters in runtime. Saves space. Fast to train
  • Quantization: Stick to bitsandbytes or AWQ (may be a bit better)
  • QLORA = Quantization + LORA
  • Predibase has open-sourced Lora Adapters in "Lora Land". Existing adapters are pretty good.
    • ghcr.io/predibase/lorax:main Docker image works on Docker compose to run locally. devices: on Docker Compose lets you specify NVIDIA GPU devices
    • Locust is a HTTP load testing lib in Python
  • Techniques for inference optimization
    • Dynamic adapters: Loads right LORAX adapters WHEN a request comes in
    • Multi-adapter batching: Process all inputs in parallel on the same GPU, but different users are post-processed using different adapters

• Tue, Jun 25, 2024. LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: #ai-coding #automation #best-practices #future #llm-ops #optimization #prompt-engineering
  • Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
  • Collect feedback: Explicit and implicit

• Thu, May 2, 2024. Hidden brain podcast. Innovation 2.0 #future #optimization
  • solve your own problem. Don't solve other people's problems. This helps you pick what you're good at
  • affordable losses. Make sure you survive
  • borrow others' spares. spare time, scrap data, anything others don't use. If you can monetize it, you can pay them back
  • focus on the controllable. Ignore what's outside your control don't even waste time on it

• Thu, Apr 18, 2024. Effort engine introduces "effort" as a parametrizable way to speed up LLMs with a quality trade-off. Works on Mistral for now. #gpu #llm-ops #optimization #prompt-engineering

• Tue, Apr 9, 2024. Do nothing. Then do less #optimization
  • Humans have a bias against inaction. Hence a strategic advantage. What can you cancel today?
  • Humans have a bias against subtraction or removal. That too is a strategic advantage. What can you remove today?
  • Humans have a bias against constraints. That's a strategic advantage. What constraint can you embrace?
  • No Yay! When declining something, add it your calendar so that when the time comes you can say yeah I got this time back

• Sat, Feb 17, 2024. Fine tuning makes economic sense only if the input tokens SAVED is twice the output token size on each call. #future #optimization #prompt-engineering

• Sun, Feb 11, 2024. Two pass generation is a clever technique to get multiple SEQUENTIAL answers in a single API request. For example the schema {'code', 'optimized_code'} will generate code and then optimize it. #code-agents #future #optimization #prompt-engineering

• Thu, Dec 21, 2023. Generic computate-intensive algorithms eventually beat domain-specific tuning, because of Moore's law. Ref #gpu #optimization

• Thu, Dec 7, 2023. 12 Tuning Strategies for RAG #markdown #optimization #prompt-engineering

• Sun, Dec 3, 2023. Unsloth offers faster lower-memory LLM QLoRA finetuning #llm-ops #optimization

• Thu, Nov 30, 2023. When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref #ai-coding #automation #code-agents #future #llm-ops #optimization #prompt-engineering #learning

• Sat, Nov 25, 2023. Learn ML optimization approach - LLMOps. Have an evaluation framework with metrics like weights and biases or tensorboard. Helps figure out where fine tuning helps and where RAG does. Soon, this will become important. #llm-ops #markdown #optimization

• Mon, Oct 23, 2023. 23 Oct 2023: Optimum is a set of performance optimization libraries for transformers #gpu #optimization