#automation | Anand S - Things I Learned

Thu, Dec 26, 2024. WOW. This is how a software agent will work alongside humans: Fix issue #5478: Add color to the line next to "Ran a XXX Command" based on return value - using @openhands-agent. #automation #code-agents #future

Sun, Dec 22, 2024. Examples of agentic AI #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
- Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI."
- Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed.
- Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts.
- Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions.
- Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies.
- Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information.
- Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing.
- Meeting notes and action items generator: A system that generates meeting notes and action items.
- O'Reilly auto parts customer service agent: An agent demonstrated using RAG.
- UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

Sat, Dec 21, 2024. Anthropic defines agents. Building effective agents + Cookbook #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
- Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
- Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
  - Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
  - Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
  - Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
  - Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
  - Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
  - Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
  - Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
- Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

Thu, Dec 12, 2024. I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: #automation #best-practices #llm-ops #optimization #prompt-engineering
- Ask better questions. (Prompt engineering.)
- Create templates or step-by-step instructions. (Chain of Thought.)
- Ask for multiple options and pick from the best options. (Agentic approach?)
- Training. (Fine tuning.)
- Price weaker responses lower. (Stratified model pricing?)

Wed, Nov 27, 2024. OpenAI allows multiple system messages. I learned this browsing through the LIDA prompts. #automation #future #llm-ops

Wed, Nov 27, 2024. Andrew Ng Explores The Rise Of Al Agents And Agentic Reasoning | BUILD 2024 Keynote #automation #code-agents #prompt-engineering
- Innovation is now a path to production. People are able to build 20 prototypes at the cost of one and see which sticks
- Machine learning is much faster. Things that took months now date days. But engineering and evaluations are only slightly faster and have become a bottleneck
- A good analogy to zero shot prompting is to ask a person to write an entire essay without pressing backspace even once
- Andrew scenes to align with the line chain definition of agentic workflow, which is about agents being able to craft their own control flows
- People find it very easy to understand agentic workflows once they read through the code
- Reflection or feedback is a useful agentic pattern
- In multi-agent collaboration, it may be the same underlying model that is acting as different agents. But just like we find it useful for the same CPU to run multiple processes and each application is its own abstraction, agents of useful abstraction

Mon, Nov 25, 2024. The OpenAI Desktop App can interact with native applications, e.g. read from Terminal, VS Code, etc. This takes it on a path to becoming a copilot for ANY apps. Putting every copilot app and every LLM integration under threat. #automation #code-agents

Mon, Nov 11, 2024. Pass a complex data structure to Claude.ai and have it create an app to visualize it. It does well. Simin Willison #automation #markdown

Sat, Nov 9, 2024. Zapier Actions are an easy way to set up custom actions like GMail / Google Calendar APIs for GPTs, since GPTs' callback URLs keep changing. But they fail often, and don't work on mobile. At least for me. #automation #chatgpt

Mon, Oct 28, 2024. Pragmatic Podcast (I forgot which one) #ai-coding-tools #automation #code-agents #future #github #prompt-engineering
- Automate changelogs for your codebases. Convert past commits into attractive release notes automatically
- AI is going to be the consumer of many tools and logs. Build converters for these
- Speed of validation such as linting, testing, etc. will allow LLMs to iterate faster and WILL become more important

Thu, Oct 17, 2024. AI seems to be slowing down apprenticeship since experts would rather use an AI than train an apprentice. Example: Robotic surgery. Ref #automation #future

Fri, Oct 4, 2024. Software 2.0 by Andrej Karpathy. #ai-coding-tools #automation #dev #markdown #prompt-engineering
- This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
  - the 2.0 programmers (data labelers) edit and grow the datasets, while
  - a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
- Adaptive UI ideas:
  - Adaptive Fields: Show only required fields based on what the user field so far.
  - Smart Inputs: Dropdowns and auto-complete based on user's context.
  - Smart Themes: Change font size, contrast, theme guessing the user's age and preferences.
  - Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs.
  - Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips.
  - Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing.
  - Smart Charts: Create the right chart that solve the user's question.
- Adaptive Back-end
  - Dynamic APIs: Create endpoints on the fly based on user needs
  - Dynamic Indexing: Create & update indices on the fly based on user needs
  - Dynamic Schema: Create & update schema on the fly based on user needs
  - Dynamic Migration: Migrate to a new database or OS or language as required
  - Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
  - Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
  - Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error
  - Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys.

Sun, Sep 15, 2024. Prompt engineering tip: Tell LLMs another AI wrote code. Else they will agree with you! #ai-coding-tools #automation #code-agents #future #llm-ops #prompt-engineering

Mon, Aug 12, 2024. Code agent frameworks to explore: #ai-coding-tools #automation #code-agents #dev #future #github #markdown #prompt-engineering

Tue, Jun 25, 2024. LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: #ai-coding-tools #automation #best-practices #future #llm-ops #optimization #prompt-engineering
- Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
- Collect feedback: Explicit and implicit

Wed, May 29, 2024. AI reporting templates are a trend. Domain expertise comes in via structuring the report template and associated prompts. #ai-coding-tools #automation #code-agents

Tue, Mar 19, 2024. Control vectors in real-time lets you control response in real-time #automation #future #prompt-engineering

Fri, Feb 23, 2024. Teknoturf is using Gen AI to #ai-coding-tools #automation #prompt-engineering #tts
- Improve prompts when teaching prompt engineering.
- Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce.

Sun, Feb 4, 2024. #PREDICTION As automation takes over these mainstream activities, people will take over the niches. Since expertise like knowledge is fractal, there will be many more segments of one in the future and it will be easier to automate clusters of similar abilities. Recommenders and brands will become even more important #automation #future

Sat, Jan 27, 2024. ⭐ OpenAI's prompt engineering strategies are an excellent start for prompt engineering. A few lessons: #automation #prompt-engineering
- Use detailed system prompts, often containing the entire instruction set, if it won't change over the course of a conversation.
- "... summary of the prior conversation could be included as part of the system message" is an interesting history compression tactic.

Sun, Jan 14, 2024. AI by API is the norm today and will grow #ai-coding-tools #automation #future
- Just having AI is no longer a differentiator
- AI is part of life, not just work
- #TODO Explore quickdrop from Stability for Maruti
- #TODO Explore Codium VS Code plugin and Continue.dev
- Hybrid systems that combine stats, ML, DL and AI models will grow
- AGI and AutoGPT resurgence
- RAG will continue to be a focus
- GPT4 will be beaten by open source models. Special purpose models beat it already
- Self hosted and cloud hosted models will grow for security
- Small language models will grow
- Productivity will be enhanced rather than replaced
- Multi modal models will grow
- Cost efficiency will grow in focus

Thu, Nov 30, 2023. When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref #ai-coding-tools #automation #code-agents #future #llm-ops #optimization #prompt-engineering

Sat, Nov 25, 2023. Agents mimic programming primitives. Switch. While. Call a function. Print. #ai-coding-tools #automation #code-agents #prompt-engineering

Mon, Nov 6, 2023. microsoft/autogen enables multi-agent conversations. #automation #code-agents #future

Thu, Oct 26, 2023. 26 Oct 2023: ChatGPT AutoExpert is a great prompt mechanism for higher accuracy, context and control. Example #ai-coding-tools #automation #chatgpt #code-agents #prompt-engineering #tts