#automation

• Estimating tool capability & task allocation
• Task breakdown
• Spec-ing: which of user personas, user-journey maps, wireframes, technical architecture, psuedo-code
• Standards: tech stack, tools, linters, security, doc standards
• Git versioning & collaboration
• Code review. (Using AI.) Providing feedback. Modularity, naming, ...
• Automated validation
• Post-mortem. Learning from errors and successes, choices LLM made

• Commands are used for:
  • Requirements: Research sub-agent, task breakdown into todos.md, creating specs.md from todos.md
  • Progress tracking: session logging, effort tracking, updating status, planning next steps
  • Project setup: initializing, adding deps, scaffolding features
  • Development: code review, debug error (five whys), explain code, refactor code
  • Optimization: optimize build, DB, caching
  • Testing: TDD, generate test cases, set up unit/integration/E2E testing, analyze coverage
  • Security: security audits, dependency vulnerability scans
  • Integration: sync tasks between GitHub and Linear (two-way issue synchronization, PR linking)
  • Deployment: prepare releases, hotfix deploys, rollbacks, containerization, CI pipeline setup
• Patterns of usage
  • Sub-agents
  • Command handoffs, i.e. one command invoking another
  • Shared among a team in a repo, enforcing standards & sharing best practices
  • Integration with specific tools / APIs (e.g. Linear)

• Context engineering
• Documentation
• Automated testing
• Standards
• Capabilities of platforms
• Modularity (and DRY vs WET)
• Code composition
• Code reviews

• Speech-to-text. Cannot stress this enough but talking to the machine means you’re more likely to share more about what you want it to do.
• I maintain some basic prompts and context for copy-pasting at the end or the beginning of what I entered.
• I ended up preloading executables on the PATH that override the default ones, steering Claude toward the right tools, e.g. running python asks it to use uv.
• I use the task tool frequently for basic parallelization and context isolation.
• Simply taking time to talk to the machine and give clear instructions outperforms elaborate pre-written prompts.
• Forcing myself to evaluate the automation has another benefit: I’m less likely to just blindly assume it helps me.

• claude --debug shows what Claude Code is doing behind a scenes -- and is a good way to understand hidden / undocumented features.
• At the end of each session, ask Claude Code: "Document learnings. What failed? What worked? What's next?"
• Have Claude Code write its own prompts by having it launch sub-agents and create common commands in .claude/commands/.
• Symlink CLAUDE.md, AGENTS.md and GEMINI.md into a CONVENTIONS.md
• Prefer creating tools / writing scripts to analyze data and feed results -- reduces input tokens.

• Auditize: citations, rationale, output screens, diffs, test results, risks, unknowns
• Auto validate. Evals, tests
• Prioritize. High z-values, big-useful-surprising areas

• Complementary skills enable a team of 1. Non-coders can code. Non-domain people get insights from data
• Solves starting trouble. It offers a first draft
• Generation. New ideas (reduces blind spots), scenarios, non-existent people, new data, new persona for surveys
• Hyper-personalization. Parts of YouTube relevant for THIS asset manager. Implication of data for me
• Automated scaling. Generate 1,000 images. Evaluate 1,000 assignments
• Saves time: debugging, research, validation, documentation, copywriting
• New ways of working. Loading event schedules into my calendar

• Documents. E.g. Docsearch, GPTs, Microsoft Copilot, Gemini
• Slides. E.g. Microsoft Copilot, Gemini
• Sheets. E.g. Microsoft Copilot, Gemini
• Code. E.g. Cursor, Claude Code
• Database. Create DB schema, ER diagrams, synthetic data, ingestion scripts, etc.
• Data (analysis). E.g. Datachat, Google Colab, Marimo
• Posters. E.g. Postgen
• Shell. E.g. Warp
• Topic modeling. E.g. classify
• Surveys. E.g. Personagen
• APIs. E.g. apiagent
• Drug regulatory submissions.
• Contracts (risk).
• Manufacturing SOPs.
• Curriculum.
• Data quality.
• Support tickets.
• Dashboards.
• IaaC / DevOps.
• Video campaigns.
• Resumes.
• Patents.

• With coding, it's testing, reviewing, deploying, use-case identification.

• Subagents are like humans -> society. The improvement is dramatic.
• "Subagents facilitate compression by operating in parallel with their own context windows, exploring different aspects of the question simultaneously before condensing..."
• "Each subagent also provides separation of concerns—distinct tools, prompts, and exploration trajectories ... (enabling) independent investigations."
• Using subagents spends ~15x more tokens. (That explained ~80% of the improved accuracy!)
• Particularly effective when tasks are independent and parallelizable. This also speeds it up.
• Teach the orchestrator how to delegate: how many subagents, what objective + output format + task boundaries (MECE to avoid overlap with other agents) in prompt, what tools.
• Teach the orchestrator how to improve agents: e.g. tools to test and rewrite tool descriptions
• Even if you evaluate a few examples, evals are surprisingly effective.
• Agents are stateful. Errors compound. Allow agents to resume. Prune history gracefully.
• Log everything to debug user-reported failures. Also monitor the kinds of decisions it took to help debug at scale.

• Sources
  • How I program with LLMs
  • How I program with agents
  • The 7 Prompting Habits of Highly Effective Engineers
  • AI Assisted Coding
  • A Glimpse of the Future
  • Agentic Coding Recommendations
  • My First Open Source AI Generated Library
  • We Can Just Measure Things
  • I Shipped a macOS App Built Entirely by Claude Code
  • Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
• Why AI coding?
  • Reduces mental energy (by creating the first draft). letting you create more.
  • Reduces starting trouble, eases effort.
  • Helps figure out how easy / tough a task really is!!
  • Most code is short-lived or has few users. AI building "throw-away" code is useful.
• Why NOT AI coding?
  • Slows you down if you know the repo well
  • Doesn't work well on large/complex/niche repos
  • Leads to over-optimism and atrophy
• Tips
  • Use for reversible decisions (2-way doors). Avoid for irreversible ones (1-way doors).
  • Fail early. Try tough bits first.
  • Fail often. Restart instead of fixing.
  • Go concurrent. Trigger multiple tasks. Ask for multiple drafts and options.
  • Give it workflow. Break down the implementation into: 1. Planning. 2. API stubs. 3. Implementation.
  • Give local context. Naming conventions, folder structure, coding style, tools (compile, test, lint), etc.
  • Conserve context. Use sub tasks and sub agents to conserve context.
  • Suggest libraries. Agents prefer writing code than using libraries, by default.
  • Give examples to follow, e.g. Write it like @filename. & -> & but &x -> &x.
  • Give screenshots and logs. These are very effective.
  • Provide goals, not instructions. Saves effort, teaches you new things.
  • Farm out research. Have specialized tools research API docs, etc. and include those in the context.
  • Keep related things together.
  • Have it write a checklist, e.g. saving it temporarily in a file.
  • Have it run code to catch its own errors.
  • Have it write tests, mocks for tests.
  • Have it see and use the app, click, play around, etc. (e.g. via playwright-mcp)
  • Have it create playbooks, examples, troubleshooting guides.
  • Have it refactor code AFTER comprehensive tests.
  • Have it think more. Use ultrathink.
  • Log extensively, by default. Improves future debugging.
  • Report errors well. What happened, why, and what to do.
  • Prefer monorepos for more context. #
  • Prefer popular libraries. LLMs know these better.
  • Prefer fast tests, tools, and libraries. Speed helps iteration.
  • Prefer small files and packages. Reduces context.
  • Prefer simple code. Avoid magic, e.g. pytest fixture injection. Functions over classes. SQL over code. Composition over inheritence.
  • Prefer specialized functions for common scenarios over DRY abstractions. Prefer fewer abstraction layers.
  • Prefer re-implementing over DRY since code is cheap.
  • Look for new tricks to learn from its code.
• Agent behaviors:
  • Simple tasks perform better. More context = more confusion.
  • Verifiable tasks are clearer for LLMs and and easier to review.
  • Useful coding agent tools: bash(cmd), patch(hunks), todo(tasks), web_nav(url), web_eval(script), web_logs(), web_screenshot(), keyword_search(keywords), codereview()
• Skills:
  • LESS Coding
  • LESS Research
  • LESS Documentation
  • LESS Operations configuration (IaaC, CI/CD, etc)
  • LESS Editor usage and expertise required
  • MORE Tests (to test the code)
  • MORE Code reviews (to test the code)
  • MORE Prompting and context creation (to write the code)
  • MORE DevOps (micro-feature deployments, deploy in parallel)
  • MORE Specs: features, requirements, APIs, tests, structure, etc.
  • MORE Analysis: security, performance.
  • MORE Tool design. Linters, SAST, DAST, Performance, etc. Semgrep, Bench Suite
  • MORE Observability: Especially for tools and LLM calls. Telemetry, log analysis and issue creation. Sentry, LogFire, etc.
• Trends:
  • Agents took time to evolve because LLMs need to be good at tool calling and long instruction following, which is just happening.
  • Agents are slow. Parallelizable tools (e.g. multiple Redis instances, container-use, CI/CD) will grow. Tool speed (e.g. fast test engines with caching) will become more important.
  • Agents generate diffs/PRs. Tools to edit and comment on these online will emerge.
  • Context gathering will widen: screenshots, logs, etc.
  • Code review process will be re-invented.
  • Personalized features. User drops a feature request via Slack. Personalized version deployed at their endpoint to test. PR sent after they are happy
  • Poor coding teams get less out of AI coding. Good communication, reviews, coding practices, testing, etc. help.
  • Agent Experience (AX) is emerging and explores: how much context to take, when & how often to ask the user questions, to how make review easier, etc.
  • Humans running multiple tasks in parallel is productive. Breaking a complex requirement into tasks (like Codex now does) helps create that task queue.
  • Agents generate technical debt faster than humans. Solving this will become a major problem/opportunity.
• "makework": made-up work that fills time or serves short-term needs.

• AI is like electricity. It's hard to define what is good for because it is good for so many things, most of them new that never existed before
• If experimentation is cheap, it makes sense to run far more experiments. Rather than think hard about what to prototype, explore how to build many diverse prototypes.
• Prototyping is now very fast but other steps like reliable evaluations for deployment still take time. But the speed of prototyping is putting pressure on other parts of the organization to go faster.
• While large language models and applications were serving human needs so far, increasingly they will serve the needs of AI and other tools.
• Since unstructured data is now more valuable, there will be a growth in data engineering on unstructured data.

• Sam Altman. Chase $0B ideas, not $0M ones. Weird + right > safe + crowded
• Gary Tan. Agency scales. Tools change, people/mindset don’t.
• Andrej Karpathy.
  • Instead of LLM memory to store facts, edit system prompt with general strategies, like the LLM writing a book for itself on how to solve problems.
  • Autonomy slider. Let user pick how far LLM acts by itself. Like the Tesla autopilot levels.
  • Make evals EASY and FAST for humans.

• PCs took 3 years to hit 20% of US population. ChatGPT took 2 years for 40%. But it's a lot cheaper, and a lot less used (0.5-3.5% of work hours). Maybe Gen AI adoption is slower than PCs.
• The jagged edge of capability: some things will become MUCH easier while others don't. The relative mix determines who goes out of a job and which tasks get fully automated. Benchmarks are rare in areas where AI is weak.
• Factory electrification took 40 years - to redesign the layout & process; change the org structure & policies; hiring & training practices. AI diffusion could take as long.
  • Therefore, the ability to re-structure a workflow end-to-end will be an advantage.
• Several areas of low AI capability will improve slowly because the feedback is slow due to safety regulations, human adoption speed, lack of clarity on what is better, slow physical feedback (e.g. growing trees), etc.
• Human intelligence is in the use of technology. AI is one more such technology.
• We know of good system safety controls in complex systems like aircrafts, power grids, engineering, chip design, healthcare, cyber-security, etc. Circuit-breakers, predefined rules, audits & monitors, access control, formal verification, etc.
• Even if everything humans do TODAY is automated, it doesn't mean we won't have work. It just shifts to what we're not doing today.
  • We stopped work 4,000 years ago, with the agricultural revolution. The plant/livestock does all the growing. We just manage them, moving stuff around.
  • We stopped work 400 years ago, with the industrial revolution. Machines do the moving. We just manage them, computing the moves.
  • We stopped work 40 years ago, with the information revolution. Computers do the computation. We just manage them, thinking how.
  • Most future tasks will be managing AI that do the thinking.

This ability to define your desired outcome in crisp, complete terms is one of the most important superpowers of the AI era.

• HubSpot MCP with a vega-lite MCP and a few others could solve many common HubSpot UI tasks.
• DarwinBox MCP, ZenDesk MCP, etc. are emerging.

• Remove redundant tags and attributes (e.g. accessibility, etc.). Explore readability.
• Add a unique gid to each element.
• Add the screenshot WITH a "Set of Marks" - "SoM" (read research paper) highlighting important clickable elements.
• Code output is brittle. Use tools / DSL - e.g. visit_url(url), click_element(text, gid, tagName), etc.

• An agent requires 3 things: a router, tools or skills, and memory.
• Agents are often sequential, but sometimes parallel execution makes sense for independent tasks that you consolidate.
• Always allow LLMs the option of NOT answering a question if there is no good answer.
• Focus prompts on the happy path. Use guard rails for edge cases.
• Here are a few "tools" an agent would need to call:
  • Clarification from user
  • Saving to memory
  • Google search
  • Edit a file introducing SPECIFIC changes
  • Search in codebase using embeddings
  • Run scripts on the shell or in a REPL (Python, Node, etc.)
  • Run code in a new container for isolation
  • Automatically discover, read an API documentation and use it
• Modify environment to enable logging and other system changes.
• When code is cheap, you can explore more ideas and hence design and product management need to approach things differently. We also need to reaching testing completely because it makes very different kinds of mistakes and we don't often have an intuition
• You can have an agent explore all the issues and full request and recent comments against the repository and summarise it for the project manager

• Agents make multiple LLM calls. Errors accumulate. So the quality of the model is key
• What's really critical: data + context + user preference
• Set up evals for subjective responses by collecting signals continuously.
• Create scaffolding for agents where errors don't accumulate. Better yet, make it FIX errors
• UX is critical. We need lots more UX styles

• Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI."
• Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed.
• Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts.
• Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions.
• Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies.
• Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information.
• Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing.
• Meeting notes and action items generator: A system that generates meeting notes and action items.
• O'Reilly auto parts customer service agent: An agent demonstrated using RAG.
• UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

• Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
• Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
  • Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
  • Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
  • Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
  • Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
  • Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
  • Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
  • Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
• Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

• Ask better questions. (Prompt engineering.)
• Create templates or step-by-step instructions. (Chain of Thought.)
• Ask for multiple options and pick from the best options. (Agentic approach?)
• Training. (Fine tuning.)
• Price weaker responses lower. (Stratified model pricing?)

• Innovation is now a path to production. People are able to build 20 prototypes at the cost of one and see which sticks
• Machine learning is much faster. Things that took months now date days. But engineering and evaluations are only slightly faster and have become a bottleneck
• A good analogy to zero shot prompting is to ask a person to write an entire essay without pressing backspace even once
• Andrew scenes to align with the line chain definition of agentic workflow, which is about agents being able to craft their own control flows
• People find it very easy to understand agentic workflows once they read through the code
• Reflection or feedback is a useful agentic pattern
• In multi-agent collaboration, it may be the same underlying model that is acting as different agents. But just like we find it useful for the same CPU to run multiple processes and each application is its own abstraction, agents of useful abstraction

• Automate changelogs for your codebases. Convert past commits into attractive release notes automatically
• AI is going to be the consumer of many tools and logs. Build converters for these
• Speed of validation such as linting, testing, etc. will allow LLMs to iterate faster and WILL become more important

• This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
  • the 2.0 programmers (data labelers) edit and grow the datasets, while
  • a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
• Adaptive UI ideas:
  • Adaptive Fields: Show only required fields based on what the user field so far.
  • Smart Inputs: Dropdowns and auto-complete based on user's context.
  • Smart Themes: Change font size, contrast, theme guessing the user's age and preferences.
  • Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs.
  • Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips.
  • Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing.
  • Smart Charts: Create the right chart that solve the user's question.
• Adaptive Back-end
  • Dynamic APIs: Create endpoints on the fly based on user needs
  • Dynamic Indexing: Create & update indices on the fly based on user needs
  • Dynamic Schema: Create & update schema on the fly based on user needs
  • Dynamic Migration: Migrate to a new database or OS or language as required
  • Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
  • Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
  • Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error
  • Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys.

• Cognition
• Factory
• Codegen

• Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
• Collect feedback: Explicit and implicit

• Improve prompts when teaching prompt engineering.
• Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce.

• Use detailed system prompts, often containing the entire instruction set, if it won't change over the course of a conversation.
• "... summary of the prior conversation could be included as part of the system message" is an interesting history compression tactic.

• Just having AI is no longer a differentiator
• AI is part of life, not just work
• Explore quickdrop from Stability for Maruti
• Explore Codium VS Code plugin and Continue.dev
• Hybrid systems that combine stats, ML, DL and AI models will grow
• AGI and AutoGPT resurgence
• RAG will continue to be a focus
• GPT4 will be beaten by open source models. Special purpose models beat it already
• Self hosted and cloud hosted models will grow for security
• Small language models will grow
• Productivity will be enhanced rather than replaced
• Multi modal models will grow
• Cost efficiency will grow in focus