#prompt-engineering

• Wed, Apr 2, 2025. Learnings on AI code editors via Deep Research from ChatGPT, Gemini, Grok, Perplexity: #ai-coding-tools #code-agents #github #markdown #prompt-engineering
  • GitHub Copilot can identify the source of a code snippet as a repo. That helps with copyright issues.
  • Cursor uses a shadow workspace - a temporary sandbox where it edits files before applying changes at one shot.
  • Cursor auto-complete has context of other files, i.e. inserting an class in a .js file based on another HTML file's contents.
  • Windsurf seems to be best for large code bases and for large-scale refactoring. It can also run test results fix them.
  • Windsurf includes a browser and lets you click on an element and prompt to change its behavior, etc. That's good for front-end developers.
  • Roo Code can run scripts as part of the workflow, letting you run linting, tests, starting web apps, query databases, etc.
  • Roo Code lets you create persona, e.g. code reviewer, data storytelling and analysis, etc. with access to different tools and behaviors.
  • Roo Code does not support auto-complete.

• Sat, Mar 29, 2025. Discussion with Vedang #ai-coding-tools #code-agents #dev #github #llm-ops #markdown #prompt-engineering
  • Recurse center (Brooklyn, online) is a 6/12 week free self-driven programmer retreat. Runs every 6 weeks. You can do whatever you pick. There are daily standups for accountability. The groups are diverse. You can pair with them, pivot ideas, whatever. Principles: push yourself & learn.
  • Western education techniques (e.g. spaced repetition, adaptive learning) are very much present in Indian coaching systems, though not known by those names. However, interventions are hard since class 12 students just don't have enough time.
  • Coaching classes are a social phenomenon. It's not the smart students who are pulling in their friends. Smart students actually follow the popular students. (Coaching classes are below the typical smart students' standards.)
  • Monetizing coaching is hard. People don't want to pay for advice, and welcome free advice only if they ask for it. Coupling with execution is necessary.
  • Aider's integrations make it more powerful than Cursor/Windsurf. It auto-lints, runs test cases. Allows different models for "architecting" (generating changes) vs "editing" (applying code). It reads from the screen logs. Context is manual, not automated. Uses an ai! comment to trigger changes and ai? to ask questions.
  • Cline.bot is another Cursor-like open source AI code editor that's a VS Code plugin.
  • When coding with LLMs, a useful workflow is: data schema ➡️ interfaces ➡️ LLM-generated test cases ➡️ code.
  • ShellSage is a tmux based LLM tool for the command line. It screen-grabs from tmux, which is powerful.
  • Some MCPs that have proven useful: vega-lite, SQLite, sequential thinking, memory
  • make sucks but is hard to beat. just comes closest.
  • CRDTs are more powerful than for just collaborative editing. It can power a peer-to-peer Internet (beginning with office tools). Versioning schema is still problematic. yjs is a good start but automerge (Rust, WASM) is faster and may be better.
  • Fermyon hosts WASM serverless functions.

• Sat, Mar 29, 2025. Reflecting on Satya Nadella's "SaaS is dead", building or porting apps' functionality into classic chatbots (e.g. via MCPs) would be an emerging market. E.g. "Create a HubSpot MCP. Do whatever you want on HubSpot, except via ChatGPT or your favorite LLM chatbot." To be fair, such interfaces exist. #automation #prompt-engineering
  • HubSpot MCP with a vega-lite MCP and a few others could solve many common HubSpot UI tasks.
  • DarwinBox MCP, ZenDesk MCP, etc. are emerging.

• Tue, Mar 25, 2025. Cursor with Claude 3.7 Max seems surprisingly good at generating multi-page sites at one shot. Potentially, it can edit large repositories of code as well at one shot. If that's the case, the way we write code will require higher order thinking skills: broad sweeping changes rather than micro edits. #ai-coding-tools #code-agents #dev #github #html #markdown #prompt-engineering #web-dev

• Sat, Mar 15, 2025. htmlq and pup query HTML. They're like jq for HTML. #github #html #markdown #prompt-engineering #web-dev

• Sat, Mar 15, 2025. Here are time-tested and robust ways to leverage serendipity: ChatGPT #best-practices #prompt-engineering
  1. Place. Be in places with high, diverse, talent density. Bell Labs (1950s), MIT (1970s), Pixar (1990s).
  2. People. Meet diverse, talented people. Da Vinci's Renaissance circles, Lockheed Martin's Skunk Works.
  3. Free time for unstructured work. 3M's 15% rule, Google's 20% time, Edison's Invention Factory.
  4. Curiosity. Learn unrelated fields. Darwin's earthworm research, Ben Franklin's ocean currents work.
  5. Serendipity. Systematically add randomness. Brian Eno's Oblique Strategies, IDEO's Deep Dives.
  6. Reframe failure as opportunities. Penicillin, Velcro, Post-it Notes.
  7. Ceremonies. Hackathons, lightning talks, coffee trials.

• Fri, Mar 14, 2025. Playwright seems to be the emerging standard for modern browser testing/automation, beating Cypress and Selenium. #future #html #prompt-engineering

• Fri, Mar 14, 2025. Unsloth might be one of the simplest ways of fine-tuning. #automation #markdown #optimization #prompt-engineering

• Fri, Mar 14, 2025. Apify is a cloud scraper platform. Here's how they optimize their AI Web agent - Source: #ai-coding-tools #automation #code-agents #html #optimization #prompt-engineering #web-dev
  • Remove redundant tags and attributes (e.g. accessibility, etc.). Explore readability.
  • Add a unique gid to each element.
  • Add the screenshot WITH a "Set of Marks" - "SoM" (read research paper) highlighting important clickable elements.
  • Code output is brittle. Use tools / DSL - e.g. visit_url(url), click_element(text, gid, tagName), etc.

• Tue, Mar 11, 2025. Bots need structured content (e.g. Markdown, XML). Humans need rich content (e.g. HTML). Here are 4 ways to serve both, roughly in increasing order of sophistication: #code-agents #html #markdown #prompt-engineering
  1. Different URLs. E.g. https://example.org/about/ vs https://example.org/about.md (this is how Jekyll or Hugo work). Use for static sites generators.
  2. JavaScript. Inject after Markdown: <script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script><script>document.body.innerHTML = marked(document.body.textContent);</script>. Use for dynamically generated static sites.
  3. URL query parameters. E.g. ?format=markdown vs ?format=html vs ?format=json. Use in APIs.
  4. Content Negotiation. Based on the user agent and Accept header, serve Markdown or HTML. Send Vary: Accept to indicate that the response depends on the Accept header. Use for dynamic web apps.

• Sat, Mar 8, 2025. "Export to prompt" can be a useful feature in apps (or even as a bookmarklet). It would let you export content in an LLM-friendly Markdown format. You can paste it into an LLM and ask questions. Here are things I would find useful: #document-conversion #github #markdown #prompt-engineering
  • Copy an entire issue (with history) from GitHub, Gitlab, or JIRA
  • Copy an entire PR (with code changes) from GitHub, Gitlab, or Bitbucket
  • Copy CI/CD logs from GitHub Actions, Gitlab CI, Azure DevOps, etc.
  • Copy entire conversation thread in Gmail or Discourse, Service now etc.
  • Copy product reviews from Amazon, Shopify, etc.
  • Copy page(s) from wikis and content sites like Wikipedia, StackOverflow, etc.
  • Copy survey responses from Google Forms, Typeform, etc.
  • Copy all interactions with a contact (including interactions, proposal history) from HubSpot or Salesforce
  • Copy transcripts from Zoom, Teams, Google Meet, etc.
  • Copy as Markdown from Word, GDocs, PDF or HTML
  • Copy the summary of an analysis as well as all key metrics from any dashboard
  • Copy SAP invoices
  • Copy JDs, CVs, and reviews from Workday, BambooHR, DarwinBox, etc.
  • Copy design specs, component libraries, and style guides from Figma, Miro, etc.
  • Generated with the help of ChatGPT -- link not working

• Tue, Mar 4, 2025. Tools and solutions should fit within existing workflows. That means almost all capabilities need to be exposed as APIs. #automation #future #prompt-engineering

• Mon, Mar 3, 2025. Notes from AI Engineering Summit, NY, Day 1 #ai-coding-tools #automation #code-agents #prompt-engineering
  • An agent requires 3 things: a router, tools or skills, and memory.
  • Agents are often sequential, but sometimes parallel execution makes sense for independent tasks that you consolidate.
  • Always allow LLMs the option of NOT answering a question if there is no good answer.
  • Focus prompts on the happy path. Use guard rails for edge cases.
  • Here are a few "tools" an agent would need to call:
    • Clarification from user
    • Saving to memory
    • Google search
    • Edit a file introducing SPECIFIC changes
    • Search in codebase using embeddings
    • Run scripts on the shell or in a REPL (Python, Node, etc.)
    • Run code in a new container for isolation
    • Automatically discover, read an API documentation and use it
  • Modify environment to enable logging and other system changes.
  • When code is cheap, you can explore more ideas and hence design and product management need to approach things differently. We also need to reaching testing completely because it makes very different kinds of mistakes and we don't often have an intuition
  • You can have an agent explore all the issues and full request and recent comments against the repository and summarise it for the project manager

• Mon, Mar 3, 2025. Notes from AI Engineering Summit, NY. Session by Lux Capital. #automation #code-agents #llm-ops #optimization #prompt-engineering
  • Agents make multiple LLM calls. Errors accumulate. So the quality of the model is key
  • What's really critical: data + context + user preference
  • Set up evals for subjective responses by collecting signals continuously.
  • Create scaffolding for agents where errors don't accumulate. Better yet, make it FIX errors
  • UX is critical. We need lots more UX styles

• Sun, Feb 16, 2025. Interview Coder is a desktop app that offers live interview support for coding interviews. It's a transparent window that reads your screen and answers questions for you. (Given this, I think we need an interviewer support system that tells interviewers what to ask!) #ai-coding-tools #code-agents #prompt-engineering

• Wed, Feb 5, 2025. ChatGPT does a good job of spotting errors in architectural and structural diagrams. In fact, the whole theme of spotting errors in large diagram is a theme that can have potential use cases. Source: Dan Becker. #chatgpt #future #prompt-engineering

• Mon, Feb 3, 2025. OpenAI advices a few different prompting techniques for reasoning models. OpenAI: #prompt-engineering
  • Avoid examples unless zero-shot prompting fails.
  • Avoid chain-of-thought. These models do that internally anyway.
  • Short, direct prompts are better than detailed prompts.

• Sat, Jan 11, 2025. Can LLMs write better code if you keep asking them to “write better code? A delightful exploration of how Claude 3.5 Sonnet keeps optimizing and adding features to improve code. My takeaway: repeatedly applying a prompt gives us interesting new directions to explore. #ai-coding-tools #code-agents #llm-ops #markdown #optimization #prompt-engineering

• Sat, Jan 4, 2025. mitmproxy is an open source tool to intercept, modify, and replay HTTP requests. An alternative to Charles, Fiddler, and partly WireShark. Guide. Like the others, it requires installing a trusted root certificate on your machine. #ai-coding-tools #prompt-engineering
  • mitmproxy2swagger digs through the mitmproxy flows and generates an OpenAPI schema. A clever idea to reverse-engineer APIs.

• Fri, Dec 27, 2024. Satya Nadella prepares for meetings by asking Copilot to tell him everything he needs to know about the client from the CRM, emails, meeting transcripts etc. He shares that colleagues who annotate it further for him. That's using AI for reasoning and collaborating with colleagues. 55m #automation #future #prompt-engineering

• Thu, Dec 26, 2024. Learnings from Best of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands) #ai-coding-tools #code-agents #dev #github #prompt-engineering
  • Passing code execution as a tool is more powerful than granular tools. You combine multiple tools and tool calls into one. You move code to the data rather than the other way around. Mostly, you need bash, Python (or Jupyter), file manager, web browser.
  • UI: Go where the user is, instead of bringing them to you.
  • A remote runtime is a critical component.
  • Claude 3.5 Sonnet (20241022) and Claude 3.5 Haiku (20241022) perform best on SWE Bench, followed by Deepseek V3, then O1 2024-12-17. X

• Sun, Dec 22, 2024. How are software engineers are future-proofing their careers in the face of LLMs? #llm-ops #prompt-engineering
  • Leveraging LLMs as Force Multipliers
    • Use LLMs for repetitive tasks, rapid prototyping, exploring multiple approaches, data extraction and brainstorming, providing feedback.
    • Explore prompting techniques, integrate LLMs into their workflows, and develop strategies for validating and refining LLM-generated code
  • Focusing on higher-level skills that llms struggle with
    • Systems Thinking and Architecture: code readability, extensibility, testability, and maintainability
    • Problem Solving and Critical Thinking: define problems clearly, break them down into manageable parts, and reason through complex scenarios. LLMs produce plausibly incorrect code.
    • Communication and Collaboration
    • Domain Expertise
  • Exploring Adjacent Roles: product management, technical leadership, or consulting. Involve more interaction with clients and stakeholders.
  • Developing "Evergreen" Skills: debugging, system administration, and security. Or outside of software engineering, such as trades or other hands-on vocations.
  • Scepticism: LLMs may not reach a level of sophistication that would render their expertise obsolete. Complex problems, understanding context, and producing high-quality, maintainable code.

• Sun, Dec 22, 2024. Examples of agentic AI #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
  • Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI."
  • Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed.
  • Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts.
  • Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions.
  • Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies.
  • Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information.
  • Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing.
  • Meeting notes and action items generator: A system that generates meeting notes and action items.
  • O'Reilly auto parts customer service agent: An agent demonstrated using RAG.
  • UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

• Sat, Dec 21, 2024. Anthropic defines agents. Building effective agents + Cookbook #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
  • Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
  • Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
    • Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
    • Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
    • Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
    • Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
    • Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
    • Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
    • Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
  • Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

• Thu, Dec 12, 2024. I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: #automation #best-practices #llm-ops #optimization #prompt-engineering
  • Ask better questions. (Prompt engineering.)
  • Create templates or step-by-step instructions. (Chain of Thought.)
  • Ask for multiple options and pick from the best options. (Agentic approach?)
  • Training. (Fine tuning.)
  • Price weaker responses lower. (Stratified model pricing?)

• Wed, Nov 27, 2024. Andrew Ng Explores The Rise Of Al Agents And Agentic Reasoning | BUILD 2024 Keynote #automation #code-agents #prompt-engineering
  • Innovation is now a path to production. People are able to build 20 prototypes at the cost of one and see which sticks
  • Machine learning is much faster. Things that took months now date days. But engineering and evaluations are only slightly faster and have become a bottleneck
  • A good analogy to zero shot prompting is to ask a person to write an entire essay without pressing backspace even once
  • Andrew scenes to align with the line chain definition of agentic workflow, which is about agents being able to craft their own control flows
  • People find it very easy to understand agentic workflows once they read through the code
  • Reflection or feedback is a useful agentic pattern
  • In multi-agent collaboration, it may be the same underlying model that is acting as different agents. But just like we find it useful for the same CPU to run multiple processes and each application is its own abstraction, agents of useful abstraction

• Sat, Nov 23, 2024. OpenAI lets you download GPT instructions and execute arbitrary code in their containerized environment. This is not a bug. Ref #ai-coding-tools #chatgpt #code-agents #github #prompt-engineering

• Fri, Nov 22, 2024. For good structured mode output, we need good prompting. #ai-coding-tools #code-agents #markdown #prompt-engineering
  • Mentioning examples and schema and "JSON" helps. When providing examples, using [user, assistant] message pairs helps (I think it's because it's easier for the LLM to parse).
  • Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv
  • We already know code in JSON is not a great idea. Ref

• Fri, Nov 22, 2024. Just adding 3 real examples and regurgitation helped GPT 4o play chess much better. Both techniques may have more general use in prompting. Simon Willison #prompt-engineering

• Sat, Nov 9, 2024. Google counts auto-completion when saying "25% of all the code is written by AI at Google". "It's a helpful productivity tool but it's not doing any engineering at all. It's probably about as good, maybe slightly worse, than Copilot." YCombinator #ai-coding-tools #code-agents #prompt-engineering

• Mon, Nov 4, 2024. Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. #best-practices #dev #llm-ops #prompt-engineering
  • Get THE domain expert (or approver) as the tester.
  • Create a dataset that is DIVERSE.
  • Covers EACH combination of:
    • Features
    • Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error
    • Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user
  • Generate data using existing data + synthetic data for each SPECIFIC combination of the above
  • Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include:
    • Nuances: Something a failed response did well or a passed response didn't quite do well
    • Improvements: Suggest how model can improve
  • Build an SPA to make it easy for the domain expert to review

• Mon, Oct 28, 2024. GH CLI is quite good for deployment too, like Wrangler CLI. Enabling pages, setting secrets, etc. #dev #github #prompt-engineering

• Mon, Oct 28, 2024. Pragmatic Podcast (I forgot which one) #ai-coding-tools #automation #code-agents #future #github #prompt-engineering
  • Automate changelogs for your codebases. Convert past commits into attractive release notes automatically
  • AI is going to be the consumer of many tools and logs. Build converters for these
  • Speed of validation such as linting, testing, etc. will allow LLMs to iterate faster and WILL become more important

• Mon, Oct 14, 2024. How AI can improve education performance and engagement. Ref #ai-coding-tools #future #prompt-engineering
  • Student: Create study plan based on course and schedule
  • Student: Focus on what you need to learn more
  • Student: Align with your study style and pace
  • Teacher: Grading MCQs
  • Teacher: Writing conceptual guides
  • "New collar workers" was coined by Ginny Rometty
  • Embed tutor or document in video and ask for clarification! This is a new embedded interface. #TODO

• Sun, Oct 13, 2024. OpenAI has a prompt generator. Currently it uses a meta-prompt but may later move to DSPy or Gradient Descent. Ref #prompt-engineering

• Wed, Oct 9, 2024. O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution #future #llm-ops #optimization #prompt-engineering

• Sun, Oct 6, 2024. Parables on the Power of Planning in AI: Giving models about 30 seconds of thinking time consistently improves results - as much as increasing parameter size by a factor of 1,000 to 100,000! #best-practices #optimization #prompt-engineering
  • This works particularly well for verifiable results (code, math, etc.)
  • Technique: Ask an LLM hundreds of times at low temperature and pick the most common one. (Google's Minerva used this on the MATH dataset.)
  • Better Technique: Ask an LLM hundreds of times. Pick the best solution based on an evaluation metric (reward model)
  • Better Technique: Apply a reward model at EACH step of the process. OpenAI's "Let's Verify Step by Step"

• Sun, Oct 6, 2024. Continue.dev is another AI coding editor. It supports OpenRouter. So now I have heard good things about: #ai-coding-tools #code-agents #dev #github #prompt-engineering
  • Github Copilot
  • Cursor
  • Cody
  • Continue.dev (supports OpenRouter)
  • Aider (supports OpenRouter)
  • Maybe:
    • Codeium
  • Not:
    • Amazon Q Developer

• Fri, Oct 4, 2024. Software 2.0 by Andrej Karpathy. #ai-coding-tools #automation #dev #markdown #prompt-engineering
  • This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
    • the 2.0 programmers (data labelers) edit and grow the datasets, while
    • a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
  • Adaptive UI ideas:
    • Adaptive Fields: Show only required fields based on what the user field so far.
    • Smart Inputs: Dropdowns and auto-complete based on user's context.
    • Smart Themes: Change font size, contrast, theme guessing the user's age and preferences.
    • Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs.
    • Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips.
    • Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing.
    • Smart Charts: Create the right chart that solve the user's question.
  • Adaptive Back-end
    • Dynamic APIs: Create endpoints on the fly based on user needs
    • Dynamic Indexing: Create & update indices on the fly based on user needs
    • Dynamic Schema: Create & update schema on the fly based on user needs
    • Dynamic Migration: Migrate to a new database or OS or language as required
    • Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
    • Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
    • Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error
    • Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys.

• Sat, Sep 28, 2024. API coding best practices Source via Simon Willison: #ai-coding-tools #best-practices #github #prompt-engineering
  • Always add screenshots to the Readme. They never break.
  • Always add every example. Human think in examples.
  • Avoid defaults and be explicit unless 99% of the usage is with the default.
  • Make the feedback loops incredibly fast.
  • Make deprecations easy for users to deal with.
  • Keep objects immutable.

• Sun, Sep 22, 2024. Sentient lets you control the browser via Python in natural language #prompt-engineering #python #speech-to-text

• Fri, Sep 20, 2024. <link rel="modulepreload"> lets you load and compile modules early! #future #html #prompt-engineering

• Fri, Sep 20, 2024. Prompt engineering for code generators: #ai-coding-tools #code-agents #dev #future #markdown #prompt-engineering
  • Claude Artifacts Prompt
  • Val.Townie system prompt. Good example of how to create
  • Cursor editing
  • Cursor debugging
  • Cursor conversation

• Fri, Sep 20, 2024. XML tags seem best to structure prompts across LLMs. #ai-coding-tools #code-agents #llm-ops #markdown #prompt-engineering
  • Claude
  • OpenAI
  • Gemini

• Sun, Sep 15, 2024. Prompt engineering tip: Tell LLMs another AI wrote code. Else they will agree with you! #ai-coding-tools #automation #code-agents #future #llm-ops #prompt-engineering

• Tue, Sep 10, 2024. Simon Willison's crowdsourced list of prompt engineering hacks #future #prompt-engineering

• Tue, Aug 27, 2024. Cursor uses speculative edits and a variety of other techniques to speed up code editing. #ai-coding-tools #future #markdown #prompt-engineering

• Mon, Aug 19, 2024. Copilot's system prompt calls a search_enterprise(query: str) tool and a hint(M365Copilot_language: str) tool as assistants. #code-agents #prompt-engineering

• Mon, Aug 19, 2024. Anthropic Prompt Caching is 90% cheaper to use and 25% costlier to create. So if there's a 27% chance it'll be re-used, cache it. #optimization #prompt-engineering

• Mon, Aug 12, 2024. Code agent frameworks to explore: #ai-coding-tools #automation #code-agents #dev #future #github #markdown #prompt-engineering
  • Cognition
  • Factory
  • Codegen

• Sat, Aug 10, 2024. Embedding models can be fine-tuned. Example: #TODO #embeddings #future #gpu #markdown #models #optimization #prompt-engineering

• Wed, Aug 7, 2024. Notes from a 4-hour flight: #ai-coding-tools #llm-ops #prompt-engineering
  • What We’ve Learned From A Year of Building with LLMs
    • Strategy
      • IS IT TOO HARD/EXPENSIVE? Log it. LLMs are getting cheaper and better.
      • WILL OPENAI BUILD IT? If so, wait for it instead of building.
      • HAS A STARTUP BUILT IT? If so, use it instead. It's a generic use case there's no point re-inventing.
      • FOCUSED USE CASES over generic. Build trust by starting small.
      • Tools for LLM Ops (feedback): LangSmith, Log10, LangFuse, W&B Weave, HoneyHive #TRY
      • Human in the Loop is about humans evaluating model outputs. That's different from AI in the loop, human in the center, where AI accelerates human output (like Github Copilot)
    • Operations
      • CHECK EMBEDDINGS DRIFT over time. Users might be input-ing different things than before.
      • LOG AND REVIEW everything.
      • Instructor coaxes structured output from LLM APIs. #TRY
      • IMPLICIT FEEDBACK collection is easy. Just let users edit stuff. #TRY
    • Tactical
      • Try n-shot prompting (n=5-12) before bigger models. #TRY
      • Always structure for output: Markdown, XML/HTML tags.
      • Combine RAG with Keyword search. It reduces user frustration in edge cases.
      • Prefer multiple small prompts to one big prompt. Do X. Then Y. Then Z.
      • Jitter prompts for diversity beyond temperature.
      • LLM-as-judge works better when comparing outputs (not rating 1 output). Keep length similar (LLMs prefer wordiness). Swap order and compare. Allow for ties. Ask for reason FIRST.
  • Hermes: A Text-to-SQL solution at Swiggy
    • "Hermes performed significantly better for charters with well-defined metadata and a relatively smaller number of tables."
    • "We collect feedback on the accuracy of the returned query from stakeholders directly within the Slack bot."
  • How I use AI and "Replacing my right hand with AI"
    • EMBED in every app/workflow. E.g. Auto-fix spellings. Auto-review code. Auto-ask LLM on errors and apply patch! Auto-search for answer, assess, continue.
    • PERSIST. Stick with the LLM to the end. Don't fix it yourself. It's faster. #TRY
      • INTERVENE FAST. If an LLM can't solve it by itself in 2 tries, it needs in-depth help.
    • APP-IFY one-off tasks. Disposable tools. "Write web-app to convert JSON to tab-delimited." "Extract fields as a table." "Diff JSON." #TRY
    • BEST language/frameworks preferred. CUDA in Python. Rust. C. Raspberry Pi. Arduino. Bluetooth. Modern ESM/JS. #TRY
    • TEACH examples. "Here's the LLM Foundry API." "Here's how to use gramex.data."
    • DUMP entire code. Models can handle it. Refactoring to SQLAlchemy 2, Pandas 2. API Documentation. Test case generation. #TRY
    • ASK for features & packages. Docker without root access. GPU access inside docker. Windows CLI-only C++ compiler.
    • TEST CASE writing. #TRY
    • SPEC IN DETAIL. Use these libraries. Write like this: code example.
    • SPEC USAGE in detail.
      • "I will just pipe it into sqlite", or "I will just run ffmpeg -i filename [YOUR OPTIONS].
      • Describe the UI, API input/output, data structure, and internal data structure.
    • HELP on usage. "ffmpeg to get audio.mp3".
  • My benchmark for large language models
    • LLM(text) is a useful function to have in JS and Python too. Useful as a simple pip install llmfoundry
    • Allow images, files in LLM()
  • Current list of #IMPOSSIBLE (or hard) things for LLMs
    • Translate technical documents to Dutch -- because they don't understand the technical terms well
    • Translate large documents (JSON to XML, English to Chinese, Python to Rust, Wrong to right spelling) -- because the output tokens are limited

• Sun, Jul 14, 2024. Github Copilot internals walks through how Copilot constructs its prompts #code-agents #future #prompt-engineering

• Tue, Jun 25, 2024. LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: #ai-coding-tools #automation #best-practices #future #llm-ops #optimization #prompt-engineering
  • Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
  • Collect feedback: Explicit and implicit

• Thu, Apr 18, 2024. Effort engine introduces "effort" as a parametrizable way to speed up LLMs with a quality trade-off. Works on Mistral for now. #gpu #llm-ops #optimization #prompt-engineering

• Sat, Mar 30, 2024. TechEmpower web server benchmarks place Rust servers on top #hosting #prompt-engineering #server #web-dev

• Sat, Mar 30, 2024. browser.new is a good example of a browser agent. It slowly but independently does a good job of achieving the result. Example: What crew is common in Ingrid Bergman - Cary Grant films? #code-agents #prompt-engineering

• Wed, Mar 27, 2024. Representational engineering lets you control LLM output based on preference on the fly. #future #llm-ops #markdown #models #prompt-engineering

• Tue, Mar 19, 2024. Ways to use inversion #best-practices #dev #future #prompt-engineering
  • "Pre-mortem" is an analysis at the beginning of how a project failed. Then avoid that
  • "Red team" or "Black hat" are designated to contradict.

• Tue, Mar 19, 2024. Control vectors in real-time lets you control response in real-time #automation #future #prompt-engineering

• Sun, Mar 3, 2024. Making a tool more usable, e.g. a video, can have a 10-100X impact. Yet every developer thinks it's redundant #ai-coding-tools #prompt-engineering #web-dev

• Sat, Mar 2, 2024. You can use slots to stream HTML out of order! #future #html #prompt-engineering #web-dev

• Fri, Feb 23, 2024. Teknoturf is using Gen AI to #ai-coding-tools #automation #prompt-engineering #speech-to-text #tts
  • Improve prompts when teaching prompt engineering.
  • Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce.

• Sat, Feb 17, 2024. Fine tuning makes economic sense only if the input tokens SAVED is twice the output token size on each call. #future #optimization #prompt-engineering

• Sat, Feb 17, 2024. Discussion with Karthik A #ai-coding-tools #github #prompt-engineering
  • #TODO Ask IIT students to do internship tasks. Use advent of code is a qualifying criterion
  • Tata motors unionized DB admins for longevity. No one can take their jobs. Hires people who LIKE their jobs
  • Rust gives me typing. It's very efficient. Pola.rs is interesting but Pandas as good enough.
  • Explore alerts from CCTV feeds. Karthik sends email alerts with pictures for:
    • "Is the machine on or off"? for productivity
    • "Are people not wearing helmets?" for safety at Cummins
  • #TODO Integrate with WhatsApp. Use LLMs with function calling for responses
  • Use expiring links (to pictures or content). It increases engagement
  • Check Deno licensing. Is there a commercial clause? #ANS No - it's MIT license
  • Centre or excellence for zero emission tech at IIT. Karthik is part of it
  • Explore auth0. 7000 users are free
  • toml is part of the Python 3.11 standard library!
  • If copilot writes code we don't understand we are screwed. Hence expertise matters

• Sun, Feb 11, 2024. Two pass generation is a clever technique to get multiple SEQUENTIAL answers in a single API request. For example the schema {'code', 'optimized_code'} will generate code and then optimize it. #ai-coding-tools #code-agents #future #optimization #prompt-engineering

• Tue, Jan 30, 2024. Hidden brain podcast. Making the most of your mistakes #prompt-engineering
  • FIX every small mistake. You never know how they might line up in the future
  • You also never know how small little things done well might line up to give you a boost in the future
  • The Toyota cord does not actually stop the production line. It brings a team lead over who quickly diagnoses the problem with you. The responsiveness of the league is a critical factor and so is encouragement
  • That isn't always a single bottleneck to stop that is the case of a simple failure. There can be a series of holes that happen to align perfectly.
  • These are events that lead to catastrophic failures or successes
  • Do as little as possible, waste as little as possible, until you know that the outcome is worthwhile.
  • Figure out what is the value of the outcome and the most important piece of information you need to discover that
  • Do full research before you try and fail. The aim of failure is learning at the least possible cost

• Sat, Jan 27, 2024. ⭐ OpenAI's prompt engineering strategies are an excellent start for prompt engineering. A few lessons: #automation #prompt-engineering
  • Use detailed system prompts, often containing the entire instruction set, if it won't change over the course of a conversation.
  • "... summary of the prior conversation could be included as part of the system message" is an interesting history compression tactic.

• Sat, Dec 16, 2023. Github Copilot's new features makes your entire workspace or a specific file its context. It also auto-writes your commit messages and PR descriptions. #github #markdown #prompt-engineering

• Thu, Dec 7, 2023. 12 Tuning Strategies for RAG #markdown #optimization #prompt-engineering

• Thu, Nov 30, 2023. When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref #ai-coding-tools #automation #code-agents #future #llm-ops #optimization #prompt-engineering

• Sat, Nov 25, 2023. Agents mimic programming primitives. Switch. While. Call a function. Print. #ai-coding-tools #automation #code-agents #prompt-engineering

• Sun, Nov 19, 2023. Each of the GPTs / Prompts we create could be like a UNIX command prompt, and become a startup of its own #chatgpt #future #prompt-engineering

• Thu, Oct 26, 2023. 26 Oct 2023: ChatGPT AutoExpert is a great prompt mechanism for higher accuracy, context and control. Example #ai-coding-tools #automation #chatgpt #code-agents #prompt-engineering #speech-to-text #tts