#dev | Anand S - Things I Learned

Thu, Dec 26, 2024. Learnings from Best of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands) #ai-coding-tools #code-agents #dev #github #prompt-engineering
- Passing code execution as a tool is more powerful than granular tools. You combine multiple tools and tool calls into one. You move code to the data rather than the other way around. Mostly, you need bash, Python (or Jupyter), file manager, web browser.
- UI: Go where the user is, instead of bringing them to you.
- A remote runtime is a critical component.
- Claude 3.5 Sonnet (20241022) and Claude 3.5 Haiku (20241022) perform best on SWE Bench, followed by Deepseek V3, then O1 2024-12-17. X

Mon, Dec 23, 2024. OpenAI introduced a developer role in addition to the system role. This is mainly for o1. The API is backward compatible - and also forward compatible. OpenAI #dev

Mon, Nov 4, 2024. Zed.dev is an AI code editor by the creators of Atom. It's written in Rust and is blazing fast. It has native AI integration. #ai-coding-tools #code-agents #dev #github #markdown

Mon, Nov 4, 2024. Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. #best-practices #dev #llm-ops #prompt-engineering
- Get THE domain expert (or approver) as the tester.
- Create a dataset that is DIVERSE.
- Covers EACH combination of:
  - Features
  - Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error
  - Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user
- Generate data using existing data + synthetic data for each SPECIFIC combination of the above
- Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include:
  - Nuances: Something a failed response did well or a passed response didn't quite do well
  - Improvements: Suggest how model can improve
- Build an SPA to make it easy for the domain expert to review

Mon, Oct 28, 2024. GH CLI is quite good for deployment too, like Wrangler CLI. Enabling pages, setting secrets, etc. #dev #github #prompt-engineering

Sun, Oct 6, 2024. Continue.dev is another AI coding editor. It supports OpenRouter. So now I have heard good things about: #ai-coding-tools #code-agents #dev #github #prompt-engineering
- Github Copilot
- Cursor
- Cody
- Continue.dev (supports OpenRouter)
- Aider (supports OpenRouter)
- Maybe:
  - Codeium
- Not:
  - Amazon Q Developer

Fri, Oct 4, 2024. Software 2.0 by Andrej Karpathy. #ai-coding-tools #automation #dev #markdown #prompt-engineering
- This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
  - the 2.0 programmers (data labelers) edit and grow the datasets, while
  - a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
- Adaptive UI ideas:
  - Adaptive Fields: Show only required fields based on what the user field so far.
  - Smart Inputs: Dropdowns and auto-complete based on user's context.
  - Smart Themes: Change font size, contrast, theme guessing the user's age and preferences.
  - Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs.
  - Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips.
  - Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing.
  - Smart Charts: Create the right chart that solve the user's question.
- Adaptive Back-end
  - Dynamic APIs: Create endpoints on the fly based on user needs
  - Dynamic Indexing: Create & update indices on the fly based on user needs
  - Dynamic Schema: Create & update schema on the fly based on user needs
  - Dynamic Migration: Migrate to a new database or OS or language as required
  - Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
  - Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
  - Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error
  - Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys.

Thu, Oct 3, 2024. Devika is an open-source clone of Devin. #dev

Wed, Oct 2, 2024. DuckDB runs inside Pyodide #dev

Mon, Sep 30, 2024. Coders are more likely to edit LLM code. Non-coders don't have that bad habit. #ai-coding-tools #code-agents #dev #github #llm-ops #markdown
- Vaishnavi and Ranjeet edited code
- Indal and Koustav didn't

Fri, Sep 20, 2024. Prompt engineering for code generators: #ai-coding-tools #code-agents #dev #future #markdown #prompt-engineering
- Claude Artifacts Prompt
- Val.Townie system prompt. Good example of how to create
- Cursor editing
- Cursor debugging
- Cursor conversation

Sat, Aug 31, 2024. Val.town's code generation system prompt uses https://maxm-imggenurl.web.val.run/the-description-of-your-image to dynamically generate images #ai-coding-tools #dev #image-generation

Mon, Aug 12, 2024. Code agent frameworks to explore: #ai-coding-tools #automation #code-agents #dev #future #github #markdown #prompt-engineering

Wed, Apr 17, 2024. Tests and homework have a purpose. It helps people figure out whether they've learnt. So: #ai-coding-tools #best-practices #dev #future #github
- Write tests that make people think! Like DuckDB workshop
- Share a list of exercises that people can explore

Sat, Mar 30, 2024. twinny is an open source VC Code Copilot alternative. #ai-coding-tools #dev #future #github

Tue, Mar 19, 2024. Ways to use inversion #best-practices #dev #future #prompt-engineering
- "Pre-mortem" is an analysis at the beginning of how a project failed. Then avoid that
- "Red team" or "Black hat" are designated to contradict.

Sun, Feb 25, 2024. demucs does an EXCELLENT job of splitting songs into drums, bass, vocals and others #ai-coding-tools #dev #markdown

Mon, Feb 19, 2024. Command line interface guidelines #ai-coding-tools #dev

Sat, Feb 17, 2024. Discussion with Vinu Yamunan #dev
- Databuck by FirstEigen. Autolysis plus monitoring
- Quality council has the data steward (maintainer of each dataset) coming together with the uses on a weekly basis to understand what quality problems to users are facing. Data owners jaundice at a lower frequency to get an understanding
- #TODO Automate rules for data quality in our projects and intranet
- Convert a config rule into business language. Explain SQL. These are good use cases for llm's
- Graph DBs are powerful for flexible data structures, but query generation needs AI or expertise. Check the Neo4J language cypher
- Explore storing SAME data in relational DBs AND in graph DBs / document DBs for different use cases
- Dallas rocketry challenge. Build a rocket that can take an egg to 800 feet exactly and land without breaking it

Mon, Jan 29, 2024. Vanna is a SQL generation LLM. An alternative to SQLCoder. This thread has a detailed discussion on SQL generation and BI #ai-coding-tools #dev

Sun, Dec 10, 2023. v0.dev creates web pages from code. Example. #dev #future #github #html #markdown #web-dev