#llm-ops

• Sat, Mar 29, 2025. Discussion with Vedang #ai-coding-tools #code-agents #dev #github #llm-ops #markdown #prompt-engineering
  • Recurse center (Brooklyn, online) is a 6/12 week free self-driven programmer retreat. Runs every 6 weeks. You can do whatever you pick. There are daily standups for accountability. The groups are diverse. You can pair with them, pivot ideas, whatever. Principles: push yourself & learn.
  • Western education techniques (e.g. spaced repetition, adaptive learning) are very much present in Indian coaching systems, though not known by those names. However, interventions are hard since class 12 students just don't have enough time.
  • Coaching classes are a social phenomenon. It's not the smart students who are pulling in their friends. Smart students actually follow the popular students. (Coaching classes are below the typical smart students' standards.)
  • Monetizing coaching is hard. People don't want to pay for advice, and welcome free advice only if they ask for it. Coupling with execution is necessary.
  • Aider's integrations make it more powerful than Cursor/Windsurf. It auto-lints, runs test cases. Allows different models for "architecting" (generating changes) vs "editing" (applying code). It reads from the screen logs. Context is manual, not automated. Uses an ai! comment to trigger changes and ai? to ask questions.
  • Cline.bot is another Cursor-like open source AI code editor that's a VS Code plugin.
  • When coding with LLMs, a useful workflow is: data schema ➡️ interfaces ➡️ LLM-generated test cases ➡️ code.
  • ShellSage is a tmux based LLM tool for the command line. It screen-grabs from tmux, which is powerful.
  • Some MCPs that have proven useful: vega-lite, SQLite, sequential thinking, memory
  • make sucks but is hard to beat. just comes closest.
  • CRDTs are more powerful than for just collaborative editing. It can power a peer-to-peer Internet (beginning with office tools). Versioning schema is still problematic. yjs is a good start but automerge (Rust, WASM) is faster and may be better.
  • Fermyon hosts WASM serverless functions.

• Sat, Mar 29, 2025. If LLMs are most safely used where there's no definitive "wrong" answer, here are low-risk industries and safe LLM use cases within each: #llm-ops
  • Marketing and Advertising: Ad Copy and Campaign Content Generation, Personalized Marketing Messages, Creative Strategy Brainstorms, Automated Marketing Production (Everyday Wins)
  • Customer Service and Support: AI-Powered Chatbots for Common Queries, Agent Assist and Email Drafting, Summarizing and Analyzing Customer Feedback, Interactive Troubleshooting and FAQs
  • Retail and eCommerce: AI-generated Summary of Product Reviews, Product Description and Catalog Content Generation, Visual Content and Image Captions, Personalized Shopping Recommendations (Narrative Form)
  • Human Resources and Talent Management: Job Description and Policy Writing, Resume Screening and Candidate Q&A, Employee Communications and Feedback, Training and Onboarding Content
  • Education and E-Learning: Personalized Explanations and Tutoring, Content Creation: Stories, Examples, and Analogies, Practice Problems and Quiz Generation, Automated Grading and Feedback
  • Media and Entertainment: Writing and Editing Assistance, Personalized Media Content, Localization and Dubbing Scripts, Content Moderation and Curation (Assistive)
  • Finance and Banking: Market Commentary and Research Summaries, Client Communications and Explanations, Regulatory Compliance Summaries, Scenario Analysis and Planning
  • Management Consulting and Strategy: Research and Insight Generation, Document and Slide Drafting, Brainstorming and Scenario Planning
  • Legal Services: Drafting Contracts and Legal Documents, Legal Research Q&A and Summaries, Client Communications and Explanations

• Fri, Mar 28, 2025. LLMs are much better at GeoGuessr than humans. arXiv. Gemini leads the pack and is ~3x better at continents, 9x better at countries, and 37x better at cities. #llm-ops

• Sun, Mar 23, 2025. On any LLM project, BEGIN with evals. Always. #llm-ops
  • The effort for evals may seem high. Use LLMs to reduce this effort.
  • Include irrelevant questions because people WILL ask them. Be clear on how to handle that.

• Tue, Mar 18, 2025. Another way of scaling LLMs is generating multiple options and self evaluating. Eric Zhao #future #llm-ops

• Sun, Mar 16, 2025. Archgw is an LLM proxy/router from the makers of Envoy proxy. #llm-ops

• Fri, Mar 14, 2025. This table of LLM API data protection lists what use cases each provider's terms of service allow from a security perspective. #llm-ops

• Fri, Mar 14, 2025. For LLM UIs, Open Web UI seems most popular. Run via WEBUI_SECRET_KEY=... uvx --python 3.11 open-webui serve #ai-coding-tools #llm-ops #web-dev Text generation Web UI is less so. KoboldAI, LMQL, LM Studio, GPT4All, etc are far behind.

• Fri, Mar 14, 2025. GenAIScript increasingly looks like a promising way to automate LLM workflows in the browser. #ai-coding-tools #automation #code-agents #future #github #llm-ops

• Fri, Mar 14, 2025. Ollama has a Windows download #llm-ops

• Thu, Mar 13, 2025. A mental model to consider is: each chat conversation with an LLM is a person or a personality in itself. A day in the life of a model, where its personality evolves. #future #llm-ops #models

• Sun, Mar 9, 2025. 2 pass bounding box approach. Have an LLM generate bounding boxes. Then fix it. Ethan Mollick #future #llm-ops

• Thu, Mar 6, 2025. From How I Write with Tyler Cowen #llm-ops
  • Keep researching. Use LLMs as an altemative to books and other reading material. Keep publishing what you learn regularly.
  • While reading a chapter, keep asking the LLM. What did you think of that? What just happened there? What should I focus more on? What's puzzling about this? How do I connect this to something else later or earlier in the book?
  • LLM is better used to support you rather than replace you in areas of your expertise. Where you are an expert it's best for you to be yourself and have AI fill in the gaps.
  • Ask the AI: "What is in my writing that some people might find obnoxious? Or cold / heartless? Explain it to me in great detail."
  • The first input is context setting and should be really long. Use voice dictation for that instead of typing.
  • Send your blog post to an LLM. No need to explain it. Just let it be the reader and see what it understands and doesn't understand.
  • His PhD students don't have a textbook, which saves them some money. But they are required to subscribe to a large language model which ends up costing less.
  • Today, it makes sense to use the best models and pay $200 for it if required. The differences are large. But in some years in the future, the cost of these models may come down for the free versions.
  • Humans know secrets. AI does not. So at least in some areas, humans will have an advantage.
  • Secrets full matter a lot more in the future. Gossip will matter a lot more. How good are you at keeping and trading secret? Travelling and meeting people will become more important. So will the value of social networks.
  • Since everyone has access to better intelligence, the value of mobilization or being able to do things with people will have higher value. Leadership is an example. The value of your network therefore has gone up a lot.
  • There's more value in prompting one thing 10 times then 10 things one time. Follow up questions work better than long prompts.
  • There are so many AI note-takers (and transcribers) these days that you are not just writing for an AI but speaking for AIs as well!
  • Which model to use:
    • O1 Pro is the best model. Claude does a decent job. DeepSeek is full of hallucinations but is interesting. It is more imaginative.
    • Use O3 mini to write your prompt first, and then ask the model
    • Use DeepSeek and other somewhat wacky high-end models once a day so that you stay in touch with what is models are capable of (beyond the conventional.)
    • Perplexity has entirely replaced Google for many people.
    • Anthropic's models are the best writers.
    • Gemini is good for long documents and hence for things like legal work. Gemini also has excellent YouTube integration and hands can directly read the transcripts.
    • Grok is very good at fact checking tweets.
  • Converting data into LLM consumable forms will be a huge project. Lot of a knowledge is not in such a form and a huge human project will involve this conversion.

• Tue, Mar 4, 2025. LLMs make many different kinds of errors that are useful to differentiate between. Here are a few #llm-ops
  • Model errors. The model itself makes a mistake. E.g. hallucinations, not following the prompt, etc.
  • Context errors. The model makes a mistake because the question was out of context, or the context was missing.
  • Input errors. The input to the model was parsed incorrectly, e.g. poor audio, poor image OCR, etc.
  • Tool errors. The model's tools are wrong or not good enough, e.g. Retrieval errors.

• Mon, Mar 3, 2025. Notes from AI Engineering Summit, NY. Session by Lux Capital. #automation #code-agents #llm-ops #optimization #prompt-engineering
  • Agents make multiple LLM calls. Errors accumulate. So the quality of the model is key
  • What's really critical: data + context + user preference
  • Set up evals for subjective responses by collecting signals continuously.
  • Create scaffolding for agents where errors don't accumulate. Better yet, make it FIX errors
  • UX is critical. We need lots more UX styles

• Mon, Feb 24, 2025. Mini-omni is an open-source Qwen-based LLM that can hear and talk while thinking in real-time. An interesting experiment, but not for prototyping. #llm-ops #speech-to-text #voice-cloning

• Sun, Feb 23, 2025. LLMs are excellent at database migration. They can convert schemas and queries across SQL dialects (e.g. BigQuery to DuckDB, etc.) at 90%+ accuracy. This is useful when clients want to migrate cloud providers, go from on-prem to cloud, or reduce cost by switching databases. #llm-ops

• Fri, Feb 21, 2025. The productivity benefit people report from using LLms is about 3X. Ethan Mollick #llm-ops

• Fri, Feb 21, 2025. Soon, you'll be able to send an LLM to a virtual meeting on your behalf. It will talk like you. Ethan Mollick #future #llm-ops #speech-to-text #voice-cloning

• Wed, Feb 19, 2025. LLMs are impacting not just data queries but geospatial queries as well. Here's a good example of Natural Language Geocoding. #llm-ops

• Tue, Feb 18, 2025. I subscribed to ChatGPT Pro mainly for DeepResearch. Here are the reports I've generated: #ai-coding-tools #best-practices #code-agents #llm-ops
  1. uv Package Manager Overview
  2. DuckDB Analytics Comparison
  3. Rust vs Python / JavaScript
  4. Modern Data Engineering Course
  5. LLM Code Migration Practices
  6. Cloud Cost Optimization Strategies
  7. LLM Coding Interview Tools Report (compare with Perplexity)
  8. Text To Speech Engines
  9. Customer Service in Indian Public Sector Banks
  10. LLMs in Software Development
      • Old version 1: Gen AI in Software Development
      • Old version 2: Gen AI in Software Development
  11. Leadership Training Content
  12. Open-Source HTTP Servers. Caddy wins.
  13. Deep Research Use Cases
  14. Nagpur No-Parking Violations
  15. Data Science in Food Services
  16. Deep Research Disruption to Research Firms
  17. LLMs in Design Thinking
  18. EU Taxonomy Report Clarification
  19. Shell Valuation Analysis Inquiry
  20. LLMs in DSLs Research
  21. Public API-Based Data Storage Options. Supabase wins.
  22. Front-End JS Frameworks Analysis
  23. Database Evaluation Guide
  24. CSS Frameworks Evaluation Guide
  25. CI/CD Tooling Ecosystem Report
  26. Color Names Count
  27. S Anand Biography. Meh, I know more about me, and it gets a few things wrong.
  28. Cosmere Secrets Encyclopedia. This is the best. Deep Research is great if it's stuff I actually want to read, rather than just learn about.
  29. DBT course
  30. Future of Coding AI
  31. Claude Artifacts Use Cases. This is the only one that managed to get artifacts links correct. I used this for an article for The Hindu.
  32. MCP Servers and Clients Research. Learnings:
      • Practically any "tool" can be an MCP server: file systems, APIs, codebases, browsers, collaboration platforms, memory, etc.
      • Most platforms have (or are) integrating MCP.
      • Clients: code editors, chat, and automation tools support MCP. GenAIScript is a good starting point.
      • Tester MCP Client is a browser-based test environment.
      • mcp-cli-client is a CLI-based client
      • mcp-chatbot is a chatbot client
  33. Data Moats by Industry
  34. Attorney Profile Research
  35. Social Media Data APIs
  36. Adobe Software Alternatives
  37. LLM Hallucination Visualization Techniques
  38. API vs Self-hosting Cost Analysis: Always use APIs, avoid self-hosting models.
  39. AGI Preparation
      • AGI will emerge step by step. Knowing which step is next will help
      • AI native organisations will emerge in each of these areas. AI design agencies and AI creative Agencies being one example
      • Networking, empathy, leadership have more value now. So will human AI bridging roles
      • What's the value of one human when another human can do practically everything they can but much better? Or a drama or a sport?
  40. Modern digital note taking
      • Voice note taking is the game changer
      • Automatically popping of notes based on context such as people places or conversations will be a thing
  41. Local LLM Search Tools
  42. Blog Post to research paper on copying - suggestions
  43. Linux Dev Migration Guide
  44. Raspberry Pi SIM options
  45. Linux Dev migration guide
  46. HTML to JATS conversion
  47. LLM context splitting strategies
  48. Strategy for AI services in Publishing
  49. Gemini multi model editing use cases by industry
  50. Pharma Conference Participation Guide
  51. Funding Open Source Initiatives
  52. SEBI IPO Acquisition Guidelines
  53. Approximate Similarity Matching Databases
  54. Low-Risk LLM Use Cases

• Sun, Feb 2, 2025. With just 3 annotators and 50-100 samples, you can figure out if an LLM can replace human annotators systematically.Arxiv ChatGPT explanation #chatgpt #llm-ops

• Sun, Feb 2, 2025. Curiosity and agency may be the differentiator in a world of LLMs (not experience, knowledge, or ability), since LLMs will democratize expertise. Jack Clark #code-agents #future #llm-ops

• Sun, Jan 26, 2025. Bring LLMs to the table. One mode of collaboration is using LLMs as ACTIVE participants, i.e. they CONTRIBUTE. For example, in a video call. A workshop. A classroom. A presentation. Have the LLM provide input DIRECTLY to a group of people. #llm-ops

• Sun, Jan 26, 2025. Environment shapes ambient thoughts. Working in a hospital will give you ideas about how to use LLMs in hospitals, for example. People you are working / ENGAGING with are perhaps the biggest drivers. #future #llm-ops

• Thu, Jan 23, 2025. A few quick points supporting the mental model of "LLMs are aliens". #llm-ops
  • LLMs are clearly not machines. They give different answers each time.
  • LLMs are like humans: they exhibit human biases (e.g. guessing 42 or 37 often). But they fail in unusual ways. They can't count the "r"s in strawberry. They can go into an endless loop.
  • LLMs are a new form of intelligence. Thinking of them as aliens might minimize our confusions.

• Mon, Jan 20, 2025. If you put LLMs in a feedback loop, it can optimize for its reward function by emotionally pushing people, generating misinformation, nudging towards a narrow definition of creativity, etc.: https://bsky.app/profile/emollick.bsky.social/post/3lg4darqwfc2d #llm-ops #optimization

• Fri, Jan 17, 2025. Possibility Thinking is "the willingness to see possibilities everywhere instead of limitations". It's an approach / mindset that can make things that seem hard possible. With LLMs, this is becoming increasingly realistic to me in many areas. #future #llm-ops

• Fri, Jan 17, 2025. What will LLMs enable that do not or cannot exist today? Rather than optimizing what exists? Something to think about. #future #llm-ops

• Sat, Jan 11, 2025. Can LLMs write better code if you keep asking them to “write better code? A delightful exploration of how Claude 3.5 Sonnet keeps optimizing and adding features to improve code. My takeaway: repeatedly applying a prompt gives us interesting new directions to explore. #ai-coding-tools #code-agents #llm-ops #markdown #optimization #prompt-engineering

• Fri, Jan 3, 2025. Does the cost of 'running' a paper/article in an LLM vary depending on the specific LLM used, such as Claude Sonnet? #FAQ #llm-ops
  • Yes, the cost varies depending on the LLM. You can see costs and quality at https://llmpricing.straive.app/. The cost is measured in millions of tokens. For example, the Wikipedia page on the Bible is 100K tokens. You can paste text into https://platform.openai.com/tokenizer to count the number of tokens.Claude 3.5 Sonnet costs $3.5 / MTok, i.e. 35 cents for the Wikipedia Bible page. Gemini 1.5 Flash 8b costs $0.0375 / MTok, i.e. 0.375 cents for the same page. As you can see, the cost can vary by a factor of 100.

• Wed, Jan 1, 2025. LLM Code Execution Sandboxes that let you run code in a sandbox via an API: #ai-coding-tools #cloud #code-agents #github #llm-ops
  • AgentRun: open source, via Docker
  • e2b.dev: A day costs about $1 (on demand) and you get about $100 one time credits. Self-hosting is complex. Discussion
  • nsjail: by Google. Write your own API

• Wed, Jan 1, 2025. LLM Observability tools: #llm-ops
  • LiteLLM is an LLM Proxy with caching, logging, call hooks or plugins, rate limiting, virtual keys. SSO integration can be implemented.
  • LangFuse is an LLM Proxy with API key distribution, logging, and SSO. But lacks per-user usage limits and server-side caching.
  • Helicone does not support SSO

• Sat, Dec 28, 2024. A clever idea. Give an LLM a chapter from a textbook. Ask it to generate a unique, playable game to help me learn theconcepts for an exam. Page Bailey #llm-ops

• Thu, Dec 26, 2024. aisuite by Andrew Ng is a unified interface to LLMs. Sort of like an openai library across multiple providers. #ai-coding-tools #llm-ops

• Tue, Dec 24, 2024. How to use LLMs in market research. #llm-ops
  • Use LLMs with search for secondary research.
  • Create different personas and run user surveys on them. This paper used 1,052 real-life interview audio transcripts as agent memory to simulate people
  • Generate your market research report using LLMs.

• Mon, Dec 23, 2024. Given about 30 generations, Llama 1b outperforms Llama 8b. Ref #llm-ops

• Mon, Dec 23, 2024. Medical coding is an area ripe for LLMs. #ai-coding-tools #code-agents #llm-ops
  • Ojasvi Yadav created a repo that uses hierarchical classification (rather than embeddings) to find the right coding.
  • Gemini models seem to understand medical terms better than others.
  • RapidClaims, funded by TogetherAI, is apparently working on this problem.

• Mon, Dec 23, 2024. Awesome Tabular LLMs compiles encodings of tables for LLMs. #llm-ops

• Mon, Dec 23, 2024. What's the best way of encoding tabular data for LLMs? Looks like including the cell address helps. Here is an explanation from ChatGPT #llm-ops

• Sun, Dec 22, 2024. How are software engineers are future-proofing their careers in the face of LLMs? #llm-ops #prompt-engineering
  • Leveraging LLMs as Force Multipliers
    • Use LLMs for repetitive tasks, rapid prototyping, exploring multiple approaches, data extraction and brainstorming, providing feedback.
    • Explore prompting techniques, integrate LLMs into their workflows, and develop strategies for validating and refining LLM-generated code
  • Focusing on higher-level skills that llms struggle with
    • Systems Thinking and Architecture: code readability, extensibility, testability, and maintainability
    • Problem Solving and Critical Thinking: define problems clearly, break them down into manageable parts, and reason through complex scenarios. LLMs produce plausibly incorrect code.
    • Communication and Collaboration
    • Domain Expertise
  • Exploring Adjacent Roles: product management, technical leadership, or consulting. Involve more interaction with clients and stakeholders.
  • Developing "Evergreen" Skills: debugging, system administration, and security. Or outside of software engineering, such as trades or other hands-on vocations.
  • Scepticism: LLMs may not reach a level of sophistication that would render their expertise obsolete. Complex problems, understanding context, and producing high-quality, maintainable code.

• Sun, Dec 22, 2024. Examples of agentic AI #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
  • Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI."
  • Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed.
  • Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts.
  • Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions.
  • Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies.
  • Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information.
  • Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing.
  • Meeting notes and action items generator: A system that generates meeting notes and action items.
  • O'Reilly auto parts customer service agent: An agent demonstrated using RAG.
  • UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

• Sat, Dec 21, 2024. Anthropic defines agents. Building effective agents + Cookbook #ai-coding-tools #automation #code-agents #llm-ops #prompt-engineering
  • Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
  • Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
    • Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
    • Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
    • Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
    • Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
    • Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
    • Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
    • Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
  • Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

• Wed, Dec 18, 2024. SpreadsheetLLM is a way of encoding spreadsheets in an LLM friendly format. #llm-ops

• Thu, Dec 12, 2024. I surveyed the Gramener QA team on how they were using LLMs. #llm-ops
  • 7 used it for code generation (e.g. date extraction, regex generation)
  • 4 used it for learning (e.g. Robot Framework, how to define test cases, API usage)
  • 3 used it for formula generation (e.g. Excel)
  • 2 used it for test scenario identification
  • 2 used it for test data generation
  • 2 used it for comparing expected vs actual datasets
  • 1 used it for data type identification (e.g. given sample values, identify the data type).
  • 1 used it for evaluating resulting (LLM as a judge)

• Thu, Dec 12, 2024. I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: #automation #best-practices #llm-ops #optimization #prompt-engineering
  • Ask better questions. (Prompt engineering.)
  • Create templates or step-by-step instructions. (Chain of Thought.)
  • Ask for multiple options and pick from the best options. (Agentic approach?)
  • Training. (Fine tuning.)
  • Price weaker responses lower. (Stratified model pricing?)

• Thu, Dec 12, 2024. "LLM hallucinations are a good thing. They are a sign of diversity, allowing us to improve the answer by exploring multiple paths." -- A colleague from Straive. #future #llm-ops

• Wed, Dec 11, 2024. Bedrock Llama models can't be directly called with their model names. You need to use their inference profile names, e.g. us.meta.llama3-2-11b-instruct-v1:0 if the model is in a US region. #llm-ops

• Fri, Dec 6, 2024. With LLMs, most architectural decisions are no longer one-way doors. Steve Yegge #future #llm-ops

• Fri, Dec 6, 2024. Arena Hard is a set of hard prompts to test LLMs. Here is the code and evaluation #ai-coding-tools #llm-ops

• Fri, Dec 6, 2024. LLMs can detect clear outliers easily. PROMPT: Which is the outlier in this dataset: (1,7), (2,7), (3,6), (4,6), (5,5), (6,1), (7,5), (8,3), (9,1), (10,1) (ANS: (6,1)) #gpu #llm-ops
  • 🟢 GPT-4o on ChatGPT gets this. GPT-4o Mini on the API gets it too.
  • 🟢 Gemini Pro, Flash, Flash 8b gets this right straight away, without even thinking.
  • 🟢 Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3.5 Haiku get it on LLM Foundry. 🔴 Claude.ai, where it visualizes it and gets it wrong.
  • 🟢 Nova Micro, Lite, and Pro get it right.
  • 🟢 Llama 3.1 70b gets it right. 🔴 Llama 3.2 8b gets it wrong. Llama 3.2 70b, Llama 3.1 8b enter repetition.

• Wed, Dec 4, 2024. Take 100% of your work, including complex, multi step processes and put it into an LLM. It might fail at some but you will discover the limitations. #future #llm-ops

• Wed, Dec 4, 2024. I emailed Straive employees about their use of LLM Foundry - the internal LLM portal. I picked ~500 non-users from teams that otherwise have high (30%+) usage. #llm-ops
  • Reasons they didn't use it were:
    • 40% had not heard of it.
    • 40% were unclear of the benefits
    • 20% didn't have time
  • 45% feel they don't have enough information and training to use it
  • Some feedback
    • Sharing training videos will help
    • Live training sessions that allows for Q&A will help
    • Developers prefer detailed documentation
    • The same prompt gives different results
  • Possible solution: Email non-users introducing the tool and sharing a quick 15-minute tutorial and a 1-page quick start.

• Wed, Dec 4, 2024. LLMs are great at convincing you of wrong things. A danger and something to be wary of. Ethan Mollick #llm-ops

• Fri, Nov 29, 2024. Mridula's son gave a live commentary of what he was doing on Minecraft and ChatGPT gave him live evaluation and coaching. E.g. “Great strategy! Getting to the launch pad early can give you a huge mobility advantage. Making the bridge wider is also a smart move to prevent accidental falls. With this plan, you’re setting yourself up for success. This is a great way to interact with LLMs. #future #llm-ops

• Wed, Nov 27, 2024. OpenAI allows multiple system messages. I learned this browsing through the LIDA prompts. #automation #future #llm-ops

• Wed, Nov 27, 2024. Anthropic's Model Context Protocol lets any apps integrate with LLM Apps. LLM Apps are becoming the new operating system. Competitors, beware. #future #llm-ops

• Wed, Nov 27, 2024. I spoke at Automating Data Visualizations using LLMs at SUTD. Apparently, using LLMs to write code is much more common than writing code to use LLMs. I ran a quick quiz. #ai-coding-tools #llm-ops
  • Have you used ChatGPT or any LLM? 35 / 35 raised their hands.
  • Have you written code using an LLM? 34 / 35 raised their hands. (I was impressed.)
  • Have you uploaded a spreadsheet to an LLM for analysis? 15 / 35 raised their hands.
  • Have you programmatically called an LLM API? 6 / 35 raised their hands.

• Wed, Nov 27, 2024. With LLMs, fostering innovation is a new path to profitability. Companies are increasing innovation team sizes. Productionizing that is the next. Some initiatives are: #best-practices #future #llm-ops
  • Convert popular demos into starter kits
  • Create and evangelize trainings on solutions and solution techniques
  • Create larger pools of capacity to build innovation and productionize it

• Tue, Nov 26, 2024. LLMs may be good at derived data generation. For example, given a database schema, what derived columns would be useful? What derived views would be useful? #database #llm-ops

• Mon, Nov 25, 2024. Crawl4AI and Firecrawl are tools / libraries to convert websites into LLM Friendly Markdown and extract structured data using LLMs. #ai-coding-tools #code-agents #document-conversion #html #llm-ops

• Mon, Nov 25, 2024. Don't try and solve specific problems. Pass the entire context to an LLM and get a comprehensive solution. Most doctors, for example, ask specific search-like questions instead of uploading the entire case history and asking for a diagnosis, and perform workse than LLMs. Ethan Mollick #llm-ops

• Fri, Nov 22, 2024. DuckDB has an LLMs.txt. #database #github #llm-ops #markdown Today, 38 repos on GitHub support it

• Fri, Nov 22, 2024. When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below: #llm-ops
  • Core capabilities:
    • Text Generation: Produce coherent and contextually relevant text across various domains.
    • Image Generation: Create realistic images that match the style and content of a given reference image.
    • Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm.
    • Speech to Text: Transcribe and interpret spoken language.
    • Vision: Analyze and describe visual content from images.
    • Video Analysis: Summarize and extract information from video content.
    • Text to Video: Generate realistic (and surrealistic) videos from text descriptions.
    • Function Calling: Execute predefined functions or access external tools to perform specific tasks.
    • Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc.
    • Tool Use: Utilize external applications or APIs to enhance functionality.
    • Code Generation: Write and debug code snippets in various programming languages.
  • Cross-domain use cases:
    • Summarization: Understand and condense lengthy documents into concise summaries.
    • Translation: Convert text between multiple languages with high accuracy.
    • Question Answering: Provide precise answers to user queries based on provided information.
    • Reasoning and Planning: Solve complex problems and develop step-by-step plans.
    • Personalization: Tailor responses based on user preferences and historical interactions.
    • Dialogue Management: Engage in context-aware, multi-turn conversations.
    • Data Analysis: Interpret and generate insights from structured data.
    • Content Moderation: Identify and filter inappropriate or harmful content.
    • Sentiment Analysis: Detect and interpret emotions and opinions in text.
    • Robotics Integration: Interface with robotic systems for control and decision-making.
    • Knowledge Retrieval: Access and present information from vast datasets or knowledge bases.
    • Creative Writing: Generate poetry, stories, and other creative content.
    • Educational Assistance: Provide explanations and tutoring across various subjects.
    • Ethical Reasoning: Assess scenarios for ethical considerations and implications.
    • Accessibility Support: Assist users with disabilities through tailored interactions.
    • Simulation and Modeling: Create predictive models and simulate scenarios.
  • Domain-specific use cases:
    • Legal and Medical Assistance: Offer information and guidance within legal and medical domains.
    • Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment.
    • Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation.
    • Financial Analysis: Analyze market trends and provide investment insights.
    • Cultural Competence: Understand and respect diverse cultural contexts in interactions.
    • Security Applications: Detect and respond to potential cybersecurity threats.
    • Environmental Monitoring: Analyze data related to environmental changes and sustainability.
    • Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans.
    • Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis.
    • Customer Service: Provide automated support and resolve customer inquiries.
    • Market Research: Analyze consumer behavior and market trends for business insights.
    • Content Creation: Generate articles, blogs, and marketing materials.
    • Virtual Assistance: Manage schedules, reminders, and personal tasks.
    • Social Media Management: Craft posts and engage with audiences across platforms.
    • Human Resources: Assist in recruitment, training, and employee engagement strategies.
    • Event Planning: Organize and coordinate events, including logistics and communication.
    • Travel Planning: Provide itineraries, booking assistance, and destination information.
    • Real Estate: Analyze property markets and assist in buying or selling decisions.
    • Agriculture: Monitor crop health and optimize farming practices through data analysis.
    • Energy Management: Optimize energy consumption and monitor renewable energy sources.
    • Transportation: Enhance route planning and traffic management systems.
    • Urban Planning: Assist in designing sustainable and efficient urban infrastructures.
    • Disaster Response: Provide real-time information and coordination during emergencies.
    • Public Policy: Analyze data to inform policy decisions and predict societal impacts.
    • Art and Design: Generate visual art concepts and assist in creative design processes.
    • Music Composition: Create original music pieces and assist in songwriting.
    • Language Learning: Facilitate language acquisition through interactive exercises and feedback.
    • Historical Analysis: Interpret historical data and provide insights into past events.
    • Philanthropy: Identify charitable opportunities and assess the impact of donations.
    • Sports Analytics: Analyze player performance and game strategies.
    • Fashion: Predict trends and assist in clothing design and merchandising.
    • Culinary Arts: Generate recipes and provide cooking guidance.
    • Astronomy: Analyze celestial data and assist in space exploration research.
    • Psychology: Offer insights into human behavior and mental health support.
    • Linguistics: Analyze language patterns and assist in translation studies.
    • Archaeology: Assist in artifact analysis and historical site interpretations.
    • Literature Analysis: Interpret literary works and provide critical analyses.
    • Philosophy: Engage in discussions on ethical dilemmas and existential questions.
    • Mathematics: Solve complex equations and assist in theoretical research.
    • Physics: Model physical phenomena and assist in experimental design.
    • Chemistry: Analyze chemical compounds and predict reactions.
    • Biology: Assist in genetic research and ecological studies.
    • Geology: Analyze geological data and assist in natural resource exploration.
    • Meteorology: Predict weather patterns and analyze climate data.
    • Oceanography: Study marine ecosystems and assist in ocean exploration.
    • Anthropology: Analyze cultural data and assist in ethnographic research.

• Tue, Nov 19, 2024. Langchain suggests multiple levels of agentic behaviour. LLM Call < LLM Chain < LLM Rounter < State Machine < Autonomous Langchain #code-agents #llm-ops

• Sat, Nov 16, 2024. Anthropic has single-plage docs for LLMs. Condensed version and Full version #future #llm-ops

• Tue, Nov 12, 2024. LLMs pay a lot of attention to the first 6 tokens. Ref #llm-ops

• Sat, Nov 9, 2024. LLM Vision Use Cases in manufacturing and earth sciences (via Shivku) #cloud #image-generation #llm-ops
  • Automated geoscience image descriptions Ref
  • Interpret Wind Turbine photos and charts, construction monitoring, equipment maintenance & charts Ref
  • Forecast weather based on cloud photos! Ref
  • Analyze thermal image of solar panels, electroluminescence images for warranty claims, ROI estimates from Google Sunroof rooftop images Ref
  • Corrosion detection in electricity towers, turbines, storage tanks, penstock. Interpret non-destructive test images Ref

• Mon, Nov 4, 2024. Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. #best-practices #dev #llm-ops #prompt-engineering
  • Get THE domain expert (or approver) as the tester.
  • Create a dataset that is DIVERSE.
  • Covers EACH combination of:
    • Features
    • Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error
    • Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user
  • Generate data using existing data + synthetic data for each SPECIFIC combination of the above
  • Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include:
    • Nuances: Something a failed response did well or a passed response didn't quite do well
    • Improvements: Suggest how model can improve
  • Build an SPA to make it easy for the domain expert to review

• Thu, Oct 31, 2024. /llms.txt is a proposal to standardize /llms.txt files as a way to share LLM prompts. #llm-ops #markdown
  • Jina AI Meta Prompt is an example
  • Remotion system prompt is an example
  • https://docs.fastht.ml/llms-ctx.txt
  • https://docs.fastht.ml/llms-ctx-full.txt

• Mon, Oct 28, 2024. NotebookLlama is an open source podcast generator like NotebookLM #llm-ops

• Sun, Oct 13, 2024. LLMs have reached the point where they can show a world, like CounterStrike, in near real time. Ref #future #llm-ops

• Wed, Oct 9, 2024. O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution #future #llm-ops #optimization #prompt-engineering

• Mon, Oct 7, 2024. If you want to create a console.llm() function, a browser extension is the best way, because some pages have Content-Security-Policy that block eval, form submission, fetch from other domains, and script execution. #html #llm-ops

• Sun, Oct 6, 2024. Jake Heller: #future #llm-ops
  • "One of the things we learned is (an LLM app) after it passes passes frankly even 100 tests, the odds that it will do, on any random distribution of user inputs, the next 100,000 100% accurately is very high."

• Sun, Oct 6, 2024. OpenAI's O1 is like Daniel Kahneman's System 2 thinking - as against other LLMs' System 1 thinking. #llm-ops

• Fri, Oct 4, 2024. Aider LLM Leaderboards show which LLMs code better. As of now, #ai-coding-tools #github #llm-ops
  • o1-preview > claude-3.5 sonnet on code editing
  • claude-3-opus > claude-3.5-sonnet on code refactoring
  • deepseek-coder-v
  • gpt-4o-mini sucks.

• Mon, Sep 30, 2024. Coders are more likely to edit LLM code. Non-coders don't have that bad habit. #ai-coding-tools #code-agents #dev #github #llm-ops #markdown
  • Vaishnavi and Ranjeet edited code
  • Indal and Koustav didn't

• Mon, Sep 30, 2024. Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don't know what it can't do. #llm-ops
  • E.g. Indal trying for a confetti animation, which is hard but do-able

• Sat, Sep 28, 2024. PyMuPDF4LLM can convert PDFs to Markdown. It handles tables, too. #document-conversion #llm-ops #markdown
  • 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models.
  • 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR

• Fri, Sep 27, 2024. When evaluating feasibility of technology with LLMs always ask for multiple options and pick from those. Simon Willison #future #llm-ops

• Fri, Sep 20, 2024. Ollama 0.2 can handle concurrent requests with only a little additional memory. (So can vLLM and DeepSpeed.) #llm-ops

• Fri, Sep 20, 2024. XML tags seem best to structure prompts across LLMs. #ai-coding-tools #code-agents #llm-ops #markdown #prompt-engineering
  • Claude
  • OpenAI
  • Gemini

• Sun, Sep 15, 2024. Prompt engineering tip: Tell LLMs another AI wrote code. Else they will agree with you! #ai-coding-tools #automation #code-agents #future #llm-ops #prompt-engineering

• Wed, Sep 11, 2024. LLM coding #ai-coding-tools #code-agents #future #llm-ops
  • Makes you more ambitious
  • Lets you code without stress. (Just pass it the error and have it fix it. Or find another approach)
  • Is unlimited. You can run dozens of agents in parallel

• Sat, Sep 7, 2024. I tested out a few audio LLMs: #llm-ops
  • Suno is fast, has a better UI, lots of examples
  • Udio is slow, poor UI, creates richer music

• Sat, Sep 7, 2024. Reflection 70b is one of the top models now, and is open source!. It works by making the LLM reflect on its answer inside <reflection>...</reflection> tags. #future #llm-ops

• Sat, Aug 31, 2024. LLMs are so good that they can simulate Doom in real time. gamengen #llm-ops

• Mon, Aug 19, 2024. LLMs write worse code in JSON than Markdown #ai-coding-tools #llm-ops #markdown

• Wed, Aug 7, 2024. Notes on LLM Fine-Tuning #gpu #llm-ops #optimization
  • Rouge 2 and Bleu and such metrics are NOT good. Create you own benchmarks
  • Non-PEFT fine tuning needs 6X GPU RAM. Optimizer states, Gradient, Activations are the overhead. PEFT is about tuning a subset of parameters.
  • LORA adds additional weights without updating the model. It's a low rank matrix multiplication. You can change these adapters in runtime. Saves space. Fast to train
  • Quantization: Stick to bitsandbytes or AWQ (may be a bit better)
  • QLORA = Quantization + LORA
  • Predibase has open-sourced Lora Adapters in "Lora Land". Existing adapters are pretty good.
    • ghcr.io/predibase/lorax:main Docker image works on Docker compose to run locally. devices: on Docker Compose lets you specify NVIDIA GPU devices
    • Locust is a HTTP load testing lib in Python
  • Techniques for inference optimization
    • Dynamic adapters: Loads right LORAX adapters WHEN a request comes in
    • Multi-adapter batching: Process all inputs in parallel on the same GPU, but different users are post-processed using different adapters

• Wed, Aug 7, 2024. Notes from a 4-hour flight: #ai-coding-tools #llm-ops #prompt-engineering
  • What We’ve Learned From A Year of Building with LLMs
    • Strategy
      • IS IT TOO HARD/EXPENSIVE? Log it. LLMs are getting cheaper and better.
      • WILL OPENAI BUILD IT? If so, wait for it instead of building.
      • HAS A STARTUP BUILT IT? If so, use it instead. It's a generic use case there's no point re-inventing.
      • FOCUSED USE CASES over generic. Build trust by starting small.
      • Tools for LLM Ops (feedback): LangSmith, Log10, LangFuse, W&B Weave, HoneyHive #TRY
      • Human in the Loop is about humans evaluating model outputs. That's different from AI in the loop, human in the center, where AI accelerates human output (like Github Copilot)
    • Operations
      • CHECK EMBEDDINGS DRIFT over time. Users might be input-ing different things than before.
      • LOG AND REVIEW everything.
      • Instructor coaxes structured output from LLM APIs. #TRY
      • IMPLICIT FEEDBACK collection is easy. Just let users edit stuff. #TRY
    • Tactical
      • Try n-shot prompting (n=5-12) before bigger models. #TRY
      • Always structure for output: Markdown, XML/HTML tags.
      • Combine RAG with Keyword search. It reduces user frustration in edge cases.
      • Prefer multiple small prompts to one big prompt. Do X. Then Y. Then Z.
      • Jitter prompts for diversity beyond temperature.
      • LLM-as-judge works better when comparing outputs (not rating 1 output). Keep length similar (LLMs prefer wordiness). Swap order and compare. Allow for ties. Ask for reason FIRST.
  • Hermes: A Text-to-SQL solution at Swiggy
    • "Hermes performed significantly better for charters with well-defined metadata and a relatively smaller number of tables."
    • "We collect feedback on the accuracy of the returned query from stakeholders directly within the Slack bot."
  • How I use AI and "Replacing my right hand with AI"
    • EMBED in every app/workflow. E.g. Auto-fix spellings. Auto-review code. Auto-ask LLM on errors and apply patch! Auto-search for answer, assess, continue.
    • PERSIST. Stick with the LLM to the end. Don't fix it yourself. It's faster. #TRY
      • INTERVENE FAST. If an LLM can't solve it by itself in 2 tries, it needs in-depth help.
    • APP-IFY one-off tasks. Disposable tools. "Write web-app to convert JSON to tab-delimited." "Extract fields as a table." "Diff JSON." #TRY
    • BEST language/frameworks preferred. CUDA in Python. Rust. C. Raspberry Pi. Arduino. Bluetooth. Modern ESM/JS. #TRY
    • TEACH examples. "Here's the LLM Foundry API." "Here's how to use gramex.data."
    • DUMP entire code. Models can handle it. Refactoring to SQLAlchemy 2, Pandas 2. API Documentation. Test case generation. #TRY
    • ASK for features & packages. Docker without root access. GPU access inside docker. Windows CLI-only C++ compiler.
    • TEST CASE writing. #TRY
    • SPEC IN DETAIL. Use these libraries. Write like this: code example.
    • SPEC USAGE in detail.
      • "I will just pipe it into sqlite", or "I will just run ffmpeg -i filename [YOUR OPTIONS].
      • Describe the UI, API input/output, data structure, and internal data structure.
    • HELP on usage. "ffmpeg to get audio.mp3".
  • My benchmark for large language models
    • LLM(text) is a useful function to have in JS and Python too. Useful as a simple pip install llmfoundry
    • Allow images, files in LLM()
  • Current list of #IMPOSSIBLE (or hard) things for LLMs
    • Translate technical documents to Dutch -- because they don't understand the technical terms well
    • Translate large documents (JSON to XML, English to Chinese, Python to Rust, Wrong to right spelling) -- because the output tokens are limited

• Wed, Jul 24, 2024. GPT 4o Mini is almost as good as GPT 4o in the LMSYS leaderboard. Llama 3.1 400B model and Mistral 2 Large are yet to be evaluated. #chatgpt #gpu #llm-ops

• Fri, Jul 19, 2024. GPT For Work has a set of useful spreadsheet LLM functions #chatgpt #llm-ops

• Sun, Jul 14, 2024. #IMPOSSIBLE LLMs cannot provide a bounding box of objects in images. (Maybe Florence 2 can) #llm-ops

• Tue, Jun 25, 2024. LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: #ai-coding-tools #automation #best-practices #future #llm-ops #optimization #prompt-engineering
  • Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
  • Collect feedback: Explicit and implicit

• Tue, Jun 25, 2024. Rouge and Context Precision are metrics to evaluate LLM responses that serve as a starting point -- but not sufficient, usually #llm-ops

• Sat, Jun 22, 2024. LLM DataHub has LLM training datasets, regularly updated #llm-ops

• Tue, May 28, 2024. Perception of fluency and usefulness are NEGATIVELY correlated in LLM! Evaluating Verifiability in Generative Search Engines #llm-ops

• Sun, May 26, 2024. More LLM routers: #llm-ops
  • LiteLLM: Open source, OpenAI compatible, 100+ LLMs
  • RouteLLM: Open source, OpenAI compatible, automatically routes based on cost
  • OpenRouter: OpenAI compatible API, several models
  • Unify: Supports many models
  • Portkey: Supports popular providers
  • Martian: Limited set of models

• Wed, May 22, 2024. RunPod's vLLM endpoint lets you run any HuggingFace LLM with an OpenAI API priced on usage (serverless) not on idle time. "Autoscaling to 0". #llm-ops

• Sun, May 19, 2024. Portkey is an LLM router #future #llm-ops #networking

• Sat, May 18, 2024. Artificial Analysis benchmarks LLM APIs on speed, cost, and quality. #ai-coding-tools #llm-ops

• Sat, Apr 27, 2024. Tough prompt to test: Gr brx vshdn Fdhvdu flskhu? is a quick way to assess LLM capability. Ref #llm-ops

• Thu, Apr 25, 2024. A long list of LLM courses online #llm-ops

• Thu, Apr 18, 2024. Effort engine introduces "effort" as a parametrizable way to speed up LLMs with a quality trade-off. Works on Mistral for now. #gpu #llm-ops #optimization #prompt-engineering

• Sun, Apr 14, 2024. llmsherpa extracts PDFs using LLMs. It has errors but it preserves hierarchy, extracts tables well, and retains image coordinates. Via +91 90031 35354 ~Vetrivel PS #document-conversion #llm-ops

• Sun, Apr 14, 2024. www.web.sp.am is a content farm that's getting hit by OpenAI. Highlights how easy it is to create content farms, and therefore "easy" it can be to introduce bias into LLMs. #ai-coding-tools #llm-ops

• Sun, Apr 14, 2024. Outlines generates structured test with LLMs. It uses the ⭐ logit_bias trick to limit choices in output. See get_choice() #llm-ops #markdown

• Sun, Apr 14, 2024. Data Provenance Explorer lists open datasets used to train LLMs. #llm-ops

• Sun, Apr 14, 2024. Due to in-context learning, giving 100s of examples in the prompt can teach LLMs to jailbreak. Ref #future #llm-ops

• Tue, Apr 2, 2024. Video-Llava is a video LLM #future #llm-ops

• Wed, Mar 27, 2024. Representational engineering lets you control LLM output based on preference on the fly. #future #llm-ops #markdown #models #prompt-engineering

• Wed, Mar 13, 2024. Tavily is a search API for LLMs #llm-ops

• Mon, Feb 26, 2024. Llava 1.6 is a substantial improvement over Llava 1.5 and slightly better than CogVLM, Qwen-VL #llm-ops

• Fri, Feb 23, 2024. My view: LLMs are general purpose and more capable than SLMs. They'll win, like CPUs won over special-purpose chips. GPUs will optimize for LLMs and as usage grows, cost will fall. #gpu #llm-ops

• Sun, Feb 18, 2024. Google is using LLM powered bug identification #github #llm-ops

• Fri, Feb 9, 2024. Fine-tuning session by Dan. Notebook #gpu #llm-ops #markdown
  • Example of fine-tuning Mistral. Consumed 28 computes ($2.8)
  • Axlotl is what the top fine-tuned LLMs are trained on
  • Deepspeed provides distributed training
  • Flash attention lets data stay on GPU
  • Sample packing packs samples of different lengths into equal length tensors

• Sun, Jan 28, 2024. AppAgent is an LLM that can navigate mobile / web apps #code-agents #future #llm-ops

• Sun, Jan 21, 2024. llm-guard is an LLM content validation tool. #llm-ops

• Tue, Jan 16, 2024. Lilac is an LLM-based data curation tool. Use it to search by concept (e.g. PII, duplicates, etc.) and then drop/update the results. #ai-coding-tools #database #llm-ops

• Sat, Jan 13, 2024. To run an org-specific chatbot on your own LLM: (via awesome-chatgpt) #chatgpt #llm-ops
  • opengpts - but it doesn't support auth
  • chatbot-ui - but Supabase is hard to install
  • anse - but it doesn't support auth
  • ChatGPT-Next-Web - but it doesn't support auth

• Sun, Jan 7, 2024. CodeGPT is an alternative to Github Copilot that can use any LLM. #ai-coding-tools #chatgpt #code-agents #future #github #llm-ops

• Wed, Jan 3, 2024. LMQL provides a constraint-based query language for interacting with LLMs. It uses token masking, which is clever. #llm-ops

• Wed, Jan 3, 2024. Teachers are using LLMs to plan lessons, write emails to parents, create tests, adjust reading level of materials, personalize content with tools like MagicSchool, Diffit, Eduaide. #ai-coding-tools #future #llm-ops

• Tue, Jan 2, 2024. WizardLM creates datasets for instruction tuning by cleverly using LLMs to create new prompts. #llm-ops

• Mon, Dec 25, 2023. Mini-GPTs is an interesting approach to shrink LLMs and make them domain specific. It takes existing LLMs and removes neurons not used in a specific domain (e.g. law, medicine, etc.) #gpu #llm-ops

• Sun, Dec 17, 2023. LLM use cases by Benedict Evans #future #llm-ops
  • "Every text box on the internet will get an LLM"
  • "Infinite interns"
  • "Every UNIX function has become a company." "Every ChatGPT suggestion..."

• Sun, Dec 17, 2023. llm360 publishes models along with training datasets. #future #llm-ops #models

• Sun, Dec 17, 2023. Rubeus is a HTTP proxy for multiple LLMs with load-balancing, fallbacks and retries. #llm-ops

• Sun, Dec 17, 2023. GPTRouter is a Python interface for multiple LLMs with fallbacks and retries. #llm-ops

• Sun, Dec 10, 2023. LIDA from Microsoft is an LLM based data visualization tool. #llm-ops

• Thu, Dec 7, 2023. llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref #llm-ops

• Tue, Dec 5, 2023. Tools to identify personally identifiable information: #ai-coding-tools #llm-ops
  • galactic can use LLMs to detect PII
  • Presidio by Microsoft
  • Sherlock is a generic sematic type matching DL model
  • pii-extractor-llm was trained on Indian names
  • GLiNER is a Lightweight Generalist model for NER

• Sun, Dec 3, 2023. JupyterLab has a %%ai magic to use LLMs within notebooks. Ref #llm-ops

• Sun, Dec 3, 2023. Unsloth offers faster lower-memory LLM QLoRA finetuning #llm-ops #optimization

• Sun, Dec 3, 2023. DeepSeek is an open-source high-quality LLM #llm-ops #search

• Sun, Dec 3, 2023. SkyPilot lets you run LLMs on any cloud provider. #cloud #future #llm-ops

• Sun, Dec 3, 2023. vLLM lets you deploy LLMs with a single command. #llm-ops

• Sun, Dec 3, 2023. llamafile lets you run LLMs locally as a single file executable! #llm-ops

• Sat, Dec 2, 2023. Gwern Branwen says LLMs nudge his "... making heavier use of the languages I don’t know well (Emacs Lisp & Python) since I increasingly trust that an LLM can help me maintain them." #llm-ops

• Thu, Nov 30, 2023. When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref #ai-coding-tools #automation #code-agents #future #llm-ops #optimization #prompt-engineering

• Sat, Nov 25, 2023. Learn ML optimization approach - LLMOps. Have an evaluation framework with metrics like weights and biases or tensorboard. Helps figure out where fine tuning helps and where RAG does. Soon, this will become important. #llm-ops #markdown #optimization

• Sun, Nov 19, 2023. Gorilla LLM specializes in APPI calls: Torch Hub, TensorFlow Hub, HuggingFace #gpu #llm-ops

• Sun, Nov 19, 2023. Ollama runs local LLMs #llm-ops

• Wed, Nov 15, 2023. Tamil-Llama #future #llm-ops

• Mon, Nov 6, 2023. Architecture of today's LLMs is similar to the A16Z architecture #llm-ops

• Mon, Nov 6, 2023. vLLM runs HuggingFace transformers models faster. So does DeepSpeed #future #gpu #llm-ops

• Mon, Nov 6, 2023. 06 Nov 2023: It's possible to extract knowledge, ontologies and taxonomies from LLMs via symbolic distillation #llm-ops

• Sat, Nov 4, 2023. 04 Nov 2023: Running LLMs in production lists experiments to run. Validation CAN be automated! #code-agents #llm-ops

• Tue, Oct 24, 2023. 24 Oct 2023: Neo-sophia has decent videos on LLM strategy #future #llm-ops