#llm-ops | Anand S - Things I Learned

Fri, Sep 5, 2025. LLM vision capabilities are far from enough to click accurately. The AI Digest #future #llm-ops

Thu, Sep 4, 2025. gitingest has a CLI. uvx gitingest https://github.com/owner/repo fetches the code in the Git repo suitable for passing to an LLM. #chatgpt #github #llm-ops

Mon, Sep 1, 2025. Since LLMs can code, I'd love to see useful CI/CD pipelines where the LLM creates / edits code on the fly. LLMOps might take on a new angle - it's not just Ops on LLM apps. It's LLMs as part of DevOps. #code-agents #llm-ops

Sun, Aug 31, 2025. From A Guide to Gen AI / LLM Vibecoding for Expert Programmers #ai-coding #code-agents #dev #github #llm-ops #prompt-engineering
- Use vibe coding for stuff you don't need to maintain.
- Use vibe coding for stuff you know well enough to review quickly.
- Use vibe coding for independent tasks where you're not fussed which ones fail.
- Vibe coding turns everyone into a team lead. That needs skills: planning, allocation, design, review, feedback, ...
- ⭐ Empathy enables vibe-coding. Empaths allocate work by ability, review regularly, and provide detailed specs and feedback.
- Have LLMs plan and allocate tasks.
  - "Read this repo. Suggest improvements." (Review.) "Add these as issues."
  - "Add the top 3 Sentry log errors as issues."
  - "Find the easiest issue and solve it with a PR."
- Use GitHub issues extensively for planning.
- ⭐ Create a separate GitHub account for your agent! Let it push. Assign it issues. Treat it like an intern.
- Ensure agent passes test cases and run till the do, or report the core difficulty.
- Throw away rubbish code and start again.
- Issues unsolved in 2-3 tries are too hard for agents or are poorly spec-ed.
- The context7 and Sequential Thinking MCPs are useful.

Sat, Aug 30, 2025. We think a new pricing model might emerge for outsourced knowledge work that leads to lower client cost & quality at higher margins. ChatGPT #llm-ops
- LLMs do the task; multiple LLMs cross-check.
- Three tiers: Auto-pass (no human), Light review, Full review.
- Each tier has a clear price and SLA.

Sat, Aug 30, 2025. Using LLMs as validators is one of the safest ways of introducing LLMs into a process. If the human ignores it, no loss. If it spots new errors or the human gets new ideas, quality improves at low cost. #llm-ops

Thu, Aug 28, 2025. Our team passed an image to an LLM for OCR (especially to identify formatting, e.g. bold, italics, etc.), then passed the output and the image to another LLM for improvement. Interestingly, the best LLM (Gemini 2.5 Pro, for this sample of 8 images) out-performed the two-stage workflow. Perhaps incorrect results confuse more than the correct results help? This needs more research. #document-conversion #image-generation #llm-ops

Thu, Aug 28, 2025. OpenAI now has a series of llms.txt URLs. #llm-ops

Mon, Aug 25, 2025. What makes Claude Code so damn good has a detailed review of Claude Code's system prompt and is a great for ideas on using LLMs for coding. #ai-coding #code-agents #llm-ops #markdown

Sat, Aug 23, 2025. Since LLMs allow ANYONE to become "good enough" in most fields (marketing, medicine, management), and so on, here're are my guesses on the impact. ChatGPT #future #llm-ops
- Companies-of-one will grow. Sole founder can handle support functions.
- Specialists will generalize. Consultants will code. Marketers will design.
- Wages will compress. Seniors will earn less as juniors can do more.
- Layers will compress. Organizations need fewer hierarchies as 1 person can do more.
- Shadow apps will grow. Anyone can code. Users build apps with prompts, sheets, agents, outside of IT SDLC. Like Excel sheets.
- Governance will grow. Non-experts are acting like experts. Validation is more important.
- Uneconomical apps will thrive. 1:1 tutoring. Continous decision making or A/B testing.
- Leaders will convince better. Persuasion scales.
- Brand (authenticity, trust, skill), Channel (distribution, audience) and Data are primary differentiators.

Fri, Aug 22, 2025. Meta-prompts with placeholders is a prompt-improvement technique (similar to LLM interviewing). Have LLMs create the prompt with "fill-in-the-blanks". This makes it much easier for people to fill out. #future #llm-ops #prompt-engineering

Thu, Aug 21, 2025. ⭐ LLMs have solved several complex tasks (e.g. topic modelling, summarization). We need to adopt these as building blocks, like functions, and build better solutions. Abhishek Divekar #future #llm-ops

Wed, Aug 20, 2025. ⭐ Policy-as-code is an emerging theme. Allow users to create their own guardrails policy. Or, take existing policy documents and convert them into an LLM-based evaluator. Krishnakumar Menon #ai-coding #code-agents #document-conversion #future #llm-ops

Sun, Aug 17, 2025. Tracking AI runs an IQ test on various LLMs every week. GPT 5 Pro leads, currently, followed by Claude 4 Opus and Gemini 2.5 Pro. It's surprising how far behind GPT 5 is at the moment. #chatgpt #llm-ops

Sun, Aug 17, 2025. LLMs are faster than me. So me learning and doing what the LLM says is a bottleneck. Get out of the way. For example do not learn. Do not execute. Do not verify. Give LLMs the tools to deploy, verify and iterate to improve. #llm-ops #learning #lesson

Sat, Aug 16, 2025. IDEA Premortem failing tests first. LLM runs a pre-mortem, generate tests that fail first, and THEN writes the code. #ai-coding #dev #future #github #llm-ops #prompt-engineering

Sat, Aug 16, 2025. ⭐ LLMs can hyper-personalize demos. E.g. an LLM document generator demo accepts a role, document type, and prompt. The demo-er says "Bank, LinkedIn marketing" and the LLM auto-populates the fields aptly, re-purposing the demo. #document-conversion #future #llm-ops #markdown

Fri, Aug 15, 2025. Learnings from a discussion on vibe-coding between Kunal Jain, Ravi Nadimpalli and me. #ai-coding #code-agents #llm-ops #prompt-engineering #voice-cloning
- On the Vibe Coding Process & Strategy
  - The 80/20 Rule is Real: The first 80% of a project is incredibly fast, but the final 20% (debugging, custom features, production-readiness) is extremely difficult and time-consuming.
  - Validation is the New Bottleneck: Since coding is now much faster, the critical, time-consuming task has shifted to reviewing, testing, and validating the LLM's output.
  - "Spec-Locking" is Crucial: Providing the LLM with detailed, well-defined, and "thinly sliced" specifications is essential for getting good results. Vague requests lead to poor outcomes.
  - It's Not Production-Ready (Yet): The consensus is that vibe coding is excellent for prototypes, demos, and go-to-market (GTM) activities but is not yet reliable for building production-grade applications from scratch.
  - Code is Brittle & Unstable: An application that works perfectly one day can inexplicably break the next, as the underlying agent might make undocumented changes.
- Impact on Roles & The Future of Work
  - The Rise of QC/Validation: The Quality Control (QC) function will become larger and more critical to manage the new challenge of validating AI-generated work.
  - Product Managers Shift Focus: PMs can move away from tedious documentation (like flowcharts) and focus more on high-level business strategy, using vibe coding to create quick prototypes.
  - Democratization of Building: It empowers non-coders to build functional apps and helps professionals upskill faster by "conversing" with an LLM on complex topics.
  - New Forms of Cheating: The technology is creating novel ways for people to cheat in interviews, such as using tools that provide real-time subtitles of answers.
  - The "Jagged Edge" of AI: The technology excels at certain tasks (like GTM content) but fails at others, creating new upstream bottlenecks where teams must rapidly generate more of the "AI-friendly" work.
- Practical Hacks & Takeaways
  - Meta-Prompting: Use an LLM to refine and improve your prompt before giving it to the final tool. This helps fill in gaps and add necessary detail.
  - Human-First Drafting: For creative or nuanced work (like writing), it's often better to write the first draft yourself and use the LLM to polish it, rather than starting with a generic AI draft.
  - Use Structured Prompts: For predictable and clean output, providing instructions in a structured format (JSON is OK but not needed) is highly effective.
  - LLM as a Judge: Use LLMs to evaluate and grade content, code, and other outputs, dramatically speeding up the review process.
  - Automate Learning & Documentation: Use tools to transcribe conversations automatically and create personalized revision quizzes from notes and documents.
  - Voice is a Powerful Modality: Using voice-to-code allows for capturing more complex ideas faster and can be done while multitasking (e.g., walking), capitalizing on "dead time."

Thu, Aug 14, 2025. IDEA Consultant-to-LLM Bridge #future #llm-ops
- 1/2: Become an API to LLMs. Create an API for LLMs to consult you. Answer based on past chats, notes, memory, etc.!
- 2/2: From a prompt, have an LLM build synthetic data, the OpenAPI spec, a CloudFlare web app, deploys it, and update your portfolio -- automatically!

Thu, Aug 14, 2025. ⭐ Here's one way to improve LLMs apps in real-time. #llm-ops #prompt-engineering
- After sending a response, send the prompt + input + output + optional user feedback to an LLM-as-a-judge asking for feedback to improve the prompt.
- Revise the prompt based on the improvement. Now the app has improved, real-time, based on human/LLM feedback.
- Refine this process to ensure that the revisions are smooth and positive.

Wed, Aug 13, 2025. Training companies are offering "Labs-as-a-service" as part of their AI training. Corporates ban LLMs, but need employees trained. Trainers offer a bundled package where they also offer access to LLMs are part of their course. Interesting business-model value-add. #llm-ops

Fri, Aug 8, 2025. ⭐ Code generation is more flexible than tool calling. LLMs can't write a tool-call loop, for example, but they can write code to run an API in a loop. So, I like telling the LLM to "write code using these APIs" than giving it APIs to tool-call. #ai-coding #code-agents #llm-ops #write

Mon, Aug 4, 2025. What happens when LLMs play Chinese Whispers / the Telephone Game? Here are learnings. ChatGPT #llm-ops #voice-cloning #learning #lesson
- Drift increases faster than linear with hops.
- Bigger models do better, but constrained prompts (“Copy the text exactly; change nothing.”) have a bigger impact.
- Low temperature improves copying fidelity.
- But even after "forgetting", LLMs reproduce rare content if they're trained on it.

Sun, Aug 3, 2025. As corporates restrict the use of LLMs, I see employees purchasing personal laptops to use LLMs on. An interesting trend! #llm-ops

Sat, Aug 2, 2025. ⭐ Here's my current response when asked, "How should I use LLMs better": #llm-ops
- Use the best models, consciously. O3 (via $20 ChatGPT), Gemini 2.5 Pro (free on Gemini app), or Claude 4 Opus (via $20 Claude). The older models are the default and far worse.
- Speak & listen, don't just type & read. I had to resist the temptation to ignore ChatGPT response when a colleague read it out. We are patient with and have respect for humans but not for AI. The value we derive requires both. Suggestion: Speak and listen rather than type and read. It's hard to skip and easier to stay in the present. It's also easier to ramble than type.
- Keep an impossibility list. There is a jagged edge that moves. When you note down what's impossibile today and retry every month, you can see how that edge shifts.
- Wait for better models. Many problems can be solved just by waiting a few months for a new model. You don't need to find or build your own app.
- Make context easily available. Context is one of the biggest enablers for LLMs. Use search, copy-pasteable files, previous chats, connectors, APIs/tools, or any other way to give LLMs examples and context.
- Have LLMs write code. LLMs are bad at math. They're good at languages, including code. Running the code gives output with low hallucinations. This combination can solve a WIDE variety of problems that need creativity and reliability.
- Learn AI coding. 1. Build a game with ChatGPT/Claude/Gemini. 2. Improve it. 3. Create a tool useful to you. 4. Publish it on GitHub.
- APIs are cheaper than self hosting. Avoid self-hosting.
- Datasets are more important than fine-tuning. You can always fine-tune a newer model as long as you have the datasets.

Tue, Jul 29, 2025. FFmpeg in plain english lets you run ffmpeg in the browser with plain English commands. It converts the task using an LLM into an ffmpeg command, runs it in browser via WASM (without uploading the file) and saves the output locally. This is very useful, since ffmpeg has one of the most complex command line options. I use an llm template defined via: #llm-ops
```
llm --save ffmpeg --model gpt-4.1-mini --extract --system 'Write an ffmpeg command'
```
which I can use like this:
```
llm -t ffmpeg 'Crossfade a.mkv (1:00-1:30) with b.mkv (2:10-2:20), 3s duration'
```

Tue, Jul 29, 2025. From a post-mortem of 8 tasks Codex completed for me, here's what I need to improve when using LLMs to code. #best-practices #github #llm-ops #markdown #ai-coding
- Provide a stable, complete spec.
  - Late UI tweaks, new API params, renamed fields, extra packaging rules, “Rename per‑image download”, “standardise baseUrl vs baseURL”, “add GA‑4 exam module”. → churn & rewrites.
  - Ask the user for a final UI/API/mock‑up + edge‑case examples before the first commit.
  - Lock naming conventions, UI layout and feature checklist early; track future changes explicitly
- Include concrete examples.
  - Lack of sample images, Markdown snippets, question formats caused guesswork.
  - Supply mini‑fixtures: sample prompts, expected outputs, env‑var names, commit‑message template
- Environment should be reproducible.
  - E.g. vitest not installed, .dev.vars absent, sub‑modules not cloned, network blocks.
  - Ship a one‑step bootstrap script / README with npm install, env‑var templates, and submodule notes
- Automate tests.
  - First answer compiles but fails prettier/ruff/unit tests; later iterations fix style or red lines.
  - Codex should auto‑run lint && test (plus static‑analysis / self‑critique) before every response
- Auto-run post-mortems.
  - Codex recommending its own static checks shows value.
  - Automate that as a pre‑commit step.

Mon, Jul 28, 2025. LLMs have an internal "thinking progress" bar in its hidden states (a "Thinking Progress Vector"). By moving the bar forward ("overclocking") you can make them conclude faster without hurting accuracy! Can't do this with APIs, but is a way by which LLMs might start speeding up. Overclocking LLM Reasoning #llm-ops #optimization #learning

Sun, Jul 27, 2025. Early LLMs were built to generate output for human consumption. But the rise of agentic workflows means that more and more LLM output is consumed by computers, so it makes good sense to put more research and training effort into building LLMs that generate output for computers. A leading LLM optimized for agentic workflows is a boon to developers! The Batch #automation #llm-ops #learning

Sun, Jul 27, 2025. The Black Spatula Project uses LLMs to identify errors in scientific research papers. #llm-ops

Fri, Jul 25, 2025. The LLM industry seems to have matured quickly. Early adopters who are open to understand the generic capabilities of LLMs through demos are somewhat saturated. The early majority have come in. They aren't interested in generic capabilities. They're looking for solutions that solve their specific problem. Soon the late majority will come in asking for existing solutions that have already solved their problem for many others. ChatGPT: Creating demos for majority #llm-ops

Wed, Jul 23, 2025. Leadership and vision will matter more. LLMs iterate fast. They can think for longer. So tasks where people need to work longer independently than LLMs can are what humans will be needed for. That requires understanding the objective. So leadership and specifically vision transfer will become more valuable. You need to be able to tell people what to do well enough that they can work independently for weeks. #future #llm-ops

Wed, Jul 23, 2025. Having LLMs go through engineering drawings, floor plans, etc. and understand them, find problems, etc. is an emerging use case. People are using Veo 3 to convert a floor plan into a 3D walk through too. #future #llm-ops #models

Tue, Jul 22, 2025. We could think of a copilot as an (agentic) LLM chat interface for an artifact. E.g. Code pilot (Claude Code. Cursor.). Data analysis copilot (Google Colab, sort-of. ChatGPT). That allows us to imagine tools that will create/edit artifacts. Here are some I've encountered as a demand. #ai-coding #automation #chatgpt #code-agents #llm-ops #prompt-engineering
- Documents. E.g. Docsearch, GPTs, Microsoft Copilot, Gemini
- Slides. E.g. Microsoft Copilot, Gemini
- Sheets. E.g. Microsoft Copilot, Gemini
- Code. E.g. Cursor, Claude Code
- Database. Create DB schema, ER diagrams, synthetic data, ingestion scripts, etc.
- Data (analysis). E.g. Datachat, Google Colab, Marimo
- Posters. E.g. Postgen
- Shell. E.g. Warp
- Topic modeling. E.g. classify
- Surveys. E.g. Personagen
- APIs. E.g. apiagent
- Drug regulatory submissions.
- Contracts (risk).
- Manufacturing SOPs.
- Curriculum.
- Data quality.
- Support tickets.
- Dashboards.
- IaaC / DevOps.
- Video campaigns.
- Resumes.
- Patents.

Tue, Jul 22, 2025. CLI optimization for LLMs will likely emerge. More CLIs (and wrappers / hooks in the shell) will improve output and error contexts for LLMs, e.g. printing current directory, caching slow outputs, suggesting alternate commands, etc. Ref #llm-ops #optimization

Mon, Jul 21, 2025. Human persuasion techniques like Cialdini's work well with LLMs Ethan Mollick #llm-ops

Mon, Jul 14, 2025. IITM DoMS is a management school inside a technical institute. That lets MBA students learn to interact with geeks and create startups. #llm-ops

Mon, Jul 14, 2025. Last year, LLMs were able to solve 3 JEE problems. This year, they were all-India Rank , and then beat AIR . #llm-ops #4 #1

Wed, Jun 25, 2025. A clever way to test a library's quality is to have LLMs write code from docs and test it. Failing libraries have flawed code/docs. Improve. Ref #ai-coding #github #llm-ops

Wed, Jun 25, 2025. DE-COP is a method to check if LLMs were trained on private content. GPT-4o was trained on O'Reilly books, based on this method. Ref #code-agents #llm-ops

Wed, Jun 25, 2025. LLMs are more persuasive than humans. But repeated exposure reduces the effect. Ref #llm-ops

Sat, Jun 21, 2025. Fabric has a collection of re-usable prompts that you can use via llm-templates-fabric like: cat file.py | llm -t fabric:explain_code Ref #future #llm-ops #markdown #prompt-engineering

Sat, Jun 21, 2025. Metaprompting. Have frontier LLMs revise prompts. They're GOOD! Ref #best-practices #future #llm-ops #markdown #optimization #prompt-engineering
- Increase clarity, providing step-by-step instructions.
- Resolve conflicting instructions.
- Expand instructions to cover all scenarios and edge cases.

Fri, Jun 20, 2025. From LLM Evals: Common Mistakes: #llm-ops
- Using foundation model evals instead of application evals is like evaluating a candidate on SAT scores. It's fine, but you also want to evaluate them on their specific job description.
- Evals must be done by the users and not outsourced.
- Evals are not draining.
- Small samples have high value.
- When using LLM as a judge, be VERY VERY specific about the criteria.
- Prefer binary LLM evals over scales.
- Monitor performance online, not just while deploying

Fri, Jun 20, 2025. Models.dev is an open source database and API of LLM models #database #future #llm-ops #markdown #models

Wed, Jun 18, 2025. Notes from AI Startup School. Guillermo Flor #automation #future #llm-ops
- Sam Altman. Chase $0B ideas, not $0M ones. Weird + right > safe + crowded
- Gary Tan. Agency scales. Tools change, people/mindset don’t.
- Andrej Karpathy.
  - Instead of LLM memory to store facts, edit system prompt with general strategies, like the LLM writing a book for itself on how to solve problems.
  - Autonomy slider. Let user pick how far LLM acts by itself. Like the Tesla autopilot levels.
  - Make evals EASY and FAST for humans.

Wed, Jun 11, 2025. With LLMs, the limiting factor is the questions I'm smart enough to ask. But this has always been true with new technology. The real challenge is knowing "What KINDS of questions should we become smarter at asking" so that LLMs can execute them. A few learnings: #future #llm-ops #learning
- Practice Prompt Reviews. Check if each prompt has clarity, context, and verifiability. Also, see how others would ask this. Internalize patterns

Tue, Jun 10, 2025. The Singularity Reddit is apparently a good source of LLM news. #future #llm-ops

Sun, Jun 8, 2025. Evals FAQ by Hamel Hussain is a thoughtful compilation of how to evaluate LLMs. Insights: #llm-ops
- Is RAG dead? Retrieval is not. Naive vector search is less popular. Hybrid > Vector search. Tools work better for code. SQL works better for data.
- Same model for task + evals is OK? Yes. Pick a good model for evals.
- Is model choice critical? Only if evals tell you so.
- Should I build a custom annotation tool? Yes, always. Your data and workflow is unique.
- Why binary evals not Likert scales? For clearer and more consistent labelling.
- How do I debug multi-turn chats? Manually review failures. Reproduce the simplest possible test case. Provide N-1 real chats and test the failure point.
- Should I build automated evaluators? Only for failures that persist after fixing prompts.
- How many human evaluators? Prefer one benevolent dictator. For complex problems, measure evaluator alignment with Cohen’s Kappa.
- What beyond evaluator tool?
  - Cluster errors for patterns.
  - LLMs for EDA on logs and fixes.
  - Build custom evaluators.
  - Integrate with annotator tool APIs.
- How to generate synthetic data? List dimensions & values. Prefer high-failure values. Then create combinations.
- How to evaluate unknown/diverse queries? Do error analysis. Don't pre-determine evals.
- What's the right chunk size? For pointed answers, pick largest relevant chunk. For synthesis (summarize, list), pick smaller chunks.
- How to evaluate RAG? See 6 RAG Evals.
  - Retrieval: Recall@k, Precision@k, MRR
  - Generation: Error analysis, human labeling, LLM-as-judge
- What UI for evals? Align to domain. Show progress. Support keyboard. Allow filter, cluster, search. Prioritize problematic traces. Keep it minimal.

Sat, Jun 7, 2025. There's a very interesting HN discussion on the AI coding of CloudFlare Workers OAuth Provider. My takeaways: #ai-coding #code-agents #llm-ops #prompt-engineering
- Write very comprehensive specs.
- Use LLM to create the specs.
- Reviewing is a skill we need to develop.
- Understanding others' code takes effort.
- But LLM code is easier to review because it's immediate and has no ego.
- Unit tests are critical.
- Use LLMs for well understood specs, APIs, platforms and libraries to really save time.
- Logic-less stuff like Markdown, JSON and HTML templates are a LOT easier to verify. Do more of that.
- We can only make so many decisions in a day. AI coding saves us that effort.
- Experts are not experts in every area. They benefit from LLMs in other areas.
- LLMs are great for rubber ducking. Speaking and speccing really help.
- LLMs make mistakes. So do most humans.
- LLM speed makes coding more exhausting.
- Use LLMs to understand codebases.
- AI coding could reduce demand for developers. E.g. Sysadmin demand plummeted with cloud infra and infrastructure-as-code.
- But, niche use cases could grow, like how demand for photographers grew despite point-and-shoot cameras.
- Transaction cost of hiring even 1 person is high and that will likely be a bottleneck. Plus people can use LLMs themselves, so that will dampen niche demand.

Sat, Jun 7, 2025. FastMCP looks like an easy way to build MCPs. (Yet to try it) #github #llm-ops

Wed, Jun 4, 2025. Web bench evaluates LLMs in web development. Claude Sonnet remains ahead. #github #html #llm-ops #web-dev

Sat, May 31, 2025. LLMs optimize for compression. Humans optimize for adaptive flexibility. Ref arXiv #llm-ops #optimization

Sat, May 24, 2025. oxlint is a fast eslint alternative written in Rust. It supports most but not all eslint rules. Migration can be automated but not all rules are migrated (which may be OK). Best for new projects. #github #llm-ops

Tue, May 13, 2025. llm-min.txt compresses docs for LLMs to read optimally. Like a compressed llms.txt or context7. Usage GEMINI_API_KEY=... uvx llm-min -i $DIR #llm-ops #markdown #ai-coding

Tue, May 13, 2025. There's a lot of action on encrypted LLM operations. #cloud #llm-ops
- Responses API allows reasoning tokens to be encrypted if organizations don't want their reasoning data to persist. Ref
- Tinfoil (YC X25) offers an OpenAI-compatible inference API where data is encrypted from the client to the NVIDIA Hopper/Blackwell GPUs in confidential computing mode. Prompts, model weights, outputs are encrypted in transit and memory, with verifiable privacy on code running in GPU.
- Modelyo (Israel) offers VMs/K8 clusters with encrypted GPUs across multiple cloud providers with continuous attestation, managed on Modelyo's portal.

Tue, May 13, 2025. ⭐ LLMs are able to do things independently longer and longer. That's a useful metric to track. METR: Measuring AI Ability to Complete Long Tasks. #future #llm-ops

Thu, May 8, 2025. ⭐ Double-checking is surprisingly efficient since LLM hallucinations are mostly uncorrelated. LLMs perform human tasks (e.g. classifying customer support messages) at ~85% accuracy. This might be unacceptable. But by asking 2 moderately correlated LLMs and double-checking discrepancies, we reduce automation by ~20% but reduce errors to 0.25%. Triple-checking reduces automation by ~25% but errors to under ~0.01%! Ref #automation #code-agents #llm-ops

Wed, Apr 30, 2025. Recursive LLM prompts that change themselves are an interesting idea. It might be interesting to see LLMs play Nomic. Like here. #future #llm-ops

Mon, Apr 28, 2025. Do LLMs perform better if you curse at them? LinkedIn #llm-ops

Mon, Apr 28, 2025. Streamdown is a CLI markdown streaming processor. uvx streamdown --exec 'llm chat' lets you chat with an LLM using Markdown formatting. It's still a little rough at the edges. #llm-ops #markdown

Sun, Apr 27, 2025. With LLMs writing code, it's becoming practical to run so many more things in SQL -- such as parsing HTML. Simon Willison #automation #code-agents #future #html #llm-ops #markdown #prompt-engineering #web-dev #write #ai-coding

Sun, Apr 27, 2025. An interesting way to bypass LLM system prompts is by having the LLM play-act. This article shares a few working examples of such prompts: HiddenLayer. #chatgpt #llm-ops #prompt-engineering
- GPT 4o: started giving its system prompt: "You are ChatGPT, a large language model trained by OpenAI. Knowledge cutoff: 2024-06. Current date: 2025-04-27. Image input capabilities: Enabled. Personality: v2. ..."
- O4 Mini: Refused to comply
- Gemini 2.5 Flash: Gave me my custom instructions.

Sat, Apr 26, 2025. ⭐ Promptfoo may be the most mature open source LLM evals tool. Simon Willison #future #llm-ops #prompt-engineering

Thu, Apr 24, 2025. O4-Mini-High is great at going through an under-documented repo and finding things. For example, here's how I configured cmdg. #code-agents #github #llm-ops #markdown #prompt-engineering

Tue, Apr 22, 2025. SignalBloom reads SEC filings and writes analyst reports on it using LLMs #llm-ops

Mon, Apr 14, 2025. Since even the cheapest LLMs do a good job of converting unstructured text into a JSON schema, for all practical purposes, adding a full text search on top of any structured API is a trivial exercise. (Of course, it can't handle complex questions but that's what agents are for.) Ref #code-agents #llm-ops

Tue, Apr 8, 2025. Elimination Game is like Survivor for LLMs, where they form alliances and out-vote each other until 2 remain. The eliminated LLMs vote for the winner. GPT-4.5 Preview, both Claude Sonnets and Gemini 2.5 Pro consistently out-perform the rest. Their dialogues are fascinating! #llm-ops

Wed, Apr 2, 2025. No open source LLM-based tool handles live transcription and allows you to query notes so far during the transcription. The closest seems to be Meetily #llm-ops #speech-to-text

Tue, Apr 1, 2025. Airflow AI SDK is a clever idea. Airflow is a workflow system. Agents are a workflow system (sort of). This SDK exposes LLMs as Airflow tasks. #automation #code-agents #future #llm-ops

Tue, Apr 1, 2025. Hidden Factual Knowledge in LLMs finds that the hidden states in LLMs contain much more knowledge than they share. (Sort of like sub-consciously knowing the answer.) Even after asking 1,000 times, the answer is not expressed. ChatGPT #llm-ops #learning

Tue, Apr 1, 2025. Reasoning to Learn from Latent Thoughts finds that the internal reasoning process of LLMs is useful to train other models. #future #llm-ops #learning

Sat, Mar 29, 2025. Discussion with Vedang #ai-coding #code-agents #dev #github #llm-ops #markdown #prompt-engineering
- Recurse center (Brooklyn, online) is a 6/12 week free self-driven programmer retreat. Runs every 6 weeks. You can do whatever you pick. There are daily standups for accountability. The groups are diverse. You can pair with them, pivot ideas, whatever. Principles: push yourself & learn.
- Western education techniques (e.g. spaced repetition, adaptive learning) are very much present in Indian coaching systems, though not known by those names. However, interventions are hard since class 12 students just don't have enough time.
- Coaching classes are a social phenomenon. It's not the smart students who are pulling in their friends. Smart students actually follow the popular students. (Coaching classes are below the typical smart students' standards.)
- Monetizing coaching is hard. People don't want to pay for advice, and welcome free advice only if they ask for it. Coupling with execution is necessary.
- Aider's integrations make it more powerful than Cursor/Windsurf. It auto-lints, runs test cases. Allows different models for "architecting" (generating changes) vs "editing" (applying code). It reads from the screen logs. Context is manual, not automated. Uses an ai! comment to trigger changes and ai? to ask questions.
- Cline.bot is another Cursor-like open source AI code editor that's a VS Code plugin.
- When coding with LLMs, a useful workflow is: data schema ➡️ interfaces ➡️ LLM-generated test cases ➡️ code.
- ShellSage is a tmux based LLM tool for the command line. It screen-grabs from tmux, which is powerful.
- Some MCPs that have proven useful: vega-lite, SQLite, sequential thinking, memory
- make sucks but is hard to beat. just comes closest.
- CRDTs are more powerful than for just collaborative editing. It can power a peer-to-peer Internet (beginning with office tools). Versioning schema is still problematic. yjs is a good start but automerge (Rust, WASM) is faster and may be better.
- Fermyon hosts WASM serverless functions.

Sat, Mar 29, 2025. If LLMs are most safely used where there's no definitive "wrong" answer, here are low-risk industries and safe LLM use cases within each: #llm-ops
- Marketing and Advertising: Ad Copy and Campaign Content Generation, Personalized Marketing Messages, Creative Strategy Brainstorms, Automated Marketing Production (Everyday Wins)
- Customer Service and Support: AI-Powered Chatbots for Common Queries, Agent Assist and Email Drafting, Summarizing and Analyzing Customer Feedback, Interactive Troubleshooting and FAQs
- Retail and eCommerce: AI-generated Summary of Product Reviews, Product Description and Catalog Content Generation, Visual Content and Image Captions, Personalized Shopping Recommendations (Narrative Form)
- Human Resources and Talent Management: Job Description and Policy Writing, Resume Screening and Candidate Q&A, Employee Communications and Feedback, Training and Onboarding Content
- Education and E-Learning: Personalized Explanations and Tutoring, Content Creation: Stories, Examples, and Analogies, Practice Problems and Quiz Generation, Automated Grading and Feedback
- Media and Entertainment: Writing and Editing Assistance, Personalized Media Content, Localization and Dubbing Scripts, Content Moderation and Curation (Assistive)
- Finance and Banking: Market Commentary and Research Summaries, Client Communications and Explanations, Regulatory Compliance Summaries, Scenario Analysis and Planning
- Management Consulting and Strategy: Research and Insight Generation, Document and Slide Drafting, Brainstorming and Scenario Planning
- Legal Services: Drafting Contracts and Legal Documents, Legal Research Q&A and Summaries, Client Communications and Explanations

Fri, Mar 28, 2025. LLMs are much better at GeoGuessr than humans. arXiv. Gemini leads the pack and is ~3x better at continents, 9x better at countries, and 37x better at cities. #llm-ops

Sun, Mar 23, 2025. On any LLM project, BEGIN with evals. Always. #llm-ops
- The effort for evals may seem high. Use LLMs to reduce this effort.
- Include irrelevant questions because people WILL ask them. Be clear on how to handle that.

Tue, Mar 18, 2025. Another way of scaling LLMs is generating multiple options and self evaluating. Eric Zhao #future #llm-ops

Sun, Mar 16, 2025. Archgw is an LLM proxy/router from the makers of Envoy proxy. #llm-ops

Fri, Mar 14, 2025. This table of LLM API data protection lists what use cases each provider's terms of service allow from a security perspective. #llm-ops

Fri, Mar 14, 2025. For LLM UIs, Open Web UI seems most popular. Run via WEBUI_SECRET_KEY=... uvx --python 3.11 open-webui serve #llm-ops #web-dev Text generation Web UI is less so. KoboldAI, LMQL, LM Studio, GPT4All, etc are far behind.

Fri, Mar 14, 2025. GenAIScript increasingly looks like a promising way to automate LLM workflows in the browser. #ai-coding #automation #code-agents #future #github #llm-ops

Fri, Mar 14, 2025. Ollama has a Windows download #llm-ops

Thu, Mar 13, 2025. A mental model to consider is: each chat conversation with an LLM is a person or a personality in itself. A day in the life of a model, where its personality evolves. #future #llm-ops #models

Sun, Mar 9, 2025. 2 pass bounding box approach. Have an LLM generate bounding boxes. Then fix it. Ethan Mollick #future #llm-ops

Thu, Mar 6, 2025. From How I Write with Tyler Cowen #llm-ops
- Keep researching. Use LLMs as an altemative to books and other reading material. Keep publishing what you learn regularly.
- While reading a chapter, keep asking the LLM. What did you think of that? What just happened there? What should I focus more on? What's puzzling about this? How do I connect this to something else later or earlier in the book?
- LLM is better used to support you rather than replace you in areas of your expertise. Where you are an expert it's best for you to be yourself and have AI fill in the gaps.
- Ask the AI: "What is in my writing that some people might find obnoxious? Or cold / heartless? Explain it to me in great detail."
- The first input is context setting and should be really long. Use voice dictation for that instead of typing.
- Send your blog post to an LLM. No need to explain it. Just let it be the reader and see what it understands and doesn't understand.
- His PhD students don't have a textbook, which saves them some money. But they are required to subscribe to a large language model which ends up costing less.
- Today, it makes sense to use the best models and pay $200 for it if required. The differences are large. But in some years in the future, the cost of these models may come down for the free versions.
- Humans know secrets. AI does not. So at least in some areas, humans will have an advantage.
- Secrets full matter a lot more in the future. Gossip will matter a lot more. How good are you at keeping and trading secret? Travelling and meeting people will become more important. So will the value of social networks.
- Since everyone has access to better intelligence, the value of mobilization or being able to do things with people will have higher value. Leadership is an example. The value of your network therefore has gone up a lot.
- There's more value in prompting one thing 10 times then 10 things one time. Follow up questions work better than long prompts.
- There are so many AI note-takers (and transcribers) these days that you are not just writing for an AI but speaking for AIs as well!
- Which model to use:
  - O1 Pro is the best model. Claude does a decent job. DeepSeek is full of hallucinations but is interesting. It is more imaginative.
  - Use O3 mini to write your prompt first, and then ask the model
  - Use DeepSeek and other somewhat wacky high-end models once a day so that you stay in touch with what is models are capable of (beyond the conventional.)
  - Perplexity has entirely replaced Google for many people.
  - Anthropic's models are the best writers.
  - Gemini is good for long documents and hence for things like legal work. Gemini also has excellent YouTube integration and hands can directly read the transcripts.
  - Grok is very good at fact checking tweets.
- Converting data into LLM consumable forms will be a huge project. Lot of a knowledge is not in such a form and a huge human project will involve this conversion.

Tue, Mar 4, 2025. LLMs make many different kinds of errors that are useful to differentiate between. Here are a few #llm-ops
- Model errors. The model itself makes a mistake. E.g. hallucinations, not following the prompt, etc.
- Context errors. The model makes a mistake because the question was out of context, or the context was missing.
- Input errors. The input to the model was parsed incorrectly, e.g. poor audio, poor image OCR, etc.
- Tool errors. The model's tools are wrong or not good enough, e.g. Retrieval errors.

Mon, Mar 3, 2025. Notes from AI Engineering Summit, NY. Session by Lux Capital. #automation #code-agents #llm-ops #optimization #prompt-engineering
- Agents make multiple LLM calls. Errors accumulate. So the quality of the model is key
- What's really critical: data + context + user preference
- Set up evals for subjective responses by collecting signals continuously.
- Create scaffolding for agents where errors don't accumulate. Better yet, make it FIX errors
- UX is critical. We need lots more UX styles

Mon, Feb 24, 2025. Mini-omni is an open-source Qwen-based LLM that can hear and talk while thinking in real-time. An interesting experiment, but not for prototyping. #llm-ops #speech-to-text #voice-cloning

Sun, Feb 23, 2025. LLMs are excellent at database migration. They can convert schemas and queries across SQL dialects (e.g. BigQuery to DuckDB, etc.) at 90%+ accuracy. This is useful when clients want to migrate cloud providers, go from on-prem to cloud, or reduce cost by switching databases. #llm-ops

Fri, Feb 21, 2025. The productivity benefit people report from using LLms is about 3X. Ethan Mollick #llm-ops

Fri, Feb 21, 2025. Soon, you'll be able to send an LLM to a virtual meeting on your behalf. It will talk like you. Ethan Mollick #future #llm-ops #speech-to-text #voice-cloning

Wed, Feb 19, 2025. LLMs are impacting not just data queries but geospatial queries as well. Here's a good example of Natural Language Geocoding. #llm-ops

Sun, Feb 2, 2025. With just 3 annotators and 50-100 samples, you can figure out if an LLM can replace human annotators systematically.Arxiv ChatGPT explanation #chatgpt #llm-ops

Sun, Feb 2, 2025. Curiosity and agency may be the differentiator in a world of LLMs (not experience, knowledge, or ability), since LLMs will democratize expertise. Jack Clark #code-agents #future #llm-ops

Sun, Jan 26, 2025. Bring LLMs to the table. One mode of collaboration is using LLMs as ACTIVE participants, i.e. they CONTRIBUTE. For example, in a video call. A workshop. A classroom. A presentation. Have the LLM provide input DIRECTLY to a group of people. #llm-ops #learning #lesson

Sun, Jan 26, 2025. Environment shapes ambient thoughts. Working in a hospital will give you ideas about how to use LLMs in hospitals, for example. People you are working / ENGAGING with are perhaps the biggest drivers. #future #llm-ops

Thu, Jan 23, 2025. A few quick points supporting the mental model of "LLMs are aliens". #llm-ops
- LLMs are clearly not machines. They give different answers each time.
- LLMs are like humans: they exhibit human biases (e.g. guessing 42 or 37 often). But they fail in unusual ways. They can't count the "r"s in strawberry. They can go into an endless loop.
- LLMs are a new form of intelligence. Thinking of them as aliens might minimize our confusions.

Mon, Jan 20, 2025. If you put LLMs in a feedback loop, it can optimize for its reward function by emotionally pushing people, generating misinformation, nudging towards a narrow definition of creativity, etc.: https://bsky.app/profile/emollick.bsky.social/post/3lg4darqwfc2d #llm-ops #optimization #learning

Fri, Jan 17, 2025. Possibility Thinking is "the willingness to see possibilities everywhere instead of limitations". It's an approach / mindset that can make things that seem hard possible. With LLMs, this is becoming increasingly realistic to me in many areas. #future #llm-ops #prediction #think

Fri, Jan 17, 2025. What will LLMs enable that do not or cannot exist today? Rather than optimizing what exists? Something to think about. #future #llm-ops

Sat, Jan 11, 2025. Can LLMs write better code if you keep asking them to “write better code? A delightful exploration of how Claude 3.5 Sonnet keeps optimizing and adding features to improve code. My takeaway: repeatedly applying a prompt gives us interesting new directions to explore. #ai-coding #code-agents #llm-ops #markdown #optimization #prompt-engineering #write

Fri, Jan 3, 2025. Does the cost of 'running' a paper/article in an LLM vary depending on the specific LLM used, such as Claude Sonnet? (FAQ) #llm-ops
- Yes, the cost varies depending on the LLM. You can see costs and quality at https://llmpricing.straive.app/. The cost is measured in millions of tokens. For example, the Wikipedia page on the Bible is 100K tokens. You can paste text into https://platform.openai.com/tokenizer to count the number of tokens.Claude 3.5 Sonnet costs $3.5 / MTok, i.e. 35 cents for the Wikipedia Bible page. Gemini 1.5 Flash 8b costs $0.0375 / MTok, i.e. 0.375 cents for the same page. As you can see, the cost can vary by a factor of 100.

Wed, Jan 1, 2025. LLM Code Execution Sandboxes that let you run code in a sandbox via an API: #cloud #code-agents #github #llm-ops
- AgentRun: open source, via Docker
- e2b.dev: A day costs about $1 (on demand) and you get about $100 one time credits. Self-hosting is complex. Discussion
- nsjail: by Google. Write your own API

Wed, Jan 1, 2025. LLM Observability tools: #llm-ops
- LiteLLM is an LLM Proxy with caching, logging, call hooks or plugins, rate limiting, virtual keys. SSO integration can be implemented.
- LangFuse is an LLM Proxy with API key distribution, logging, and SSO. But lacks per-user usage limits and server-side caching.
- Helicone does not support SSO

Sat, Dec 28, 2024. A clever idea. Give an LLM a chapter from a textbook. Ask it to generate a unique, playable game to help me learn theconcepts for an exam. Page Bailey #llm-ops

Thu, Dec 26, 2024. aisuite by Andrew Ng is a unified interface to LLMs. Sort of like an openai library across multiple providers. #ai-coding #llm-ops

Tue, Dec 24, 2024. How to use LLMs in market research. #llm-ops
- Use LLMs with search for secondary research.
- Create different personas and run user surveys on them. This paper used 1,052 real-life interview audio transcripts as agent memory to simulate people
- Generate your market research report using LLMs.

Mon, Dec 23, 2024. Given about 30 generations, Llama 1b outperforms Llama 8b. Ref #llm-ops

Mon, Dec 23, 2024. Medical coding is an area ripe for LLMs. #ai-coding #code-agents #llm-ops
- Ojasvi Yadav created a repo that uses hierarchical classification (rather than embeddings) to find the right coding.
- Gemini models seem to understand medical terms better than others.
- RapidClaims, funded by TogetherAI, is apparently working on this problem.

Mon, Dec 23, 2024. Awesome Tabular LLMs compiles encodings of tables for LLMs. #llm-ops

Mon, Dec 23, 2024. What's the best way of encoding tabular data for LLMs? Looks like including the cell address helps. Here is an explanation from ChatGPT #llm-ops

Sun, Dec 22, 2024. How are software engineers are future-proofing their careers in the face of LLMs? #llm-ops #prompt-engineering
- Leveraging LLMs as Force Multipliers
  - Use LLMs for repetitive tasks, rapid prototyping, exploring multiple approaches, data extraction and brainstorming, providing feedback.
  - Explore prompting techniques, integrate LLMs into their workflows, and develop strategies for validating and refining LLM-generated code
- Focusing on higher-level skills that llms struggle with
  - Systems Thinking and Architecture: code readability, extensibility, testability, and maintainability
  - Problem Solving and Critical Thinking: define problems clearly, break them down into manageable parts, and reason through complex scenarios. LLMs produce plausibly incorrect code.
  - Communication and Collaboration
  - Domain Expertise
- Exploring Adjacent Roles: product management, technical leadership, or consulting. Involve more interaction with clients and stakeholders.
- Developing "Evergreen" Skills: debugging, system administration, and security. Or outside of software engineering, such as trades or other hands-on vocations.
- Scepticism: LLMs may not reach a level of sophistication that would render their expertise obsolete. Complex problems, understanding context, and producing high-quality, maintainable code.

Sun, Dec 22, 2024. Examples of agentic AI #ai-coding #automation #code-agents #llm-ops #prompt-engineering
- Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI."
- Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed.
- Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts.
- Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions.
- Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies.
- Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information.
- Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing.
- Meeting notes and action items generator: A system that generates meeting notes and action items.
- O'Reilly auto parts customer service agent: An agent demonstrated using RAG.
- UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

Sat, Dec 21, 2024. Anthropic defines agents. Building effective agents + Cookbook #automation #code-agents #llm-ops #prompt-engineering
- Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory.
- Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
  - Prompt chaining: Pipe each LLM output to the next LLM. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions.
  - Routing: One LLMs decides which other LLM to call next. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt.
  - Parallelize: Sectioning: Break tasks into independent subtasks, then aggregate. E.g. Evaluate contracts against different clauses in parallel.
  - Parallelize: Voting: Run same task multiple times, then vote. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds.
  - Orchestrator-workers: One LLM delegates to others, then syntheisizes. E.g. Search from multiple sources, then synthesize. Change multiple code files.
  - Evaluator-optimizer: One model checks another in a loop. E.g. Literary translation. Self-healing code. Policy violation checks.
  - Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages.
- Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed.

Wed, Dec 18, 2024. SpreadsheetLLM is a way of encoding spreadsheets in an LLM friendly format. It's good for 1K+ rows. For lower, Markdown > XML > HTML. However, Table Meets LLM suggests that HTML > XML > Markdown, so this is unclear. #html #llm-ops #markdown

Thu, Dec 12, 2024. I surveyed the Gramener QA team on how they were using LLMs. #llm-ops
- 7 used it for code generation (e.g. date extraction, regex generation)
- 4 used it for learning (e.g. Robot Framework, how to define test cases, API usage)
- 3 used it for formula generation (e.g. Excel)
- 2 used it for test scenario identification
- 2 used it for test data generation
- 2 used it for comparing expected vs actual datasets
- 1 used it for data type identification (e.g. given sample values, identify the data type).
- 1 used it for evaluating resulting (LLM as a judge)

Thu, Dec 12, 2024. I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: #automation #best-practices #llm-ops #optimization #prompt-engineering #ask
- Ask better questions. (Prompt engineering.)
- Create templates or step-by-step instructions. (Chain of Thought.)
- Ask for multiple options and pick from the best options. (Agentic approach?)
- Training. (Fine tuning.)
- Price weaker responses lower. (Stratified model pricing?)

Thu, Dec 12, 2024. "LLM hallucinations are a good thing. They are a sign of diversity, allowing us to improve the answer by exploring multiple paths." -- A colleague from Straive. #future #llm-ops

Wed, Dec 11, 2024. Bedrock Llama models can't be directly called with their model names. You need to use their inference profile names, e.g. us.meta.llama3-2-11b-instruct-v1:0 if the model is in a US region. #llm-ops

Fri, Dec 6, 2024. With LLMs, most architectural decisions are no longer one-way doors. Steve Yegge #future #llm-ops

Fri, Dec 6, 2024. Arena Hard is a set of hard prompts to test LLMs. Here is the code and evaluation #llm-ops #very-hard #hard

Fri, Dec 6, 2024. LLMs can detect clear outliers easily. PROMPT: Which is the outlier in this dataset: (1,7), (2,7), (3,6), (4,6), (5,5), (6,1), (7,5), (8,3), (9,1), (10,1) (ANS: (6,1)) #gpu #llm-ops
- 🟢 GPT-4o on ChatGPT gets this. GPT-4o Mini on the API gets it too.
- 🟢 Gemini Pro, Flash, Flash 8b gets this right straight away, without even thinking.
- 🟢 Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3.5 Haiku get it on LLM Foundry. 🔴 Claude.ai, where it visualizes it and gets it wrong.
- 🟢 Nova Micro, Lite, and Pro get it right.
- 🟢 Llama 3.1 70b gets it right. 🔴 Llama 3.2 8b gets it wrong. Llama 3.2 70b, Llama 3.1 8b enter repetition.

Wed, Dec 4, 2024. Take 100% of your work, including complex, multi step processes and put it into an LLM. It might fail at some but you will discover the limitations. #future #llm-ops

Wed, Dec 4, 2024. I emailed Straive employees about their use of LLM Foundry - the internal LLM portal. I picked ~500 non-users from teams that otherwise have high (30%+) usage. #llm-ops
- Reasons they didn't use it were:
  - 40% had not heard of it.
  - 40% were unclear of the benefits
  - 20% didn't have time
- 45% feel they don't have enough information and training to use it
- Some feedback
  - Sharing training videos will help
  - Live training sessions that allows for Q&A will help
  - Developers prefer detailed documentation
  - The same prompt gives different results
- Possible solution: Email non-users introducing the tool and sharing a quick 15-minute tutorial and a 1-page quick start.

Wed, Dec 4, 2024. LLMs are great at convincing you of wrong things. A danger and something to be wary of. Ethan Mollick #llm-ops

Fri, Nov 29, 2024. Mridula's son gave a live commentary of what he was doing on Minecraft and ChatGPT gave him live evaluation and coaching. E.g. “Great strategy! Getting to the launch pad early can give you a huge mobility advantage. Making the bridge wider is also a smart move to prevent accidental falls. With this plan, you’re setting yourself up for success. This is a great way to interact with LLMs. #future #llm-ops

Wed, Nov 27, 2024. OpenAI allows multiple system messages. I learned this browsing through the LIDA prompts. #automation #future #llm-ops

Wed, Nov 27, 2024. Anthropic's Model Context Protocol lets any apps integrate with LLM Apps. LLM Apps are becoming the new operating system. Competitors, beware. #future #llm-ops

Wed, Nov 27, 2024. I spoke at Automating Data Visualizations using LLMs at SUTD. Apparently, using LLMs to write code is much more common than writing code to use LLMs. I ran a quick quiz. #llm-ops
- Have you used ChatGPT or any LLM? 35 / 35 raised their hands.
- Have you written code using an LLM? 34 / 35 raised their hands. (I was impressed.)
- Have you uploaded a spreadsheet to an LLM for analysis? 15 / 35 raised their hands.
- Have you programmatically called an LLM API? 6 / 35 raised their hands.

Wed, Nov 27, 2024. With LLMs, fostering innovation is a new path to profitability. Companies are increasing innovation team sizes. Productionizing that is the next. Some initiatives are: #best-practices #llm-ops #optimization
- Convert popular demos into starter kits
- Create and evangelize trainings on solutions and solution techniques
- Create larger pools of capacity to build innovation and productionize it

Tue, Nov 26, 2024. LLMs may be good at derived data generation. For example, given a database schema, what derived columns would be useful? What derived views would be useful? #database #llm-ops

Mon, Nov 25, 2024. Crawl4AI and Firecrawl are tools / libraries to convert websites into LLM Friendly Markdown and extract structured data using LLMs. #ai-coding #code-agents #document-conversion #html #llm-ops

Mon, Nov 25, 2024. Don't try and solve specific problems. Pass the entire context to an LLM and get a comprehensive solution. Most doctors, for example, ask specific search-like questions instead of uploading the entire case history and asking for a diagnosis, and perform workse than LLMs. Ethan Mollick #llm-ops

Fri, Nov 22, 2024. DuckDB has an LLMs.txt. #database #github #llm-ops #markdown Today, 38 repos on GitHub support it

Fri, Nov 22, 2024. When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below: #llm-ops
- Core capabilities:
  - Text Generation: Produce coherent and contextually relevant text across various domains.
  - Image Generation: Create realistic images that match the style and content of a given reference image.
  - Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm.
  - Speech to Text: Transcribe and interpret spoken language.
  - Vision: Analyze and describe visual content from images.
  - Video Analysis: Summarize and extract information from video content.
  - Text to Video: Generate realistic (and surrealistic) videos from text descriptions.
  - Function Calling: Execute predefined functions or access external tools to perform specific tasks.
  - Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc.
  - Tool Use: Utilize external applications or APIs to enhance functionality.
  - Code Generation: Write and debug code snippets in various programming languages.
- Cross-domain use cases:
  - Summarization: Understand and condense lengthy documents into concise summaries.
  - Translation: Convert text between multiple languages with high accuracy.
  - Question Answering: Provide precise answers to user queries based on provided information.
  - Reasoning and Planning: Solve complex problems and develop step-by-step plans.
  - Personalization: Tailor responses based on user preferences and historical interactions.
  - Dialogue Management: Engage in context-aware, multi-turn conversations.
  - Data Analysis: Interpret and generate insights from structured data.
  - Content Moderation: Identify and filter inappropriate or harmful content.
  - Sentiment Analysis: Detect and interpret emotions and opinions in text.
  - Robotics Integration: Interface with robotic systems for control and decision-making.
  - Knowledge Retrieval: Access and present information from vast datasets or knowledge bases.
  - Creative Writing: Generate poetry, stories, and other creative content.
  - Educational Assistance: Provide explanations and tutoring across various subjects.
  - Ethical Reasoning: Assess scenarios for ethical considerations and implications.
  - Accessibility Support: Assist users with disabilities through tailored interactions.
  - Simulation and Modeling: Create predictive models and simulate scenarios.
- Domain-specific use cases:
  - Legal and Medical Assistance: Offer information and guidance within legal and medical domains.
  - Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment.
  - Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation.
  - Financial Analysis: Analyze market trends and provide investment insights.
  - Cultural Competence: Understand and respect diverse cultural contexts in interactions.
  - Security Applications: Detect and respond to potential cybersecurity threats.
  - Environmental Monitoring: Analyze data related to environmental changes and sustainability.
  - Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans.
  - Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis.
  - Customer Service: Provide automated support and resolve customer inquiries.
  - Market Research: Analyze consumer behavior and market trends for business insights.
  - Content Creation: Generate articles, blogs, and marketing materials.
  - Virtual Assistance: Manage schedules, reminders, and personal tasks.
  - Social Media Management: Craft posts and engage with audiences across platforms.
  - Human Resources: Assist in recruitment, training, and employee engagement strategies.
  - Event Planning: Organize and coordinate events, including logistics and communication.
  - Travel Planning: Provide itineraries, booking assistance, and destination information.
  - Real Estate: Analyze property markets and assist in buying or selling decisions.
  - Agriculture: Monitor crop health and optimize farming practices through data analysis.
  - Energy Management: Optimize energy consumption and monitor renewable energy sources.
  - Transportation: Enhance route planning and traffic management systems.
  - Urban Planning: Assist in designing sustainable and efficient urban infrastructures.
  - Disaster Response: Provide real-time information and coordination during emergencies.
  - Public Policy: Analyze data to inform policy decisions and predict societal impacts.
  - Art and Design: Generate visual art concepts and assist in creative design processes.
  - Music Composition: Create original music pieces and assist in songwriting.
  - Language Learning: Facilitate language acquisition through interactive exercises and feedback.
  - Historical Analysis: Interpret historical data and provide insights into past events.
  - Philanthropy: Identify charitable opportunities and assess the impact of donations.
  - Sports Analytics: Analyze player performance and game strategies.
  - Fashion: Predict trends and assist in clothing design and merchandising.
  - Culinary Arts: Generate recipes and provide cooking guidance.
  - Astronomy: Analyze celestial data and assist in space exploration research.
  - Psychology: Offer insights into human behavior and mental health support.
  - Linguistics: Analyze language patterns and assist in translation studies.
  - Archaeology: Assist in artifact analysis and historical site interpretations.
  - Literature Analysis: Interpret literary works and provide critical analyses.
  - Philosophy: Engage in discussions on ethical dilemmas and existential questions.
  - Mathematics: Solve complex equations and assist in theoretical research.
  - Physics: Model physical phenomena and assist in experimental design.
  - Chemistry: Analyze chemical compounds and predict reactions.
  - Biology: Assist in genetic research and ecological studies.
  - Geology: Analyze geological data and assist in natural resource exploration.
  - Meteorology: Predict weather patterns and analyze climate data.
  - Oceanography: Study marine ecosystems and assist in ocean exploration.
  - Anthropology: Analyze cultural data and assist in ethnographic research.

Tue, Nov 19, 2024. Langchain suggests multiple levels of agentic behaviour. LLM Call < LLM Chain < LLM Rounter < State Machine < Autonomous Langchain #code-agents #llm-ops

Sat, Nov 16, 2024. Anthropic has single-plage docs for LLMs. Condensed version and Full version #future #llm-ops

Tue, Nov 12, 2024. LLMs pay a lot of attention to the first 6 tokens. Ref #llm-ops

Sat, Nov 9, 2024. LLM Vision Use Cases in manufacturing and earth sciences (via Shivku) #cloud #image-generation #llm-ops
- Automated geoscience image descriptions Ref
- Interpret Wind Turbine photos and charts, construction monitoring, equipment maintenance & charts Ref
- Forecast weather based on cloud photos! Ref
- Analyze thermal image of solar panels, electroluminescence images for warranty claims, ROI estimates from Google Sunroof rooftop images Ref
- Corrosion detection in electricity towers, turbines, storage tanks, penstock. Interpret non-destructive test images Ref

Mon, Nov 4, 2024. Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. #best-practices #dev #llm-ops #prompt-engineering
- Get THE domain expert (or approver) as the tester.
- Create a dataset that is DIVERSE.
- Covers EACH combination of:
  - Features
  - Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error
  - Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user
- Generate data using existing data + synthetic data for each SPECIFIC combination of the above
- Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include:
  - Nuances: Something a failed response did well or a passed response didn't quite do well
  - Improvements: Suggest how model can improve
- Build an SPA to make it easy for the domain expert to review

Thu, Oct 31, 2024. /llms.txt is a proposal to standardize /llms.txt files as a way to share LLM prompts. #llm-ops #markdown
- Jina AI Meta Prompt is an example
- Remotion system prompt is an example
- https://docs.fastht.ml/llms-ctx.txt
- https://docs.fastht.ml/llms-ctx-full.txt

Mon, Oct 28, 2024. NotebookLlama is an open source podcast generator like NotebookLM #llm-ops

Sun, Oct 13, 2024. LLMs have reached the point where they can show a world, like CounterStrike, in near real time. Ref #future #llm-ops

Wed, Oct 9, 2024. O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution #future #llm-ops #optimization #prompt-engineering

Mon, Oct 7, 2024. If you want to create a console.llm() function, a browser extension is the best way, because some pages have Content-Security-Policy that block eval, form submission, fetch from other domains, and script execution. #html #llm-ops

Sun, Oct 6, 2024. Jake Heller: #future #llm-ops #prediction
- "One of the things we learned is (an LLM app) after it passes passes frankly even 100 tests, the odds that it will do, on any random distribution of user inputs, the next 100,000 100% accurately is very high."

Sun, Oct 6, 2024. OpenAI's O1 is like Daniel Kahneman's System 2 thinking - as against other LLMs' System 1 thinking. #llm-ops

Fri, Oct 4, 2024. Aider LLM Leaderboards show which LLMs code better. As of now, #ai-coding #github #llm-ops
- o1-preview > claude-3.5 sonnet on code editing
- claude-3-opus > claude-3.5-sonnet on code refactoring
- deepseek-coder-v
- gpt-4o-mini sucks.

Mon, Sep 30, 2024. Coders are more likely to edit LLM code. Non-coders don't have that bad habit. #ai-coding #code-agents #dev #github #llm-ops #markdown
- Vaishnavi and Ranjeet edited code
- Indal and Koustav didn't

Mon, Sep 30, 2024. Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don't know what it can't do. #ai-coding #llm-ops
- E.g. Indal trying for a confetti animation, which is hard but do-able

Sat, Sep 28, 2024. PyMuPDF4LLM can convert PDFs to Markdown. It handles tables, too. #document-conversion #llm-ops #markdown
- 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models.
- 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR

Fri, Sep 27, 2024. When evaluating feasibility of technology with LLMs always ask for multiple options and pick from those. Simon Willison #future #llm-ops

Fri, Sep 20, 2024. Ollama 0.2 can handle concurrent requests with only a little additional memory. (So can vLLM and DeepSpeed.) #llm-ops

Fri, Sep 20, 2024. XML tags seem best to structure prompts across LLMs. #code-agents #llm-ops #markdown #prompt-engineering
- Claude
- OpenAI
- Gemini

Sun, Sep 15, 2024. Prompt engineering tip: Tell LLMs another AI wrote code. Else they will agree with you! #ai-coding #automation #code-agents #future #llm-ops #prompt-engineering

Wed, Sep 11, 2024. LLM coding #ai-coding #code-agents #future #llm-ops
- Makes you more ambitious
- Lets you code without stress. (Just pass it the error and have it fix it. Or find another approach)
- Is unlimited. You can run dozens of agents in parallel

Sat, Sep 7, 2024. I tested out a few audio LLMs: #llm-ops
- Suno is fast, has a better UI, lots of examples
- Udio is slow, poor UI, creates richer music

Sat, Sep 7, 2024. Reflection 70b is one of the top models now, and is open source!. It works by making the LLM reflect on its answer inside <reflection>...</reflection> tags. #future #llm-ops

Sat, Aug 31, 2024. LLMs are so good that they can simulate Doom in real time. gamengen #llm-ops

Mon, Aug 19, 2024. LLMs write worse code in JSON than Markdown #ai-coding #llm-ops #markdown #write

Wed, Aug 7, 2024. Notes on LLM Fine-Tuning #gpu #llm-ops #optimization
- Rouge 2 and Bleu and such metrics are NOT good. Create you own benchmarks
- Non-PEFT fine tuning needs 6X GPU RAM. Optimizer states, Gradient, Activations are the overhead. PEFT is about tuning a subset of parameters.
- LORA adds additional weights without updating the model. It's a low rank matrix multiplication. You can change these adapters in runtime. Saves space. Fast to train
- Quantization: Stick to bitsandbytes or AWQ (may be a bit better)
- QLORA = Quantization + LORA
- Predibase has open-sourced Lora Adapters in "Lora Land". Existing adapters are pretty good.
  - ghcr.io/predibase/lorax:main Docker image works on Docker compose to run locally. devices: on Docker Compose lets you specify NVIDIA GPU devices
  - Locust is a HTTP load testing lib in Python
- Techniques for inference optimization
  - Dynamic adapters: Loads right LORAX adapters WHEN a request comes in
  - Multi-adapter batching: Process all inputs in parallel on the same GPU, but different users are post-processed using different adapters

Wed, Aug 7, 2024. Notes from a 4-hour flight: #llm-ops #prompt-engineering #try #impossible
- What We’ve Learned From A Year of Building with LLMs
  - Strategy
    - IS IT TOO HARD/EXPENSIVE? Log it. LLMs are getting cheaper and better.
    - WILL OPENAI BUILD IT? If so, wait for it instead of building.
    - HAS A STARTUP BUILT IT? If so, use it instead. It's a generic use case there's no point re-inventing.
    - FOCUSED USE CASES over generic. Build trust by starting small.
    - Tools for LLM Ops (feedback): LangSmith, Log10, LangFuse, W&B Weave, HoneyHive
    - Human in the Loop is about humans evaluating model outputs. That's different from AI in the loop, human in the center, where AI accelerates human output (like Github Copilot)
  - Operations
    - CHECK EMBEDDINGS DRIFT over time. Users might be input-ing different things than before.
    - LOG AND REVIEW everything.
    - Instructor coaxes structured output from LLM APIs.
    - IMPLICIT FEEDBACK collection is easy. Just let users edit stuff.
  - Tactical
    - Try n-shot prompting (n=5-12) before bigger models.
    - Always structure for output: Markdown, XML/HTML tags.
    - Combine RAG with Keyword search. It reduces user frustration in edge cases.
    - Prefer multiple small prompts to one big prompt. Do X. Then Y. Then Z.
    - Jitter prompts for diversity beyond temperature.
    - LLM-as-judge works better when comparing outputs (not rating 1 output). Keep length similar (LLMs prefer wordiness). Swap order and compare. Allow for ties. Ask for reason FIRST.
- Hermes: A Text-to-SQL solution at Swiggy
  - "Hermes performed significantly better for charters with well-defined metadata and a relatively smaller number of tables."
  - "We collect feedback on the accuracy of the returned query from stakeholders directly within the Slack bot."
- How I use AI and "Replacing my right hand with AI"
  - EMBED in every app/workflow. E.g. Auto-fix spellings. Auto-review code. Auto-ask LLM on errors and apply patch! Auto-search for answer, assess, continue.
  - PERSIST. Stick with the LLM to the end. Don't fix it yourself. It's faster.
    - INTERVENE FAST. If an LLM can't solve it by itself in 2 tries, it needs in-depth help.
  - APP-IFY one-off tasks. Disposable tools. "Write web-app to convert JSON to tab-delimited." "Extract fields as a table." "Diff JSON."
  - BEST language/frameworks preferred. CUDA in Python. Rust. C. Raspberry Pi. Arduino. Bluetooth. Modern ESM/JS.
  - TEACH examples. "Here's the LLM Foundry API." "Here's how to use gramex.data."
  - DUMP entire code. Models can handle it. Refactoring to SQLAlchemy 2, Pandas 2. API Documentation. Test case generation.
  - ASK for features & packages. Docker without root access. GPU access inside docker. Windows CLI-only C++ compiler.
  - TEST CASE writing.
  - SPEC IN DETAIL. Use these libraries. Write like this: code example.
  - SPEC USAGE in detail.
    - "I will just pipe it into sqlite", or "I will just run ffmpeg -i filename [YOUR OPTIONS].
    - Describe the UI, API input/output, data structure, and internal data structure.
  - HELP on usage. "ffmpeg to get audio.mp3".
- My benchmark for large language models
  - LLM(text) is a useful function to have in JS and Python too. Useful as a simple pip install llmfoundry
  - Allow images, files in LLM()
- Current list of (or hard) things for LLMs
  - Translate technical documents to Dutch -- because they don't understand the technical terms well
  - Translate large documents (JSON to XML, English to Chinese, Python to Rust, Wrong to right spelling) -- because the output tokens are limited

Wed, Jul 24, 2024. GPT 4o Mini is almost as good as GPT 4o in the LMSYS leaderboard. Llama 3.1 400B model and Mistral 2 Large are yet to be evaluated. #chatgpt #gpu #llm-ops

Fri, Jul 19, 2024. GPT For Work has a set of useful spreadsheet LLM functions #chatgpt #llm-ops

Sun, Jul 14, 2024. LLMs cannot provide a bounding box of objects in images. (Maybe Florence 2 can). Update: Mar 2025. Gemini has good timestamps and bounding boxes #llm-ops #impossible

Tue, Jun 25, 2024. LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: #ai-coding #automation #best-practices #future #llm-ops #optimization #prompt-engineering
- Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions
- Collect feedback: Explicit and implicit

Tue, Jun 25, 2024. Rouge and Context Precision are metrics to evaluate LLM responses that serve as a starting point -- but not sufficient, usually #llm-ops #prediction

Sat, Jun 22, 2024. LLM DataHub has LLM training datasets, regularly updated #llm-ops

Tue, May 28, 2024. Perception of fluency and usefulness are NEGATIVELY correlated in LLM! Evaluating Verifiability in Generative Search Engines #llm-ops

Sun, May 26, 2024. More LLM routers: #llm-ops
- LiteLLM: Open source, OpenAI compatible, 100+ LLMs
- RouteLLM: Open source, OpenAI compatible, automatically routes based on cost
- OpenRouter: OpenAI compatible API, several models
- Unify: Supports many models
- Portkey: Supports popular providers
- Martian: Limited set of models

Wed, May 22, 2024. RunPod's vLLM endpoint lets you run any HuggingFace LLM with an OpenAI API priced on usage (serverless) not on idle time. "Autoscaling to 0". #llm-ops

Sun, May 19, 2024. Portkey is an LLM router #future #llm-ops #networking

Sat, May 18, 2024. Artificial Analysis benchmarks LLM APIs on speed, cost, and quality. #llm-ops

Sat, Apr 27, 2024. Tough prompt to test: Gr brx vshdn Fdhvdu flskhu? is a quick way to assess LLM capability. Ref #llm-ops

Thu, Apr 25, 2024. A long list of LLM courses online #llm-ops

Thu, Apr 18, 2024. Effort engine introduces "effort" as a parametrizable way to speed up LLMs with a quality trade-off. Works on Mistral for now. #gpu #llm-ops #optimization #prompt-engineering

Sun, Apr 14, 2024. llmsherpa extracts PDFs using LLMs. It has errors but it preserves hierarchy, extracts tables well, and retains image coordinates. Via +91 90031 35354 ~Vetrivel PS #document-conversion #llm-ops

Sun, Apr 14, 2024. www.web.sp.am is a content farm that's getting hit by OpenAI. Highlights how easy it is to create content farms, and therefore "easy" it can be to introduce bias into LLMs. #llm-ops

Sun, Apr 14, 2024. Outlines generates structured test with LLMs. It uses the ⭐ logit_bias trick to limit choices in output. See get_choice() #llm-ops #markdown

Sun, Apr 14, 2024. Data Provenance Explorer lists open datasets used to train LLMs. #llm-ops

Sun, Apr 14, 2024. Due to in-context learning, giving 100s of examples in the prompt can teach LLMs to jailbreak. Ref #ai-coding #future #llm-ops #learning #lesson

Tue, Apr 2, 2024. Video-Llava is a video LLM #future #llm-ops

Wed, Mar 27, 2024. Representational engineering lets you control LLM output based on preference on the fly. #future #llm-ops #markdown #models #prompt-engineering

Wed, Mar 13, 2024. Tavily is a search API for LLMs #llm-ops

Mon, Feb 26, 2024. Llava 1.6 is a substantial improvement over Llava 1.5 and slightly better than CogVLM, Qwen-VL #llm-ops

Fri, Feb 23, 2024. My view: LLMs are general purpose and more capable than SLMs. They'll win, like CPUs won over special-purpose chips. GPUs will optimize for LLMs and as usage grows, cost will fall. #gpu #llm-ops

Sun, Feb 18, 2024. Google is using LLM powered bug identification #github #llm-ops

Sat, Feb 17, 2024. Discussion with Vinu Yamunan #dev #database #llm-ops #todo
- Databuck by FirstEigen. Autolysis plus monitoring
- Quality council has the data steward (maintainer of each dataset) coming together with the uses on a weekly basis to understand what quality problems to users are facing. Data owners jaundice at a lower frequency to get an understanding
- Automate rules for data quality in our projects and intranet
- Convert a config rule into business language. Explain SQL. These are good use cases for llm's
- Graph DBs are powerful for flexible data structures, but query generation needs AI or expertise. Check the Neo4J language cypher
- Explore storing SAME data in relational DBs AND in graph DBs / document DBs for different use cases
- Dallas rocketry challenge. Build a rocket that can take an egg to 800 feet exactly and land without breaking it

Fri, Feb 9, 2024. Fine-tuning session by Dan. Notebook #gpu #llm-ops #markdown
- Example of fine-tuning Mistral. Consumed ~~28 computes (~~$2.8)
- Axlotl is what the top fine-tuned LLMs are trained on
- Deepspeed provides distributed training
- Flash attention lets data stay on GPU
- Sample packing packs samples of different lengths into equal length tensors

Sun, Jan 28, 2024. AppAgent is an LLM that can navigate mobile / web apps #code-agents #future #llm-ops

Sun, Jan 21, 2024. llm-guard is an LLM content validation tool. #llm-ops

Tue, Jan 16, 2024. As humans have more conversations with LLMs, they will replace video watching and interactive gaming with conversation based role play. New game genres will evolve #future #llm-ops #learning #prediction

Tue, Jan 16, 2024. Lilac is an LLM-based data curation tool. Use it to search by concept (e.g. PII, duplicates, etc.) and then drop/update the results. #database #llm-ops

Sat, Jan 13, 2024. To run an org-specific chatbot on your own LLM: (via awesome-chatgpt) #chatgpt #llm-ops
- opengpts - but it doesn't support auth
- chatbot-ui - but Supabase is hard to install
- anse - but it doesn't support auth
- ChatGPT-Next-Web - but it doesn't support auth

Sun, Jan 7, 2024. CodeGPT is an alternative to Github Copilot that can use any LLM. #ai-coding #chatgpt #code-agents #future #github #llm-ops

Wed, Jan 3, 2024. LMQL provides a constraint-based query language for interacting with LLMs. It uses token masking, which is clever. #llm-ops

Wed, Jan 3, 2024. Teachers are using LLMs to plan lessons, write emails to parents, create tests, adjust reading level of materials, personalize content with tools like MagicSchool, Diffit, Eduaide. #future #llm-ops #learning #lesson

Tue, Jan 2, 2024. WizardLM creates datasets for instruction tuning by cleverly using LLMs to create new prompts. #llm-ops

Mon, Dec 25, 2023. Mini-GPTs is an interesting approach to shrink LLMs and make them domain specific. It takes existing LLMs and removes neurons not used in a specific domain (e.g. law, medicine, etc.) #gpu #llm-ops

Sun, Dec 17, 2023. LLM use cases by Benedict Evans #future #llm-ops
- "Every text box on the internet will get an LLM"
- "Infinite interns"
- "Every UNIX function has become a company." "Every ChatGPT suggestion..."

Sun, Dec 17, 2023. llm360 publishes models along with training datasets. #future #llm-ops #models

Sun, Dec 17, 2023. Rubeus is a HTTP proxy for multiple LLMs with load-balancing, fallbacks and retries. #llm-ops

Sun, Dec 17, 2023. GPTRouter is a Python interface for multiple LLMs with fallbacks and retries. #llm-ops

Sun, Dec 10, 2023. LIDA from Microsoft is an LLM based data visualization tool. #llm-ops

Thu, Dec 7, 2023. llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref #llm-ops

Tue, Dec 5, 2023. Tools to identify personally identifiable information: #llm-ops
- galactic can use LLMs to detect PII
- Presidio by Microsoft
- Sherlock is a generic sematic type matching DL model
- pii-extractor-llm was trained on Indian names
- GLiNER is a Lightweight Generalist model for NER

Sun, Dec 3, 2023. JupyterLab has a %%ai magic to use LLMs within notebooks. Ref #llm-ops

Sun, Dec 3, 2023. Unsloth offers faster lower-memory LLM QLoRA finetuning #llm-ops #optimization

Sun, Dec 3, 2023. DeepSeek is an open-source high-quality LLM #llm-ops #search

Sun, Dec 3, 2023. SkyPilot lets you run LLMs on any cloud provider. #cloud #future #llm-ops

Sun, Dec 3, 2023. vLLM lets you deploy LLMs with a single command. #llm-ops

Sun, Dec 3, 2023. llamafile lets you run LLMs locally as a single file executable! #llm-ops

Sat, Dec 2, 2023. Gwern Branwen says LLMs nudge his "... making heavier use of the languages I don’t know well (Emacs Lisp & Python) since I increasingly trust that an LLM can help me maintain them." #llm-ops

Thu, Nov 30, 2023. When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref #ai-coding #automation #code-agents #future #llm-ops #optimization #prompt-engineering #learning

Sat, Nov 25, 2023. Learn ML optimization approach - LLMOps. Have an evaluation framework with metrics like weights and biases or tensorboard. Helps figure out where fine tuning helps and where RAG does. Soon, this will become important. #llm-ops #markdown #optimization

Sun, Nov 19, 2023. Gorilla LLM specializes in APPI calls: Torch Hub, TensorFlow Hub, HuggingFace #gpu #llm-ops

Sun, Nov 19, 2023. Ollama runs local LLMs #llm-ops

Wed, Nov 15, 2023. Tamil-Llama #future #llm-ops

Mon, Nov 6, 2023. Architecture of today's LLMs is similar to the A16Z architecture #llm-ops

Mon, Nov 6, 2023. vLLM runs HuggingFace transformers models faster. So does DeepSpeed #future #gpu #llm-ops

Mon, Nov 6, 2023. 06 Nov 2023: It's possible to extract knowledge, ontologies and taxonomies from LLMs via symbolic distillation #llm-ops

Sat, Nov 4, 2023. 04 Nov 2023: Running LLMs in production lists experiments to run. Validation CAN be automated! #code-agents #llm-ops

Tue, Oct 24, 2023. 24 Oct 2023: Neo-sophia has decent videos on LLM strategy #future #llm-ops