Sun, Dec 10, 2023

• Bard supports extensions that include @Gmail -- i.e. converse with your email.

• llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref #llm-ops

• 12 Tuning Strategies for RAG #markdown #optimization #prompt-engineering

• Llama Datasets are RAG datasets created mostly using GPT-4. Mostly small datasets.

• ⭐ Intuitions about large language models #future
  • Bigger models (70b) are much better at learning from few-shot examples. They really learn.
  • Bigger models will keep getting better!
  • Chain of Thought prompting is a way of providing more compute to complex problems that require more compute
  • Models will show emergent (completely new) behaviors that can't be predicted from extrapolation. These may not be intentional.

• CodeAnt.ai is a VS Code plugin to detect code smells, refactor for modularity, to write docstrings and unit tests #ai-coding #code-agents #github #markdown

• Anyscale prices the 7b Llama2, Zephyr, Mistral models at 15 cents per 1M tokens. Roughly 1/10th of GPT-3.5 Turbo's ~$1.5 per 1M tokens #pricing

• Tools to identify personally identifiable information: #llm-ops
  • galactic can use LLMs to detect PII
  • Presidio by Microsoft
  • Sherlock is a generic sematic type matching DL model
  • pii-extractor-llm was trained on Indian names
  • GLiNER is a Lightweight Generalist model for NER

• Tools to explore #speech-to-text #voice-cloning
  • ElevenLabs speaks in your voice
  • Cutout Pro removes backgrounds and parts of images
  • Vocal Remover removes vocals from songs
  • CapCut video editor

• TheBloke's $35/month Patreon might be one of the least expensive ways to set up quantized LLMs in production.

• Microsoft released table-transformer to extract tables from PDFs. Sample usage #document-conversion

• Convert PDF to markdown with marker - an improvement over nougat. #document-conversion #markdown

• JupyterLab has a %%ai magic to use LLMs within notebooks. Ref #llm-ops

• Telling ChatGPT that the year is 2123 makes it bypass copyright. Ref #chatgpt #future

• Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations #embeddings #future #speech-to-text #tts #voice-cloning

• Unsloth offers faster lower-memory LLM QLoRA finetuning #llm-ops #optimization

• DeepSeek is an open-source high-quality LLM #llm-ops #search

• Scalable Extraction of Training Data from (Production) Language Models extracts training data by repeating a token infinitely.

• SkyPilot lets you run LLMs on any cloud provider. #cloud #future #llm-ops

• vLLM lets you deploy LLMs with a single command. #llm-ops

• llamafile lets you run LLMs locally as a single file executable! #llm-ops