Sun, Dec 10, 2023

Weekly Notes and Discoveries | Recall | S Anand

Bard supports extensions that include @Gmail -- i.e. converse with your email.

llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref #llm-ops

12 Tuning Strategies for RAG #markdown #optimization #prompt-engineering

Llama Datasets are RAG datasets created mostly using GPT-4. Mostly small datasets.

⭐ Intuitions about large language models #future
- Bigger models (70b) are much better at learning from few-shot examples. They really learn.
- Bigger models will keep getting better!
- Chain of Thought prompting is a way of providing more compute to complex problems that require more compute
- Models will show emergent (completely new) behaviors that can't be predicted from extrapolation. These may not be intentional.

CodeAnt.ai is a VS Code plugin to detect code smells, refactor for modularity, to write docstrings and unit tests #ai-coding #code-agents #github #markdown

Anyscale prices the 7b Llama2, Zephyr, Mistral models at 15 cents per 1M tokens. Roughly 1/10th of GPT-3.5 Turbo's ~$1.5 per 1M tokens #pricing

Tools to identify personally identifiable information: #llm-ops
- galactic can use LLMs to detect PII
- Presidio by Microsoft
- Sherlock is a generic sematic type matching DL model
- pii-extractor-llm was trained on Indian names
- GLiNER is a Lightweight Generalist model for NER

Tools to explore #speech-to-text #voice-cloning
- ElevenLabs speaks in your voice
- Cutout Pro removes backgrounds and parts of images
- Vocal Remover removes vocals from songs
- CapCut video editor

TheBloke's $35/month Patreon might be one of the least expensive ways to set up quantized LLMs in production.

Microsoft released table-transformer to extract tables from PDFs. Sample usage #document-conversion

Convert PDF to markdown with marker - an improvement over nougat. #document-conversion #markdown

JupyterLab has a %%ai magic to use LLMs within notebooks. Ref #llm-ops

Telling ChatGPT that the year is 2123 makes it bypass copyright. Ref #chatgpt #future

Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations #embeddings #future #speech-to-text #tts #voice-cloning

Unsloth offers faster lower-memory LLM QLoRA finetuning #llm-ops #optimization

DeepSeek is an open-source high-quality LLM #llm-ops #search

Scalable Extraction of Training Data from (Production) Language Models extracts training data by repeating a token infinitely.

SkyPilot lets you run LLMs on any cloud provider. #cloud #future #llm-ops

vLLM lets you deploy LLMs with a single command. #llm-ops

llamafile lets you run LLMs locally as a single file executable! #llm-ops