Kamal Hassan: "A farmer invests in crops. I'm an actor. So I invest in films." As a technologist, I guess I would invest in technology.
"A person who has much more to give is unfazed by overwhelming demands because there is too much in him to overwhelm. He gives you 2 options in place of one."
Anyscale and Fireworks AI have the lowest error rates (5xx, 429) and rate limits across providers
Groq and Anthropic are among the highest, OpenAI is among the lowest, Google is in-between
OpenAI has lower error rates and lower latency than Azure
They have a ~35% cache hit rate
A few quick points supporting the mental model of "LLMs are aliens". #llm-ops
LLMs are clearly not machines. They give different answers each time.
LLMs are like humans: they exhibit human biases (e.g. guessing 42 or 37 often). But they fail in unusual ways. They can't count the "r"s in strawberry. They can go into an endless loop.
LLMs are a new form of intelligence. Thinking of them as aliens might minimize our confusions.
ChatGPT's Scheduled Tasks are pretty bad at fetching the latest news. Its use of search is poor. (I'm not sure if it actually searches.) I need to figure out other use cases for it. Possible options are: #chatgpt