This is a raw stream of working notes. Some may be wrong, and mostly these will have little context, but I believe sharing raw thoughts is valuable in a way that polished blog posts are not.
* "less is more" (proverb) * "perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away" (Antoine de Saint Exupéry) * "omit needless words" (Strunk & White) * "It is vain to do with more what can be done with fewer" (Occam'
The Double Diamond is, roughly, a design framework consisting of 4 steps: 1. Diverging on problem (Discover). Explore widely to gather a broad range of insights and challenges related to the problem. 2. Converging on problem (Define). Analyze and synthesize the gathered insights to define a clear and specific problem statement. 3. Diverging on solution (Develop). Brainstorm and generate a
All top LLMs, including all GPT-family and Llama-family models, generate predictions one token at a time. It's inherent to the architecture, and applies to models running behind an API as well as local or self-deployed models. Armed with this knowledge, we can make a very accurate model of what the LLM's response time
90% of time when people say "Retrieval-augmented generation" they mean that the index is built using an embedding model like OpenAI's text-embedding-002 and a vector database like Chroma, but it doesn't have to be this way. Retrieval is a long-standing problem in computer science -- a couple of PhD students made
Retrieval-augmented generation, or RAG, is a fancy term hiding a simple idea: Problem: LLMs can reason, but they don't have the most relevant facts about your situation. They don't know the location of your user, or the most relevant passage from the knowledge base, or what the current list of customers is in their CRM.
Apps with no Memory are boring. Compare a static website from the 90s with any SaaS or social network or phone app: the former knows nothing about you, the latter knows a lot. From UI preferences (dark or light mode?) to basic personal data (name?) to your friend list to what things you engage with the most, today's
After attending the Ray Summit in San Francisco this week, I realized I had previously discounted several interesting things. Here's what I now want to explore more. Semantic Kernel I've gotten so used to langchain that I haven't really considered switching... all the while loving to hate it. As I was ranting with someone
Across the many experiments I've made this year (and which I've written about here) I've felt the need for better tools. Specifically, for the past few months I have been building copilots, and doing so from scratch takes a bunch of work every time. So we decided to release OpenCopilot: an OSS framework which
The stages of an LLM app seem to go like this: * Hardcode the first prompt, get the end-to-end app working. * Realise that the answers are bad. * Do some prompt engineering. * Realise the answers are still bad. * Do some more prompt engineering. * Discover vector databases!!!1 * Dump a ton of data as plain strings into the vector db for
GPT APIs are slow. Just in the past week, the OpenAI community has had 20+ questions around that. And not only is it rare for users to tolerate 30-second response times in any app, it is also extremely annoying to develop when even basic tests take several minutes to run. (Before diving into these tricks, you might want to
