Tutorials, news reactions, tool notes, personal miscellany, and observations outside the main topics.
An object that has a lot of momentum is hard to stop. A bowling ball. An ocean liner. A person who will not allow themselves to be derailed. Physically, an object of any amount of momentum could be stopped in a very short time, if enormous force is exerted. And the same could apply, in theory, to mental affairs. But
Rather than try to fix things, all you have to do is notice, non-judgementally, that you’re doing them. Even while you’re involved in this non-judgemental noticing, you will notice a barrage of impulses to try different things, to intervene, to try, to fix. That’s fine, just notice them non-judgementally, too. Source: "The Inner
Here's a hypothesis: I think I'll always find something I am curious about. I am a naturally curious person, and I would actually guess that everyone is? It seems impossible that there would ever be a moment where nothing could interest me. But I think curiosity arises from lack of Dopamine. Basically, boredom. If you never
No. You can see for yourself: setting the temperature variable to 0 (meaning you always sample the most likely token from the output distribution), you'd expect GPT-3.5 and 4 to produce the same output every time you call them. However, they don't. Why do LLMs produce different outputs across different runs? I've
Llama 3 is already training according to Zuck. There are conflicting sources & rumors, and the release date claims vary across all of 2024. For GPT-5 there are even no reliable rumors; if training started within the past few months then my back of napkin is that it may be launched early 2025, because OpenAI has ~2 year major
I'll share more thoughts on OpenAI Dev Day announcements soon, but one huge problem for any developer is LLM API latency. And boy, did OpenAI deliver. On a quick benchmark I ran: * gpt-4-1106-preview ("gpt-4-turbo") runs in 18ms/token * gpt-3.5-turbo-1106 ("the newest version of gpt-3.5&
After attending the Ray Summit in San Francisco this week, I realized I had previously discounted several interesting things. Here's what I now want to explore more. Semantic Kernel I've gotten so used to langchain that I haven't really considered switching... all the while loving to hate it. As I was ranting with someone
GPT APIs are slow. Just in the past week, the OpenAI community has had 20+ questions around that. And not only is it rare for users to tolerate 30-second response times in any app, it is also extremely annoying to develop when even basic tests take several minutes to run. (Before diving into these tricks, you might want to
This is a short link-post to a new repo I just released. While working on Why AutoGPT fails and how to fix it I created a handy web-browsing Tool for langchain Agent middlewares and now finally got around to open-sourcing it. Here is the repository: github.com/taivop/agentreader.
Some of the LLM apps we've been experimenting with have been extremely slow, so we asked ourselves: what do GPT APIs' response times depend on? It turns out that response time mostly depends on the number of output tokens generated by the model. Why? Because LLM latency is linear in output token count. But the theory isn&
