My chats with LLMs
I spend maybe 15% of my waking hours talking to LLMs, and I don't think that's uncommon in the Bay Area, but almost none of these conversations are ever shared on the internet.
I think there's unique value in sharing chats; it's one of the best windows into your brain. Almost all the content you'd publish otherwise (interviews, blog posts, books) are about topics you're expert in, or at least have thought a lot about. Chat threads are more often about topics you know nothing about, making them much more vulnerable.
I'm also interested in learning how my usage patterns differ from others'. Many people I've spoken to have said things like "I don't use LLMs for learning because I don't trust them. They hallucinate with confidence." I have personally never had an issue with hallucinations, even when ChatGPT first launched, and I think it's because of the way that I use LLMs. I don't care for citations, and I'm generally not looking for high confidence of any particular fact. LLMs are their own type of source. They've seen a lot of patterns in the world, and I just want them to show me the patterns.
This evidently differs from how my cofounder Axel uses LLMs. He uses LLMs strictly on thinking mode, because he's always willing to wait longer for a more correct answer. I find this mind-boggling. Partly because I don't have the patience to wait 30 seconds for an answer, but also because for the purposes I use LLMs for, thinking usually makes the answer less useful for me. It causes the LLM to stray further from regurgitating what it's seen. I want the regurgitation, I want a statistical sense of what most experts would say.
You probably need a sample of questions I ask to make sense of that, so here it is. This is a very filtered down sample, as I wouldn't want to share most of them, but it is representative of my general style.
The business of wealth symbols
The US's entrepreneurial origins and our founding fathers
More random, but were interesting to me at points in time:
There's also plenty of chats where I make LLMs do some grunt work for me, such as trying to find an obscure Feynman quote I had once heard.
Most of those chats need too much context to make sense of though.
Some more thoughts I've formed after collecting this list:
I use Claude much more for advice in social situations. It seems to have much better EQ, which I hear is largely due to post-training on Amanda Askell. For example, Claude has a very different answer than ChatGPT for "Do women on average enjoy talking about agi timelines at parties less than men do?". (This may be in part because Askell herself hates talking about AGI at parties.)
I also know several people that leave the memory feature of ChatGPT on. I can't imagine doing this myself. I'm constantly lying to GPT to test how it responds with different assumptions, and I also have some anti-bias techniques that depend on a clean context each chat, such as flipping a loaded question. I would use memory if I could enable it only for particular projects, but this capability doesn't seem to exist yet.
I don't do any fancy prompt engineering techniques besides de-biasing techniques like above. Ever since RLHF, I don't think things like priming are actually useful for Q+A, though they're probably useful for agentic workflows.