id
type
5
status
21
review version
0
cleanup version
0
pending deletion
0
created at
2025-03-27 22:23:24
updated at
2025-11-15 13:47:24
url
https://chamath.substack.com/p/a-conversation-with-one-of-my-8090
url length
65
url crc
35584
url crc32
3191114496
location type
1
canonical status
10
canonical page id
domain id
domain tld
0
domain parts
0
originating warc id
-
originating url
https://chamathreads.substack.com/sitemap.xml
source type
1
page id
title
excerpt
content
I talked to Sina Sojoodi (X/LinkedIn), one of my co-founders at 8090, after we published our Deep Dive on AI last week. Let me know if this is interesting to you.Chamath: There’s this great story of how OpenAI accidentally discovered LLMs. Walk me through what happened there.Sina: LLM Grokking – this is funny. It's almost like how penicillin was discovered.OpenAI made this discovery where typically in traditional machine learning, there's a point of overfitting, where you stop training because the model is just memorizing the training set rather than learning to generalize.But OpenAI accidentally ran their experiment for days instead of hours, well beyond this overfitting point. And they reached what they call the 'grokking zone', where somehow, even though the training should have been completed many times over, the model started to actually understand concepts and show emergent behaviors beyond memorization.This discovery completely changed how we approach training transformer models...
author
Chamath Palihapitiya
updated
1764632404
block type
0
extracted fields
36
extracted bits
article author
full content
detected location
0
detected language
1 (English)
category id
index version
2025110801
paywall score
0
spam phrases
0
text nonlatin
0
text cyrillic
0
text characters
3183
text words
587
text unique words
318
text lines
48
text sentences
33
text paragraphs
11
text words per sentence
17
text matched phrases
6
text matched dictionaries
2
links self subdomains
0
links other subdomains
1
links other domains
0
links spam adult
0
links spam random
0
links spam expired
0
links ext activities
0
links ext ecommerce
0
links ext finance
0
links ext crypto
0
links ext booking
0
links ext news
0
links ext leaks
0
links ext ugc
11
links ext klim
0
links ext generic
0
image author
featured image