Breadcrumbs
These are things I've come across that I find inspiring or thought-provoking.
Art, articles, ideas, videos, books, poems, and websites to send you down the rabbit-hole.
Literature is getting faster
In 2017, for instance, I started to wonder how much time passes, on average, across a page of a novel. Literary-critical tradition suggested that there had been a pretty stable balance between “scene” (minute-by-minute description) and “summary” (which may cover weeks or years) until modernists started to leave out the summary and make every page breathlessly immediate [1]. But when I sat down with two graduate students (Sabrina Lee and Jessica Mercado) to manually characterize a thousand passages from fiction, we found instead a long trend. The average length of time represented in 250 words of fiction had been getting steadily shorter since the early eighteenth century. There was a trend toward immediacy, in other words, but modernism didn’t begin the trend [2].
Figure 3. The average length of time described in 250 words of narration.
Craft Is the Antidote to Slop (article)
"Truly inspired creation, often seems rooted in something beyond mere human effort or secular concern. [...] no purely secular building has ever approached the grandeur of Chartres or any of the old cathedrals, and no secular music has reached the spiritual depth of Bach. Slop, in all its forms, is the result of attempting to conjure life and meaning without labor or place in history. The Kingdom of God is not merely a distant future state but is, in a sense, present alongside us, requiring our active participation in building and tending it through meaningful labor. It is no coincidence that Christ was a carpenter and Moses a shepherd. Demons thrive in these [AI] realms. Not metaphorical demons, but the actual spiritual forces that have always sought to separate humanity from meaningful engagement with creation as we see in Genesis 3. The demonic recognizes in our shortcuts the perfect opportunity: tempt humans away from the difficult labor of making, growing, and building with our own hands and minds. Instead, offer an endless stream of effortless consumption-images without artists, music without musicians, stories without storytellers. The devil's oldest strategy is, of course, promising godlike creation without godlike effort."
Oil Paint Color Mixing with Algorithms (article)
Created by Zsolt M. Kovacs-Vajna, based on his collection of spectral profiles of numerous paint pigments and brands.
He has built a suite of software for converting between digital color profiles and physical paint mixing recipes.
He has written extensively about color science and the nuances that go into color perception and rendering. 
Codex Seraphinianus (book)
A surrealist, illustrated encyclopedia created by Italian artist, architect, and designer Luigi Serafini
Beautiful illustrations, annotated with an untranslatable language. 
The Library of Babel, Jorge Luis Borges (short story)
Interesting exploration of the bounds of language.
Tangential to NLP/ language modeling, and I feel an essential read for anyone working with large language models.
When you pray, move your feet
A master was traveling with one of his disciples. The disciple was in charge of taking care of the camel. They came in the night, tired, to a caravanserai. It was the disciple’s duty to tether the camel; he didn’t bother about it, he left the camel outside. Instead of that he simply prayed. He said to Allah, “Take care of the camel,” and fell asleep.
In the morning the camel was gone—stolen or wandered away. The master asked, “What happened to the camel? Where is the camel?”
The disciple said, “I don’t know. You ask Allah, because I had told him to take care of the camel. I was so tired that I fell asleep immediately, so I don’t know. I am not responsible either, because I had told him, and very clearly! There was no missing the point. Not only once in fact, I told him thrice. You go on teaching ‘Trust Allah,’ so I trusted. Now don’t look at me with anger.”
The master said, “Trust in Allah, but tether your camel first—because Allah has no other hands than yours.” -Osho
Religion and Science both require Faith (video)
Great discussion with John Lennox on the compatibility of religion and science. This was my first exposure to his ideas and a great gateway to his work.
What is Emergence? (article)
What causes emergence in systems? I sketched an answer I found promising and elegant: error correction. This is a term from information theory, where you can encode the signals along a noisy channel to reduce that noise. Your phone works because of error correction.
Well, I think macroscales of systems are basically encodings that add error correction to the causal relationships of a system. That is, they reduce uncertainty about “What causes what?” And this added error correction just is what emergence is. So if a macroscale “emerges” from its underlying microscale it’s because it adds, uniquely, a certain amount of error correction, in that there’s a clearer answer to “What causes what?” up at that macroscale. -Erik Hoel (Science of Emergence)
What Is ChatGPT Doing … and Why Does It Work? (Stephen Wolfram, 2023 article)
we might ask what the “effective information content” is of human language and what’s typically said with it. There’s the raw corpus of examples of language. And then there’s the representation in the neural net of ChatGPT. That representation is very likely far from the “algorithmically minimal” representation (as we’ll discuss below). But it’s a representation that’s readily usable by the neural net. And in this representation it seems there’s in the end rather little “compression” of the training data; it seems on average to basically take only a bit less than one neural net weight to carry the “information content” of a word of training data.
You can shape the world (Gary Tan video)
"The world was built and designed by people no smarter than yourself. When you realize you can shape and control life, that changes everything."
How To Be Successful (Sam Altman article)
A powerful set of axioms for success that I read through at least once a year.
Once you have figured out what to do, be unstoppable about getting your small handful of priorities accomplished quickly. I have yet to meet a slow-moving person who is very successful.
School Is Not Enough (Simon Sarris article)
Often the parents of precocious children do not appreciate this interest. They think a child engrossed in Minecraft is simply playing video games, instead of longing to build. It is the parent’s job to identify this motivation, and then leverage technology and their own resources to encourage something more meaningful. “He’s going to do great things one day.” In his thirties? Why not today?
Create More, Consume Less
“We have to create culture, don’t watch TV, don’t read magazines, don’t even listen to NPR. Create your own roadshow. The nexus of space and time where you are now is the most immediate sector of your universe, and if you’re worrying about Michael Jackson or Bill Clinton or somebody else, then you are disempowered, you’re giving it all away to icons, icons which are maintained by an electronic media so that you want to dress like X or have lips like Y. This is shit-brained, this kind of thinking. That is all cultural diversion, and what is real is you and your friends and your associations, your highs, your orgasms, your hopes, your plans, your fears. And we are told ‘no’, we’re unimportant, we’re peripheral. ‘Get a degree, get a job, get a this, get a that.’ And then you’re a player, you don’t want to even play in that game. You want to reclaim your mind and get it out of the hands of the cultural engineers who want to turn you into a half-baked moron consuming all this trash that’s being manufactured out of the bones of a dying world.” Terrance McKenna
The Lamp Post Heretics
Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, "Let us first of all consider, my brethren, the value of Light. If Light be in itself good-" At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten min-utes, and they go about congratulating each other on their unmedi-aeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark. -G.K. Chesterton, Heretics
Different languages, similar encoding efficiency
Comparison of the information density of different spoken languages (ie, syllables spoken vs information conveyed) 
A Simple and Profound Introduction to Self-Inquiry by Sri Mooji (video)
A lecture I return to every few years to return to the core truth and cut through the layers of the ego.
This is what I always recommend to people who express interest in meditation or non-duality, but are complete beginners.
It is an intellectual line of self questioning rather than a meditation, which I think is more effective and approachable for most people.
The Hollow Crown of the King
“ For within the hollow crown
That rounds the mortal temples of a king
Keeps Death his court; and there the antic sits,
Scoffing his state and grinning at his pomp,
Allowing him a breath, a little scene,
To monarchize, be fear’d, and kill with looks;
Infusing him with self and vain conceit,
As if this flesh which walls about our life
Were brass impregnable; and humour’d thus,
Comes at the last and with a little pin
Bores through his castle wall, and farewell king!”
-William Shakespeare, Richard II, Act 3, Scene 2
Code: The Hidden Language of Computer Hardware and Software (book)
Before I read this book, computers were a black box to me.
I had no idea how electricity could give life to mysterious 'computer chips'.
This book will bring you from having zero knowledge of how a computer works, to deriving an understanding of logic gates and complex modern CPU architecture.
This book uses diagrams extensively, so get a PHYSICAL copy.
1985 Electrical Components Catalog
Many cool 80's technology company logos 
Karpathy Zero to Hero (video series)
The absolute essential series for anyone working in ML, especially language modeling.
Speak Memory
The cradle rocks above an abyss, and common sense wells us that our existence is but a brief crack of light between two eternities of darkness.
Although the two are identical twins, man, as a rule, views the prenatal abyss with more calm than the one he is heading for (at some forty-five hundred heartbeats an hour).
I know, however, of a young chronophobiac who experienced something like panic when looking for the first time at homemade movies that had been taken a few weeks before his birth.
He saw a world that was practically unchanged-the same house, the same people-and then realized that he did not exist there at all and that nobody mourned his absence.
He caught a glimpse of his mother waving from an upstairs window, and that unfamiliar gesture disturbed him, as if it were some mysterious farewell.
But what particularly frightened him was the sight of a brand-new baby carriage standing there on the porch, with the smug, encroaching air of a coffin; even that was empty, as if, in the reverse course of events, his very bones had dis-integrated.
-Nabokov, Speak Memory
Wings of Desire (short film)
Centered around two angels discussing the small human joys of life.
Refinement Culture
Refinement culture can be summarized as a general streamlining and removal of any unique characteristics.
Refinement Culture is one dimensional and removes variety from the environment.
It's optimized.
