An Ode to Killed Darlings

Dear Felienne!

I have written, deleted, and rewritten the start of this homework at least twenty times. I am on vacation; part of my brain is switched off, and today I just don’t have it in me to come up with a good, catchy opener that will draw our readers in. So let me just cut right to the chase…

Modeling How People Think

In our last episode of Computer Science Off Course, we discussed Sherry Turkle’s book “The Second Self”.¹ I have now read chapter 7, “The New Philosophers of Artificial Intelligence”, and there is this one section that stuck out to me and got me thinking. This section describes an approach to modeling human thinking, developed by early AI scientists Herbert Simon and Allen Newell. They had this idea that to arrive at a computer model of people’s thinking, one could observe people as they solve a particular problem. For instance, they observed people solving cryptarithmetic puzzles, in which digits are replaced by letters in a math equation. People were asked to talk while they solved such problems, and the researchers recorded what the participants did, what they said, and their eye movements. Subsequently, they analyzed these rich data. From these data, the different procedures and strategies people used to solve the puzzles in question could be distilled and turned into programs that simulate them. Since not all people solve problems in the same way, different programs would be developed for each type of problem. However, Herbert Simon and Allen Newell held the idea that all these different individual programs would still ”have the same general form” (p. 225) and that this general form characterizes ”how people think” (p. 225).

Process Over Outcome

I have many critical thoughts about that paradigm, but I want to approach things more positively today. So let me share some things I like about this: First, I love how their approach focuses on the problem-solving process rather than just the resulting solutions. I am also impressed by how much their method values the mistakes people make along the way, the pauses they take to think, and the notes they scribble down. It would have been so easy (and maybe even tempting?) to leave those things out when creating those computer simulations. However, when developing their programs, Herbert Simon and Allen Newell did not abstract these detours and pauses away, but went to build simulations that…

will simulate the solution process in every detail: not only the correct moves, but also the false starts and how they are undone, the pauses, the glance to recall a piece of information written on a corner of a worksheet, and the exclamations that punctuate the subject’s progress. (p. 255)

I personally find this approach more respectful of human intelligence than the idea that it could be captured by learning from, e.g., finished work, like the published books and scientific articles fed to LLMs.

No Case for Mass Surveillance

Please do not misunderstand my words. I am not making a case for mass surveillance of people, just so we can train AI with this data. I am not arguing that we need to study people’s behavior in all their detail or capture everything they say and do for the sake of better AI models. No! In fact, it is a nightmare to think of all this data from smart TVs, smart doorbells, smart speakers, and robot cleaners that undoubtedly shows how people solve the various problems associated with living their everyday lives, and to imagine that it could be combined to train AI. Aside from the ethical objections, I do not think this would be necessary, even if one were interested in creating ‘better AI’. As I have mentioned many times before, I believe computers do not need to imitate us humans, and they can better develop their own ‘computery’ strategies of figuring things out.

Yet, I do believe there is some value in mistakes, detours, pauses, and all the tiny, weird steps (sighs, forehead-rubbing, finger-tabbing) we humans take when we try to figure something out. I am convinced that there is value in all the attempts we delete, the side-steps we take, the darlings we kill, the thoughts we throw out. The 20 different beginnings for this homework that I have tried out and discarded say more about me, my thinking, and my reasoning and who I am than this finished text does.

This is also why I am extremely happy that companies cannot access such data on how people find their way towards a finished text to train LLMs at scale. Yes, the current so-called “reasoning models” exhibit behaviors that, on a superficial level, seem more similar to how humans approach problems. They tackle challenges step by step and fix issues in the text they produce before providing their final output. But when it comes to training their models, companies do not have access to all the books that ended up in the bin, all those texts that never saw the light of day. They do not see the original versions of academic papers that were later revised to please reviewers, nor the many words that were cut to meet the word limit. No, the typical training data for LLMs consists of finished, published books, not the darlings that got killed.² And while I am not happy that companies can scrape this very text from my website, I get some pleasure from the thought that at least they don’t have access to the 20 different types of beginnings I wrote and later deleted. This was my struggle, and I am happy that I am the only one learning from it — that not everything is up for grabs!

Big Tech Can’t Have It Both Ways

But what about online editors, documents living in the cloud, and companies spying on every word we type, you might ask! Yes, it worries me that companies like Microsoft could eventually change their policies and use people’s blank-page struggles, evolving drafts, and intermediate versions to train machine learning models. Yet, at least, they face a beautiful dilemma. If they succeed in shoving their AI models down people’s throats and get people to use AI for their writing, the companies will not be able to observe people’s genuine behavior. Even when spying on every keyboard click and mouse movement, they will not see people change words, try alternatives, and repair the flow of their text. Instead, they will observe how people avoid the discomfort of a blinking cursor on a blank page, prompt an LLM to get them started, or accept AI-generated revisions and suggestions. If companies get people to use their models, at least they won’t get to learn from their authentic human struggles. They can’t have it both ways! In times like these, I can find beauty in things like that.

Darlings Killed by AI

But then again, while AI won’t get to learn from the many darlings we humans have killed, we humans also will not get to enjoy the many texts that AI kills. If I, for one, had provided Felienne’s homework instructions³ to an AI along with this initial draft, the AI would certainly have pointed out to me that I have gone off course and missed the mark. It would have guided me back to the original assignment and provided helpful suggestions, even without being asked. This text might have been rewritten or deleted. And while it would never have entered any human eyes, it would surely remain available in some data center for training their AI. Of course, I am not claiming this would have been a huge loss in this specific case. But I am really worried about the many texts we lose to AI at large… So let me make sure this isn’t one of them and post it raw and fresh, exactly like this, before AI changes my mind!⁴

Best,
Hanna