Open-endedness in Society
Curiosity and Interestingness: Rethinking Education, Research, and the Power of Implicit Knowledge via Novelty.
Do clearly-defined objectives help us make impactful discoveries? Or does a focus on goals and outcomes close off our curiosity and limit our potential? Perhaps open-ended creativity drives more successful outcomes?
Machine learning researcher Kenneth Stanley addresses these questions in his book, Why Greatness Cannot Be Planned. His conjectures have profound implications for how we live our lives, how we use curiosity, and objectives in education and elsewhere. They also overlap with Critical Rationalism, an evolutionary epistemology developed by philosopher Sir Karl Popper and quantum physicist David Deutsch.
Education and Research
We mostly think of project management, research, and education systems as objective-based processes. Professor Stanley suggests that this might be wrong and that interestingness-driven or novelty search leads to better outcomes. Let’s look at some examples:
Who are more successful in life? Children who are force fed standardized curricula with the objective to pass certain tests, or children who follow their wide-ranging curiosities and develop an expertise in one or a few areas that is unique and profound?
Who makes more valuable research contributions? A scientist who plans their research program in painstaking detail with the objective of achieving a certain outcome, or a scientist who is pursuing interesting and novel research directions?
Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.—Vannevar Bush, Science — the Endless Frontier, p. 12.
Variance/criteria tradeoff
Can unlimited unbounded open-ended exploration be bad? In the extremes, potentially. A scientist might spend their entire life researching the same obscure topic without any success. A student might not learn about basic math skills, which makes it hard to function in society. An extreme variance of outcomes will lead to many successes but also to a few failures. Venture capitalists know this all too well. Some risks are worth taking.
What successes does an open-ended system create? A student can obsess with physics for years and then invent a new breakthrough that brings about an energy abundance that leads to a broad-based productivity increase for billions. We are often curious about the solution to a problem. One breakthrough can change the lives of billions. Capitalism, our economic system, incentivizes solving the problems of other humans. (At least that is the case in theory. In a socialistic or Interventionist economy this incentive is reduced because of high taxes or regulations). Millions around the world escape poverty in a few years or decade thanks to this system. Over centuries this dynamic is more pronounced. Capitalism won’t make most of us as rich as Jeff Bezos, but it’ll make most of our grandchildren richer than him—just as it made most of us richer than Rockefeller in nearly every way that matters.
The solution is a minimal success criteria. The emphasis here should be on minimal. For the scientist this might mean moving to a different field after a decade of failures. For the student this might mean learning basic arithmetic and writing, so they can apply for an apartment and calculate their expenses. In Kenneth’s terminology these criteria might also be called initial stepping stones.
Stepping stones are the steps that lead to discoveries and breakthroughs. They are unknowable, and only exist in the adjacent possible. Rather than through predetermined objectives we discover them through experimentation and curiosity. For example, when developing a new social platform a designer might discover a novel user interface interaction that leads to a completely new set of unintended but valuable user behaviors. After years of developing large language models, an AI research company releases an experimental AI chatbot product. This leads them to discover a new business opportunity for a chatbot platform with a plugin marketplace. In a. talk at TTI/Vanguard's Stanley said:
People need the ability to follow their radical paths to their bitter ends so that we can see where those stepping stones might lead. Otherwise, we get this mediocre washout effect compromise for all but preference of none.
Unfortunately current research funding practices and our education system ignore the above insights. On average, the scientist who cannot show their pathway to success is unlikely to get funding. And students spend decades in school memorizing a mandated curriculum and rarely are given the space to explore on their own.
Visionaries
Following this framework of open-endedness and stepping stones, we can also understand a visionary as the first person that recognizes that success is only a few stepping stones away. Stanley writes:
Contrary to popular belief, great inventors don't peer into the distant future. [...] The successful inventor asks where we can get from here rather than how we can get there. It's a subtle yet profound difference. Instead of wasting effort on far-off grandiose visions, they concentrate on the edge of what's possible today.
For instance, Markus Persson combined recent technologies and concepts from existing games to create Minecraft, which he later sold to Microsoft for $2.5 billion. Steve Jobs, too, focused on available technologies and led the development of the iPad, rather than pursuing an overambitious futuristic AI technology. Donald W. Braben Scientific Freedom (2008) wrote:
The urgent need is for a few infl uential leaders to recognize that policies aiming to maximize efficiency will work only when efficiencies can be measured reliably. At the margins of intellectual endeavor where scientists are grappling with diffi cult or intractable problems the authorities should expect such policies to fail. They might even be the worst possible policies in such circumstances. Until recently, we had visionary leaders, and academic research was wisely left to look after itself. In meritocratic societies vision was once an essential qualification for leadership. But the environment has changed.
Critical Rationalism
Goals are fallible
All knowledge is conjectural, including that about our goals and plans. Since our goals are based on our theories, we can be wrong about what goals are worth pursuing. You might happily pursue a goal until a criticism of that pursuit arises. This might be an alternative goal that cannot be pursued in parallel, or a possible negative outcome should the goal be achieved.
All of these criticisms of your goal are unknowable when you set off to achieve it. This doesn't only apply to simple to-do lists, product roadmaps, and multi-year plans, but is summarized by a general inability to predict the growth of knowledge. David Deutsch in Beginning of Infinity writes:
The future of civilization is unknowable, because the knowledge that is going to affect it has yet to be created. [...]
[...] People in 1900 did not consider the internet or nuclear power unlikely: they did not conceive of them at all.
No good explanation can predict the outcome, or the probability of an outcome, of a phenomenon whose course is going to be significantly affected by the creation of new knowledge.
Not only the planning process is error-prone but also goals themselves. When you set off toward a goal, you will learn new things, and these new discoveries will change how you think about your goal. The goal itself needs to be malleable. A daydream or random conversation might make you realize that the item on your to-do or life goal list is unproductive for what is important to you. Thus, interesting should be top priority.
Implicit ideas are knowledge
"...you trust your gut instinct when it tells you something important is around the corner, but you should trust it, even if you can't explain what that something is." (Why Greatness Cannot Be Planned. p. 10)
In other words, explicit ideas shouldn't have authority over inexplicit ideas. We often experience these via emotions and intuitions. Some have described interestingness as the curiosity emotion.
So, Interest is mostly inexplicit. For example, it’s very hard to articulate why something is interesting or why we are curious about a topic. (I think we can to some extent and I plan to publish an article about the nature of curiosity soon). A feeling of drudgery or lack of interest can hint at a less fruitful or unproductive task.
Scientists often rely on implicit knowledge to guide their thinking and to generate new hypotheses and theories, even if they cannot always articulate precisely how or why this knowledge is relevant. Inexplicit ideas should be taken seriously because they represent valuable information.
Deutsch talks about implicit ideas and how they interact with explicit ideas when discussing the Fun Criterion.
Goal justification
You don't need to make up a tortured reason to justify every little impulse you feel. (Why Greatness Cannot Be Planned. p. 10)
Justificationism is wrong (knowledge is not justified true belief by some authority). Knowledge grows via conjecture and criticism. We can make the following two points about goal justification:
Goal justification is unnecessary: The absence of criticism suffices. (I wrote about criticism here).
Goal justification is impossible: Justifying a decision would require justifying the goal and all parent goals and so on. The result is an infinite regress.
Novelty search in ML and understanding
Professor Stanley co-invented Novelty Search, an evolutionary search method that rewards behavioral novelty.
The approach in novelty search is to identify novelty as a proxy for stepping stones. Novelty search has been shown to be effective at solving deceptive problems in constrained search spaces, and it can often outperform traditional objective-based search methods when deceptiveness increases [1][2][3].
In a deceptive problem, following the gradient of the objective function leads to local optima, which may not necessarily be the global optimum. Building a successful company is a deception problem. If you narrowly optimize for a few business metrics you’ll neglect novel experiments that might 100x your user base. Solving a maze is a deceptive problem. Exploring novel areas of the maze is more fruitful than running around in the area that seems to be closes to the exit.
Tracking novelty requires little change to any evolutionary algorithm aside from replacing the fitness function with a novelty metric. E.g. during the prioritization of search area to explore you might discard potential areas that have the least novelty.
Algorithms like novelty search can be more effective in solving deceptive problems, as they focus on exploring novel solutions instead of solely following the gradient of the objective function.
Understanding reality is a deception problem too. To think it is not is the error of empiricism. David Deutsch writes:
…, whenever we make an error, it is an error in the explanation of something. That is why appearances can be deceptive, and it is also why we, and our instruments, can correct for that deceptiveness. The growth of knowledge consists of correcting misconceptions in our theories. (BoI, p. 39)
And
Trying to know the unknowable leads inexorably to error and self-deception. Among other things, it creates a bias towards pessimism. (BoI, p. 196)
If we want to make progress, personally and as a society, we should adopt novelty search for search problems we encounter, avoid narrowly, in-fallible goals, engage in many high variance high return projects, and follow our curiosity.
Seek open-endedness.
Thanks to Logan Chipkin, Henrik Karlsson, Luke Piette, Tapa Ghosh, Matthew Siu, and Aaron Stupple for early feedback.
If this article was interesting you might also enjoy our interview with Stanley:
