Hype Cycles & the Principle of Optimism

Why we should be more optimistic and even tolerate excesses of optimism in Silicon Valley (while still roasting Pollyannas whenever it's fun)

One of the reasons I’ll never stop casual-posting is that I periodically discover broader interest in an idea that I would have anticipated. Recently, I restacked an observation Alberto Romero made about the “AI bubble,” noting as a comment to it that there’s more defensibility to hype-cycles and “manias” than might be apparent. I wouldn’t dispute that they’re ridiculous, but they occur for reasons more justifiable than many critics understand; they’re not solely the result of broken psychology, financial profiteering, or mob dynamics, but are also frequently productive, consequential, and skill-developing.

Since more than one person reached out to say this note was useful to them, I thought I’d expand on it a bit. I do think that being optimistic is important in this field, and probably in many others, and I’ll start with why that might be so from a philosophical perspective.

The Principle of Optimism

David Deutsch devotes an entire chapter in his remarkably useful book The Beginning of Infinity to optimism. He opens by approvingly quoting Karl Popper:

“The possibilities that lie in the future are infinite. When I say ‘It is our duty to remain optimists,’ this includes not only the openness of the future but also that which all of us contribute to it by everything we do: we are all responsible for what the future holds in store. Thus it is our duty, not to prophesy evil but, rather, to fight for a better world.”

Many will find this assertion appealing, but it is in the nature of the fight described that the disagreements begin; Popper views it as a fight against limitations and the unknown, while many prefer to think of it as a fight against some other. Online culture in particular, which as Chris Best has noted is increasingly predominant over any other, tends for various reasons to reward those who “prophesy evil” and lay blame for it too, as well as those gifted at excoriating real evils and evildoers or even mere strawmen representations thereof; it is indeed this latter group —the critics of cartoons— who probably do best in most feeds, and who increasingly color IRL cultures and scenes too. Reading most of what’s posted, one will soon have the sense that every problem in the world is caused by idiots who don’t know how stupid they are or are too evil to will the good, and thus that one should develop a critical, paranoid sensibility that first seeks to find whether something can be criticized.If it can, it should be abandoned quickly, for to associate with what can be criticized is to expose oneself to the risk of being dogpiled into oblivion. These are the games of the cultures of feeds.

If a mould for the individual results from this scrum, it is the person beyond criticism, an archetype of all mass media systems. The concept of “being cool” is, in my opinion, mostly to do with this, incidentally, but more broadly we can say that what is implicitly valorized is the person whose identity, configuration of activities, beliefs, tone, and history falsifies any attempt at criticisms one imagines coming from the many schools represented in media. This person is, of course, a kind of nothing, because the project is one of total risk-aversion; it may happen that this or that type of behavior or attitude is safe in any given generation, but across time, nothing is safe, and so nothing is what the culturally-informed person becomes. Hence the overlap between “being cool” and “being nihilistic,” even if one is cheery while doing so.

However: we generally learn by doing, especially but not exclusively at the frontiers of knowledge —which include not only novel science but every prosaic effort that has not already been accomplished, even in our personal lives— and we can only “do” where we achieve the commitment needed for activation. If we are to learn how to solve a given problem, we must try to solve it; if we want to try and solve it, we must believe, to some extent, that we can solve it. We might grant that it is unlikely, that our odds are long, that we could blow it, but if we believe a problem cannot be solved, or that a solution is utterly unworkable, we are unable to even try with the energy and investment needed to make progress.

Thus: since much knowledge comes from trying, and since trying requires belief in the possibility of success, there are reasons to encourage optimism as a general attitude. As Deutsch believes that “[i]f something is permitted by the laws of physics, then the only thing that can prevent it from being technologically possible is not knowing how,” it follows that for almost all efforts, it’s at least potentially useful to be optimistic.

The role of investors

An interesting aspect to the way that Silicon Valley works is that investors famously expect large shares of the companies they fund to fail. That is: they are not trying to avoid error; they are trying to maximize the likelihood of rare success. You can see how this maps to a preference for optimism and a variance from a culture of criticism: it’s of no matter to them if some goofball (or a hundred of them) believes too fervently or naively in the likelihood of their projects succeeding, because they really only need a small number to in fact succeed.

This alone would account for lots of Silicon Valley’s tendency towards optimism, but there’s an additional layer of great importance. In trying to build a company, one develops a range of skills and types of knowledge that are genericallyvaluable for company-building, regardless of what happens with one’s (perhaps naively optimistic) first effort. The harder one works to bring about a successful enterprise, too, the likelier one is to develop these skills, which range from operational to technical to psychological, and how hard one works is in part a function of one’s optimism! Many of these skills still require experience, especially ones involving self-management, perspective, emotional regulation, cultural dynamism, and other subtle or complex phenomena.

The reason so many in this scene love the idea of Roosevelt’s “[person] in the arena” is not solely their desire to romanticize their lives or insulate themselves from criticism by claiming that critics are fundamentally clueless spectators; it’s also that being in the arena really generates specific, important knowledge that only that experience can yield. The more one has been “in the arena,” the likelier that one will know how to operate there. And only optimists tend to jump into arenas that entail profound risks: painful effort, conflict, discomfort, financial loss, humiliation, and so on. I don’t want to overstate this; many learn little or nothing from trying to accomplish things; many learn the wrong lessons entirely, misreading their experiences and getting worse and worse to work with. All these things —entrepreneurship, the arts, the practice of science— are so causally dense and so intertwined with chance that it’s quite easy for the epistemic situation to be mixed. But it’s not for nothing that investors wind up selecting for optimists, or that optimism tends to be a common quality of successful entrepreneurs.

Those who try learn more than those who do not; those who learn more about how to try have greater success.

Hype cycles

If all or most of the winners in a scene —again, an industry or artistic or any other scene— share common properties, and if that scene attracts large numbers of interested entrants, it’s inevitable that those properties will because “part of the culture.” They’re reified in commentaries and analyses and histories and biographies and memes, and they’re imitated by those who want to participate in said scenes. Because of the reduction in nuance that inevitably accompanies scale, Silicon Valley culture runs the gamut from measured, informed, humble optimism about the possibility of improving this or that part of the world through diligent and rigorously-scrutinized efforts and invents all the way to imbecilic, narcissistic, and /or delusional Pollyanna-ish belief in the self, or in some technology or product, no matter what.

Whereas in general most builders are equally focused on problems and obstacles and limitations as on potential ideal outcomes, the further one gets from those actually engaged in making things the more one encounters people who seem determined never to allow this or that aspect of reality to harm their mood: people in denial of complexity and wildly credulous, people dispensing and consuming hype as a lifestyle, people whose primary motivation seems to be the maintenance of a mood they perhaps hope makes them seem like visionaries. Competitive dynamics amplify this. Just a few months ago, saying “I think LLMs will be important, but I don’t know that they’ll do much more than they do now” would have been worse than boring; it threatened to harsh the vibe! Countless parties in which status overlaps with wealth and both with one’s ability to seem informed and imaginative about the technological landscape required that everyone pretend, together, that within months, we’d see Sam Altman announce the immediate availability to Plus and Pro customers of God, now at your beck and call.

Together, these cultural (or cargo-cultural) and social forces mean that investors’ justifiable preference for optimistic doers and their outsize representation among the winners of the industry leads to Silicon Valley having a well-known hype problem. Trend after trend, launch after launch, bubble after bubble, the industry beclowns itself; grandiosity seeps into the most minor announcements, startups working on improved toasters devise missions that sound more ambitious than that of the United Nations, and countless Twitter accounts war over whether WAGMI or, if you’re in doubt, the degree to which you’re NGMI.

It’s enough to make anyone overcorrect; but as Frank Lantz has noted, overcorrection is the root of a great deal of error (and frequently evil).

The critical path

Many of us wind up allergic to hype. Especially if you’re a serious person, sensitive to the truth-value of statements, bothered by charlatans or bullshit, or merely proud of one’s own intelligence and discernment, it’s hard to avoid; it’s surely even worse if you’ve been harmed in some way by the costs of hype cycles. Many are the builders who have exchanged a steady income at e.g. Google for a crypto startup that went nowhere, and in the aftermath of it all, an obvious target for blame is excessive optimism, in the form of individuals or of the scene, or even in itself. Gritty realism is a mode of operation just like optimism, and in many fields it’s extremely useful; you’d rather those working on aviation systems, for example, be skeptical and “prove it” types, rather than “fake it till you make it” types! Engineering as a discipline now selects for both these types, and depending on background and personality, any individual might become one or the other.

I am sure there are lots of people who need to be counseled to calm down, get serious, and be more skeptical of trends and of themselves. But in management, I often encounter the opposite, perhaps as a function of my own personality: designers and engineers and data scientists who have perhaps gone too far, have become cynical to the point of interfering with their ability to be generative and collaborative and energetic in their daily work. If I have any hope for this post, it’s to provide a means for anyone in that category to recover some sense of sanguinity about our field and its associated culture.

My own experience in tech is not an uncommon one. I believed far too much in my first startup; I was far too cynical at my second; I bailed to Facebook after my third; and only in my fourth and fifth roles have I had what I think is a useful sense of how to manage myself philosophically and psychologically. David Cole was the first to express the importance of combining humility with optimism as a means of keeping oneself in check even as one pursued ambitious goals. David Deutsch may be correct that only “not knowing how” can stop us from doing anything physically possible; but it’s also important to recognize that “not knowing how” is not easily addressed, and most of all that we ourselves are agents of ignorance and confusion as often as not. That things are possible is useful to know, but there’s a difference between optimism as a philosophical position and deranged confidence or arrogance as personal traits.

Still, now, quite separate from my own mood and perspective on optimism and pessimism, I have a sense of the value of both modes, so much so that their excesses do not bother me or seem like crises to address (or matters of identity, intelligence, or morality). It is a matter of judgment when to lean into one or the other, and no there’s no shortage of examples of the utility and cost of each.

• Optimists, even naive ones, drive an enormous amount of productive activity, have the energy needed to animate basically slothful humans towards good ends, learn critical skills required of anyone hoping to achieve things through coordinated efforts as they do so, and are regularly correct in their views.

• Pessimists, even rancorous ones, improve the thinking and planning of optimists, highlight avoidable errors, restrain destructive flights of fancy (or at least keep them at sound altitudes), and hound operations and cultures into becoming more rigorous, truth-seeking, and criticism-incorporating.

If I’d counsel anything, it’s that one try not to calcify into either, and especially not as a means to achieving psychological safety or a sense of superiority. As ever: “Let facts create you!” Each is called for in particular moments, and balanced, they can both contribute to the development of knowledge and the creation of successful enterprises of any kind. And this kind of creation is more valuable than “never looking foolish” or “never seeming too critical” or anything of the sort. Behavior, not identity, is what matters for collective endeavors; output, not inner essence, should be the focus of a professional working towards ends in the real world.

If optimists incidentally generate moronic hype-cycles, and if pessimists seem more concerned with looking cool than doing anything worth a damn, these are just the unfortunate by-products of an eternal tension none of us can claim to have cleanly resolved, and one which we probably shouldn’t hope to.

Seokwoo Nam, Between Optimism and Discouragement, 2024.