Thursday, May 17, 2007

Black Swans, Rats, and Rejection

Success is a mixture of luck and timing.

--Julian Barnes

Nassim Nicholas Taleb is a trader (and passionate reader of fiction and poetry) who wrote a surprise bestseller entitled Fooled by Randomness (now followed up by his soon-to-be bestseller The Black Swan). Fooled by Randomness is a marvelous, and even more, an important book, and I would recommend it to anyone; but for purposes of this post, I will summarize Taleb’s main points (in a highly oversimplified fashion):

1) Luck (or, if you prefer, chance or randomness) has far more to do with success than anyone wants to admit; and

2) The most important factors in outcomes are often those that are unforeseen, unanticipated, and beyond anyone’s control.

The concept of the “Black Swan” relates to the unforeseen: Prior to the European discovery of Australia, Europeans believed, quite logically, that all swans were white, almost by definition. Finding black swans in Australia upset the rules—they were not only an unforeseen event, but an event that never could have been foreseen.

Taleb believes that the big forces that move markets are the Black Swans, and in publishing he gives the example of JK Rowling and the Harry Potter books. In retrospect it may be possible for people to rationalize the way the first Harry Potter stormed the bastions of bestsellerdom, but prior to the event no one saw it coming—certainly not all the publishers who turned it down, and not even the folks at Bloomsbury (who paid an advance of 3,000 pounds).

Grumpy Old Bookman has already treated Taleb’s thesis as applied to writing in an excellent (and lengthy) essay entitled On the Survival of Rats in the Slush Pile. Should you decide to read it, be prepared for the fact that the essay is quite depressing (he isn’t called ‘Grumpy’ for nothing). The bottom line, which Taleb would endorse, is that talent (however defined) and diligence are necessary for success in writing, but far from sufficient; success requires talent and diligence, plus a generous helping of luck.

Grumpy Old Bookman’s point seems to be that if you can avoid trying to be a writer, you’ll probably be happier at almost anything else, since the odds of success are so low, and since you have so little control over the outcome. True enough: I wouldn’t recommend writing to anyone unless they were driven to do it.

On the other hand, I think there is one more lesson for writers that can be derived from Fooled by Randomness, though that lesson is not brought out clearly in the book. Taleb is an options trader, and he makes his money by betting on Black Swans. Since Black Swans can’t be foreseen, what Taleb apparently does is take option positions counter to the prevailing wisdom—many, many such positions. And the prevailing wisdom is usually right, so on almost every bet he makes, Taleb loses money; only a little money, admittedly, but it needs a strong stomach to watch most of the investment positions you take go bad. When Taleb wins, however, he wins big, because the market has a large move in an unexpected direction, and this jackpot not only wipes away all the small losses he has made, but also makes him his living.

This strikes me as rather akin to the world a writer faces. Unless you are talented, diligent, and extraordinarily lucky, many of your experiments won’t work on the page; with those that do work, you will most likely be relentlessly rejected; when you are published, it will likely be to small acclaim and even smaller sales. But if you don’t keep taking those bets—bets you are almost certain to lose in any given instance—then you will probably never write something that works, or find a publisher, or achieve any critical or financial success. You have to buy a ticket to play, and you have to buy many many tickets to have any chance of succeeding.

In other words, frequent failure and rejection, both artistically and financially, should be expected as a part of the process. Then again, as Lawrence Block has said, it doesn’t matter how many times you are told ‘no’ when that can all be wiped away by a single ‘yes.’

There is the risk that the ‘yes’ may never come; it certainly didn’t come for Van Gogh, who never sold a painting despite the aggressive marketing of his art-dealer brother. (On the other hand, if Vincent hadn’t shot himself at age 37, he might have had better luck waiting down the road; killing yourself is the ultimate way of announcing that you're not going to play anymore.) All in all, I have to agree with Grumpy—if you can avoid being a writer, you’ll probably be happier. But if you decide to be involved in this goofy industry, be prepared for the fact that failure (however defined) and rejection will be constant companions. As it turns out, they aren't such bad guys once you see them for what they are: reminders that you're still in the game and still taking those bets.


may said...

I enjoyed reading this post quite a bit although (forgive) being a sucker for probability theory I am slightly upset when the word "randomness" is used in its colloquial meaning.

I recommend this article by Gregory Chaitin: “Randomness and Mathematical Proof". It's exhaustive and clearly written - simply beautiful.

David Isaak said...

I'll check out the Chaitin article (if I can find it).

But Taleb's thesis actually uses 'randomness' in a fairly mathematical sense. For example, he takes the example of picking stock market moves, and shows that, with enough people making predictions/investments, that even if the people did it by flipping a coin, at least one person would have 'predicted' every market move correctly. In fact, you get a bell curve of the total number of right and wrong answers per analyst.

What Taleb calls being "fooled by randomness" is the way that people then go on to look at the guy way out on the tail and conclude that he must be a genius based on his performance. The math says that such a guy will probably exist, even if he's a coin-flipping machine, but we humans proceed to lionize the guy and read all kinds of meanings into his pronouncements.

In general, Taleb says that we are story-making animals, and that we therefore see patterns and meaning where they don't really exist. He says that in his own line of work, the hardest thing for him to do is stick to what he knows is mathematically true rather than going with what his gut is telling him.

Interesting problem. Because story-making animals is exactly what we are.

Jeremy James said...

Excellent post, David. My favorite so far. Long live irrational exuberance.

May said...

Thank you for the further explanation. I want to read that book. Sounds wonderful.

Here is the link to the article (I don't know how to link it):

cate sweeney said...

Hi David
Is there a randomness in the fact I've just been reading about the Blak Swan theory somewhere else this week too.
Interesting stuff.
But hey I think luck helps, but talent does too.
Like the cover of your book by the way.

David Isaak said...

Irrational Exuberance is my middle name, Jeremy. Well, actually, just 'Irrational'.

David Isaak said...

"Here is the link to the article (I don't know how to link it):"

Learning how to paste a hyperlink isn't worth the trouble, trust me. But I grabbed the link and went out to the article. Looks cool--but I'll need a few cups of coffee first!

David Isaak said...

Hi, Cate!

Yeah, talent helps and may even be necessary. Ditto on diligence. They just don't seem to be guarantees of success.

The sad truth is that in my innermost heart I think it's all kind of mystical and fraught with significance, and when I don't succeed it is a reflection on my whole inner person. Everything is connected and it all means something. Taleb is a challenge to that kind of thinking.

You just heard about Black Swans the other day? Taleb would call that coincidence. Carl Jung would call it synchronicity. You could convince me of either, depending on my mood...

May said...

...searched for the book and did not find it. If I ordered it would take a couple of weeks. I am not a patient person.

David Thayer said...

This reminded me that the Swan Brewery in Darwin Australia was damaged by a collision with a freighter during a cyclone.
Black Swan Lager survived the disaster.

David Isaak said...

A freighter collided with a brewery? That's some cyclone! (Swan Lager is from Darwin? Who knew?)