Tatiana Samoilova as Anna-bobanna

So back in March, 2009 I blogged about a phenomenon I called the Anna Karenina Yield Anomaly. In short, I postulated that in the production of crops the idea of a national ‘good year’ pretty much means everyone had a good yield and a national ‘bad year’ meant that some had an OK year and some were having a terrible year. And thus I made myself seem more literate than I am by linking that phenomenon back to Tolstoy and his line “All happy families are happy in the same way. All miserable families are miserable in their own way.”

So a couple weeks ago a big ass storm conspired against me and resulted in me listening to the audio version of Guns Germs and Steel by Jared Diamond while stuck in the Minneapolis airport. I was struck that Diamond also used the Anna Kerenina quote when discussing the domestication of animals. He presented the idea that all domesticated animals are similar in a number of key characteristics and all the non-domesticated critters are different and failed to be put under the yolk of humans for their own special reasons.

Don't call him Jared Yoder. He hates that.

Ok, so Diamond and I both like to make connections between Tolstoy and modern agriculture. And all us white guys like to look smarter than we probably are (actually he really is that smart, I’m just gonna fake it till I make it). But besides those obvious things, what’s behind the Tolstoy quote? How applicable is this idea to other situations?

I spent a long time in the Minneapolis Airport pondering what conditions result in what I began to call the Tolstoy Dichotomy: one outcome and all are similar vs. other outcome with different causes. OK, just for transparency, I was also trying not to tap my toe or do anything else obviously gay while in the pooper. That’s why I listen to audio books and not music when stuck in MSP.

While it’s not exactly profound, it seems to me that any time a given outcome of a process is dependent on many necessary and few/no sufficient conditions then the final set of actors who went through that process will comprise one group where all necessary conditions were met and another group where varying different conditions were not met. So all the actors in one group are the same but the actors in the other group are in that group together, but all for a different set of reasons.

In crop yields this makes a lot of sense. The growing season is 8 months + for corn in the Midwest US. During that period a LOT of things can go wrong. There can be a spring that is wet and delays planting, like this year. There can be a mid-season drought, like 1987. A big cloud can park itself over the Midwest and limit light and temp and increase flooding (1993). But all good years mean that there was moisture at the right times, not too much heat during pollination, etc. So with crops there are many necessary conditions and no sufficient conditions for a good crop.

One of the interesting analytical consequences of this dichotomy is that the two groups have very different correlations. If you are measuring the group that met all the conditions, they are a homogeneous bunch (i.e. high correlation). But the group that fell off the wagon for one reason or another… well they are all pretty heterogeneous (i.e. low correlation). Depending on your model it may be very important to consider the difference between the correlations in these groups and then vary your covariance accordingly. Or you can just assume that correlations are stable over time and find yourself kicked in the nuts by reality. Which ever. It’s your call.

So, as I said earlier, not exactly profound but every time you see a system with many necessary conditions and  no sufficient conditions be like me and pull a Tolstoy Anna Kerenina quote from your butt. You’ll be glad you did.

Leo Tolstoy

Leo Tolstoy begins Anna Karenina with the famous lines: “All happy families are happy in the same way. All miserable families are miserable in their own way.”

I did my graduate thesis on the spatial relationship of stochastic variables. The stochastic variable which I found most interesting at the time was farmer corn yields. Since yields can’t go any lower than zero I had a preconceived notion that in bad years all farmer yields were bunched up down close to zero and on good years yields were very spread out with some farmers having average yields and some having nearly double their average yields.  I was pretty surprised when I found that good year yield distributions look like this:

IL Corn 2006

and bad years look like this:

IL Corn 2002

I was surprised by the notion that when yields are relatively low, like in 2002, the distribution has so much more dispersion. And in “good” years, like 2006, the distribution is rather compact. This pattern holds across other crops, other states, and different levels of aggregation/disaggregation.

So in happy years all farmers are happy together, but in miserable years each one in miserable in their own special way.

There are interesting spatial correlation impacts of this which are quite intuitive. If a good year is defined as ‘everyone does well’ then the spatial correlation is quite high since everyone is doing well. But bad years are defined by some doing ok, and some doing quite bad. This results in much lower spatial correlation.

Sooooo… BOTH the distribution shape AND the correlation matrix of crop yields must vary with yield level. That makes a proper stochastic model a bit tricky, no? Typically we assume the correlation matrix to be fixed and not a variable. That’s why it’s never boring to be a stochastic agricultural economist.

Footnote: The above graphs are PDFs based on county yields, not producer yields, but yields are fractal so the producer level yield distributions have a very similar shape.  Data is from the National Agricultural Statistical Service (NASS).  PDFs were done in R using the density function.