From Pushkin to ChatGPT: Markov's 1913 Analysis

The Birth of Statistical Language Analysis

In 1913, in a small apartment in St. Petersburg, Andrey Markov was doing something that would seem bizarre to his contemporaries: he was counting vowels in Pushkin's poetry. Not for literary analysis, not for prosody, but for pure mathematics. This wasn't just any counting exercise - it was the birth of statistical natural language processing, though it would take the world nearly a century to fully appreciate its significance.

Imagine doing this without a computer. No Ctrl+F. No Python. Not even a calculator. Just Markov, a copy of "Eugene Onegin," and meticulous notes. The next time you ask ChatGPT for a recipe or get your grammar checked by Grammarly, spare a thought for Markov's cramping hand and ink-stained fingers.

A Symphony of Manual Computation

Let's appreciate what Markov had to do manually, step by step:

1. Count exactly 20,000 characters (no copy-paste to count!)
2. Create 200 sequences of 100 characters each
3. For each sequence, count vowels and consonants
4. Track transitions between vowels and consonants
5. Calculate probabilities without a calculator
6. Verify all calculations multiple times

Today, we can replicate his entire analysis in milliseconds. Our Python script does in the blink of an eye what took Markov months of dedicated work:

# What took Markov months, we do in a few lines
def clean_text(text):
    text = text.replace("\n", " ")
    text = " ".join(text.split())
    text = text.lower()
    return text[:20000]  # Exactly as Markov did

vowels = ["a", "e", "i", "o", "u"]
text = clean_text(get_text())  # Instant!

The Data That Changed Everything

Here's where it gets fascinating. Markov's original distribution reads like a mysterious code:

# Markov's original distribution (1913)
markov_dict = {
    37: 3,  38: 1,  39: 6,  40: 18, 41: 12,  # Lower tail
    42: 31, 43: 43, 44: 29, 45: 25, 46: 17,  # Peak region
    47: 12, 48: 2,  49: 1                     # Upper tail
}

Each number here represents hours of work. The '43: 43' entry? That's the mode of his distribution, found after counting thousands of characters. Today, we plot this in matplotlib with a single line. In 1913, each data point was a victory of human persistence.

The Revolutionary Insight

But here's where Markov's genius truly shines. He wasn't just counting - he was proving something profound about the nature of language itself. His data showed that:

# Modern recreation of Markov's key findings
vowel_vowel = vowel_vowel_count(text)
p_1 = vowel_vowel / total_vowels      # ≈ 0.128
p_0 = vowel_con / total_consonants    # ≈ 0.432

# The crucial difference
delta = p_1 - p_0                     # ≈ 0.128

This difference - this tiny, precious 0.128 - was revolutionary. It proved that letters in language aren't independent. Each character's probability depends on its predecessor, a concept so fundamental to modern NLP that we almost take it for granted.

The Mathematics of Poetry

Consider what Markov discovered in Pushkin's verses. When he found that:

He wasn't just finding numbers - he was quantifying the rhythm of Russian poetry. The same patterns we find in modern English text, proving that language's statistical nature transcends both time and culture.

From Pushkin to Python

Our modern recreation reveals something beautiful. Running Markov's analysis on English text:

# Results from our modern analysis
print("Vowel-vowel transitions:", vowel_vowel)
print("Vowel-consonant transitions:", vowel_con)
print("Coefficient of dispersion:", (1 + delta)/(1 - delta))

# The patterns persist across languages and centuries!

The patterns he found in Russian poetry emerge in English prose, in French novels, in Spanish tweets. The statistical structure of language is universal, and Markov found it with nothing but paper, pencil, and persistence.

The Long Shadow of 1913

The Eternal Lesson

Every time you use a language model - every autocomplete suggestion, every machine translation, every AI-generated text - you're standing on the shoulders of a mathematician who spent months counting vowels in poetry. Markov's work teaches us something profound: groundbreaking insights don't always require cutting-edge technology. Sometimes they just need patience, persistence, and the ability to see patterns where others see only poetry.

Modern Implications

Today's language models process billions of tokens per second, using neural architectures that would seem like science fiction to Markov. But at their core, they're still building on his fundamental insight: language has structure, and that structure can be quantified.

The next time you're frustrated with a slow computer or a long-running script, remember Markov in 1913, counting vowels by candlelight, proving that sometimes the most important breakthroughs come not from processing power, but from the power of human persistence.

# A toast to Markov in Python
def markov_tribute():
    return "Here's to the mathematician who counted letters " + \
           "so we could count on AI"

# The future of NLP, built on a foundation of vowels and consonants