One of the most famous stories in predictive analytics has two different versions. I happened to meet someone who said she'd worked on the model.
Years ago, Target became famous for something rather more interesting than selling groceries and household goods.
Its data scientists had developed a model capable of predicting whether a customer was pregnant based on her shopping behaviour.
Not exactly the sort of thing you expect your supermarket to know.
The story became legendary after a 2012 New York Times article by Charles Duhigg.
According to the article, an angry father walked into a Target store near Minneapolis.
His teenage daughter had been receiving coupons for baby clothes, maternity products and nursery furniture.
He was furious.
Why on earth was a supermarket sending pregnancy-related advertisements to his daughter, who was still in high school?
The store manager apologised.
A few days later, he called the father again.
This time, the father apologised.
His daughter was indeed pregnant.
Apparently, the supermarket knew something he didn't.
But there were two versions of the story
Over the years, I've encountered two slightly different interpretations.
Version one: The girl didn't even know she was pregnant. Target's predictive model figured it out before she did.
Version two: The girl already knew. Her shopping behaviour had changed, the model detected those changes, and Target started sending pregnancy-related offers.
The only person who didn't know was her father.
Both versions are interesting.
But they're not remotely the same thing.
The first suggests an almost supernatural predictive ability.
The second is an impressive, but considerably more understandable, application of behavioural analytics.
Target's system analysed purchasing patterns across multiple product categories, looking for combinations associated with pregnancy.
The original NYT reporting described a model developed by statistician Andrew Pole and Target's analytics team, using approximately 25 product signals to estimate the likelihood and stage of pregnancy.
Very clever.
But not clairvoyance.
Then I met someone who'd worked on it
Some time ago, I was talking with a senior director of Data Science at an American company.
During the conversation, we somehow got onto the subject of predictive analytics and Target.
And then came one of those strange professional moments.
She told me she had been involved in developing that famous pregnancy-prediction model.
The one I'd read about years earlier.
The one that had become a standard example in discussions about data mining, consumer behaviour and privacy.
Naturally, I had a question.
Not about the algorithms.
Not about the accuracy metrics.
Not even about how Target had managed to turn ordinary shopping data into something so personal.
I wanted to settle the argument.
Did the girl already know she was pregnant?
According to my conversation with her, yes.
The girl knew.
Her father didn't.
That's the version she gave me. I can't independently verify that detail, and the original public reporting doesn't explicitly establish when the daughter herself learned about the pregnancy.
But it does make the story rather less mysterious.
The model didn't necessarily discover something before the person experiencing it.
It inferred something she hadn't shared with her family.
And Target's marketing department made that inference unexpectedly visible.
The more interesting problem
For me, the important part of the Target story has never been whether the algorithm was particularly brilliant.
It's what happens when a company knows something about you that you haven't chosen to tell anyone else.
There's a difference between predicting which breakfast cereal somebody might buy next Tuesday and inferring a sensitive personal fact.
Even if the prediction is accurate.
Especially if it's accurate.
The Target case became a classic example of the tension between predictive marketing and privacy.
For more on the psychology behind marketing influence, see Dark Gestalt.
A system can produce a commercially useful prediction while creating an entirely different problem when that prediction reaches the customer.
Or, in this case, the customer's family.
And this is still relevant in the age of AI, when the ability to connect seemingly unrelated behavioural signals has become an important part of increasingly capable analytical systems.
And one more thing
I always find it amusing when you reach a point in your career where you meet people you've never heard of, but whose work you know extremely well.
You know the product.
You know the research.
You know the story everybody tells about it.
And then, over an ordinary conversation, you discover that the person sitting opposite you was involved.
Oh, and there's another detail I liked about this particular encounter.
She also writes children's books.
Apparently, building predictive models that can infer people's most private life events and writing stories for children are perfectly compatible professional interests.
I rather like that combination.