How is generative AI like a forged Vermeer?
I came to this question because one kind of AI is intuitive to me--the kind where a machine can be trained to distinguish between a photo of Lou Reed and a photo of a chocolate sundae.
(Lou Reed eating a chocolate sundae, according to GenAI)
Such AI models conform to the prescribed, god-given role of statistical models. Namely, they are descriptive, or at best, predictive. “Is this a photo of Lou Reed or of a sundae?” “Is this borrower likely to miss payments?” “Will this cookie click on my ad?”
Give a model a bunch of data, and it should give you a score, an answer, an analysis.
It’s not supposed to bloody make something.
Generative AI never really made sense to me, so I had to understand it better.
The Best of Adversaries
It wasn’t intuitive to the rest of humanity either, and it took a computer science prodigy named Ian Goodfellow to figure it out.
Goodfellow was freezing his tuchis in a PhD program at the University of Montreal in 2014 when he wrote a paper about an idea he and his colleagues called “generative adversarial networks.”
Their breakthrough idea was: what if we pit two AI models against each other to achieve a whole new level of result, one that blows away the results we have seen to date.
In this new approach, one model would be a generative model. The generative model’s job was to take its training data--or a slice of it, a “random input vector”--and create a new set of data.
(Generative Adversarial Network schema - machinelearningmastery.com)
This new data set would be designed to look as much like the training data as possible… with some noise thrown in to ensure it is not identical.
Goodfellow & Co. paired that model with a second model. This second model would be called a discriminatory model.
This second model’s job would be to look at the generative model’s output… compare it to the training data… and see if it can tell the difference. Is this really a picture of chocolate sundae, or is it a lousy copy?
The discriminatory model would stamp the output with a probability score. Like an Olympic judge, the discriminatory model would hold up a little sign with a number from 0 to 1 judging whether the new data, provided by the generative model, was real or fake.
The generative model and the discriminatory model would go around and around until the discriminatory model was stumped and started stamping every new output with a 0.5. In other words, the midpoint between 0 and 1. In other words, until the discriminatory model pleaded, “I can’t tell the difference.”
So you can see why the art forgery metaphor is apt.
A Real Faker
In 1945, a Dutchman named Han van Meegeren was about to be tried and executed for treason because he had been selling off national treasures--priceless Vermeer paintings--to the Nazis. With his life on the line, van Meegeren put up an extraordinary defense: “I can’t be a traitor, because these are not national treasures. They are forgeries--and I made them.”
(Vermeer vs van Meegeren, from Cultured Magazine)
Van Meegeren’s career as an original artist had floundered, but he had labored lovingly over his techniques as a forger.
So we can imagine van Meegeren as our generative model, striving to achieve an ever-more-authentic effect, to fool his discriminatory model.
Van Meegeren found a worthy adversary to play this role: the museum curator Dr. Abraham Bredius. Bredius was not only caustic and grudge-holding, he was famous for, in the 1930s, declaring 60 out of 690 extant Rembrandts to be inauthentic. A fearless professional skeptic, in other words, willing to call out mistakes and fakes.
Van Meegeren vs. Bredius! It was Holmes vs Moriarty. Edison vs. Tesla.
And so we may pit them against one another, in completely theoretical, but illustrative, rounds of adversarial cognition.
Getting to Maybe
To open, van Meegeren’s gambit was to mix his own paints, using original, Renaissance ingredients--lapis lazuli for blue, cinnabar for red.
[We can imagine Dr. Bredius evaluating van Meegeren’s early works and providing a score: say, a 0.1 chance of being an original Vermeer.]
In response, Van Meegeren--still playing the role of generative model, ever-adaptive--would make his own badger-hair paintbrushes, just like the ones Vermeer used, to make his strokes look identical.
[The Bredius discriminatory model says: 0.2 chance of being a real Vermeer]
Van Meegeren would then find, and use, original, actual, 17th century canvases to paint on.
[Okay, the discriminatory model is starting to be impressed, and says: 0.3]
Van Meegeren learned to mix his paints with Bakelite plastic, then cook them at 240 degrees, so they would crack like paint dating from the Renaissance. Lilac oil was required to keep the paint from discoloring in the heat, and Van Meegeren would keep lilacs in his studio to explain the smell.
Crafty.
[Discriminatory model says: 0.4]
Finally, van Meegeren would fill in the cracks with India Ink to make the paint look older and dirtier.
The Discriminatory model, at last, says: 0.5. “Can’t tell the difference.” In real life, Bredius declared Van Meegeren’s forgery, Christ at Emmaus, as “every inch a Vermeer.”
“We have here,” Dr. Bredius exulted, “the masterpiece of Johannes Vermeer of Delft.” The discriminatory model was fooled.
Stop and Smell the Lilacs
(Images of faces generated by the first adversarial model)
Van Meegeren in real life--that is to say, when he was not playing the role of a generative AI model in an extended riff in a blog--was a weirdo: an egotist, a drunk, a morphine addict, a philanderer, and a thief, who ripped off buyers to the tune of $60 million in today’s money.
Yet there is something of a little miracle in his ability to make new Vermeers.
And there is a uncanny parallel between the Montreal grad students pumping out images of faces… trying to make each successive round a more and more crisp synthesis of its data originals… and van Meegeren laboring to make Woman Reading echo Woman in Blue Reading a Letter by Vermeer.
“I've always loved a forger,” says Frank Wynn, a Van Meegeren biographer. It’s easy to see why. The cat and mouse. The pompous critics duped. Silly rich people--and evil Nazis--handing over their bucks to a trickster.
Now that trickster spirit lives inside our generative AI models, which tirelessly study the data of humanity’s oeuvre. Innumerable forgers, quietly peering at the Vermeers in the gallery, that they may better whip out easy, fast fakes.
So breathe deep, my fellow citizens of a post-gen-AI world, and smell the lilacs.
They have a 0.5 chance of being real.
