The Melbourne artist taking control of generative AI for humanity’s sake

February 10, 2023

Mark Pesce

Mark Pesce invented the technology for 3D on the Web, has written seven books, was for seven years a judge on the ABC's "The New Inventors", founded postgraduate programs at USC and AFTRS, holds an honorary appointment at Sydney University, is a multiple-award-winning columnist for The Register, pens another column for IEEE Spectrum, and is a professional futurist and public speaker. Pesce hosts both the award-winning "The Next Billion Seconds" and "This Week in Startups Australia" podcasts.

One of the persistently recurring tropes of science fiction films – from Lawnmower Man all the way to Transcendence and a fair few besides – involves the plot of a human being (always, it seems, a man) trying to cheat death by technological means.

With a bit of jiggery-pokery and hand waving, the mind of the soon-to-be-deceased is somehow ‘uploaded’ into a vessel made of stronger stuff.

(Though if you have ever had a hard drive die on you, you might sense that ‘forever’ probably has a use-by date of no more than 48 months.)

After the ‘uploading’, that’s when the nasty bits start happening: all the phones on Earth begin to ring, or machines – driven by the now fully evil uploaded intelligence – begin to hunt down and kill anyone who gets in their way. It all makes for a very satisfying form of popcorn-demanding cinema. But none of it is really science. It’s all just so much fiction. Like it or not, our minds remain inextricably indivisible from our physical being. There’s no way to move our minds outside our heads, and as such, no life beyond the body. Yet.

Though there are hints.

In mid-January 2023, a lawyer and programmer named Matthew Butterick filed an enormous lawsuit against all of the organisations responsible for creating stable diffusion (that’s mostly a consortium of affiliated open-source firms), DALL-E (created by OpenAI) and MidJourney (another startup). At its essence the lawsuit contends that by vacuuming up every available image accessible on the Internet – in order to train their ‘diffusion’ models – they had effectively absorbed the style and, more than that, some of the artistic substance, of many artists. Those artists had no choice in the matter, could not ‘opt out’, nor place their works beyond the reach of this massive collection operation, only looking on as each of these new generative AI tools faithfully replicated key visual features from their own works within entirely new ones. (A very recent paper shows how to work backward to extract the original works from these models.)

Those artists had no choice in the matter, could not ‘opt out’ […] looking on as each of these new generative AI tools faithfully replicated key visual features from their own works.

That imitation was a big part of the initial draw to these new ‘generative AI’ tools. I could type “Paint the Sydney Opera House in the style of J. W. Turner” and get a work that looked as though it might have been from his hand. The same holds true for many artists – including some, such as Andy Warhol, whose works remain under copyright, plus others who continue to practice. Does this imitation represent a credible threat to their artistic creativity? That’s a big question – one that we’ll likely spend the next decade or two arguing, just as we did at the advent of photography, nearly two hundred years ago. Wherever that lands, there is now the widespread belief that a computer program can generate a work that’s “good enough” to satisfy the public.

a woman sitting behind a bowl of fruit painted in the style of Frida Kahlo, produced by a generative AI — *AI art generated using the prompt: “A still life of a woman sitting behind a bowl of fruit painted in the style of Frida Kahlo” with Stable Diffusion*.

What is ‘good enough’? For Turner, stormy and bright, diffuse colours. For Frida Kahlo, a primitive-cum-modernist mixture of fantasy and realism. Warhol might be represented in a quasi-illustrated style, mixing watercolor and pencil and print – as seen on the covers of his Interview magazine. Would Andy – who loved a bit of kit – be thrilled, terrified or simply bemused by generative AI?

Most likely, he’d be looking for a way to make some money with it.

But before we come to that, we need to ask exactly what has been copied here? The distillation of all of these images into an incredibly compressed ‘weighting’ reduces artistic expressiveness to a set of visual tics – these strokes make a Turner, those colours and perspective a Kahlo, and this pastel wash a Warhol. That reduction reveals the artist, as it strips away the content and allows us to focus on technique – which generative AIs have no trouble replicating.

Does this imitation represent a credible threat to their artistic creativity?

But is an artist simply a repeatable set of visual tics? That appears to be the Big Question posed by generative AI – less a question about art itself than a question about what the artist offers to their art. An artist who is all tics such as Leroy Neiman (once among the most sought-after painters in the world), will eventually seen as trite and boring. An artist who continually reinvents their capacity for creation (Picasso being the canonical example) transcends to a longer-lasting fame.

That reinvention may in fact be the heart of the matter – or, more precisely, the art of the matter, because no generative AI can reinvent itself beyond the weightings of its model. It simply follows the math to its inevitable conclusion.

Yet there might be a space for these tics. Consider Melbourne-based artist Anthony Breslin, who built a career out of an accessible, fun, almost pop aesthetic with works that clearly owe a lot to modernism, post-expressionism, pop, urban, Indigenous Australia – and many other schools. Although widely differing in subject material, Breslin’s paintings also have a sort of internal consistency of style that you’d expect – such as the inclusion of ‘found’ materials, such as buttons, set into the canvas.

Breslin has had a good run as an artist – a run that might be drawing to a close as he battles two kinds of cancer. It’s the kind of confrontation with limits that gets people to thinking about their legacy – and a shot at immortality.

This is where a new startup out of Europe – VAIsual – comes into the story. The founders immediately understood how generative AI datasets would leave artists feeling as though they’d been stolen from, so they built a tool that allowed artists to create specific datasets around their own work. Rather than training against hundreds of millions of images, VAIsual trains against a smaller set from an individual artist. This process does still reduce the artist to a series of tics – or, in the parlance of generative AI, ‘prompts’. But it also replicates the style of an artist’s work – indefinitely.

Anthony Breslin found that an irresistible opportunity. In October 2022, VAIsual and Breslin announced the ‘Bresinator’ – an AI model trained to be able to generate an infinite series of new works in the style of Anthony Breslin – with his approval and with some of the proceeds generated from sales of that model going to him. One firm quickly licensed the Breslinator for their own generative AI toolsets – and now Breslin looks forward to a future where he will continue to create new works long after he passes.

This soft upload of Breslin’s artistic essence offers him a chance to connect with generations to come.

While generative AI forces us to ask hard questions about human creativity, it also offers us new ways to celebrate it. Rather than something tinged with sci-fi menace, this ‘soft upload’ of Breslin’s artistic essence offers him a chance to connect with generations to come. It’s not the same as the human act of creation, but it could bring people joy for decades to come. That’s not a bad legacy.