Rob Adams a Painter's Blog painter's progress

June 24, 2024


Filed under: Artificial Intelligence — Tags: — Rob Adams @ 6:40 pm

So here I was for the third time in my life with a new medium on my hands. There had been the arrival of 2d computer graphics and then 3d modelling and rendering. These had caused some disruption. The first to photographers and layout artists who were either wiped out or in the case of photographers cut down from little gods to mere mortals. As always the superstars survived but the rest went back do doing mundane stuff like weddings.

This new revolution was somehow different. Nobody was using it for anything. People were having fun playing with the tools but the images were glanced at for a second and then forgotten. I set about seeing what I could make. Surely, I thought, my years of experience and drawing skills would allow me to do something of note.

I decided to return to my teenage passion of sci-fi illustration. The medium seemed tailor made for this. I decided I didn’t want photographs. Most of the Ai users were either doing Manga or seeking to make what they called ‘realistic’ which meant photographic. So I sought a painted feel. If you prompt for an artistic style or a famous artist the Ai model will attempt to emulate it. Here’s Rubens in space…

Vermeer reimagined. Here I hit the strangest problem. I didn’t feel I’d created these images. I had caused them to be made but somehow they didn’t belong to me in the same way a drawing or painting did. It was very hard to move out from a particular genre.

You could flit between genres and even mix them together. Here’s Rackham and Japanese Woodcut…

As a young illustrator I would have been delighted with any of these. Alas the images came with no sense of achievement.

As I learnt more I realised that the Photoshop plugin was very limited. It was a user interface for Stable Diffusion. I read a little and found there were other ways of using the same Models where you had more control. I need to explain models here. To make an Ai diffusion model you need millions of image/text pairs. Then the vectors are calculated between them all. So the vector angle between ‘man’ and ‘woman’ would be zero if calculated from the position of ‘animal’ but much wider if calculated from the position of ‘gender’ A diffusion model is the result of many billions of these calculations.

The end result is that if you type in ‘cat’ and input a field of random noise as well and then put it through your model. It will come out the other side a little more cat like, take that image and put it through again and it will be more cat like still. Do the same thirty times and you have a realistic moggy. I won’t go into the theory more as there are many better explanations out there by people who understand the maths better than I do.

I settled on an interface called ComfyUi which was node based. I was familiar with node based programs as I had used them to make shaders in 3d programs. Essentially you have a box of parts which you can plug together. The interface is quite forbidding and the documentation patchy… here’s an image of what I was faced with.

The rectangles are nodes each of which performs a simple operation. The ‘noodles’ are the connections between them. This workflow is for image to image, where you start with an image and then type a prompt for how you would like it to be remade. You can set the level of change and mix different images together. My friends and the press appear to believe you type in any old nonsense and a brilliant image will pop out. This is sort of true but the image will be pretty much random. If you want a specific image things get much harder. I was soon producing fairly finished images.

The only trouble with these is they were the same as everyone else’s images. There was still no ‘me’ in them. I put that to one side. I had only scratched the surface of the various methods to get what you wanted. It was soon clear I was never going to put in a sketch and a few sentences and the right image would pop out. To learn more I had to watch many You Tube videos and look at other people’s images. I came to an uncomfortable conclusion. Everyone was making every kind of image. Personal style was nonexistent in the medium. After a lot of study and experimentation I began to make some slightly more crafted images.

After a few months I could make pretty much anything I intended in any style. The core problem remained. I felt almost zero attachment to the resulting images and would certainly never sign one. On the other hand making them was great fun. As a child my favourite toy was a huge Meccano set. I spent many hours building complicated machines and vehicles. ComfyUI was like the biggest Meccano set ever, the ways of plugging the nodes together was endless. I had the thought that my attitude to the creations was similar. I never played with the cranes and cars I made as a child. I would finish making the model, look at it for a few moments and then take it apart again. I would never even show them to anyone.

There were so many people struggling with the medium that I began to make Videos teaching the basics… I tried to smuggle some of the basics of composition and colour theory into them but people had zero interest in the nuts and bolts of image making. I would correct the distorted hands but most users wouldn’t bother. There was never any compositional thought. The subject was always dead centre of frame.

So far on my quest I had come to many different conclusions and then had to abandon them. What were the millions of images being made for? Were they entirely disposable? I am still undecided.

June 21, 2024

Down the Rabbit Hole

Filed under: Artificial Intelligence,Uncategorized — Tags: — Rob Adams @ 10:13 am

When I started this blog it was to document my journey from being a commercial artist to being a painter of pictures to go in frames. I had thought the transition would be easy, after all, I had skills built up over years of illustration. I tried to set down what I knew here, to help others, but also to sort out what I knew and why in my own head.

You can read the results in earlier posts and see that my journey wasn’t exactly an easy one. There was much about picture making I hadn’t learnt and a lot I had learnt which was wrong or my understanding was flawed. I stopped blogging because I’d covered most aspects and it was turning into an illustrated diary. It’s gratifying that my erratic posts still get lots of hits—the ones on perspective especially so.

Also, I had retired to a cottage in Dorset and was fully engaged in painting and drawing my way around the county. So life was very pleasant and my pictures were selling well. Then, like Alice, a white rabbit passed by.

The first inkling was this:

Jason M. Allen, entered this image in an art competition and won. I thought it was a very strange image but interesting. Reading further I found out it was Ai generated in a program called Midjourney. Also, artists weren’t happy and said it wasn’t art. It was a classic reaction to any new thing. I’d been an early adopter of computer graphics and 3d so I knew what it’s like to be told what you do isn’t art.

This was in 2022 and a year slid by with me vaguely following the controversy. I saw other images from Dalle-e, often of plastic-looking women with no expression, or deranged cityscapes of swoopy sci-fi buildings. I rather liked these, they were reminiscent of the cover art I’d so admired as a twenty year old. Roger Dean churned out by the yard. I tried a few of the sites where you type in “A cat on a bicycle.” and a small image would pop out and wasn’t impressed.

Then towards the end of 2023 I saw there was a free plugin for Photoshop called Alpaca and it said it would turn my drawings into finished images. It was in beta so I thought, ‘Why not?’ Getting it was easy and during the process I ended up with Discord account… but that is a story for later. It presented me with a pretty straightforward interface so I did a simple drawing and wrote a short sentence describing what I wanted. Here’s my sketch.

After a few moments this came back.

Lots of things wrong but somehow it had almost made sense of my squiggles. This was my introduction to the mysteries of prompting. After reading a bit online I made a more focussed prompt. Apparently you had to reference art styles. So I added short description of the girl and put “Cathedral” and “Pixar” in my prompt. The result set me back on my heels.

Not perfect, but half an hour retouch and it could be a finished illustration. A thought surfaced, this was the end for legions of illustrators. Not the stars with the hosts of adoring fans, but the foot soldiers supplying the needs of advertisers and publishers. The many concept artists and storyboard slaves. I very much wanted it to be worse than a human produced image. I tried to convince myself it lacked soul and magic.

Most of my artist friends could manage this sleight of mind seemingly without effort. I have always had trouble with self delusion, not because I don’t delude myself, but because I am too good at it and have to keep it in check. Self delusion is the well spring of imagination, so it has its uses. The images being made weren’t better than the best that humans could make, but they were a lot better than most illustrators could have managed.

There I was. The rabbit hole was open at my feet should I jump down it? First of all I needed to know more. I needed to know how this thing operated. I found a site called Huggingface which had many learned papers with complex equations. I looked at You Tube vids by people who said they knew how it worked but plainly hadn’t read the science or understood the maths. At the same time the press was screaming about deep fakes and how we would all be killed by some malign Ai intelligence who dispensed with old-fashioned humans.

As a young illustrator I’d dreamed of doing Sci-fi illustrations. Once I got good enough to have an artist’s agent the jobs I actually got were romance covers and spot illustrations for Woman’s Own! This, I thought, would be the perfect medium to churn them out. I soon found it was far from easy. The Ai seemed to have a mind of its own. If I tried to constrain it too much I got a mangled mess of pixels, but if I allowed it too much freedom I would get a random image. Here’s a few examples from my early tests.

If, back in 1990, I had offered any of these to an art buyer they would have bitten my hand off. My feelings about this were very mixed but I was hooked now and falling rapidly down the rabbit’s burrow. Like Alice my journey thereafter would be surreal and unexpected.

« Newer PostsOlder Posts »

Powered by WordPress

error: Content is protected !!