AI generated art for books?

DiscussieFine Press Forum

Sluit je aan bij LibraryThing om te posten.

AI generated art for books?

1punkzip
sep 2, 2022, 2:07 pm

Looks like a way to save money on commissioning art - wonder if any titles (not necessarily fine or small press) have done this. AI generated is getting very good...

https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists...

2NathanOv
Bewerkt: sep 2, 2022, 4:00 pm

>1 punkzip: I don't know that I'd like it to become a trend, but it could be a fascinating concept for a relevant sci-fi title, like A Space Odyssey or Neuromancer. That could also allow it to be rather lavishly illustrated.

I've been hoping for Thornwillow to do I Have No Mouth & I Must Scream as a Dispatch title, and think a twisted piece of AI-generated art could make a great frontispiece for something like that as well.

I definitely wouldn't want presses to just start generating art for their books instead of hiring illustrators. It has to be part of a specific concept, and then well thought out and curated.

3Glacierman
Bewerkt: sep 2, 2022, 3:54 pm

>2 NathanOv: "I've been hoping for Thornwillow to do I Have No Mouth & I Must Scream as a Dispatch title...."

But...but...I want No Reply Press to do that story!

>1 punkzip: BTW, I can't access that story. Seems the Times wants me to subscribe to read it.

4NathanOv
sep 2, 2022, 4:00 pm

>3 Glacierman: "But...but...I want No Reply Press to do that story!"

They'd be a great publisher for it as well! Maybe if their upcoming sci-fi trilogy has a succesful launch, it'll become an ongoing series.

5SDB2012
Bewerkt: sep 2, 2022, 4:05 pm

+1 for Harlan Ellison

EDIT: I'd much rather have a cheeseburger cooked by a robot that can clean up the kitchen than AI art. When do we get that?

6astropi
Bewerkt: sep 2, 2022, 5:10 pm

As someone that has read a lot of Harlan Ellison... well, he's not my favorite science fiction and/or fantasy writer. In fact, I really would not recommend him to most people. Personal opinion of course. I know "I Have No Mouth" is an award-winning short story. But, I honestly found it boring. Like most of HE's work, I find Mouth to be an angry piece and reading it (despite the sci-fi post-apocalyptic setting) feels like reading through an angry teenager's diary. And in fact, just about every interview I've seen with him as well as others seems to suggest that he was actually an angry individual (there's actually a collection of his work titled "Angry Candy"). Again, just my opinion.

>1 punkzip: very interesting, but the NYT is obnoxious and has a paywall. I recommend theverge
https://www.theverge.com/2022/9/1/23332684/ai-generated-artwork-wins-state-fair-...

And here's the work:

7A.Godhelm
sep 2, 2022, 5:20 pm

I've been working with with Stable Diffusion for weeks now precisely to generate book based artwork and it's staggeringly good at generating a very wide range of styles. Maybe 1/20 of the results are good enough to publish. A good portion, 3/10 are full of various glitches. As long as you have time to generate results and act like the art director in changing the input slightly and grabbing the good results - or if you were to send what you liked to just be cleaned up and finished by an artist, this seems like a gamechanger.

There's a very interesting ethical debate happening in artist communities about this since all the AIs need to be "trained" on other people's work. This is something Stable Diffusion for instance really shines at; to get great results it helps to pick known artists that do styles you're looking for. From Moebius to Rembrandt, Giger and Beksiński or more recent digital illustrators. The debate is about to what extent the AI should have a right to mine the output of these artists and reproduce it.

We're definitely going to have a similar debate soon about AIs that can be trained on authors and replicate similar styles of stories. I doubt it'll be perfect, but by the same token you can simply tweak the inputs and sort hits from misses, it doesn't need to be 100% to present a challenge for authors.

8NathanOv
Bewerkt: sep 2, 2022, 5:24 pm

>6 astropi: I think "an angry piece from an angry writer" is a (at least somewhat) fair descriptor, and I think part of the appeal. I don't know that it's fair to interpret HE's apparent anger here as juvenile though, and it's a pretty typical tone for moral allegory in sci-fi.

EDIT: I also do not want to hijack this into an HE discussion thread though!

9punkzip
sep 2, 2022, 6:01 pm

>6 astropi: I'm a NYT digital subscriber. It used to be that they let you read 10 articles free a month but perhaps that has changed.

10astropi
sep 2, 2022, 6:07 pm

>8 NathanOv: Fair points indeed :)
Still, having seen interviews with HE and read so many of his works, I just can't read anything he wrote without an "angry voice"! Don't mean to hijack this thread either...

>7 A.Godhelm: It's really going to be interesting to see where this goes. Human art has always taken form by imitating and making homages to nature... and now we have computers producing art. Not sure why that's a problem? If computer-generated art is "not art" as some might argue, then what about digital art? That's after all a human-machine hybrid! At the end of the day, I think fully computer-generated art will be accepted. I suspect it will be in its own category so humans don't feel too intimidated :)

ps This isn't anything new. Here's a very relevant Twilight Zone episode - "The Brain Center at Whipple's"
https://www.dailymotion.com/video/x7yopcq
There are many bromides applicable here: 'too much of a good thing', 'tiger by the tail', 'as you sow so shall you reap'. The point is that, too often, Man becomes clever instead of becoming wise; he becomes inventive and not thoughtful; and sometimes, as in the case of Mr. Whipple, he can create himself right out of existence. As in tonight's tale of oddness and obsolescence, in the Twilight Zone.

11jroger1
sep 2, 2022, 6:10 pm

>9 punkzip:
I’m a NYT digital subscriber and am able to read an unlimited number of articles. I probably average 10 a day. I forward many of them to my wife and she is able to read them too. A call to their customer service might be in order.

12punkzip
sep 2, 2022, 6:13 pm

>11 jroger1: What I meant was that non-subscribers used to be able to read 10 articles free a month. I posted the link as I'm a subscriber, didn't realize non-subscribers couldn't see it.

13originaux
sep 3, 2022, 2:52 am

Perhaps AI could replace most journalists at NYT?

A giant computer named WOKE.

14What_What
sep 3, 2022, 10:28 am

>10 astropi: I agree with you - it seems to me art has always been inspired by what’s come before, so to what extent does a distinction need to be made whether it’s being done by a computer or a human. However, just like we wouldn’t put a robot in a 100m race with humans, we probably shouldn’t allow machine made art into competition with human-made art?

15SDB2012
sep 3, 2022, 11:17 am

>14 What_What: It depends on how you define art. Is art a beautiful picture or a unique way of looking at the world we can learn from or experience a new perspective?

16SF-72
sep 3, 2022, 12:01 pm

It's fascinating how advanced this is. I wouldn't have thought that you could 'create' art like this based on giving an AI descriptions.

As for the question if photography is on the same level as other art (e. g. painting etc.): In my country they're even taxed differently. What the state considers 'proper' original art is taxed at 7%, photos (or prints for that matter) at 19%. So that discussion is definitely not over at all, although I completely agree that a good photographer can create art. It's just different from what was traditionally considered art.

But I do worry about jobs for artists / illustrators with these AI options on the horizon.

17rsmac
sep 3, 2022, 12:34 pm

Well this definitely is going to be a problem for any illustrator doing digital painting in a fantasy/surrealism style because this is just as good as a lot of those.

I haven't seen an AI that can mimic, say, a wood engraving style or anything more precise and deliberate, so more traditional artists should be safe for now.

18NathanOv
Bewerkt: sep 3, 2022, 12:46 pm

>17 rsmac: I would be fascinated to see an AI trained on traditional woodcuts. I would have to imagine it having trouble conforming to all the “rules” that come from the physical limitations of the medium.

I see very little likelihood of publishers just beginning to replace illustrators with AI, though I could potentially see trade publishers or some limited series generating art for books that would’ve been otherwise unillustrated.

At the end of the day, it still requires a human with a strong creative concept and an artistic eye to prompt and curate AI generated art, though.

19abysswalker
sep 3, 2022, 1:37 pm

>18 NathanOv: "I would be fascinated to see an AI trained on traditional woodcuts. I would have to imagine it having trouble conforming to all the rules that come from the physical limitations of the medium."

On a lark I just tried the following prompt on Midjourney:

One morning, when Gregor Samsa woke from troubled dreams, he found himself transformed in his bed into a horrible vermin. He lay on his armour-like back, and if he lifted his head a little he could see his brown belly, slightly domed and divided by arches into stiff sections, wood engraving, detailed, dramatic lighting




And here is one from a different stable diffusion platform:

a wood engraving of a castle, Albrecht Dürer, detailed, elegant, gloomy, dramatic lighting



Neither is perfect by any means, but not bad either, and both together took me 30 seconds to generate.

20NathanOv
Bewerkt: sep 3, 2022, 1:47 pm

>19 abysswalker: Wow! It may have committed some impossibilities with the shading and detail work, but pretty spot on with the style and format.

I wouldn’t question at all if I saw that first one in a printing of Metamorphosis (and on a separate note, would love to see an edition of it done in that style) though at the same time neither are quite artistic standouts.

That castle does have me wondering though how an AI would interpret something like Lovecraft’s impossible architecture.

21grifgon
sep 3, 2022, 1:44 pm

It all comes down to what art is. If it's just decoration, then I'm sure AI will eventually do as well as the best of us.

22gmacaree
sep 3, 2022, 1:47 pm

I don't know how I'd define art, but I do know that if we keep calling machine learning 'AI' I will lose my mind.

23rsmac
sep 3, 2022, 1:48 pm

>19 abysswalker: The Kafka looks a little like the illustrations Ben Jones (Folio's A Clockwork Orange, Secret Agent) did for The Metamorphosis:

https://www.heartagency.com/artists/ben-jones/image/?image=7850

24NathanOv
Bewerkt: sep 3, 2022, 1:57 pm

>22 gmacaree: Machine Learning is a subset of AI. If you “teach” a machine something (or give it parameters to self-train), you are essentially granting it an “artificial” intelligence. They’re not synonymous, but are both applicable here.

25abysswalker
sep 3, 2022, 1:56 pm

>22 gmacaree: re: "AI" and machine learning: that used to annoy me as well, but I think the ship has sailed. It's kind of an arbitrary signifier anyways.

26abysswalker
sep 3, 2022, 1:59 pm

>20 NathanOv: "That castle does have me wondering though how an AI would interpret something like Lovecraft’s impossible architecture"

Coincidentally, about a month ago I used the prompt "brutalist cthulhu" on Midjourney:



27NathanOv
sep 3, 2022, 2:29 pm

>26 abysswalker: Cthulhu as an impossible building is such an off-the-rails interpretation of Lovecraft. I think I’ll be a little disappointed when these platforms become more grounded and human-like.

For now though, they certainly make for great inspiration and conceptualization.

28originaux
sep 3, 2022, 2:51 pm

Interesting.



Used the text "Boris Johnson as Lohengrin" on Midjourney.

29gmacaree
sep 3, 2022, 2:57 pm

>24 NathanOv: I studied machine learning at postgraduate level, I am perfectly well aware of how it's classified (and that that classification is nonsense).

30NathanOv
Bewerkt: sep 3, 2022, 3:37 pm

>29 gmacaree: I apologize if my response sounded supercilious. I consult on a pretty wide variety of machine learning and AI-driven projects, so a lot of my work is boiling it down into simplest possible terms.

Just about every major tech leader, university or research department classifies machine learning as an application of AI.

When it comes to what “an AI” is as an entity, the term is almost always used to refer to ML models with self-training parameters.

I understand they haven’t always been categorized together, but that’s where we are now.

EDIT: I am not remotely a computer scientist or AI expert, though.

31punkzip
Bewerkt: sep 4, 2022, 8:44 am

>21 grifgon: "It all comes down to what art is. If it's just decoration, then I'm sure AI will eventually do as well as the best of us."

I think it depends on what we mean by "art". If, as is typically done in the fine art world, one differentiates between fine art and illustration (applied art), I don't think fine art is threatened nearly as much by AI as illustration. Arion Press for example, often uses fine artists (i.e. artists whose work is in galleries and museums) for their books. I've noticed that many people here don't like the results and appreciate illustration more. Illustration is much more commonly used for books, and there is no reason that AI cannot do as good a job as human illustrators without the cost, because illustration IS largely decorative. I can easily imagine books which will use AI-generated art to provide many more illustrations than it is possible for human illustrators to create (at least at a reasonable cost) and these lavishly illustrated (by AI) books eventually driving book illustrators out of business.

32Glacierman
Bewerkt: sep 4, 2022, 1:43 pm

>31 punkzip: "I can easily imagine books which will use AI-generated art to provide many more illustrations than it is possible for human illustrators to create (at least at a reasonable cost) and these lavishly illustrated (by AI) books eventually driving book illustrators out of business."

I must respectfully disagree. The human element will always be present. AI is soulless. It can attempt to copy the human spirit, but will always fail. That is why AI "art" will remain naught but a fad. And if I should be wrong, then that is a world I would not wish to inhabit.

BTW, "Illustration" IS Art, just a different variety and the term is often applied by the fine art snobs to denigrate such works, some of which are absolutely stunning in their technique and style. "Digital art" is not AI, as it is made by a human intelligence using digital tools.

And remember, one man's art is another man's trash.

33punkzip
Bewerkt: sep 4, 2022, 3:44 pm

>32 Glacierman: I have to also respectfully disagree on both accounts.

I have little doubt that AI generated illustration will eventually become predominant.

No one is saying that illustration isn’t art - but it’s applied rather than fine art. In practical terms there is a vast difference between a fine artist and an illustrator- although they share similar skills. If one goes to school for art the training is different. In terms of education illustration is closer to graphic design than it is to fine art (and some illustrators with formal education were trained as graphic designers). What a fine artist and an illustrator actually do is quite different. In commercial terms while the vast majority of fine artists struggle the most successful fine artists have works which are shown in major galleries and museums and command eye-dropping prices for their works - far more than any illustrator can command. Are there any illustrators you like whose work has been shown at MOMA? Why not?

As for what art is, there is a vast literature on this topic in philosophy of art/aesthetics- referring specifically to fine art rather than illustration.

34originaux
sep 4, 2022, 2:10 pm

Picasso ILLUSTRATED Lysistrata…

35punkzip
Bewerkt: sep 4, 2022, 2:15 pm

Fine artists do take commercial projects - for example Arion Press. But they do it in their own way - which is why many people don’t appreciate what Picasso did for Lysistrata.

36originaux
sep 4, 2022, 2:21 pm

Do you mean that ”illustrators” never do it in their own way?

37punkzip
Bewerkt: sep 4, 2022, 4:39 pm

I love the use of scare quotes for illustration and illustrator. You do realize that schools offer degrees in illustration specifically?

38Glacierman
sep 4, 2022, 5:11 pm

>37 punkzip: Yes. And the conflict between the two will continue, probably forever.

39astropi
sep 4, 2022, 6:45 pm

How the Fine Art Market is a Scam
https://www.youtube.com/watch?v=Dw5kme5Q_Yo

I know a number of professional artists. They're primarily illustrators and they universally agree that "fine art" is in many ways a scam - or at any rate, the way fine art is marketed and sold most definitely is.

40NathanOv
Bewerkt: sep 4, 2022, 7:07 pm

>33 punkzip: “I have little doubt that AI generated illustration will eventually become predominant.”

I know I’m jumping back a few comments, but that’s certainly a bold prediction. Even digital art has yet to overtake traditional mediums, and I’m skeptical it will ever do so.

I could see us ending up with a few prominent AI-assisted illustrators, but even then we’d be crediting the human curator. The largest disruption I could imagine would be a major publishing house starting to generate art for their publications, but that still seems like the type of thing that would be constrained to a single series.

41zackapplewhite
sep 25, 2022, 10:09 am

I recently got in a heated debate over this topic, specifically in the genre of TTRPGs (tabletop roleplaying games) and will try to translate some of the arguments and counter arguments as best I can for authors and publishers.

Argument: History is repeating itself an the feelings and protests of artists will do nothing to stop it.

Counter argument:
1.Whether the emotions felt by artists will make any historical difference has nothing to do with the merit of their feelings and the justification for their anger. A feeling is not measured by its effectiveness same as your opinions are not measured by their effectiveness. They are measured, evaluated, and shared within one’s self and the general public. The sharing of them is natural and healthy, and attempts to quell these intangible parts of human nature have, “historically”, been equally ineffective.
2. And why is it that a author's feelings about money and production should be considered more important than the feelings felt by the artist? Why MUST you make a profit in your earliest days of publishing? What’s more, how many artists create and share their work for FREE on a daily basis just for the chance that others will see their work and its value, follow their work, and someday give them money.

Argument: This is a resource that should only be used by those who could never afford an artist in the first place. Of course we would all prefer to get exactly what we want from a talented and original designer and will pay for such as soon as funds are made available, but no such funds can be raised unless the book is put into production and sold in the first place. Thus, the value of these AI covers.

Counter Argument:
1. This logic assumes that should the author/publisher have money, that of course they would pay the artist. Tell me then, should your AI art generated book cover help sell your book so well that you gain 100k in profits will you never use AI again? I think it unlikely. I think it would be highly likely that you would “cut out the middle man” mostly if not entirely because that would be the prudent business decision. It’s all "stepping stones" and "last resorts" until someone strikes gold, then its “cutting costs”, it's “optimizing production”, etc.
2. All that said, of course I want authors and artists alike to turn profits. Frankly, assuming you start with purely digital materials and don't count the cost of your own labor (as is almost universally the case for creatives) it's pretty darn difficult to make a loss. Literally ANY sale is profit at that point. (I would know, as I have sold a number of digital products)

Argument: My choices are either to design it myself, shell out anywhere from $200-$2,000, or settle for the good-enough art provided by AI. I am as poor of a designer as I am poor and AI is my only feasible option.

Counter Argument:
1. This is just willful ignorance. You've decided to forget Public domain, Creative Commons, & royalty free materials. Not to mention Canva, photography, learning a skill, collaborating with artists and sharing those profits. Much of this I have done with Applewhite Games. My first TTRPG Kickstarter was a collaboration. Otherwise I wouldn’t have had much in the way of art and maps. Still the quality wasn’t up to the “high standards” many claim to be necessary, and yet it raised a couple hundred dollars. After which, my wife swooped in, polished her talents, and Applewhite Games was born.
2. Additionally, I didn’t even start in TTRPGs, I sold customized poems. $1 for digital, $5 for handwritten. After a few campaigns of similar nature I had enough money to print a book, to hire my cousin for cover design, and I made profits from the very beginning. Those profits helped to fund Applewhite Games. So consider this, why not change the scope of your work to fund your other endeavors? Why not go with the previously mentioned resources, to collaborate, learn a new skill, or work your way into the money you need to get the art that your work deserves?

You can choose to do none of these, as is your right. It’s harder, takes longer, costs money and probably isn’t what you want to do. I get that. But don’t tell me there weren’t a number of perfectly viable alternatives that you said no to or refused to consider in the first place.

42bacchus.
sep 25, 2022, 12:13 pm

I believe, when the time comes, it’d be more appropriate for artists to hold copyrights on their humane samples and use their own ML algorithms to generate images based on their input.