AI for Art 🖼️🎛️ [article requires update]
Sam KatarheinFirst of all. Art's that i do is not fully my merit, i just connected my vision, technologies [AI] and some art skill.
All ai artworks/pictures are the merit of the work of all the artists/photographers/sculptors who created learning material and engineers who brought this technology to life. Even if these groups of people isn't friends.
IMPORTANT INFO: This is not a "step by step" guide how to use AI in your artwork, this is a general user look to possibilities of using AI for creating pictures. A bit of technical moments you can find bellow too. Keep in mind that I'm still experimenting and don't have clear pipeline that 100% work. [sorry for my English].
Another important note: everything getting better with practice. AI usage requires practice too. Without practice results will be drastically bad.
At this moment i see three main ways of creating pictures with AI assistance:
- Generate image and fix broken moments.
- Generate image based on your control image [sketch, photo, art, 3d, etc] and fix broken moment.
- Enhance/Change existing picture with generative AI and fix broken moments.
About technical moments
Let's suggest that you're not banned in google and have good enough pc for generating images locally. By good PC i meant that your Graphic card is any RTX Nvidia 8Gb+. Ofcource you can generate images even on old graphic card with 4-6gb VRAM, but this will be drastically slow. If you don't have good graphic card and looking for new one i can recommend these options that will work at least 3 years upfront. You can find cheaper alternatives - but at my opinion other graphic cards will outdate too fast and will make no profit in performance/price balance.
- Nvidia RTX 4060Ti (16gb version! 8gb version is bad!). ~500$. A lot of graphic memory. More = better. As example 4070 have 12gb of graphic memory, don't ask me why.
- Nvidia RTX 4070 Super (16gb)
- Nvidia RTX 4080 Super (16gb). ~1000$. Much faster than 4060ti but also 2 times expensive. requires good power supply.
- Nvidia RTX 4090 (24gb). ~1700$. Most expensive and best variant for rich people. You'll not find anything better at this moment. requires good power supply.
My current graphic card is 3070ti (8gb) because i have notebook, it works fine but have some limitations time to time.
By "not banned in google" i meant that you can write "how to install Automatic 1111 stable diffusion locally" in Youtube or Google search bar. I can recommend Youtube channels like Olivio Sarikas, Matt Wolfe, Aitrepreneur for good guides about installation and usage.
Update: currently i use this fork of A1111 that have better optimization. https://github.com/lllyasviel/stable-diffusion-webui-forge
Also i'm learning how to use ComfyUI too
When you finally installed Stable Diffusion locally - you'll need download furry models for generating [because classic models is not capable with good furry art generation].
For models i can recommend site CivitAI [requires login to see nsfw models] and "Furry Diffusion" Discord
I'm not discord user and don't know how to invite someone to the server, so you need to google it. "Furry Diffusion" Discord have a lot of useful info with good prompts and a lot of people who share their experience/tips and models!
About models, here's some that i use last 2-3 months:
- SeaArt Furry XL [good for enhancing]
- IndigoFurryMix XL [ mix of SeaArt and PonyDiff i guess? anyway - good for pose refs]
- PonyDiffusion XL [good for pose refs too, a lot of custom style loras on civit ai]
- EasyFluff sd1.5[a bit outdated due to resolution but fine for weak graphic cards]
- Be curious! Download any and test!
About extensions for stable diffusion, i sometimes use:
- ControlNet [to control generation process, explanation later]
- Infinite Image Browsing [comfort image viewer for saved images and promts]
- Sd Webui Tagcomplete [for hints during prompting]
- CFG Rescale [only for VPRED models like easyfluff - outdated]
- Be curious! Keep eye's on Youtube and watch channels about AI generation.
Finally, after several tutorials your stable diffusion will looks something like this:
About art-theoretical moments
I hightly reccomend to check youtube channels like Marco Bucci, Marc Brunet, Sinix Design, Bobby Chiu etc. Especially 10 minute classes from Marco Bucci. This will improve your artistic vision so you'll have better taste of composition, color, design, and many many other stuff.
FIRST OF ALL 🎛️AI may be good at "rendering" parts like donuts or a cat fur, but AI will never build that idea from "0" just by pressing button. People don't understand that this is main point why ai will never replace artists. Not because ai can't generate hands correctly, but because ai can't tell a story.
Just imagine how powerful AI can be in hands of this artist. AI can save hundreds and thousand of hours in perspective!
Artists life is short - i would rather let them to paint ideas with AI, instead thousands hours rendering bushes on background.
Let's take another example, just random art from twitter.
I don't think that you can explane this idea to AI and generate that from "0"? Maybe this artist don't draw a perfect cloth or shadows - but this artist show you idea. Idea that ai can't handle by slap. AI understand how to render realistic shorts or hoodie, but AI don't understand message of the art.
Why left character trying to refuse pie and looks worrying? Why right character trying to feed left and have semi-evil funny smile? Why he's trying to give first bite of pie to his friend? Why there's a flowers on that t-short? THIS IS ALL ABOUT IDEA AND MESSAGE.
Altrough if we will look at the other side of question - sometimes people like simple ideas like "cute cat sitting on table". This is sphere where ai can hande. People will generate thousands arts with cats on tables and future viewer will become less interest in this type of art. Unless you'll bring something realy unigue in your generated picture.
Okie, i hope you understood my take and we can back to generation methods.
№1 Generate image and fix broken moments. [updating]
First of all - Be curious! watch guides about "prompting" on Youtube. Look at tips from "Furry Diffusion" Discord users and try copy prompts of other users. No need to reinventing the wheel [untill you're expirienced enough for mixing]
At this moment creating arts including 2+ characters is very problematic for AI. And i don't even tell about situations when you have idea in mind and want a special poses. In this case artists have pretty good safe zone.
№2 Generate image based on your control image [sketch, photo, art, 3d, etc] and fix broken moment.
Draw a sketch and put it into ControlNet extension [watch youtube tutors]. Write prompt "masterpiece Sword, white background, simple background". Get random result! Fix bad moments manually or by "rolling" inPaint function if you're enough lucky and have time.
Keep in mind that models is not "universal", some models can be good at drawing "swords" some models really bad!
What if i want a special design for my sword? AI DONT CARE 😅
Can you get something good looking from AI - yes. ✅
Can you get what you want from AI - no. ❌
This is why AI can't replace artists - especially when we talk about commissions. There's no way to clearly explain your reference sheet and idea of art to AI. At this moment if you'll train special models [lora] for your character - AI will not follow your color patterns and idea good enough to give you perfect final look.
Exception for popular characters that have really A LOT of fanarts, hundreds of arts with Nick Wild actually help ai to understand that this is fox with orange and white fur.
But even if you'll generate popular character from sketch - you'll still need to fight with artifacts and broken hands, etc.
№3 Enhance/Change existing picture with generative AI and fix broken moments. [method that i use 70% of time]
Usually i use Ai assist after shading step. Here's my usual pipeline of work:
TADA 🎂🎉
Final thoughts from Artist and AI user.
If you think that AI technologies will replace artists - Nope. And it will be unable to replace artist for many&many years upfront. Most of AI limitations based on logical things that can't be solved just by better technologies. You'll easily understand that if you'll try to do something with AI.
BUT AI can be really good assistant in workflow. Especially if you're mid lvl artist like me. And i just can't imagine how powerful this tool can be in hands of really professional artists. That's all. If i'll have any changes or additional thoughts i'll update this article.
Be curious! 🐶