Basic facts about Stable Diffusion

AI is a new technology, and I (@elitepics1 from Instagram) see your endless interest in the generation of images. Here's some basic knowledge (as of November 2023):

Q: What program do you use to create pictures?

A: Stable Diffusion.

Q: How Stable Diffusion works from the user's point of view?

A: The most widely used technique is text-to-image: you choose style (a.k.a. model), write textual description of what you want to see (a.k.a. prompts), define technical settings (resolution, AI's freedom of action, and others), and start generation. After some intense computing you get a picture. Second interesting technique is image-to-image where you use a source picture that will be completely re-drawn according to your prompts. You can use even a hand-written sketch to transform it into a piece of art.

Q: How can I use Stable Diffusion?

A: You can either use web services or install it on your PC.

Q: What are pros and cons of using web services based on Stable Diffusion?

A: Web services based on Stable Diffusion give opportunity to create pictures, but they have many limitations regarding your freedom to make anything you want (some subjects are banned or censored) and, more important, quantity and quality of images.

Q: What are pros and cons of using Stable Diffusion on your PC?

A: You get full control of content that you make, but it results in slower speed and necessity to figure the settings yourself.

Q: What PC is needed to run Stable Diffusion?

A: There are several versions of Stable Diffusion with understandable interface that can be installed on PC. Any desktop computer with a good GPU can be used. For technical reasons nVidia GPUs are generally more suitable for Stable Diffusion and therefore are strongly recommended. The GPU must have at least 6Gb RAM (8Gb and more are recommended; without it it will be hard to create high-resolution pictures). I strongly recommend GPUs that have 2000+ stream processors ("s.p." or "CUDA cores"), otherwise you won't be able to make pictures in a reasonable amount of time. I once launched Stable Diffusion on 4Gb GTX 1050 Ti, but it was very slow and wasn't able to unlock the potential.

Q: What downloadable version of Stable Diffusion do you use?

A: Stable Diffusion web UI by Automatic1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui

Q: Is it easy to use Stable Diffusion?

A: It's easy to make simple pictures, but more complex pictures will need experience and attention. Study the manuals to understand basic. I wrote down effects of different settings and prompts for several days. Good understanding of computing will also help. Anyway, the fast-growing and friendly community will help you.

Q: What's the speed of creation of pictures?

A: Time to generate a picture strongly depends on your PC, and the most significant parameter is number of s.p.'s (CUDA cores) of your GPU. If you have nVidia 3000/4000 GPU, small pictures (512*512 is the basic resolution) will take less than 10-15 seconds (depends on settings and your PC). Bigger images - more time: 1024*1024 picture will take nearly 4 times more time. High-resolution image can take 1 or even 2 minutes, if your GPU have enough RAM to start such task. You can estimate the time using charts from here: https://www.tomshardware.com/pc-components/gpus/stable-diffusion-benchmarks

Q: Can I create any pictures in any style using the default preferences?

A: It will be hard. It's much better to download special "models" to create content you want to in some specific style. Some models are suitable for anime-style pictures, others are photorealistic, and so on. You can use https://civitai.com/ to download new models. I also strongly recommend to use this site to see which prompts other creators are using.

Q: Can I control face for portraits?

A: Yes, you can use LORAs and hypernetworks. Hypernetworks can be created easily, LORAs are harder to make, but more effective. Please don't make hypernetworks and LORAs with your ex ;)

Q: Can I create videos with Stable Diffusion?

A: Yes, there're several extensions to work with video (Deforum, Animator and others). However, generation of video will take all resources of your PC for hours.

Q: There're many other generative AIs (Midjourney, DALL-E 2 and others). Are they worse than Stable Diffusion?

A: They're neither worse, nor better in quality of images, they're just different. What makes Stable Diffusion better is that it is open-sourced and therefore free to use and improve by enthusiasts. It's like Wikipedia in the world of closed paid encyclopedias.

Q: So, I can download a program, write some prompts, and get perfect pictures, right?

A: Sadly, but no. Generative AI so far has many problems, such as bad understanding of human anatomy (especially fingers for some reason). It also lacks understanding of integrity of pictures. Creating pictures is more like an alchemy from fantasy books and games than real chemistry with strict rules: sometimes you have to add the rabbit's foot to have a good result, but you don't understand why you have to do this. 80% of pictures I created have incurable problems, 15% have small problems, 5% are good enough to share. There're many strange rules that can't be explained without good understanding of machine learning and other special knowledge, so be prepared for unexpected problems.

Yours, https://www.instagram.com/elitepics1/