Training our own checkpoint

We had to jump on the Generative AI craze! As early adopters of new tech decided to look into it and train our checkpoint based on Stable Diffusion 1.5, we were blown away. We uploaded several images of our guinea pig Teodor and let Google Collab do the computation using a publically available Jupyter notebook.

We customized it to fit our vision and had a checkpoint that we were excited to try out. As true nerds, we decided not to download and use a GUI but to scribble some Python code in a Jupyter notebook, load up the checkpoint, let it run for hours, and put the M1 chip to the test.

We tried many different prompts and we kept track of the prompts and the results respectively by committing every iteration to a git repository using the actual prompt as a commit message and looking at the commit history.

The general theme of course was our slogan: The cloud heroes behind your platform

Results

The prompt we used is:

teodor as powerful superhero trying to solve a complex problem, show his face, intricate, elegant, highly detailed, digital painting, concept art, smooth, sharp focus, illustration

We used the following instance prompt

photo of teodor

The prompt above allowed us to refer to the character as teodor in our prompts

Our class prompt was simply

photo of a man

We found that 75 num_inference_steps produce better results, although online discussions suggest that 50 is sufficient.

Training dataset

Some of the images we used to train the model:

Source Code

The code below is aimed at the M1 chip; most examples online are tailored to NVIDIA GPUs, so it took some research to figure out exactly how to run it on M1.

import torch
import os
import uuid
from IPython.display import Image
from diffusers import StableDiffusionPipeline

unique_filename = str(uuid.uuid4()) + ".png"

#https://huggingface.co/docs/diffusers/optimization/mps
pipe = StableDiffusionPipeline.from_pretrained("forwardforce/teodor")
pipe = pipe.to("mps")

# Recommended if your computer has < 64 GB of RAM
pipe.enable_attention_slicing()
generator = torch.manual_seed(9)

prompt = "teodor as powerful superhero trying to solve a complex problem, show his face, intricate, elegant, highly detailed, digital painting, concept art, smooth, sharp focus, illustration"

_ = pipe(prompt, num_inference_steps=75, generator=generator)


# Results match those from the CPU device after the warmup pass.
image = pipe(prompt).images[0]

image.save("./" + unique_filename)

os.system('git add . && git commit -m "' + prompt + '" && git push origin main')

Image("./" + unique_filename)

Give credit when credit is due

Our admiration Huggingface for providing the platform and all the blog posts and StabilityAI for doing the magic, funding it, and open-sourcing it! ❤️