# Attribute Control Supplementary Material

This supplementary material contains a reference implementation of the Attribute Control method. Please refer to https://github.com/CompVis/attribute-control/ for an updated version.

## Usage
### Setup
Just install the requirements via `pip install -r requirements.txt`, then you're ready to go. For usage, see the examples below, everything else that's needed (model checkpoints) will be downloaded automatically.

### Inference
We provide a range of learned deltas for SDXL used in the paper at `pretrained_deltas`. These can also be used for models such as SD 1.5 or LDM3D by just loading them as usual.

### Real Image Editing
We also provide an example for real image editing at `notebooks/real_image_editing` based on ReNoise and SDXL Turbo.
This allows real image editing with our method. 

### Creating new Attribute Deltas
When creating deltas for new attributes, start by creating a config for them akin to `configs/prompts/people/age.yaml`. There are multiple entries of base prompts that correspond to the attribute in a neutral, "negative", and "positive" direction. Please make sure to use the same noun for all the prompts per entry and specify it as the `pattern_target`.
You can also specify a list of prefixes that contain various other words that will be added before the main prompt to help obtain more robust deltas. The syntax used finds all sets of words enclosed in braces (e.g., `{young,old}`) and then generates all combinations of words in the braces.

#### Learning-based Method
The best method to obtain deltas is the learning-based method, although it takes substantially longer than the naive method (see below)

To obtain a delta with the naive method, use:
```shell
python learn_delta.py device=cuda:0 model=sdxl prompts=people/age
```
This will save the delta at `outputs/learn_delta/people/age/runs/<date>/<time>/checkpoints/delta.pt`, which you can then directly use as shown in the example notebooks.

This will typically require slightly more than 24GB of VRAM for training (26GB when training on an A100, although this will likely change with newer versions of diffusers and PyTorch). If you want to train on smaller hardware, you can enable gradient checkpointing (typically called activation checkpointing, but we'll stick to diffusers terminology here) by launching the training as
```shell
python learn_delta.py device=cuda:0 model=sdxl prompts=people/age model.compile=False +model.gradient_checkpointing=True
```
In our experiments, this enabled training deltas with a 11.5GB VRAM budget, at the cost of slower training.

#### Naive CLIP Difference Method
The simplest method to obtain deltas is the naive CLIP difference-based method. With it, you can obtain a delta in a few seconds on a decent GPU. It is substantially worse than the proper learned method though.

To obtain a delta with the naive method, use (same arguments as for the learning-based method):
```shell
python learn_delta_naive_clip.py device=cuda:0 model=sdxl prompts=people/age
```
This will save the delta at `outputs/learn_delta_naive_clip/people/age/runs/<date>/<time>/checkpoints/delta.pt`, which you can then directly use as shown in the example notebooks.
