Random Seed:  1123
cuda:0 will be used.
Loading params from results/Fairfaces/2023-01-29/04:57:35_fairfaces_resnet/config.txt
Latent dims: 256
Verifying against the following:
 ['Inv. to Head Flip']
Loaded dataset with classes: 2 (Male, Female) and attributes: 3 (Black, Indian, East Asian)
--------------------
Classes: ['Male', 'Female']
Conditioning attributes: [['Black', 'Indian', 'East Asian']]
Conditional losses: ['CE'] (total conditional_ldims: 3)
--------------------
Dataset: Fairfaces
Batch Size: 1
Training Batches: 85019
Validation Batches: 858
Test Batches: 867
--------------------

-------------------------
Feature Detection Network
-------------------------
===============================================================================================
Layer (type:depth-idx)                        Output Shape              Param #
===============================================================================================
FDN                                           [1, 512, 1, 1]            --
├─Sequential: 1-1                             [1, 512, 1, 1]            --
│    └─Conv2d: 2-1                            [1, 64, 32, 32]           9,408
│    └─BatchNorm2d: 2-2                       [1, 64, 32, 32]           128
│    └─ReLU: 2-3                              [1, 64, 32, 32]           --
│    └─MaxPool2d: 2-4                         [1, 64, 16, 16]           --
│    └─Sequential: 2-5                        [1, 64, 16, 16]           --
│    │    └─BasicBlock: 3-1                   [1, 64, 16, 16]           73,984
│    │    └─BasicBlock: 3-2                   [1, 64, 16, 16]           73,984
│    └─Sequential: 2-6                        [1, 128, 8, 8]            --
│    │    └─BasicBlock: 3-3                   [1, 128, 8, 8]            230,144
│    │    └─BasicBlock: 3-4                   [1, 128, 8, 8]            295,424
│    └─Sequential: 2-7                        [1, 256, 4, 4]            --
│    │    └─BasicBlock: 3-5                   [1, 256, 4, 4]            919,040
│    │    └─BasicBlock: 3-6                   [1, 256, 4, 4]            1,180,672
│    └─Sequential: 2-8                        [1, 512, 2, 2]            --
│    │    └─BasicBlock: 3-7                   [1, 512, 2, 2]            3,673,088
│    │    └─BasicBlock: 3-8                   [1, 512, 2, 2]            4,720,640
│    └─AdaptiveAvgPool2d: 2-9                 [1, 512, 1, 1]            --
===============================================================================================
Total params: 11,176,512
Trainable params: 11,176,512
Non-trainable params: 0
Total mult-adds (M): 148.06
===============================================================================================
Input size (MB): 0.05
Forward/backward pass size (MB): 3.24
Params size (MB): 44.71
Estimated Total Size (MB): 48.00
===============================================================================================


-------------------------
Classification Head
-------------------------
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
ClassificationHead                       [1, 2]                    --
├─Sequential: 1-1                        [1, 2]                    --
│    └─Flatten: 2-1                      [1, 512]                  --
│    └─ReLU: 2-2                         [1, 512]                  --
│    └─Linear: 2-3                       [1, 2]                    1,026
==========================================================================================
Total params: 1,026
Trainable params: 1,026
Non-trainable params: 0
Total mult-adds (M): 0.00
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.00
Estimated Total Size (MB): 0.01
==========================================================================================


-------------------------
Encoding Head (1 layer)
-------------------------
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
VaeInvertibleEncodingHead                [1, 256]                  --
├─InvertibleEncodingHead: 1-1            [1, 512]                  --
│    └─Sequential: 2-1                   [1, 512]                  --
│    │    └─Flatten: 3-1                 [1, 512]                  --
│    │    └─Linear: 3-2                  [1, 512]                  262,144
==========================================================================================
Total params: 262,144
Trainable params: 262,144
Non-trainable params: 0
Total mult-adds (M): 0.26
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 0.01
Params size (MB): 2.10
Estimated Total Size (MB): 2.11
==========================================================================================

-------------------------
Encoding Head (2 layers)
-------------------------
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
VaeInvertibleEncodingHead                [1, 256]                  --
├─InvertibleEncodingHead: 1-1            [1, 512]                  --
│    └─Sequential: 2-1                   [1, 512]                  --
│    │    └─Flatten: 3-1                 [1, 512]                  --
│    │    └─Linear: 3-2                  [1, 512]                  262,144
│    │    └─LeakyReLU: 3-3               [1, 512]                  --
│    │    └─Linear: 3-4                  [1, 512]                  262,144
==========================================================================================
Total params: 524,288
Trainable params: 524,288
Non-trainable params: 0
Total mult-adds (M): 0.52
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 0.02
Params size (MB): 4.19
Estimated Total Size (MB): 4.21
==========================================================================================

-------------------------
Encoding Head (3 layers)
-------------------------
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
VaeInvertibleEncodingHead                [1, 256]                  --
├─InvertibleEncodingHead: 1-1            [1, 512]                  --
│    └─Sequential: 2-1                   [1, 512]                  --
│    │    └─Flatten: 3-1                 [1, 512]                  --
│    │    └─Linear: 3-2                  [1, 512]                  262,144
│    │    └─LeakyReLU: 3-3               [1, 512]                  --
│    │    └─Linear: 3-4                  [1, 512]                  262,144
│    │    └─LeakyReLU: 3-5               [1, 512]                  --
│    │    └─Linear: 3-6                  [1, 512]                  262,144
==========================================================================================
Total params: 786,432
Trainable params: 786,432
Non-trainable params: 0
Total mult-adds (M): 0.79
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 0.02
Params size (MB): 6.29
Estimated Total Size (MB): 6.32
==========================================================================================

-------------------------
Decoder
-------------------------
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
Decoder                                  [1, 3, 64, 64]            --
├─Sequential: 1-1                        [1, 3, 64, 64]            --
│    └─Linear: 2-1                       [1, 512]                  131,584
│    └─Reshape: 2-2                      [1, 512, 1, 1]            --
│    └─ConvTranspose2d: 2-3              [1, 512, 4, 4]            4,194,816
│    └─LeakyReLU: 2-4                    [1, 512, 4, 4]            --
│    └─ConvTranspose2d: 2-5              [1, 256, 8, 8]            2,097,408
│    └─BatchNorm2d: 2-6                  [1, 256, 8, 8]            512
│    └─LeakyReLU: 2-7                    [1, 256, 8, 8]            --
│    └─ConvTranspose2d: 2-8              [1, 128, 16, 16]          524,416
│    └─BatchNorm2d: 2-9                  [1, 128, 16, 16]          256
│    └─LeakyReLU: 2-10                   [1, 128, 16, 16]          --
│    └─ConvTranspose2d: 2-11             [1, 64, 32, 32]           131,136
│    └─BatchNorm2d: 2-12                 [1, 64, 32, 32]           128
│    └─LeakyReLU: 2-13                   [1, 64, 32, 32]           --
│    └─ConvTranspose2d: 2-14             [1, 64, 64, 64]           65,600
│    └─BatchNorm2d: 2-15                 [1, 64, 64, 64]           128
│    └─LeakyReLU: 2-16                   [1, 64, 64, 64]           --
│    └─ConvTranspose2d: 2-17             [1, 3, 64, 64]            1,731
│    └─Tanh: 2-18                        [1, 3, 64, 64]            --
==========================================================================================
Total params: 7,147,715
Trainable params: 7,147,715
Non-trainable params: 0
Total mult-adds (M): 745.81
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 6.20
Params size (MB): 28.59
Estimated Total Size (MB): 34.79
==========================================================================================


-------------------------
Model inversion verified w.r.t 0.001 (maximum inversion difference was 1.3251622021925868e-12)
-------------------------
Loading MODEL from results/Fairfaces/2023-01-29/04:57:35_fairfaces_resnet/models/Epoch_49_acc_0.82_vaeloss_1144.87.tar

Batches 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:02<00:00, 152.44 batch/s]

Models test accuracy is : 0.8240000605583191

