# Results: 3D Nerve Segmentation QPBO Benchmark¶

Faster Multi-Object Segmentation using Parallel Quadratic Pseudo-Boolean Optimization, ICCV 2021 Paper

Authors: Niels Jeppesen (niejep@dtu.dk) and Patrick M. Jensen (patmjen@dtu.dk)

This notebook is used to analyze the benchmark results from the ParallelNerveSegmentation3DPart1.ipynb notebook. The benchmark is testing the performance of three different QPBO implementations: K-QPBO, M-QPBO and P-QPBO. The K-QPBO imlementation found in the `thinqpbo` package, which is almost identical to the original implementation by Vladimir Kolmogorov. P-QPBP is our new parallel QPBO implementation and M-QPBO is our serial QPBO implementation.

In [1]:

    import os
    from glob import glob

    import matplotlib.pyplot as plt
    import numpy as np
    import pandas as pd
    import scipy.optimize

## Load and display results¶

First we load the benchmark results from the CSV files. Once we've loaded the results we display the dataframe. Change the variables in the cell below to save figures or change directories.

In [2]:

    # Should figures be saved?
    save_figures = False

    # Directory to save figures.
    figure_dir = 'figures'

    # Directory with benchmark results.
    benchmark_dir = '../benchmark/nerve_benchmarks/'
    benchmark_paths = glob(os.path.join(benchmark_dir, 'parallel_qpbo_benchmark_results_*.csv'))
    benchmark_paths

Out[2]:

    ['../benchmark/nerve_benchmarks\\parallel_qpbo_benchmark_results_20210617-130430.csv',
     '../benchmark/nerve_benchmarks\\parallel_qpbo_benchmark_results_20210617-175617.csv']

In [3]:

    df_all = pd.read_csv(benchmark_paths[0], index_col=0)
    for p in benchmark_paths[1:]:
        df_all = df_all.append(pd.read_csv(p, index_col=0), ignore_index=True)

    df_all

Out[3]:

| Class | NodeCount | EdgeCount | BuildTime | SolveTime | WeakPersistenciesTime | TwiceEnergy | Timestamp | CpuCount | ShortName | SystemName | SystemCpu | SystemCpuCount | NerveCount
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
0 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | 363748800 | 2124073454 | 80.653347 | 581.638645 | 9.369483 | 7574420340 | 2021-06-17 00:02:53.123763 | 1 | Para (1) | n-62-11-56 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
1 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | 363748800 | 2124073454 | 77.510277 | 581.703961 | 9.508867 | 7574420340 | 2021-06-17 00:14:21.601783 | 1 | Para (1) | n-62-11-56 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
2 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | 363748800 | 2124073454 | 78.704689 | 558.576859 | 5.895914 | 7574420340 | 2021-06-17 00:25:16.826738 | 1 | Para (1) | n-62-11-56 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
3 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | 363748800 | 2124073454 | 73.538075 | 582.549264 | 9.268695 | 7574420340 | 2021-06-17 00:36:41.875594 | 1 | Para (1) | n-62-11-56 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
4 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | 363748800 | 2124073454 | 77.922847 | 577.593899 | 9.768811 | 7574420340 | 2021-06-17 00:48:03.866198 | 1 | Para (1) | n-62-11-56 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ...
164 | QPBOInt | 818434800 | 4864255488 | 897.128082 | 5340.008811 | 37.700538 | 311358216 | 2021-06-17 12:57:57.881656 | -1 | QPBO | n-62-11-52 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
165 | QPBOInt | 818434800 | 4864255488 | 862.803517 | 4986.645045 | 25.902894 | 311358216 | 2021-06-17 14:37:41.872564 | -1 | QPBO | n-62-11-52 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
166 | QpboCapInt32ArcIdxUInt64NodeIdxUInt32 | 818434800 | 4864255488 | 107.169996 | 3746.184516 | 28.300907 | 14876534453474 | 2021-06-17 15:43:38.488163 | -1 | Qpbo | n-62-11-52 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
167 | QpboCapInt32ArcIdxUInt64NodeIdxUInt32 | 818434800 | 4864255488 | 130.182441 | 3868.691022 | 15.895734 | 14876534453474 | 2021-06-17 16:51:21.945680 | -1 | Qpbo | n-62-11-52 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216
168 | QpboCapInt32ArcIdxUInt64NodeIdxUInt32 | 818434800 | 4864255488 | 114.698075 | 3703.791146 | 14.528365 | 14876534453474 | 2021-06-17 17:56:12.716527 | -1 | Qpbo | n-62-11-52 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 216

169 rows × 14 columns

### Show configurations¶

To get an overview of the data we've loaded, we print the different configurations.

In [4]:

    print('Classes:')
    for n in df_all['Class'].unique().tolist():
        print(f'\t{n}')

    print('Nerve counts:')
    for n in df_all['NerveCount'].unique().tolist():
        print(f'\t{n}')

    print('Node counts:')
    for n in df_all['NodeCount'].unique().tolist():
        print(f'\t{n}')

    print('Edge counts:')
    for n in df_all['EdgeCount'].unique().tolist():
        print(f'\t{n}')

    print('CPU counts:')
    for n in df_all['SystemCpuCount'].unique().tolist():
        print(f'\t{n}')

    Classes:
    	ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32
    	QPBOInt
    	QpboCapInt32ArcIdxUInt32NodeIdxUInt32
    	ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32
    	QpboCapInt32ArcIdxUInt64NodeIdxUInt32
    Nerve counts:
    	216
    Node counts:
    	363748800
    	818434800
    Edge counts:
    	2124073454
    	4864255488
    CPU counts:
    	32

### Change short name¶

For the purpose of plotting, we update the `ShotName` column values.

In [5]:

    df = df_all.copy()
    df.loc[df['Class'] == 'QPBOInt', 'ShortName'] = 'K-QPBO'
    df.loc[df['Class'].str.startswith('QpboCap'), 'ShortName'] = 'M-QPBO'
    df.loc[df['Class'].str.startswith('ParallelQpboCap'), 'ShortName'] = 'P-QPBO'
    df.loc[df['CpuCount'] != -1, 'ShortName'] += ' (' + df['CpuCount'].astype(np.str) + ')'
    df['SystemCpuCount'] = df['SystemCpuCount'].astype(np.int16)
    df['TotalTime'] = df['BuildTime'] + df['SolveTime'] + df['WeakPersistenciesTime']

### Filter out results for other systems and configurations¶

We only want to work with results from a specific system and configuration, so we filter out other. This should have no effect for the data included in the supplementary material.

In [6]:

    mask = df['NerveCount'] == 216
    mask &= df['SystemCpu'].str.contains('Gold 6226R')
    mask &= (~df['Class'].str.contains('CapInt') | df['Class'].str.contains('CapInt32'))

    df = df[mask]

### Group and show solve times¶

We can now group the data to get an overview of the solve times for N1 and N2 for each algorithm and thread configuration. The results are used in the paper, where we report the minimum (best) solve time for each group.

In [7]:

    df_group = df.groupby(['NodeCount', 'EdgeCount', 'Class', 'SystemCpu', 'CpuCount'])
    df_group[['SolveTime']].describe()

Out[7]:

|  |  |  |  | SolveTime
--- | --- | --- | --- | --- | ---
|  |  |  |  | count | mean | std | min | 25% | 50% | 75% | max
NodeCount | EdgeCount | Class | SystemCpu | CpuCount |  |  |  |  |  |  |  |
363748800 | 2124073454 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 1 | 10.0 | 579.307223 | 11.875941 | 558.328071 | 578.605085 | 582.126613 | 585.055149 | 594.456977
2 | 10.0 | 319.854369 | 1.576121 | 317.279246 | 318.922619 | 319.809541 | 320.916663 | 322.176483
4 | 10.0 | 195.173083 | 1.620437 | 192.559520 | 194.516620 | 195.562399 | 195.923060 | 197.797684
8 | 10.0 | 131.076683 | 2.272190 | 128.020746 | 129.337352 | 131.077217 | 132.822038 | 134.440500
16 | 10.0 | 97.358785 | 2.053922 | 94.463284 | 95.732710 | 97.290871 | 99.037019 | 100.375021
24 | 10.0 | 90.315316 | 3.335071 | 83.846947 | 87.996215 | 90.835559 | 92.905286 | 94.450838
32 | 10.0 | 86.877261 | 3.241845 | 79.910048 | 85.663351 | 87.359990 | 88.837200 | 90.543388
40 | 10.0 | 76.847025 | 3.325948 | 74.392834 | 74.999791 | 75.735599 | 76.222564 | 85.000041
48 | 10.0 | 78.900290 | 1.834891 | 76.210279 | 77.995580 | 78.616811 | 79.771020 | 81.765956
56 | 10.0 | 80.018581 | 2.378089 | 77.410612 | 78.180758 | 79.773072 | 80.899562 | 84.921396
64 | 10.0 | 81.107813 | 1.599401 | 79.379471 | 79.766957 | 80.890435 | 81.978347 | 84.027035
QPBOInt | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 10.0 | 841.373733 | 4.298516 | 836.261239 | 836.729420 | 843.534726 | 843.862978 | 848.021129
QpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 10.0 | 627.905059 | 0.083595 | 627.784739 | 627.860083 | 627.894342 | 627.978344 | 628.024728
818434800 | 4864255488 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 1 | 3.0 | 2531.628094 | 96.514441 | 2429.012243 | 2487.149097 | 2545.285952 | 2582.936020 | 2620.586087
2 | 3.0 | 1374.460305 | 3.726936 | 1370.174875 | 1373.218386 | 1376.261898 | 1376.603021 | 1376.944144
4 | 3.0 | 796.362165 | 1.229802 | 794.954621 | 795.928841 | 796.903061 | 797.065936 | 797.228811
8 | 3.0 | 472.311812 | 2.091032 | 471.016149 | 471.105661 | 471.195172 | 472.959643 | 474.724114
16 | 3.0 | 325.156917 | 2.024735 | 322.850512 | 324.414543 | 325.978575 | 326.310120 | 326.641664
24 | 3.0 | 284.964727 | 9.873397 | 276.624371 | 279.513898 | 282.403425 | 289.134905 | 295.866385
32 | 3.0 | 268.076653 | 3.722665 | 263.876128 | 266.631259 | 269.386390 | 270.176916 | 270.967441
40 | 3.0 | 248.331926 | 3.224294 | 244.620095 | 247.278665 | 249.937235 | 250.187842 | 250.438449
48 | 3.0 | 246.858319 | 7.648632 | 239.071892 | 243.106841 | 247.141790 | 250.751532 | 254.361275
56 | 3.0 | 253.982875 | 9.031339 | 243.653976 | 250.778060 | 257.902144 | 259.147324 | 260.392505
64 | 3.0 | 256.466345 | 15.813020 | 240.437121 | 248.672562 | 256.908003 | 264.480956 | 272.053909
QPBOInt | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 3.0 | 5127.883808 | 187.043128 | 4986.645045 | 5021.821307 | 5056.997570 | 5198.503190 | 5340.008811
QpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 3.0 | 3772.888895 | 85.631975 | 3703.791146 | 3724.987831 | 3746.184516 | 3807.437769 | 3868.691022

## Plot results¶

To plot the results, we define a plot function.

In [8]:

    def time_plot(df, column, label, title=None, filename=None, relative_to=None, speedup=False, per_thread=False, return_ax=False, ax=None, plot_label=None, amdahl_fit=False):
        """Used for plotting results of the N1 and N2 segmentation tasks."""

        # Copy dataframe to avoid changing original.
        df = df.copy()

        pqpbo_only = False
        max_value = False

        if relative_to is not None:
            # If plotting the data relative to a specific configuration (e.g. K-QPBO).
            df[column] /= df[df['ShortName'] == relative_to][column].min()
            if speedup:
                # If we're plotting th speed-up we inverse the relative values. We will then be taking the max, rather than min values.
                max_value = True
                df[column] = 1 / df[column]
                if per_thread:
                    # If we're calculating speed-up per thread (efficiency).
                    pqpbo_only = True
                    cpu_mask = df['CpuCount'] > 1
                    df.loc[cpu_mask, column] /= df.loc[cpu_mask, 'CpuCount']

        # Separate data.
        df_qpbo = df[df['Class'].str.startswith('QPBO')]
        df_mqpbo = df[df['Class'].str.startswith('QpboCap')]
        df = df[df['Class'].str.startswith('ParallelQpboCap')]

        # Group data.
        df_group = df.groupby(['Class', 'SystemCpu', 'CpuCount'], sort=False)
        df_group.describe()

        # Create new plot or re-use.
        if ax is None:
            fig, ax = plt.subplots(1, 1, figsize=(5, 2.5))
            style = '.-'
        else:
            style = '*-'

        color = None
        if not pqpbo_only:
            ax.axhline(df_qpbo['SolveTime'].min(), color=plt.cm.Set1(0), ls='--', label=df_qpbo['ShortName'].iloc[0])
            ax.axhline(df_mqpbo['SolveTime'].min(), color=plt.cm.Set1(1), ls=':', label=df_mqpbo['ShortName'].iloc[0])
            color = plt.cm.Set1(2)

        # Take min or max value.
        if max_value:
            agg = df_group[column].max().reset_index()
        else:
            agg = df_group[column].min().reset_index()

        if relative_to is not None and speedup and per_thread:
            agg['SolveTime'] /= agg['SolveTime'].max()
            ax.set_ylim(0, 1)
            ax.set_yticks(np.arange(0, 1.1, .1))

        # Plot data.
        agg.plot(ax=ax, x='CpuCount', y='SolveTime', style=style, color=color, label='P-QPBO' if plot_label is None else plot_label)

        if return_ax:
            return ax

        if relative_to is not None and speedup and not per_thread:
            if amdahl_fit:
                amdahl_p = scipy.optimize.least_squares(lambda p: agg['SolveTime']/agg['SolveTime'][0] - 1/(1 - p + p/agg['CpuCount']), 0).x[0]
                plt.plot(agg['CpuCount'], agg['SolveTime'][0]/(1 - amdahl_p + amdahl_p/agg['CpuCount']), '--', color='gray', label=f"Amdahl's law fit (p={amdahl_p:3.2f})")
            ymax = int(ax.get_ylim()[1] + 0.5)
            ax.set_ylim(0, ymax)
            ax.set_yticks(np.arange(0, ymax + 1, 1), minor=True)

        ax.set_xlim(df['CpuCount'].min() - 1, df['CpuCount'].max() + 1)
        ax.legend()
        ax.grid(axis='y')
        ax.set_ylabel(label)
        ax.set_xticks(df['CpuCount'])
        ax.set_xlabel('Number of threads')
        plt.tight_layout()
        if save_figures and filename is not None:
            plt.savefig(os.path.join(figure_dir, filename))
        ax.set_title(title)
        plt.show()

### N1 plots¶

Let's plot the results for the N1 task. The third plot is used in the paper.

In [9]:

    # Keep results from the smallest task (N1).
    mask = df['NodeCount'] == df['NodeCount'].min()
    df_n1 = df[mask]

    # Append to figure file name.
    fig_append = '_n1'

    # Create title (only used in notebook).
    title = f"{df_n1['SystemCpu'].iloc[0].split('@')[0].strip()} ({df_n1['SystemCpuCount'].iloc[0]} CPUs), 32-bit capacities"

    # Create plot with solve time.
    time_plot(df_n1, 'SolveTime', 'Solve time (s)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r{fig_append}.pdf')
    # Create plot with relative solve time.
    time_plot(df_n1, 'SolveTime', 'Solve time (relative)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r_rel{fig_append}.pdf', relative_to='K-QPBO')
    # Create plot with relative speed-up.
    time_plot(df_n1, 'SolveTime', 'Solve time speed-up (times)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r_relx{fig_append}_amdahl1.pdf', relative_to='K-QPBO', speedup=True, amdahl_fit=True)

### N2 plots¶

And then the results for the N2 task. The third plot is used in the paper.

In [10]:

    # Keep results from the largest task (N2).
    mask = df['NodeCount'] == df['NodeCount'].max()
    df_n2 = df[mask]

    # Append to figure file name.
    fig_append = '_n2'

    # Create title (only used in notebook).
    title = f"{df_n2['SystemCpu'].iloc[0].split('@')[0].strip()} ({df_n2['SystemCpuCount'].iloc[0]} CPUs), 32-bit capacities"

    # Create plot with solve time.
    time_plot(df_n2, 'SolveTime', 'Solve time (s)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r{fig_append}.pdf')
    # Create plot with relative solve time.
    time_plot(df_n2, 'SolveTime', 'Solve time (relative)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r_rel{fig_append}.pdf', relative_to='K-QPBO')
    # Create plot with relative speed-up.
    time_plot(df_n2, 'SolveTime', 'Solve time speed-up (times)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r_relx{fig_append}_amdahl1.pdf', relative_to='K-QPBO', speedup=True, amdahl_fit=True)

### N1 and N2 thread efficiency comparison¶

We can also compare the per thread efficiency for each task. The baseline for each task is the time it takes for P-QPBO(1). If P-QPBO(20) is 20 times faster than P-QPBO(1) is has an efficincy of one. If it is 10 times faster the effeciency is 0.5, and so on.

In [11]:

    # Create title (only used in notebook).
    title = f"{df['SystemCpu'].iloc[0].split('@')[0].strip()} ({df['SystemCpuCount'].iloc[0]} CPUs), 32-bit capacities"

    # Create efficiency plot for both tasks.
    ax = time_plot(df_n1, 'SolveTime', 'Thread effciency (relative)', relative_to='K-QPBO', speedup=True, per_thread=True, return_ax=True, plot_label='P-QPBO(N1)')
    time_plot(df_n2, 'SolveTime', 'Thread effciency (relative)', title=title, filename=f'plot_qpbo_solve_cap32_xeon_gold_6226r_relxt.pdf', relative_to='K-QPBO', speedup=True, per_thread=True, ax=ax, plot_label='P-QPBO(N2)')

## Calculate relative speed-up as table¶

Get speed-up numbers to use in text and tables.

In [12]:

    df_group = df_n1.groupby(['Class', 'SystemCpu', 'CpuCount'], sort=False)
    df = df_group['SolveTime'].min().reset_index()
    ref_val = df[df['Class'] == 'QPBOInt']['SolveTime'].iloc[0]
    df['SpeedUp'] = ref_val / df['SolveTime']
    df['ReductionK'] = (ref_val - df['SolveTime']) / ref_val
    ref_val = df[df['Class'] == 'QpboCapInt32ArcIdxUInt32NodeIdxUInt32']['SolveTime'].iloc[0]
    df['ReductionM'] = (ref_val - df['SolveTime']) / ref_val
    df.style.highlight_max(subset=['SpeedUp'], color='yellow')

Out[12]:

| Class | SystemCpu | CpuCount | SolveTime | SpeedUp | ReductionK | ReductionM
--- | --- | --- | --- | --- | --- | --- | ---
0 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 1 | 558.328071 | 1.497795 | 0.332352 | 0.110638
1 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 2 | 317.279246 | 2.635726 | 0.620598 | 0.494605
2 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 4 | 192.559520 | 4.342871 | 0.769738 | 0.693271
3 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 8 | 128.020746 | 6.532232 | 0.846913 | 0.796075
4 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 16 | 94.463284 | 8.852765 | 0.887041 | 0.849529
5 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 24 | 83.846947 | 9.973664 | 0.899736 | 0.866440
6 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 79.910048 | 10.465032 | 0.904444 | 0.872711
7 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 40 | 74.392834 | 11.241153 | 0.911041 | 0.881499
8 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 48 | 76.210279 | 10.973077 | 0.908868 | 0.878604
9 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 56 | 77.410612 | 10.802928 | 0.907432 | 0.876692
10 | ParallelQpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 64 | 79.379471 | 10.534981 | 0.905078 | 0.873556
11 | QPBOInt | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 836.261239 | 1.000000 | 0.000000 | -0.332083
12 | QpboCapInt32ArcIdxUInt32NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 627.784739 | 1.332083 | 0.249296 | 0.000000

In [13]:

    df_group = df_n2.groupby(['Class', 'SystemCpu', 'CpuCount'], sort=False)
    df = df_group['SolveTime'].min().reset_index()
    ref_val = df[df['Class'] == 'QPBOInt']['SolveTime'].iloc[0]
    df['SpeedUp'] = ref_val / df['SolveTime']
    df['ReductionK'] = (ref_val - df['SolveTime']) / ref_val
    ref_val = df[df['Class'] == 'QpboCapInt32ArcIdxUInt64NodeIdxUInt32']['SolveTime'].iloc[0]
    df['ReductionM'] = (ref_val - df['SolveTime']) / ref_val
    df.style.highlight_max(subset=['SpeedUp'], color='yellow')

Out[13]:

| Class | SystemCpu | CpuCount | SolveTime | SpeedUp | ReductionK | ReductionM
--- | --- | --- | --- | --- | --- | --- | ---
0 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 1 | 2429.012243 | 2.052952 | 0.512897 | 0.344182
1 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 2 | 1370.174875 | 3.639422 | 0.725231 | 0.630062
2 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 4 | 794.954621 | 6.272868 | 0.840583 | 0.785367
3 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 8 | 471.016149 | 10.586994 | 0.905544 | 0.872829
4 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 16 | 322.850512 | 15.445678 | 0.935257 | 0.912832
5 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 24 | 276.624371 | 18.026774 | 0.944527 | 0.925313
6 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 32 | 263.876128 | 18.897674 | 0.947083 | 0.928755
7 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 40 | 244.620095 | 20.385263 | 0.950945 | 0.933954
8 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 48 | 239.071892 | 20.858349 | 0.952058 | 0.935452
9 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 56 | 243.653976 | 20.466093 | 0.951139 | 0.934215
10 | ParallelQpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | 64 | 240.437121 | 20.739913 | 0.951784 | 0.935084
11 | QPBOInt | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 4986.645045 | 1.000000 | 0.000000 | -0.346362
12 | QpboCapInt32ArcIdxUInt64NodeIdxUInt32 | Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz | -1 | 3703.791146 | 1.346362 | 0.257258 | 0.000000

In [ ]: