Kyle R Bryenton, 2025-11-18

The following package was used for the analysis of the article: 
    WTMAD-4: A Fair Weighting Scheme for GMTKN55
    Kyle R. Bryenton, Erin R. Johnson

This version pertains to the response to reviewers comments. A major revision was made to
- Remove new XDM, TS, and MBD related data to focus on the weighting scheme.  This will be included in a future article.
- The WTMAD-4 weight bins were removed in favour of an individualized floating-point value used to weight each benchmark
- Comparisons are made between fitting between: 
  - (a) D3_10:   10 D3-corrected hybrid functionals, 
  - (b) ALL_115: All 115 DC-functionals obtained from literature.


The analysis proceeded in the following order, indicated by the directory structure:

01_FITTING_WTMAD4 
Consists of two fit sets: D3_10 and ALL_115
The weights for WTMAD-4 were based off this data.

02_SCRAPED_DATA
The raw, scraped data used for this study. 
The format seen is whatever was obtained from copying the ESI of the respective PDFs. 
The headers contain the article it came from, as well as some shell script one-liners to help with processing it.

03_EVAL_RESULTS
For data newly acquired, this data is paired down from the output format from aoterodelaroza's refdata/40_eval_driver 
See: https://github.com/aoterodelaroza/refdata
For the scraped data, it was put in a similar form to be processed using the same post-processing bash script.

04_GMTKN55_ANALYSIS_D3_10 and 04_GMTKN55_ANALYSIS_ALL_115
Using the data from 03_EVAL_RESULTS, the ProcessGMTKN55.sh shell script generates tables of outputs.
A copy of the script is provided.
Newest version obtainable from https://github.com/KyleBryenton/FHIaims-Toolbox

05_DC_COMPARISON [Omitted. Previously contained XDM, TS, and MBD data. Will be included in future publication.]
Following the same processing steps outlined above, XDM, TS, MBD, MBD-NL, and D3(BJ) were compared from this data.

06_WEIGHTS_D3_10 and 06_WEIGHTS_ALL_115
To determine the % contribution to total WTMAD, a Mathematica script (provided) was used that took inputs from 04_GMTKN55_ANALYSIS
The outputs are stored here, breaking down the results for each functional.

07_PLOTS_AND_DATA_D3_10 and 07_PLOTS_AND_DATA_ALL_115
More Mathematica scripts used to generate the data and plots in the main article and ESI are included here.

#----------------------------------------

## REFERENCE DATA TABLE
 # Rearrange rows to change the subset ordering in the output.
 # Comment out a row to exclude that subset from the output.
 # All errors will adjust accordingly, automatically.
 #     Column 1: Subset Name
 #     Column 2: Number of systems in the subset
 #     Column 3: Average relative absolute energy \bar{|\Delta E|} in kcal/mol        [Used for WTMAD-2]
 #     Column 4: Weight (=10.0 if deltaE < 7.5, =0.1 if deltaE > 75, = 1.0 otherwise) [Used for WTMAD-1]
 #     Column 5: Indexing (0 = basicsmall, 1 = isolarge, 2 = barriers, 3 = intermolNCI, 4 = intramolNCI)
 #     Column 6: Weights such that each "typical" functional+DC gives equal weights for each benchmark. [Used for WTMAD-4]
 # To obtain ref energies from your .din, use the following one-liner:
 #     cat *.din | awk '/^0$/ { getline; print }' | awk '{sum+=sqrt($1*$1); n++} END {if(n>0) printf "%.2f\n", sum/n}'     
weight_Type=1 #1 = D3-10, 2 = ALL-115
if ((weight_Type == 1)) ; then
    ### WTMAD-4 Weights determined using D3-10 Dataset
    GMTKN55_info=(
      "AL2X6        6     35.88     1.0    0    3.27 "
      "ALK8         8     62.60     1.0    0    1.30 "
      "ALKBDE10    10    100.69     0.1    0    1.06 "
      "BH76RC      30     21.39     1.0    0    2.55 "
      "DC13        13     54.98     1.0    0    0.78 "
      "DIPCS10     10    654.26     0.1    0    1.65 "
      "FH51        51     31.01     1.0    0    2.39 "
      "G21EA       25     33.62     1.0    0    2.15 "
      "G21IP       36    257.61     0.1    0    1.63 "
      "G2RC        25     51.26     1.0    0    1.12 "
      "HEAVYSB11   11     58.02     1.0    0    2.73 "
      "NBPRC       12     27.71     1.0    0    2.32 "
      "PA26        26    189.05     0.1    0    1.98 "
      "RC21        21     35.70     1.0    0    1.45 "
      "SIE4x4      16     33.72     1.0    0    0.469"
      "TAUT15      15      3.05    10.0    0    5.80 "
      "W4-11      140    306.91     0.1    0    1.07 "
      "YBDE18      18     49.28     1.0    0    2.55 "
      "BSR36       36     16.20     1.0    1    2.03 "
      "C60ISO       9     98.25     0.1    1    1.19 "
      "CDIE20      20      4.06    10.0    1    5.75 "
      "DARC        14     32.47     1.0    1    1.44 "
      "ISO34       34     14.57     1.0    1    4.49 "
      "ISOL24      24     21.92     1.0    1    1.99 "
      "MB16-43     43    468.39     0.1    1    0.329"
      "PArel       20      4.63    10.0    1    5.51 "
      "RSE43       43      7.60     1.0    1    5.27 "
      "BH76        76     18.61     1.0    2    1.53 "
      "BHDIV10     10     45.33     1.0    2    1.66 "
      "BHPERI      26     20.87     1.0    2    2.29 "
      "BHROT27     27      6.27    10.0    2   11.0  "
      "INV24       24     31.85     1.0    2    4.41 "
      "PX13        13     33.36     1.0    2    1.18 "
      "WCPT18      18     34.99     1.0    2    1.77 "
      "ADIM6        6      3.36    10.0    3   36.8  "
      "AHB21       21     22.49     1.0    3    6.24 "
      "CARBHB12    12      6.04    10.0    3    5.50 "
      "CHB6         6     26.79     1.0    3    4.14 "
      "HAL59       59      4.59    10.0    3    9.63 "
      "HEAVY28     28      1.24    10.0    3   14.0  "
      "IL16        16    109.04     0.1    3   11.5  "
      "PNICO23     23      4.27    10.0    3    7.86 "
      "RG18        18      0.58    10.0    3   29.9  "
      "S22         22      7.30    10.0    3   13.5  "
      "S66         66      5.47    10.0    3   18.4  "
      "WATER27     27     81.14     0.1    3    1.43 "
      "ACONF       15      1.83    10.0    4   61.2  "
      "Amino20x4   80      2.44    10.0    4   22.6  "
      "BUT14DIOL   64      2.80    10.0    4   30.0  "
      "ICONF       17      3.27    10.0    4   20.0  "
      "IDISP        6     14.22     1.0    4    2.50 "
      "MCONF       51      4.97    10.0    4   21.2  "
      "PCONF21     18      1.62    10.0    4    8.66 "
      "SCONF       17      4.60    10.0    4   19.1  "
      "UPU23       23      5.72    10.0    4   10.1  "
    )
elif ((weight_Type == 2)) ; then
    ### WTMAD-4 Weights determined using ALL-115 Dataset
    GMTKN55_info=(
      "AL2X6        6     35.88     1.0    0    2.67 "
      "ALK8         8     62.60     1.0    0    1.30 "
      "ALKBDE10    10    100.69     0.1    0    1.25 "
      "BH76RC      30     21.39     1.0    0    2.85 "
      "DC13        13     54.98     1.0    0    0.881"
      "DIPCS10     10    654.26     0.1    0    1.37 "
      "FH51        51     31.01     1.0    0    2.82 "
      "G21EA       25     33.62     1.0    0    2.37 "
      "G21IP       36    257.61     0.1    0    1.93 "
      "G2RC        25     51.26     1.0    0    1.42 "
      "HEAVYSB11   11     58.02     1.0    0    1.97 "
      "NBPRC       12     27.71     1.0    0    3.09 "
      "PA26        26    189.05     0.1    0    2.32 "
      "RC21        21     35.70     1.0    0    1.82 "
      "SIE4x4      16     33.72     1.0    0    0.486"
      "TAUT15      15      3.05    10.0    0    6.01 "
      "W4-11      140    306.91     0.1    0    1.12 "
      "YBDE18      18     49.28     1.0    0    2.26 "
      "BSR36       36     16.20     1.0    1    2.87 "
      "C60ISO       9     98.25     0.1    1    1.08 "
      "CDIE20      20      4.06    10.0    1    6.60 "
      "DARC        14     32.47     1.0    1    1.77 "
      "ISO34       34     14.57     1.0    1    5.04 "
      "ISOL24      24     21.92     1.0    1    2.02 "
      "MB16-43     43    468.39     0.1    1    0.299"
      "PArel       20      4.63    10.0    1    5.90 "
      "RSE43       43      7.60     1.0    1    4.45 "
      "BH76        76     18.61     1.0    2    1.56 "
      "BHDIV10     10     45.33     1.0    2    1.80 "
      "BHPERI      26     20.87     1.0    2    2.35 "
      "BHROT27     27      6.27    10.0    2   14.3  "
      "INV24       24     31.85     1.0    2    4.47 "
      "PX13        13     33.36     1.0    2    1.38 "
      "WCPT18      18     34.99     1.0    2    1.93 "
      "ADIM6        6      3.36    10.0    3   17.4  "
      "AHB21       21     22.49     1.0    3    8.15 "
      "CARBHB12    12      6.04    10.0    3    7.28 "
      "CHB6         6     26.79     1.0    3    4.18 "
      "HAL59       59      4.59    10.0    3    9.09 "
      "HEAVY28     28      1.24    10.0    3   14.6  "
      "IL16        16    109.04     0.1    3    9.95 "
      "PNICO23     23      4.27    10.0    3    9.22 "
      "RG18        18      0.58    10.0    3   26.0  "
      "S22         22      7.30    10.0    3   16.4  "
      "S66         66      5.47    10.0    3   19.8  "
      "WATER27     27     81.14     0.1    3    1.68 "
      "ACONF       15      1.83    10.0    4   44.0  "
      "Amino20x4   80      2.44    10.0    4   23.2  "
      "BUT14DIOL   64      2.80    10.0    4   25.8  "
      "ICONF       17      3.27    10.0    4   22.2  "
      "IDISP        6     14.22     1.0    4    2.55 "
      "MCONF       51      4.97    10.0    4   15.8  "
      "PCONF21     18      1.62    10.0    4    8.98 "
      "SCONF       17      4.60    10.0    4   15.8  "
      "UPU23       23      5.72    10.0    4   11.6  "
    )
else
    echo "ERROR: weight_Type Not Supported. Exiting..." >&2
    exit 1
fi