Random versus systematic errors in computed reaction enthalpies

Jimmy Kromann, Alexander Welford, Anders Christensen, and I have been testing the connectivity-based hierarchy (CBH) approach, developed by Raghavachari and co-workers, for semempirical methods. For most methods the improvement has been quite modest and here I discuss why (you'll need to read the paper to follow what I write here).

PM6
For example, for PM6 the RMSE relative to G4 for the 23 parent reactions is 9.4 kcal/mol, which increases to 16.4 kcal/mol for CBH-1 and decreases to 7.1 kcal/mol for CBH-2.  For comparison, the corresponding values for HF/6-311G(d,p) are 15.2, 11.4, and 3.0 kcal/mol, respectively.

To see what's going on, let's look at the CBH-1 correction for the first reaction in the set. The PM6 error is -0.3 kcal/mol for the parent reaction and the CBH-1 correction is -12.7 kcal/mol. PM6 is doing great for the parent reaction so, ideally, the CBH-1 correction should be 0.  Why is it so large in magnitude?

It is a Diels-Alder reaction, so two double bonds are turned into four single bonds and the CBH-1 reaction is (using SMILES notation) 4 C + 2 C=C -> 4 CC.  The error in the heats of formation (HOF) relative to G4 are C: -5.6, C=C -3.3, and CC: -4.1 kcal/mol. So the CBH-1 correction is -[4(-4.1) - 2(-3.3) - 4(-5.6)] = -12.6 kcal/mol.  So the CBH-1 correction is large because the error for C is somewhat larger than for CC and, especially, C=C.

These errors also shows that the 0 kcal/mol error for the parent reaction is clearly fortuitous. If the errors in the HOFs of the reactants and products are similar to those observed for C, C=C, and CC then one would expect an error in the reaction energy of about 3-6 kcal/mol since you go from 2 to 1 molecule.

HF/6-311G(d,p)
Let's compare the PM6 results to HF/6-311G(d,p). The error for the parent reaction is -12.2 kcal/mol, which decreases to -7.9 with the CBH-1 correction.  The error in the HOFs relative to G4 are C: -95.4, C=C -140.6, and CC: -166.8 kcal/mol. So the CBH-1 correction is -[4(-166.8) - 2(-140.6) - 4(-95.4)] = 4.4 kcal/mol, which when added to -12.2 lowers the error in reaction energy to -7.9 kcal/mol.  Clearly the errors in HOFs are much than larger for PM6, yet most of the error cancels.  Why?

The answer is that the magnitude of the HOF error scales with the number and types of bonds in the molecule.  For example, the HOF-error of C=CC is -213.2 kcal/mol, which is approximately the sum of the HOF-errors of CC and C=C, minus the error for C [-166.8 -140.6 + 95.4 = -212.0 kcal/mol].  This type of error scaling makes HF-results very amenable to improvement using the CBH approach.

Implications for optimisation of new semiempirical methods
Most NDDO-based semi-empirical methods are optimised by reducing the HOF-RMSE (or MUE). If the absolute errors for predictions are consistently below, say, 1 kcal/mol then the maximum possible reaction energy-error will be 2, 3, and 4 kcal/mol for a A -> B, A + B -> C, and A + B -> C + D type reactions respectively.

However, if the errors are larger (say 5 kcal/mol on average) then the best case scenario for this approach is one where the errors are as systematic as possible (i.e. that all are as close to 5 kcal/mol as possible).  Then the reaction energy-error will be around 0 for A -> B and A + B -> C + D reactions and typically around 5 kcal/mol for A + B -> C reaction (which one could correct for since it is consistent).

As shown by the HF example, another approach could be to minimise the the error in a "bond-corrected HOF" for larger molecules, e.g. minimise the errors in HOF(C), HOF(C=C), HOF(CC), and [HOF(C=CC) - HOF(C=C) - HOF(CC) + HOF(C)] and then present a CBH-1-corrected HOF for C=CC to the user.

This approach might make it easier to find more generally applicable parameters that better and systematically minimise the error, since the underlying HF approach "naturally" gives larger errors for larger systems and we are no longer asking the parameters to "undo that" by searching for small errors independent of system size.

Can we tell from the fragment-HOF errors if CBH-corrections will improve results?
No, but perhaps we can identify cases where the correction makes it much, much worse.  For example, the fifth reaction in the set is also a Diels-Alder reaction, so the CBH-1 correction is the same as for reaction 1, but the PM6 error for the parent reaction is 9.1 kcal/mol, so the CBH-1 correction lowers the error to -3.2 kcal/mol.

On the other hand, the PM6 CBH-2 correction for Reaction 23 is -44.2 kcal/mol, which is unusually large (and makes things much worse).  The CBH-2 reaction 2 C#C + 3 CC + 5 C=C=C -> 6 C=C + 5 C#CC and the large error comes from the fact that the HOF error is 7.6 kcal/mol, much larger and of opposite sign than the other HOF-errors in the reaction, plus the fact that it is multiplied by 5.

However, it is not enough to simply look for outliers. The largest PM6 HOF error among the fragment we have studied so far is 11.7 kcal/mol for NN, and this enters in to the CBH-2 correction for R10, but this correction actually lowers the error relative to the parent reaction. 

Perhaps the best one can do is to view corrections that are significantly larger than 20 kcal/mol with suspicion since such large errors are rarely observed for the parent reactions.


This work is licensed under a Creative Commons Attribution 4.0 International License.

Atom_mapper: determining atom correspondence between reactants and products

Many TS search algorithms interpolate between reactants and products and require that the atom order is identical. This is time consuming to enforce by hand and hard to automate, but this paper by the folks at Schrödinger describe a solution that seems to work pretty well. The general idea is also fairly easy to implement if you are familiar with RDKit or similar cheminformatics tools and here I describe my implementation so far.

The basic idea
If two molecules are identical but have different atom numbering then RDKit can map the atoms using "GetSubstructMatch". So (pseudocode) "OCC.GetSubstructMatch(CCO)" gives "(2,1,0)" meaning that atom 2 in CCO corresponds to atom 0 in OCC and so forth.  And "RenumberAtoms(CCO, (2,1,0))" gives OCC.

Let's say our reaction is CH3-CH=O -> CH2=CH-OH.  "product.GetSubstructMatch(reactant)" will not give a match because the molecules are different. To focus on connectivity we remove bond orders: CH3-CH-O -> CH2-CH-OH, but the molecules are still different. The next step is to break each bond in turn and compare. For the reactant you get [H + CH2-CH-O, CH3 + CH-OH, H + CH3-C-O, O + CH3-CH2] and the first entry will match the last entry in the corresponding list for the product [H + CH-CH-OH, CH2 + CH-OH, H + CH2-C-OH, H + CH2-CH-O].

It is very easy to check to for equivalence by converting the list of fragmented molecules to a set of corresponding SMILES strings and looking for identical elements in the two sets: "matches = list(set(react_frag_smiles) & set(prod_frag_smiles))". If there is a match then the atoms can be matched as before because they are identical (note that the bond breaking does not alter the atom numbering of the molecule). If no match is found then the procedure is repeated for the fragments.

One problem is that equivalent atoms are not matched in 3D. For example, if you label (i.e. number) the equivalent H atoms in a methyl group then the methyl group is chiral and RDKit will create a random chirality.  So, the Cartesian coordinated of methyl hydrogens in the CCO example above are not necessarily superimposable, which creates problems in TS search algorithms that use interpolation.

Following the ideas presented in the Schrödinger paper, I try to solve this issue by switching pairs of equivalent atoms and comparing the RMSDs between reactant and product before and after.  However, this will only work if the reactants and products have the same conformation, so I use a modified version of ConstrainedEmbed (ConstrainedEmbedRP) which uses either Cartesian or distance constraints to match two structures using a UFF force field minimization of one of the structures.

Using ConstrainedEmbed has the added advantage of producing reactant and product structures that are in the same orientation and, therefore, often good starting points for interpolation.

Atom_mapper
* I have implemented these ideas in Atom_mapper and made it available on GitHub under the MIT license.
Usage  
* The input is SMILES strings of reactant and products but the code can be adapted to work with user-supplied coordinates via sdf files.
* The output is two sdf files labelled "template" and "mol" with (hopefully) identical atom ordering. Template is the most rigid of the two and the coordinates come from an unconstrained UFF minimization. Mol is superimposed on template using ConstrainedEmbedRP.
* Both geometries should probably be minimized using the energy function you plan to use for the TS search and, depending on the optimization algorithm, the structures should probably be rigidly superimposed before interpolation
Current limitations
* I haven't tested the code extensively. If you run into cases where it doesn't work, file a bug report on GitHub
* No conformation search is done so the conformations chosen will be random. Solution: start with coordinates rather than SMILES.
* If reactants or products consists of two molecules their relative orientation resulting from the UFF minimization will be somewhat arbitrary. This is not a problem if only the reactants or products consists of two molecules, since the coordinates will be superimposed on a more rigid molecule. However, it is a problem if the reactants and products both consist of two molecules. Solution: start with coordinates rather than SMILES, but not sure how to determine best intermolecular orientation automatically.
* The code only permutes pairs of equivalent atoms so it won't work for equivalent groups, e.g. methyl groups in CC(C)(C)CO because we need to switch several pairs (corresponding to C and 3 H atoms) simultaneously.  Not sure how to fix that easily. Suggestions welcome.

There are also some general limitations to this "active bonds" approach as described in the article.


This work is licensed under a Creative Commons Attribution 4.0