« Teaching 2D NMR? | Main | How Accurate should NMR Predictions be? »

May 23, 2007

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d834fe3a5753ef00d83546ad1d53ef

Listed below are links to weblogs that reference NMRShiftDB, ACD/Labs, and Modgraph:

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

cesar

is good

Antony Williams

I'll add that the availability of the NMRShiftDB to perform this type of analysis is a true public service in terms of having a diverse database of this size to analyze. I've blogged elsewhere (http://www.chemspider.com/blog/?p=13) that, despite the fact that it is not perfect, NO database of this size is free of errors. The literature is loaded with poor assignments and, without care, they carry into the database. This is a high quality test set! So, thanks to the NMRShiftDB team. At least two groups have been able to perform an analysis of performance of NMR prediction algorithms. Are there others out there???

Ryan Sasaki

Tony makes an excellent point. The NMRShiftDB team should be absolutely acknowledged here (and they are within the document)

Bottom line, commercial products can conduct comparisons for databases that they create themselves...but it is difficult to make them free from bias. As mentioned, the NMRShiftDB is populated by scientists and spectroscopists worldwide. This is good data.

The NMRShiftDB is a terrific resource for the validation and comparison of different NMR predictors. I encourage others to experiment and I'll be happy to share the results on here (provided it is done right and overlap is acknowledged). I think the document provided in the entry above is a great way to evaluate the performance of different NMR predictors...and it's easy to do.

Jeff Seymour

I am Jeff Seymour, Marketing Manager of Modgraph Consultants and I am responding to the articles on your BLOG which started on May 23rd http://www.acdlabs.typepad.com/my_weblog/2007/05/nmrshiftdb_acdl.html comparing ACD/CNMR Predictor to NMRPredict.

In the PDF document which is attached to the May 23rd BLOG you state "we have (had) an opportunity to compare performance with another commercial product, NMRPredict provided by Modgraph Consultants, Ltd" and conclude that ACD "significantly outperforms the algorithms of Robien".

In your BLOG of May 30th http://acdlabs.typepad.com/my_weblog/2007/05/update_robien_o.html you state “ACD/CNMR Predictor vastly outperforms the prediction accuracy of the CSEARCH algorithm. The average deviation in ACD/CNMR predictor was 1.59 ppm compared to an average deviation of 2.22 ppm in CSEARCH.”

The problem is that your were looking at prediction values from Wolfgang Robien’s web page from March 12th http://nmrpredict.orc.univie.ac.at/csearchlite/enjoy_its_free.html and you have assumed that Wolfgang was using NMRPredict to generate his results. WOLFGANG WAS NOT USING NMRPREDICT IN HIS ARTICLE – as he clearly stated.

Wolfgang’s CSEARCH program can (as with ACD) use both HOSE code databases and Neural Network technology for its predictions. Wolfgang has over 750,000 data at his disposal. The most accurate predictions will always come from using a combination of well verified HOSE code databases and Neural Network technology.

In his original paper Wolfgang only used a Neural Network from 1996, there were no HOSE code databases used. This is not surprising. Wolfgang’s intention in his article was clearly to demonstrate how in a few hours he could find glaring errors in the NMRShiftDB database. His intention was not, as you seem to have assumed, to show how accurate his predictions could be.

Wolfgang’s CSEARCH program is indeed the basis of NMRPredict. However, together with Wolfgang, we have added significant enhancements, such as 'auto-stereo recognition', different utilization of solvent-dependent predictions and 'BEST selection’, which are not available in CSEARCH.

NMRPredict uses a database of over 345,000 records, a Neural Network and also includes a “BEST–selection” routine to choose which of the HOSE code or Neural network values to use for each carbon atom.

We have now re-run the NMRShiftDB database using the NMRPredict program and have come up with an average deviation of 1.40 ppm compared to the 1.59 ppm in ACD CNMR/Predictor 10.5. Details can be found at http://www.modgraph.co.uk/product_nmr.htm, http://www.modgraph.co.uk/product_nmr_shiftdb.htm and http://nmrpredict.orc.univie.ac.at/csearchlite/Robien2Ryan_May31_2007.html

In conclusion, using a set of data which you described as being “of size and quality to serve as a fair and reliable validation set to evaluate the performance of ACD/CNMR Predictor in terms of accuracy of NMR prediction” the outcome is that NMRPredict vastly outperforms the prediction accuracy of ACD/CNMR Predictor rather than the other way round. The average deviation in NMRPredict was 1.40 ppm compared to an average deviation of 1.59 ppm in ACD/CNMR Predictor version 10.5, already compensating for your somewhat smaller structural overlap.

Ryan Sasaki

Thanks for your comments, Jeff.

I believe I have responded to the points in your comments in my latest entry here:

http://acdlabs.typepad.com/my_weblog/2007/06/robiens_and_mod.html

The comments to this entry are closed.