Ryan's Blog on NMR Software

False Negatives and False Positives are Waiting...

Great post from Derek Lowe from In the Pipeline the other day talking about the dangers of not quality checking those fine-looking starting compounds for your project. Chemistry happens and yes, mistakes do too.

In fact, it appears that Derek has been on a kick as of late referring to personal QC.

I Can Has Ugly Molecules?

Oops.

I thought this would once again be a good opportunity to provide you with a link to a poster Sergey Golotvin presented at ENC 2008 entitled, "Validating the Quality of Large Collections of NMR Spectra Automatically".

Long story short, 15,000 1H NMR Spectra from the Aldrich collection were evaluated in complete automation and the software was able to confirm 88% of the collection as having chemical structures that were consistent with the respective spectra. In addition, 4% were flagged by the software as being inconsistent. A closer, manual look at those 5% revealed that there were indeed some truly wrong structures (or incorrect tautomers) in the collection.

This was evaluating the 1H NMR data only. Using additional 2D experiments, such as HSQC, will likely improve these results.

Just an example of a check an organization can build into their process for additional QC of their registration database for example.

Is it perfect? Absolutely not. There are perhaps a few more false positives in there that the software didn't catch, and of course the software provided some false negatives as well, annoying because presumable someone has to look over them manually only to realize that they were indeed the right structure all along. But at least this doesn't involve manually pining over 15,000 spectra!

We continue to run these datasets, and actually have a consortium consisting of several NMR experts in the industry we call ASCI (Automated Structure Confirmation Initiative) where we are testing and validating this technology in the real pharmaceutical world. Identifying the common areas where false negatives and false positives occur and trying to address them with algorithms.

Will we ever solve all the problems, especially in the world of novel chemistry? Of course not, and for that matter there are some existing problems that appear to be too hard to solve.

But that being said, what is the acceptable limit of false positives and false negatives for automated verification by software for the verification of  registered compounds in a library?

Interested in hearing your thoughts.

Looking for a Great Weekend Read?

In fact, to borrow a phrase from a colleague, this might be the defacto article on Computer Assisted Structure Elucidation (CASE) for the next decade!

This article written by Mikhail Elyashberg, Antony Williams, and Gary Martin spans across two issues of the review journal, Progress in Nuclear Magnetic Resonance Spectroscopy. 

This article entitled, entitled, "Computer-Assisted Structure Verification and Elucidation Tools in NMR-based Structure Elucidation" is available online and you can review a preview of the content at:

http://dx.doi.org/10.1016/j.pnmrs.2007.04.003

This is a very important and comprehensive review of modern expert CASE systems over many years. It includes specific examples of complex natural product structures that have been automatically elucidated using such systems.

I thank the authors of this publication for their contributions in this area, and the efforts they have now put forth to communicate this story to the scientific community.

Please obtain a copy for yourself, I can promise that it is a very informational and intriguing read for those of you who do NMR regularly.

New Blog by Arvin Moser

Those of you who are current users may recognize the name Arvin Moser as he has spent many years as both a Technical Support Specialist and Application Scientist at ACD/Labs.

Arvin has decided to start a blog and share his knowledge and experience about structure elucidation. I think this blog promises to be a very interesting one as Arvin has a wealth of experience in both manual and computer assisted structure elucidation (CASE).

From Arvin's About Page:

My goal is to focus on the science of data interpretation and structure elucidation. I would like to pass on my experiences including what I have learnt from the experts. By sharing these experiences with the scientific community, I think an emerging elucidator can be better equipped with handling anything that comes their way.

Visit Arvin's Blog here.

More on Indirect Covariance

Gary Martin has added a comment to an earlier post I wrote on Indirect Covariance and was kind enough to post an updated publication list on this topic. I have simply copy and pasted his comments in this post to provide more exposure to his list:

Ryan, it has been a while since you updated the publication list on indirect covariance methods, so I thought it appropriate to do that with a post:

F. Zhang and R. Bruschweiler, J. Am. Chem. Soc., 126, 13180 (2004).

K. A. Blinov, N. I. Larin, M. P. Kvasha, A. Moser, A. J. Williams, and G. E. Martin, Magn. Reson. Chem., 43, 999 (2005).

K. A. Blinov, N. I. Larin, A. J. Williams, M. Zell, and G. E. Martin, Magn. Reson. Chem., 44, 107 (2006).

K. A. Blinov, N. I. Larin, A. J. Williams, K. A. Mills, and G. E. Martin, J. Heterocyclic Chem., 43, 163 (2006).

K. A. Blinov, A. J. Williams, B. D. Hilton, P. A. Irish, and G. E. Martin, Magn. Reson. Chem., 45 544 (2007).

W. Schoefberger, V. Smrečki, D. Vikić-Topić, and N. Müller, Magn. Reson. Chem., 45, 583 (2007).

G.E. Martin, P. A. Irish, B. D. Hilton, K. A. Blinov, and A. J. Williams, Magn. Reson. Chem., 45, 624 (2007).

G.E. Martin, B.D. Hilton, P.A. Irish, K.A. Blinov, and A.J. Williams, J. Heterocyclic Chem., 44, 1219 (2007).

G.E. Martin, B.D. Hilton, P.A. Irish, K.A. Blinov, and A.J. Williams, J. Nat. Prod., 70, 1393 (2007).

G.E. Martin, P. A. Irish, B. D. Hilton, K. A. Blinov, and A. J. Williams, Magn. Reson. Chem., 45, 883 (2007).

B. Hu, J.-P. Amourex, and J. Trebose, Solid State Nuclear Magnetic Resonance, 31, 163 (2007).

G.E. Martin, B.D. Hilton, P.A. Irish, K.A. Blinov, and A.J. Williams, J. Nat. Prod., 70, 1966 (2007).

D. A. Snyder, Y. Xu, D. Yang, and R. Bruschweiler, J. Am. Chem. Soc., 129, 14126 (2007).

G.E. Martin, B. D. Hilton, K. A. Blinov, and A. J. Williams, Magn. Reson. Chem., 46, 138 (2008).

G.E. Martin, B.D. Hilton, K. A. Blinov, and A. J. Williams, J. Heterocyclic Chem., 45, in press (2008).

This is the list I currently have. Note that this list does not contain additional papers that pertain to homonuclear covariance processing methods. I don't have that list of publications handy that I can cut and paste into this post.

Thanks Gary!

Fringe Benefits and Knowledge Management

Last week I blogged about Phil Keyes' and Anthony Macherone's applications of NMR software towards automated structure confirmation.

A few months back, I pointed you to Steve Coombes' workflow when working with ACD/Structure Elucidator.

Phil had a very nice section in his presentation about the "fringe benefits" he was able to derive outside of the main goal of the project, "Automated Structure Verification".

Specifically, Phil pointed to a couple of fringe benefits:

1) A spectral database is grown as a result of the automated structure confirmation. This database is heavily searchable and can be used as a resource within the company. Building the database is part of the workflow. No extra work needs to be done.

2) The software provides an assignment starting point. In running the verification algorithm, the software automatically attempts to assign multiplets in the 1D and 2D spectra, provides feedback of the quality of those assignments, along with the ability to easily edit them:

Anthony Macherone also mentioned automatically storing data in a searchable database as an additional benefit to conducting automated structure confirmation in his presentation.

On a different application, Steve Coombes spoke a lot about the additional benefits he receives out of ACD/Structure Elucidator.

In this presentation Steve really stresses the knowledge management angle from Structure Elucidator. Sure, the software can help elucidate the chemical structure of unknowns, but it also supports the ability to store the knowledge you gain from working on your data.

In Steve's opinion this is what separates ACD/Labs software from many other packages out there. The "ability to extract the information and knowledge for further use"

It's not just the ability to build databases with structures and spectra. The key is the ability to assign that data electronically and store it in a searchable database. That's knowledge.

And of course by retaining that knowledge through electronic assignments, you can share that knowledge with the software by training the predictions and improving elucidation and verification performance. 

I'd like to thanks these guys for teaching me a nice "marketing" lesson. It's not always about the main application of the software. Always be on the lookout for "fringe benefits"

Meet Quindolinocryptotackiene

Tony over at ChemSpider takes us on a trip down memory lane to one of the most successful stories surrounding Computer Assisted Structure Elucidation (CASE).

It is also the best example of achieving symbiosis between a spectroscopist (in this case Gary Martin) and software (ACD/Structure Elucidator) I have ever seen.

He is referring to "Solving a structure computationally after 10 years of human effort" that was presented by Gary and Tony at the ASP Meeting in 2003 (It's a long presentation but skip to slide 44 to get to the meat of the presentation).

There is also a publication on this story. 

Tony's purpose for resurrecting this story is as follows:

Now, we THINK we have it elucidated correctly. However, we would like to confirm it. Synthesis of the molecule in question, further NMR data generation and a crystal structure would help finish this work fully. This is a call to organic chemists to participate in a hobby project. Anybody want to help? We guarantee a publication etc. The structure is shown below. Contact me at antonyDOTwilliamsATChemspiderDOTcom. Thanks!

Hopefully someone is willing to step up to the plate.

I'd also like to take this opportunity to, once again, point out that CASE is not simply about piling a bunch of data in a piece of software and getting the answer out the other end. Sure this is possible, but it usually benefits when an experienced spectroscopist works with it and shares their knowledge of the existing chemistry. I think Gary's story is a perfect example of that.

That being said, in Gary's case, along with comments I have received from Dr. Shaun Tennant (another elucidator user) in the past, the software is an unbiased approach that will propose some things that the spectroscopist simply might not think about. Knowledge can sometimes be your enemy.

Applications of Automated Structure Verification with NMR Software- Part 2

Yesterday I blogged about how Phil Keyes has applied automated structure verification at Lexicon Pharmaceuticals to help validate compound registrations in an open access environment.

Links to the latest performance statistics of our automated structure verification solution for both 1D ¹H and combined 1D ¹H and 2D HSQC structure verification can be found in the previous post.

As promised, today I will highlight the application of automated structure verification that Anthony Macherone has employed at ASDI.

Anthony works in a high-throughput environment where more than 1000 compounds are directed to 1D ¹H NMR analysis per week. Based on this workload, he has implemented a very nice workflow in his laboratory. In his presentation, Anthony mentioned that it in his line of work, the ultimate goals are to:

1. Maximize instrument efficiency
2. Maximize throughput
3. Be cost effective

Sounds like some pretty good goals to me. How Anthony is able to achieve this is of course the really interesting part.

Anthony describes his workflow in three phases, the pre-game, middle-game, and end-game. In the pre-game he uses proprietary software (not ACD/Labs) to screen the compounds and "bin" them into appropriate analytical techniques. In doing so he does not have to run a full battery of analytical data on every compound that is screened. In the middle-game, he automates the sample preparation and acquisition using well-plates and the help of robots.

The end-game is where Anthony employs ACD/Labs software. Once the data is acquired, he applies a custom macro to automatically:

1. Attach chemical structures to appropriate FID files
2. Process the data (FT, phasing, baseline correction, and integration)
3. Run the ACD/Labs automated structure verification algorithm (Provide a red light/green light data assessment)
4. Store the data in a searchable database

Following the data acquisition and analysis, Anthony only needs to manually evaluate the ambiguous or questionable results (i.e. red light data)

Make sure to check out Anthony's presentation for more details regarding the advantages of these phases, time-savings, accuracy, etc.:

Anthony Macherone- High-Throughput NMR Analysis: The End Game

Again, I would like to thank both Phil Keyes and Anthony Macherone for sharing their applications at our New Jersey User Meeting last week.

Applications of Automated Structure Verification with NMR Software- Part 1

Several posts back I pointed you to a couple of articles ACD/Labs were involved in with regards to automated structure verification.

I have pointed to these articles, but I have spent little time talking about it. I will now.

For those new to this idea, it involves using software to automatically confirm the consistency between a chemical structure and an NMR spectrum using NMR prediction. Lee Griffiths from AstraZeneca has done excellent work over the years in this field. Lee was kind enough to present at our European User's Meeting last year to share a summary of his approach towards automated structure using 1D ¹H and ¹³C, and 2D HSQC data.  This presentation can be downloaded here.

In addition, by doing a simple search for "Griffiths" on the Magentic Resonance in Chemistry webpage, you'll find a whole bunch of relevant articles.

We initially published a validation on the performance of automated structure verification using just 1D ¹H NMR data. We then proceeded to publish again recently to compare that to the performance of a combined verification approach using 1D ¹H and 2D HSQC data.

As a result of these and other studies, much of the focus of late by ACD/Labs has been on the performance of automated structure verification using 1D ¹H and 2D HSQC NMR data.

These publications along with posters we presented at SMASH and ENC on this topic should give you a general idea about the performance and accuracy of this approach.

I am not going to discuss the performance of this approach today but rather focus on the real-world applications and performance in an industrial setting.

Last Thursday I was in New Brunswick, New Jersey at our New Jersey User's Meeting where I was blown away by two terrific presentations by our guest speakers, Phil Keyes from Lexicon Pharmaceuticals and Anthony Macherone from ASDI.

Two different applications in two different environments. I'll talk about Phil's today, and Anthony's tomorrow.  Phil's is interesting as he is setting up a really cool system to significantly improve how analytical data is handled in an open access environment, and further to validate Lexicon's compound registration database.

In my opinion, the real crucial thing to point out here is the evolution of an open access environment from a more traditional analytical services setup. It used to be that NMR Spectroscopists would run and handle all the analytical data for compounds that a chemist produced, verify their structures for them, and give them the thumbs up or thumbs down. In this environment, spectroscopists were getting a look at the data from all compounds entering the registration database. In an open access environment this is no longer the case. While NMR spectroscopist certainly see lots of this data still, and they will likely eventually see a compounds data during it's pharmaceutical R&D life cycle, the reality is that there are still going to be some incorrectly or questionably verified structures in a company's registration database that will go on for further testing. Towards the evolution of open access NMR, somewhere along the way, it became OK for compounds to get registered without being approved by an analytical expert. Of course, these aren't being registered blindly, chemists are approving these and in most cases they are more than qualified to do so and are doing a good job. However, I have yet to talk to a NMR spectroscopist who has NOT seen compounds registered incorrectly.

My point is of course to not pick on chemists here. Sometimes these mistakes are unavoidable and the data LOOKS right. Sometimes there is nothing in the ¹H NMR spectrum or the LC-MS that suggests that there is anything different present. The key is to better identify when these instances arise in the registration database. Can an automated structure verification solution with NMR software replace and outperform the QC of a chemist for good in an open access environment? No, not right now anyway.

However, the key statement is in Phil's presentation:

"Integrating a system to perform automated compound verification provides value by highlighting compounds for which structural data is complex and subject to interpretation."

Sure there are going to be false positives and false negatives with an automated approach. The question is, if 50 out of 1000 compounds being registered by chemists are incorrect, is there value in automated software highlighting 40 of them?

False negatives can be annoying because it involves the spectroscopist to do unnecessary work on a sample that was correct all along. But other times it might point out the need to run more experiments to prove that it is indeed the right structure. Ideally ALL of the data gets manually evaluated, but in the age of open access NMR where chemists are outnumbering spectroscopists 100:1 in some organizations this is clearly no longer plausible. But is there a balance here? While it isn't plausible to manually evaluate the data for say 1000 compounds, would it be feasible to manually evaluate the 300 of the 1000 samples that software has highlighted as complex or subject to interpretation?

Phil's and Anthony's presentations will be available on the ACD/Labs website shortly, but for my readers, you get advanced access to these presentations.

Phil Keyes- Validating Compound Registrations with Automated NMR Verification in Open Access

For those who want to do advanced reading on the topic for tomorrow's blog entry:

Anthony Macherone- High-Throughput NMR Analysis: The End Game

Another NMR Blog

I'd like to point out another NMR blogger to all my readers out there.

Glenn Facey, Facility Manager from the University of Ottawa has created a blog specifically for his NMR users. While he is doing this to provide a resource to the University's students, I think there are some nice tips and tricks in there specifically for NMR data acquisition and processing. There are a couple of irrelevant housekeeping posts for those not attending the university, but other than that it is a very useful resource for beginner NMR users and students at other academic institutions.

Check it out here:

http://www.u-of-o-nmr-facility.blogspot.com/

I should point out that Glenn's work is an EXCELLENT application of blogging. I think that all instrument facility managers at academic institutions should have a blog. While we are at it, the same can be said for industry (blogs can be internal as well). It's a great place to talk about instrument maintenance and downtime (avoid the 2-3 emails a week you get from the facility manager) but more importantly to offer the NMR users tips and tricks over time. Students generally only get one in-depth training session with their NMR spectroscopist a blog offers the ability to provide students with a running commentary from the NMR expert. Contrary to popular belief, students probably aren't going to read the instruction manual and while spectroscopists put some work in creating a cheat sheet for them, these generally get lost in the bottomless pile of data that graduate students are collecting. You leave one in the instrument room as a resource? Go check, I bet it isn't there anymore :).

Here's another NMR Facility Blog run by Tim Burrows at the University of Toronto:

http://www.chem.utoronto.ca/facilities/nmr/NMRBlog/

There's no reason NOT to do this. It's dead easy. If you can email, you can blog.

Glenn has provided a nice standard to build upon:

P.S. Glenn actually taught our very own application scientist, Arvin Moser, everything he knows ;)

If you are a facility manager for an academic institution and you blog. Please let me know, I will be sure to mention you on here at some point. I think you can provide a great resource for not only your students, but the entire academic community!

2DNMR.com

Remaining on the topic of Computer-Assisted Structure Elucidation (CASE), one of our most experienced users, Shaun Tennant, has devoted a website to Structure Elucidator where he shares some of his experiences with the software and provides a guide for users including his own tips and tricks to get started.

A must read for any elucidator users out there or anyone interested in how the software works.

Check out his site at www.2DNMR.com