Ryan's Blog on NMR Software

Time Flies...1 Year Anniversary of Ryan's Blog on NMR Software

I can't believe it has been one full year since my very first blog post. 73 posts 90 comments, >500 subscribers later...here we are.

I hope that over the past year I have accomplished what I set out to do: to provide an updateable resource for people interested in NMR software, more specifically ACD/Labs software.

Instead of attempting to write a meaningful post that would appear fitting, I thought I would reflect back on my favorite 5 (OK, I couldn't resist I had to go with 7!) posts of my rookie year as a blogger:

Here they are, IMO:

7. The Truths and Myths of ACD/Structure Elucidator
6. The Price of NMR Software
5. The Curse of Knowledge
4. Where is the Quality? Assumption vs. Assurance?
3. Fringe Benefits and Knowledge Management
2. NMRShiftDB, ACD/Labs, and Modgraph (This prompted quite a lot of dialog around the web, there's a lot of reading here)
1. The Purgatory Database

A big thank you to all those who have subscribed to my blog and those who have offered comments, emails, etc. that have inspired me to continue on with this endeavor. A blog is only as good as it's followers.

A big thanks to Antony Williams, Gary Martin, Phil Keyes, Rich Apodaca, Wolfgang Robien, Malcolm Beckett, Jeff Seymour, Jean Claude Bradley, Egon Willighagen,  Mark Mowery, Christoph Steinbeck, and Stefan Kuhn for their inspiration, insights, opinions, comments, and overall spreading of the word through the blogosphere and beyond.

I would like to thank Daria Thorp, Antony Williams, Brent Lefebvre, Mark Bayliss, Arvin Moser, Andrew Anderson, and Patrick Wheeler for their internal support.

And of course our NMR Development Project Leaders Sergey Golotovin, Kirill Blinov, and Mikhail Kvasha without of course this blog would not exist.

Here's to another year of NMR Software blogging!

Automated Structure Verification by 1H NMR Only

I've blogged several times about the progress and applications of automated structure verification with the help of ACD/Labs software.

There are really two main approaches right now:

1. Combined verification which includes automatically verifying the correspondence between a proposed chemical structure and the 1D ¹H and 2D HSQC spectra. I've blogged about the publications and applications of this method previously.
2. ¹H only verification. Of course the first approach is preferable from an accuracy standpoint as the additional information gained from the HSQC spectrum increases the selectivity and specificity of the results.

However, I always get questions on how well we perform on ¹H NMR only because in some organizations and environments it is simply not feasible to always run an HSQC in tandem with a routine ¹H NMR analysis.

As mentioned in previous posts, we've already conducted a validation study on this approach and it was published in a 2006 article in MRC. We've continued to investigate and validate this approach and we recently presented our latest results using 1D ¹H NMR data only at ENC 2008.

The poster highlighted a study on the automatic evaluation of over 15,000 Aldrich compounds and spectra from Aldrich NMR Spectra Database.

The results of this study revealed that the software was able to confirm 88%of all spectra as consistent and flagged less than 5% as inconsistent.

One of the more interesting discoveries in this study was that it revealed some truly wrong structures in the Aldrich NMR database.

More information on these as well as shortcomings in prediction, processing, and analysis are provided in the poster that can be downloaded:

http://www.acdlabs.com/download/publ/2008/enc08_aldrich.pdf

Do You Run 15N, 19F, and 31P Experiments?

As I am closely approaching my one year anniversary of this blog (how time flies!), Arvin's post entitled, "How Do I Know if my Unknown Contains a Fluorine Atom",  reminded me an X-nuclei related post from almost a year ago.

In that post I highlighted a best practice document written by Gary Martin for acquiring ¹⁵N-¹H heteronuclear shift correlation data where he highlights an interesting application for ¹⁵N NMR prediction.

If you think that only benefits of a ¹⁵N NMR Predictor is for structure verification or validation, Gary is providing you with some additional tips and suggestions.

Check out his document here.

I want to personally thank Gary for his incredible contributions to the NMR community as well as for his guidance, collaboration, and contributions to the ongoing development and growth of ACD/Labs software.

Love/Hate Continued...

To follow my post from yesterday, Rich made a very interesting comment that I was hoping to address in today's post.

He suggests that perhaps a NMR Lite would be a good approach. I gave reason as to why we didn't take that approach and strip away existing features from 1D NMR Processor when developing the processing component in ACD/1D NMR Assistant yesterday.

But Rich makes a very good point in his comments:

For everyone else, it's all too easy to not even look at a piece of software who's fundamental purpose is obscured by layer upon layer of expert features. They just tune out and the only ones who end up using the software and giving feedback are the power users - which unfortunately reinforces the misconception.

For me, the key is to make sure that all these features don't get in the way of the primary reason someone is using the software. I don't have a problem with a wealth of features as long as it does not interfere with the main workflow.

Cathy Sierra is a blogger that I dearly miss for her daily insight on how to "create passionate users". Here's one of her takes. Specifically I like:

One of the themes I heard over and over at ETech and SXSW (Jason Fried, Craig Newmark, and others) was the developer mantra of "get out of the way." In other words, build the thing so that it stays the hell out of the way and lets the user get on with what they really want to do.

 

So as an adaptation of Rich's comment. Make the 20% REALLY clear, and hide the other 80% but still offer it.

I'm not sure it's the perfect solution, but I think it's a different approach.

For example, here's the first thing a user will see when they open a raw data file in 1D NMR Processor:

On the other hand, here is ACD/1D NMR Assistant:

Of course you don't want to create an environment where a user is drilling through menus looking for useful features, but we do provide users with the ability to hide or show toolbar buttons and action buttons on the interface so they can choose the interface most appropriate to them which can be altered as they get more comfortable with the software.

What do you think Rich?

Anyone else have an opinion they want to share in the comments section?

Avoiding the Love/Hate Relationship in Software

I had a conversation with Geoff, one of my ACD/Labs colleagues just yesterday.

He provided me with a great quote from a person he was talking to about software and usability.

He said:

"User-Friendly...hrmph...what that means to me is: love it for the first week, hate it forever!"

I think that's a great quote.

I think usability is incredibly important and should take high priority in the development of a software package.

However, while it is important we can't forget the term USE in "Ease of Use"

I find a lot of software packages out there that are very pretty, very intuitive, and very easy to use. They make a great first impression. You can use it for few days and fall in love. However, often times what happens is that after a week you decide you want to do more sophisticated things with the software but you can't. All of a sudden that love, turns to frustration, and sometimes ends up in hate.

I think this is one of the more difficult aspects of software development. I think most Product Managers and developers want their products to be easy to use. But you can't go overboard and oversimplify.

We went through the simplification process with the development of ACD/1D NMR Assistant. In the very early stages, we thought it might be a good idea to create a 1D NMR Processor Lite for chemists. We knew that 1D NMR Assistant would have the same NMR processing component as ACD/1D NMR Processor. But we also came to the conclusion that the interface and workflow in a duplicated form would not be appropriate for a new market, after all ACD/1D NMR Processor was developed for years with the NMR Spectroscopist in mind as they were the ones using it, providing feedback, and NFRs (New Feature Requests).

So while ACD/1D NMR Processor contains many features we don't anticipate most chemists will make use of (a quantitation tool, macro capabilities, group macro capabilities, intelligent bucketing, multiple baseline and phase correction algorithms, arithmetic, peak fitting, etc.) we decided to leave EVERYTHING in there. Perhaps 1 in 50 will become frustrated when their software can't execute a particular function for them, we're hoping we've avoided this issue by including all the features available in 1D NMR Processor within the Assistant package.

Here's hoping that 1D NMR Assistant is a software that you love the first week and love forever.

At least until the next version ;)

If you haven't tried it yet, download a 30 day trial here.

Linking to Meaningful Data in an ELN World

In a previous post, I asked the question, why does paper spectra continue to persist in chemistry?

Of course there is the next challenge, as Rich Apodaca points out on his Depth-First Blog in an earlier post:

The previous article in this series, suggested that the same dynamic applied to the compilation, management, and sharing of spectral data by chemists. More to the point:   

... cheminformatics has failed to deliver an inexpensive, robust, and truly usable solution to the problem of compiling, managing, and sharing spectral data for scientists of average computer skills. ...

To be sure, there are tools that address parts of the problem. But no solution addresses them all and that's why scientists and publishers resort to using obviously inferior solutions like PDFs.

Whether or not organizations and groups are resorting to inferior solutions is up for debate because it of course depends on the expectations of the end user. But his comments definitely struck a chord with me.

So the next question is:

"What is the best way to connect my analytical data to my ELN records TODAY?"

By far, the most common way that I have seen organizations connect the analytical data from our software to ELNs is via PDF.

But as Rich mentions in yet another post, for people who are looking to build on experiments or model or compile the results, static PDF images are practically useless.

I couldn't agree more.

So why do organizations choose this route?

The three biggest reasons I have heard are:

1. File size limitations in the ELN
2. The lack of a standard and supported analytical data format that is generic, open, lockable, and widely supported for years to come.
3. Currently,  PDF is more controlled for legacy support than analytical data.

As a result, PDF is the only reasonable approach for many, and it is certainly better than not connecting to a record of the data at all. 

I think the key is for vendors to work horizontally and to combine their strengths to deliver as Rich suggests a:

an inexpensive, robust, and truly usable solution to the problem of compiling, managing, and sharing spectral data for scientists of average computer skills.

But the file format remains an issue.

Work by the ASTM E13.15 Commitee has been ongoing for the past 5-6 years towards a universal analytical data file format. This file format is called AnIML (Analytical Information Markup Language), the developing XML standard for analytical chemistry data. Most vendors support the general directions of the ASTM E13.15 for a universal data format for analytical data.

A final note on the role of MEANINGFUL data in an electronic world. When I refer to meaningful data, I am referring to knowledge gained and stored in an actual data file as opposed to a static PDF. One of the unique features that ACD/Labs has maintained over the years is the ability to electronically assign NMR data to chemical structures to truly capture not only the data but the knowledge gained from the experiment. I think not leveraging this knowledge is an awful shame, especially in an electronic world, but I think it will come.

As of right now, While it is common that NMR Spectroscopists will assign their data electronically, it is very rare to find a group of chemists in the pharmaceutical industry, for example, who routinely use their processing tools to assign their data. Why?

1. They might not have the right software tools
2. It is not required. In fact, in some cases I have learned that it is forbidden. Why spend the time it takes to assign the data if it is not required or permitted?

A static PDF is indeed proof that an experiment was run, but does it contain information that supports a proof of the proposed structure? Where is the knowledge that was gained from this exercise?

I think 1D NMR Assistant significantly reduces the amount of time it takes to electronically assign a spectrum so now it is just a matter of finding an easy way to tie this assigned analytical data to the ELN.

Try it our for yourself here.

I think there is a real opportunity here.

What are your thoughts?

Would you prefer electronic data over PDFs?

Is simply raw or processed data enough?

How important is maintaining the knowledge gained from the experiment (i.e. assignments)?

Thanks to Rich for the multiple inspirations for this and previous posts.

Using NMR for Quantitative Analysis

Do you currently use NMR for quantitative analysis?

At the University of Ottawa NMR Facility Blog, Glenn Facey provides some acquisition tips for ¹H NMR spectra.

If you perform quantitative analysis on your spectra, how do you do it? Do you do it manually by hand, or do you use some software to help?

I am not sure how many of you are aware of Quanalyst, a quantitation tool available in ACD/1D and 2D NMR Processor that can measure different spectral attributes and automatically measure the result.

For example, some common applications of the tool are to:

• Calculate the ratio of components in a sample mixture. ( I think this application is especially useful for those in the chemical industry)

• Reaction Monitoring

• Finding and quantifying a multiplet for a specific atom

• Quantifying coupling constant changes in a specific multiplet across a series of spectrum

Here are a couple of application notes available that describe a few of the different applications of Quanalyst:

Reaction Monitoring with Quanalyst (PDF)

Optimizing the Process of Quantifying: From Manual to High-Throughput (PDF)

I will also provide you with a link to an old movie on Quanalyst from an older version of ACD/1D NMR Processor that I still think will give you an idea of how Quanlyst works. Note, if you can't view the movie, you can download it here.

Perhaps it's time for a new version of this movie!...I'll try to get to that and share it with you.

In the meantime, if you are currently doing quantitative analysis, I invite you to share your thoughts, insights, workflows, etc. in the comments section.

Download a Trial of ACD/1D NMR Assistant!

Apologies for not posting much lately. It has been a very busy month for me.

I have some great news to share. Starting today, ACD/Labs is offering free, 30-day download trials of the new ACD/1D NMR Assistant software. I have blogged a lot about this product over the last several months (start at the bottom), and now here's your chance to try it and provide feedback.

Learn more about the product and access the download here.

While I will certainly continue to blog about this product, I also want to point you to a new blog, The ACD/1D NMR Assistant Blog.

The primary purpose for this blog is to support the download campaign and to provide resources for downloaders so they can learn how to use specific features in the software. There are already several videos up and more to come. We will also provide brief commentaries on product development, design, features, etc. Please do not hesitate to comment or ask a question in the comment section. We want this to be an interactive experience where all users can benefit.

While I will also be an active author on this blog, I am also welcoming several other authors to contribute. The ACD/Labs I LOVE NMR Club consisting of Myself, Brent Lefebvre, Arvin Moser, Andrew Anderson. Joe Dimartino, and Patrick Wheeler will all contribute to the blog with their own thoughts, experiences, and tips regarding chemists and their interaction with NMR data.

So go ahead and download the software and be sure to check out the 1D NMR Assistant Blog.

I welcome your feedback on either blog and I hope you like the software!

Let's See your Printed Spectra Do This!- Part 2.

Back to the chemists with their ELN who continue to resort to their paper spectra.

Is it just an old habit?

No. I think it is something else. In fact I think there are two major (and completely understandable) reasons why some chemist continue to resist the complete transition to the electronic world:

1. One of our users who I really enjoy speaking with when I get a chance once told me, "The only way you are going to get chemist to fully adopt these tools is when they can access and interact with the spectra JUST as fast as they can do it on paper." I think it's a great point. For some chemists having to sit in front of a new piece (or old piece) of software to try and get your data out can be a daunting task. But I think significant strides have been made in this regard. With the transition to open access, NMR Spectroscopists have done a nice job (in conjunction with software vendors) to automate processing and create software macros to provide chemists with access to fully processed spectra. In addition, I think that the usability of NMR software for manual processing (See Shortcut Mode for example) has greatly improved over the years, but there is of course still work to do.
2. I think the other reason is that perhaps the benefits associated with electronic handling of NMR data have historically been not convincing enough or educated clearly enough to the user. For example, I have beaten the multiplet report topic to death on here, but I am continually amazed by the number of chemists who have had our software for a long time that aren't aware of this feature.

But perhaps easy access to data from your desk and formatted multiplet reports is not enough for a chemist to let go of the paper and embrace the electronic world. In fact, I am convinced it isn't. After all, an NMR spectrum is a means to an end. Sure, the delivery of the results in a readable form in a convenient place is essential and it has increased producitivity in open access environments, but the bottom line is that there is a reason the chemist ran an NMR experiment. In most cases that reason is to determine if their compound's proposed structure is consistent with their spectrum. Historically, NMR processing software has not provided any assistance in regards to data interpretation. 

I am hoping that ACD/1D NMR Assistant addresses this challenge and finally convinces the chemist to let go of their paper spectra for good and fully embrace the electronic world. To see how 1D NMR Assistant helps chemists interpret and assign their ¹H NMR spectra check out my previous vlog postings (with video) here and here.

Are You Attached to the Paper Printouts of Your Spectra?-Part 1

This is the topic I will be presenting on at our annual ENC Symposium on March 9, 2008. If you happen to be attending this conference in Asilomar, check out the agenda and register here.

I mentioned this before, but I follow the Depth-First Blog authored by Rich Apodaca rather closely and I highly recommend it.

I mention it again because Rich had a very interesting post a few weeks back inspired by a discussion about Electronic Lab Notebooks (ELNs) that took place on Derek Lowe's In the Pipeline Blog (which is another blog I follow frequently and highly recommend!)

Fascinating posts and discussion for sure!

Rich notes:

The wasteful process of entombing valuable scientific data often begins with the paper lab notebook, so the subject of ELNs should be of great interest to anyone involved in creating, using, or reprocessing chemical information.

Why do paper notebooks continue to persist in chemistry?

The issue is complex, but in my view stems from the lack of a truly usable and affordable tool. Although the term "tool" may suggest software, it actually involves a much more complex beast consisting of hardware, software, an ergonomic hardware/software user interface, and a computer network. In chemistry, the problem is compounded by the centrality of chemical structures and the inability of most generic ELN products to capture or use them. Given these constraints, and the costs associated with creating and marketing general-purpose products designed to work within them, it's not surprising that many organizations decide to roll their own ELN. And it's even less surprising that many others decide sticking with paper is a better option - at least for now.

I think this is a good argument, and I want to add to it with the question, "Why does paper spectra continue to persist in chemistry?"

Personally, I am amazed by the number of times I have encountered groups who are already using ELNs, but are still routinely using paper spectra. Sure when the time comes to attach their PDFs (Rich has shared his feelings on this format as well, but that's an entirely different discussion!) to their notebook records, they will extract the electronic file, but until that point many chemists will walk back to the instrument room, pick up their data printout, study it, and then toss it in the recycling bin or toss it on their bench!

For example, I was recently visiting with some chemists in the pharmaceutical industry who were providing me with feedback on our products and discussing their workflow. They mentioned that while they religiously use the desktop NMR processing software for viewing, processing, analysis, interpretation, and reporting to their ELN right on their laptops, in their lab...many chemists from their group still make the walk to the instrument room for their piece of paper instead.

Paperless environment?

Old habits, I guess.

Why use a piece of paper when you can access the data electronically? Meaning you can zoom in and zoom out on regions of the spectra to take a closer look? I guess in most cases, you don't really need to do that but in many labs where the data can be accessed electronically on a laptop in your lab, why make the walk back to the instrument room for that piece of paper?

What are your thoughts? Are you still attached to the paper printout? If so, why?

I'll share my opinion in part 2 of this topic tomorrow.