Pre- and post-publication peer review: some new tools

There has been some interesting developments in peer review lately:

Altmetric it is a bookmarklet you install in your browser.  Once installed go to any recent publication on the journal web page and click on the bookmarklet to get altmetric data (once you have installed it try this computational chemistry paper as an example).  Most importantly you get links to twitter comments and blog posts (I hear Google+ and PubPeer integration is on the way) where a lot of post-publication peer review happens.

I think this is the future of post-publication peer review: don't worry about where it happens, just make sure it can be easily found.

PubPeer is discussion forum centered around journal articles and preprints (click here for a computational chemistry example). These discussion fora are notoriously difficult to get off the ground so I am impressed how much discussion is already going on there.  PubPeer has recently made an extension that adds links to PubPeer comments on Pubmed search results and journal websites.

PeerLibrary allows you to create and share annotated versions of papers.  If you think about it, the most common form of post-publication peer review is the highlighting of text and margin notes we all make when we read a paper.  PeerLibrary allows you to share this.

I do most of my annotations using iAnnotate on my iPad and uploading pre-annotated pdf files does not seem to work for all pages, so I am not sure how much I will use this service yet.  Also, the text in the uploaded files are currently quite grainy. But I like the general idea very much and I'll probably use it more if/when these things get fixed. Anyway, you can see what I have messed around with so far here.

Publons is a site for listing, and getting credit for, the reviews you do.  This appeals greatly to me since usually review anonymously. I have just has a brief look at it (you can see my profile here) but I think my workflow will be the following:

(1) Prepare (e.g. paste in) the review and de-select "Has this article been published?".  This will just list the article title and journal.  As the site states "We can not publish the content of your review until the article has been published, as this would be unfair to the author if they decided to resubmit the article."

(2) Once the article is published, you can make the actual review visible.  It would be nice if Publons could keep track of this (using the info you provided) and alert you.

You can also review papers already published (in which case it's a bit like PubPeer) and you can also review anonymously.


Have I missed some cool reviewing tools?  If so, leave a comment.


This work is licensed under a Creative Commons Attribution 4.0

Notes from Michael Staton's talk at EduDisrupt

Last Wednesday I attended the morning session of the EduDisrupt meeting in Copenhagen.  The talk that interested me most was by venture capitalist Michael Staton, a former history teacher (some things he has written here and here).  Towards the end of the talk he flashed up four slides with companies he has invested in and some of them sounded pretty interesting.  I managed to take picture of the last 3 so I could look them up later, which is what I am doing now.

Slide 1

I am pretty sure Udemy and Edmodo was on it.

Slide 2 (shown above)

Talent sourcing: Brilliant

Evidenced hiring: HireArt

Industry assessment: Kalibrr

Slide 3

Peer motivation: NovoEd

Coached curriculum: Bloc

Expert guidance: popexpert

Slide 4

Distributed schooling:

AltSchool

mSchool

General Assembly

Gap year

Can we, teachers or universities, provide similar services or use similar tools to identify, help, retain, and promote students?



This work is licensed under a Creative Commons Attribution 4.0

Protease variants paper is out

Our methodology to find in a stochastic way variants for HIV-1 protease is finally published! You can read it and download it through this link.

The residues located by our methodology as the most apt to be changed, so as to bind better the new peptide substrate. Check out the paper to learn more.

The work behind this paper has been long and painful, but hopefully worth it. But that's no news, most papers are born that way.
The algorithm we used is based on PyRosetta, a Python interpreter to Rosetta. Structures were further evaluated with FMO single points at the MP2 level of theory, which accounts for a big part of the spent time.

Kids today and the good old days when lectures worked

A lecture boring students 600 years ago source

A few days ago the flipped classroom concept made the front page of the major Danish newspaper Politiken.  The occasion was the appointment of a new president of Roskilde University, Hanne Leth Andersen, a professor of education. Among other things, the interview motivated the use of the flipped classroom approach by arguing that incoming students have changed  in recent years and that they loose patience with hour-long lectures. 

Having given interviews myself, I know it's hard to separate what Leth Andersen actually said and how it is presented in the article, so I think the article is being unfair to students in that respect. Lectures never really worked well and we have known that for a long time.

Perhaps the most celebrated lectures in the natural sciences are the Feynman Lectures on Physics, which he gave to Caltech students more than 50 years ago. Here is what Feynman wrote in the preface to the lecture notes in 1963 (emphasis mine): 

The question, of course, is how well this experiment has succeeded. My own point of view—which, however, does not seem to be shared by most of the people who worked with the students—is pessimistic. I don't think I did very well by the students. When I look at the way the majority of the students handled the problems on the examinations, I think that the system is a failure. 

I think, however, that there isn't any solution to this problem of education other than to realize that the best teaching can be done only when there is a direct individual relationship between a student and a good teacher—a situation in which the student discusses the ideas, thinks about the things, and talks about the things. It's impossible to learn very much by simply sitting in a lecture, or even by simply doing problems that are assigned. 

One of the most common components of the flipped classroom approach is peer instruction, pioneered by Eric Mazur at Harvard University, in which lecture is replaced by in-class discussion and voting using clickers. Mazur switched to this approach in the early 1990's because he found that lecturing lead to rote-memorization and little conceptual understanding among his elite Harvard pre-med students.  He was lead to this realization by a series of papers published in 1985 by David Hestenes that demonstrated this general trend based on test results from ca 1000 students taught by 7 different instructors at two different universities in the early 1980's.

Carl Wieman related this story from another pioneer in science education, Joe Redish, who came to the same realization in the late 1970's:

Even though the students thought his lectures were wonderful, Joe wondered how much they were actually learning. So he hired a graduate student to grab students at random as they filed out of class at the end of the lecture and ask, “What was the lecture you just heard about?” It turned out that the students could respond with only with the vaguest of generalities.

I'm not saying students haven't changed. And the way we teach students need to change. But in the case of "the lecture" as the primary teaching tool, these two things are largely unrelated.

This work is licensed under a Creative Commons Attribution 4.0

New manuscript: A third-generation dispersion and third-generation hydrogen bonding corrected PM6 method: PM6-D3H+

After a long fight, I can finally say that I've submitted the method I've been working on. PM6-D3H+. Well, the method is actually 'stolen' pieces from other great work, implemented in GAMESS.

The MS is submitted to +PeerJ and is therefore also available on preprint:
https://peerj.com/preprints/353v1

Abstract:
We present new dispersion and hydrogen bond corrections to the PM6 method, PM6-D3H+, and its implementation in the GAMESS program. The method combines the DFT-D3 dispersion correction by Grimme et al with a modified version of the H+ hydrogen bond correction by Korth. Overall, the interaction energy of PM6-D3H+ is very similar to PM6-DH2 and PM6-DH+, with RMSD and MAD values within 0.02 kcal/mol of one another. The main difference is that the geometry optimizations of 88 complexes result in 82, 6, 0, and 0 geometries with 0, 1, 2, and $\ge$ 3 imaginary frequencies using PM6-D3H+ implemented in GAMESS, while the corresponding numbers for PM6-DH+ implemented in MOPAC are 54, 17, 15, and 2. The PM6-D3H+ method as implemented in GAMESS offers an attractive alternative to PM6-DH+ in MOPAC in cases where the LBFGS optimizer must be used and a vibrational analysis is needed, e.g. when computing vibrational free energies. While the GAMESS implementation is up to 10 times slower for geometry optimizations of proteins in bulk solvent, compared to MOPAC, it is sufficiently fast to make geometry optimizations of small proteins practically feasible.

The method is implemented in GAMESS, and will be available to public as soon as possible.

A general chemistry course meets Wolfram Alpha

Even the most thoughtful, dedicated teachers spend enormously more time worrying about their lectures than they do about their homework assignments, which I think is a mistake. Extended, highly focused mental processing is required to build those little proteins that make up the long-term memory. No matter what happens in the relatively brief period students spend in the classroom, there is not enough time to develop the long-term memory structures required for subject mastery. 

To ensure that the necessary extended effort is made, and that it is productive, requires carefully designed homework assignments, grading policies, and feedback. 

Carl Wieman 

In a previous post I showed some examples of how some chemistry problems posted on Reddit and Yahoo can be solved simply by typing them into Wolfram Alpha (WA) and suggested that we should revisit the general chemistry curriculum in light of new tools such as WA.  

To get more data I signed up for the Introduction to Chemistry MOOC offered by Coursera.  In this course the homework consists of 8 quizzes, each with between 11 and 15 questions plus a Pre-Course Concept Assessment Quiz. Here's what I found.

Pre-Course Concept Assessment Quiz
7 out of 15 questions could be done by typing them into WA.  For example: Solve the following system of two equations with two unknowns: x + y = 1 and 5x + y = 2. This questions tests for manual skills not really needed anymore, much like "what is the square root of 2?".

This question was much better: An architect presents a 3 inch wide by 4 inch deep by 3 inch tall model of a new central campus dorm. If the final building foundation is 126 feet wide, then how tall will the building be?  Of course this can also be solved with WA but the student must reformulate the question first.  I usually don't count questions such as this as square-root-of-2 problems.

Week 1 Introduction
4 out of 15 questions could be done by typing them into WA. For example: Wavelength of orange light is 0.00000060 m, scientific notation is ______ m.  A conceptual question on scientific notation would be much more useful.  Questions like this is also trivial in WA: A Boeing 747 carries 1.834 x 10^5 liters of jet fuel. Convert this volume to cm3.  These are "wasted" questions.

4 questions concerned significant figures, which WA does not handle.  For example: Perform the following calculation and input the answer expressed to the correct number of significant figures:  80720 ÷ (15.3 – 7.009) × 1.86.  This is an important skill that must be taught.  Even though it's not used in the rest of the course :).

Week 2 Matter and Energy

10 out of 13 questions could be done by typing them into WA.  For example: Name the following compound: CaF2.  Even: Which of the following neutral atoms has the smallest first ionization energy? Si, Sc, Sr, B, N.  A much better question is why the order is what it is, but how to phrase that as a multiple choice question?

Here's one that WA couldn't answer: Identify each of the following ions with their correct chemical symbol: Species with 8 protons and 10 electrons; Species with 30 protons and 28 electrons.

Week 3 Chemical Composition, Solutions, and Dissolution Equations

8 out of 11 questions could be done by typing them into WA.  Here I count questions like: A 12.0 g sample of HF is dissolved in water to give 3.1×102 mL of solution. The concentration of the solution is: that take a little bit of thought.  But there are completely straight forward questions like: Dinitrogen monoxide (more commonly called nitrous oxide) has the chemical formula N2O. What is the mass percent oxygen in nitrous oxide?.  Also, simple trial and error becomes effortless: A compound containing phosphorus and oxygen has a molar mass of 219.9 g/mol and an empirical formula of P2O3. Select its molecular formula from the following list: P6O8, P4O6, PO, none of the formulas listed are correct, P2O3

Here's one that requires some thought: A sample of sodium dichromate, Na2Cr2O7, is placed into a container by itself. The sample of material in the container is analyzed, and it is found to contain exactly 0.67 moles of sodium atoms. How many moles of oxygen atoms are in this sample?

Week 4 Chemical Composition, Solutions, Dissolution and Precipitation

1 out of 11 questions could be done by typing them into WA and it is: Determine the oxidation state of the nitrogen in each of the following molecules or ions: NO2^-1

WA is simply not yet equipped to handle questions like: A solution is known to contain only one type of anion. Addition of Tl1+ ion to the solution had no apparent effect (all ions remained in solution), but addition of Ba2+ ion resulted in a precipitate. Which anion is present? SO4^2-, Cl1-, I1-, NO3^1-

Week 5 no quiz

Week 6 Atomic Structure

5 out of 14 questions could be done by typing them into WA.  For example: Use the periodic table to write the electron configuration for the following element: Ba. By itself it is a pointless question. What you really want to know is: "How many core and valence electrons are in the following neutral atom? Se" which is also readily available from WA.  

And just because a question cannot be answered easily with WA doesn't mean it's a good question. For example, what's the point of this question?: Give the number of s, p, d, and f electrons in the following neutral atom when it is in the ground state.

I liked this one though: 

Below is the energy level diagram (not drawn to scale) representing the transitions made by an electron in a hydrogen atom that result in the observed lines of both the absorption and emission spectra. Some are in the visible region, and some are not.  4 different energy photons are represented (approximate wavelengths are given in parentheses): infrared (~ 10-4 m), red (~ 10-6 m), blue (~ 10-7 m), ultraviolet (~ 10-8 m).  Match the transition (a - h) with the photon described (approximate wavelengths are given in parentheses.) Your answer input should be a single, lower case letter. (Please note: This is not a problem for which a calculator is required. Your knowledge of the Bohr model of the atom and the relative energies of transitions is all that is needed.)

Week 7 Molecular Structures and Shapes 
4 out of 11 questions could be done by typing them into WA.  Drawing Lewis structures is becoming a square-root-of-2 problem: Select the correct Lewis structure for the following ion. N3-. But most of the questions in this section are quite good and not immediately answerable by WA.

Week 8 Ideal Gas Law and Intermolecular Forces
2 out of 11 questions could be done by typing them into WA.  For example: A container of 8.03 x 10-3 moles of hydrogen gas has a volume of 20.9 mL and a temperature of 20.8 degrees C.  You could argue that this example requires some processing of the information given, but certainly all the "heavy lifting" in terms of units and conversions is done.

But, again, most of the questions in this section are quite good and not immediately answerable by WA.

Week 9 Solution Calculations
7 out of 12 questions could be done by typing them into WA.  For example imagine being completely stuck on "What is the mass of fructose, also known as fruit sugar (C6H12O6), in a 127 mL sample of glucose solution that has a concentration of a 1.44 M?". Simply type in 127 mL 1.44 M fructose in WA.

Surprisingly WA can't handle "What is the mass percent concentration of the solution if 11.9 g of ethanol is dissolved in 67.4 g of water? " directly.

Summary
So, 42% (48/113) of the questions can be easily done with WA and at least a 3rd are what I would call square-root-of-2 questions - questions that are no longer really meaningful in and of themselves.  And in many weeks the majority of the questions are like that.  That's a waste of very valuable student time and attention.

I should mention that there are also advanced problems sets that "need to be completed in order to achieve a statement of accomplishment with distinction."  Many of these are quite interesting problems that I think could be assigned to all students if they are taught to use WA effectively.  This is what we should be aiming for:

1. An 8-year-old child who weighs 66 pounds needs to be treated for a novel influenza A (H1N1) infection. For a child of this size, the total daily recommended amount of the antiviral drug Oseltamivir is 4.0 mg of drug per kg of body weight. The total daily amount of medication should be divided into two equal doses. (Source: Clinical Infectious Diseases 2009; 48:1003–1032.) A liquid suspension of this medication contains 12 mg Oseltamivir per mL. How many mg of the antiviral drug should be given to the child for her first dose?

1. Look for the chemistry terms and unfamiliar words. Do you understand all of the terms?
2. What is the question asking for?
3. Write down in words a short sketch for how you would solve this problem. What are the steps to solve this problem? Is any necessary information missing? Which information is provided that you do not need to answer the question?
4. Solve Problem 1: the answer is _______ mg. 

This work is licensed under a Creative Commons Attribution 4.0

Citations: some numbers from Denmark

Just out of curiosity I checked Web of Science (WOS) and found:

The most cited paper with a co-author working in Denmark is "Improved methods for building protein models in electron-density maps and the location of errors in these models" published in 1991 in Acta Crystallographica Section A with 12,625 citations.  The second most cited paper has 6,303 citations.  The Acta Cryst. A paper is the 7th most cited paper on the topic of "chemistry" (as defined by WOS) worldwide.

If we restrict the search to "chemistry" (as defined by WOS) then it is Improved prediction of signal peptides: SignalP 3.0 published in Journal of Molecular Biology in 2004 with 4,260 citations.

I would classify the latter paper as bioinformatics and for some reason the Acta Cryst. A paper didn't show up, so lets add "dept chem" to the search instead of restricting the search by subject. One of the co-authors on the Acta Cryst. A paper is from the Department of Chemistry at the University of Aarhus, so that's the top one. The next one is "Identification of prokaryotic and eukaryotic signal peptides and prediction of their cleavage sites" published in Protein Engineering in 1997 with 4,368 citations.

That still smells like bioinformatics to me, but just shows how versatile chemists are.  Anyway, the third most cited paper is definitely in the realm of traditional chemistry: "Peptidotriazoles on solid phase: [1,2,3]-triazoles by regiospecific copper(I)-catalyzed 1,3-dipolar cycloadditions of terminal alkynes to azides" published in Journal of Organic Chemistry in 2002 with 3,251 citations.

This work is licensed under a Creative Commons Attribution 4.0