SAMPL Challenge

Last updated

SAMPL (Statistical Assessment of the Modeling of Proteins and Ligands) is a set of community-wide blind challenges aimed to advance computational techniques as standard predictive tools in rational drug design. [1] [2] [3] [4] [5] A broad range of biologically relevant systems with different sizes and levels of complexities including proteins, host–guest complexes, and drug-like small molecules have been selected to test the latest modeling methods and force fields in SAMPL. New experimental data, such as binding affinity and hydration free energy, are withheld from participants until the prediction submission deadline, so that the true predictive power of methods can be revealed. The most recent SAMPL5 challenge contains two prediction categories: the binding affinity of host–guest systems, and the distribution coefficients of drug-like molecules between water and cyclohexane. [6] [7] Since 2008, the SAMPL challenge series has attracte interest from scientists engaged in the field of computer-aided drug design (CADD) [8] [9] [10] The current SAMPL organizers include John Chodera, Michael K. Gilson, David Mobley, and Michael Shirts. [11]

Contents

Project significance

The SAMPL challenge seeks to accelerate progress in developing quantitative, accurate drug discovery tools by providing prospective validation and rigorous comparisons for computational methodologies and force fields. Computer-aided drug design methods have been considerably improved over time, along with the rapid growth of high-performance computing capabilities. However, their applicability in the pharmaceutical industry are still highly limited, due to the insufficient accuracy. Lacking large-scale prospective validations, methods tend to suffer from over-fitting the pre-existing experimental data. To overcome this, SAMPL challenges have been organized as blind tests: each time new datasets are carefully designed and collected from academic or industrial research laboratories, and measurements are released shortly after the deadline of prediction submission. Researchers then can compare those high-quality, prospective experimental data with the submitted estimates. A key emphasis is on lessons learned, allowing participants in future challenges to benefit from modeling improvements made based on earlier challenges.

SAMPL has historically focused on the properties of host–guest systems and drug-like small molecules. These simply model systems require considerably less computational resources to simulate than protein systems, and thus converge more quickly. Through careful design, these model systems can be used to focus on one particular or a subset of simulation challenges.[ further explanation needed ] [12] The past several SAMPL host–guest, hydration free energy and log D challenges revealed the limitations in generalized force fields, [13] [14] facilitated the development of solvent models, [15] [16] and highlighted the importance of properly handling protonation states and salt effects. [17] [18]

Participation

Registration and participation is free for SAMPL challenges. Beginning with SAMPL7, challenge participation data was posted on the SAMPL website, [19] as well as the GitHub page for the specific challenge. Instructions, input files and results were then provided through GitHub (earlier challenges provided content primarily through D3R for SAMPL4-5, and via other means for earlier SAMPLs). Participants were allowed to submit multiple predictions through the D3R website, either anonymously or with research affiliation. Since the SAMPL2 challenge, all participants have been invited to attend the SAMPL workshops and submit manuscripts to describe their results. After a peer-review process, the resulting papers, along with the overview papers which summarize all submitting data, were published in the special issues of the Journal of Computer-Aided Molecular Design. [20]

Funding

The SAMPL project was recently funded by the NIH (grant GM124270-01A1), for the period of Sept. 2018 through August 2022, to allow the design of future SAMPL challenges to drive advances in the areas they are most needed for modeling efforts. [9] [10] The effort is spearheaded by David L. Mobley (UC Irvine) with co-investigators John D. Chodera (MSKCC), Bruce C. Gibb (Tulane), and Lyle Isaacs (Maryland). Currently challenges and workshops are run in partnership with the NIH-funded Drug Design Data Resource, but this will likely change over time as funding for the two projects is not coupled.

Funding also allowed a broadening of scope of SAMPL; through SAMPL6, its role had been seen as primarily focused on physical properties, with D3R handling protein-ligand challenges. However, the funded effort broadened its focus to include systems which will drive improvements in modeling, including potentially suitable protein-ligand systems. This is still in contrast to D3R, which relies on donated datasets of pharmaceutical interest, whereas SAMPL challenges are specifically designed to focus on specific modeling challenges.

History

Earlier SAMPL challenges

The first SAMPL exercise, SAMPL0 (2008) [21] focused on the predictions of solvation free energies of 17 small molecules. A research group at Stanford University and scientists at OpenEye Scientific Software carried out the calculations. Despite the informal format, SAMPL0 laid the groundwork for the following SAMPL challenges.

SAMPL1 (2009) [22] and SAMPL2 challenges (2010) [1] were organized by OpenEye and continued to focus on predicting solvation free energies of drug-like small molecules. Attempts were also made to predict binding affinities, binding poses and tautomer ratios. Both challenges attracted significant participations from computational scientists and researchers in academia and industry.

SAMPL3 and SAMPL4

The blinded data sets for host–guest binding affinities were introduced for the first time in SAMPL3 (2011-2012), [3] along with solvation free energies for small molecules and the binding affinity data for 500 fragment-like tyrosine inhibitors. Three host molecules were all from the cucurbituril family. The SAMPL3 challenge received 103 submissions from 23 research groups worldwide. [2]

Different from the prior three SAMPL events, the SAMPL4 exercise (2013-2014) [4] [5] was coordinated by academic researchers, with logistical support from OpenEye. Datasets in SAMPL4 consisted of binding affinities for host–guest systems and HIV integrase inhibitors, as well as hydration free energies of small molecules. Host molecules included cucurbit[7]uril (CB7) and octa-acid. The SAMPL4 hydration challenge involved 49 submissions from 19 groups. The participation of the host–guest challenge also grew significantly compared to SAMPL3. The workshop was held at Stanford University in September, 2013.

SAMPL5

The protein-ligand challenges were separated from SAMPL in SAMPL5 (2015-2016) [6] [7] and were distributed as the new Grand Challenges of the Drug Design Data Resource (D3R). [23] SAMPL5 allowed participants to make predictions of the binding affinities of three sets of host–guest systems: an acyclic CB7 derivative and two host from the octa-acid family. Participants were also encouraged to submit predictions for binding enthalpies. A wide array of computational methods were tested, including density functional theory (DFT), molecular dynamics, docking, and metadynamics. The distribution coefficient predictions were introduced for the first time, receiving total of 76 submissions from 18 researcher groups or scientists for a set of 53 small molecules. The workshop was held in March, 2016 at University of California, San Diego as part of the D3R workshop. The top-performing methods in the host–guest challenge yielded encouraging yet imperfect correlations with experimental data, accompanied by large, systematic shifts relative to experiment. [24] [25]

SAMPL6

The SAMPL6 testing systems include cucurbit[8]uril, octa-acid, tetra-endo-methyl octa-acid, and a series of fragment-like small molecules. The host–guest, conformational sampling and pKa prediction challenges of SAMPL6 are now closed. The SAMPL6 workshop was jointly run with the D3R workshop in February 2018 at the Scripps Institution of Oceanography [26] and a SAMPL special issue of the Journal of Computer Aided Molecular Design reported many of the results. A SAMPL6 Part II challenge focused on a small octanol-water partition coefficient prediction set and was followed by a virtual workshop on May 16, 2019 and a joint D3R/SAMPL workshop in San Diego in August 2019. A special issue or special section of JCAMD is planned to report the results.[ needs update ] SAMPL6 inputs and results are available via the SAMPL6 GitHub repository.

SAMPL7

SAMPL7 again included host-guest challenges and a physical property challenge. A protein-ligand binding challenge on PHIPA fragments was also included. Host-guest binding focused on several small molecules binding to octa-acid and exo-octa-acid; binding of two compounds to a series of cyclodextrin derivatives; and binding of a series of small molecules to a clip-like guest known as TrimerTrip. A SAMPL7 virtual workshop took place and is available online. A SAMPL7 physical properties challenge is currently ongoing. Plans for a EuroSAMPL in-person workshop in Fall 2020 were derailed by COVID-19 and the workshop is being conducted virtually. SAMPL7 inputs and (as challenge components are completed, results) are available via the SAMPL6 GitHub repository.

SAMPL8

SAMPL8 included host-guest components on binding of drugs of abuse to CB8, and a series of small molecules to Gibb Deep Cavity Cavitands (GDCCs), as detailed on the SAMPL8 GitHub repository. An additional pKa and logD challenge focused on pK and logD prediction for a series of drug-like molecules.

SAMPL9

SAMPL9 is in planning stages, except that a SAMPL9 host-guest challenge on a host from Lyle Isaacs' group is currently underway. Details are available on the SAMPL9 GitHub repository

SAMPL Special Issues

SAMPL Publications

A relatively complete list of SAMPL-related publications is maintained by the SAMPL organizers; more than 150 related papers have been published.

Future challenges

SAMPL is slated to continue its focus on physical property prediction, including logP and logD values, pKa prediction, host–guest binding, and other properties, as well as broadening to include a protein-ligand component. [9] Some data is planned to be collected directly by the SAMPL co-investigators (Chodera, Gibb and Isaacs), but industry partnerships and internships are also proposed. [9]

See also

Related Research Articles

<span class="mw-page-title-main">Computational chemistry</span> Branch of chemistry

Computational chemistry is a branch of chemistry that uses computer simulation to assist in solving chemical problems. It uses methods of theoretical chemistry, incorporated into computer programs, to calculate the structures and properties of molecules, groups of molecules, and solids. The importance of this subject stems from the fact that, with the exception of some relatively recent findings related to the hydrogen molecular ion, achieving an accurate quantum mechanical depiction of chemical systems analytically, or in a closed form, is not feasible. The complexity inherent in many-body problem exacerbates the challenge of providing detailed descriptions in quantum mechanical systems. While computational results normally complement the information obtained by chemical experiments, it can in some cases predict unobserved chemical phenomena.

<span class="mw-page-title-main">Drug design</span> Invention of new medications based on knowledge of a biological target

Drug design, often referred to as rational drug design or simply rational design, is the inventive process of finding new medications based on the knowledge of a biological target. The drug is most commonly an organic small molecule that activates or inhibits the function of a biomolecule such as a protein, which in turn results in a therapeutic benefit to the patient. In the most basic sense, drug design involves the design of molecules that are complementary in shape and charge to the biomolecular target with which they interact and therefore will bind to it. Drug design frequently but not necessarily relies on computer modeling techniques. This type of modeling is sometimes referred to as computer-aided drug design. Finally, drug design that relies on the knowledge of the three-dimensional structure of the biomolecular target is known as structure-based drug design. In addition to small molecules, biopharmaceuticals including peptides and especially therapeutic antibodies are an increasingly important class of drugs and computational methods for improving the affinity, selectivity, and stability of these protein-based therapeutics have also been developed.

Quantitative structure–activity relationship models are regression or classification models used in the chemical and biological sciences and engineering. Like other regression models, QSAR regression models relate a set of "predictor" variables (X) to the potency of the response variable (Y), while classification QSAR models relate the predictor variables to a categorical value of the response variable.

<span class="mw-page-title-main">Molecular recognition</span> Type of non-covalent bonding

The term molecular recognition refers to the specific interaction between two or more molecules through noncovalent bonding such as hydrogen bonding, metal coordination, hydrophobic forces, van der Waals forces, π-π interactions, halogen bonding, or resonant interaction effects. In addition to these direct interactions, solvents can play a dominant indirect role in driving molecular recognition in solution. The host and guest involved in molecular recognition exhibit molecular complementarity. Exceptions are molecular containers, including e.g. nanotubes, in which portals essentially control selectivity.

<span class="mw-page-title-main">Docking (molecular)</span> Prediction method in molecular modeling

In the field of molecular modeling, docking is a method which predicts the preferred orientation of one molecule to a second when a ligand and a target are bound to each other to form a stable complex. Knowledge of the preferred orientation in turn may be used to predict the strength of association or binding affinity between two molecules using, for example, scoring functions.

Macromolecular docking is the computational modelling of the quaternary structure of complexes formed by two or more interacting biological macromolecules. Protein–protein complexes are the most commonly attempted targets of such modelling, followed by protein–nucleic acid complexes.

SciTegic was a San Diego-based software company that developed and marketed informatics software to the pharmaceutical and biotechnology industries.

<span class="mw-page-title-main">Virtual screening</span>

Virtual screening (VS) is a computational technique used in drug discovery to search libraries of small molecules in order to identify those structures which are most likely to bind to a drug target, typically a protein receptor or enzyme.

In the fields of computational chemistry and molecular modelling, scoring functions are mathematical functions used to approximately predict the binding affinity between two molecules after they have been docked. Most commonly one of the molecules is a small organic compound such as a drug and the second is the drug's biological target such as a protein receptor. Scoring functions have also been developed to predict the strength of intermolecular interactions between two proteins or between protein and DNA.

Inte:Ligand was founded in Maria Enzersdorf, Lower Austria (Niederösterreich) in 2003. They established the company headquarters on Mariahilferstrasse in Vienna, Austria that same year.

The program UCSF DOCK was created in the 1980s by Irwin "Tack" Kuntz's Group, and was the first docking program. DOCK uses geometric algorithms to predict the binding modes of small molecules. Brian K. Shoichet, David A. Case, and Robert C.Rizzo are codevelopers of DOCK.

Computational Resources for Drug Discovery (CRDD) is one of the important silico modules of Open Source for Drug Discovery (OSDD). The CRDD web portal provides computer resources related to drug discovery on a single platform. It provides computational resources for researchers in computer-aided drug design, a discussion forum, and resources to maintain a wiki related to drug discovery, predict inhibitors, and predict the ADME-Tox property of molecules. One of the major objectives of CRDD is to promote open source software in the field of chemoinformatics and pharmacoinformatics.

<span class="mw-page-title-main">BindingDB</span>

BindingDB is a public, web-accessible database of measured binding affinities, focusing chiefly on the interactions of proteins considered to be candidate drug-targets with ligands that are small, drug-like molecules. As of March, 2011, BindingDB contains about 650,000 binding data, for 5,700 protein targets and 280,000 small molecules. BindingDB also includes a small collection of host–guest binding data of interest to chemists studying supramolecular systems.

Lead Finder is a computational chemistry tool designed for modeling protein-ligand interactions. This application is useful for conducting molecular docking studies and quantitatively assessing ligand binding and biological activity. Lead Finder offers free access to individual users, especially those in non-commercial and academic settings.

<span class="mw-page-title-main">SR-57227</span> Chemical compound

SR-57227 is a potent and selective agonist at the 5HT3 receptor, with high selectivity over other serotonin receptor subtypes and good blood–brain barrier penetration.

Rommie E. Amaro is a professor and endowed chair of chemistry and biochemistry and the director of the National Biomedical Computation Resource at the University of California, San Diego. Her research focuses on development of computational methods in biophysics for applications to drug discovery.

Yvonne Connolly Martin is an American cheminformatics and computer-aided drug design expert who rose to the rank of Senior Volwiler Research Fellow at Abbott Laboratories. Trained in chemistry at Northwestern University, she became a leader in collaborative science aimed at discovering and developing bioactive molecules as therapeutic agents, with her contributions proceeding from application of methods to understand how descriptors of molecular shapes and physicochemical properties relate to their biological activity. She is the author of a seminal volume in cheminformatics, Quantitative Drug Design, and has been the recipient of numerous awards in her field, including being named as a fellow of the American Association for the Advancement of Science (1985) and of the International Union of Pure and Applied Chemistry (2000), and receiving the Herman Skolnik Award (2009) and the Award for Computers in Chemical and Pharmaceutical Research (2017) from the American Chemical Society.

Molecular Operating Environment (MOE) is a drug discovery software platform that integrates visualization, modeling and simulations, as well as methodology development, in one package. MOE scientific applications are used by biologists, medicinal chemists and computational chemists in pharmaceutical, biotechnology and academic research. MOE runs on Windows, Linux, Unix, and macOS. Main application areas in MOE include structure-based design, fragment-based design, ligand-based design, pharmacophore discovery, medicinal chemistry applications, biologics applications, structural biology and bioinformatics, protein and antibody modeling, molecular modeling and simulations, virtual screening, cheminformatics & QSAR. The Scientific Vector Language (SVL) is the built-in command, scripting and application development language of MOE.

Namasivayam Gautham is a retired Professor Emeritus at the Centre of Advance Study in Crystallography and Biophysics, University of Madras. He is known for his work on DNA Crystallography, protein structure prediction and molecular docking.

<span class="mw-page-title-main">David Weininger</span> American chemist (1952–2016)

David Weininger was an American cheminformatician and entrepreneur. He was most notable for inventing the chemical line notations for structures (SMILES), substructures (SMARTS) and reactions (SMIRKS). He also founded Daylight Chemical Information Systems, Inc.

References

  1. 1 2 Geballe, Matthew T.; Skillman, A. Geoffrey; Nicholls, Anthony; Guthrie, J. Peter; Taylor, Peter J. (2010-05-09). "The SAMPL2 blind prediction challenge: introduction and overview". Journal of Computer-Aided Molecular Design. 24 (4): 259–279. Bibcode:2010JCAMD..24..259G. doi:10.1007/s10822-010-9350-8. ISSN   0920-654X. PMID   20455007. S2CID   12363968.
  2. 1 2 Skillman, A. Geoffrey (2012-05-24). "SAMPL3: blinded prediction of host–guest binding affinities, hydration free energies, and trypsin inhibitors". Journal of Computer-Aided Molecular Design. 26 (5): 473–474. Bibcode:2012JCAMD..26..473S. doi:10.1007/s10822-012-9580-z. ISSN   0920-654X. PMID   22622621. S2CID   8335429.
  3. 1 2 Muddana, Hari S.; Varnado, C. Daniel; Bielawski, Christopher W.; Urbach, Adam R.; Isaacs, Lyle; Geballe, Matthew T.; Gilson, Michael K. (2012-02-25). "Blind prediction of host–guest binding affinities: a new SAMPL3 challenge". Journal of Computer-Aided Molecular Design. 26 (5): 475–487. Bibcode:2012JCAMD..26..475M. doi:10.1007/s10822-012-9554-1. ISSN   0920-654X. PMC   3383923 . PMID   22366955.
  4. 1 2 Muddana, Hari S.; Fenley, Andrew T.; Mobley, David L.; Gilson, Michael K. (2014-03-06). "The SAMPL4 host–guest blind prediction challenge: an overview". Journal of Computer-Aided Molecular Design. 28 (4): 305–317. Bibcode:2014JCAMD..28..305M. doi:10.1007/s10822-014-9735-1. ISSN   0920-654X. PMC   4053502 . PMID   24599514.
  5. 1 2 Mobley, David L.; Wymer, Karisa L.; Lim, Nathan M.; Guthrie, J. Peter (2014-03-11). "Blind prediction of solvation free energies from the SAMPL4 challenge". Journal of Computer-Aided Molecular Design. 28 (3): 135–150. Bibcode:2014JCAMD..28..135M. doi:10.1007/s10822-014-9718-2. ISSN   0920-654X. PMC   4006301 . PMID   24615156.
  6. 1 2 Yin, Jian; Henriksen, Niel M.; Slochower, David R.; Shirts, Michael R.; Chiu, Michael W.; Mobley, David L.; Gilson, Michael K. (2016-09-22). "Overview of the SAMPL5 host–guest challenge: Are we doing better?". Journal of Computer-Aided Molecular Design. 31 (1): 1–19. doi:10.1007/s10822-016-9974-4. ISSN   0920-654X. PMC   5241188 . PMID   27658802.
  7. 1 2 Bannan, Caitlin C.; Burley, Kalistyn H.; Chiu, Michael; Shirts, Michael R.; Gilson, Michael K.; Mobley, David L. (2016-09-27). "Blind prediction of cyclohexane–water distribution coefficients from the SAMPL5 challenge". Journal of Computer-Aided Molecular Design. 30 (11): 927–944. Bibcode:2016JCAMD..30..927B. doi:10.1007/s10822-016-9954-8. ISSN   0920-654X. PMC   5209301 . PMID   27677750.
  8. L, Mobley, David; D, Chodera, John; K, Gilson, Michael (2017-06-21). "Results of the 2017 Roadmap survey of the Statistical Assessment of Modeling of Proteins and Ligands (SAMPL) challenge community". EScholarship.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  9. 1 2 3 4 Mobley, David L.; Chodera, John D.; Isaacs, Lyle; Gibb, Bruce C. (2016). "Advancing predictive modeling through focused development of model systems to drive new modeling innovations". Zenodo. doi:10.5281/zenodo.163963.
  10. 1 2 Mobley, David L. (2016-10-05). "Advancing predictive modeling through focused development of model systems to drive new modeling innovations". eScholarship.
  11. "D3R | SAMPL".
  12. Mobley, David L.; Gilson, Michael K. (2016-12-08). "Predicting binding free energies: Frontiers and benchmarks". bioRxiv   10.1101/074625 .
  13. Muddana, Hari S.; Gilson, Michael K. (2012-01-25). "Prediction of SAMPL3 host–guest binding affinities: evaluating the accuracy of generalized force-fields". Journal of Computer-Aided Molecular Design. 26 (5): 517–525. Bibcode:2012JCAMD..26..517M. doi:10.1007/s10822-012-9544-3. ISSN   0920-654X. PMC   3383906 . PMID   22274835.
  14. Mobley, David L.; Liu, Shaui; Cerutti, David S.; Swope, William C.; Rice, Julia E. (2011-12-24). "Alchemical prediction of hydration free energies for SAMPL". Journal of Computer-Aided Molecular Design. 26 (5): 551–562. doi:10.1007/s10822-011-9528-8. ISSN   0920-654X. PMC   3583515 . PMID   22198475.
  15. Pal, Rajat Kumar; Haider, Kamran; Kaur, Divya; Flynn, William; Xia, Junchao; Levy, Ronald M.; Taran, Tetiana; Wickstrom, Lauren; Kurtzman, Tom; Gallicchio, Emilio (2016-09-30). "A combined treatment of hydration and dynamical effects for the modeling of host–guest binding thermodynamics: the SAMPL5 blinded challenge". Journal of Computer-Aided Molecular Design. 31 (1): 29–44. doi:10.1007/s10822-016-9956-6. ISSN   0920-654X. PMC   5477994 . PMID   27696239.
  16. Brini, Emiliano; Paranahewage, S. Shanaka; Fennell, Christopher J.; Dill, Ken A. (2016-09-08). "Adapting the semi-explicit assembly solvation model for estimating water-cyclohexane partitioning with the SAMPL5 molecules". Journal of Computer-Aided Molecular Design. 30 (11): 1067–1077. Bibcode:2016JCAMD..30.1067B. doi:10.1007/s10822-016-9961-9. ISSN   0920-654X. PMC   5261860 . PMID   27632227.
  17. Sure, Rebecca; Antony, Jens; Grimme, Stefan (2014-03-27). "Blind Prediction of Binding Affinities for Charged Supramolecular Host–Guest Systems: Achievements and Shortcomings of DFT-D3". The Journal of Physical Chemistry B. 118 (12): 3431–3440. doi:10.1021/jp411616b. ISSN   1520-6106. PMID   24588346.
  18. Klamt, Andreas; Eckert, Frank; Reinisch, Jens; Wichmann, Karin (2016-07-26). "Prediction of cyclohexane-water distribution coefficients with COSMO-RS on the SAMPL5 data set". Journal of Computer-Aided Molecular Design. 30 (11): 959–967. Bibcode:2016JCAMD..30..959K. doi:10.1007/s10822-016-9927-y. ISSN   0920-654X. PMID   27460058. S2CID   33238269.
  19. "SAMPL | Challenges". samplchallenges.github.io. Retrieved 2020-07-03.
  20. "Journal of Computer-Aided Molecular Design - All Volumes & Issues - Springer". link.springer.com. Retrieved 2017-01-12.
  21. Nicholls, Anthony; Mobley, David L.; Guthrie, J. Peter; Chodera, John D.; Bayly, Christopher I.; Cooper, Matthew D.; Pande, Vijay S. (2008-02-01). "Predicting Small-Molecule Solvation Free Energies: An Informal Blind Test for Computational Chemistry". Journal of Medicinal Chemistry. 51 (4): 769–779. doi:10.1021/jm070549+. ISSN   0022-2623. PMID   18215013. S2CID   35414610.
  22. Guthrie, J. Peter (2009-04-09). "A Blind Challenge for Computational Solvation Free Energies: Introduction and Overview" . The Journal of Physical Chemistry B. 113 (14): 4501–4507. doi:10.1021/jp806724u. ISSN   1520-6106. PMID   19338360.
  23. Gathiaka, Symon; Liu, Shuai; Chiu, Michael; Yang, Huanwang; Stuckey, Jeanne A.; Kang, You Na; Delproposto, Jim; Kubish, Ginger; Dunbar, James B. (2016-09-30). "D3R grand challenge 2015: Evaluation of protein–ligand pose and affinity predictions". Journal of Computer-Aided Molecular Design. 30 (9): 651–668. Bibcode:2016JCAMD..30..651G. doi:10.1007/s10822-016-9946-8. ISSN   0920-654X. PMC   5562487 . PMID   27696240.
  24. Yin, Jian; Henriksen, Niel M.; Slochower, David R.; Gilson, Michael K. (2016-09-16). "The SAMPL5 host–guest challenge: computing binding free energies and enthalpies from explicit solvent simulations by the attach-pull-release (APR) method". Journal of Computer-Aided Molecular Design. 31 (1): 133–145. doi:10.1007/s10822-016-9970-8. ISSN   0920-654X. PMC   5241238 . PMID   27638809.
  25. Bosisio, Stefano; Mey, Antonia S. J. S.; Michel, Julien (2016-08-08). "Blinded predictions of host–guest standard free energies of binding in the SAMPL5 challenge" (PDF). Journal of Computer-Aided Molecular Design. 31 (1): 61–70. doi:10.1007/s10822-016-9933-0. ISSN   0920-654X. PMID   27503495. S2CID   31110594.
  26. "D3R 2018 Workshop". D3R. 2018-02-22. Retrieved 2022-03-27.