We are thrilled to announce the launch of the prestigious Outstanding Early Career Research Award, aimed at recognising and celebrating outstanding contributions in the realm of Digital Discovery. This initiative seeks to honour the dedication, innovation, and impactful research endeavours of promising early career researchers.
We looked back at the exceptionally high-quality and innovative Digital Discovery research published during the previous calendar year and put together a shortlist of articles based on a variety of metrics including article downloads, Altmetric score, and citations. The shortlist was reviewed by the journal's Editorial Board members based on the science presented and its potential future impact.
Our 2023 winners
Assessment of chemistry knowledge in large language models that generate code
Andrew White, Glen Hocky, Heta Gandhi, Mehrad Ansari, Sam Cox, Geemi Wellawatte, Subarna Sasmal, Ziyue Yang, Kangxin Liu, Yuvraj Singh and Willmor Peña Ccoa
Upon release of the Codex model (a variant of GPT-3 tuned on code) by OpenAI in August 2021, the authors began experimenting and discovered that this Large Language Model (LLM) could instantly generate short code snippets that would be typical of what we would ask students to generate either in research or for graduate course work, and that the model also had some inherent chemistry knowledge. This led us to publish our first article in the new journal Digital Discovery reporting our findings and giving predictions for the future, which have largely panned out!
However, we had a lingering question of how much does this or similar models really know chemistry, and what are the best way to pose questions to get reliable answers. To assess that, members from our two groups generated example problems spanning a wide range of chemistry, chemical engineering, and cheminformatics topics.
The trick was that we posed the problems in the context of asking the model to write a python function that returns the answer, this way we could automatically execute the code and assess whether the model knew that kind of information (e.g. the ideal gas law, how to calculate molecular properties from a string expressing a molecule, to simulate a random walk).
Codex proved highly accurate in these cases (even when problems weren’t posed in English), and we could improve the accuracy through various ‘prompt engineering’ strategies. This led us to be very optimistic about the power of these models to lower the barrier to performing many common chemistry tasks.
Another forgotten aspect of this paper is that we built a really nifty software tool for querying different language model APIs, getting back code, and executing that code on our example problems in order to do the evaluation (https://github.com/whitead/nlcc). We even built a Slack bot that could use this code to answer chemistry questions.
While we thought this tool would be really useful, upon the release of ChatGPT in November 2022, the general public had access to a conversational interface and our tool was therefore much less useful even in our own groups! Improvements to the model including the ChatGPT refinement and GPT-4 have made the results even more accurate in our original benchmark, and reduced the need for some of the prompt engineering strategies we tested (such as telling the model ‘I am an expert programmer’).
Read this paper
Guidelines for nominators
Find out who is eligible for this award, about the nomination process and see who is on the selection panel.
To be eligible to receive a Digital Discovery Outstanding Early Career Researcher Award:
- Researchers must have gained their first independent research position in the ten years preceding the award year.
- Researchers must have published an article in the journal in the previous calendar year.
- All submissions will be subject to initial assessment and peer review as appropriate according to the journal's guidelines.
The editorial team draw up a shortlist of papers based on a variety of metrics including article downloads, Altmetric score, and citations.
Winner(s) are selected by the Digital Discovery Editorial and Advisory Boards.
Selection panel
Our previous Outstanding Paper Award
This award was previously dedicated to recognising outstanding work published in Digital Discovery, celebrating the contributions of authors at all career stages.
We are proud to honour past recipients of the Outstanding Paper Award. As of 2023, this award has been retired and succeeded by the Outstanding Early Career Research Award, which continues our tradition of celebrating exceptional publications and contributions.