In the field of biomedicine, promise is mixed with pitfalls. What does our future with artificial intelligence look like? What challenges might we face?
Experts from the Morgridge Institute addressed these questions during a Fearless Science Speaker Series lecture on March 21, 2024. A full recording can be viewed in full above.
In the spirit of this topic, we wanted to demonstrate the potential capabilities (and limitations) of AI in summarizing the event. We tested a tool that is widely recognized by the general public and readily available to use: ChatGPT.
How accurate is the AI-generated text? Can we rely on these tools to improve our workflow? Read the following summary, with our review below.
The following summary was written by running the video transcript through ChatGPT 3.5. Light edits were made for length.
Morgridge CEO Brad Schwartz introduced the institute’s focus on curiosity-driven biomedical research. He emphasized the role of artificial intelligence (AI) as a tool to decipher the intricate workings of biological processes that will allow us to unravel mysteries about health and disease.
Tony Gitter, investigator in virology and research computing, recounted an engaging anecdote from a college course he taught, introducing the evolving perceptions of AI among students and the broader understanding of its applications beyond neural networks and deep learning (1).
Biomedical imaging investigator Juan Caicedo delved into the application of AI in cellular biology, particularly in analyzing microscopy images to decipher cellular activities. He emphasized the challenges posed by the complexity of biological systems and the need for AI to unravel hidden patterns in biomedical images.
Ron Stewart, investigator in bioinformatics, then explored the use of AI in analyzing biomedical text data, highlighting the overwhelming volume of scientific literature and the potential of AI to uncover hidden relationships between concepts. He introduced the Kinderminer algorithm for identifying associations between biomedical terms and showcased its potential applications in drug repurposing and adverse drug event detection.
Gitter returned to discuss his group’s research on protein engineering, where AI and computational simulations play a crucial role in optimizing protein functions for therapeutic purposes. He explained the challenges of modifying protein sequences to enhance desired outcomes and outlined their innovative approach of combining biophysical simulations with experimental data to train AI models for accurate predictions.
In a lively Q&A, the speakers delved into a nuanced discussion surrounding the limitations and potentials of AI in biomedicine.
The speakers acknowledged advancements in AI, particularly in web data analysis and natural language processing, where AI models like GPT have showcased impressive accuracy (2). However, they emphasized the need for caution, highlighting instances where AI models produce inaccurate or even fabricated results (3). Despite the progress, there remains a margin for error that demands careful consideration, especially in critical applications like medical diagnosis and drug development.
Discussion also delved into the interpretability of AI models, with insights shared on the challenges of discerning between AI-generated and human-authored content. Techniques such as analyzing punctuation in text and inconsistencies in image shadows were explored as potential indicators of AI-generated data.
Moreover, the conversation expanded to address the role of AI in forecasting infectious disease pandemics. Researchers discussed how AI techniques, particularly in protein modeling and drug development, offer valuable insights for predicting viral mutations and prioritizing therapeutic targets. Collaborative efforts between AI researchers and wet lab scientists were highlighted as crucial for validating AI predictions and improving model accuracy.
Questions emerged regarding the iterative nature of AI model development and evaluation. Strategies such as active learning and incremental model refinement were discussed as means to efficiently iterate on AI models, allowing researchers to validate and adjust their approaches based on real-time feedback.
Overall, the event provided a comprehensive overview of the diverse applications of AI in biomedical research, its current capabilities, limitations, and future prospects in biomedicine, underscoring the importance of interdisciplinary collaboration and cautious optimism in harnessing AI’s potential for transformative advancements in biomedical research (4).
In reviewing this story for publication, the panel of experts agree that the AI-generated summary is a decent example of the potential use of these kinds of tools in our daily workflows. However, they note that while ChatGPT produced a sufficient summary, there are still omissions, mild errors, and an over-generalization of the event.
“It does adequately cover the main beats of the event, but it loses what made our event engaging and our research at Morgridge unique,” says Gitter. “GPT summaries tend to do that.”
He suggests that a more powerful large language model may be more accurate in picking up on the nuance of the discussion, but we would need the time and resources to test multiple models to make that conclusion. Stewart and Caicedo agree that the summary, while mostly accurate, it is somewhat vague, and more thorough editing would be necessary to better explain technical details and provide relevant context.
Some of the inconsistencies of the AI-generated text are footnoted here:
- Gitter explains that the intent of this anecdote was as a rhetorical device, and most of the details were fiction.
- “Web data analysis” is a vague term — does it mean analyzing data via a web application, or analyzing data downloaded from the web? This topic was not fully discussed in the session, and the speakers mentioned other applications besides natural language processing and GPT.
- While inaccuracies are a valid concern, it’s mostly in regards to large language models. It is an over-generalization to say that all AI models produce fabricated results.
- The speakers provided brief examples of the possibilities of AI in biomedicine, but it was far from a “comprehensive overview” — there are many more examples and much more to discover and learn about the promises and pitfalls of these tools.