Are You Asking the Right Questions? Lessons Learned from the SIIM 2023 Large Language Model Prompt Competition

Artificial intelligence (AI) chatbots driven by large language models (LLMs), like ChatGPT, offer opportunities to improve efficiency in imaging informatics use cases. However, simple prompts for these models do not always return desired responses and poorly designed prompts may predispose to limitations associated with LLMs, such as model hallucination. Prompt engineering is the process of designing effective prompts that generate high-quality, clear, and accurate responses.

Learning about prompt engineering skills can build an understanding of the potential and limitations of LLMs, allowing for maximizing effective, safe use of LLMs. However, there is still varying degree of knowledge/proficiency with LLMs and access to the latest LLM outside the informatics/engineering community, including in high-skill sectors like medicine, making an argument for education on prompt engineering for those using the most accessible, publicly available version of GPT (GPT-3.5). The webinar will conclude with insights and concrete examples from the AI Playground prompts that we developed for SIIM23.

Winners from the SIIM 2023 AI Playground ChatGPT challenge will join the conversation for a discussion regarding their strategy in designing prompts for imaging informatics use cases.

Objectives

• Introduce the concept of prompt engineering
• Understand the potential uses and limitations of large language models (LLMs)
• Learn how to leverage prompt engineering to optimize prompt for efficient LLM use
• Review example prompts from the SIIM23 AI Playground

Resources

• Career Essentials in Generative AI Course
• Prompt Engineering for ChatGPT
• Best practices for prompt engineering with OpenAI API
• Prompt Engineering Guide
• AI Weirdness

CE Credit

SIIM has approved this activity for 1.0 hours of SIIM IIP Credits towards certification and re-certification by the American Board of Imaging Informatics (ABII).