Chatbot Essentials & Advanced Prompting Strategies

Mastering AI Chatbots: Optimizing Research through Effective Use

Mag. Dr. Hannah Metzler
Complexity Science Hub & Medical University of Vienna

Slides: https://hannahmetzler.eu/ai_skills

The Basics: How to talk to an LLM Chatbot?

Use search engines vs. LLMs

Search engine

• A few short keywords
• Answer: many single results
  
  
• Real-time information & news via internet

LLM Chatbot

• Long prompts with lots of details, context, examples & explanations
• Answer is summarised
• Answer is statistically probable: prototypical, average, generalized
• Generalizes across time, no real time-access

LLMs: not ideal for news and up to date knowledge research, better adapted for things that generalize across time (binstorming, managing projects, creating documents…) Meta-Regeln, Kontext, FRagen/Antwort schrittweise: https://www.youtube.com/watch?v=cfl7q1llkso&t=5s Minute 30 - vielleicht gut als Vereinfachung meines Prompting Teils? Tipp: neuer Chat - neuer kontext notwendig

General tips for working with LLM Chatbots

• There is no correct way of prompting that applies universally. We are all figuring this out together constantly.
• We can’t predict what will work in a particular use case. [1].
• This is the worst AI you will ever use. [2]
• Always invite AI to the table. [2]
• Use it often and try it out on many different tasks.
• Practice is all you need to get good at prompting.
• Use it for the first 80% of your tasks (co-intelligence/co-worker).

Voice conversations

• Gemini/ChatGPT App: Mac (& Windows?) & Smartphones
• Browser: Gemini, or Chrome extension Voice Up

Voice Up

• Hold SPACE (outside text input) to record, release to submit
• ESC to stop & transcribe text without submitting

Gemini/ChatGPT App

• Microphone to record, stop to transcribe

• Advanced voice mode (headphone, Gemini symbol)

Using it often: Everyday life ideas

• Recipes: Suggestions based on existing ingredients
• Shopping lists: vegetable curry, weekly meal plan, workshop
• Concert suggestions
• Personalized travel planning
• Household: Stain or smell removal, houseplant care
• Ideas for presents
• Mindfulness meditation guide
• Basic medical advice (Ibuprofen vs. Paracetamol)
• Summarizing a long law document on chicken welfare
• Estimating prices (me selling my old cello)
• Personalized fitness plan
• Finding a web hosting service

all links only work if I am logged in??/

5 minutes exercise

• Pick one of the ideas, and try it out.

• Experiment with using voice control.

Effective prompting

Effective prompting in a nutshell

Treat AI just like an infinitely patient new co-worker who forgets everything you tell them each new conversation. (Ethan Mollick)

1. Co-worker: Learn what it can do well by using it in areas of your expertise.
2. New on the job: Be specific & clear about what you want.
3. Forgetful: Provide detailed background information every time.
4. Patient: Ask for many options & select those you like. Abundance!

Vereinfacht: https://www.youtube.com/watch?v=cfl7q1llkso&t=5s bei Minute 33

Kontext, Frage, Format, Beispiele, Rollen, Zielgruppe, Kanal. Constraints vielleicht entfernen?

1) Treat it like a co-worker.

• LLMs behave more like people than software/machines.
• Work with it, don’t give it orders.
• You are the expert: Start by using it in areas of your expertise.
• You’ll learn where hallucinations are a big deal, and where they are not over time.
• Give feedback & engage in dialogue. Performance improves very quickly.
• Work on your prompts until you get useful output

2) New on the job: Clear instructions

You don’t want a report on the pros and cons in remote learning, you want a report on the pros and cons in remote learning appropriate for a regional university in the Midwestern US and that might convince a business school Dean to fund a new remote learning program. (E. Mollick)

2) Clear & specific instructions

• Give step-by-step instructions
• Provide examples
• Give feedback for improvement (dialogue)
• Include constraints
• Specify tone & style
• Specify output format

On a slide: include an example on specific & step-by-step instructions here. Use something where the LLM can create a list, then choose the best. Like E. Mollick here: https://www.oneusefulthing.org/p/getting-started-with-ai-good-enough, in the figure after “working with AI ia a dialogue, not an order.” e.g.: “Write an abstract of this paper” “Write an abstract of this paper for the journal Nature human behavior. Focus on the robust methods, the causal design, the novelty and implications of results. Here are instructions on how to write a good abstract: …

Give good or bad examples

• An example of the output you want (or do not want) the model to produce.
• Zero-shot vs. one-shot learning

What you could use:

• Emails
• Abstract
• Social media posts/thread and paper

• Presentation slides
• Paragraph from paper
• Previous recommendation/ application/ cover letters

Style and Output format

Style, tone, modality

• formal vs. easy to understand
• caring, professional, bold (more examples)
• style of a famous person, researcher, book
• Style of your text example

Output format

• length (300 words)
• structure (e.g., bullet points, paragraphs)
• Markdown, JSON, csv, coding language
• list of steps
• a table (Markdown format is handy for copy pasting)

Constraints or whitelist

• What you don’t want the model to do.
• The task you want your model to stick to.

Example constraints

• “Rely strictly on the provided text, without including external information.” (summarizing text)
• “Answer only questions about topic X.” (for a conversational chatbot)

3) Forgetful: Provide detailed context

• Purpose, background, specific details
• Copy/paste easily available information (documents, instruction manuals, previous conversations, emails…)
• Use a role/persona (when it’s useful, will not always help):
• As a …, You are a …, Act as a … (PhD student, biologist, journalist, empathetic coach, prompt engineer…)
• Audience expertise level: Explain it like to a … (5-year old, someone with a PhD in biology,…)

4) Patience & abundance

• Ask for several options and select one you like.
• 15 sentences, 30 ideas, 3 abstracts
• Push for:
• variation (“give me ideas that are 80% weirder”),
• recombination (“combine ideas 12 and 16”) and
• expansion (“more ideas like number 12”)

Example from during course preparation

• ChatGPT conversation
• Task: Create exercise to practice prompting
• Context: skill training, my previous slides
• Audience: you
• Provided example exercise: Writing an email (see here)
• Output: 3 Quarto slides with one exercise each

Managing expectations

• Usually not all of these details in every prompt
• Talk “into it”, iterate and add necessary aspects required to improve the answer
• Prompting is not hard to learn - there is no perfection.
• (Probably) only a useful short-term skill:
• LLMs already got very good at prompting themselves (reasoning)
• LLMs are getting integrated into software/interfaces

Exercise Options 1 & 2

Practice instructing your new, infinitely patient & forgetful co-worker.

1. Translate a paper into a general audience presentation.
   • Context: Paper, audience, focus…
2. Do a review of a/your paper.
   • Context/clarity: reviewer guidelines (general, psychology)
   • Good reviews you have done/received as examples
   • Inspiration for a prompt to improve text

Exercise Option 3

3. Write the discussion of a finished paper.
   • Context: Intro, Methods, Results
   • Experiment with different writing instructions (see here)
   • Experiment with letting the model suggest different outlines as bullet points, improve those, then let the model write full sentences.
   • Write one paragraph after another. Improve. Add each paragraph to the initial prompt with entire paper once you like it.

Example for writing style instructions

Example of an Writing Style Guide:

1. Structure and Formatting

• Use simple, clear language focused on one main idea per sentence
• Employ short paragraphs (2-4 sentences) to enhance readability
• Begin with a concise, attention-grabbing title and subtitle
• Use headings and subheadings to organize content logically
• Incorporate formatting elements (e.g., bold text, italics) sparingly for emphasis
• Use bullet points or numbered lists for key takeaways or steps

2. Content Organization

• Start with the most important information (inverted pyramid style)
• Ensure each paragraph focuses on a single main idea
• Organize content into clear sections, each building upon the previous one
• Include occasional single-sentence paragraphs for impact

3. Flow and Engagement

• Open with a compelling hook to capture reader interest
• Pose thought-provoking questions to involve the reader
• Maintain a strong flow of ideas throughout the text
• Use transitional phrases to connect ideas and paragraphs
• Create and use strong “concept handles” (catchy phrases that sum up complex ideas)
• Use analogies and metaphors sparingly to explain complex concepts
• Incorporate relevant real or hypothetical scenarios to illustrate points

4. Depth and Conciseness

• Provide depth on key points without becoming verbose
• Break down complex ideas into digestible chunks
• Use precise language to convey maximum information in minimal words
• Begin with concrete examples before introducing abstract principles
• Use multiple, diverse examples to illustrate complex points
• Apply introduced ideas to real-world scenarios to demonstrate their relevance

5. Persuasion and Balance

• Present a balanced view by acknowledging multiple perspectives
• Address potential counterarguments proactively
• Use appropriate language to connect with your target audience

6. Technical and Scientific Writing

• Explain technical concepts in accessible language
• Provide context for why the topic matters to the reader or the broader field

7. Tone and Voice

• Employ active voice and direct language
• Maintain a professional yet accessible tone
• Avoid being overly academic or jargon-heavy
• Use an engaging and slightly conversational style to create a sense of dialogue
• Present ideas thoughtfully, showing respect for the reader’s intelligence
• Convey complex ideas efficiently without sacrificing depth
• Maintain a forward-looking and slightly enthusiastic tone, especially when discussing potential developments
• Balance optimism about possibilities with acknowledgment of challenges
• Aim to inform and engage without being pedantic or oversimplifying

Advanced prompting strategies

Using section headers

• Use section headers
• Markdown format

# Role

You are a ...

# Context

- Bullet point 1
- Bullet point 2

# Task

1. Do X
2. Do Y

# Examples

## Example 1

Text

## Example 2

Text

Markdown

• Markdown guide and cheatsheet
• Easy to use text-to-html conversion tool
• Copying from Chatbot to Google docs without loosing the formatting

Basic Markdown:

Headings	# for H1, ## for H2
Text format	**bold **, *italic*
Lists	1. Item (ordered) - Item (unordered)
Tables	| Column 1 | Column 2 | | ——– | ——– | | Text | Text |
Links	[Link Text](URL)
Images	
Horizontal Rule	- - -

Hallucinations

• LLMs invent facts/sources that don’t exist.
• Why?
• Statistical patterns in training data
• Predicting the next most likely word.
• Trained to give an answer, not to be truthful.
• Likely = plausible & hard to recognize
• Does not have fact knowledge or real-time access.
• Cannot reflect on its own processes

Improvements for hallucinations

• Web search possibility helps a lot.
• But the problem is built in, will continue to occur.

Strategies for verification I

• 4-eyes principle: Model does 80%, 20% human [2]
• Expertise and fact knowledge matters. Start using LLMs in your area of expertise. [1]
• Check this claim with 5 sources [2]
• Chain of Verification: Check online for 4 facts to make sure that this answer is correct [2]
• Ask for better prompts: Why am I not getting the answer I am looking for? Why do you produce false responses?
• Book example in Erklär mir die Welt

Strategies for verification II

• Write if you’re unsure or necessary information, say “I don’t have enough information to answer this”. (One useful thing)
• Enter the same prompt in different models and compare, e.g. at https://claude.ai, https://poe.com, https://anakin.ai
• Ask other models: Enter output of model 1 and ask: “Is this correct? Can you back this up with sources? Do these books exist?” (Example: ChatGPT to Claude, Claude to ChatGPT)
• Use LLM tools that provide references: Perplexity AI, Elicit, Dimensions Research AI, ScholarGPT

Exercise: Correcting hallucinations (20 min)

1. Use a prompt that could lead to hallucinations
2. Apply different verification strategies

• “Please recommend 5 easy to read non-technical books about how to use LLM chatbots for research”, books about: psychology/biology
• “Summarize the findings of studies that show that evidence on the connection between mental health and social media use is often misleading. Include a reference list at the end of your summary.”
• Quote from an expert on a specific topic: “What does Jonathan Haidt say about why evidence on social media and mental health is not conclusive?”
• Or: ask about a historical event, details from a biography, most recent statistical data on unemployment in the Maldives etc.