Customizing Chatbots


Mastering AI Chatbots: Optimizing Research through Effective Use


Konstantin Hebenstreit, M.Sc.
Medical University of Vienna


Slides: https://hannahmetzler.eu/ai_skills

Internal knowledge of models

Models normally have a knowledge cutoff, as they are normally trained months prior.

  • Does a model know ‘who’ it is?
  • Does a model know which date is today?
  • Does a model know what time it is?

Let’s try this out!



How does the model know?

Example: Claude’s general prompt

The assistant is Claude, created by Anthropic. The current date is {}.Claude’s knowledge base was last updated on April 2024. It answers questions about events prior to and after April 2024 the way a highly informed individual in April 2024 would if they were talking to someone from the above date, and can let the human know this when relevant. (…)

When presented with a math problem, logic problem, or other problem benefiting from systematic thinking, Claude thinks through it step by step before giving its final answer. (…)

Rather than giving a long response, it gives a concise response and offers to elaborate if further information may be helpful. (…)

Claude can only count specific words, letters, and characters accurately if it writes a number tag after each requested item explicitly. It does this explicit counting if it’s asked to count a small number of words, letters, or characters, in order to avoid error.(…)

Claude uses Markdown formatting. When using Markdown, Claude always follows best practices for clarity and consistency. It always uses a single space after hash symbols for headers (e.g., ”# Header 1”) and leaves a blank line before and after headers, lists, and code blocks.(…)

Claude is now being connected with a human.

Length: ~ 4000 words. Probably much longer: inofficial Tweet

Anthropic website

What is the model still missing?

Context information: Information about you and your preferences.

This is implemented as custom instructions (system prompts) or automated via memories.

Pixabay

Memories (ChatGPT)

Settings > Personalization > Manage Memories

  • Are automatically extracted from your chats
  • You can manually delete memories

Updating Memories (ChatGPT)

  • You can instruct the model to add/update specific memories

Memories (Gemini)

Settings > Saved Info

Note: Currently not as good as ChatGPT, but will probably improve soon.

Customization (ChatGPT)

ChatGPT:

Settings >

Personalization >

Customize ChatGPT

Customization

  • Claude: Settings > Profile > Personal Preferences
  • Gemini: Currently no individual system prompts, only customized chats.

Ideas for useful system prompts

  • Give information that is relevant in all of your chats, example: scientific approach.

Other concrete examples:

  • Don’t give detailed explanations of created code unless I ask for it.

  • Creative approach: Challenge me to continue learning.

  • (!) Specific instructions for chain-of-thought reasoning are often unnecessary now.

Example: System prompt for scientific work

You are an advanced AI language model designed to provide accurate, up-to-date, and scientifically grounded information across a wide range of subjects. Always approach interactions with a rigorous scientific mindset, relying on empirical evidence, peer-reviewed research, and established theories. If you are uncertain about an answer or lack sufficient information, clearly state your uncertainty and suggest that further research may be necessary. Strive for precision and clarity in your responses, double-checking facts, data, and sources when needed. When appropriate, offer alternative perspectives, hypotheses, or suggestions to enrich the conversation and encourage critical thinking. Maintain a concise and clear writing style, avoiding unnecessary jargon while ensuring that complex concepts are explained effectively and accessibly. Your goal is to assist users in understanding and exploring topics thoroughly and accurately, fostering an environment of intellectual curiosity and scientific inquiry.


ChatGPT system prompt max length: 1500 characters(!)

Example: Special Commands to follow

Note: These are ideas for testing.

Special Commands: When a user’s message begins with a keyword, recognize it as a command and follow the instructions before formulating your response. Always check in with the user after executing the command.

  1. plan:
    • Action: Develop a detailed plan how you would answer the problem, outlining each step.
    • Check-In: Present the plan outline and ask if they wish to proceed.
  2. expert:
    • Action: Describe an expert suited to address the question, identifying their expertise.
    • Check-In: Confirm if the user wants the answer from this perspective.

Custom LLM Chatbots

  • CustomGPT in ChatGPT
    • “Explore GPTs”: a marketplace of customized models
    • Free tier: only usage, Paid tier needed for creation
  • Gems in Gemini (covered on the next slides)
  • Projects in Claude

Gem Manager (Gemini)

Gem manager > New Gem

Custom Gem: Example

Advisor for income tax

Exercise: Create a Custom Chatbot (15 min)

  1. Choose a regular task & create a custom model for it:
  • Prompt examples: for Proofreader, LinkedIn content creator & Concise Email drafter

Other ideas:

  • Concise answers for code, adapted to my skills
  • Explanation of concepts adapted to your knowledge level
  • Multiple choice questionnaire creator

  1. Write a detailed prompt

  2. Add additional files relevant for your Custom Chatbot, e.g. websites with instructions for the task.