04_Separating_Data_and_Instructions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 4: Separating Data and Instructions\n",
    "\n",
    "- [Lesson](#lesson)\n",
    "- [Exercises](#exercises)\n",
    "- [Example Playground](#example-playground)\n",
    "\n",
    "## Setup\n",
    "\n",
    "Run the following setup cell to load your API key and establish the `get_completion` helper function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import python's built-in regular expression library\n",
    "import re\n",
    "import boto3\n",
    "from botocore.exceptions import ClientError\n",
    "import json\n",
    "\n",
    "# Import the hints module from the utils package\n",
    "from utils import hints\n",
    "\n",
    "# Retrieve the MODEL_NAME variable from the IPython store\n",
    "%store -r modelId\n",
    "%store -r region\n",
    "\n",
    "bedrock_client = boto3.client(service_name='bedrock-runtime', region_name=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(prompt, system_prompt=None):\n",
    "    inference_config = {\n",
    "        \"temperature\": 0.0,\n",
    "        \"maxTokens\": 200\n",
    "    }\n",
    "    converse_api_params = {\n",
    "        \"modelId\": modelId,\n",
    "        \"messages\": [{\"role\": \"user\", \"content\": [{\"text\": prompt}]}],\n",
    "        \"inferenceConfig\": inference_config\n",
    "    }\n",
    "    if system_prompt:\n",
    "        converse_api_params[\"system\"] = [{\"text\": system_prompt}]\n",
    "    try:\n",
    "        response = bedrock_client.converse(**converse_api_params)\n",
    "        text_content = response['output']['message']['content'][0]['text']\n",
    "        return text_content\n",
    "\n",
    "    except ClientError as err:\n",
    "        message = err.response['Error']['Message']\n",
    "        print(f\"A client error occured: {message}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Lesson\n",
    "\n",
    "Oftentimes, you don't want to write full prompts, but instead want **prompt templates that can be modified later with additional input data before submitting to Claude**. This might come in handy if you want Claude to do the same thing every time, but the data that Claude uses for its task might be different each time. \n",
    "\n",
    "Luckily, you can do this pretty easily by **separating the fixed skeleton of the prompt from variable user input, then substituting the user input into the prompt** before sending the full prompt to Claude. \n",
    "\n",
    "Below, you'll walk step by step through how to write a substitutable prompt template, as well as how to substitute in user input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "In this first example, you're asking Claude to act as an animal noise generator. Notice that the full prompt submitted to Claude is just the `PROMPT_TEMPLATE` substituted with the input (in this case, \"Cow\"). Notice that the word \"Cow\" replaces the `ANIMAL` placeholder via an f-string when you print out the full prompt.\n",
    "\n",
    "**Note:** You don't have to call your placeholder variable anything in particular in practice. You called it `ANIMAL` in this example, but just as easily, you could have called it `CREATURE` or `A` (although it's generally good to have your variable names be specific and relevant so that your prompt template is easy to understand even without the substitution, just for user parseability). Just make sure that whatever you name your variable is what you use for the prompt template f-string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cow\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why would you want to separate and substitute inputs like this? Well, **prompt templates simplify repetitive tasks**. Let's say you build a prompt structure that invites third party users to submit content to the prompt (in this case the animal whose sound they want to generate). These third party users don't have to write or even see the full prompt. All they have to do is fill in variables.\n",
    "\n",
    "You do this substitution here using variables and f-strings, but you can also do it with the format() method.\n",
    "\n",
    "**Note:** Prompt templates can have as many variables as desired!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When introducing substitution variables like this, it is very important to **make sure Claude knows where variables start and end** (vs. instructions or task descriptions). Let's look at an example where there is no separation between the instructions and the substitution variable.\n",
    "\n",
    "To our human eyes, it is very clear where the variable begins and ends in the prompt template below. However, in the fully substituted prompt, that delineation becomes unclear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, **Claude thinks \"Yo Claude\" is part of the email it's supposed to rewrite**! You can tell because it begins its rewrite with \"Dear Claude\". To the human eye, it's clear, particularly in the prompt template where the email begins and ends, but it becomes much less clear in the prompt after substitution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How do you solve this? **Wrap the input in XML tags**! We did this below, and as you can see, there's no more \"Dear Claude\" in the output.\n",
    "\n",
    "[XML tags](https://docs.anthropic.com/claude/docs/use-xml-tags) are angle-bracket tags like `<tag></tag>`. They come in pairs and consist of an opening tag, such as `<tag>`, and a closing tag marked by a `/`, such as `</tag>`. XML tags are used to wrap around content, like this: `<tag>content</tag>`.\n",
    "\n",
    "**Note:** While Claude can recognize and work with a wide range of separators and delimeters, we recommend that you **use specifically XML tags as separators** for Claude, as Claude was trained specifically to recognize XML tags as a prompt organizing mechanism. Outside of function calling, **there are no special sauce XML tags that Claude has been trained on that you should use to maximally boost your performance**. Anthropic has purposefully made Claude very malleable and customizable this way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see another example of how XML tags can help us. \n",
    "\n",
    "In the following prompt, **Claude incorrectly interprets what part of the prompt is the instruction vs. the input**. It incorrectly considers `Each is about an animal, like rabbits` to be part of the list due to the formatting, when the user (the one filling out the `SENTENCES` variable) presumably did not want that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "SENTENCES = \"\"\"- I like how cows sound\n",
    "- This sentence is about spiders\n",
    "- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
    "\n",
    "- Each is about an animal, like rabbits.\n",
    "{SENTENCES}\"\"\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fix this, you just need to **surround the user input sentences in XML tags**. This shows Claude where the input data begins and ends despite the misleading hyphen before `Each is about an animal, like rabbits.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "SENTENCES = \"\"\"- I like how cows sound\n",
    "- This sentence is about spiders\n",
    "- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
    "\n",
    "- Each is about an animal, like rabbits.\n",
    "<sentences>\n",
    "{SENTENCES}\n",
    "</sentences>\"\"\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** In the incorrect version of the \"Each is about an animal\" prompt, you had to include the hyphen to get Claude to respond incorrectly in the way you wanted to for this example. This is an important lesson about prompting: **small details matter**! It's always worth it to **scrub your prompts for typos and grammatical errors**. Claude is sensitive to patterns (in its early years, before finetuning, it was a raw text-prediction tool), and it's more likely to make mistakes when you make mistakes, smarter when you sound smart, sillier when you sound silly, and so on.\n",
    "\n",
    "If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Exercises\n",
    "- [Exercise 4.1 - Haiku Topic](#exercise-41---haiku-topic)\n",
    "- [Exercise 4.2 - Dog Question with Typos](#exercise-42---dog-question-with-typos)\n",
    "- [Exercise 4.3 - Dog Question Part 2](#exercise-42---dog-question-part-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.1 - Haiku Topic\n",
    "Modify the `PROMPT` so that it's a template that will take in a variable called `TOPIC` and output a haiku about the topic. This exercise is just meant to test your understanding of the variable templating structure with f-strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "TOPIC = \"Pigs\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Create a haiku about pigs\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(\"pigs\", text.lower()) and re.search(\"haiku\", text.lower()))\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_4_1_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.2 - Dog Question with Typos\n",
    "Fix the `PROMPT` by adding XML tags so that Claude produces the right answer. \n",
    "\n",
    "Try not to change anything else about the prompt. The messy and mistake-ridden writing is intentional, so you can see how Claude reacts to such mistakes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "QUESTION = \"ar cn brown?\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(\"brown\", text.lower()))\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_4_2_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.3 - Dog Question Part 2\n",
    "Fix the `PROMPT` **WITHOUT** adding XML tags. Instead, remove only one or two words from the prompt.\n",
    "\n",
    "Just as with the above exercises, try not to change anything else about the prompt. This will show you what kind of language Claude can parse and understand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "QUESTION = \"ar cn brown?\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(\"brown\", text.lower()))\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_4_3_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Congrats!\n",
    "\n",
    "If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Example Playground\n",
    "\n",
    "This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cow\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "SENTENCES = \"\"\"- I like how cows sound\n",
    "- This sentence is about spiders\n",
    "- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
    "\n",
    "- Each is about an animal, like rabbits.\n",
    "{SENTENCES}\"\"\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Variable content\n",
    "SENTENCES = \"\"\"- I like how cows sound\n",
    "- This sentence is about spiders\n",
    "- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
    "\n",
    "- Each is about an animal, like rabbits.\n",
    "<sentences>\n",
    "{SENTENCES}\n",
    "</sentences>\"\"\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "",
   "name": ""
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}