05_Formatting_Output_and_Speaking_for_Claude.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 5: Formatting Output and Speaking for Claude\n",
    "\n",
    "- [Lesson](#lesson)\n",
    "- [Exercises](#exercises)\n",
    "- [Example Playground](#example-playground)\n",
    "\n",
    "## Setup\n",
    "\n",
    "Run the following setup cell to load your API key and establish the `get_completion` helper function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import python's built-in regular expression library\n",
    "import re\n",
    "import boto3\n",
    "from botocore.exceptions import ClientError\n",
    "import json\n",
    "\n",
    "# Import the hints module from the utils package\n",
    "from utils import hints\n",
    "\n",
    "# Retrieve the MODEL_NAME variable from the IPython store\n",
    "%store -r modelId\n",
    "%store -r region\n",
    "\n",
    "bedrock_client = boto3.client(service_name='bedrock-runtime', region_name=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(prompt, system_prompt=None, prefill=None):\n",
    "    inference_config = {\n",
    "        \"temperature\": 0.5,\n",
    "        \"maxTokens\": 200\n",
    "    }\n",
    "    converse_api_params = {\n",
    "        \"modelId\": modelId,\n",
    "        \"messages\": [{\"role\": \"user\", \"content\": [{\"text\": prompt}]}],\n",
    "        \"inferenceConfig\": inference_config\n",
    "    }\n",
    "    if system_prompt:\n",
    "        converse_api_params[\"system\"] = [{\"text\": system_prompt}]\n",
    "    if prefill:\n",
    "        converse_api_params[\"messages\"].append({\"role\": \"assistant\", \"content\": [{\"text\": prefill}]})\n",
    "    try:\n",
    "        response = bedrock_client.converse(**converse_api_params)\n",
    "        text_content = response['output']['message']['content'][0]['text']\n",
    "        return text_content\n",
    "\n",
    "    except ClientError as err:\n",
    "        message = err.response['Error']['Message']\n",
    "        print(f\"A client error occured: {message}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Lesson\n",
    "\n",
    "**Claude can format its output in a wide variety of ways**. You just need to ask for it to do so!\n",
    "\n",
    "One of these ways is by using XML tags to separate out the response from any other superfluous text. You've already learned that you can use XML tags to make your prompt clearer and more parseable to Claude. It turns out, you can also ask Claude to **use XML tags to make its output clearer and more easily understandable** to humans."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "Remember the 'poem preamble problem' we solved in Chapter 2 by asking Claude to skip the preamble entirely? It turns out we can also achieve a similar outcome by **telling Claude to put the poem in XML tags**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Rabbit\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is this something we'd want to do? Well, having the output in **XML tags allows the end user to reliably get the poem and only the poem by writing a short program to extract the content between XML tags**.\n",
    "\n",
    "An extension of this technique is to **put the first XML tag in the `assistant` turn. When you put text in the `assistant` turn, you're basically telling Claude that Claude has already said something, and that it should continue from that point onward. This technique is called \"speaking for Claude\" or \"prefilling Claude's response.\"\n",
    "\n",
    "Below, we've done this with the first `<haiku>` XML tag. Notice how Claude continues directly from where we left off."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cat\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"<haiku>\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN:\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN:\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Claude also excels at using other output formatting styles, notably `JSON`. If you want to enforce JSON output (not deterministically, but close to it), you can also prefill Claude's response with the opening bracket, `{`}."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cat\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"{\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is an example of **multiple input variables in the same prompt AND output formatting specification, all done using XML tags**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First input variable\n",
    "EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
    "\n",
    "# Second input variable\n",
    "ADJECTIVE = \"olde english\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
    "\n",
    "# Prefill for Claude's response (now as an f-string with a variable)\n",
    "PREFILL = f\"<{ADJECTIVE}_email>\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Bonus lesson\n",
    "\n",
    "If you are calling Claude through the API, you can pass the closing XML tag to the `stop_sequences` parameter to get Claude to stop sampling once it emits your desired tag. This can save money and time-to-last-token by eliminating Claude's concluding remarks after it's already given you the answer you care about.\n",
    "\n",
    "If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Exercises\n",
    "- [Exercise 5.1 - Steph Curry GOAT](#exercise-51---steph-curry-goat)\n",
    "- [Exercise 5.2 - Two Haikus](#exercise-52---two-haikus)\n",
    "- [Exercise 5.3 - Two Haikus, Two Animals](#exercise-53---two-haikus-two-animals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.1 - Steph Curry GOAT\n",
    "Forced to make a choice, Claude designates Michael Jordan as the best basketball player of all time. Can we get Claude to pick someone else?\n",
    "\n",
    "Change the `PREFILL` variable to **compell Claude to make a detailed argument that the best basketball player of all time is Stephen Curry**. Try not to change anything except `PREFILL` as that is the focus of this exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Who is the best basketball player of all time? Please choose one specific player.\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT, prefill=PREFILL)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(\"Warrior\", text))\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_5_1_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.2 - Two Haikus\n",
    "Modify the `PROMPT` below using XML tags so that Claude writes two haikus about the animal instead of just one. It should be clear where one poem ends and the other begins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"\"\n",
    "\n",
    "# number of haikus\n",
    "NUMBER = \"\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write {NUMBER} haiku(s) about {ANIMAL}. Put each haiku(s) in {PREFILL} tags.\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT, prefill=PREFILL)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(\n",
    "        (re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
    "        and (text.count(\"\\n\") + 1) > 5\n",
    "    )\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_5_2_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.3 - Two Haikus, Two Animals\n",
    "Modify the `PROMPT` below so that **Claude produces two haikus about two different animals**. Use `{ANIMAL1}` as a stand-in for the first substitution, and `{ANIMAL2}` as a stand-in for the second substitution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First input variable\n",
    "ANIMAL1 = \"Cat\"\n",
    "\n",
    "# Second input variable\n",
    "ANIMAL2 = \"Dog\"\n",
    "\n",
    "# number of haikus\n",
    "NUMBER = \"two\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL1}. Put it in <haiku> tags.\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(\"tail\", text.lower()) and re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(response)\n",
    "print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_5_3_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Congrats!\n",
    "\n",
    "If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Example Playground\n",
    "\n",
    "This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Rabbit\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(PROMPT)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cat\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"<haiku>\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN:\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN:\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variable content\n",
    "ANIMAL = \"Cat\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
    "\n",
    "# Prefill for Claude's response\n",
    "PREFILL = \"{\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# First input variable\n",
    "EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
    "\n",
    "# Second input variable\n",
    "ADJECTIVE = \"olde english\"\n",
    "\n",
    "# Prompt template with a placeholder for the variable content\n",
    "PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
    "\n",
    "# Prefill for Claude's response (now as an f-string with a variable)\n",
    "PREFILL = f\"<{ADJECTIVE}_email>\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
    "print(\"USER TURN\")\n",
    "print(PROMPT)\n",
    "print(\"\\nASSISTANT TURN\")\n",
    "print(PREFILL)\n",
    "print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
    "print(get_completion(PROMPT, prefill=PREFILL))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "",
   "name": ""
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}