run_phi4_in_colab_using_R.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyPJx2c9rRni7vYBErq78daa",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "ir",
      "display_name": "R"
    },
    "language_info": {
      "name": "R"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/textspur/prompting_open_llm/blob/main/run_phi4_in_colab_using_R.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Running Phi-4 and other LLMs in R using Google Colab for free through the rollama R-Package"
      ],
      "metadata": {
        "id": "G59XJkiIYOZD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Local Large Language Models (LLMs) are becoming increasingly important for researchers and practitioners. This tutorial will show you how to run Microsoft’s Phi-4 model using R, Ollama, and the rollama package — all for free using Google Colab!\n",
        "\n",
        "Why Colab?\n",
        "- Provides easy GPU access\n",
        "- No local installation required"
      ],
      "metadata": {
        "id": "9zas-rA3YT3q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Setting Up Ollama in Colab**\n",
        "\n",
        "Since we’re in Colab, we need to install Ollama in our temporary environment. This can be achieved with a few commands:"
      ],
      "metadata": {
        "id": "PQalWm2e_-PX"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SGU1MHWltyD2",
        "outputId": "1a12cfc5-a30b-4d77-98ec-8dd9f09a044e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "WARNING: systemd is not running\n",
            "WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies.\n"
          ]
        }
      ],
      "source": [
        "# Run the Ollama Linux install script\n",
        "output <- system(\"curl -fsSL https://ollama.com/install.sh | sh\", intern = TRUE)\n",
        "cat(output, sep = \"\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Start Ollama on the machine"
      ],
      "metadata": {
        "id": "pjQxJA9fDaJC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Start the Ollama service\n",
        "system(\"ollama serve\", wait = FALSE)"
      ],
      "metadata": {
        "id": "_ccfnGz5qH3n"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check if Ollama is running:"
      ],
      "metadata": {
        "id": "DWIp3K9yDeXY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Verify Ollama is running\n",
        "output <- system(\"curl http://localhost:11434\", intern = TRUE)\n",
        "cat(output, sep = \"\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pmYRc_qFriDF",
        "outputId": "70aa4111-60a3-49bd-f0ae-911ffb1ee510"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in system(\"curl http://localhost:11434\", intern = TRUE):\n",
            "“running command 'curl http://localhost:11434' had status 7”\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "If the previous code chunk resulted in \"Ollama is running\" we are good to go."
      ],
      "metadata": {
        "id": "b8E5EEB_DmxK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Installing and Setting Up rollama**\n",
        "\n",
        "The rollama package provides an interface to Ollama. Learn more about the rollama R-Package https://jbgruber.github.io/rollama/"
      ],
      "metadata": {
        "id": "vYCDoG6arIWJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "install.packages(\"rollama\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q23ZM0gjpcv8",
        "outputId": "b3eeed52-0080-4f56-ae55-a9e4d4dc0136"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Installing package into ‘/usr/local/lib/R/site-library’\n",
            "(as ‘lib’ is unspecified)\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "library(rollama)"
      ],
      "metadata": {
        "id": "A0ByDp47urYn"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check if ollama is running:"
      ],
      "metadata": {
        "id": "W2oqB03tEaqx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ping_ollama()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wcpkLqx3p93R",
        "outputId": "568cec0d-9a28-4d6a-ec48-66710717c400"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[1m\u001b[22m\u001b[32m▶\u001b[39m Ollama (v0.5.4) is running at \u001b[3m\u001b[34m<http://localhost:11434>\u001b[39m\u001b[23m!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In order to use a generative language model, we need to download one. Ollama provides a list of supported models here: https://ollama.com/search   \n",
        "Pulling a model might take a few minutes."
      ],
      "metadata": {
        "id": "2_1DpAwSEjBq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pull_model(\"phi4\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vc2cZcJvu9qR",
        "outputId": "fcf523e6-063d-406e-8644-89e784893d33"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[32m✔\u001b[39m model phi4 pulled succesfully\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Basic Usage**\n",
        "\n",
        "Let’s start with a simple query to test our setup. The first query after loading the model will be slower than subsequent queries."
      ],
      "metadata": {
        "id": "VNTw1ToIy-9B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "answer <- query(\n",
        " \"Why is the sky blue? Answer with one sentence.\",\n",
        " output = \"text\",\n",
        " model = \"phi4\",\n",
        " screen = FALSE\n",
        ")\n",
        "print(answer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UcOPgkJZvNYo",
        "outputId": "6eb32d6e-195f-4e4d-daae-fa61829775fe"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1] \"The sky appears blue because molecules in Earth's atmosphere scatter shorter wavelengths of sunlight, like blue and violet, more than longer wavelengths such as red and yellow, but our eyes are more sensitive to blue light.\"\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Advanced Applications**\n",
        "\n",
        "*Text Classification*\n",
        "\n",
        "Phi-4 can be used for various NLP tasks. Here’s an example of text classification:"
      ],
      "metadata": {
        "id": "MEPQsQQOZ4kA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prompt <- \"Answer with just the correct category (one or two words). Classify this text as either 'populist' or 'not populist':\n",
        "'It's time to take back our country back from corrupt elites and put the power back where it belongs: in the hands of the people.'\"\n",
        "\n",
        "answer <- query(prompt, output = \"text\",  model= \"phi4\", screen = FALSE)\n",
        "print(answer)"
      ],
      "metadata": {
        "id": "1SnGGpU3zcfQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c736914f-c6cc-4790-a3b7-1896b883f9ed"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1] \"Populist  \\n\"\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Working with Data frames and a few-shot prompt**\n",
        "\n",
        "One of the powerful features of rollama is the ability to process entire data frames. Here’s how to annotate multiple texts:"
      ],
      "metadata": {
        "id": "-uc7FeTC-6IW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "unseen_texts <- c(\n",
        "  \"Many kids find school boring\",\n",
        "  \"Everybody should have access to good education and healthcare\"\n",
        ")\n",
        "\n",
        "df <- data.frame(text = unseen_texts)\n",
        "df\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 161
        },
        "id": "qEekUCkq-omb",
        "outputId": "c176243a-9844-4c95-b3c9-8b9c2b4a0aea"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table class=\"dataframe\">\n",
              "<caption>A data.frame: 2 × 1</caption>\n",
              "<thead>\n",
              "\t<tr><th scope=col>text</th></tr>\n",
              "\t<tr><th scope=col>&lt;chr&gt;</th></tr>\n",
              "</thead>\n",
              "<tbody>\n",
              "\t<tr><td>Many kids find school boring                                 </td></tr>\n",
              "\t<tr><td>Everybody should have access to good education and healthcare</td></tr>\n",
              "</tbody>\n",
              "</table>\n"
            ],
            "text/markdown": "\nA data.frame: 2 × 1\n\n| text &lt;chr&gt; |\n|---|\n| Many kids find school boring                                  |\n| Everybody should have access to good education and healthcare |\n\n",
            "text/latex": "A data.frame: 2 × 1\n\\begin{tabular}{l}\n text\\\\\n <chr>\\\\\n\\hline\n\t Many kids find school boring                                 \\\\\n\t Everybody should have access to good education and healthcare\\\\\n\\end{tabular}\n",
            "text/plain": [
              "  text                                                         \n",
              "1 Many kids find school boring                                 \n",
              "2 Everybody should have access to good education and healthcare"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This creates a data frame with a ‘text’ column. Next, we generate annotation queries using rollama’s make_query() function with two examples and a system prompt."
      ],
      "metadata": {
        "id": "9QBS8j5D_M6l"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create examples for few-shot learning\n",
        "annotated_examples <- tibble::tribble(\n",
        " ~text, ~answer,\n",
        " \"the pizza tastes terrible\", \"'not political'\",\n",
        " \"the state should make sure no child goes hungry\", \"'political'\"\n",
        ")\n",
        "\n",
        "# Generate queries with examples\n",
        "queries <- make_query(\n",
        " template = \"{text}\\n{prompt}\",\n",
        " text = df$text,\n",
        " prompt = \"Categories: 'political' or 'not political'\",\n",
        " system = \"You are an expert annotator who classifies texts.\n",
        " Answer with just the correct category.\",\n",
        " example = annotated_examples\n",
        ")"
      ],
      "metadata": {
        "id": "-1ULbbGvzjst"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(queries)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a7NSZFds9f_v",
        "outputId": "0191822b-9299-478a-b022-07cfcdfd5f2d"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[1]]\n",
            "\u001b[90m# A tibble: 6 × 2\u001b[39m\n",
            "  role      content                                                            \n",
            "  \u001b[3m\u001b[90m<chr>\u001b[39m\u001b[23m     \u001b[3m\u001b[90m<glue>\u001b[39m\u001b[23m                                                             \n",
            "\u001b[90m1\u001b[39m system    You are an expert annotator who classifies texts.\n",
            " Answer with just…\n",
            "\u001b[90m2\u001b[39m user      the pizza tastes terrible\n",
            "Categories: 'political' or 'not political'\n",
            "\u001b[90m3\u001b[39m assistant 'not political'                                                    \n",
            "\u001b[90m4\u001b[39m user      the state should make sure no child goes hungry\n",
            "Categories: 'politi…\n",
            "\u001b[90m5\u001b[39m assistant 'political'                                                        \n",
            "\u001b[90m6\u001b[39m user      Many kids find school boring\n",
            "Categories: 'political' or 'not politi…\n",
            "\n",
            "[[2]]\n",
            "\u001b[90m# A tibble: 6 × 2\u001b[39m\n",
            "  role      content                                                            \n",
            "  \u001b[3m\u001b[90m<chr>\u001b[39m\u001b[23m     \u001b[3m\u001b[90m<glue>\u001b[39m\u001b[23m                                                             \n",
            "\u001b[90m1\u001b[39m system    You are an expert annotator who classifies texts.\n",
            " Answer with just…\n",
            "\u001b[90m2\u001b[39m user      the pizza tastes terrible\n",
            "Categories: 'political' or 'not political'\n",
            "\u001b[90m3\u001b[39m assistant 'not political'                                                    \n",
            "\u001b[90m4\u001b[39m user      the state should make sure no child goes hungry\n",
            "Categories: 'politi…\n",
            "\u001b[90m5\u001b[39m assistant 'political'                                                        \n",
            "\u001b[90m6\u001b[39m user      Everybody should have access to good education and healthcare\n",
            "Categ…\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can execute the queries and store the results in a new column of the data frame:"
      ],
      "metadata": {
        "id": "xmrX6KRMaV2f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df$annotation <- query(\n",
        "    q = queries,\n",
        "    output = \"text\",\n",
        "    model= \"phi4\",\n",
        "    screen = FALSE\n",
        ")\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 161
        },
        "id": "7FmA_eIKzrJ2",
        "outputId": "4a046417-87d3-416d-fb9c-8977165838e6"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table class=\"dataframe\">\n",
              "<caption>A data.frame: 2 × 2</caption>\n",
              "<thead>\n",
              "\t<tr><th scope=col>text</th><th scope=col>annotation</th></tr>\n",
              "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
              "</thead>\n",
              "<tbody>\n",
              "\t<tr><td>Many kids find school boring                                 </td><td>'not political'</td></tr>\n",
              "\t<tr><td>Everybody should have access to good education and healthcare</td><td>'political'    </td></tr>\n",
              "</tbody>\n",
              "</table>\n"
            ],
            "text/markdown": "\nA data.frame: 2 × 2\n\n| text &lt;chr&gt; | annotation &lt;chr&gt; |\n|---|---|\n| Many kids find school boring                                  | 'not political' |\n| Everybody should have access to good education and healthcare | 'political'     |\n\n",
            "text/latex": "A data.frame: 2 × 2\n\\begin{tabular}{ll}\n text & annotation\\\\\n <chr> & <chr>\\\\\n\\hline\n\t Many kids find school boring                                  & 'not political'\\\\\n\t Everybody should have access to good education and healthcare & 'political'    \\\\\n\\end{tabular}\n",
            "text/plain": [
              "  text                                                          annotation     \n",
              "1 Many kids find school boring                                  'not political'\n",
              "2 Everybody should have access to good education and healthcare 'political'    "
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Remember that Colab environments are temporary. Any models you download will need to be re-downloaded in new sessions. Consider this when planning long-running tasks. Save your notebook to Google Drive to preserve your code and configurations for future use."
      ],
      "metadata": {
        "id": "D9L9KTLr_ypM"
      }
    }
  ]
}