1-Data-Cleaning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DtD8-wrt-f-0"
   },
   "source": [
    "# Data Cleaning\n",
    "\n",
    "This is a fork of https://github.com/adashofdata/nlp-in-python-tutorial.  Rather than using the transcripts of stand-up comedians, it uses the text of New Testament books of the Bible.\n",
    "\n",
    "- Created Nov 2022\n",
    "- Revised Nov 2024\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "3dDv5BlMHTQu"
   },
   "outputs": [],
   "source": [
    "# # Load the Drive helper and mount\n",
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive')\n",
    "# !pip install cloudscraper\n",
    "\n",
    "import os\n",
    "# base_dir = '/content/drive/MyDrive/ColabData/NLP_Demo/'\n",
    "base_dir = '.'\n",
    "\n",
    "try:\n",
    "    os.mkdir(base_dir)\n",
    "except FileExistsError:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qVnxnNpr-f-2"
   },
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XvozsI2Y-f-2"
   },
   "source": [
    "This notebook goes through a necessary step of any data science project - data cleaning. Data cleaning can be a time consuming and unenjoyable task, yet it's a very important one. Keep in mind, \"garbage in, garbage out\". Feeding dirty data into a model will give us results that are meaningless.\n",
    "\n",
    "Specifically, we'll be walking through:\n",
    "\n",
    "1. **Getting the data** - in this case, we'll be scraping data from a website\n",
    "2. **Cleaning the data** - we will walk through popular text pre-processing techniques\n",
    "3. **Organizing the data** - we will organize the cleaned data into a way that is easy to input into other algorithms\n",
    "\n",
    "The output of this notebook will be clean, organized data in two standard text formats:\n",
    "\n",
    "1. **Corpus** - a collection of text\n",
    "2. **Document-Term Matrix** - word counts in matrix format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DX9J6b7n-f-3"
   },
   "source": [
    "## Problem Statement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dT9N7Vs9-f-3"
   },
   "source": [
    "As a reminder, our goal is to look at text of New Testament books and note their similarities and differences."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VIiTgbgs-f-3"
   },
   "source": [
    "## Getting The Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "euR6I5_S-f-3"
   },
   "source": [
    "The text of the NET of the bible is freely available using an API at http://labs.bible.org/api/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "-1FDqm8M-f-3",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import cloudscraper  # instead of requests\n",
    "import pickle\n",
    "\n",
    "book_names = [\n",
    "    'Matthew',\n",
    "    'Mark',\n",
    "    'Luke',\n",
    "    'John',\n",
    "    'Acts',\n",
    "    'Romans',\n",
    "    '1 Corinthians',\n",
    "    '2 Corinthians',\n",
    "    'Galatians',\n",
    "    'Ephesians',\n",
    "    'Philippians',\n",
    "    'Colossians',\n",
    "    '1 Thessalonians',\n",
    "    '2 Thessalonians',\n",
    "    '1 Timothy',\n",
    "    '2 Timothy',\n",
    "    'Titus',\n",
    "    'Philemon',\n",
    "    'Hebrews',\n",
    "    'James',\n",
    "    '1 Peter',\n",
    "    '2 Peter',\n",
    "    '1 John',\n",
    "    '2 John',\n",
    "    '3 John',\n",
    "    'Jude',\n",
    "    'Revelation',\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "5i7qoeN8-f-4"
   },
   "outputs": [],
   "source": [
    "# Request book text and store as list of lists (takes a few seconds to run)\n",
    "\n",
    "def get_chapters(book):\n",
    "    scraper = cloudscraper.create_scraper(\n",
    "        browser = {'browser': 'chrome',\n",
    "                   'platform': 'linux',\n",
    "                   'desktop': True}\n",
    "    )\n",
    "\n",
    "    # Request book text and store as list of lists (takes a few seconds to run)\n",
    "    ch = 0\n",
    "    print(f'{book:>16s} ', end='')\n",
    "    chapters = []\n",
    "    while ch < 30: # no NT book has more than 28 chapters\n",
    "        ch += 1\n",
    "        prefix = f'{ch}:1'\n",
    "        url = f\"https://labs.bible.org/api/?passage={book.replace(' ', '%20')}+{ch}&formatting=plain\"\n",
    "        chapter_text = scraper.get(url).text\n",
    "        if prefix == chapter_text[:len(prefix)]:\n",
    "            print('.', end='')\n",
    "            chapters.append(chapter_text)\n",
    "        else:\n",
    "            ch -= 1\n",
    "            break\n",
    "    print(' '*(30-ch), f\"{ch:2d} chapter{'s' if ch > 1 else ''}\")\n",
    "    return chapters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "RjtoDoHv-f-4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./NET_pickled/Matthew.pkl already exists\n",
      "./NET_pickled/Mark.pkl already exists\n",
      "./NET_pickled/Luke.pkl already exists\n",
      "./NET_pickled/John.pkl already exists\n",
      "./NET_pickled/Acts.pkl already exists\n",
      "./NET_pickled/Romans.pkl already exists\n",
      "./NET_pickled/1 Corinthians.pkl already exists\n",
      "./NET_pickled/2 Corinthians.pkl already exists\n",
      "./NET_pickled/Galatians.pkl already exists\n",
      "./NET_pickled/Ephesians.pkl already exists\n",
      "./NET_pickled/Philippians.pkl already exists\n",
      "./NET_pickled/Colossians.pkl already exists\n",
      "./NET_pickled/1 Thessalonians.pkl already exists\n",
      "./NET_pickled/2 Thessalonians.pkl already exists\n",
      "./NET_pickled/1 Timothy.pkl already exists\n",
      "./NET_pickled/2 Timothy.pkl already exists\n",
      "./NET_pickled/Titus.pkl already exists\n",
      "./NET_pickled/Philemon.pkl already exists\n",
      "./NET_pickled/Hebrews.pkl already exists\n",
      "./NET_pickled/James.pkl already exists\n",
      "./NET_pickled/1 Peter.pkl already exists\n",
      "./NET_pickled/2 Peter.pkl already exists\n",
      "./NET_pickled/1 John.pkl already exists\n",
      "./NET_pickled/2 John.pkl already exists\n",
      "./NET_pickled/3 John.pkl already exists\n",
      "./NET_pickled/Jude.pkl already exists\n",
      "./NET_pickled/Revelation.pkl already exists\n"
     ]
    }
   ],
   "source": [
    "# Download books of the New Testament and pickle for later use\n",
    "import os\n",
    "pickled_dir = os.path.join(base_dir, 'NET_pickled')\n",
    "\n",
    "# Make a new directory to hold the text files\n",
    "try:\n",
    "    os.mkdir(pickled_dir)\n",
    "except FileExistsError:\n",
    "    pass# Load the Drive helper and mount\n",
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive')\n",
    "\n",
    "\n",
    "for i, book in enumerate(book_names):\n",
    "    filename = f'{pickled_dir}/{book}.pkl'\n",
    "    if not os.path.exists(filename):\n",
    "        chapters = get_chapters(book)\n",
    "        print(f'Creating {filename}')\n",
    "        with open(filename, 'wb') as file:\n",
    "            pickle.dump(chapters, file)\n",
    "    else:\n",
    "        print(f'{filename} already exists')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "zaUHPW1Z-f-4",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load pickled files\n",
    "pickled_dir = os.path.join(base_dir, 'NET_pickled')\n",
    "data = {}\n",
    "for i, b in enumerate(book_names):\n",
    "    filename = f'{pickled_dir}/{b}.pkl'\n",
    "    with open(filename, 'rb') as file:\n",
    "        data[b] = pickle.load(file)\n",
    "\n",
    "book_chapters = [len(data[b]) for b in book_names]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "b15EypKU-f-4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['Matthew', 'Mark', 'Luke', 'John', 'Acts', 'Romans', '1 Corinthians', '2 Corinthians', 'Galatians', 'Ephesians', 'Philippians', 'Colossians', '1 Thessalonians', '2 Thessalonians', '1 Timothy', '2 Timothy', 'Titus', 'Philemon', 'Hebrews', 'James', '1 Peter', '2 Peter', '1 John', '2 John', '3 John', 'Jude', 'Revelation'])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Double check to make sure data has been loaded properly\n",
    "data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "BAC2KFIK-f-4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The book of John has 21 chapters\n",
    "len(data['John'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "f85nm19I-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the father of Jacob, Jacob the father of Judah and his brothers,  3 Judah the father of Perez and Zerah (by Tamar), Perez the father of Hezron, Hezron the father of Ram,  4 Ram the father of Amminadab, Amminadab the father of Nahshon, Nahshon the father of Salmon,  5 Salmon the father of Boaz (by Rahab), Boaz the father of Obed (by Ruth), Obed the father of Jesse,  6 and Jesse the father of David the king.David was the father of Solomon (by the wife of Uriah), 7 Solomon the father of Rehoboam, Rehoboam the father of Abijah, Abijah the father of Asa,  8 Asa the father of Jehoshaphat, Jehoshaphat the father of Joram, Joram the father of Uzziah,  9 Uzziah the father of Jotham, Jotham the father of Ahaz, Ahaz the father of Hezekiah,  10 Hezekiah the father of Manasseh, Manasseh the father of Amon, Amon the father of Josiah,  11 and Josiah the father of Jeconiah and his brothers, at the time of the deportation to Babylon.  12 After the deportation to Babylon, Jeconiah became the father of Shealtiel, Shealtiel the father of Zerubbabel,  13 Zerubbabel the father of Abiud, Abiud the father of Eliakim, Eliakim the father of Azor,  14 Azor the father of Zadok, Zadok the father of Achim, Achim the father of Eliud,  15 Eliud the father of Eleazar, Eleazar the father of Matthan, Matthan the father of Jacob,  16 and Jacob the father of Joseph, the husband of Mary, by whom Jesus was born, who is called Christ.  17 So all the generations from Abraham to David are fourteen generations, and from David to the deportation to Babylon, fourteen generations, and from the deportation to Babylon to Christ, fourteen generations.  18 Now the birth of Jesus Christ happened this way. While his mother Mary was engaged to Joseph, but before they came together, she was found to be pregnant through the Holy Spirit.  19 Because Joseph, her husband to be, was a righteous man, and because he did not want to disgrace her, he intended to divorce her privately.  20 When he had contemplated this, an angel of the Lord appeared to him in a dream and said, “Joseph, son of David, do not be afraid to take Mary as your wife because the child conceived in her is from the Holy Spirit.  21 She will give birth to a son and you will name him Jesus because he will save his people from their sins.”  22 This all happened so that what was spoken by the Lord through the prophet would be fulfilled:  23 “Look! The virgin will conceive and give birth to a son, and they will name him Emmanuel,” which means “God with us.”  24 When Joseph awoke from sleep he did what the angel of the Lord told him. He took his wife,  25 but did not have marital relations with her until she gave birth to a son, whom he named Jesus.  ',\n",
       " '2:1 After Jesus was born in Bethlehem in Judea, in the time of King Herod, wise men from the East came to Jerusalem  2 saying, “Where is the one who is born king of the Jews? For we saw his star when it rose and have come to worship him.”  3 When King Herod heard this he was alarmed, and all Jerusalem with him.  4 After assembling all the chief priests and experts in the law, he asked them where the Christ was to be born.  5 “In Bethlehem of Judea,” they said, “for it is written this way by the prophet:  6 ‘And you, Bethlehem, in the land of Judah,are in no way least among the rulers of Judah,for out of you will come a ruler who will shepherd my people Israel.’”  7 Then Herod privately summoned the wise men and determined from them when the star had appeared.  8 He sent them to Bethlehem and said, “Go and look carefully for the child. When you find him, inform me so that I can go and worship him as well.” 9 After listening to the king they left, and once again the star they saw when it rose led them until it stopped above the place where the child was.  10 When they saw the star they shouted joyfully. 11 As they came into the house and saw the child with Mary his mother, they bowed down and worshiped him. They opened their treasure boxes and gave him gifts of gold, frankincense, and myrrh.  12 After being warned in a dream not to return to Herod, they went back by another route to their own country.  13 After they had gone, an angel of the Lord appeared to Joseph in a dream and said, “Get up, take the child and his mother and flee to Egypt, and stay there until I tell you, for Herod is going to look for the child to kill him.”  14 Then he got up, took the child and his mother during the night, and went to Egypt.  15 He stayed there until Herod died. In this way what was spoken by the Lord through the prophet was fulfilled: “I called my Son out of Egypt.”  16 When Herod saw that he had been tricked by the wise men, he became enraged. He sent men to kill all the children in Bethlehem and throughout the surrounding region from the age of two and under, according to the time he had learned from the wise men.  17 Then what was spoken by Jeremiah the prophet was fulfilled:  18 “A voice was heard in Ramah, weeping and loud wailing, Rachel weeping for her children, and she did not want to be comforted, because they were gone.”  19 After Herod had died, an angel of the Lord appeared in a dream to Joseph in Egypt  20 saying, “Get up, take the child and his mother, and go to the land of Israel, for those who were seeking the child’s life are dead.”  21 So he got up and took the child and his mother and returned to the land of Israel.  22 But when he heard that Archelaus was reigning over Judea in place of his father Herod, he was afraid to go there. After being warned in a dream, he went to the regions of Galilee.  23 He came to a town called Nazareth and lived there. Then what had been spoken by the prophets was fulfilled, that Jesus would be called a Nazarene.  ']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# More checks\n",
    "data['Matthew'][:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7188AiW8-f-5"
   },
   "source": [
    "## Cleaning The Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_8JF7vFY-f-5"
   },
   "source": [
    "When dealing with numerical data, data cleaning often involves removing null values and duplicate data, dealing with outliers, etc. With text data, there are some common data cleaning techniques, which are also known as text pre-processing techniques.\n",
    "\n",
    "With text data, this cleaning process can go on forever. There's always an exception to every cleaning step. So, we're going to follow the MVP (minimum viable product) approach - start simple and iterate. Here are a bunch of things you can do to clean your data. We're going to execute just the common cleaning steps here and the rest can be done at a later point to improve our results.\n",
    "\n",
    "**Common data cleaning steps on all text:**\n",
    "* Make text all lower case\n",
    "* Remove punctuation\n",
    "* Remove numerical values\n",
    "* Remove common non-sensical text (e.g. newline character)\n",
    "* Tokenize text\n",
    "* Remove stop words\n",
    "\n",
    "**More data cleaning steps after tokenization:**\n",
    "* Stemming / lemmatization\n",
    "* Parts of speech tagging\n",
    "* Create bi-grams or tri-grams\n",
    "* Deal with typos\n",
    "* And more..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "g9n7g18i-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Matthew'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at our data again\n",
    "next(iter(data.keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "yftrIX8E-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the father of Jacob, Jacob the father of Judah and his brothers,  3 Judah the father of Perez and Zerah (by Tamar), Perez the father of Hezron, Hezron the father of Ram,  4 Ram the father of Amminadab, Amminadab the father of Nahshon, Nahshon the father of Salmon,  5 Salmon the father of Boaz (by Rahab), Boaz the father of Obed (by Ruth), Obed the father of Jesse,  6 and Jesse the father of David the king.David was the father of Solomon (by the wife of Uriah), 7 Solomon the father of Rehoboam, Rehoboam the father of Abijah, Abijah the father of Asa,  8 Asa the father of Jehoshaphat, Jehoshaphat the father of Joram, Joram the father of Uzziah,  9 Uzziah the father of Jotham, Jotham the father of Ahaz, Ahaz the father of Hezekiah,  10 Hezekiah the father of Manasseh, Manasseh the father of Amon, Amon the father of Josiah,  11 and Josiah the father of Jeconiah and his brothers, at the time of the deportation to Babylon.  12 After the deportation to Babylon, Jeconiah became the father of Shealtiel, Shealtiel the father of Zerubbabel,  13 Zerubbabel the father of Abiud, Abiud the father of Eliakim, Eliakim the father of Azor,  14 Azor the father of Zadok, Zadok the father of Achim, Achim the father of Eliud,  15 Eliud the father of Eleazar, Eleazar the father of Matthan, Matthan the father of Jacob,  16 and Jacob the father of Joseph, the husband of Mary, by whom Jesus was born, who is called Christ.  17 So all the generations from Abraham to David are fourteen generations, and from David to the deportation to Babylon, fourteen generations, and from the deportation to Babylon to Christ, fourteen generations.  18 Now the birth of Jesus Christ happened this way. While his mother Mary was engaged to Joseph, but before they came together, she was found to be pregnant through the Holy Spirit.  19 Because Joseph, her husband to be, was a righteous man, and because he did not want to disgrace her, he intended to divorce her privately.  20 When he had contemplated this, an angel of the Lord appeared to him in a dream and said, “Joseph, son of David, do not be afraid to take Mary as your wife because the child conceived in her is from the Holy Spirit.  21 She will give birth to a son and you will name him Jesus because he will save his people from their sins.”  22 This all happened so that what was spoken by the Lord through the prophet would be fulfilled:  23 “Look! The virgin will conceive and give birth to a son, and they will name him Emmanuel,” which means “God with us.”  24 When Joseph awoke from sleep he did what the angel of the Lord told him. He took his wife,  25 but did not have marital relations with her until she gave birth to a son, whom he named Jesus.  ',\n",
       " '2:1 After Jesus was born in Bethlehem in Judea, in the time of King Herod, wise men from the East came to Jerusalem  2 saying, “Where is the one who is born king of the Jews? For we saw his star when it rose and have come to worship him.”  3 When King Herod heard this he was alarmed, and all Jerusalem with him.  4 After assembling all the chief priests and experts in the law, he asked them where the Christ was to be born.  5 “In Bethlehem of Judea,” they said, “for it is written this way by the prophet:  6 ‘And you, Bethlehem, in the land of Judah,are in no way least among the rulers of Judah,for out of you will come a ruler who will shepherd my people Israel.’”  7 Then Herod privately summoned the wise men and determined from them when the star had appeared.  8 He sent them to Bethlehem and said, “Go and look carefully for the child. When you find him, inform me so that I can go and worship him as well.” 9 After listening to the king they left, and once again the star they saw when it rose led them until it stopped above the place where the child was.  10 When they saw the star they shouted joyfully. 11 As they came into the house and saw the child with Mary his mother, they bowed down and worshiped him. They opened their treasure boxes and gave him gifts of gold, frankincense, and myrrh.  12 After being warned in a dream not to return to Herod, they went back by another route to their own country.  13 After they had gone, an angel of the Lord appeared to Joseph in a dream and said, “Get up, take the child and his mother and flee to Egypt, and stay there until I tell you, for Herod is going to look for the child to kill him.”  14 Then he got up, took the child and his mother during the night, and went to Egypt.  15 He stayed there until Herod died. In this way what was spoken by the Lord through the prophet was fulfilled: “I called my Son out of Egypt.”  16 When Herod saw that he had been tricked by the wise men, he became enraged. He sent men to kill all the children in Bethlehem and throughout the surrounding region from the age of two and under, according to the time he had learned from the wise men.  17 Then what was spoken by Jeremiah the prophet was fulfilled:  18 “A voice was heard in Ramah, weeping and loud wailing, Rachel weeping for her children, and she did not want to be comforted, because they were gone.”  19 After Herod had died, an angel of the Lord appeared in a dream to Joseph in Egypt  20 saying, “Get up, take the child and his mother, and go to the land of Israel, for those who were seeking the child’s life are dead.”  21 So he got up and took the child and his mother and returned to the land of Israel.  22 But when he heard that Archelaus was reigning over Judea in place of his father Herod, he was afraid to go there. After being warned in a dream, he went to the regions of Galilee.  23 He came to a town called Nazareth and lived there. Then what had been spoken by the prophets was fulfilled, that Jesus would be called a Nazarene.  ',\n",
       " '3:1 In those days John the Baptist came into the wilderness of Judea proclaiming,  2 “Repent, for the kingdom of heaven is near.”  3 For he is the one about whom the prophet Isaiah had spoken: “The voice of one shouting in the wilderness, ‘Prepare the way for the Lord, make his paths straight.’”  4 Now John wore clothing made from camel’s hair with a leather belt around his waist, and his diet consisted of locusts and wild honey.  5 Then people from Jerusalem, as well as all Judea and all the region around the Jordan, were going out to him,  6 and he was baptizing them in the Jordan River as they confessed their sins.  7 But when he saw many Pharisees and Sadducees coming to his baptism, he said to them, “You offspring of vipers! Who warned you to flee from the coming wrath?  8 Therefore produce fruit that proves your repentance,  9 and don’t think you can say to yourselves, ‘We have Abraham as our father.’ For I tell you that God can raise up children for Abraham from these stones!  10 Even now the ax is laid at the root of the trees, and every tree that does not produce good fruit will be cut down and thrown into the fire.  11 “I baptize you with water, for repentance, but the one coming after me is more powerful than I am—I am not worthy to carry his sandals! He will baptize you with the Holy Spirit and fire.  12 His winnowing fork is in his hand, and he will clean out his threshing floor and will gather his wheat into the storehouse, but the chaff he will burn up with inextinguishable fire!”  13 Then Jesus came from Galilee to John to be baptized by him in the Jordan River.  14 But John tried to prevent him, saying, “I need to be baptized by you, and yet you come to me?”  15 So Jesus replied to him, “Let it happen now, for it is right for us to fulfill all righteousness.” Then John yielded to him.  16 After Jesus was baptized, just as he was coming up out of the water, the heavens opened and he saw the Spirit of God descending like a dove and coming to rest on him.  17 And a voice from heaven said, “This is my one dear Son; in him I take great delight.”  ',\n",
       " '4:1 Then Jesus was led by the Spirit into the wilderness to be tempted by the devil.  2 After he fasted 40 days and 40 nights he was famished.  3 The tempter came and said to him, “If you are the Son of God, command these stones to become bread.”  4 But he answered, “It is written, ‘Man does not live by bread alone, but by every word that comes from the mouth of God.’”  5 Then the devil took him to the holy city, had him stand on the highest point of the temple,  6 and said to him, “If you are the Son of God, throw yourself down. For it is written, ‘He will command his angels concerning you’ and ‘with their hands they will lift you up, so that you will not strike your foot against a stone.’”  7 Jesus said to him, “Once again it is written: ‘You are not to put the Lord your God to the test.’” 8 Again, the devil took him to a very high mountain, and showed him all the kingdoms of the world and their grandeur.  9 And he said to him, “I will give you all these things if you throw yourself to the ground and worship me.”  10 Then Jesus said to him, “Go away, Satan! For it is written: ‘You are to worship the Lord your God and serve only him.’” 11 Then the devil left him, and angels came and began ministering to his needs. 12 Now when Jesus heard that John had been imprisoned, he went into Galilee.  13 While in Galilee, he moved from Nazareth to make his home in Capernaum by the sea, in the region of Zebulun and Naphtali,  14 so that what was spoken by the prophet Isaiah would be fulfilled:  15 “Land of Zebulun and land of Naphtali, the way by the sea, beyond the Jordan, Galilee of the Gentiles— 16 the people who sit in darkness have seen a great light,and on those who sit in the region and shadow of death a light has dawned.”  17 From that time Jesus began to preach this message: “Repent, for the kingdom of heaven is near!”  18 As he was walking by the Sea of Galilee he saw two brothers, Simon (called Peter) and Andrew his brother, casting a net into the sea (for they were fishermen).  19 He said to them, “Follow me, and I will turn you into fishers of people!”  20 They left their nets immediately and followed him.  21 Going on from there he saw two other brothers, James the son of Zebedee and his brother John, in a boat with their father Zebedee, mending their nets. Then he called them.  22 They immediately left the boat and their father and followed him.  23 Jesus went throughout all of Galilee, teaching in their synagogues, preaching the gospel of the kingdom, and healing every kind of disease and sickness among the people.  24 So a report about him spread throughout Syria. People brought to him all who suffered with various illnesses and afflictions, those who had seizures, paralytics, and those possessed by demons, and he healed them.  25 And large crowds followed him from Galilee, the Decapolis, Jerusalem, Judea, and beyond the Jordan River.  ',\n",
       " '5:1 When he saw the crowds, he went up the mountain. After he sat down his disciples came to him.  2 Then he began to teach them by saying:  3 “Blessed are the poor in spirit, for the kingdom of heaven belongs to them.  4 Blessed are those who mourn, for they will be comforted.  5 Blessed are the meek, for they will inherit the earth.  6 Blessed are those who hunger and thirst for righteousness, for they will be satisfied.  7 Blessed are the merciful, for they will be shown mercy.  8 Blessed are the pure in heart, for they will see God.  9 Blessed are the peacemakers, for they will be called the children of God.  10 Blessed are those who are persecuted for righteousness, for the kingdom of heaven belongs to them.  11 Blessed are you when people insult you and persecute you and say all kinds of evil things about you falsely on account of me.  12 Rejoice and be glad because your reward is great in heaven, for they persecuted the prophets before you in the same way.  13 “You are the salt of the earth. But if salt loses its flavor, how can it be made salty again? It is no longer good for anything except to be thrown out and trampled on by people!  14 You are the light of the world. A city located on a hill cannot be hidden.  15 People do not light a lamp and put it under a basket but on a lampstand, and it gives light to all in the house.  16 In the same way, let your light shine before people, so that they can see your good deeds and give honor to your Father in heaven.  17 “Do not think that I have come to abolish the law or the prophets. I have not come to abolish these things but to fulfill them. 18 I tell you the truth, until heaven and earth pass away not the smallest letter or stroke of a letter will pass from the law until everything takes place.  19 So anyone who breaks one of the least of these commands and teaches others to do so will be called least in the kingdom of heaven, but whoever obeys them and teaches others to do so will be called great in the kingdom of heaven.  20 For I tell you, unless your righteousness goes beyond that of the experts in the law and the Pharisees, you will never enter the kingdom of heaven!  21 “You have heard that it was said to an older generation, ‘Do not murder,’ and ‘whoever murders will be subjected to judgment.’  22 But I say to you that anyone who is angry with a brother will be subjected to judgment. And whoever insults a brother will be brought before the council, and whoever says ‘Fool’ will be sent to fiery hell.  23 So then, if you bring your gift to the altar and there you remember that your brother has something against you,  24 leave your gift there in front of the altar. First go and be reconciled to your brother and then come and present your gift.  25 Reach agreement quickly with your accuser while on the way to court, or he may hand you over to the judge, and the judge hand you over to the warden, and you will be thrown into prison.  26 I tell you the truth, you will never get out of there until you have paid the last penny!  27 “You have heard that it was said, ‘Do not commit adultery.’  28 But I say to you that whoever looks at a woman to desire her has already committed adultery with her in his heart.  29 If your right eye causes you to sin, tear it out and throw it away! It is better to lose one of your members than to have your whole body thrown into hell.  30 If your right hand causes you to sin, cut it off and throw it away! It is better to lose one of your members than to have your whole body go into hell.  31 “It was said, ‘Whoever divorces his wife must give her a legal document.’  32 But I say to you that everyone who divorces his wife, except for immorality, makes her commit adultery, and whoever marries a divorced woman commits adultery.  33 “Again, you have heard that it was said to an older generation, ‘Do not break an oath, but fulfill your vows to the Lord.’  34 But I say to you, do not take oaths at all—not by heaven because it is the throne of God,  35 not by earth because it is his footstool, and not by Jerusalem because it is the city of the great King.  36 Do not take an oath by your head because you are not able to make one hair white or black.  37 Let your word be ‘Yes, yes’ or ‘No, no.’ More than this is from the evil one.  38 “You have heard that it was said, ‘An eye for an eye and a tooth for a tooth.’  39 But I say to you, do not resist the evildoer. But whoever strikes you on the right cheek, turn the other to him as well. 40 And if someone wants to sue you and take your tunic, let him have your coat also.  41 And if anyone forces you to go one mile, go with him two.  42 Give to the one who asks you, and do not reject the one who wants to borrow from you.  43 “You have heard that it was said, ‘Love your neighbor’ and ‘hate your enemy.’  44 But I say to you, love your enemy and pray for those who persecute you,  45 so that you may be like your Father in heaven, since he causes the sun to rise on the evil and the good, and sends rain on the righteous and the unrighteous.  46 For if you love those who love you, what reward do you have? Even the tax collectors do the same, don’t they? 47 And if you only greet your brothers, what more do you do? Even the Gentiles do the same, don’t they? 48 So then, be perfect, as your heavenly Father is perfect.  ',\n",
       " '6:1 “Be careful not to display your righteousness merely to be seen by people. Otherwise you have no reward with your Father in heaven.  2 Thus whenever you do charitable giving, do not blow a trumpet before you, as the hypocrites do in synagogues and on streets so that people will praise them. I tell you the truth, they have their reward!  3 But when you do your giving, do not let your left hand know what your right hand is doing,  4 so that your gift may be in secret. And your Father, who sees in secret, will reward you.  5 “Whenever you pray, do not be like the hypocrites because they love to pray while standing in synagogues and on street corners so that people can see them. Truly I say to you, they have their reward!  6 But whenever you pray, go into your inner room, close the door, and pray to your Father in secret. And your Father, who sees in secret, will reward you.  7 When you pray, do not babble repetitiously like the Gentiles because they think that by their many words they will be heard.  8 Do not be like them, for your Father knows what you need before you ask him.  9 So pray this way: “Our Father in heaven, may your name be honored,  10 may your kingdom come, may your will be done on earth as it is in heaven. 11 Give us today our daily bread,  12 and forgive us our debts, as we ourselves have forgiven our debtors.  13 And do not lead us into temptation, but deliver us from the evil one.  14 “For if you forgive others their sins, your heavenly Father will also forgive you.  15 But if you do not forgive others, your Father will not forgive you your sins.  16 “When you fast, do not look sullen like the hypocrites, for they make their faces unattractive so that people will see them fasting. I tell you the truth, they have their reward!  17 When you fast, anoint your head and wash your face,  18 so that it will not be obvious to others when you are fasting, but only to your Father who is in secret. And your Father, who sees in secret, will reward you.  19 “Do not accumulate for yourselves treasures on earth, where moth and devouring insect destroy and where thieves break in and steal.  20 But accumulate for yourselves treasures in heaven, where moth and devouring insect do not destroy, and thieves do not break in and steal.  21 For where your treasure is, there your heart will be also.  22 “The eye is the lamp of the body. If then your eye is healthy, your whole body will be full of light. 23 But if your eye is diseased, your whole body will be full of darkness. If then the light in you is darkness, how great is the darkness!  24 “No one can serve two masters, for either he will hate the one and love the other, or he will be devoted to the one and despise the other. You cannot serve God and money.  25 “Therefore I tell you, do not worry about your life, what you will eat or drink, or about your body, what you will wear. Isn’t there more to life than food and more to the body than clothing?  26 Look at the birds in the sky: They do not sow, or reap, or gather into barns, yet your heavenly Father feeds them. Aren’t you more valuable than they are? 27 And which of you by worrying can add even one hour to his life?  28 Why do you worry about clothing? Think about how the flowers of the field grow; they do not work or spin.  29 Yet I tell you that not even Solomon in all his glory was clothed like one of these!  30 And if this is how God clothes the wild grass, which is here today and tomorrow is tossed into the fire to heat the oven, won’t he clothe you even more, you people of little faith?  31 So then, don’t worry saying, ‘What will we eat?’ or ‘What will we drink?’ or ‘What will we wear?’  32 For the unconverted pursue these things, and your heavenly Father knows that you need them.  33 But above all pursue his kingdom and righteousness, and all these things will be given to you as well. 34 So then, do not worry about tomorrow, for tomorrow will worry about itself. Today has enough trouble of its own.  ',\n",
       " '7:1 “Do not judge so that you will not be judged.  2 For by the standard you judge you will be judged, and the measure you use will be the measure you receive. 3 Why do you see the speck in your brother’s eye, but fail to see the beam of wood in your own?  4 Or how can you say to your brother, ‘Let me remove the speck from your eye,’ while there is a beam in your own?  5 You hypocrite! First remove the beam from your own eye, and then you can see clearly to remove the speck from your brother’s eye.  6 Do not give what is holy to dogs or throw your pearls before pigs; otherwise they will trample them under their feet and turn around and tear you to pieces.  7 “Ask and it will be given to you; seek and you will find; knock and the door will be opened for you.  8 For everyone who asks receives, and the one who seeks finds, and to the one who knocks, the door will be opened.  9 Is there anyone among you who, if his son asks for bread, will give him a stone?  10 Or if he asks for a fish, will give him a snake?  11 If you then, although you are evil, know how to give good gifts to your children, how much more will your Father in heaven give good gifts to those who ask him!  12 In everything, treat others as you would want them to treat you, for this fulfills the law and the prophets.  13 “Enter through the narrow gate because the gate is wide and the way is spacious that leads to destruction, and there are many who enter through it.  14 How narrow is the gate and difficult the way that leads to life, and there are few who find it!  15 “Watch out for false prophets, who come to you in sheep’s clothing but inwardly are voracious wolves.  16 You will recognize them by their fruit. Grapes are not gathered from thorns or figs from thistles, are they? 17 In the same way, every good tree bears good fruit, but the bad tree bears bad fruit.  18 A good tree is not able to bear bad fruit, nor a bad tree to bear good fruit.  19 Every tree that does not bear good fruit is cut down and thrown into the fire.  20 So then, you will recognize them by their fruit.  21 “Not everyone who says to me, ‘Lord, Lord,’ will enter into the kingdom of heaven—only the one who does the will of my Father in heaven.  22 On that day, many will say to me, ‘Lord, Lord, didn’t we prophesy in your name, and cast out demons in your name, and do many powerful deeds in your name?’  23 Then I will declare to them, ‘I never knew you. Go away from me, you lawbreakers!’  24 “Everyone who hears these words of mine and does them is like a wise man who built his house on rock.  25 The rain fell, the flood came, and the winds beat against that house, but it did not collapse because its foundation had been laid on rock.  26 Everyone who hears these words of mine and does not do them is like a foolish man who built his house on sand.  27 The rain fell, the flood came, and the winds beat against that house, and it collapsed—it was utterly destroyed!”  28 When Jesus finished saying these things, the crowds were amazed by his teaching,  29 because he taught them like one who had authority, not like their experts in the law.  ',\n",
       " '8:1 After he came down from the mountain, large crowds followed him.  2 And a leper approached and bowed low before him, saying, “Lord, if you are willing, you can make me clean.”  3 He stretched out his hand and touched him saying, “I am willing. Be clean!” Immediately his leprosy was cleansed.  4 Then Jesus said to him, “See that you do not speak to anyone, but go, show yourself to the priest, and bring the offering that Moses commanded, as a testimony to them.”  5 When he entered Capernaum, a centurion came to him asking for help:  6 “Lord, my servant is lying at home paralyzed, in terrible anguish.” 7 Jesus said to him, “I will come and heal him.”  8 But the centurion replied, “Lord, I am not worthy to have you come under my roof! Instead, just say the word and my servant will be healed.  9 For I too am a man under authority, with soldiers under me. I say to this one, ‘Go!’ and he goes, and to another ‘Come!’ and he comes, and to my slave ‘Do this!’ and he does it.” 10 When Jesus heard this he was amazed and said to those who followed him, “I tell you the truth, I have not found such faith in anyone in Israel!  11 I tell you, many will come from the east and west to share the banquet with Abraham, Isaac, and Jacob in the kingdom of heaven,  12 but the sons of the kingdom will be thrown out into the outer darkness, where there will be weeping and gnashing of teeth.”  13 Then Jesus said to the centurion, “Go; just as you believed, it will be done for you.” And the servant was healed at that hour.  14 Now when Jesus entered Peter’s house, he saw his mother-in-law lying down, sick with a fever.  15 He touched her hand, and the fever left her. Then she got up and began to serve them.  16 When it was evening, many demon-possessed people were brought to him. He drove out the spirits with a word, and healed all who were sick.  17 In this way what was spoken by the prophet Isaiah was fulfilled: “He took our weaknesses and carried our diseases.”  18 Now when Jesus saw a large crowd around him, he gave orders to go to the other side of the lake. 19 Then an expert in the law came to him and said, “Teacher, I will follow you wherever you go.”  20 Jesus said to him, “Foxes have dens, and the birds in the sky have nests, but the Son of Man has no place to lay his head.”  21 Another of the disciples said to him, “Lord, let me first go and bury my father.”  22 But Jesus said to him, “Follow me, and let the dead bury their own dead.”  23 As he got into the boat, his disciples followed him.  24 And a great storm developed on the sea so that the waves began to swamp the boat. But he was asleep.  25 So they came and woke him up saying, “Lord, save us! We are about to die!”  26 But he said to them, “Why are you cowardly, you people of little faith?” Then he got up and rebuked the winds and the sea, and it was dead calm.  27 And the men were amazed and said, “What sort of person is this? Even the winds and the sea obey him!”  28 When he came to the other side, to the region of the Gadarenes, two demon-possessed men coming from the tombs met him. They were extremely violent, so that no one was able to pass by that way.  29 They cried out, “Son of God, leave us alone! Have you come here to torment us before the time?”  30 A large herd of pigs was feeding some distance from them.  31 Then the demons begged him, “If you drive us out, send us into the herd of pigs.”  32 And he said, “Go!” So they came out and went into the pigs, and the herd rushed down the steep slope into the lake and drowned in the water.  33 The herdsmen ran off, went into the town, and told everything that had happened to the demon-possessed men.  34 Then the entire town came out to meet Jesus. And when they saw him, they begged him to leave their region.  ',\n",
       " '9:1 After getting into a boat he crossed to the other side and came to his own town. 2 Just then some people brought to him a paralytic lying on a stretcher. When Jesus saw their faith, he said to the paralytic, “Have courage, son! Your sins are forgiven.”  3 Then some of the experts in the law said to themselves, “This man is blaspheming!”  4 When Jesus perceived their thoughts he said, “Why do you respond with evil in your hearts?  5 Which is easier, to say, ‘Your sins are forgiven’ or to say, ‘Stand up and walk’?  6 But so that you may know that the Son of Man has authority on earth to forgive sins”—then he said to the paralytic—“Stand up, take your stretcher, and go home.”  7 So he stood up and went home.  8 When the crowd saw this, they were afraid and honored God who had given such authority to men.  9 As Jesus went on from there, he saw a man named Matthew sitting at the tax booth. “Follow me,” he said to him. So he got up and followed him.  10 As Jesus was having a meal in Matthew’s house, many tax collectors and sinners came and ate with Jesus and his disciples.  11 When the Pharisees saw this they said to his disciples, “Why does your teacher eat with tax collectors and sinners?”  12 When Jesus heard this he said, “Those who are healthy don’t need a physician, but those who are sick do.  13 Go and learn what this saying means: ‘I want mercy and not sacrifice.’ For I did not come to call the righteous, but sinners.”  14 Then John’s disciples came to Jesus and asked, “Why do we and the Pharisees fast often, but your disciples don’t fast?”  15 Jesus said to them, “The wedding guests cannot mourn while the bridegroom is with them, can they? But the days are coming when the bridegroom will be taken from them, and then they will fast.  16 No one sews a patch of unshrunk cloth on an old garment because the patch will pull away from the garment and the tear will be worse.  17 And no one pours new wine into old wineskins; otherwise the skins burst and the wine is spilled out and the skins are destroyed. Instead they put new wine into new wineskins and both are preserved.”  18 As he was saying these things, a leader came, bowed low before him, and said, “My daughter has just died, but come and lay your hand on her and she will live.”  19 Jesus and his disciples got up and followed him.  20 But a woman who had been suffering from a hemorrhage for 12 years came up behind him and touched the edge of his cloak.  21 For she kept saying to herself, “If only I touch his cloak, I will be healed.”  22 But when Jesus turned and saw her he said, “Have courage, daughter! Your faith has made you well.” And the woman was healed from that hour.  23 When Jesus entered the leader’s house and saw the flute players and the disorderly crowd,  24 he said, “Go away, for the girl is not dead but asleep!” And they began making fun of him.  25 But when the crowd had been forced outside, he went in and gently took her by the hand, and the girl got up.  26 And the news of this spread throughout that region.  27 As Jesus went on from there, two blind men began to follow him, shouting, “Have mercy on us, Son of David!”  28 When he went into the house, the blind men came to him. Jesus said to them, “Do you believe that I am able to do this?” They said to him, “Yes, Lord.”  29 Then he touched their eyes saying, “Let it be done for you according to your faith.”  30 And their eyes were opened. Then Jesus sternly warned them, “See that no one knows about this!” 31 But they went out and spread the news about him throughout that entire region.  32 As they were going away, a man who was demon-possessed and unable to speak was brought to him.  33 After the demon was cast out, the man who had been mute began to speak. The crowds were amazed and said, “Never has anything like this been seen in Israel!”  34 But the Pharisees said, “By the ruler of demons he casts out demons!”  35 Then Jesus went throughout all the towns and villages, teaching in their synagogues, preaching the good news of the kingdom, and healing every kind of disease and sickness.  36 When he saw the crowds, he had compassion on them because they were bewildered and helpless, like sheep without a shepherd.  37 Then he said to his disciples, “The harvest is plentiful, but the workers are few.  38 Therefore ask the Lord of the harvest to send out workers into his harvest-ready fields.” ',\n",
       " '10:1 Jesus called his 12 disciples and gave them authority over unclean spirits so they could cast them out and heal every kind of disease and sickness. 2 Now these are the names of the 12 apostles: first, Simon (called Peter), and Andrew his brother; James son of Zebedee and John his brother;  3 Philip and Bartholomew; Thomas and Matthew the tax collector; James the son of Alphaeus, and Thaddaeus;  4 Simon the Zealot and Judas Iscariot, who betrayed him.  5 Jesus sent out these 12, instructing them as follows: “Do not go on a road that leads to Gentile regions and do not enter any Samaritan town.  6 Go instead to the lost sheep of the house of Israel.  7 As you go, preach this message: ‘The kingdom of heaven is near!’  8 Heal the sick, raise the dead, cleanse lepers, cast out demons. Freely you received, freely give.  9 Do not take gold, silver, or copper in your belts;  10 no bag for the journey; or an extra tunic or sandals or staff; for the worker deserves his provisions.  11 Whenever you enter a town or village, find out who is worthy there and stay with them until you leave.  12 As you enter the house, greet those within it.  13 And if the house is worthy, let your peace come on it, but if it is not worthy, let your peace return to you.  14 And if anyone will not welcome you or listen to your message, shake the dust off your feet as you leave that house or that town.  15 I tell you the truth, it will be more bearable for the region of Sodom and Gomorrah on the day of judgment than for that town!  16 “I am sending you out like sheep surrounded by wolves, so be wise as serpents and innocent as doves.  17 Beware of people, because they will hand you over to councils and flog you in their synagogues.  18 And you will be brought before governors and kings because of me, as a witness to them and to the Gentiles.  19 Whenever they hand you over for trial, do not worry about how to speak or what to say, for what you should say will be given to you at that time.  20 For it is not you speaking, but the Spirit of your Father speaking through you.  21 “Brother will hand over brother to death, and a father his child. Children will rise against parents and have them put to death.  22 And you will be hated by everyone because of my name. But the one who endures to the end will be saved!  23 Whenever they persecute you in one town, flee to another! I tell you the truth, you will not finish going through all the towns of Israel before the Son of Man comes.  24 “A disciple is not greater than his teacher, nor a slave greater than his master.  25 It is enough for the disciple to become like his teacher, and the slave like his master. If they have called the head of the house ‘Beelzebul,’ how much worse will they call the members of his household!  26 “Do not be afraid of them, for nothing is hidden that will not be revealed, and nothing is secret that will not be made known.  27 What I say to you in the dark, tell in the light, and what is whispered in your ear, proclaim from the housetops.  28 Do not be afraid of those who kill the body but cannot kill the soul. Instead, fear the one who is able to destroy both soul and body in hell. 29 Aren’t two sparrows sold for a penny? Yet not one of them falls to the ground apart from your Father’s will. 30 Even all the hairs on your head are numbered.  31 So do not be afraid; you are more valuable than many sparrows.  32 “Whoever, then, acknowledges me before people, I will acknowledge before my Father in heaven.  33 But whoever denies me before people, I will deny him also before my Father in heaven.  34 “Do not think that I have come to bring peace to the earth. I have not come to bring peace but a sword!  35 For I have come to set a man against his father, a daughter against her mother, and a daughter-in-law against her mother-in-law,  36 and a man’s enemies will be the members of his household.  37 “Whoever loves father or mother more than me is not worthy of me, and whoever loves son or daughter more than me is not worthy of me.  38 And whoever does not take up his cross and follow me is not worthy of me.  39 Whoever finds his life will lose it, and whoever loses his life because of me will find it.  40 “Whoever receives you receives me, and whoever receives me receives the one who sent me.  41 Whoever receives a prophet in the name of a prophet will receive a prophet’s reward. Whoever receives a righteous person in the name of a righteous person will receive a righteous person’s reward.  42 And whoever gives only a cup of cold water to one of these little ones in the name of a disciple, I tell you the truth, he will never lose his reward.”  ',\n",
       " '11:1 When Jesus had finished instructing his 12 disciples, he went on from there to teach and preach in their towns.  2 Now when John heard in prison about the deeds Christ had done, he sent his disciples to ask a question: 3 “Are you the one who is to come, or should we look for another?”  4 Jesus answered them, “Go tell John what you hear and see:  5 The blind see, the lame walk, lepers are cleansed, the deaf hear, the dead are raised, and the poor have good news proclaimed to them 6 —and blessed is anyone who takes no offense at me!”  7 While they were going away, Jesus began to speak to the crowd about John: “What did you go out into the wilderness to see? A reed shaken by the wind?  8 What did you go out to see? A man dressed in soft clothing? Look, those who wear soft clothing are in the palaces of kings!  9 What did you go out to see? A prophet? Yes, I tell you, and more than a prophet!  10 This is the one about whom it is written: “‘Look, I am sending my messenger ahead of you, who will prepare your way before you.’  11 “I tell you the truth, among those born of women, no one has arisen greater than John the Baptist. Yet the one who is least in the kingdom of heaven is greater than he is!  12 From the days of John the Baptist until now the kingdom of heaven has suffered violence, and forceful people lay hold of it.  13 For all the prophets and the law prophesied until John appeared. 14 And if you are willing to accept it, he is Elijah, who is to come.  15 The one who has ears had better listen!  16 “To what should I compare this generation? They are like children sitting in the marketplaces who call out to one another,  17 “‘We played the flute for you, yet you did not dance; we wailed in mourning, yet you did not weep.’  18 For John came neither eating nor drinking, and they say, ‘He has a demon!’  19 The Son of Man came eating and drinking, and they say, ‘Look at him, a glutton and a drunk, a friend of tax collectors and sinners!’ But wisdom is vindicated by her deeds.”  20 Then Jesus began to criticize openly the cities in which he had done many of his miracles because they did not repent.  21 “Woe to you, Chorazin! Woe to you, Bethsaida! If the miracles done in you had been done in Tyre and Sidon, they would have repented long ago in sackcloth and ashes.  22 But I tell you, it will be more bearable for Tyre and Sidon on the day of judgment than for you!  23 And you, Capernaum, will you be exalted to heaven? No, you will be thrown down to Hades! For if the miracles done among you had been done in Sodom, it would have continued to this day.  24 But I tell you, it will be more bearable for the region of Sodom on the day of judgment than for you!”  25 At that time Jesus said, “I praise you, Father, Lord of heaven and earth, because you have hidden these things from the wise and intelligent, and have revealed them to little children.  26 Yes, Father, for this was your gracious will. 27 All things have been handed over to me by my Father. No one knows the Son except the Father, and no one knows the Father except the Son and anyone to whom the Son decides to reveal him. 28 Come to me, all you who are weary and burdened, and I will give you rest.  29 Take my yoke on you and learn from me because I am gentle and humble in heart, and you will find rest for your souls.  30 For my yoke is easy to bear, and my load is not hard to carry.”  ',\n",
       " '12:1 At that time Jesus went through the grain fields on a Sabbath. His disciples were hungry, and they began to pick heads of wheat and eat them. 2 But when the Pharisees saw this they said to him, “Look, your disciples are doing what is against the law to do on the Sabbath.”  3 He said to them, “Haven’t you read what David did when he and his companions were hungry— 4 how he entered the house of God and ate the sacred bread, which was against the law for him or his companions to eat, but only for the priests?  5 Or have you not read in the law that the priests in the temple desecrate the Sabbath and yet are not guilty?  6 I tell you that something greater than the temple is here.  7 If you had known what this means: ‘I want mercy and not sacrifice,’ you would not have condemned the innocent.  8 For the Son of Man is lord of the Sabbath.”  9 Then Jesus left that place and entered their synagogue.  10 A man was there who had a withered hand. And they asked Jesus, “Is it lawful to heal on the Sabbath?” so that they could accuse him.  11 He said to them, “Would not any one of you, if he had one sheep that fell into a pit on the Sabbath, take hold of it and lift it out?  12 How much more valuable is a person than a sheep! So it is lawful to do good on the Sabbath.”  13 Then he said to the man, “Stretch out your hand.” He stretched it out and it was restored, as healthy as the other.  14 But the Pharisees went out and plotted against him, as to how they could assassinate him.  15 Now when Jesus learned of this, he went away from there. Great crowds followed him, and he healed them all.  16 But he sternly warned them not to make him known.  17 This fulfilled what was spoken by the prophet Isaiah:  18 “Here is my servant whom I have chosen, the one I love, in whom I take great delight. I will put my Spirit on him, and he will proclaim justice to the nations.  19 He will not quarrel or cry out, nor will anyone hear his voice in the streets. 20 He will not break a bruised reed or extinguish a smoldering wick, until he brings justice to victory.  21 And in his name the Gentiles will hope.”  22 Then they brought to him a demon-possessed man who was blind and mute. Jesus healed him so that he could speak and see.  23 All the crowds were amazed and said, “Could this one be the Son of David?”  24 But when the Pharisees heard this they said, “He does not cast out demons except by the power of Beelzebul, the ruler of demons!”  25 Now when Jesus realized what they were thinking, he said to them, “Every kingdom divided against itself is destroyed, and no town or house divided against itself will stand.  26 So if Satan casts out Satan, he is divided against himself. How then will his kingdom stand?  27 And if I cast out demons by Beelzebul, by whom do your sons cast them out? For this reason they will be your judges.  28 But if I cast out demons by the Spirit of God, then the kingdom of God has already overtaken you.  29 How else can someone enter a strong man’s house and steal his property, unless he first ties up the strong man? Then he can thoroughly plunder the house.  30 Whoever is not with me is against me, and whoever does not gather with me scatters.  31 For this reason I tell you, people will be forgiven for every sin and blasphemy, but the blasphemy against the Spirit will not be forgiven.  32 Whoever speaks a word against the Son of Man will be forgiven. But whoever speaks against the Holy Spirit will not be forgiven, either in this age or in the age to come.  33 “Make a tree good and its fruit will be good, or make a tree bad and its fruit will be bad, for a tree is known by its fruit.  34 Offspring of vipers! How are you able to say anything good, since you are evil? For the mouth speaks from what fills the heart.  35 The good person brings good things out of his good treasury, and the evil person brings evil things out of his evil treasury.  36 I tell you that on the day of judgment, people will give an account for every worthless word they speak.  37 For by your words you will be justified, and by your words you will be condemned.”  38 Then some of the experts in the law along with some Pharisees answered him, “Teacher, we want to see a sign from you.”  39 But he answered them, “An evil and adulterous generation asks for a sign, but no sign will be given to it except the sign of the prophet Jonah.  40 For just as Jonah was in the belly of the huge fish for three days and three nights, so the Son of Man will be in the heart of the earth for three days and three nights.  41 The people of Nineveh will stand up at the judgment with this generation and condemn it because they repented when Jonah preached to them—and now, something greater than Jonah is here!  42 The queen of the South will rise up at the judgment with this generation and condemn it because she came from the ends of the earth to hear the wisdom of Solomon—and now, something greater than Solomon is here!  43 “When an unclean spirit goes out of a person, it passes through waterless places looking for rest but does not find it. 44 Then it says, ‘I will return to the home I left.’ When it returns, it finds the house empty, swept clean, and put in order.  45 Then it goes and brings with it seven other spirits more evil than itself, and they go in and live there, so the last state of that person is worse than the first. It will be that way for this evil generation as well!” 46 While Jesus was still speaking to the crowds, his mother and brothers came and stood outside, asking to speak to him.  47  Someone told him, “Look, your mother and your brothers are standing outside wanting to speak to you.” 48 To the one who had said this, Jesus replied, “Who is my mother and who are my brothers?”  49 And pointing toward his disciples he said, “Here are my mother and my brothers!  50 For whoever does the will of my Father in heaven is my brother and sister and mother.”  ',\n",
       " '13:1 On that day after Jesus went out of the house, he sat by the lake.  2 And such a large crowd gathered around him that he got into a boat to sit while the whole crowd stood on the shore.  3 He told them many things in parables, saying: “Listen! A sower went out to sow.  4 And as he sowed, some seeds fell along the path, and the birds came and devoured them.  5 Other seeds fell on rocky ground where they did not have much soil. They sprang up quickly because the soil was not deep.  6 But when the sun came up, they were scorched, and because they did not have sufficient root, they withered.  7 Other seeds fell among the thorns, and they grew up and choked them.  8 But other seeds fell on good soil and produced grain, some a hundred times as much, some sixty, and some thirty.  9 The one who has ears had better listen!”  10 Then the disciples came to him and said, “Why do you speak to them in parables?”  11 He replied, “You have been given the opportunity to know the secrets of the kingdom of heaven, but they have not.  12 For whoever has will be given more, and will have an abundance. But whoever does not have, even what he has will be taken from him.  13 For this reason I speak to them in parables: Although they see they do not see, and although they hear they do not hear nor do they understand.  14 And concerning them the prophecy of Isaiah is fulfilled that says: “‘You will listen carefully yet will never understand, you will look closely yet will never comprehend.  15 For the heart of this people has become dull;they are hard of hearing, and they have shut their eyes, so that they would not see with their eyes and hear with their ears and understand with their hearts and turn, and I would heal them.’  16 “But your eyes are blessed because they see, and your ears because they hear.  17 For I tell you the truth, many prophets and righteous people longed to see what you see but did not see it, and to hear what you hear but did not hear it. 18 “So listen to the parable of the sower:  19 When anyone hears the word about the kingdom and does not understand it, the evil one comes and snatches what was sown in his heart; this is the seed sown along the path.  20 The seed sown on rocky ground is the person who hears the word and immediately receives it with joy.  21 But he has no root in himself and does not endure; when trouble or persecution comes because of the word, immediately he falls away.  22 The seed sown among thorns is the person who hears the word, but worldly cares and the seductiveness of wealth choke the word, so it produces nothing.  23 But as for the seed sown on good soil, this is the person who hears the word and understands. He bears fruit, yielding a hundred, sixty, or thirty times what was sown.”  24 He presented them with another parable: “The kingdom of heaven is like a person who sowed good seed in his field.  25 But while everyone was sleeping, an enemy came and sowed darnel among the wheat and went away.  26 When the plants sprouted and produced grain, then the darnel also appeared.  27 So the slaves of the landowner came and said to him, ‘Sir, didn’t you sow good seed in your field? Then where did the darnel come from?’  28 He said, ‘An enemy has done this!’ So the slaves replied, ‘Do you want us to go and gather it?’ 29 But he said, ‘No, since in gathering the darnel you may uproot the wheat along with it. 30 Let both grow together until the harvest. At harvest time I will tell the reapers, “First collect the darnel and tie it in bundles to be burned, but then gather the wheat into my barn.”’”  31 He gave them another parable: “The kingdom of heaven is like a mustard seed that a man took and sowed in his field.  32 It is the smallest of all the seeds, but when it has grown it is the greatest garden plant and becomes a tree, so that the wild birds come and nest in its branches.”  33 He told them another parable: “The kingdom of heaven is like yeast that a woman took and mixed with three measures of flour until all the dough had risen.”  34 Jesus spoke all these things in parables to the crowds; he did not speak to them without a parable.  35 This fulfilled what was spoken by the prophet: “I will open my mouth in parables, I will announce what has been hidden from the foundation of the world.” 36 Then he left the crowds and went into the house. And his disciples came to him saying, “Explain to us the parable of the darnel in the field.”  37 He answered, “The one who sowed the good seed is the Son of Man.  38 The field is the world and the good seed are the people of the kingdom. The poisonous weeds are the people of the evil one,  39 and the enemy who sows them is the devil. The harvest is the end of the age, and the reapers are angels.  40 As the poisonous weeds are collected and burned with fire, so it will be at the end of the age.  41 The Son of Man will send his angels, and they will gather from his kingdom everything that causes sin as well as all lawbreakers.  42 They will throw them into the fiery furnace, where there will be weeping and gnashing of teeth.  43 Then the righteous will shine like the sun in the kingdom of their Father. The one who has ears had better listen!  44 “The kingdom of heaven is like a treasure, hidden in a field, that a person found and hid. Then because of joy he went and sold all that he had and bought that field.  45 “Again, the kingdom of heaven is like a merchant searching for fine pearls.  46 When he found a pearl of great value, he went out and sold everything he had and bought it.  47 “Again, the kingdom of heaven is like a net that was cast into the sea that caught all kinds of fish. 48 When it was full, they pulled it ashore, sat down, and put the good fish into containers and threw the bad away.  49 It will be this way at the end of the age. Angels will come and separate the evil from the righteous  50 and throw them into the fiery furnace, where there will be weeping and gnashing of teeth.  51 “Have you understood all these things?” They replied, “Yes.” 52 Then he said to them, “Therefore every expert in the law who has been trained for the kingdom of heaven is like the owner of a house who brings out of his treasure what is new and old.”  53 Now when Jesus finished these parables, he moved on from there.  54 Then he came to his hometown and began to teach the people in their synagogue. They were astonished and said, “Where did this man get such wisdom and miraculous powers?  55 Isn’t this the carpenter’s son? Isn’t his mother named Mary? And aren’t his brothers James, Joseph, Simon, and Judas?  56 And aren’t all his sisters here with us? So where did he get all this?”  57 And so they took offense at him. But Jesus said to them, “A prophet is not without honor except in his hometown and in his own house.”  58 And he did not do many miracles there because of their unbelief.  ',\n",
       " '14:1 At that time Herod the tetrarch heard reports about Jesus,  2 and he said to his servants, “This is John the Baptist. He has been raised from the dead! And because of this, miraculous powers are at work in him.”  3 For Herod had arrested John, bound him, and put him in prison on account of Herodias, his brother Philip’s wife,  4 because John had repeatedly told him, “It is not lawful for you to have her.”  5 Although Herod wanted to kill John, he feared the crowd because they accepted John as a prophet.  6 But on Herod’s birthday, the daughter of Herodias danced before them and pleased Herod,  7 so much that he promised with an oath to give her whatever she asked.  8 Instructed by her mother, she said, “Give me the head of John the Baptist here on a platter.”  9 Although it grieved the king because of his oath and the dinner guests, he commanded it to be given.  10 So he sent and had John beheaded in the prison.  11 His head was brought on a platter and given to the girl, and she brought it to her mother.  12 Then John’s disciples came and took the body and buried it and went and told Jesus.  13 Now when Jesus heard this he went away from there privately in a boat to an isolated place. But when the crowd heard about it, they followed him on foot from the towns.  14 As he got out he saw the large crowd, and he had compassion on them and healed their sick.  15 When evening arrived, his disciples came to him saying, “This is an isolated place and the hour is already late. Send the crowds away so that they can go into the villages and buy food for themselves.”  16 But he replied, “They don’t need to go. You give them something to eat.”  17 They said to him, “We have here only five loaves and two fish.”  18 “Bring them here to me,” he replied.  19 Then he instructed the crowds to sit down on the grass. He took the five loaves and two fish, and looking up to heaven he gave thanks and broke the loaves. He gave them to the disciples, who in turn gave them to the crowds.  20 They all ate and were satisfied, and they picked up the broken pieces left over, 12 baskets full.  21 Not counting women and children, there were about 5,000 men who ate.  22 Immediately Jesus made the disciples get into the boat and go ahead of him to the other side, while he dispersed the crowds.  23 And after he sent the crowds away, he went up the mountain by himself to pray. When evening came, he was there alone.  24 Meanwhile the boat, already far from land, was taking a beating from the waves because the wind was against it. 25 As the night was ending, Jesus came to them walking on the sea.  26 When the disciples saw him walking on the water they were terrified and said, “It’s a ghost!” and cried out with fear.  27 But immediately Jesus spoke to them: “Have courage! It is I. Do not be afraid.”  28 Peter said to him, “Lord, if it is you, order me to come to you on the water.”  29 So he said, “Come.” Peter got out of the boat, walked on the water, and came toward Jesus.  30 But when he saw the strong wind he became afraid. And starting to sink, he cried out, “Lord, save me!”  31 Immediately Jesus reached out his hand and caught him, saying to him, “You of little faith, why did you doubt?”  32 When they went up into the boat, the wind ceased.  33 Then those who were in the boat worshiped him, saying, “Truly you are the Son of God.”  34 After they had crossed over, they came to land at Gennesaret.  35 When the people there recognized him, they sent word into all the surrounding area, and they brought all their sick to him.  36 They begged him if they could only touch the edge of his cloak, and all who touched it were healed.  ',\n",
       " '15:1 Then Pharisees and experts in the law came from Jerusalem to Jesus and said,  2 “Why do your disciples disobey the tradition of the elders? For they don’t wash their hands when they eat.”  3 He answered them, “And why do you disobey the commandment of God because of your tradition?  4 For God said, ‘Honor your father and mother’ and ‘Whoever insults his father or mother must be put to death.’  5 But you say, ‘If someone tells his father or mother, “Whatever help you would have received from me is given to God,”  6 he does not need to honor his father.’ You have nullified the word of God on account of your tradition.  7 Hypocrites! Isaiah prophesied correctly about you when he said,  8 “‘This people honors me with their lips, but their hearts are far from me,  9 and they worship me in vain, teaching as doctrines the commandments of men.’”  10 Then he called the crowd to him and said, “Listen and understand.  11 What defiles a person is not what goes into the mouth; it is what comes out of the mouth that defiles a person.”  12 Then the disciples came to him and said, “Do you know that when the Pharisees heard this saying they were offended?”  13 And he replied, “Every plant that my heavenly Father did not plant will be uprooted.  14 Leave them! They are blind guides. If someone who is blind leads another who is blind, both will fall into a pit.”  15 But Peter said to him, “Explain this parable to us.”  16 Jesus said, “Even after all this, are you still so foolish?  17 Don’t you understand that whatever goes into the mouth enters the stomach and then passes out into the sewer?  18 But the things that come out of the mouth come from the heart, and these things defile a person.  19 For out of the heart come evil ideas, murder, adultery, sexual immorality, theft, false testimony, slander.  20 These are the things that defile a person; it is not eating with unwashed hands that defiles a person.”  21 After going out from there, Jesus went to the region of Tyre and Sidon.  22 A Canaanite woman from that area came and cried out, “Have mercy on me, Lord, Son of David! My daughter is horribly demon-possessed!”  23 But he did not answer her a word. Then his disciples came and begged him, “Send her away because she keeps on crying out after us.”  24 So he answered, “I was sent only to the lost sheep of the house of Israel.”  25 But she came and bowed down before him and said, “Lord, help me!”  26 “It is not right to take the children’s bread and throw it to the dogs,” he said.  27 “Yes, Lord,” she replied, “but even the dogs eat the crumbs that fall from their masters’ table.”  28 Then Jesus answered her, “Woman, your faith is great! Let what you want be done for you.” And her daughter was healed from that hour.  29 When he left there, Jesus went along the Sea of Galilee. Then he went up a mountain, where he sat down.  30 Then large crowds came to him bringing with them the lame, blind, crippled, mute, and many others. They laid them at his feet, and he healed them.  31 As a result, the crowd was amazed when they saw the mute speaking, the crippled healthy, the lame walking, and the blind seeing, and they praised the God of Israel.  32 Then Jesus called his disciples and said, “I have compassion on the crowd because they have already been here with me three days and they have nothing to eat. I don’t want to send them away hungry since they may faint on the way.”  33 The disciples said to him, “Where can we get enough bread in this desolate place to satisfy so great a crowd?”  34 Jesus said to them, “How many loaves do you have?” They replied, “Seven—and a few small fish.”  35 After instructing the crowd to sit down on the ground,  36 he took the seven loaves and the fish, and after giving thanks, he broke them and began giving them to the disciples, who then gave them to the crowds.  37 They all ate and were satisfied, and they picked up the broken pieces left over, seven baskets full.  38 Not counting children and women, there were 4,000 men who ate.  39 After sending away the crowd, he got into the boat and went to the region of Magadan.  ',\n",
       " '16:1 Now when the Pharisees and Sadducees came to test Jesus, they asked him to show them a sign from heaven.  2 He said, “When evening comes you say, ‘It will be fair weather because the sky is red,’  3 and in the morning, ‘It will be stormy today because the sky is red and darkening.’ You know how to judge correctly the appearance of the sky, but you cannot evaluate the signs of the times.  4 A wicked and adulterous generation asks for a sign, but no sign will be given to it except the sign of Jonah.” Then he left them and went away.  5 When the disciples went to the other side, they forgot to take bread.  6 “Watch out,” Jesus said to them, “beware of the yeast of the Pharisees and Sadducees.”  7 So they began to discuss this among themselves, saying, “It is because we brought no bread.”  8 When Jesus learned of this, he said, “You who have such little faith! Why are you arguing among yourselves about having no bread?  9 Do you still not understand? Don’t you remember the five loaves for the 5,000, and how many baskets you took up?  10 Or the seven loaves for the 4,000 and how many baskets you took up?  11 How could you not understand that I was not speaking to you about bread? But beware of the yeast of the Pharisees and Sadducees!”  12 Then they understood that he had not told them to be on guard against the yeast in bread, but against the teaching of the Pharisees and Sadducees.  13 When Jesus came to the area of Caesarea Philippi, he asked his disciples, “Who do people say that the Son of Man is?”  14 They answered, “Some say John the Baptist, others Elijah, and others Jeremiah or one of the prophets.”  15 He said to them, “But who do you say that I am?”  16 Simon Peter answered, “You are the Christ, the Son of the living God.”  17 And Jesus answered him, “You are blessed, Simon son of Jonah, because flesh and blood did not reveal this to you, but my Father in heaven!  18 And I tell you that you are Peter, and on this rock I will build my church, and the gates of Hades will not overpower it.  19 I will give you the keys of the kingdom of heaven. Whatever you bind on earth will have been bound in heaven, and whatever you release on earth will have been released in heaven.”  20 Then he instructed his disciples not to tell anyone that he was the Christ.  21 From that time on Jesus began to show his disciples that he must go to Jerusalem and suffer many things at the hands of the elders, chief priests, and experts in the law, and be killed and on the third day be raised.  22 So Peter took him aside and began to rebuke him: “God forbid, Lord! This must not happen to you!”  23 But he turned and said to Peter, “Get behind me, Satan! You are a stumbling block to me because you are not setting your mind on God’s interests, but on man’s.” 24 Then Jesus said to his disciples, “If anyone wants to become my follower, he must deny himself, take up his cross, and follow me.  25 For whoever wants to save his life will lose it, but whoever loses his life because of me will find it.  26 For what does it benefit a person if he gains the whole world but forfeits his life? Or what can a person give in exchange for his life?  27 For the Son of Man will come with his angels in the glory of his Father, and then he will reward each person according to what he has done.  28 I tell you the truth, there are some standing here who will not experience death before they see the Son of Man coming in his kingdom.”  ',\n",
       " '17:1 Six days later Jesus took with him Peter, James, and John the brother of James, and led them privately up a high mountain.  2 And he was transfigured before them. His face shone like the sun, and his clothes became white as light.  3 Then Moses and Elijah also appeared before them, talking with him.  4 So Peter said to Jesus, “Lord, it is good for us to be here. If you want, I will make three shelters—one for you, one for Moses, and one for Elijah.”  5 While he was still speaking, a bright cloud overshadowed them, and a voice from the cloud said, “This is my one dear Son, in whom I take great delight. Listen to him!”  6 When the disciples heard this, they were overwhelmed with fear and threw themselves down with their faces to the ground. 7 But Jesus came and touched them. “Get up,” he said. “Do not be afraid.”  8 When they looked up, all they saw was Jesus alone. 9 As they were coming down from the mountain, Jesus commanded them, “Do not tell anyone about the vision until the Son of Man is raised from the dead.”  10 The disciples asked him, “Why then do the experts in the law say that Elijah must come first?”  11 He answered, “Elijah does indeed come first and will restore all things.  12 And I tell you that Elijah has already come. Yet they did not recognize him, but did to him whatever they wanted. In the same way, the Son of Man will suffer at their hands.”  13 Then the disciples understood that he was speaking to them about John the Baptist.  14 When they came to the crowd, a man came to him, knelt before him,  15 and said, “Lord, have mercy on my son because he has seizures and suffers terribly, for he often falls into the fire and into the water.  16 I brought him to your disciples, but they were not able to heal him.”  17 Jesus answered, “You unbelieving and perverse generation! How much longer must I be with you? How much longer must I endure you? Bring him here to me.”  18 Then Jesus rebuked the demon and it came out of him, and the boy was healed from that moment.  19 Then the disciples came to Jesus privately and said, “Why couldn’t we cast it out?”  20 He told them, “It was because of your little faith. I tell you the truth, if you have faith the size of a mustard seed, you will say to this mountain, ‘Move from here to there,’ and it will move; nothing will be impossible for you.”  22 When they gathered together in Galilee, Jesus told them, “The Son of Man is going to be betrayed into the hands of men.  23 They will kill him, and on the third day he will be raised.” And they became greatly distressed.  24 After they arrived in Capernaum, the collectors of the temple tax came to Peter and said, “Your teacher pays the double drachma tax, doesn’t he?” 25 He said, “Yes.” When Peter came into the house, Jesus spoke to him first, “What do you think, Simon? From whom do earthly kings collect tolls or taxes—from their sons or from foreigners?”  26 After he said, “From foreigners,” Jesus said to him, “Then the sons are free.  27 But so that we don’t offend them, go to the lake and throw out a hook. Take the first fish that comes up, and when you open its mouth, you will find a four-drachma coin. Take that and give it to them for me and you.”  ',\n",
       " '18:1 At that time the disciples came to Jesus saying, “Who is the greatest in the kingdom of heaven?”  2 He called a child, had him stand among them,  3 and said, “I tell you the truth, unless you turn around and become like little children, you will never enter the kingdom of heaven!  4 Whoever then humbles himself like this little child is the greatest in the kingdom of heaven.  5 And whoever welcomes a child like this in my name welcomes me.  6 “But if anyone causes one of these little ones who believe in me to sin, it would be better for him to have a huge millstone hung around his neck and to be drowned in the open sea.  7 Woe to the world because of stumbling blocks! It is necessary that stumbling blocks come, but woe to the person through whom they come.  8 If your hand or your foot causes you to sin, cut it off and throw it away. It is better for you to enter life crippled or lame than to have two hands or two feet and be thrown into eternal fire.  9 And if your eye causes you to sin, tear it out and throw it away. It is better for you to enter into life with one eye than to have two eyes and be thrown into fiery hell.  10 “See that you do not disdain one of these little ones. For I tell you that their angels in heaven always see the face of my Father in heaven.  12 What do you think? If someone owns a hundred sheep and one of them goes astray, will he not leave the ninety-nine on the mountains and go look for the one that went astray?  13 And if he finds it, I tell you the truth, he will rejoice more over it than over the ninety-nine that did not go astray.  14 In the same way, your Father in heaven is not willing that one of these little ones be lost.  15 “If your brother sins, go and show him his fault when the two of you are alone. If he listens to you, you have regained your brother.  16 But if he does not listen, take one or two others with you, so that at the testimony of two or three witnesses every matter may be established.  17 If he refuses to listen to them, tell it to the church. If he refuses to listen to the church, treat him like a Gentile or a tax collector.  18 “I tell you the truth, whatever you bind on earth will have been bound in heaven, and whatever you release on earth will have been released in heaven.  19 Again, I tell you the truth, if two of you on earth agree about whatever you ask, my Father in heaven will do it for you.  20 For where two or three are assembled in my name, I am there among them.”  21 Then Peter came to him and said, “Lord, how many times must I forgive my brother who sins against me? As many as seven times?”  22 Jesus said to him, “Not seven times, I tell you, but seventy-seven times!  23 “For this reason, the kingdom of heaven is like a king who wanted to settle accounts with his slaves.  24 As he began settling his accounts, a man who owed 10,000 talents was brought to him.  25 Because he was not able to repay it, the lord ordered him to be sold, along with his wife, children, and whatever he possessed, and repayment to be made. 26 Then the slave threw himself to the ground before him, saying, ‘Be patient with me, and I will repay you everything.’  27 The lord had compassion on that slave and released him, and forgave him the debt.  28 After he went out, that same slave found one of his fellow slaves who owed him 100 silver coins. So he grabbed him by the throat and started to choke him, saying, ‘Pay back what you owe me!’ 29 Then his fellow slave threw himself down and begged him, ‘Be patient with me, and I will repay you.’  30 But he refused. Instead, he went out and threw him in prison until he repaid the debt.  31 When his fellow slaves saw what had happened, they were very upset and went and told their lord everything that had taken place.  32 Then his lord called the first slave and said to him, ‘Evil slave! I forgave you all that debt because you begged me!  33 Should you not have shown mercy to your fellow slave, just as I showed it to you?’  34 And in anger his lord turned him over to the prison guards to torture him until he repaid all he owed.  35 So also my heavenly Father will do to you, if each of you does not forgive your brother from your heart.”  ',\n",
       " '19:1 Now when Jesus finished these sayings, he left Galilee and went to the region of Judea beyond the Jordan River.  2 Large crowds followed him, and he healed them there.  3 Then some Pharisees came to him in order to test him. They asked, “Is it lawful to divorce a wife for any cause?”  4 He answered, “Have you not read that from the beginning the Creator made them male and female,  5 and said, ‘For this reason a man will leave his father and mother and will be united with his wife, and the two will become one flesh’?  6 So they are no longer two, but one flesh. Therefore what God has joined together, let no one separate.”  7 They said to him, “Why then did Moses command us to give a certificate of dismissal and to divorce her?” 8 Jesus said to them, “Moses permitted you to divorce your wives because of your hard hearts, but from the beginning it was not this way.  9 Now I say to you that whoever divorces his wife, except for immorality, and marries another commits adultery.”  10 The disciples said to him, “If this is the case of a husband with a wife, it is better not to marry!”  11 He said to them, “Not everyone can accept this statement, except those to whom it has been given.  12 For there are some eunuchs who were that way from birth, and some who were made eunuchs by others, and some who became eunuchs for the sake of the kingdom of heaven. The one who is able to accept this should accept it.” 13 Then little children were brought to him for him to lay his hands on them and pray. But the disciples scolded those who brought them.  14 But Jesus said, “Let the little children come to me and do not try to stop them, for the kingdom of heaven belongs to such as these.”  15 And he placed his hands on them and went on his way.  16 Now someone came up to him and said, “Teacher, what good thing must I do to gain eternal life?” 17 He said to him, “Why do you ask me about what is good? There is only one who is good. But if you want to enter into life, keep the commandments.”  18 “Which ones?” he asked. Jesus replied, “Do not murder, do not commit adultery, do not steal, do not give false testimony,  19 honor your father and mother, and love your neighbor as yourself.”  20 The young man said to him, “I have wholeheartedly obeyed all these laws. What do I still lack?”  21 Jesus said to him, “If you wish to be perfect, go sell your possessions and give the money to the poor, and you will have treasure in heaven. Then come, follow me.”  22 But when the young man heard this he went away sorrowful, for he was very rich.  23 Then Jesus said to his disciples, “I tell you the truth, it will be hard for a rich person to enter the kingdom of heaven!  24 Again I say, it is easier for a camel to go through the eye of a needle than for a rich person to enter into the kingdom of God.”  25 The disciples were greatly astonished when they heard this and said, “Then who can be saved?”  26 Jesus looked at them and replied, “This is impossible for mere humans, but for God all things are possible.”  27 Then Peter said to him, “Look, we have left everything to follow you! What then will there be for us?”  28 Jesus said to them, “I tell you the truth: In the age when all things are renewed, when the Son of Man sits on his glorious throne, you who have followed me will also sit on 12 thrones, judging the 12 tribes of Israel.  29 And whoever has left houses or brothers or sisters or father or mother or children or fields for my sake will receive a hundred times as much and will inherit eternal life.  30 But many who are first will be last, and the last first.  ',\n",
       " '20:1 “For the kingdom of heaven is like a landowner who went out early in the morning to hire workers for his vineyard.  2 And after agreeing with the workers for the standard wage, he sent them into his vineyard.  3 When it was about nine o’clock in the morning, he went out again and saw others standing around in the marketplace without work.  4 He said to them, ‘You go into the vineyard too, and I will give you whatever is right.’  5 So they went. When he went out again about noon and three o’clock that afternoon, he did the same thing.  6 And about five o’clock that afternoon he went out and found others standing around, and said to them, ‘Why are you standing here all day without work?’  7 They said to him, ‘Because no one hired us.’ He said to them, ‘You go and work in the vineyard too.’ 8 When it was evening the owner of the vineyard said to his manager, ‘Call the workers and pay them their wages starting with the last hired until the first.’  9 When those hired about five o’clock came, each received a full day’s pay.  10 And when those hired first came, they thought they would receive more. But each one also received the standard wage.  11 When they received it, they began to complain against the landowner,  12 saying, ‘These last fellows worked one hour, and you have made them equal to us who bore the hardship and burning heat of the day.’  13 And the landowner replied to one of them, ‘Friend, I am not treating you unfairly. Didn’t you agree with me to work for the standard wage?  14 Take what is yours and go. I want to give to this last man the same as I gave to you.  15 Am I not permitted to do what I want with what belongs to me? Or are you envious because I am generous?’  16 So the last will be first, and the first last.”  17 As Jesus was going up to Jerusalem, he took the twelve aside privately and said to them on the way,  18 “Look, we are going up to Jerusalem, and the Son of Man will be handed over to the chief priests and the experts in the law. They will condemn him to death,  19 and will turn him over to the Gentiles to be mocked and flogged severely and crucified. Yet on the third day, he will be raised.”  20 Then the mother of the sons of Zebedee came to him with her sons, and kneeling down she asked him for a favor.  21 He said to her, “What do you want?” She replied, “Permit these two sons of mine to sit, one at your right hand and one at your left, in your kingdom.”  22 Jesus answered, “You don’t know what you are asking! Are you able to drink the cup I am about to drink?” They said to him, “We are able.”  23 He told them, “You will drink my cup, but to sit at my right and at my left is not mine to give. Rather, it is for those for whom it has been prepared by my Father.”  24 Now when the other 10 heard this, they were angry with the two brothers.  25 But Jesus called them and said, “You know that the rulers of the Gentiles lord it over them, and those in high positions use their authority over them.  26 It must not be this way among you! Instead whoever wants to be great among you must be your servant,  27 and whoever wants to be first among you must be your slave— 28 just as the Son of Man did not come to be served but to serve, and to give his life as a ransom for many.”  29 As they were leaving Jericho, a large crowd followed them.  30 Two blind men were sitting by the road. When they heard that Jesus was passing by, they shouted, “Have mercy on us, Lord, Son of David!”  31 The crowd scolded them to get them to be quiet. But they shouted even more loudly, “Lord, have mercy on us, Son of David!”  32 Jesus stopped, called them, and said, “What do you want me to do for you?”  33 They said to him, “Lord, let our eyes be opened.”  34 Moved with compassion, Jesus touched their eyes. Immediately they received their sight and followed him.  ',\n",
       " '21:1 Now when they approached Jerusalem and came to Bethphage, at the Mount of Olives, Jesus sent two disciples,  2 telling them, “Go to the village ahead of you. Right away you will find a donkey tied there, and a colt with her. Untie them and bring them to me.  3 If anyone says anything to you, you are to say, ‘The Lord needs them,’ and he will send them at once.”  4 This took place to fulfill what was spoken by the prophet:  5 “Tell the people of Zion, ‘Look, your king is coming to you, unassuming and seated on a donkey, and on a colt, the foal of a donkey.’”  6 So the disciples went and did as Jesus had instructed them.  7 They brought the donkey and the colt and placed their cloaks on them, and he sat on them.  8 A very large crowd spread their cloaks on the road. Others cut branches from the trees and spread them on the road.  9 The crowds that went ahead of him and those following kept shouting, “Hosanna to the Son of David! Blessed is the one who comes in the name of the Lord! Hosanna in the highest!”  10 As he entered Jerusalem the whole city was thrown into an uproar, saying, “Who is this?”  11 And the crowds were saying, “This is the prophet Jesus, from Nazareth in Galilee.”  12 Then Jesus entered the temple area and drove out all those who were selling and buying in the temple courts and turned over the tables of the money changers and the chairs of those selling doves.  13 And he said to them, “It is written, ‘My house will be called a house of prayer,’ but you are turning it into a den of robbers!”  14 The blind and lame came to him in the temple courts, and he healed them.  15 But when the chief priests and the experts in the law saw the wonderful things he did and heard the children crying out in the temple courts, “Hosanna to the Son of David,” they became indignant  16 and said to him, “Do you hear what they are saying?” Jesus said to them, “Yes. Have you never read, ‘Out of the mouths of children and nursing infants you have prepared praise for yourself’?” 17 And leaving them, he went out of the city to Bethany and spent the night there.  18 Now early in the morning, as he returned to the city, he was hungry.  19 After noticing a fig tree by the road he went to it, but found nothing on it except leaves. He said to it, “Never again will there be fruit from you!” And the fig tree withered at once.  20 When the disciples saw it they were amazed, saying, “How did the fig tree wither so quickly?”  21 Jesus answered them, “I tell you the truth, if you have faith and do not doubt, not only will you do what was done to the fig tree, but even if you say to this mountain, ‘Be lifted up and thrown into the sea,’ it will happen.  22 And whatever you ask in prayer, if you believe, you will receive.”  23 Now after Jesus entered the temple courts, the chief priests and elders of the people came up to him as he was teaching and said, “By what authority are you doing these things, and who gave you this authority?”  24 Jesus answered them, “I will also ask you one question. If you answer me then I will also tell you by what authority I do these things.  25 Where did John’s baptism come from? From heaven or from people?” They discussed this among themselves, saying, “If we say, ‘From heaven,’ he will say, ‘Then why did you not believe him?’  26 But if we say, ‘From people,’ we fear the crowd, for they all consider John to be a prophet.”  27 So they answered Jesus, “We don’t know.” Then he said to them, “Neither will I tell you by what authority I am doing these things.  28 “What do you think? A man had two sons. He went to the first and said, ‘Son, go and work in the vineyard today.’  29 The boy answered, ‘I will not.’ But later he had a change of heart and went.  30 The father went to the other son and said the same thing. This boy answered, ‘I will, sir,’ but did not go.  31 Which of the two did his father’s will?” They said, “The first.” Jesus said to them, “I tell you the truth, tax collectors and prostitutes will go ahead of you into the kingdom of God!  32 For John came to you in the way of righteousness, and you did not believe him. But the tax collectors and prostitutes did believe. Although you saw this, you did not later change your minds and believe him.  33 “Listen to another parable: There was a landowner who planted a vineyard. He put a fence around it, dug a pit for its winepress, and built a watchtower. Then he leased it to tenant farmers and went on a journey.  34 When the harvest time was near, he sent his slaves to the tenants to collect his portion of the crop.  35 But the tenants seized his slaves, beat one, killed another, and stoned another. 36 Again he sent other slaves, more than the first, and they treated them the same way.  37 Finally he sent his son to them, saying, ‘They will respect my son.’  38 But when the tenants saw the son, they said to themselves, ‘This is the heir. Come, let’s kill him and get his inheritance!’  39 So they seized him, threw him out of the vineyard, and killed him.  40 Now when the owner of the vineyard comes, what will he do to those tenants?”  41 They said to him, “He will utterly destroy those evil men! Then he will lease the vineyard to other tenants who will give him his portion at the harvest.”  42 Jesus said to them, “Have you never read in the scriptures: “‘The stone the builders rejected has become the cornerstone. This is from the Lord, and it is marvelous in our eyes’?  43 “For this reason I tell you that the kingdom of God will be taken from you and given to a people who will produce its fruit.  44 The one who falls on this stone will be broken to pieces, and the one on whom it falls will be crushed.”  45 When the chief priests and the Pharisees heard his parables, they realized that he was speaking about them.  46 They wanted to arrest him, but they were afraid of the crowds because the crowds regarded him as a prophet.  ',\n",
       " '22:1 Jesus spoke to them again in parables, saying:  2 “The kingdom of heaven can be compared to a king who gave a wedding banquet for his son.  3 He sent his slaves to summon those who had been invited to the banquet, but they would not come.  4 Again he sent other slaves, saying, ‘Tell those who have been invited, “Look! The feast I have prepared for you is ready. My oxen and fattened cattle have been slaughtered, and everything is ready. Come to the wedding banquet.”’  5 But they were indifferent and went away, one to his farm, another to his business.  6 The rest seized his slaves, insolently mistreated them, and killed them. 7 The king was furious! He sent his soldiers, and they put those murderers to death and set their city on fire.  8 Then he said to his slaves, ‘The wedding is ready, but the ones who had been invited were not worthy.  9 So go into the main streets and invite everyone you find to the wedding banquet.’  10 And those slaves went out into the streets and gathered all they found, both bad and good, and the wedding hall was filled with guests.  11 But when the king came in to see the wedding guests, he saw a man there who was not wearing wedding clothes.  12 And he said to him, ‘Friend, how did you get in here without wedding clothes?’ But he had nothing to say.  13 Then the king said to his attendants, ‘Tie him up hand and foot and throw him into the outer darkness, where there will be weeping and gnashing of teeth!’  14 For many are called, but few are chosen.”  15 Then the Pharisees went out and planned together to entrap him with his own words.  16 They sent to him their disciples along with the Herodians, saying, “Teacher, we know that you are truthful and teach the way of God in accordance with the truth. You do not court anyone’s favor because you show no partiality. 17 Tell us then, what do you think? Is it right to pay taxes to Caesar or not?”  18 But Jesus realized their evil intentions and said, “Hypocrites! Why are you testing me?  19 Show me the coin used for the tax.” So they brought him a denarius.  20 Jesus said to them, “Whose image is this, and whose inscription?”  21 They replied, “Caesar’s.” He said to them, “Then give to Caesar the things that are Caesar’s, and to God the things that are God’s.” 22 Now when they heard this they were stunned, and they left him and went away.  23 The same day Sadducees (who say there is no resurrection) came to him and asked him,  24 “Teacher, Moses said, ‘If a man dies without having children, his brother must marry the widow and father children for his brother.’  25 Now there were seven brothers among us. The first one married and died, and since he had no children he left his wife to his brother.  26 The second did the same, and the third, down to the seventh.  27 Last of all, the woman died.  28 In the resurrection, therefore, whose wife of the seven will she be? For they all had married her.”  29 Jesus answered them, “You are deceived because you don’t know the scriptures or the power of God.  30 For in the resurrection they neither marry nor are given in marriage, but are like angels in heaven.  31 Now as for the resurrection of the dead, have you not read what was spoken to you by God,  32 ‘I am the God of Abraham, the God of Isaac, and the God of Jacob’? He is not the God of the dead but of the living!”  33 When the crowds heard this, they were amazed at his teaching.  34 Now when the Pharisees heard that he had silenced the Sadducees, they assembled together.  35 And one of them, an expert in religious law, asked him a question to test him:  36 “Teacher, which commandment in the law is the greatest?”  37 Jesus said to him, “‘Love the Lord your God with all your heart, with all your soul, and with all your mind.’  38 This is the first and greatest commandment.  39 The second is like it: ‘Love your neighbor as yourself.’  40 All the law and the prophets depend on these two commandments.”  41 While the Pharisees were assembled, Jesus asked them a question: 42 “What do you think about the Christ? Whose son is he?” They said, “The son of David.”  43 He said to them, “How then does David by the Spirit call him ‘Lord,’ saying,  44 “‘The Lord said to my lord, “Sit at my right hand,until I put your enemies under your feet”’?  45 If David then calls him ‘Lord,’ how can he be his son?”  46 No one was able to answer him a word, and from that day on no one dared to question him any longer.  ',\n",
       " '23:1 Then Jesus said to the crowds and to his disciples,  2 “The experts in the law and the Pharisees sit on Moses’ seat.  3 Therefore pay attention to what they tell you and do it. But do not do what they do, for they do not practice what they teach.  4 They tie up heavy loads, hard to carry, and put them on men’s shoulders, but they themselves are not willing even to lift a finger to move them.  5 They do all their deeds to be seen by people, for they make their phylacteries wide and their tassels long.  6 They love the place of honor at banquets and the best seats in the synagogues  7 and elaborate greetings in the marketplaces and to have people call them ‘Rabbi.’ 8 But you are not to be called ‘Rabbi,’ for you have one Teacher and you are all brothers.  9 And call no one your ‘father’ on earth, for you have one Father, who is in heaven.  10 Nor are you to be called ‘teacher,’ for you have one Teacher, the Christ.  11 The greatest among you will be your servant.  12 And whoever exalts himself will be humbled, and whoever humbles himself will be exalted.  13 “But woe to you, experts in the law and you Pharisees, hypocrites! You keep locking people out of the kingdom of heaven! For you neither enter nor permit those trying to enter to go in.  15 “Woe to you, experts in the law and you Pharisees, hypocrites! You cross land and sea to make one convert, and when you get one, you make him twice as much a child of hell as yourselves!  16 “Woe to you, blind guides, who say, ‘Whoever swears by the temple is bound by nothing. But whoever swears by the gold of the temple is bound by the oath.’ 17 Blind fools! Which is greater, the gold or the temple that makes the gold sacred?  18 And, ‘Whoever swears by the altar is bound by nothing. But if anyone swears by the gift on it he is bound by the oath.’ 19 You are blind! For which is greater, the gift or the altar that makes the gift sacred?  20 So whoever swears by the altar swears by it and by everything on it.  21 And whoever swears by the temple swears by it and the one who dwells in it.  22 And whoever swears by heaven swears by the throne of God and the one who sits on it.  23 “Woe to you, experts in the law and you Pharisees, hypocrites! You give a tenth of mint, dill, and cumin, yet you neglect what is more important in the law—justice, mercy, and faithfulness! You should have done these things without neglecting the others. 24 Blind guides! You strain out a gnat yet swallow a camel!  25 “Woe to you, experts in the law and you Pharisees, hypocrites! You clean the outside of the cup and the dish, but inside they are full of greed and self-indulgence.  26 Blind Pharisee! First clean the inside of the cup, so that the outside may become clean too! 27 “Woe to you, experts in the law and you Pharisees, hypocrites! You are like whitewashed tombs that look beautiful on the outside but inside are full of the bones of the dead and of everything unclean.  28 In the same way, on the outside you look righteous to people, but inside you are full of hypocrisy and lawlessness.  29 “Woe to you, experts in the law and you Pharisees, hypocrites! You build tombs for the prophets and decorate the graves of the righteous.  30 And you say, ‘If we had lived in the days of our ancestors, we would not have participated with them in shedding the blood of the prophets.’  31 By saying this you testify against yourselves that you are descendants of those who murdered the prophets.  32 Fill up then the measure of your ancestors!  33 You snakes, you offspring of vipers! How will you escape being condemned to hell?  34 “For this reason I am sending you prophets and wise men and experts in the law, some of whom you will kill and crucify, and some you will flog in your synagogues and pursue from town to town,  35 so that on you will come all the righteous blood shed on earth, from the blood of righteous Abel to the blood of Zechariah son of Barachiah, whom you murdered between the temple and the altar.  36 I tell you the truth, this generation will be held responsible for all these things!  37 “O Jerusalem, Jerusalem, you who kill the prophets and stone those who are sent to you! How often I have longed to gather your children together as a hen gathers her chicks under her wings, but you would have none of it! 38 Look, your house is left to you desolate!  39 For I tell you, you will not see me from now until you say, ‘Blessed is the one who comes in the name of the Lord!’”  ',\n",
       " '24:1 Now as Jesus was going out of the temple courts and walking away, his disciples came to show him the temple buildings.  2 And he said to them, “Do you see all these things? I tell you the truth, not one stone will be left on another. All will be torn down!”  3 As he was sitting on the Mount of Olives, his disciples came to him privately and said, “Tell us, when will these things happen? And what will be the sign of your coming and of the end of the age?”  4 Jesus answered them, “Watch out that no one misleads you.  5 For many will come in my name, saying, ‘I am the Christ,’ and they will mislead many.  6 You will hear of wars and rumors of wars. Make sure that you are not alarmed, for this must happen, but the end is still to come.  7 For nation will rise up in arms against nation, and kingdom against kingdom. And there will be famines and earthquakes in various places. 8 All these things are the beginning of birth pains.  9 “Then they will hand you over to be persecuted and will kill you. You will be hated by all the nations because of my name.  10 Then many will be led into sin, and they will betray one another and hate one another.  11 And many false prophets will appear and deceive many,  12 and because lawlessness will increase so much, the love of many will grow cold.  13 But the person who endures to the end will be saved.  14 And this gospel of the kingdom will be preached throughout the whole inhabited earth as a testimony to all the nations, and then the end will come.  15 “So when you see the abomination of desolation—spoken about by Daniel the prophet—standing in the holy place” (let the reader understand),  16 “then those in Judea must flee to the mountains.  17 The one on the roof must not come down to take anything out of his house,  18 and the one in the field must not turn back to get his cloak.  19 Woe to those who are pregnant and to those who are nursing their babies in those days!  20 Pray that your flight may not be in winter or on a Sabbath.  21 For then there will be great suffering unlike anything that has happened from the beginning of the world until now, or ever will happen.  22 And if those days had not been cut short, no one would be saved. But for the sake of the elect those days will be cut short.  23 Then if anyone says to you, ‘Look, here is the Christ!’ or ‘There he is!’ do not believe him. 24 For false messiahs and false prophets will appear and perform great signs and wonders to deceive, if possible, even the elect.  25 Remember, I have told you ahead of time.  26 So then, if someone says to you, ‘Look, he is in the wilderness,’ do not go out, or ‘Look, he is in the inner rooms,’ do not believe him. 27 For just like the lightning comes from the east and flashes to the west, so the coming of the Son of Man will be.  28 Wherever the corpse is, there the vultures will gather.  29 “Immediately after the suffering of those days, the sun will be darkened, and the moon will not give its light; the stars will fall from heaven, and the powers of heaven will be shaken.  30 Then the sign of the Son of Man will appear in heaven, and all the tribes of the earth will mourn. They will see the Son of Man arriving on the clouds of heaven with power and great glory.  31 And he will send his angels with a loud trumpet blast, and they will gather his elect from the four winds, from one end of heaven to the other.  32 “Learn this parable from the fig tree: Whenever its branch becomes tender and puts out its leaves, you know that summer is near.  33 So also you, when you see all these things, know that he is near, right at the door.  34 I tell you the truth, this generation will not pass away until all these things take place.  35 Heaven and earth will pass away, but my words will never pass away.  36 “But as for that day and hour no one knows it—not even the angels in heaven—except the Father alone.  37 For just like the days of Noah were, so the coming of the Son of Man will be.  38 For in those days before the flood, people were eating and drinking, marrying and giving in marriage, until the day Noah entered the ark.  39 And they knew nothing until the flood came and took them all away. It will be the same at the coming of the Son of Man.  40 Then there will be two men in the field; one will be taken and one left.  41 There will be two women grinding grain with a mill; one will be taken and one left.  42 “Therefore stay alert because you do not know on what day your Lord will come.  43 But understand this: If the owner of the house had known at what time of night the thief was coming, he would have been alert and would not have let his house be broken into.  44 Therefore you also must be ready because the Son of Man will come at an hour when you do not expect him. 45 “Who then is the faithful and wise slave, whom the master has put in charge of his household, to give the other slaves their food at the proper time?  46 Blessed is that slave whom the master finds at work when he comes.  47 I tell you the truth, the master will put him in charge of all his possessions.  48 But if that evil slave should say to himself, ‘My master is staying away a long time,’  49 and he begins to beat his fellow slaves and to eat and drink with drunkards,  50 then the master of that slave will come on a day when he does not expect him and at an hour he does not foresee,  51 and will cut him in two, and assign him a place with the hypocrites, where there will be weeping and gnashing of teeth.  ',\n",
       " '25:1 “At that time the kingdom of heaven will be like 10 virgins who took their lamps and went out to meet the bridegroom.  2 Five of the virgins were foolish, and five were wise.  3 When the foolish ones took their lamps, they did not take extra olive oil with them.  4 But the wise ones took flasks of olive oil with their lamps.  5 When the bridegroom was delayed a long time, they all became drowsy and fell asleep.  6 But at midnight there was a shout, ‘Look, the bridegroom is here! Come out to meet him.’ 7 Then all the virgins woke up and trimmed their lamps.  8 The foolish ones said to the wise, ‘Give us some of your oil because our lamps are going out.’  9 ‘No,’ they replied. ‘There won’t be enough for you and for us. Go instead to those who sell oil and buy some for yourselves.’  10 But while they had gone to buy it, the bridegroom arrived, and those who were ready went inside with him to the wedding banquet. Then the door was shut.  11 Later, the other virgins came too, saying, ‘Lord, lord! Let us in!’  12 But he replied, ‘I tell you the truth, I do not know you!’  13 Therefore stay alert because you do not know the day or the hour.  14 “For it is like a man going on a journey, who summoned his slaves and entrusted his property to them.  15 To one he gave five talents, to another two, and to another one, each according to his ability. Then he went on his journey.  16 The one who had received five talents went off right away and put his money to work and gained five more.  17 In the same way, the one who had two gained two more.  18 But the one who had received one talent went out and dug a hole in the ground and hid his master’s money in it. 19 After a long time, the master of those slaves came and settled his accounts with them.  20 The one who had received the five talents came and brought five more, saying, ‘Sir, you entrusted me with five talents. See, I have gained five more.’  21 His master answered, ‘Well done, good and faithful slave! You have been faithful in a few things. I will put you in charge of many things. Enter into the joy of your master.’  22 The one with the two talents also came and said, ‘Sir, you entrusted two talents to me. See, I have gained two more.’  23 His master answered, ‘Well done, good and faithful slave! You have been faithful with a few things. I will put you in charge of many things. Enter into the joy of your master.’  24 Then the one who had received the one talent came and said, ‘Sir, I knew that you were a hard man, harvesting where you did not sow, and gathering where you did not scatter seed,  25 so I was afraid, and I went and hid your talent in the ground. See, you have what is yours.’  26 But his master answered, ‘Evil and lazy slave! So you knew that I harvest where I didn’t sow and gather where I didn’t scatter?  27 Then you should have deposited my money with the bankers, and on my return I would have received my money back with interest!  28 Therefore take the talent from him and give it to the one who has 10.  29 For the one who has will be given more, and he will have more than enough. But the one who does not have, even what he has will be taken from him.  30 And throw that worthless slave into the outer darkness, where there will be weeping and gnashing of teeth.’  31 “When the Son of Man comes in his glory and all the angels with him, then he will sit on his glorious throne.  32 All the nations will be assembled before him, and he will separate people one from another like a shepherd separates the sheep from the goats.  33 He will put the sheep on his right and the goats on his left.  34 Then the king will say to those on his right, ‘Come, you who are blessed by my Father, inherit the kingdom prepared for you from the foundation of the world.  35 For I was hungry and you gave me food, I was thirsty and you gave me something to drink, I was a stranger and you invited me in, 36 I was naked and you gave me clothing, I was sick and you took care of me, I was in prison and you visited me.’  37 Then the righteous will answer him, ‘Lord, when did we see you hungry and feed you, or thirsty and give you something to drink?  38 When did we see you a stranger and invite you in, or naked and clothe you?  39 When did we see you sick or in prison and visit you?’  40 And the king will answer them, ‘I tell you the truth, just as you did it for one of the least of these brothers or sisters of mine, you did it for me.’  41 “Then he will say to those on his left, ‘Depart from me, you accursed, into the eternal fire that has been prepared for the devil and his angels!  42 For I was hungry and you gave me nothing to eat, I was thirsty and you gave me nothing to drink.  43 I was a stranger and you did not receive me as a guest, naked and you did not clothe me, sick and in prison and you did not visit me.’  44 Then they too will answer, ‘Lord, when did we see you hungry or thirsty or a stranger or naked or sick or in prison, and did not give you whatever you needed?’  45 Then he will answer them, ‘I tell you the truth, just as you did not do it for one of the least of these, you did not do it for me.’  46 And these will depart into eternal punishment, but the righteous into eternal life.” ',\n",
       " '26:1 When Jesus had finished saying all these things, he told his disciples,  2 “You know that after two days the Passover is coming, and the Son of Man will be handed over to be crucified.”  3 Then the chief priests and the elders of the people met together in the palace of the high priest, who was named Caiaphas.  4 They planned to arrest Jesus by stealth and kill him. 5 But they said, “Not during the feast, so that there won’t be a riot among the people.”  6 Now while Jesus was in Bethany at the house of Simon the leper,  7 a woman came to him with an alabaster jar of expensive perfumed oil, and she poured it on his head as he was at the table.  8 When the disciples saw this, they became indignant and said, “Why this waste?  9 It could have been sold at a high price and the money given to the poor!”  10 When Jesus learned of this, he said to them, “Why are you bothering this woman? She has done a good service for me.  11 For you will always have the poor with you, but you will not always have me!  12 When she poured this oil on my body, she did it to prepare me for burial.  13 I tell you the truth, wherever this gospel is proclaimed in the whole world, what she has done will also be told in memory of her.”  14 Then one of the twelve, the one named Judas Iscariot, went to the chief priests  15 and said, “What will you give me to betray him into your hands?” So they set out 30 silver coins for him.  16 From that time on, Judas began looking for an opportunity to betray him.  17 Now on the first day of the Feast of Unleavened Bread the disciples came to Jesus and said, “Where do you want us to prepare for you to eat the Passover?”  18 He said, “Go into the city to a certain man and tell him, ‘The Teacher says, “My time is near. I will observe the Passover with my disciples at your house.”’”  19 So the disciples did as Jesus had instructed them, and they prepared the Passover.  20 When it was evening, he took his place at the table with the twelve.  21 And while they were eating he said, “I tell you the truth, one of you will betray me.”  22 They became greatly distressed and each one began to say to him, “Surely not I, Lord?”  23 He answered, “The one who has dipped his hand into the bowl with me will betray me.  24 The Son of Man will go as it is written about him, but woe to that man by whom the Son of Man is betrayed! It would be better for him if he had never been born.”  25 Then Judas, the one who would betray him, said, “Surely not I, Rabbi?” Jesus replied, “You have said it yourself.” 26 While they were eating, Jesus took bread, and after giving thanks he broke it, gave it to his disciples, and said, “Take, eat, this is my body.”  27 And after taking the cup and giving thanks, he gave it to them, saying, “Drink from it, all of you,  28 for this is my blood, the blood of the covenant, that is poured out for many for the forgiveness of sins.  29 I tell you, from now on I will not drink of this fruit of the vine until that day when I drink it new with you in my Father’s kingdom.”  30 After singing a hymn, they went out to the Mount of Olives.  31 Then Jesus said to them, “This night you will all fall away because of me, for it is written: “‘I will strike the shepherd,and the sheep of the flock will be scattered.’  32 But after I am raised, I will go ahead of you into Galilee.”  33 Peter said to him, “If they all fall away because of you, I will never fall away!”  34 Jesus said to him, “I tell you the truth, on this night, before the rooster crows, you will deny me three times.”  35 Peter said to him, “Even if I must die with you, I will never deny you.” And all the disciples said the same thing.  36 Then Jesus went with them to a place called Gethsemane, and he said to the disciples, “Sit here while I go over there and pray.”  37 He took with him Peter and the two sons of Zebedee, and he became anguished and distressed.  38 Then he said to them, “My soul is deeply grieved, even to the point of death. Remain here and stay awake with me.”  39 Going a little farther, he threw himself down with his face to the ground and prayed, “My Father, if possible, let this cup pass from me! Yet not what I will, but what you will.”  40 Then he came to the disciples and found them sleeping. He said to Peter, “So, couldn’t you stay awake with me for one hour?  41 Stay awake and pray that you will not fall into temptation. The spirit is willing, but the flesh is weak.”  42 He went away a second time and prayed, “My Father, if this cup cannot be taken away unless I drink it, your will must be done.”  43 He came again and found them sleeping; they could not keep their eyes open.  44 So leaving them again, he went away and prayed for the third time, saying the same thing once more.  45 Then he came to the disciples and said to them, “Are you still sleeping and resting? Look, the hour is approaching, and the Son of Man is betrayed into the hands of sinners.  46 Get up, let us go. Look! My betrayer is approaching!”  47 While he was still speaking, Judas, one of the twelve, arrived. With him was a large crowd armed with swords and clubs, sent by the chief priests and elders of the people.  48 (Now the betrayer had given them a sign, saying, “The one I kiss is the man. Arrest him!”)  49 Immediately he went up to Jesus and said, “Greetings, Rabbi,” and kissed him.  50 Jesus said to him, “Friend, do what you are here to do.” Then they came and took hold of Jesus and arrested him.  51 But one of those with Jesus grabbed his sword, drew it out, and struck the high priest’s slave, cutting off his ear.  52 Then Jesus said to him, “Put your sword back in its place! For all who take hold of the sword will die by the sword.  53 Or do you think that I cannot call on my Father and that he would send me more than 12 legions of angels right now?  54 How then would the scriptures that say it must happen this way be fulfilled?”  55 At that moment Jesus said to the crowd, “Have you come out with swords and clubs to arrest me like you would an outlaw? Day after day I sat teaching in the temple courts, yet you did not arrest me.  56 But this has happened so that the scriptures of the prophets would be fulfilled.” Then all the disciples left him and fled.  57 Now the ones who had arrested Jesus led him to Caiaphas, the high priest, in whose house the experts in the law and the elders had gathered.  58 But Peter was following him from a distance, all the way to the high priest’s courtyard. After going in, he sat with the guards to see the outcome.  59 The chief priests and the whole Sanhedrin were trying to find false testimony against Jesus so that they could put him to death.  60 But they did not find anything, though many false witnesses came forward. Finally two came forward  61 and declared, “This man said, ‘I am able to destroy the temple of God and rebuild it in three days.’”  62 So the high priest stood up and said to him, “Have you no answer? What is this that they are testifying against you?”  63 But Jesus was silent. The high priest said to him, “I charge you under oath by the living God, tell us if you are the Christ, the Son of God.”  64 Jesus said to him, “You have said it yourself. But I tell you, from now on you will see the Son of Man sitting at the right hand of the Power and coming on the clouds of heaven.”  65 Then the high priest tore his clothes and declared, “He has blasphemed! Why do we still need witnesses? Now you have heard the blasphemy!  66 What is your verdict?” They answered, “He is guilty and deserves death.”  67 Then they spat in his face and struck him with their fists. And some slapped him, 68 saying, “Prophesy for us, you Christ! Who hit you?”  69 Now Peter was sitting outside in the courtyard. A slave girl came to him and said, “You also were with Jesus the Galilean.”  70 But he denied it in front of them all: “I don’t know what you’re talking about!”  71 When he went out to the gateway, another slave girl saw him and said to the people there, “This man was with Jesus the Nazarene.”  72 He denied it again with an oath, “I do not know the man!”  73 After a little while, those standing there came up to Peter and said, “You really are one of them too—even your accent gives you away!”  74 At that he began to curse, and he swore with an oath, “I do not know the man!” At that moment a rooster crowed.  75 Then Peter remembered what Jesus had said: “Before the rooster crows, you will deny me three times.” And he went outside and wept bitterly.  ',\n",
       " '27:1 When it was early in the morning, all the chief priests and the elders of the people plotted against Jesus to execute him.  2 They tied him up, led him away, and handed him over to Pilate the governor.  3 Now when Judas, who had betrayed him, saw that Jesus had been condemned, he regretted what he had done and returned the 30 silver coins to the chief priests and the elders,  4 saying, “I have sinned by betraying innocent blood!” But they said, “What is that to us? You take care of it yourself!” 5 So Judas threw the silver coins into the temple and left. Then he went out and hanged himself.  6 The chief priests took the silver and said, “It is not lawful to put this into the temple treasury, since it is blood money.”  7 After consulting together they bought the Potter’s Field with it, as a burial place for foreigners.  8 For this reason that field has been called the “Field of Blood” to this day.  9 Then what was spoken by Jeremiah the prophet was fulfilled: “They took the 30 silver coins, the price of the one whose price had been set by the people of Israel,  10 and they gave them for the potter’s field, as the Lord commanded me.”  11 Then Jesus stood before the governor, and the governor asked him, “Are you the king of the Jews?” Jesus said, “You say so.” 12 But when he was accused by the chief priests and the elders, he did not respond.  13 Then Pilate said to him, “Don’t you hear how many charges they are bringing against you?”  14 But he did not answer even one accusation, so that the governor was quite amazed.  15 During the feast the governor was accustomed to release one prisoner to the crowd, whomever they wanted.  16 At that time they had in custody a notorious prisoner named Jesus Barabbas.  17 So after they had assembled, Pilate said to them, “Whom do you want me to release for you, Jesus Barabbas or Jesus who is called the Christ?”  18 (For he knew that they had handed him over because of envy.)  19 As he was sitting on the judgment seat, his wife sent a message to him: “Have nothing to do with that innocent man; I have suffered greatly as a result of a dream about him today.”  20 But the chief priests and the elders persuaded the crowds to ask for Barabbas and to have Jesus killed.  21 The governor asked them, “Which of the two do you want me to release for you?” And they said, “Barabbas!” 22 Pilate said to them, “Then what should I do with Jesus who is called the Christ?” They all said, “Crucify him!” 23 He asked, “Why? What wrong has he done?” But they shouted more insistently, “Crucify him!” 24 When Pilate saw that he could do nothing, but that instead a riot was starting, he took some water, washed his hands before the crowd and said, “I am innocent of this man’s blood. You take care of it yourselves!”  25 In reply all the people said, “Let his blood be on us and on our children!”  26 Then he released Barabbas for them. But after he had Jesus flogged, he handed him over to be crucified.  27 Then the governor’s soldiers took Jesus into the governor’s residence and gathered the whole cohort around him.  28 They stripped him and put a scarlet robe around him,  29 and after braiding a crown of thorns, they put it on his head. They put a staff in his right hand, and kneeling down before him, they mocked him: “Hail, king of the Jews!”  30 They spat on him and took the staff and struck him repeatedly on the head.  31 When they had mocked him, they stripped him of the robe and put his own clothes back on him. Then they led him away to crucify him.  32 As they were going out, they found a man from Cyrene named Simon, whom they forced to carry his cross.  33 They came to a place called Golgotha (which means “Place of the Skull”)  34 and offered Jesus wine mixed with gall to drink. But after tasting it, he would not drink it. 35 When they had crucified him, they divided his clothes by throwing dice.  36 Then they sat down and kept guard over him there.  37 Above his head they put the charge against him, which read: “This is Jesus, the king of the Jews.”  38 Then two outlaws were crucified with him, one on his right and one on his left.  39 Those who passed by defamed him, shaking their heads  40 and saying, “You who can destroy the temple and rebuild it in three days, save yourself! If you are God’s Son, come down from the cross!”  41 In the same way even the chief priests—together with the experts in the law and elders—were mocking him: 42 “He saved others, but he cannot save himself! He is the king of Israel! If he comes down now from the cross, we will believe in him!  43 He trusts in God—let God, if he wants to, deliver him now because he said, ‘I am God’s Son’!”  44 The robbers who were crucified with him also spoke abusively to him.  45 Now from noon until three, darkness came over all the land.  46 At about three o’clock Jesus shouted with a loud voice, “Eli, Eli, lema sabachthani?” that is, “My God, my God, why have you forsaken me?”  47 When some of the bystanders heard it, they said, “This man is calling for Elijah.”  48 Immediately one of them ran and got a sponge, filled it with sour wine, put it on a stick, and gave it to him to drink. 49 But the rest said, “Leave him alone! Let’s see if Elijah will come to save him.”  50 Then Jesus cried out again with a loud voice and gave up his spirit.  51 Just then the temple curtain was torn in two, from top to bottom. The earth shook and the rocks were split apart.  52 And tombs were opened, and the bodies of many saints who had died were raised.  53 (They came out of the tombs after his resurrection and went into the holy city and appeared to many people.) 54 Now when the centurion and those with him who were guarding Jesus saw the earthquake and what took place, they were extremely terrified and said, “Truly this one was God’s Son!”  55 Many women who had followed Jesus from Galilee and given him support were also there, watching from a distance.  56 Among them were Mary Magdalene, Mary the mother of James and Joseph, and the mother of the sons of Zebedee.  57 Now when it was evening, there came a rich man from Arimathea, named Joseph, who was also a disciple of Jesus.  58 He went to Pilate and asked for the body of Jesus. Then Pilate ordered that it be given to him. 59 Joseph took the body, wrapped it in a clean linen cloth,  60 and placed it in his own new tomb that he had cut in the rock. Then he rolled a great stone across the entrance of the tomb and went away.  61 (Now Mary Magdalene and the other Mary were sitting there, opposite the tomb.)  62 The next day (which is after the day of preparation) the chief priests and the Pharisees assembled before Pilate  63 and said, “Sir, we remember that while that deceiver was still alive he said, ‘After three days I will rise again.’ 64 So give orders to secure the tomb until the third day. Otherwise his disciples may come and steal his body and say to the people, ‘He has been raised from the dead,’ and the last deception will be worse than the first.”  65 Pilate said to them, “Take a guard of soldiers. Go and make it as secure as you can.”  66 So they went with the soldiers of the guard and made the tomb secure by sealing the stone.  ',\n",
       " '28:1 Now after the Sabbath, at dawn on the first day of the week, Mary Magdalene and the other Mary went to look at the tomb.  2 Suddenly there was a severe earthquake, for an angel of the Lord descending from heaven came and rolled away the stone and sat on it.  3 His appearance was like lightning, and his clothes were white as snow.  4 The guards were shaken and became like dead men because they were so afraid of him.  5 But the angel said to the women, “Do not be afraid; I know that you are looking for Jesus, who was crucified.  6 He is not here, for he has been raised, just as he said. Come and see the place where he was lying.  7 Then go quickly and tell his disciples, ‘He has been raised from the dead. He is going ahead of you into Galilee. You will see him there.’ Listen, I have told you!”  8 So they left the tomb quickly, with fear and great joy, and ran to tell his disciples.  9 But Jesus met them, saying, “Greetings!” They came to him, held on to his feet and worshiped him.  10 Then Jesus said to them, “Do not be afraid. Go and tell my brothers to go to Galilee. They will see me there.”  11 While they were going, some of the guards went into the city and told the chief priests everything that had happened.  12 After they had assembled with the elders and formed a plan, they gave a large sum of money to the soldiers,  13 telling them, “You are to say, ‘His disciples came at night and stole his body while we were asleep.’  14 If this matter is heard before the governor, we will satisfy him and keep you out of trouble.”  15 So they took the money and did as they were instructed. And this story is told among the Jews to this day.  16 So the 11 disciples went to Galilee to the mountain Jesus had designated.  17 When they saw him, they worshiped him, but some doubted.  18 Then Jesus came up and said to them, “All authority in heaven and on earth has been given to me.  19 Therefore go and make disciples of all nations, baptizing them in the name of the Father and the Son and the Holy Spirit,  20 teaching them to obey everything I have commanded you. And remember, I am with you always, to the end of the age.”  ']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Notice that our dictionary is currently in key: book, value: list of text format\n",
    "next(iter(data.values()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "kgyJU30d-f-5"
   },
   "outputs": [],
   "source": [
    "# We are going to change this to key: book, value: string format\n",
    "def combine_text(list_of_text):\n",
    "    \"\"\"Takes a list of text and combines them into one large chunk of text.\"\"\"\n",
    "    combined_text = ' '.join(list_of_text)\n",
    "    return combined_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "XzNXACG0-f-5"
   },
   "outputs": [],
   "source": [
    "# Combine it!\n",
    "data_combined = {key: [combine_text(value)] for (key, value) in data.items()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "ZnzzvT5z-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                             book_text\n",
       "Matthew          1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...\n",
       "Mark             1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...\n",
       "Luke             1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...\n",
       "John             1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...\n",
       "Acts             1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...\n",
       "Romans           1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...\n",
       "1 Corinthians    1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...\n",
       "2 Corinthians    1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...\n",
       "Galatians        1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...\n",
       "Ephesians        1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...\n",
       "Philippians      1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...\n",
       "Colossians       1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...\n",
       "1 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...\n",
       "2 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...\n",
       "1 Timothy        1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...\n",
       "2 Timothy        1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...\n",
       "Titus            1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...\n",
       "Philemon         1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...\n",
       "Hebrews          1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...\n",
       "James            1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...\n",
       "1 Peter          1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...\n",
       "2 Peter          1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...\n",
       "1 John           1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...\n",
       "2 John           1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...\n",
       "3 John           1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...\n",
       "Jude             1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...\n",
       "Revelation       1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h..."
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can either keep it in dictionary format or put it into a pandas dataframe\n",
    "import pandas as pd\n",
    "pd.set_option('max_colwidth', 150)\n",
    "\n",
    "data_df = pd.DataFrame.from_dict(data_combined).transpose()\n",
    "data_df.columns = ['book_text']\n",
    "data_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "_jumADbG-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Christ.  2 May mercy, peace, and love be lavished on you!  3 Dear friends, although I have been eager to write to you about our common salvation, I now feel compelled instead to write to encourage you to contend earnestly for the faith that was once for all entrusted to the saints.  4 For certain men have secretly slipped in among you—men who long ago were marked out for the condemnation I am about to describe—ungodly men who have turned the grace of our God into a license for evil and who deny our only Master and Lord, Jesus Christ.  5 Now I desire to remind you (even though you have been fully informed of these facts once for all) that Jesus, having saved the people out of the land of Egypt, later destroyed those who did not believe.  6 You also know that the angels who did not keep within their proper domain but abandoned their own place of residence, he has kept in eternal chains in utter darkness, locked up for the judgment of the great Day.  7 So also Sodom and Gomorrah and the neighboring towns, since they indulged in sexual immorality and pursued unnatural desire in a way similar to these angels, are now displayed as an example by suffering the punishment of eternal fire.  8 Yet these men, as a result of their dreams, defile the flesh, reject authority, and insult the glorious ones.  9 But even when Michael the archangel was arguing with the devil and debating with him concerning Moses’ body, he did not dare to bring a slanderous judgment, but said, “May the Lord rebuke you!”  10 But these men do not understand the things they slander, and they are being destroyed by the very things that, like irrational animals, they instinctively comprehend.  11 Woe to them! For they have traveled down Cain’s path, and because of greed have abandoned themselves to Balaam’s error; hence, they will certainly perish in Korah’s rebellion.  12 These men are dangerous reefs at your love feasts, feasting without reverence, feeding only themselves. They are waterless clouds, carried along by the winds; autumn trees without fruit—twice dead, uprooted;  13 wild sea waves, spewing out the foam of their shame; wayward stars for whom the utter depths of eternal darkness have been reserved.  14 Now Enoch, the seventh in descent beginning with Adam, even prophesied of them, saying, “Look! The Lord is coming with thousands and thousands of his holy ones,  15 to execute judgment on all, and to convict every person of all their thoroughly ungodly deeds that they have committed, and of all the harsh words that ungodly sinners have spoken against him.”  16 These people are grumblers and fault-finders who go wherever their desires lead them, and they give bombastic speeches, enchanting folks for their own gain.  17 But you, dear friends—recall the predictions foretold by the apostles of our Lord Jesus Christ.  18 For they said to you, “At the end of time there will come scoffers, propelled by their own ungodly desires.”  19 These people are divisive, worldly, devoid of the Spirit.  20 But you, dear friends, by building yourselves up in your most holy faith, by praying in the Holy Spirit,  21 maintain yourselves in the love of God while anticipating the mercy of our Lord Jesus Christ that brings eternal life.  22 And have mercy on those who waver;  23 save others by snatching them out of the fire; have mercy on others, coupled with a fear of God, hating even the clothes stained by the flesh.  24 Now to the one who is able to keep you from falling, and to cause you to stand, rejoicing, without blemish before his glorious presence,  25 to the only God our Savior through Jesus Christ our Lord, be glory, majesty, power, and authority, before all time, and now, and for all eternity. Amen.  '"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at the text of Jude\n",
    "data_df.book_text.loc['Jude']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "OJdXKnGl-f-5"
   },
   "outputs": [],
   "source": [
    "# First round of cleaning:\n",
    "# Remove the chapter and verse numbering (and any other numbers)\n",
    "import re\n",
    "import string\n",
    "\n",
    "def remove_chap_verse_numbering(text):\n",
    "    \"\"\"Remove chapter & verse numbering and newlines\"\"\"\n",
    "    text = re.sub('[0-9]+:[0-9]+', '', text)  # remove all chapter:verse references\n",
    "    text = re.sub('[0-9]+', '', text)         # remove all remaining numbers\n",
    "    text = re.sub('\\n', '', text)             # remove newlines\n",
    "    return text\n",
    "\n",
    "round1 = lambda x: remove_chap_verse_numbering(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "g4QNQ8wE-f-5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.   Abraham was the father of Isaac, Isaac the father of...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>The beginning of the gospel of Jesus Christ, the Son of God.  As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead of y...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>Now many have undertaken to compile an account of the things that have been fulfilled among us,   like the accounts passed on to us by those who ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>In the beginning was the Word, and the Word was with God, and the Word was fully God.   The Word was with God in the beginning.   All things were...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>I wrote the former account, Theophilus, about all that Jesus began to do and teach   until the day he was taken up to heaven, after he had given ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.   This gospel he promised beforehand through his pr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,   to the church of God that is in Corinth, to ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saints w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)   and all the brot...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.   Grace and peace to you from...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.   Grace and...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,   to the saints, the faithful brothers and sisters in Christ, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!   We ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.   Grace and peace to you from...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,   to Timothy, my genuine child in the faith....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,   to Timothy, my dear child. Grace, mer...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in keepin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,   to Apphia our sister, to Archippus ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>After God spoke long ago in various portions and in various ways to our ancestors through the prophets,   in these last days he has spoken to us ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>From James, a slave of God and the Lord Jesus Christ, to the  tribes dispersed abroad. Greetings!   My brothers and sisters, consider it nothing ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithynia)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been gra...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and our...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),   because of th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>From the elder, to Gaius my dear brother, whom I love in truth.   Dear friend, I pray that all may go well with you and that you may be in good h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Christ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to his ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                             book_text\n",
       "Matthew           This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.   Abraham was the father of Isaac, Isaac the father of...\n",
       "Mark              The beginning of the gospel of Jesus Christ, the Son of God.  As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead of y...\n",
       "Luke              Now many have undertaken to compile an account of the things that have been fulfilled among us,   like the accounts passed on to us by those who ...\n",
       "John              In the beginning was the Word, and the Word was with God, and the Word was fully God.   The Word was with God in the beginning.   All things were...\n",
       "Acts              I wrote the former account, Theophilus, about all that Jesus began to do and teach   until the day he was taken up to heaven, after he had given ...\n",
       "Romans            From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.   This gospel he promised beforehand through his pr...\n",
       "1 Corinthians     From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,   to the church of God that is in Corinth, to ...\n",
       "2 Corinthians     From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saints w...\n",
       "Galatians         From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)   and all the brot...\n",
       "Ephesians         From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.   Grace and peace to you from...\n",
       "Philippians       From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.   Grace and...\n",
       "Colossians        From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,   to the saints, the faithful brothers and sisters in Christ, ...\n",
       "1 Thessalonians   From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!   We ...\n",
       "2 Thessalonians   From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.   Grace and peace to you from...\n",
       "1 Timothy         From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,   to Timothy, my genuine child in the faith....\n",
       "2 Timothy         From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,   to Timothy, my dear child. Grace, mer...\n",
       "Titus             From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in keepin...\n",
       "Philemon          From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,   to Apphia our sister, to Archippus ...\n",
       "Hebrews           After God spoke long ago in various portions and in various ways to our ancestors through the prophets,   in these last days he has spoken to us ...\n",
       "James             From James, a slave of God and the Lord Jesus Christ, to the  tribes dispersed abroad. Greetings!   My brothers and sisters, consider it nothing ...\n",
       "1 Peter           From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithynia)...\n",
       "2 Peter           From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been gra...\n",
       "1 John            This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and our...\n",
       "2 John            From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),   because of th...\n",
       "3 John            From the elder, to Gaius my dear brother, whom I love in truth.   Dear friend, I pray that all may go well with you and that you may be in good h...\n",
       "Jude              From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Christ...\n",
       "Revelation        The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to his ..."
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at the updated text\n",
    "data_clean = pd.DataFrame(data_df.book_text.apply(round1))\n",
    "data_clean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "N5TkjJPp-f-5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Apply a second round of cleaning:\n",
    "# Make lowercase and remove punctuation\n",
    "import re\n",
    "import string\n",
    "\n",
    "def lcase_and_remove_punctuation(text):\n",
    "    \"\"\"Make text lowercase and remove punctuation\"\"\"\n",
    "    text = text.lower()\n",
    "    text = re.sub(f'[{re.escape(string.punctuation)}]', '', text)\n",
    "    return text\n",
    "\n",
    "round2 = lambda x: lcase_and_remove_punctuation(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "sGzqpx2K-f-6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>this is the record of the genealogy of jesus christ the son of david the son of abraham   abraham was the father of isaac isaac the father of jac...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>the beginning of the gospel of jesus christ the son of god  as it is written in the prophet isaiah “look i am sending my messenger ahead of youwh...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>now many have undertaken to compile an account of the things that have been fulfilled among us   like the accounts passed on to us by those who w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>in the beginning was the word and the word was with god and the word was fully god   the word was with god in the beginning   all things were cre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>i wrote the former account theophilus about all that jesus began to do and teach   until the day he was taken up to heaven after he had given ord...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>from paul a slave of christ jesus called to be an apostle set apart for the gospel of god   this gospel he promised beforehand through his prophe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>from paul called to be an apostle of christ jesus by the will of god and sosthenes our brother   to the church of god that is in corinth to those...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>from paul an apostle of christ jesus by the will of god and timothy our brother to the church of god that is in corinth with all the saints who a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>from paul an apostle not from men nor by human agency but by jesus christ and god the father who raised him from the dead   and all the brothers ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>from paul an apostle of christ jesus by the will of god to the saints in ephesus the faithful in christ jesus   grace and peace to you from god o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>from paul and timothy slaves of christ jesus to all the saints in christ jesus who are in philippi with the overseers and deacons   grace and pea...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>from paul an apostle of christ jesus by the will of god and timothy our brother   to the saints the faithful brothers and sisters in christ at co...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>from paul and silvanus and timothy to the church of the thessalonians in god the father and the lord jesus christ grace and peace to you   we tha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>from paul and silvanus and timothy to the church of the thessalonians in god our father and the lord jesus christ   grace and peace to you from g...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>from paul an apostle of christ jesus by the command of god our savior and of christ jesus our hope   to timothy my genuine child in the faith gra...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>from paul an apostle of christ jesus by the will of god to further the promise of life in christ jesus   to timothy my dear child grace mercy and...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>from paul a slave of god and apostle of jesus christ to further the faith of god’s chosen ones and the knowledge of the truth that is in keeping ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>from paul a prisoner of christ jesus and timothy our brother to philemon our dear friend and colaborer   to apphia our sister to archippus our fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>after god spoke long ago in various portions and in various ways to our ancestors through the prophets   in these last days he has spoken to us i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>from james a slave of god and the lord jesus christ to the  tribes dispersed abroad greetings   my brothers and sisters consider it nothing but j...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>from peter an apostle of jesus christ to those temporarily residing abroad in pontus galatia cappadocia the province of asia and bithynia who are...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>from simeon peter a slave and apostle of jesus christ to those who through the righteousness of our god and savior jesus christ have been granted...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>this is what we proclaim to you what was from the beginning what we have heard what we have seen with our eyes what we have looked at and our han...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>from the elder to an elect lady and her children whom i love in truth and not i alone but also all those who know the truth   because of the trut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>from the elder to gaius my dear brother whom i love in truth   dear friend i pray that all may go well with you and that you may be in good healt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>from jude a slave of jesus christ and brother of james to those who are called wrapped in the love of god the father and kept for jesus christ   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>the revelation of jesus christ which god gave him to show his servants what must happen very soon he made it clear by sending his angel to his se...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                             book_text\n",
       "Matthew           this is the record of the genealogy of jesus christ the son of david the son of abraham   abraham was the father of isaac isaac the father of jac...\n",
       "Mark              the beginning of the gospel of jesus christ the son of god  as it is written in the prophet isaiah “look i am sending my messenger ahead of youwh...\n",
       "Luke              now many have undertaken to compile an account of the things that have been fulfilled among us   like the accounts passed on to us by those who w...\n",
       "John              in the beginning was the word and the word was with god and the word was fully god   the word was with god in the beginning   all things were cre...\n",
       "Acts              i wrote the former account theophilus about all that jesus began to do and teach   until the day he was taken up to heaven after he had given ord...\n",
       "Romans            from paul a slave of christ jesus called to be an apostle set apart for the gospel of god   this gospel he promised beforehand through his prophe...\n",
       "1 Corinthians     from paul called to be an apostle of christ jesus by the will of god and sosthenes our brother   to the church of god that is in corinth to those...\n",
       "2 Corinthians     from paul an apostle of christ jesus by the will of god and timothy our brother to the church of god that is in corinth with all the saints who a...\n",
       "Galatians         from paul an apostle not from men nor by human agency but by jesus christ and god the father who raised him from the dead   and all the brothers ...\n",
       "Ephesians         from paul an apostle of christ jesus by the will of god to the saints in ephesus the faithful in christ jesus   grace and peace to you from god o...\n",
       "Philippians       from paul and timothy slaves of christ jesus to all the saints in christ jesus who are in philippi with the overseers and deacons   grace and pea...\n",
       "Colossians        from paul an apostle of christ jesus by the will of god and timothy our brother   to the saints the faithful brothers and sisters in christ at co...\n",
       "1 Thessalonians   from paul and silvanus and timothy to the church of the thessalonians in god the father and the lord jesus christ grace and peace to you   we tha...\n",
       "2 Thessalonians   from paul and silvanus and timothy to the church of the thessalonians in god our father and the lord jesus christ   grace and peace to you from g...\n",
       "1 Timothy         from paul an apostle of christ jesus by the command of god our savior and of christ jesus our hope   to timothy my genuine child in the faith gra...\n",
       "2 Timothy         from paul an apostle of christ jesus by the will of god to further the promise of life in christ jesus   to timothy my dear child grace mercy and...\n",
       "Titus             from paul a slave of god and apostle of jesus christ to further the faith of god’s chosen ones and the knowledge of the truth that is in keeping ...\n",
       "Philemon          from paul a prisoner of christ jesus and timothy our brother to philemon our dear friend and colaborer   to apphia our sister to archippus our fe...\n",
       "Hebrews           after god spoke long ago in various portions and in various ways to our ancestors through the prophets   in these last days he has spoken to us i...\n",
       "James             from james a slave of god and the lord jesus christ to the  tribes dispersed abroad greetings   my brothers and sisters consider it nothing but j...\n",
       "1 Peter           from peter an apostle of jesus christ to those temporarily residing abroad in pontus galatia cappadocia the province of asia and bithynia who are...\n",
       "2 Peter           from simeon peter a slave and apostle of jesus christ to those who through the righteousness of our god and savior jesus christ have been granted...\n",
       "1 John            this is what we proclaim to you what was from the beginning what we have heard what we have seen with our eyes what we have looked at and our han...\n",
       "2 John            from the elder to an elect lady and her children whom i love in truth and not i alone but also all those who know the truth   because of the trut...\n",
       "3 John            from the elder to gaius my dear brother whom i love in truth   dear friend i pray that all may go well with you and that you may be in good healt...\n",
       "Jude              from jude a slave of jesus christ and brother of james to those who are called wrapped in the love of god the father and kept for jesus christ   ...\n",
       "Revelation        the revelation of jesus christ which god gave him to show his servants what must happen very soon he made it clear by sending his angel to his se..."
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at the updated text\n",
    "data_clean = pd.DataFrame(data_clean.book_text.apply(round2))\n",
    "data_clean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tSgN67By-f-6"
   },
   "source": [
    "**NOTE:** This data cleaning aka text pre-processing step could go on for a while, but we are going to stop for now. After going through some analysis techniques, if you see that the results don't make sense or could be improved, you can come back and make more edits such as:\n",
    "* Mark 'sinning' and 'sin' as the same word (stemming / lemmatization)\n",
    "* Combine 'thank you' into one term (bi-grams)\n",
    "* And a lot more..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vl2LA5l5-f-6"
   },
   "source": [
    "## Organizing The Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yX0kqkVx-f-6"
   },
   "source": [
    "I mentioned earlier that the output of this notebook will be clean, organized data in two standard text formats:\n",
    "1. **Corpus** - a collection of text\n",
    "2. **Document-Term Matrix** - word counts in matrix format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9JLMX0BL-f-6"
   },
   "source": [
    "### Corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HvVLjUhZ-f-6"
   },
   "source": [
    "We already created a corpus in an earlier step. The definition of a corpus is a collection of texts, and they are all put together neatly in a pandas dataframe here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "OLYlVvj7-f-6",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                             book_text\n",
       "Matthew          1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...\n",
       "Mark             1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...\n",
       "Luke             1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...\n",
       "John             1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...\n",
       "Acts             1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...\n",
       "Romans           1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...\n",
       "1 Corinthians    1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...\n",
       "2 Corinthians    1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...\n",
       "Galatians        1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...\n",
       "Ephesians        1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...\n",
       "Philippians      1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...\n",
       "Colossians       1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...\n",
       "1 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...\n",
       "2 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...\n",
       "1 Timothy        1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...\n",
       "2 Timothy        1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...\n",
       "Titus            1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...\n",
       "Philemon         1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...\n",
       "Hebrews          1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...\n",
       "James            1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...\n",
       "1 Peter          1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...\n",
       "2 Peter          1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...\n",
       "1 John           1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...\n",
       "2 John           1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...\n",
       "3 John           1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...\n",
       "Jude             1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...\n",
       "Revelation       1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h..."
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at our dataframe\n",
    "data_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "xk17CRYw-f-6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_text</th>\n",
       "      <th>num_chapters</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h...</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                             book_text  \\\n",
       "Matthew          1:1 This is the record of the genealogy of Jesus Christ, the son of David, the son of Abraham.  2 Abraham was the father of Isaac, Isaac the fathe...   \n",
       "Mark             1:1 The beginning of the gospel of Jesus Christ, the Son of God. 2 As it is written in the prophet Isaiah, “Look, I am sending my messenger ahead ...   \n",
       "Luke             1:1 Now many have undertaken to compile an account of the things that have been fulfilled among us,  2 like the accounts passed on to us by those ...   \n",
       "John             1:1 In the beginning was the Word, and the Word was with God, and the Word was fully God.  2 The Word was with God in the beginning.  3 All things...   \n",
       "Acts             1:1 I wrote the former account, Theophilus, about all that Jesus began to do and teach  2 until the day he was taken up to heaven, after he had gi...   \n",
       "Romans           1:1 From Paul, a slave of Christ Jesus, called to be an apostle, set apart for the gospel of God.  2 This gospel he promised beforehand through hi...   \n",
       "1 Corinthians    1:1 From Paul, called to be an apostle of Christ Jesus by the will of God, and Sosthenes, our brother,  2 to the church of God that is in Corinth,...   \n",
       "2 Corinthians    1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother, to the church of God that is in Corinth, with all the saint...   \n",
       "Galatians        1:1 From Paul, an apostle (not from men, nor by human agency, but by Jesus Christ and God the Father who raised him from the dead)  2 and all the ...   \n",
       "Ephesians        1:1 From Paul, an apostle of Christ Jesus by the will of God, to the saints [in Ephesus], the faithful in Christ Jesus.  2 Grace and peace to you ...   \n",
       "Philippians      1:1 From Paul and Timothy, slaves of Christ Jesus, to all the saints in Christ Jesus who are in Philippi, with the overseers and deacons.  2 Grace...   \n",
       "Colossians       1:1 From Paul, an apostle of Christ Jesus by the will of God, and Timothy our brother,  2 to the saints, the faithful brothers and sisters in Chri...   \n",
       "1 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God the Father and the Lord Jesus Christ. Grace and peace to you!  2...   \n",
       "2 Thessalonians  1:1 From Paul and Silvanus and Timothy, to the church of the Thessalonians in God our Father and the Lord Jesus Christ.  2 Grace and peace to you ...   \n",
       "1 Timothy        1:1 From Paul, an apostle of Christ Jesus by the command of God our Savior and of Christ Jesus our hope,  2 to Timothy, my genuine child in the fa...   \n",
       "2 Timothy        1:1 From Paul, an apostle of Christ Jesus by the will of God, to further the promise of life in Christ Jesus,  2 to Timothy, my dear child. Grace,...   \n",
       "Titus            1:1 From Paul, a slave of God and apostle of Jesus Christ, to further the faith of God’s chosen ones and the knowledge of the truth that is in kee...   \n",
       "Philemon         1:1 From Paul, a prisoner of Christ Jesus, and Timothy our brother, to Philemon, our dear friend and colaborer,  2 to Apphia our sister, to Archip...   \n",
       "Hebrews          1:1 After God spoke long ago in various portions and in various ways to our ancestors through the prophets,  2 in these last days he has spoken to...   \n",
       "James            1:1 From James, a slave of God and the Lord Jesus Christ, to the 12 tribes dispersed abroad. Greetings!  2 My brothers and sisters, consider it no...   \n",
       "1 Peter          1:1 From Peter, an apostle of Jesus Christ, to those temporarily residing abroad (in Pontus, Galatia, Cappadocia, the province of Asia, and Bithyn...   \n",
       "2 Peter          1:1 From Simeon Peter, a slave and apostle of Jesus Christ, to those who through the righteousness of our God and Savior, Jesus Christ, have been ...   \n",
       "1 John           1:1 This is what we proclaim to you: what was from the beginning, what we have heard, what we have seen with our eyes, what we have looked at and ...   \n",
       "2 John           1:1 From the elder, to an elect lady and her children, whom I love in truth (and not I alone, but also all those who know the truth),  2 because o...   \n",
       "3 John           1:1 From the elder, to Gaius my dear brother, whom I love in truth.  2 Dear friend, I pray that all may go well with you and that you may be in go...   \n",
       "Jude             1:1 From Jude, a slave of Jesus Christ and brother of James, to those who are called, wrapped in the love of God the Father and kept for Jesus Chr...   \n",
       "Revelation       1:1 The revelation of Jesus Christ, which God gave him to show his servants what must happen very soon. He made it clear by sending his angel to h...   \n",
       "\n",
       "                 num_chapters  \n",
       "Matthew                    28  \n",
       "Mark                       16  \n",
       "Luke                       24  \n",
       "John                       21  \n",
       "Acts                       28  \n",
       "Romans                     16  \n",
       "1 Corinthians              16  \n",
       "2 Corinthians              13  \n",
       "Galatians                   6  \n",
       "Ephesians                   6  \n",
       "Philippians                 4  \n",
       "Colossians                  4  \n",
       "1 Thessalonians             5  \n",
       "2 Thessalonians             3  \n",
       "1 Timothy                   6  \n",
       "2 Timothy                   4  \n",
       "Titus                       3  \n",
       "Philemon                    1  \n",
       "Hebrews                    13  \n",
       "James                       5  \n",
       "1 Peter                     5  \n",
       "2 Peter                     3  \n",
       "1 John                      5  \n",
       "2 John                      1  \n",
       "3 John                      1  \n",
       "Jude                        1  \n",
       "Revelation                 22  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's include the number of chapters in each book\n",
    "data_df['num_chapters'] = book_chapters\n",
    "data_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "zfVoEwso-f-6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Let's pickle it for later use\n",
    "data_df.to_pickle(os.path.join(base_dir, 'corpus.pkl'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vnLUs3TO-f-6"
   },
   "source": [
    "### Document-Term Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RsBUE6IS-f-6"
   },
   "source": [
    "For many of the techniques we'll be using in future notebooks, the text must be tokenized, meaning broken down into smaller pieces. The most common tokenization technique is to break down text into words. We can do this using scikit-learn's CountVectorizer, where every row will represent a different document and every column will represent a different word.\n",
    "\n",
    "In addition, with CountVectorizer, we can remove stop words. Stop words are common words that add no additional meaning to text such as 'a', 'the', etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "AVl1sO1L-f-7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>aaron</th>\n",
       "      <th>abaddon</th>\n",
       "      <th>abandon</th>\n",
       "      <th>abandoned</th>\n",
       "      <th>abandoning</th>\n",
       "      <th>abandons</th>\n",
       "      <th>abba</th>\n",
       "      <th>abel</th>\n",
       "      <th>abhor</th>\n",
       "      <th>abiathar</th>\n",
       "      <th>...</th>\n",
       "      <th>zealous</th>\n",
       "      <th>zebedee</th>\n",
       "      <th>zebulun</th>\n",
       "      <th>zechariah</th>\n",
       "      <th>zenas</th>\n",
       "      <th>zerah</th>\n",
       "      <th>zerubbabel</th>\n",
       "      <th>zeus</th>\n",
       "      <th>zion</th>\n",
       "      <th>zionhe</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Matthew</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mark</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Luke</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Acts</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Romans</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Corinthians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Corinthians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Galatians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ephesians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Colossians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Thessalonians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Thessalonians</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Timothy</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Timothy</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Titus</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philemon</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hebrews</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>James</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 Peter</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 Peter</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1 John</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2 John</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3 John</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jude</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Revelation</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>27 rows × 6612 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 aaron  abaddon  abandon  abandoned  abandoning  abandons  \\\n",
       "Matthew              0        0        0          0           0         0   \n",
       "Mark                 0        0        0          0           0         0   \n",
       "Luke                 1        0        0          0           0         0   \n",
       "John                 0        0        1          0           0         1   \n",
       "Acts                 1        0        2          3           0         0   \n",
       "Romans               0        0        0          1           0         0   \n",
       "1 Corinthians        0        0        0          0           0         0   \n",
       "2 Corinthians        0        0        0          1           0         0   \n",
       "Galatians            0        0        0          0           0         0   \n",
       "Ephesians            0        0        0          0           0         0   \n",
       "Philippians          0        0        0          0           0         0   \n",
       "Colossians           0        0        0          0           0         0   \n",
       "1 Thessalonians      0        0        0          0           0         0   \n",
       "2 Thessalonians      0        0        0          0           0         0   \n",
       "1 Timothy            0        0        0          0           0         0   \n",
       "2 Timothy            0        0        0          0           0         0   \n",
       "Titus                0        0        0          0           0         0   \n",
       "Philemon             0        0        0          0           0         0   \n",
       "Hebrews              3        0        1          0           1         0   \n",
       "James                0        0        0          0           0         0   \n",
       "1 Peter              0        0        0          0           0         0   \n",
       "2 Peter              0        0        0          0           0         0   \n",
       "1 John               0        0        0          0           0         0   \n",
       "2 John               0        0        0          0           0         0   \n",
       "3 John               0        0        0          0           0         0   \n",
       "Jude                 0        0        0          2           0         0   \n",
       "Revelation           0        1        0          0           0         0   \n",
       "\n",
       "                 abba  abel  abhor  abiathar  ...  zealous  zebedee  zebulun  \\\n",
       "Matthew             0     1      0         0  ...        0        6        2   \n",
       "Mark                1     0      0         1  ...        0        4        0   \n",
       "Luke                0     1      0         0  ...        0        1        0   \n",
       "John                0     0      0         0  ...        0        1        0   \n",
       "Acts                0     0      0         0  ...        1        0        0   \n",
       "Romans              1     0      2         0  ...        1        0        0   \n",
       "1 Corinthians       0     0      0         0  ...        0        0        0   \n",
       "2 Corinthians       0     0      0         0  ...        0        0        0   \n",
       "Galatians           1     0      0         0  ...        1        0        0   \n",
       "Ephesians           0     0      0         0  ...        0        0        0   \n",
       "Philippians         0     0      0         0  ...        0        0        0   \n",
       "Colossians          0     0      0         0  ...        0        0        0   \n",
       "1 Thessalonians     0     0      0         0  ...        0        0        0   \n",
       "2 Thessalonians     0     0      0         0  ...        0        0        0   \n",
       "1 Timothy           0     0      0         0  ...        0        0        0   \n",
       "2 Timothy           0     0      0         0  ...        0        0        0   \n",
       "Titus               0     0      0         0  ...        0        0        0   \n",
       "Philemon            0     0      0         0  ...        0        0        0   \n",
       "Hebrews             0     2      0         0  ...        0        0        0   \n",
       "James               0     0      0         0  ...        0        0        0   \n",
       "1 Peter             0     0      0         0  ...        0        0        0   \n",
       "2 Peter             0     0      0         0  ...        0        0        0   \n",
       "1 John              0     0      0         0  ...        0        0        0   \n",
       "2 John              0     0      0         0  ...        0        0        0   \n",
       "3 John              0     0      0         0  ...        0        0        0   \n",
       "Jude                0     0      0         0  ...        0        0        0   \n",
       "Revelation          0     0      0         0  ...        0        0        1   \n",
       "\n",
       "                 zechariah  zenas  zerah  zerubbabel  zeus  zion  zionhe  \n",
       "Matthew                  1      0      1           2     0     1       0  \n",
       "Mark                     0      0      0           0     0     0       0  \n",
       "Luke                    12      0      0           1     0     0       0  \n",
       "John                     0      0      0           0     0     1       0  \n",
       "Acts                     0      0      0           0     2     0       0  \n",
       "Romans                   0      0      0           0     0     1       1  \n",
       "1 Corinthians            0      0      0           0     0     0       0  \n",
       "2 Corinthians            0      0      0           0     0     0       0  \n",
       "Galatians                0      0      0           0     0     0       0  \n",
       "Ephesians                0      0      0           0     0     0       0  \n",
       "Philippians              0      0      0           0     0     0       0  \n",
       "Colossians               0      0      0           0     0     0       0  \n",
       "1 Thessalonians          0      0      0           0     0     0       0  \n",
       "2 Thessalonians          0      0      0           0     0     0       0  \n",
       "1 Timothy                0      0      0           0     0     0       0  \n",
       "2 Timothy                0      0      0           0     0     0       0  \n",
       "Titus                    0      1      0           0     0     0       0  \n",
       "Philemon                 0      0      0           0     0     0       0  \n",
       "Hebrews                  0      0      0           0     0     1       0  \n",
       "James                    0      0      0           0     0     0       0  \n",
       "1 Peter                  0      0      0           0     0     1       0  \n",
       "2 Peter                  0      0      0           0     0     0       0  \n",
       "1 John                   0      0      0           0     0     0       0  \n",
       "2 John                   0      0      0           0     0     0       0  \n",
       "3 John                   0      0      0           0     0     0       0  \n",
       "Jude                     0      0      0           0     0     0       0  \n",
       "Revelation               0      0      0           0     0     1       0  \n",
       "\n",
       "[27 rows x 6612 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We are going to create a document-term matrix using CountVectorizer, and exclude common English stop words\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "cv = CountVectorizer(stop_words='english')\n",
    "data_cv = cv.fit_transform(data_clean.book_text)\n",
    "data_dtm = pd.DataFrame(data_cv.toarray(), columns=cv.get_feature_names_out())\n",
    "data_dtm.index = data_clean.index\n",
    "data_dtm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "RVVkVg1F-f-7",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Let's pickle it for later use\n",
    "data_dtm.to_pickle(os.path.join(base_dir, 'dtm.pkl'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "5mWKHJbQ-f-7",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Let's also pickle the cleaned data (before we put it in document-term matrix format) and the CountVectorizer object\n",
    "data_clean.to_pickle(os.path.join(base_dir, 'data_clean.pkl'))\n",
    "pickle.dump(cv, open(os.path.join(base_dir, 'cv.pkl'), 'wb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ip0pGYOz-f--"
   },
   "source": [
    "## Additional Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YemorzFQ-f-_"
   },
   "source": [
    "1. Can you add an additional regular expression to the clean_text_round2 function to further clean the text?\n",
    "2. Play around with CountVectorizer's parameters. What is ngram_range? What is min_df and max_df?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wK3IfJ2w-f-_"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}