Mid_2024.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyNulR0drxa6eA1k7g1NSu87"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## Data Science for Cybersecurity, 2024@NCCU\n",
        "\n",
        "Midterm, **2022/04/1 (Mon) 09:00 to 2020/04/10 (Wed) 23:59. UTC+8**\n",
        "\n",
        "* Make sure you copy this file to YOUR OWN Google Drive before starting to take the exam.\n",
        "* You MUST send your .ipynb file (NOT a colab sharable link) to the Homework Submission System (if any problem, send it to TA via email. Late submission is allowed, but with a 15% per day penalty).\n",
        "* You CANNOT discuss midterm with anyone.\n",
        "* You CANNOT share your file with any others except TA.\n",
        "* If possible, write some comments in your codes so that TA can easily understand your codes.\n",
        "* You are allowed to use AI-tools, python, excel, and/or orange. You can use any codes from our lecture and/or the Internet. But you need to explain how and why you use them in your analysis.\n",
        "* Answer all questions.\n",
        "* You may post images/screenshot in this file.\n",
        "---\n"
      ],
      "metadata": {
        "id": "sIm9jRBOMTcy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #0\n",
        "\n",
        "What is your name, department and student ID?"
      ],
      "metadata": {
        "id": "YpoWgy2lM794"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #0</font>\n",
        "\n",
        "Name:\n",
        "\n",
        "Department:\n",
        "\n",
        "Student ID:\n",
        "\n",
        "---"
      ],
      "metadata": {
        "id": "xwwdjSTJM6o-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #1\n",
        "\n",
        "1-1. Please explain three security concepts: safety, security, and assurance. And add some comments by using your own examples (not necessary to be a cybersecurity example; my example in the class is 'COVID-19 campus measures and control').\n",
        "\n",
        "1-2. What is the relationship between the following security concepts: cost and risk (hint: probability and impcat). How do we evaluate a security incident (i.e., security assessment)?\n",
        "\n",
        "1-3. Why do we use \"data analysis\" methods to analysis cybersecurity? Is it feasable to do so? (You may use the answer of 1-1 and 1-2 to further discuss this question.) Can you give us some examples (or scenairo) that \"data analysis\" may fail when we apply it to cybersecurity?"
      ],
      "metadata": {
        "id": "hH_JU_QMNBWo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #1</font>\n",
        "\n"
      ],
      "metadata": {
        "id": "k2H_1oMjNoUA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #2\n",
        "\n",
        "For **ALL** the following questions in 2, please state your solutions, list some python libraries that can help you to quickly solve such problem, and tell us some concepts how do they solve such problem.\n",
        "\n",
        "2-1. There are several problems when making a model. For exmaple, there might be too many data points in the training data. How do you deal with such problem?\n",
        "\n",
        "2-2 What the the major issues if you apply such solution to your 'too-many-data' dataset?\n",
        "\n",
        "2-3 What if I do not want to perform sampling in 2-2 while training? (Hint: gradient descent)\n",
        "\n",
        "2-4. If there are too few data points in the traning data, how do you deal with it?\n",
        "\n",
        "2-5. What the the major issues if you apply such solution to your 'too-few-data' dataset?\n",
        "\n",
        "2-6. What if there are too many features in the traning data? Any solution?\n",
        "\n",
        "2-7. What if there are too few features in the traning data? Any solution?\n",
        "\n",
        "2-8. For a classification problem, if the number of data in the each class in the training data is imbalanced, how do you deal with it?\n"
      ],
      "metadata": {
        "id": "ldX6Uo9bN23m"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #2</font>\n"
      ],
      "metadata": {
        "id": "I76EwZxnOp60"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "###Question #3:\n",
        "\n",
        "3-1. Tree-based algorithms can deal with both numeric data and categorical data. We show that SVM can deal with numeric data in the class. Could SVM manage categorical data? Why or why not?\n",
        "\n",
        "\n",
        "3-2. Is there any generalized method that we can convert any categorical data to numeric form? Show me one method, and what are the issues of such methods?\n",
        "\n",
        "\n",
        "3-3. PCA is one of the method to reduce the traning data dimension. How do we evaluate how many dimensions we should keep?\n",
        "\n",
        "\n",
        "3-4. Is PCA (i.e., dimension reduction) harmful to the later preduction process? Why or why not?\n",
        "\n",
        "3-5. Maximum likelihood (or cross-entropy) is used for evaluate catagorical target. What if we use MSE (minial square error) to evaluate catagorical target?\n"
      ],
      "metadata": {
        "id": "JWP6UwbtO65L"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #3</font>\n"
      ],
      "metadata": {
        "id": "pGSJYmMIPHZd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #4:\n",
        "\n",
        "* Use Dataset: Malware Calls to perform classification.\n",
        "* https://github.com/hsiaom26/DS4CS-24/blob/main/data/F14S1940_callonly_tfds.tgz\n",
        "* Show your confusion matrix, accuracy, F1 score.\n",
        "* You may apply any alorithms (but no neural network is allowed, we will talk about it later), any preprocessing, any postprocessing, and etc.\n",
        "* Must show analysis steps, not only answers.\n"
      ],
      "metadata": {
        "id": "7gfOug-dQTby"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# get data from github\n",
        "\n",
        "import requests\n",
        "\n",
        "url = 'https://github.com/hsiaom26/DS4CS-24/raw/main/data/F14S1940_callonly_tfds.zip'\n",
        "\n",
        "response = requests.get(url)\n",
        "\n",
        "if response.status_code == 200:\n",
        "    # Open the file for writing\n",
        "    with open('F14S1940_callonly_tfds.zip', 'wb') as f:\n",
        "        # Write the response content to the file\n",
        "        f.write(response.content)\n",
        "else:\n",
        "    print('Error: Failed to download the file')"
      ],
      "metadata": {
        "id": "qydWNGfFQk65"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!unzip -q F14S1940_callonly_tfds.zip"
      ],
      "metadata": {
        "id": "LSezwsVvRSPb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!ls -al F14S1940_callonly_tfds"
      ],
      "metadata": {
        "id": "OHgc3MPKRUQY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #4</font>\n",
        "\n",
        "At least, confusion matrix, accuracy, and F1-score must be shown."
      ],
      "metadata": {
        "id": "5EGfq8IpRV_1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #5\n",
        "\n",
        "According to your analysis in Quesion 4.\n",
        "\n",
        "5-1. What is the most important feature in this dataset?\n",
        "\n",
        "5-2. Two families `kryptik_g_529` and `kryptik_g_547` are in the dataset, do you think it is appropreate to devide the smaples into these two sets? Or do you think using one class, for example `kryptik_g_`, as the family label on these samples is good enough? Why?\n",
        "\n",
        "5-3. `allaple_woj_g_98_year2017` has 615 samples. Do you think we should further devide these samples into many smaller sub-families? Why?\n",
        "\n",
        "5-4. Who and who could be the family 'sibling'? Why? (open question)\n",
        "\n",
        "5-5. Which distance function do you use? Why do you choose this distance function rather than the others?"
      ],
      "metadata": {
        "id": "voj91PwgRk9C"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #5</font>"
      ],
      "metadata": {
        "id": "Z-jtnKYbT7yF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Question #6:\n",
        "It is always difficult to deal with text data. Try to give us TWO methods to preprocess text-based data to numbers (or vectors), so that we can easily apply linear regression or logistic regression on texts?\n"
      ],
      "metadata": {
        "id": "uwbHOolvT60P"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #6</font>\n",
        "\n",
        "\n",
        "Answer (1).\n",
        "\n",
        "Answer (2)."
      ],
      "metadata": {
        "id": "8FJr0k7NUB_0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "###Question #7:\n",
        "\n",
        "7-1. Use the training data of `NID` dataset to perform calssificaiton. Is data normalization is useful for gradient-descent-like method? Show us evidence.\n",
        "\n",
        "7-2. You should apply PCA (dimension reduction) first or data normalization/standardization first? Why?\n",
        "\n",
        "7-3. Why classification accuracy may not be a very good index to evaluate the usefulness of a classification model? Could you exaplin why by using confusion matrix?\n",
        "\n"
      ],
      "metadata": {
        "id": "QRbseGRMUWtL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #7</font>"
      ],
      "metadata": {
        "id": "ylY2ZlW6Wu-U"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "###Question #8:\n",
        "\n",
        "If you have mulitple models, but they may have different classification results to the testing data, how to come up a final decision?\n",
        "\n"
      ],
      "metadata": {
        "id": "doInShQwWzt5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color=red>Answer #8</font>\n",
        "\n"
      ],
      "metadata": {
        "id": "Q6kjfXhEW2IC"
      }
    }
  ]
}