MultinomialNBXGBoost.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the first few assignments, we have learned how to infer part based components (known as mutational signatures) generated by particular mutational processes using Non-negative Matrix Factorization (NMF). By doing this, we are trying to reconstruct the mutation catalog in a given sample with mutational signatures and their contributions.\n",
    "\n",
    "In this group project, you will use similar mutational profiles and signature activities to predict cancer types but with much larger sample size. \n",
    "You should:\n",
    "* Separate the data into training and test groups within each cancer type.\n",
    "* Find out which features are informative for the prediction of the cancer type (label). You should combine the profiles and activities and use each data type independently.\n",
    "* Implement different models for classification of the samples given the input data and evaluate the model performance using test data to avoid overfitting. Explain briefly how does each model that you have used work.\n",
    "* Report model performance, using standard machine learning metrics such as confusion matrices etc. \n",
    "* Compare model performance across methods and across cancer types, are some types easier top predict than others.\n",
    "* Submit a single Jupyter notebook as the final report and present that during the last assignment session "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data include both mutational catalogs from multiple cancers and the predicted activities in the paper [\"Alexandrov LB, et al. (2020) The repertoire of mutational signatures in human cancer\"](https://www.nature.com/articles/s41586-020-1943-3). The data either are generated from whole human genome (WGS) or only exomes regions (WES). Since the exome region only constitutes about 1% of human genome, the total mutation numbers in these samples are, of course, much smaller. So if you plan to use WGS together with WES data, remember to normalize the profile for each sample to sum up to 1.\n",
    "\n",
    "Note that, the data is generated from different platforms by different research groups, some of them (e.g. labeled with PCAWG, TCGA) are processed with the same bioinformatics pipeline. Thus, these samples will have less variability related to data processing pipelines."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cancer types might be labeled under the same tissue, e.g. 'Bone-Benign','Bone-Epith', which can also be combined together or take the one has more samples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a link to background reading [\"Pan-Cancer Analysis of Whole Genomes\"](https://www.nature.com/collections/afdejfafdb). Have a look especially the paper [\"A deep learning system accurately classifies primary and metastatic cancers using passenger mutation patterns\"](https://www.nature.com/articles/s41467-019-13825-8)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 411,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mutational catalogs and activities - WGS data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 412,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Mutation type</th>\n",
       "      <th>Trinucleotide</th>\n",
       "      <th>Biliary-AdenoCA::SP117655</th>\n",
       "      <th>Biliary-AdenoCA::SP117556</th>\n",
       "      <th>Biliary-AdenoCA::SP117627</th>\n",
       "      <th>Biliary-AdenoCA::SP117775</th>\n",
       "      <th>Biliary-AdenoCA::SP117332</th>\n",
       "      <th>Biliary-AdenoCA::SP117712</th>\n",
       "      <th>Biliary-AdenoCA::SP117017</th>\n",
       "      <th>Biliary-AdenoCA::SP117031</th>\n",
       "      <th>...</th>\n",
       "      <th>Uterus-AdenoCA::SP94540</th>\n",
       "      <th>Uterus-AdenoCA::SP95222</th>\n",
       "      <th>Uterus-AdenoCA::SP89389</th>\n",
       "      <th>Uterus-AdenoCA::SP90503</th>\n",
       "      <th>Uterus-AdenoCA::SP92460</th>\n",
       "      <th>Uterus-AdenoCA::SP92931</th>\n",
       "      <th>Uterus-AdenoCA::SP91265</th>\n",
       "      <th>Uterus-AdenoCA::SP89909</th>\n",
       "      <th>Uterus-AdenoCA::SP90629</th>\n",
       "      <th>Uterus-AdenoCA::SP95550</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>C&gt;A</td>\n",
       "      <td>ACA</td>\n",
       "      <td>269</td>\n",
       "      <td>114</td>\n",
       "      <td>105</td>\n",
       "      <td>217</td>\n",
       "      <td>52</td>\n",
       "      <td>192</td>\n",
       "      <td>54</td>\n",
       "      <td>196</td>\n",
       "      <td>...</td>\n",
       "      <td>117</td>\n",
       "      <td>233</td>\n",
       "      <td>94</td>\n",
       "      <td>114</td>\n",
       "      <td>257</td>\n",
       "      <td>139</td>\n",
       "      <td>404</td>\n",
       "      <td>97</td>\n",
       "      <td>250</td>\n",
       "      <td>170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C&gt;A</td>\n",
       "      <td>ACC</td>\n",
       "      <td>148</td>\n",
       "      <td>56</td>\n",
       "      <td>71</td>\n",
       "      <td>123</td>\n",
       "      <td>36</td>\n",
       "      <td>139</td>\n",
       "      <td>54</td>\n",
       "      <td>102</td>\n",
       "      <td>...</td>\n",
       "      <td>90</td>\n",
       "      <td>167</td>\n",
       "      <td>59</td>\n",
       "      <td>64</td>\n",
       "      <td>268</td>\n",
       "      <td>75</td>\n",
       "      <td>255</td>\n",
       "      <td>78</td>\n",
       "      <td>188</td>\n",
       "      <td>137</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 2782 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  Mutation type Trinucleotide  Biliary-AdenoCA::SP117655  \\\n",
       "0           C>A           ACA                        269   \n",
       "1           C>A           ACC                        148   \n",
       "\n",
       "   Biliary-AdenoCA::SP117556  Biliary-AdenoCA::SP117627  \\\n",
       "0                        114                        105   \n",
       "1                         56                         71   \n",
       "\n",
       "   Biliary-AdenoCA::SP117775  Biliary-AdenoCA::SP117332  \\\n",
       "0                        217                         52   \n",
       "1                        123                         36   \n",
       "\n",
       "   Biliary-AdenoCA::SP117712  Biliary-AdenoCA::SP117017  \\\n",
       "0                        192                         54   \n",
       "1                        139                         54   \n",
       "\n",
       "   Biliary-AdenoCA::SP117031  ...  Uterus-AdenoCA::SP94540  \\\n",
       "0                        196  ...                      117   \n",
       "1                        102  ...                       90   \n",
       "\n",
       "   Uterus-AdenoCA::SP95222  Uterus-AdenoCA::SP89389  Uterus-AdenoCA::SP90503  \\\n",
       "0                      233                       94                      114   \n",
       "1                      167                       59                       64   \n",
       "\n",