Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Project Tasks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the first few assignments, we have learned how to infer part based components (known as mutational signatures) generated by particular mutational processes using Non-negative Matrix Factorization (NMF). By doing this, we are trying to reconstruct the mutation catalog in a given sample with mutational signatures and their contributions.\n",
"\n",
"In this group project, you will use similar mutational profiles and signature activities to predict cancer types but with much larger sample size. \n",
"You should:\n",
"* Separate the data into training and test groups within each cancer type.\n",
"* Find out which features are informative for the prediction of the cancer type (label). You should combine the profiles and activities and use each data type independently.\n",
"* Implement different models for classification of the samples given the input data and evaluate the model performance using test data to avoid overfitting. Explain briefly how does each model that you have used work.\n",
"* Report model performance, using standard machine learning metrics such as confusion matrices etc. \n",
"* Compare model performance across methods and across cancer types, are some types easier top predict than others.\n",
"* Submit a single Jupyter notebook as the final report and present that during the last assignment session "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data include both mutational catalogs from multiple cancers and the predicted activities in the paper [\"Alexandrov LB, et al. (2020) The repertoire of mutational signatures in human cancer\"](https://www.nature.com/articles/s41586-020-1943-3). The data either are generated from whole human genome (WGS) or only exomes regions (WES). Since the exome region only constitutes about 1% of human genome, the total mutation numbers in these samples are, of course, much smaller. So if you plan to use WGS together with WES data, remember to normalize the profile for each sample to sum up to 1.\n",
"\n",
"Note that, the data is generated from different platforms by different research groups, some of them (e.g. labeled with PCAWG, TCGA) are processed with the same bioinformatics pipeline. Thus, these samples will have less variability related to data processing pipelines."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Cancer types might be labeled under the same tissue, e.g. 'Bone-Benign','Bone-Epith', which can also be combined together or take the one has more samples."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is a link to background reading [\"Pan-Cancer Analysis of Whole Genomes\"](https://www.nature.com/collections/afdejfafdb). Have a look especially the paper [\"A deep learning system accurately classifies primary and metastatic cancers using passenger mutation patterns\"](https://www.nature.com/articles/s41467-019-13825-8)."
]
},
{
"cell_type": "code",
"execution_count": 351,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import re"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Mutational catalogs and activities - WGS data"
]
},
{
"cell_type": "code",
"execution_count": 352,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Mutation type</th>\n",
" <th>Trinucleotide</th>\n",
" <th>Biliary-AdenoCA::SP117655</th>\n",
" <th>Biliary-AdenoCA::SP117556</th>\n",
" <th>Biliary-AdenoCA::SP117627</th>\n",
" <th>Biliary-AdenoCA::SP117775</th>\n",
" <th>Biliary-AdenoCA::SP117332</th>\n",
" <th>Biliary-AdenoCA::SP117712</th>\n",
" <th>Biliary-AdenoCA::SP117017</th>\n",
" <th>Biliary-AdenoCA::SP117031</th>\n",
" <th>...</th>\n",
" <th>Uterus-AdenoCA::SP94540</th>\n",
" <th>Uterus-AdenoCA::SP95222</th>\n",
" <th>Uterus-AdenoCA::SP89389</th>\n",
" <th>Uterus-AdenoCA::SP90503</th>\n",
" <th>Uterus-AdenoCA::SP92460</th>\n",
" <th>Uterus-AdenoCA::SP92931</th>\n",
" <th>Uterus-AdenoCA::SP91265</th>\n",
" <th>Uterus-AdenoCA::SP89909</th>\n",
" <th>Uterus-AdenoCA::SP90629</th>\n",
" <th>Uterus-AdenoCA::SP95550</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>C>A</td>\n",
" <td>ACA</td>\n",
" <td>269</td>\n",
" <td>114</td>\n",
" <td>105</td>\n",
" <td>217</td>\n",
" <td>52</td>\n",
" <td>192</td>\n",
" <td>54</td>\n",
" <td>196</td>\n",
" <td>...</td>\n",
" <td>117</td>\n",
" <td>233</td>\n",
" <td>94</td>\n",
" <td>114</td>\n",
" <td>257</td>\n",
" <td>139</td>\n",
" <td>404</td>\n",
" <td>97</td>\n",
" <td>250</td>\n",
" <td>170</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>C>A</td>\n",
" <td>ACC</td>\n",
" <td>148</td>\n",
" <td>56</td>\n",
" <td>71</td>\n",
" <td>123</td>\n",
" <td>36</td>\n",
" <td>139</td>\n",
" <td>54</td>\n",
" <td>102</td>\n",
" <td>...</td>\n",
" <td>90</td>\n",
" <td>167</td>\n",
" <td>59</td>\n",
" <td>64</td>\n",
" <td>268</td>\n",
" <td>75</td>\n",
" <td>255</td>\n",
" <td>78</td>\n",
" <td>188</td>\n",
" <td>137</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>2 rows × 2782 columns</p>\n",
"</div>"
],
"text/plain": [
" Mutation type Trinucleotide Biliary-AdenoCA::SP117655 \\\n",
"0 C>A ACA 269 \n",
"1 C>A ACC 148 \n",
"\n",
" Biliary-AdenoCA::SP117556 Biliary-AdenoCA::SP117627 \\\n",
"0 114 105 \n",
"1 56 71 \n",
"\n",
" Biliary-AdenoCA::SP117775 Biliary-AdenoCA::SP117332 \\\n",
"0 217 52 \n",
"1 123 36 \n",
"\n",
" Biliary-AdenoCA::SP117712 Biliary-AdenoCA::SP117017 \\\n",
"0 192 54 \n",
"1 139 54 \n",
"\n",
" Biliary-AdenoCA::SP117031 ... Uterus-AdenoCA::SP94540 \\\n",
"0 196 ... 117 \n",
"1 102 ... 90 \n",
"\n",
" Uterus-AdenoCA::SP95222 Uterus-AdenoCA::SP89389 Uterus-AdenoCA::SP90503 \\\n",
"0 233 94 114 \n",
"1 167 59 64 \n",
"\n",
Loading
Loading full blame...