From Protein Structure to Function with Bioinformatics.pdf

104 T. Nugent and D.T. JonesFig. 4.6 Eleven proteomes were filtered using the MEMSAT3 TM/globular discriminator. Thosepredicted to be TM proteins were subject to full TM topology prediction. X-axis: TM helix count.Y-axis: Number of proteinscolicin used to form pores in the outer membranes of competing bacteria.Furthermore, errors in databases are not infrequent and add an element of noise.While such noise is often well tolerated by machine learning methods, the problemis more significant in smaller data sets.Another issue that needs to be addressed is homology in the data, with most datasets being reduced at a level of 30–40% sequence identity. Since structural TMprotein data is at a premium, this level is perhaps slightly higher than that whichwould be applied to globular protein data sets. Although there is an increased riskof overfitting, this is necessary to ensure training sets are of sufficient size. Allmachine learning methods have multiple free parameters and thus have the potentialto overfit. That is, rather than identifying a pattern in a sequence, an example maybe learned ‘by heart’, including any noise that the sequence may contain. A methodthat has been overfitted is typically able to reproduce its training examples accurately,but will perform poorly on examples that it has not seen before. It is importantthat, when assessing the accuracy of a prediction method, homology in bothtraining and test data sets is reduced in order to avoid overfitting.In all cases, it is important that stringent cross-validation is performed. Crossvalidationis the statistical practice of partitioning a data set into subsets such thata single subset is validated on a model trained using the remaining subsets, and the