Exercise:AllergenicityPrediction

Exercise written by Carolina Barra Quaglia

Introduction

Climate change and the increasing demand for sustainable food production are driving the search for alternative protein sources. Algae are particularly interesting because they are abundant, can be cultivated without agricultural land, and may provide a valuable source of proteins and other nutrients.

However, introducing a new protein source into the human diet also raises an important question: could some of these proteins cause allergic reactions?

Food allergens are generally proteins that are recognized by the immune system and can trigger an inappropriate immune response in susceptible individuals. Experimental characterization of every protein in a new food source is expensive and time-consuming. Bioinformatics can therefore play an important role in identifying proteins with potential allergenic properties and prioritizing candidates for experimental investigation.

Learning Objectives

After completing this exercise, you should be able to:

• Use biological databases and sequence similarity to investigate potential protein allergens

• Explain how sequence similarity can provide evidence for potential allergenicity and cross-reactivity, and recognize its limitations

• Build and evaluate a simple machine-learning model for allergenicity prediction

• Identify limitations of a prediction model and explore strategies to improve its performance

The research question

Researchers at DTU Food are interested in developing a high-protein food product based on brown algae. Before introducing a new protein source into the human diet, however, we need to consider its potential allergenicity.

Your task is therefore to investigate:

Could proteins from brown algae represent an allergenic risk, and how could we predict this risk computationally?

We will focus particularly on Ectocarpus siliculosus, a species belonging to the brown algae (Phaeophyceae, taxid: 2870).

We will first explore how known allergens and sequence similarity can be used to assess potential allergenicity. We will then build our own machine-learning classifier for allergen prediction using a Random Forest. Finally, you will try to improve the predictor using alternative features, preprocessing strategies, machine-learning algorithms, or other approaches of your choice.

PART A: Investigating Allergenicity

Where Would You Start?

Every good bioinformatics investigation starts by asking what information is already available.

• Q1: What would be your first approach to determine whether brown algae contain potential allergens? Find one database containing experimentally characterized allergens. Consider what biological information you would search for and what properties of the proteins might provide evidence of allergenic potential.

We could try to run NetAllergen on our protein of interest, but the server will crash if too many people start to run jobs, instead we will follow an alternative approach

Now imagine that no computational predictor specifically designed for allergenicity has yet been developed. How could you attempt to predict allergenicity in silico? Which existing protein predictions or sequence properties could potentially be used as proxies for allergenicity? There is not necessarily a single correct solution here. Think about what biological properties characterize allergens and what computational tools could provide information about those properties.

Sequence Similarity and Cross-Reactivity

Allergic reactions are not always restricted to the protein that originally caused sensitization. Antibodies recognizing one allergen may sometimes recognize structurally or evolutionarily related proteins from another organism. Is there any approach to search for sequence similarity to a known allergen? Is there another approaches to search than sequence similarity? What about structure

• Q2: Based on the above is there any other tool beyond NetAllergen that can be used as a proxy to predict allergenicity in silico?

• Q3: Based on your investigation, could immunotherapy potentially reduce allergic responses to proteins found in brown algae? Use sequence or protein-family evidence to justify your answer. What would have to be true about the algal proteins for such cross-reactivity to be plausible?

PART B: Can you beat BLAST?

You will now develop your own allergenicity predictor using a Random Forest classifier in Google Colab. If you have not used Google Colab before, you can find an introduction here

Open the notebook for this exercise: Notebook

Make a copy of the notebook in your own gmail account. Run the notebook step by step rather than simply selecting "Run all". Your goal is to understand what happens to the data at each stage.

You will first need to upload the data to google Colab. The data is placed in DTU Learn as data.csv

• Q4: What will the model see? Inspect the data. How is the data structured? List the type of input features.

Now pick a random seed (replace the default 42) and run the feature-generation and Random Forest training steps in the notebook.

The notebook trains and tests the model using the different sequence partitions. For simplicity these partitions are provided to you in the data. However, remember that these are not random partitions of the data, instead the sequences have been clustered according to sequence homology so no similar proteins are contained across partitions. Predictions from the independent test partitions are then combined to obtain an overall evaluation of the model.

Run the evaluation section of the notebook.

It will report several performance measures:

Accuracy — fraction of all predictions that are correct
Precision — fraction of proteins predicted as allergens that actually are allergens
Recall (sensitivity) — fraction of allergens correctly identified by the predictor
Specificity — fraction of non-allergens correctly identified.
MCC (Matthews Correlation Coefficient) — overall agreement between predictions and observations, taking all four outcomes of the confusion matrix into account.
ROC-AUC — ability of the model to discriminate allergens from non-allergens across different classification thresholds.

• Q5: How well does it work? Report the performance of your predictor

Inspect the confusion matrix produced by the notebook.

The four possible outcomes are:

True positive (TP): allergen → predicted allergen  
True negative (TN): non-allergen → predicted non-allergen  
False positive (FP):non-allergen → predicted allergen  
False negative (FN): allergen → predicted non-allergen

• Q6: Where does the model make mistakes? Imagine that this predictor is being used as an initial "screening tool for proteins in a novel food".

Accuracy is not the whole story Two models can have similar accuracy while behaving differently towards allergens and non-allergens.

Inspect the ROC curve generated by the notebook. The diagonal line represents the performance expected from random classification. An AUC of approximately 0.5 corresponds to random discrimination, whereas a value approaching 1.0 indicates increasingly good separation between the two classes.

• Q7: Why it makes more sense to use AUC than accuracy to report performance. Record the AUC of your model.

You now have a baseline model. Let's play with one of the random forest parameters. Number of estimators.

• Q8: How the number of estimators 10, 100, 1000 affects model performance?

PART C Can you beat the baseline?

Now it is your turn. Modify the computational approach and determine whether you can improve allergenicity prediction.

You are free to modify the pipeline. For example, you could investigate:

• different Random Forest parameters
• alternative machine-learning algorithms
• feature selection or preprocessing
• strategies for dealing with class imbalance
• alternative sequence representations

You do not need to try everything. A well-motivated modification that you evaluate correctly is more interesting than testing many methods without understanding them.

You may use ChatGPT or other language models to help you modify the code. However, you should understand what you changed and be able to explain why you expected it to help.

• Q9: Provide a rationale for the change and report the new performance.

• Q10: Did your modification improve allergenicity prediction?

Conclusion =

You should now have experience with two complementary approaches to investigating protein allergenicity:

Knowledge and similarity-based analysis: using experimental evidence, known allergens, sequence similarity, and biological knowledge to identify potential cross-reactivity.

Machine-learning-based prediction: using protein sequence information to train a model that distinguishes allergens from non-allergens.

Most importantly, consider that predictive performance alone does not guarantee that a model has learned the underlying biology. Dataset construction, sequence redundancy, feature selection, and independent evaluation all strongly influence the apparent performance of a bioinformatics predictor.

Done!