Exercise:AllergenicityPrediction

Exercise written by Carolina Barra Quaglia

Introduction

Climate change and the increasing demand for sustainable food production are driving the search for alternative protein sources. Algae are particularly interesting because they are abundant, can be cultivated without agricultural land, and may provide a valuable source of proteins and other nutrients.

However, introducing a new protein source into the human diet also raises an important question: could some of these proteins cause allergic reactions?

Food allergens are generally proteins that are recognized by the immune system and can trigger an inappropriate immune response in susceptible individuals. Experimental characterization of every protein in a new food source is expensive and time-consuming. Bioinformatics can therefore play an important role in identifying proteins with potential allergenic properties and prioritizing candidates for experimental investigation.

Learning Objectives

After completing this exercise, you should be able to:

• Identify appropriate databases and experimental evidence for investigating protein allergenicity

• Explain how sequence similarity to known allergens can provide evidence of potential cross-reactivity or sensitisation

• Use sequence-search tools such as BLAST to investigate relationships between allergens

• Explain how existing bioinformatics predictions can be used as proxies when a dedicated predictor is unavailable

• Describe the major steps required to construct a supervised machine-learning predictor from biological sequence data

• Explain why sequence clustering is important when separating biological data into training and test sets

• Train and evaluate a Random Forest classifier for protein allergenicity.

• Interpret commonly used measures of predictive performance.

• Identify potential limitations of a machine-learning model and propose strategies for improving its performance.

• Design and evaluate an improved allergenicity-prediction strategy.

The research question

Researchers at DTU Food are interested in developing a high-protein food product based on brown algae. Before introducing a new protein source into the human diet, however, we need to consider its potential allergenicity.

Your task is therefore to investigate:

Could proteins from brown algae represent an allergenic risk, and how could we predict this risk computationally?

We will focus particularly on Ectocarpus siliculosus, a species belonging to the brown algae (Phaeophyceae, taxid: 2870).

We will first explore how known allergens and sequence similarity can be used to assess potential allergenicity. We will then build our own machine-learning classifier for allergen prediction using a Random Forest. Finally, you will try to improve the predictor using alternative features, preprocessing strategies, machine-learning algorithms, or other approaches of your choice.

PART A: Investigating Allergenicity

Where Would You Start?

Every good bioinformatics investigation starts by asking what information is already available.

Q1A: What would be your first approach to determine whether brown algae contain potential allergens? Find one database containing experimentally characterized allergens. Consider what biological information you would search for and what properties of the proteins might provide evidence of allergenic potential.

Q1B: Which approach would you use to predict allergic responses of a new protein? NetAllergen

Now imagine that no computational predictor specifically designed for allergenicity has yet been developed.

Q1C: How could you attempt to predict allergenicity in silico? Which existing protein predictions or sequence properties could potentially be used as proxies for allergenicity? There is not necessarily a single correct solution here. Think about what biological properties characterize allergens and what computational tools could provide information about those properties.

Sequence Similarity and Cross-Reactivity

Allergic reactions are not always restricted to the protein that originally caused sensitization. Antibodies recognizing one allergen may sometimes recognize structurally or evolutionarily related proteins from another organism.

ALK has developed allergen immunotherapy treatments against house dust mites. One of the major allergens from Dermatophagoides pteronyssinus is Der p 1.

Following prolonged allergen immunotherapy, many patients show reduced allergic symptoms towards house dust mites. Interestingly, reductions in reactions to some other allergens have also been observed.

Let us investigate one possible explanation. Retrieve the protein sequence of Der p 1 and investigate whether similar proteins occur among known allergens from other species.

Go to the NCBI BLAST server In particular, consider allergens associated with papaya and pineapple (Ananas).

Q2: Can you identify a possible molecular explanation for cross-reactivity between these allergens? Consider: Sequence similarity, Protein family and function, Conserved regions that could potentially be recognized by the same antibodies. Do your observations support the hypothesis that these proteins could exhibit immunological cross-reactivity? Explain your reasoning.

Back to Brown Algae We can now return to our original food-safety problem.

Q3: Based on your investigation, could an immunotherapy targeting an existing allergen potentially reduce allergic responses to proteins found in brown algae? Use sequence or protein-family evidence to justify your answer.What would have to be true about the algal proteins for such cross-reactivity to be plausible?

PART B: Can you beat BLAST

You will now develop your own allergenicity predictor using a Random Forest classifier in Google Colab. If you have not used Google Colab before, you can find an introduction here

Open the notebook for this exercise: Notebook

Make a copy of the notebook in your own gmail account. Run the notebook step by step rather than simply selecting "Run all". Your goal is to understand what happens to the data at each stage.

You will first need to upload the data to google Colab. The data is placed in DTU Learn as data.csv

Q4: What will the model see? Inspect the data. How is the data structured? List the type of input features.

• Now pick a random seed (replace the default 42) and run the feature-generation and Random Forest training steps in the notebook.

The notebook trains and tests the model using the different sequence partitions. For simplicity these partitions are provided to you in the data. However, remember that these are not random partitions of the data, instead the sequences have been clustered according to sequence homology so no similar proteins are contained across partitions. Predictions from the independent test partitions are then combined to obtain an overall evaluation of the model.

Run the evaluation section of the notebook.

It will report several performance measures:

Accuracy — fraction of all predictions that are correct

Precision — fraction of proteins predicted as allergens that actually are allergens

Recall (sensitivity) — fraction of allergens correctly identified by the predictor

Specificity — fraction of non-allergens correctly identified.

MCC (Matthews Correlation Coefficient) — overall agreement between predictions and observations, taking all four outcomes of the confusion matrix into account.

ROC-AUC — ability of the model to discriminate allergens from non-allergens across different classification thresholds.

Q5: How well does it work? Report the performance of your predictor

Inspect the confusion matrix produced by the notebook.

The four possible outcomes are:

True positive (TP): allergen → predicted allergen

True negative (TN): non-allergen → predicted non-allergen

False positive (FP):non-allergen → predicted allergen

False negative (FN): allergen → predicted non-allergen

Q6: Where does the model make mistakes? Imagine that this predictor is being used as an initial "screening tool for proteins in a novel food".

Accuracy is not the whole story Two models can have similar accuracy while behaving differently towards allergens and non-allergens.

Inspect the ROC curve generated by the notebook. The diagonal line represents the performance expected from random classification. An AUC of approximately 0.5 corresponds to random discrimination, whereas a value approaching 1.0 indicates increasingly good separation between the two classes.

Q7: Why it makes more sense to use AUC than accuracy to report performance. Record the AUC of your model.

You now have a 'baseline model

Q8: How the number of estimators 10, 100, 1000 affects model performance?

Keep these performance values. In Part C, your task will be to determine whether you can improve them.

PART C Can you build a better predictor

Now it is your turn.

The Random Forest from Part B provides our baseline model. Your task is to modify the computational pipeline and determine whether you can improve allergenicity prediction.

You are free to experiment.

For example, you could investigate:

Different machine-learning algorithms.

Random Forest hyperparameters.

Alternative representations of protein sequences.

Additional sequence-derived features.

Feature selection or dimensionality reduction.

Different preprocessing or normalization strategies.

Strategies for dealing with class imbalance.

Regularization to reduce overfitting.

Alternative clustering or train/test splitting strategies.

Combining several predictors or biological features.

You are also welcome to explore approaches that are not suggested above.

Documentation and examples for machine-learning methods implemented in scikit-learn can be found here:

https://scikit-learn.org/stable/index.html

You are allowed to use ChatGPT or other large language models in this part of the exercise. However, you should understand the changes you introduce into the code and be able to explain why you expected them to improve the model.

If programming is new to you, a good strategy is to start with the existing notebook and ask an LLM to help you modify one component at a time. Test each modification and determine whether it actually improves performance.

The Challenge

Try to obtain better predictive performance than the baseline model from Part B.

Be careful: improving performance on the training data does not necessarily mean that you have built a better predictor.

Your model should generalize to proteins it has not encountered before.

Q9: Describe the changes you made to the original pipeline.

For each major modification, briefly explain why you expected it to improve allergenicity prediction.

Q10: Report the performance of your improved predictor and compare it with the baseline Random Forest from Part B.

Did your changes actually improve performance?

Q11: What would you try next if you had more time? Think beyond simply changing machine-learning algorithms. What additional biological information or data could potentially make allergenicity prediction more accurate?

This exercise can also serve as inspiration for the mini research project in the second part of the course. If you identify an interesting direction for improving allergenicity prediction, you are welcome to develop it further.

Conclusion =

You should now have experience with two complementary approaches to investigating protein allergenicity:

Knowledge and similarity-based analysis: using experimental evidence, known allergens, sequence similarity, and biological knowledge to identify potential cross-reactivity.

Machine-learning-based prediction: using protein sequence information to train a model that distinguishes allergens from non-allergens.

Most importantly, consider that predictive performance alone does not guarantee that a model has learned the underlying biology. Dataset construction, sequence redundancy, feature selection, and independent evaluation all strongly influence the apparent performance of a bioinformatics predictor.

Done!