Exercise:AllergenicityPrediction
Exercise written by Carolina Barra Quaglia
Introduction
Climate change and the increasing demand for sustainable food production are driving the search for alternative protein sources. Algae are particularly interesting because they are abundant, can be cultivated without agricultural land, and may provide a valuable source of proteins and other nutrients.
However, introducing a new protein source into the human diet also raises an important question: could some of these proteins cause allergic reactions?
Food allergens are generally proteins that are recognized by the immune system and can trigger an inappropriate immune response in susceptible individuals. Experimental characterization of every protein in a new food source is expensive and time-consuming. Bioinformatics can therefore play an important role in identifying proteins with potential allergenic properties and prioritizing candidates for experimental investigation.
Learning Objectives
After completing this exercise, you should be able to:
- Identify appropriate databases and experimental evidence for investigating protein allergenicity
- Explain how sequence similarity to known allergens can provide evidence of potential cross-reactivity or sensitisation
- Use sequence-search tools such as BLAST to investigate relationships between allergens
- Explain how existing bioinformatics predictions can be used as proxies when a dedicated predictor is unavailable
- Describe the major steps required to construct a supervised machine-learning predictor from biological sequence data
- Explain why sequence clustering is important when separating biological data into training and test sets
- Train and evaluate a Random Forest classifier for protein allergenicity.
- Interpret commonly used measures of predictive performance.
- Identify potential limitations of a machine-learning model and propose strategies for improving its performance.
- Design and evaluate an improved allergenicity-prediction strategy.
The research question
Researchers at DTU Food are interested in developing a high-protein food product based on brown algae. Before introducing a new protein source into the human diet, however, we need to consider its potential allergenicity.
Your task is therefore to investigate:
Could proteins from brown algae represent an allergenic risk, and how could we predict this risk computationally?
We will focus particularly on Ectocarpus siliculosus, a species belonging to the brown algae (Phaeophyceae, taxid: 2870).
We will first explore how known allergens and sequence similarity can be used to assess potential allergenicity. We will then build our own machine-learning classifier for allergen prediction using a Random Forest. Finally, you will try to improve the predictor using alternative features, preprocessing strategies, machine-learning algorithms, or other approaches of your choice.
PART A: Investigating Allergenicity
Where Would You Start?
Every good bioinformatics investigation starts by asking what information is already available.
Q1A: What would be your first approach to determine whether brown algae contain potential allergens? Find one database containing experimentally characterized allergens. Consider what biological information you would search for and what properties of the proteins might provide evidence of allergenic potential.
Q1B: Which approach would you use to predict allergic responses of a new protein? NetAllergen
Now imagine that no computational predictor specifically designed for allergenicity has yet been developed.
Q1C: How could you attempt to predict allergenicity in silico? Which existing protein predictions or sequence properties could potentially be used as proxies for allergenicity? There is not necessarily a single correct solution here. Think about what biological properties characterize allergens and what computational tools could provide information about those properties.
Sequence Similarity and Cross-Reactivity
Allergic reactions are not always restricted to the protein that originally caused sensitization. Antibodies recognizing one allergen may sometimes recognize structurally or evolutionarily related proteins from another organism.
ALK has developed allergen immunotherapy treatments against house dust mites. One of the major allergens from Dermatophagoides pteronyssinus is Der p 1.
Following prolonged allergen immunotherapy, many patients show reduced allergic symptoms towards house dust mites. Interestingly, reductions in reactions to some other allergens have also been observed.
Let us investigate one possible explanation.
Go to the NCBI BLAST server:
https://blast.ncbi.nlm.nih.gov/Blast.cgi
Retrieve the protein sequence of Der p 1 and investigate whether similar proteins occur among known allergens from other species.
In particular, consider allergens associated with papaya and pineapple (Ananas).
Q2: Can you identify a possible molecular explanation for cross-reactivity between these allergens?
Consider:
Sequence similarity.
Protein family and function.
Conserved regions that could potentially be recognized by the same antibodies.
Do your observations support the hypothesis that these proteins could exhibit immunological cross-reactivity? Explain your reasoning.
Back to Brown Algae
We can now return to our original food-safety problem.
Q3: Based on your investigation, could an immunotherapy targeting an existing allergen potentially reduce allergic responses to proteins found in brown algae?
Use sequence or protein-family evidence to justify your answer.
What would have to be true about the algal proteins for such cross-reactivity to be plausible?
PART B: Develop your own allergenicity predictor
We will now move from investigating individual proteins to developing a computational method capable of predicting allergenicity.
Our first predictor will use a Random Forest classifier.
You will work in Google Colab. If you have not used Google Colab before, you can find an introduction here
Open the notebook for this exercise:
Make a copy of the notebook in your own account, so you can modify it.
Run the notebook step by step rather than simply selecting Run all. Your goal is to understand what happens to the data at each stage.
The Machine-Learning Pipeline
Pay particular attention to the following steps:
1. Data Collection and Filtering
You need to upload the data file from DTU Learn called data.csv
Inspect the allergen and non-allergen datasets used to construct the predictor.
Consider:
What constitutes a positive example?
What constitutes a negative example?
Are the two datasets comparable?
Could the way the data were collected introduce biases into the predictor?
2. Sequence Clustering
Biological sequence datasets frequently contain highly similar proteins.
If nearly identical proteins occur in both the training and test sets, a predictor may appear to perform extremely well simply because it has already encountered very similar examples during training.
Inspect how clustering is performed in the notebook.
Q4: Why is clustering important before constructing the training and test datasets?
What might happen to the estimated performance if highly similar sequences were allowed to occur in both sets?
3. Training and Test Split
The dataset is divided into data used for training the model and independent data used for evaluating it.
Q5: Why should the final test dataset not be used during model training or model optimization?
4. Training
We will use a Random Forest as our first machine-learning model.
Inspect the features used as input to the classifier and how the model is trained.
Q6: What information about a protein sequence is the Random Forest actually using to distinguish allergens from non-allergens?
5. Evaluation
Finally, evaluate your trained model on the test dataset.
Do not rely on a single performance measure. Inspect the evaluation metrics produced by the notebook and consider what each tells you about the model.
Q7: Report the performance of your allergenicity predictor.
Include the relevant performance metrics and briefly interpret the results.
Q8: Based on these results, what do you consider the main limitations of the current predictor?
PART C Can you build a better predictor
Now it is your turn.
The Random Forest from Part B provides our baseline model. Your task is to modify the computational pipeline and determine whether you can improve allergenicity prediction.
You are free to experiment.
For example, you could investigate:
Different machine-learning algorithms.
Random Forest hyperparameters.
Alternative representations of protein sequences.
Additional sequence-derived features.
Feature selection or dimensionality reduction.
Different preprocessing or normalization strategies.
Strategies for dealing with class imbalance.
Regularization to reduce overfitting.
Alternative clustering or train/test splitting strategies.
Combining several predictors or biological features.
You are also welcome to explore approaches that are not suggested above.
Documentation and examples for machine-learning methods implemented in scikit-learn can be found here:
https://scikit-learn.org/stable/index.html
You are allowed to use ChatGPT or other large language models in this part of the exercise. However, you should understand the changes you introduce into the code and be able to explain why you expected them to improve the model.
If programming is new to you, a good strategy is to start with the existing notebook and ask an LLM to help you modify one component at a time. Test each modification and determine whether it actually improves performance.
The Challenge
Try to obtain better predictive performance than the baseline model from Part B.
Be careful: improving performance on the training data does not necessarily mean that you have built a better predictor.
Your model should generalize to proteins it has not encountered before.
Q9: Describe the changes you made to the original pipeline.
For each major modification, briefly explain why you expected it to improve allergenicity prediction.
Q10: Report the performance of your improved predictor and compare it with the baseline Random Forest from Part B.
Did your changes actually improve performance?
Q11: What would you try next if you had more time? Think beyond simply changing machine-learning algorithms. What additional biological information or data could potentially make allergenicity prediction more accurate?
This exercise can also serve as inspiration for the mini research project in the second part of the course. If you identify an interesting direction for improving allergenicity prediction, you are welcome to develop it further.
Conclusion =
You should now have experience with two complementary approaches to investigating protein allergenicity:
Knowledge and similarity-based analysis: using experimental evidence, known allergens, sequence similarity, and biological knowledge to identify potential cross-reactivity.
Machine-learning-based prediction: using protein sequence information to train a model that distinguishes allergens from non-allergens.
Most importantly, consider that predictive performance alone does not guarantee that a model has learned the underlying biology. Dataset construction, sequence redundancy, feature selection, and independent evaluation all strongly influence the apparent performance of a bioinformatics predictor.
Done!