Parallel programming - Revision history

WikiSysop: /* Exercises */

2025-11-06T13:29:37Z

Exercises

← Older revision		Revision as of 15:29, 6 November 2025
Line 12:		Line 12:
	== Exercises ==		== Exercises ==
	''Read a fasta file, find the complement strand of each entry, save result in new fasta file.''<br>		''Read a fasta file, find the complement strand of each entry, save result in new fasta file.''<br>
	Note: The python you should use on '''pupil1''' server is /~~opt~~/~~anaconda3_2021~~.11/bin/python3 - there are 2 pythons, so it is possible to choose wrong.<br>		Note: The python you should use on '''pupil1''' server is /home/ctools/anaconda3-2024.10-1/bin/python3 - there are 2 pythons, so it is possible to choose wrong.<br>
	'''1)'''<br>		'''1)'''<br>
	In lesson 6, Distributed computing, you created 2-3 programs and used the Queueing System to submit your sub tasks. You are going to repeat this exercise, but this time do not use the QS to do the sub tasks, but use		In lesson 6, Distributed computing, you created 2-3 programs and used the Queueing System to submit your sub tasks. You are going to repeat this exercise, but this time do not use the QS to do the sub tasks, but use

WikiSysop: /* Material for the lesson */

2024-06-17T09:28:11Z

Material for the lesson

← Older revision		Revision as of 11:28, 17 June 2024
Line 7:		Line 7:
	Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=bb4fc4d2-c0af-44bd-b181-af270122cf03 Libraries to use and how they work]<br>		Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=bb4fc4d2-c0af-44bd-b181-af270122cf03 Libraries to use and how they work]<br>
	Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=5aa86527-f501-44b6-9ee4-af270122a685 Parent/Child relationships]<br>		Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=5aa86527-f501-44b6-9ee4-af270122a685 Parent/Child relationships]<br>
	Powerpoint: [https://teaching.healthtech.dtu.dk/material/22112/~~HPCLife06~~-Programming.ppt Programming]<br>		Powerpoint: [https://teaching.healthtech.dtu.dk/material/22112/22112_09-Programming.ppt Programming]<br>
	Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=8ff003cd-f906-4d42-9f3d-af2701228c6d What is expected from exercises]		Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=8ff003cd-f906-4d42-9f3d-af2701228c6d What is expected from exercises]

WikiSysop: Created page with "{| width=500 style="float:right; margin-left: 10px; margin-top: -56px;" |Previous: Algorithms |Next: More parallelism |} == Material for the lesson == Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=6d186b7e-f5a7-4e1f-b613-af270122f263 Parallel Programming - problems]
Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=bb4fc4d2-c0af-44bd-b181-af270122cf03 Libraries to use and how they work]
Video: [https://panopto.dtu.dk/Panopto/Pages..."

2024-03-06T10:55:18Z

Created page with "{| width=500 style="float:right; margin-left: 10px; margin-top: -56px;" |Previous: Algorithms |Next: More parallelism |} == Material for the lesson == Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=6d186b7e-f5a7-4e1f-b613-af270122f263 Parallel Programming - problems] Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=bb4fc4d2-c0af-44bd-b181-af270122cf03 Libraries to use and how they work] Video: [https://panopto.dtu.dk/Panopto/Pages..."

New page

{| width=500 style="float:right; margin-left: 10px; margin-top: -56px;"
|Previous: [[Algorithms]]
|Next: [[More parallelism]]
|}
== Material for the lesson ==
Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=6d186b7e-f5a7-4e1f-b613-af270122f263 Parallel Programming - problems] 
Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=bb4fc4d2-c0af-44bd-b181-af270122cf03 Libraries to use and how they work] 
Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=5aa86527-f501-44b6-9ee4-af270122a685 Parent/Child relationships] 
Powerpoint: [https://teaching.healthtech.dtu.dk/material/22112/HPCLife06-Programming.ppt Programming] 
Video: [https://panopto.dtu.dk/Panopto/Pages/Viewer.aspx?id=8ff003cd-f906-4d42-9f3d-af2701228c6d What is expected from exercises]

== Exercises ==
''Read a fasta file, find the complement strand of each entry, save result in new fasta file.'' 
Note: The python you should use on '''pupil1''' server is /opt/anaconda3_2021.11/bin/python3 - there are 2 pythons, so it is possible to choose wrong. 
'''1)''' 
In lesson 6, Distributed computing, you created 2-3 programs and used the Queueing System to submit your sub tasks. You are going to repeat this exercise, but this time do not use the QS to do the sub tasks, but use
'''subprocess''' and '''joblib''' as shown in the powerpoint example. The code is the same as second exercise except for the part that do the sub tasks, and waits for them to finish. 
You must still use the QS to submit the initial main job.

'''2)''' 
If you feel this was too easy, then the administrator should only index the fasta file and launch workers with the indexed info using '''joblib''' and an internal worker function.
The workers will create the complement strand entry as a file and the administrator will collect the pieces as in above. This is using techniques we have already covered. The result will be the same as above, but faster.

'''3)''' 
For an extra challenge, then create a different worker function that writes the complement stand directly back into the file the right place as has been discussed earlier.
Here you need your own "private" fasta file, because you can not write in mine in /home/projects/pr_course.

Time everything to see the difference. Don't use more than 8 workers at a time - '''joblib''' helps with that.