Difference between revisions of "CSC352 Class Page 2017"
(→ ) |
(→ ) |
||
(117 intermediate revisions by the same user not shown) | |||
Line 4: | Line 4: | ||
<br /> | <br /> | ||
<center>[[CSC_352_--_2017 | Home]] | [[CSC352 Syllabus -- Spring 2017 | Syllabus]] | [[CSC352 Class Page 2017 | Schedule]] | | <center>[[CSC_352_--_2017 | Home]] | [[CSC352 Syllabus -- Spring 2017 | Syllabus]] | [[CSC352 Class Page 2017 | Schedule]] | | ||
− | [[CSC352_Class_Page_2017#Links_and_Resources | Links & Resources]]</center><br /> | + | [[CSC352_Class_Page_2017#Links_and_Resources | Links & Resources]] | [[CSC352_Project_Page_2017 | Final Project]]</center><br /> |
<br /> | <br /> | ||
{| class="mw-collapsible mw-collapsed" style="width:100%" border="1" | {| class="mw-collapsible mw-collapsed" style="width:100%" border="1" | ||
Line 287: | Line 287: | ||
::* '''Requirements for newsletter''': Add an image or figure to your newsletter. It could be a photo of you at the top, as the author, or a graph, or a logo. You pick the jpeg or png, and incorporate it in your newsletter. | ::* '''Requirements for newsletter''': Add an image or figure to your newsletter. It could be a photo of you at the top, as the author, or a graph, or a logo. You pick the jpeg or png, and incorporate it in your newsletter. | ||
| | | | ||
− | * | + | * [[CSC352 Game of Life Homework 1| Game of Life Homework]] |
| | | | ||
* [https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingQueue.html Java BlockingQueues] | * [https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingQueue.html Java BlockingQueues] | ||
Line 312: | Line 312: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* Game of Life, revisited |
+ | ::* Deadlocks, starvation, and the Dining Philosophers problem | ||
+ | ::* Thread states | ||
+ | ::* Measuring performance: speedup, bash scripts, R graphs | ||
+ | ::* Performance measures | ||
+ | ::* [[Media:CSC352_week6_2017.pdf | Slides Week 6 (pdf)]] | ||
+ | ::* [[Media:CSC352_CTutorial_2017.pdf | Slides for C Tutorial (pdf)]]. [[Solutions_to_Introduction_to_C_Presentation| Solutions for exercises]]. | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* <font color="magenta">Paper Presentation: Unreasonable, and Top500</font> | + | ::* <font color="magenta">Paper Presentation: Unreasonable (Lujun), and Top500</font> |
| | | | ||
− | * | + | * [[CSC352 Game of Life Homework 2| Homework 2]], due Thursday after the break, 3/23/17, at 1:00 p.m. |
| | | | ||
* | * | ||
|} | |} | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 339: | Line 346: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* C & MPI |
+ | ::* [[CSC352_GCC-Generated_Assembly_ARM_%26_Pentium| Comparing executable C code]] for ARM and for Pentium | ||
+ | ::* [[CSC352 MPI pi2.c program | Computing Pi]] with MPI | ||
+ | ::* [[Game of Life in C and MPI | Game of Life]] in C | ||
+ | <br /> | ||
+ | ::* [[Media:CSC352_Week7_MPISlides.pdf| MPI slides (pdf)]] | ||
+ | ::* [[Media:CSC352_Week7_CTutorial.pdf| C Tutorial slides (pdf)]] | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* <font color="magenta">Paper Presentation: Success of MPI, and General purpose vs GPU</font> | + | ::* A word about the [[CSC352_Project_Page_2017| final project]] |
− | ::* <font color="red">Newsletter due</font> | + | ::* <font color="magenta">Paper Presentation: Success of MPI (Zainab), and General purpose vs GPU</font> |
+ | ::* <font color="red">Newsletter due</font>. This week, add a sentence to each summary that explains why you picked this news item. Why does the news you reported matters? How is it important to you? Does it connect in any way to something you have read or programmed? Also, this week, add an a section to your newsletter that refers to Amdahl's law. The purpose of this is to have you create an equation with Latex. So, you can simply explain what Amdahl's law is, or you can find a bit of news in the past year that mentions Amdahl's Law, and take this opportunity to create an equation: Speedup = T(1)/T(N). | ||
| | | | ||
− | * | + | * Prepare a 1-page description of the project you will present at the end of the semester. Submit it before March 21, 1:00 p.m. to Moodle. See this [[CSC352_Project_Page_2017| page]] for information. |
| | | | ||
− | * | + | * [https://computing.llnl.gov/tutorials/mpi/ MPI Tutorials] by Blaise Barney of the Lawrence Livermore National Laboratory |
+ | * [http://cslibrary.stanford.edu/101/EssentialC.pdf Essential C] by Nick Parlante, Stanford U. | ||
|} | |} | ||
<br /> | <br /> | ||
Line 354: | Line 369: | ||
| | | | ||
'''SPRING BREAK''' | '''SPRING BREAK''' | ||
+ | <br /> | ||
+ | <!-- [[Image:CalvinDancing.gif | center]] --> | ||
|} | |} | ||
<!-- ============================================================= --> | <!-- ============================================================= --> | ||
<!-- ============================================================= --> | <!-- ============================================================= --> | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 375: | Line 393: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* 1-page project description due on Moodle |
+ | ::* C & MPI | ||
+ | ::* [http://www.mpich.org/static/docs/latest/www3/ All MPI functions (for reference only)] | ||
+ | ::* [[Media:CSC352_Week8_CTutorial.pdf| C-Tutorial Slides (pdf)]] | ||
+ | ::* [[Media:CSC352_Week8_MPISlides.pdf| MPI slides (pdf)]] | ||
+ | |||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* <font color="magenta">Paper Presentations: Virtualization, and View of Cloud</font> | + | ::* <font color="magenta">Paper Presentations: Virtualization, and View of Cloud (Kathleen)</font> |
− | + | ::* [[Setup_MPI_on_Hadoop_Cluster | Running MPI programs on Local Cluster (Hadoop Cluster)]] | |
+ | ::* [[Media:CSC352_Week8_MPISlides.pdf| MPI slides (pdf)]] | ||
| | | | ||
* | * | ||
| | | | ||
− | * | + | * [http://cslibrary.stanford.edu/101/EssentialC.pdf Essential C] by Nick Parlante, Stanford U. |
+ | * [https://computing.llnl.gov/tutorials/mpi/ MPI Tutorials] by Blaise Barney of the Lawrence Livermore National Laboratory | ||
+ | |||
+ | * [http://star.mit.edu/cluster/docs/latest/overview.html StarCluster overview]: must read. StarCluster is a set of Python programs that one installs on one's laptop to control clusters on Amazon Web Services (AWS). StarCluster can easily setup MPI and Hadoop clusters. | ||
|} | |} | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 402: | Line 430: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* MPI, Continued |
+ | ::* [[Media:CSC352_Week9_MPI.pdf | Slides (pdf)]] | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* <font color="magenta">Paper Presentations: MapReduce and (Nobody got fired + Beyond Hadoop)</font> | + | ::* MPI, Continued |
+ | ::* [[Media:CSC352_Week9_MPI.pdf | Slides (pdf)]] | ||
+ | ::* [[Tutorial:_Create_an_MPI_Cluster_on_the_Amazon_Elastic_Cloud_(EC2)| MPI on AWS, Part 1 (Hello World!)]] | ||
+ | ::* <font color="magenta">Paper Presentations: MapReduce (Sam) and (Nobody got fired + Beyond Hadoop) (Grace)</font> | ||
::* <font color="red">Newsletter due</font> | ::* <font color="red">Newsletter due</font> | ||
Line 413: | Line 445: | ||
|} | |} | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 424: | Line 457: | ||
{| class="mw-collapsible " style="width:100%" border="1" | {| class="mw-collapsible " style="width:100%" border="1" | ||
|- style="background:#dddddd;" | |- style="background:#dddddd;" | ||
− | |width="60%"| '''Topics''': | + | |width="60%"| '''Topics''': MPI, AWS |
|width="15%"| '''Lab/Hw''' | |width="15%"| '''Lab/Hw''' | ||
|width="25%"| '''Reading''' | |width="25%"| '''Reading''' | ||
Line 430: | Line 463: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* More MPI functions |
+ | ::* [[Tutorial:_Create_an_MPI_Cluster_on_the_Amazon_Elastic_Cloud_(EC2)| MPI on AWS, Part 1 (Hello World!)]] | ||
+ | ::* [[Computing_Pi_on_an_AWS_MPI-Cluster | MPI on AWS, Part 2 (Computing Pi)]] | ||
+ | ::* [[media:CSC352_Slides_Week10MPIAWS_2017.pdf| slides (pdf)]] | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* <font color="magenta">Paper Presentations: Spark and Microsoft Bet Future</font> | + | ::* <font color="magenta">Paper Presentations: Spark (Youyou), and Microsoft Bet Future (Isaiah & Muriel)</font> |
| | | | ||
− | * | + | * [[CSC352 Homework 3 2017| Homework 3]] |
| | | | ||
* | * | ||
|} | |} | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 457: | Line 494: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* Finish [[Computing_Pi_on_an_AWS_MPI-Cluster | tutorial on MPI on AWS (Computing Pi)]] |
+ | ::* Map-Reduce | ||
+ | :::* Introduction to Functional Programming | ||
+ | :::* Implementing Map-Reduce in Python | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* | + | ::* Class cancelled |
− | |||
| | | | ||
Line 468: | Line 507: | ||
|} | |} | ||
<br /> | <br /> | ||
+ | |||
==== ==== | ==== ==== | ||
---- | ---- | ||
Line 484: | Line 524: | ||
|-valign="top" | |-valign="top" | ||
| | | | ||
− | * '''Tuesday''' | + | * '''Tuesday/Thursday''' |
− | * ' | + | ::* <font color="magenta">Paper Presentations: Wikipedia dumps (Angie), and Can Cloud Reach Top500 (Vega)</font> |
− | ::* | + | ::* Map-Reduce |
+ | :::* Implementing Map-Reduce in Python: various problems solved with Map-Reduce | ||
+ | ::::* [[Map-Reduce_Examples| Example page]] | ||
+ | ::::* [[CSC352_Game_of_Life_in_Map-Reduce| Solving Conway's Game of Life with Map-Reduce]] | ||
+ | |||
+ | ::* [[Media:CSC352_Week12_MapReduce.pdf | Map-Reduce slides (pdf)]] | ||
+ | |||
| | | | ||
* | * | ||
Line 511: | Line 557: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* 3 | + | ::* Labs |
+ | :::* [[Tutorial:_Creating_a_Hadoop_Cluster_on_Amazon_AWS | Lab 1: Hadoop on AWS]] | ||
+ | :::* [[Hadoop_Tutorial_2.3_--_Running_WordCount_in_Python_on_AWS | Lab 2:Hadoop on AWS: streaming Python]] | ||
+ | :::* [[Hadoop_Tutorial_1.1_--_Generating_Task_Timelines | Lab 3: Creating Task Graphs]] | ||
+ | |||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* | + | ::* Project Presentations: |
+ | :::* Isaiah | ||
| | | | ||
− | * | + | * [[CSC352 Homework 4 2017| Homework 4, Game of Life using Map-Reduce on AWS]] |
| | | | ||
* | * | ||
Line 538: | Line 589: | ||
| | | | ||
* '''Tuesday''' | * '''Tuesday''' | ||
− | ::* | + | ::* Project Presentations: |
+ | :::* Grace (lunch time) | ||
+ | :::* Kathleen | ||
+ | :::* Riley | ||
* '''Thursday''' | * '''Thursday''' | ||
− | ::* | + | ::* Project Presentations: |
+ | :::* Sam & Angie (lunch time) | ||
+ | :::* Lujun & Vega | ||
+ | :::* Youyou and Zainab | ||
| | | | ||
* | * | ||
Line 547: | Line 604: | ||
|} | |} | ||
+ | ==== ==== | ||
+ | ---- | ||
+ | {| style="width:100%" border="0" | ||
+ | |width="45%"|'''Exam Week ''' May 9, 12 | ||
+ | |width="20%"| ''' ''' | ||
+ | |width="25%"| ''' ''' | ||
+ | |- | ||
+ | |} | ||
+ | <br /> | ||
+ | {| class="mw-collapsible " style="width:100%" border="1" | ||
+ | |- style="background:#bb8fce;" | ||
+ | |width="100%"| '''Final Exam Week''' | ||
+ | |-valign="top" | ||
+ | | | ||
+ | * '''Friday, May 12, 4:00 p.m.''': pdf of project due on Moodle. | ||
+ | * '''Friday, May 12, 4:00 p.m.''': Hadoop Game of Life due. | ||
+ | |} | ||
+ | <br /> | ||
+ | |||
+ | =Links & Resources= | ||
+ | <br /> | ||
+ | ==Map-Reduce== | ||
+ | <br /> | ||
+ | * [https://www.tutorialspoint.com/hadoop/hadoop_mapreduce.htm Map-Reduce Tutorial from tutorialpoint.com] | ||
+ | * [https://www.slideshare.net/andreaiacono/mapreduce-34478449 Good slides on Slide-Share by Andrea Jacono] | ||
<br /> | <br /> | ||
− | = | + | =Final Project= |
<br /> | <br /> | ||
:Go to the [[CSC352 Project Page 2017|project page]] for information on the projects. | :Go to the [[CSC352 Project Page 2017|project page]] for information on the projects. | ||
<br /> | <br /> | ||
+ | |||
=Links and Resources= | =Links and Resources= | ||
<br /> | <br /> | ||
+ | ==Cloud Services== | ||
+ | {|class="wikitable" | ||
+ | !colspan="2"|Cloud Services | ||
+ | |- | ||
+ | ! style="width: 30%;" | Name | ||
+ | ! Description | ||
+ | |- | ||
+ | | | ||
+ | [http://www.bitfusion.io/ bitfusion.io] | ||
+ | | | ||
+ | Provides AMIs on AWS with GPUs attached. The AMIs are already loaded with tensorflow and python, as well as Jupyter for remote development of Python programs. They offer a 5-day free trial. | ||
+ | |- | ||
+ | | | ||
+ | [https://www.digitalocean.com/ Digital Ocean] | ||
+ | | | ||
+ | A whole cloud infrastructure similar to AWS, but (apparently) not running on AWS. Offers a cluster infrastructure with many packages preloaded, including Node.js, Rails, Redis, MongoDB, Cassandra, Docker, MediaWiki, Django, Drupal, Drone, LAMP, LEMP, Ruby, etc. | ||
+ | Offers a $10 credit when first signing up. | ||
+ | |||
+ | |} | ||
+ | |||
==Latex== | ==Latex== | ||
<br /> | <br /> | ||
Line 573: | Line 676: | ||
===MPI=== | ===MPI=== | ||
* [https://computing.llnl.gov/tutorials/mpi/ Introduction to MPI], by Blaise Barney, Lawrence Livermore National Laboratory. Another short but excellent coverage of a topic in parallel processing, this time MPI. | * [https://computing.llnl.gov/tutorials/mpi/ Introduction to MPI], by Blaise Barney, Lawrence Livermore National Laboratory. Another short but excellent coverage of a topic in parallel processing, this time MPI. | ||
+ | * [http://mpitutorial.com/tutorials/launching-an-amazon-ec2-mpi-cluster/ Launching and Amazon EC2 MPI Cluster] | ||
+ | |||
===Hadoop=== | ===Hadoop=== | ||
* [[CSC352 MapReduce/Hadoop Class Notes| Thiebaut's class notes]] | * [[CSC352 MapReduce/Hadoop Class Notes| Thiebaut's class notes]] | ||
+ | * [https://blog.cloudera.com/blog/2014/01/how-to-create-a-simple-hadoop-cluster-with-virtualbox/ How to create a simple hadoop cluster with VirtualBox] | ||
+ | * [http://star.mit.edu/cluster/docs/latest/plugins/hadoop.html?highlight=hadoop StarCluster's Hadoop Plugin] | ||
===Microprocessors=== | ===Microprocessors=== | ||
Line 744: | Line 851: | ||
<br /> | <br /> | ||
<br /> | <br /> | ||
− | [[Category:CSC352]][[Category:Classes]] | + | [[Category:CSC352]][[Category:Classes]][[Category:Hadoop]][[Category:MPI]][[Category:Java]][[Category:Threads]] |
Latest revision as of 11:50, 25 April 2017
--D. Thiebaut (talk) 10:37, 2 December 2016 (EST)
Topics to be covered: | Week (approx) |
|
Week 1 |
|
Week 2 |
|
Week 3 |
|
Week 4 |
|
Week 6 |
|
Week 8 |
|
Week 10 |
|
Week 12 |
|
Week 13 |
Week 1 Jan 26, 27 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 2 Jan 31, Feb 2 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 3 Feb 7, 9 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 4 Feb 14, 16 |
Topics: | Lab/Hw | Reading |
|
|
Two papers to read for Thrusday. In class discussion. |
Week 5 Feb 21 |
Topics: | Lab/Hw | Reading |
|
Week 6 Feb 28, Mar 2 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 7 Mar 7, 9 |
Topics: | Lab/Hw | Reading |
|
|
|
SPRING BREAK
|
Week 8 Mar 21, 23 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 9 Mar 28, 30 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 10 Apr 4, 6 |
Topics: MPI, AWS | Lab/Hw | Reading |
|
|
Week 11 Apr 11, 13 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 12 Apr 18, 20 |
Topics: | Lab/Hw | Reading |
|
|
|
Week 13 Apr 25, 27 |
Topics: | Lab/Hw | Reading |
|
|
Week 14 May 2, 4 |
Topics: | Lab/Hw | Reading |
|
|
|
Exam Week May 9, 12 |
Final Exam Week |
|
Links & Resources
Map-Reduce
Final Project
- Go to the project page for information on the projects.
Links and Resources
Cloud Services
Cloud Services | |
---|---|
Name | Description |
Provides AMIs on AWS with GPUs attached. The AMIs are already loaded with tensorflow and python, as well as Jupyter for remote development of Python programs. They offer a 5-day free trial. | |
A whole cloud infrastructure similar to AWS, but (apparently) not running on AWS. Offers a cluster infrastructure with many packages preloaded, including Node.js, Rails, Redis, MongoDB, Cassandra, Docker, MediaWiki, Django, Drupal, Drone, LAMP, LEMP, Ruby, etc. Offers a $10 credit when first signing up. |
Latex
Here are some nice video tutorials on using Latex to write papers.
- Latex tutorial (video)
- How to write a thesis in Latex (video)
- Presentations with Beamer (video)
- TikZ examples (video), for drawing diagrams with lines, circles, ellipses, etc.
Smith Elements of Style
On-Line Resources
- Introduction to Parallel Processing, by Blaise Barney, Lawrence Livermore National Laboratory. A good read. Covers most of the important topics.
MPI
- Introduction to MPI, by Blaise Barney, Lawrence Livermore National Laboratory. Another short but excellent coverage of a topic in parallel processing, this time MPI.
- Launching and Amazon EC2 MPI Cluster
Hadoop
- Thiebaut's class notes
- How to create a simple hadoop cluster with VirtualBox
- StarCluster's Hadoop Plugin
Microprocessors
Papers
This is a tentative and non exhaustive list of papers scheduled for reading this semester.
Introduction
Paper | Pages |
---|---|
|
50 |
|
2 |
General/Parallelism
Paper | Pages |
---|---|
|
5 |
|
7 |
|
5 |
|
8 |
MPI
Paper | Pages |
---|---|
|
11 |
GPUs
Paper | Pages |
---|---|
6 |
Virtualization
Paper | Pages |
---|---|
5 |
Cloud
Paper | Pages |
---|---|
|
1.5 |
|
9 |
|
13 |
|
5 |
|
2 |
|
10 |
|
19 |
|
8 |
Ischia, Italy — May 18 - 20, 2009. |
4 |