CSC352 Class Page 2013

--D. Thiebaut (talk) 11:15, 9 August 2013 (EDT)

---

Main Page|Syllabus|Project Page | PIAZZA

Weekly Schedule

Week	Topics	Reading
Week 1 9/3	Tuesday Syllabus Introduction to final project Approach Programming Testing ==> paper (see 2011 paper for example). Parallelism: going to the source: Interrupts! 8086 type of interrupts (simplified) Interrupt Vector Interrupt Priority Context Switch Stack and Stack Frame Global and Local Variables What is a process? What is a thread? Thursday Goals of multithreading: Enhanced performance Increased throughput Greater user responsiveness What should we remember 5 years from now? Introduction to a graph we'll use all throughout the semester. The idea of threads Thread 1 ----------------------|====|-------------------------> time Thread 2 ------------|====|-----------------------------------> time Multithreaded programming. Computing Pi Mini Lab 1 Regroup and gather statistics on the different machines in the classroom Comments on bimonthly newsletter The format should be similar to the ACM Tech News format. The header should contain a title, your name, the class (CSC352) and the date Each paragraph should have a header with a title, the source of news, the date, and possibly a link to the full article. The paragraph describing a news item should be between 3 to 10 lines, give or take. Write 1 full page to 2 pages, depending on the richness of events in the technology field Feel free to present N-1 topics with just 3 lines, and 1 topic which you highlight with a longer paragraph. Topics: anything related to parallelism: computers, mobile platforms, cloud, companies, new software, new algorithms, conferences, people in the field, etc. Good sources of information to get started: the ACM Tech News Wired magazine The NYT Technology Bits MIT Technology Review The Official Google blog Review: Tom's Hardware Some of the sites listed in this page's top 40 list. Recommendation for news aggregator: Feedly.com Homework: play with Latex. Find or adapt a document template for your needs (minimalist is the name of the game at this point), and start gathering news bits. First newsletter due Thursday Sept. 19th. The ACM Tech News format is a good and simple format to emulate.	Definition of a Process Definition of a Thread Coverage (with examples) of Processes and Threads, provided by Oracle Definition of an interrupt References on Latex The Latex Project A not-so-short Introduction to Latex (pdf) The TeX User Group WikiBooks guide to Latex Users Guide to Writing a Thesis in a Physics/Astronomy Institute of the University of Bonn
Week 2 9/10	Tuesday Introduction to measuring performance. Comparing execution times. Introduction to Speedup( N ), where N is the number of threads, or the number of processors. Amdahl's Law A bit of Computer Architecture: Cores and Caches (last slide taken from [www.cs.utexas.edu/users/mckinley/352/lectures/16.pdf http://www.cs.utexas.edu/users/mckinley/352/lectures/16.pdf]) Synchronizing Java Threads Using Bash to run a program multiple times Thursday Discussion of A View of Parallel Processing from Berkeley. Prepare a 1- to 2-page summary of the paper in Latex. Hand-in the summary in class. No summaries will be accepted after class. Some topics taken from the paper: Moore's Law: Processor-DRAM gap increasing (graph taken from www.cs.virginia) Barnes and Hut approach to N-Body problem Monte Carlo: see the current Homework #1 and Solution Example of Many-Core architecture: (Image taken from URL: http://www.altera.com/technology/system-design/articles/2012/multicore-many-core.html) nanometers: where are we now? ( Image taken from http://en.wikipedia.org/wiki/22_nanometer) Ring network connecting cores in Intel architecture (http://semiaccurate.com/2012/08/28/intel-details-knights-corner-architecture-at-long-last/) Short preparation for Maggie Lind's tour of the SCMA on Tuesday. Meeting place is entrance of SCMA. What the project is about can be included in the field of Culturomics Homework #1, due 9/19/13 and Solution	Java Synchronization methods
Week 3 9/17	Tuesday: Guided tour by Maggie Lind at the SCMA. Meeting in entrance of museum. Thursday: Newsletter #1 due today! Organization and Anatomy of the View from Berkeley paper. How to prepare a talk: An example: "The Butler Did It!" (and also this) Java synchronization, a presentation Producer-Consumer Lab Homework 2, will be due 9/28/13 at midnight. Solution programs	All the data structures of interest (concurrent non-blocking and blocking) can be found in the Oracle documentation. The information is a bit cryptic, but you need to get comfortable with it! BlockingQueue ConcurrentLinkedQueue
Week 4 9/24	Tuesday: Guest Lecture/Informal discussion with by Tim Draper Some questions to start the conversation: How has the cloud infrastructure changed entrepreneurship, if at all? There is a whole ecosystem growing around the cloud services offered by Amazon and the other players: new companies offering services and using Amazon's AWS for example. What are some of the most interesting companies/ideas/technologies emerging that you have discovered or been involved with? There is tremendous worries about the safety and privacy of data in the cloud. Is this an area of growth students should consider? What other area of growth do you see that students should keep in their view-sight? If a graduating major is interested in joining a start-up company, what are the signs she should be looking for before joining such a group? Some students are interested in a management track, starting at a big company and climbing fast. What is your advice for best preparing for this type of career? What is the most exciting development in your eyes happening now with cloud technology? It has been said that the 21st century is the century of the entrepreneur. Do you see this as true? Companies rise and fall. Microsoft was once the place where all our majors wanted to go. The most prestigious company for programmers. Now it's Google, and Facebook. Which company(ies) do you see as potential new meccas for programmers? If somebody were to form a start-up with friends. Say 10 people. Who/What/Where? Who should the people be? What field should they be experts in? Where should the company locate? Review of Homework 1 and its Solution. Understand static variables don't use global random generators! /usr/bin/time multiplies time by the # of cores for threaded applications be sure to understand if you need the same random seed or a different seed in your experiments create a different user on your laptop with no extra applications loaded in the background (e.g. Skype): less stress on the O.S. Thursday Mountain Day!
Week 5 10/1	Tuesday (Grace Hopper Conference) Introduction to Packing pdf and ppt Studying the Red-Black Tree data-structure Why is it not thread-safe? How can we make it thread-safe? Devise a test to verify that the modifications have resulted in a thread-safe class Profiling Java applications (introduction to Java's GC). Thursday (Grace Hopper Conference) Newsletter #2 due today. Please include 1 news item about some form of image collage, representation of many images in some form, hopefully digital. Also, please use a Latex feature you haven't used in your first newsletter Elaborating a roadmap for the final [Project]
Week 6 10/8	Tuesday An Introduction to C ( keynote) (we stopped at "Arrays") C solutions for Intro. to C Exercises Thursday: 50-min Presentation by Rocco Piccinino, Head of the Young Science Library, in FH345 on using the library resources for research. Homework 3 will be due 10/22. Solution Programs	MPI by Blaise Barney, at Lawrence Livermore National Laboratory: an excellent reference on MPI
Week 7 10/15	Tuesday: Fall Break Thursday Introduction to the Final Project An Introduction to MPI ( keynote). You may want to install MPI on your MacBook (We stopped on Thursday on the MPI_Send() function.)	Super MPI-Introduction from the Lawrence Livermore Nat. Lab
Week 8 10/22	Tuesday Paper presentation: Learning from the Success of MPI, presented by Gavi ( Bibtex) Hadoop0 accounts Learn how to become rsync champions! Continuation of the introduction to MPI ( keynote). We stopped on Thursday on the MPI_Send() function. Code for the pi2.c program computing Pi using summation of a series Newsletter #3 due today! Thursday Continuation of the introduction to MPI ( keynote) Introduction on how to operate a MySQL database ( keynote) A project-oriented MPI example. Bring your Mac! Homework 4 on C and MPI. Solution programs
Week 9 10/29	Tuesday Paper presentation: Server Virtualization Architecture and Implementation presented by Emily A few words about newsletters Where is What on Hadoop0 MySQL Exercises Combining C, MySQL and MPI: combing through a lot of code Project discussion Thursday Creating an MPI Cluster on Amazon, ( Accompanying keynote), followed by a second tutorial on computing Pi on a 10-node AWS cluster.
Week 10 11/5	Tuesday: Otelia Cromwell Day Thursday: Paper presentation: MapReduce: Simplified Data Processing on Large Clusters presented by Sharon Pamela Newsletter #4 due today!. Please include at least one image, and at least one news item covering some form of project that could be related or influential for our own wiki-collage project. See this document on writing theses for information about the inclusion of images in Latex. The end section has a good list of sites that have good coverage of Latex topics. There is also plenty of information on the Web about this subject. Preparation for Homework 5: attaching EBS volumes. We'll do a lab in class to create and attach an EBS volume to your AWS cluster. Homework 5 and Solution
Week 11 11/12	Tuesday Paper presentation: General-Purpose vs. GPU: Comparisons of Many-Cores on Irregular Workloads, presented by Yoshie Questions about the paper: What kind of paper is this? Broad distribution? Research? Small group? Organization? Abstract? Introduction? Definition of specialized terms? Early enough in the paper? Are the contributions of paper clear? The section on related research sufficient? What is being compared? Similar machines? Hardware? Software? Are authors partial? Do they have a stake? How does the paper advance the state of research? What does it tell us about the way computer systems evolve? Thinking about the project What do we know better about the overall project. What pieces have we looked at? What is it we don't know? Can we turn any of these questions into a project? Thursday A few comments on Manager/Worker paradigm in MPI: not the only one. Many logical communication networks do not match star pattern Continuation of Project Discussion Quick Introduction to Hadoop/MapReduce ( Accompanying keynote) MapReduce lab on AWS Script to run Hadoop WordCount program on AWS	Yahoo has some very good reading material on Hadoop. One reason is that they may be one of the largest users of AWS and of Hadoop. Yahoo Developers Network: Tutorial on Hadoop. All the chapters are worth reading!
Week 12 11/19	Tuesday: 1 month to go (exactly) before the project is due (Dec. 19)! Student-directed work (DT @ INFOCOMP 2013) Finish the MapReduce lab on AWS and make sure you do the Challenge 2 part of the lab. Food for thought: some videos I suggest one of you connects her laptop to the projection system and you all watch these videos together. After each one, discuss it as a group. Take notes and be ready to share your comments during Thursday's class when we resume our regular schedule. The Cave 2 Project at the University of Illinois: Just another hardware solution for presenting the user with a large number of pixels; in this case 27320 x 3072 pixels. Short, 3 minutes. The Creators Projects video This video is not necessarily anything that can work for us, but it's just "food for thought." Just a different way an artist has come up to make still pictures interesting to look at. Short, 6 minutes. O'Reilly Radar Videos Tim O'Reilly is a visionary who figured out a long time ago that computer technology was an exploding field and he started a very successful line of books to support all new technology projects that were emerging and promising. The books all have animals on them and are uniquely easy to spot. O'Reilly now also has an on-line channel (O'Reilly Radar), and organizes conferences with top researchers and intellectuals in the field of computer science. The first video is with Doug Cutting, one of the creators of Hadoop. He makes some very good points about what Hadoop is, what it is good at, and what it might not be good at (Homework 5 lesson?). After Cutting you can skip the 2nd interview (about video technology) and zip to the 3rd interview with Jeremy Howard, at time-tag 13:47. Then learn about big data and analytics, and what is said of data scientists. About 12 minutes total. Good interview of Tim O'Reilly describing Web 2.0, and his view of a data-driven Internet. 8-minute long. You may want to think about how our wikipedia data (images, stats) relate to what is said about data as described in the interview. About 8 minutes. The next video filmed in June 2013 presents Bruno Fernandez-Ruiz of Yahoo, who speaks about Hadoop since 2005, Hadoop today, and what is ahead. An important type of data property Fernandez-Ruiz is interested in is timeliness, which we haven't really looked at for our project, but you will see that it could apply easily to the dynamics of wikipedia. Some interesting statistics about the number of servers, the size of the HDFS they use, the number of processes are given. About 17 minutes. If you have at least 25 minutes left before the class time is over, do the MapReduce-Python lab, without attempting the challenges at the end. We'll do these together. Thursday: Tentative guest lecture: Nick Howe on CUDA and GPUs Some thoughts about INFOCOMP 2013 ( keynote) Going over Homework #5 ( Walking a 2-Level Directory in C)
Week 13 11/26	Tuesday: No newsletter due Paper presentation: A View of Cloud Computing presented by Danaë. 5-minute project presentations (everybody) Instead of a newsletter, you may turn today a draft of an introduction to your final project. If you have too much work this week, you can turn this in on 12/3. A bit of Bash The challenges of the MapReducing in Python lab Thursday: Thanksgiving Break
Week 14 12/3	Tuesday Paper presentation: The Unreasonable Effectiveness of Data presented by Julia Instead of a newsletter, you need to turn in a draft of an introduction to your final project (unless you submitted it last week). The challenges of the MapReducing in Python lab. We have done Challenge #1 last time. We'll look at Challenge #2 and #3. Some feedback on Homework #5 and one solution. MapReduce task graphs Thursday Distribution of Map and Reduce tasks over time Project work and discussion 20-minute individual session (in class) to go over project, questions, setup, etc...
Week 15 12/10	Tuesday: Last Day of Class 20-minute presentations of projects. Suggested outline: The context: how your project fits in the overall pictures Has other similar work been done and documented before What you decided to do The challenges The choices The target experiments Preliminary results Expected results Possible directions for continuing research after the project An afternoon of packing circular crepes, including some imaginative variations...

Links and Resources

Latex

• Latex tutorial (video)
• How to write a thesis in Latex (video)
• Presentations with Beamer (video)
• TikZ examples (video)

Smith Elements of Style

• "Writing Papers" from the Smith College Jacobson Center for writing

On-Line Resources

• Introduction to Parallel Processing, by Blaise Barney, Lawrence Livermore National Laboratory. A good read. Covers most of the important topics.
• Introduction to MPI, by Blaise Barney, Lawrence Livermore National Laboratory. Another short but excellent coverage of a topic in parallel processing, this time MPI.
• A 90-Minute Guide to Modern Microprocessors

Classics

• Designing and Building Parallel Programs, by Ian Foster. A relatively old reference (1995), with still good information.

Papers

This is a tentative and non exhaustive list of papers scheduled for reading this semester.

Introduction

Paper	Pages
The Landscape of Parallel Computing Research: A View From Berkely, 2006, still good! (very long paper)	50
Update on a view from Berkeley, 2010. (short paper)	2

General/Parallelism

Paper	Pages
Parallel Computing with Patterns and Frameworks, 2010, XRDS.	5
Understanding Throughput-Oriented Architectures, CACM, 2010.	7
The Unreasonable Effectiveness of Data, by Halevy, Norvig, Pereira, IEEE Intelligent Systems, IEEE Intelligent Systems, March 2009, Vol. 24, No. 2, pp. 8-12.	5

MPI

Paper	Pages
Learning from the Success of MPI, by WIlliam D. Gropp, Argonne National Lab, 2002.	11

GPUs

Paper	Pages
General-Purpose vs. GPU: Comparisons of Many-Cores on Irregular Workloads, 2010	6

Virtualization

Paper	Pages
Server Virtualization Architecture and Implementation, xrds, 2009	5

Cloud

Paper	Pages
The NIST Definition of Cloud Computing (Draft) (very short paper)	1.5
A View of Cloud Computing, 2010, By Armbrust, Michael and Fox, Armando and Griffith, Rean and Joseph, Anthony D. and Katz, Randy and Konwinski, Andy and Lee, Gunho and Patterson, David and Rabkin, Ariel and Stoica, Ion and Zaharia, Matei.	9
MapReduce: SImplified Data Processing on Large Clusters, by Dean and Ghemawat, First published in OSDI 2004, also in Commun. ACM 51, 1 (January 2008), 107-113.	13
Nobody ever got fired for using Hadoop on a cluster, Rowstron, Antony and Narayanan, Dushyanth and Donnelly, Austin and O'Shea, Greg and Douglas, Andrew	5
Beyond Hadoop, Gregory Mone, CACM, 2013. (short paper).	2

Project-Related

Paper	Pages
Processing Wikipedia Dumps: A Case-Study comparing the XGrid and MapReduce Approaches, D. Thiebaut, Yang Li, Diana Jaunzeikare, Alexandra Cheng, Ellysha Raelen Recto, Gillian Riggs, Xia Ting Zhao, Tonje Stolpestad, and Cam Le T Nguyen, in proceedings of 1st Int'l Conf. On Cloud Computing and Services Science (CLOSER 2011), Noordwijkerhout, NL, May 2011. (longer version)	8