CSC352 Homework 3 2013

--D. Thiebaut (talk) 14:38, 9 October 2013 (EDT)

---

This assignment is due on 10/22/13 at 11:59 p.m. You can work in pairs on this if you desire, but only if you follow the pair-programming model, where two programmers work together at one computer.
When you have finished this assignment, we will have learnt enough MPI for you to move on to the next step of making this a parallel application in MPI!

The Idea

The idea is to start programming for our collage app. At some point, it is very likely that we will want to have some Linux machines run some C/MPI programs in the background to scale images on the fly, or perform some time consuming task, and there will be a Processing/Java program that will be in charge of interacting with the user(s) and display the collage.

So we need to figure out how to interface Java and C programs. This is the purpose of this assignment.

Interfacing C and Java

One way to exchange data between the C program (we'll call it the producer) and the Java program (the consumer) is to use a Linux fifo. Please read the information about mkfifo in Wikipedia. Mkfifo allows your programs to be written as if they were exchanging data by having the producer write the data to a file, and the consumer read the data from the same file. Mkfifo makes the file temporary (it does remain in the directory at the end, though. You have to erase it yourself). That simple!

Good Tutorial on the Subject

The tutorial "How to connect a C/C++ process to a Java application" (link) is very good. Please give it a quick reading. It will give you an idea of what we have to do in C and Java for the two to talk to each other. Below I created a simpler version of the Producer and Consumer for you to play with.

A Producer in C: producer1.c

/*
producer1.c
D. Thiebaut
This program generates N random numbers in an array,
sorts them, and sends them out to a java application
through a Linux Fifo.  See the Linux command mkfifo
for more information.

The java application is called consumer1.java.
To compile and run both:

   gcc -o producer1 producer1.c
   javac Consumer1.java
 
   mkfifo temp
 
   java Consumer1 &           # run in the background
   ./producer1

This example is inspired by the good tutorial at this URL: 
http://research.engineering.wustl.edu/~beardj/CtoJava.html
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <inttypes.h>
#include <stdint.h>

#define N 100

// -----------------------------------------------------
// comp(): a comparator function to compare to different
// integers
int comp (const void * elem1, const void * elem2) {
  int f = *( (int*) elem1 ); // typecast elem1 to become a pointer to an int,
  int s = *( (int*) elem2 ); // then dereference elem1 to get the int it's pointing to
  if (f > s) return  1;
  if (f < s) return -1;
  return 0;
}

// ===================== M A I N =======================
int main(int argc, const char** argv){
  int i, array[N];
  const char* outFile = "temp";

  //--- seed random number generator ---
  srand ( 123 );
  for ( i=0; i<N; i++ )
    array[i]  = rand() % 1000; // numbers in [0,1000[

  //--- sort the N ints in the array "array",  ---
  //--- using comparator comp()                ---
  qsort( array, N, sizeof( int ), comp );

  //--- open fifo for writing ---
  FILE *fp_out = fopen(outFile,"w");

  //--- output dimension N, then all N ints ---
  fprintf( fp_out, "%d\n", N );
  for ( i=0; i<N; i++ )
    fprintf( fp_out, "%d\n", array[i] );

   //--- close output fifo ---
   fclose(fp_out);
   return( 0 ); // success
}

The Java Consumer: Consumer1.java

/*
  Consumer1.java
  D. Thiebaut
  the consumer part of the producer-consumer example made of 
  producer1.c and consumer1.java

  To compile and run:
   gcc -o producer1 producer1.c
   javac Consumer1.java
 
   mkfifo temp
 
   java Consumer1 &           # run in the background
   ./producer1

  Example inspired by the good tutorial at 
  http://research.engineering.wustl.edu/~beardj/CtoJava.html
 */
import java.io.BufferedReader;
import java.io.IOException;
import java.io.FileReader;
import java.util.ArrayList;

public class Consumer1{    
    //--- the external FIFO we'll use.  It's a file ---
    //--- that Linux will manage as a FIFO for us   ---
    private static final String FIFO = "temp"; 
    
    public static void main(String[] args){
	BufferedReader in = null;
	ArrayList<Integer> array = new ArrayList<Integer>();
	int N = 0, count = 0;
	boolean firstLine = true;

	try{
	    in = new BufferedReader(new FileReader(FIFO));
	    while ( in.ready() ) {
		String line = in.readLine().trim();
		int x = Integer.parseInt( line );
		if ( firstLine ) {
		    N = x;
		    firstLine = false;
		}
		else {
		    array.add( x );
		    count++;
		}
	    }
	    in.close();
	} catch( IOException ex ){
	    System.err.println( "IO Exception at buffered read!!" );
	    System.exit( -1 );
	}
	
	//--- we're done.  Display what we received
	System.out.println( "Expected " + N + " ints, received " + count + " ints");
	for ( int x: array )
	    System.out.print( x + " " );
	System.out.println( "\n\n" );
   }
}

Compile and Run

Julia discovered that the steps below work on Beowulf and not Beowulf2. Nice catch!
--D. Thiebaut (talk) 16:10, 20 October 2013 (EDT)

• This should work on your MacBook, as mkfifo should be part of OS X, but if you run into trouble, run this on Beowulf.
• Create the C and Java programs listed above.
• Compile both:

      gcc -o producer1  producer1.c
      javac Consumer1.java

• Create a file named temp (that's the name used by both the C and Java programs when accessing the external file) that Linux will use as a fifo:

      mkfifo temp

• Run both programs, starting with the consumer, since it is waiting on the producer to actually process the data, and then with the producer. Adding the & symbol at the end of the line starting the java program runs it in the background. It is important to do so, otherwise you would be stuck waiting for Consumer1 to be done before being able to type a new command, but Consumer1 cannot be done until you have started producer1...

       java Consumer1  &
       ./producer1

       Expected 100 ints, received 100 ints
       8 9 16 46 60 67 79 99 100 104 107 109 114 131 135 156 166 167 187 190 225 236 243 254 
       261 272 272 275 275 293 293 294 305 317 321 330 330 335 340 344 351 371 373 400 403 
       427 434 439 449 455 459 467 469 511 515 519 522 523 531 534 539 540 553 563 578 592 
       598 606 620 623 634 667 677 685 696 702 711 715 718 719 720 725 761 766 798 804 804 
       810 814 828 857 895 911 912 930 936 938 979 989 992

Running Linux Commands from C programs

This section assumes that the computer you are using has ImageMagick installed on it. If it is not on your computer, you may want to install it, or do your work on Beowulf.

ImageMagick is a collection of many utilities. The two most well-known are identify and convert. The first one gives you information about the image you give it, as in this example:

[xgridmac] ~/classes/352/2013/java+C$: ls -l
-rw-r--r--  1 thiebaut  staff  14080 Dec 10  2010 smith.gif

[xgridmac] ~/classes/352/2013/java+C$: identify smith.gif
smith.gif GIF 277x120 277x120+0+0 8-bit sRGB 32c 14.1KB 0.000u 0:00.000

The second one is convert which you can use to resize images, as in:

[xgridmac] ~/classes/352/2013/java+C$: convert smith.gif -resize 50% smith.jpg
[xgridmac] ~/classes/352/2013/java+C$: ls -ltr
-rw-r--r--  1 thiebaut  staff  14080 Dec 10  2010 smith.gif
-rw-r--r--  1 thiebaut  staff   4033 Oct  9 15:16 smith.jpg

Notice that we have a new file, smith.jpg in the directory, and that its size is much smaller than the original smith.gif. If we identify the new file we get:

[xgridmac] ~/classes/352/2013/java+C$: identify smith.jpg
smith.jpg JPEG 139x60 139x60+0+0 8-bit sRGB 4.03KB 0.000u 0:00.019

The new file is 139x60 while the original was 277x120. So we have reduced the width and height by 50%, indeed!

A C Program that runs identify

The inspiration for the code below is taken from this page.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAXDIM 1000

int main() {
  FILE *pf;
  char command[100];
  char data[MAXDIM];
  char fileName[] = "smith.gif"; // name of the file to identify

  // Execute a process listing
  strcpy( command, "identify " ); // imageMagic command to get file info
  strcat( command, fileName );     // becomes "identify smith.gif"
 
  // Run the identify command and setup a pipe for reading 
  // the information returned by this command.
  pf = popen( command, "r" ); 
 
  if ( !pf ) {
    fprintf(stderr, "Could not open pipe for output.\n");
    return 1;
  }
 
  // Grab data from process execution. fgets gets a string from a file
  // or from a pipe
  fgets(data, MAXDIM , pf);
 
  // Print grabbed data to the screen.
  printf( "Identify returns: %s\n\n",data); 
 
  if ( pclose(pf) != 0 )
    fprintf(stderr," Error: Failed to close command stream \n");
 
  return 0;
}

You should be able to figure out what is happening in this program. Here the idea of a pipe is slightly different from what we used with the producer/consumer of the first part of this homework assignment. Here a pipe is just a way to capture the output or input of a program in a temporary file. The C function popen() is sued to execute a Linux command, and get the output of this command as a pipe (fp). This way the C program cat get what the command would have normally outputted to the screen. Makes sense?

Implement this program and play with it.

Your Assignment

Your assignment is to write a pair of producer/consumer programs, one in C and the other one in Java that talk to each other through fifos: one fifo for the producer to talk to the consumer, and one for the consumer to respond to the producer.

1. The Java program will send the names of three image files to the C program through a fifo.
2. (You need to have 3 images in the same directory where your programs will reside.)
3. The C program will receive each file name, and for each one and it will call convert to resize it by 50%.
4. The C program then runs identify on the new file.
5. It then sends back to the java program the information produced by identify.
6. The java program will then print this information back on the screen.
7. Use a bash file to make everything run! Call the bash file doHomework3.sh

Optional and Extra Credits (1/3 Point, on top of 4 points total)

Make the java program talk to 3 different C programs, and give each one one photo to resize.

Submission

• Call the java program Master1.java
• Call the C program worker1.c
• Pick 3 image files of your choosing
• Submit all files on beowulf, as follows:

  submit doHomework3.sh
  submit hw3 Master1.java
  submit hw3 worker1.c
  submit hw3 yourImage1.jpg
  submit hw3 yourImage2.jpg
  submit hw3 yourImage3.jpg

Note that the names of your images do not have to be standardized. Your java program will have these names hard-coded in it. They can be jpg, png, or gif, depending on what image you want to work with.