ptolemy.actor.ptalon.lib
Class MapReduceAlgorithm

java.lang.Object
  java.lang.Thread
      ptolemy.actor.ptalon.lib.MapReduceAlgorithm

All Implemented Interfaces:

java.lang.Runnable

Direct Known Subclasses:

ReverseLink, WordCount

public abstract class MapReduceAlgorithm
extends java.lang.Thread

The MapReduce Algorithm.

See MapReduce: Simplified Data Processing on Large Clusters

Since:

Ptolemy II 6.1

Version:

$Id: MapReduceAlgorithm.java 53095 2009-04-12 19:12:45Z cxh $

Author:

Adam Cataldo

Accepted Rating:

Red (cxh)

Proposed Rating:

Red (cxh)

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler

Field Summary

private boolean	_noMoreInputs This is set externally to true when the reduceValues queue no longer needs to accept values.
private boolean	_threadDone True when the run method is done being called.
private boolean	_threadError True if an exception was thrown accessing the blocking queue.
java.lang.String	reduceKey The key for the reduce algorithm, which should be set externally.
java.util.List<java.lang.String>	reduceOutput The list generated by the reduce algorithm, which should be read externally.
java.util.concurrent.BlockingQueue<java.lang.String>	reduceValues The values for the reduce algorithm, which should be set externally.

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

MapReduceAlgorithm()

Method Summary

protected boolean	isQueueEmpty() Return true if no more elements will be added to the list.
boolean	isReduceFinished() Return true if the reduce is finished.
abstract java.util.List<KeyValuePair>	map(java.lang.String key, java.lang.String value) Subclasses should implement their map method here.
abstract java.util.List<java.lang.String>	reduce(java.lang.String key, java.util.concurrent.BlockingQueue<java.lang.String> values) Subclasses should implement their reduce method here, calling the take method of the BlockingQueue to get the next value, and checking the parameter noMoreInputs to test if no more values can be put on the queue.
void	run() This is used to call the reduce algorithm.
void	setNoMoreInputs() Set this true when no more inputs values will be given to the reduce method queue.

Methods inherited from class java.lang.Thread

activeCount, checkAccess, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

reduceKey

public java.lang.String reduceKey

The key for the reduce algorithm, which should be set externally.

reduceValues

public java.util.concurrent.BlockingQueue<java.lang.String> reduceValues

The values for the reduce algorithm, which should be set externally.

reduceOutput

public java.util.List<java.lang.String> reduceOutput

The list generated by the reduce algorithm, which should be read externally.

_threadDone

private boolean _threadDone

True when the run method is done being called.

_threadError

private boolean _threadError

True if an exception was thrown accessing the blocking queue.

_noMoreInputs

private boolean _noMoreInputs

This is set externally to true when the reduceValues queue no longer needs to accept values.

Constructor Detail

MapReduceAlgorithm

public MapReduceAlgorithm()

Method Detail

isReduceFinished

public boolean isReduceFinished()
                         throws IllegalActionException

Return true if the reduce is finished.

Returns:

Return true if the reduce is finished.

Throws:

IllegalActionException - If there was a thread error.

map

public abstract java.util.List<KeyValuePair> map(java.lang.String key,
                                                 java.lang.String value)

Subclasses should implement their map method here.

Parameters:

key - The key passed to the map method.

value - The value passed to the map method.

Returns:

The list of key value pairs for the given input.

reduce

public abstract java.util.List<java.lang.String> reduce(java.lang.String key,
                                                        java.util.concurrent.BlockingQueue<java.lang.String> values)
                                                 throws java.lang.InterruptedException

Subclasses should implement their reduce method here, calling the take method of the BlockingQueue to get the next value, and checking the parameter noMoreInputs to test if no more values can be put on the queue. The last value put on the queue may be the empty string. If this is the case, just discard it.

Parameters:

key - The key to reduce over.

values - The queue of values in reduction.

Returns:

The reduced list of valeus.

Throws:

java.lang.InterruptedException - If thrown while reducing.

run

public void run()

This is used to call the reduce algorithm.

Specified by:

run in interface java.lang.Runnable

Overrides:

run in class java.lang.Thread

setNoMoreInputs

public void setNoMoreInputs()

Set this true when no more inputs values will be given to the reduce method queue.

isQueueEmpty

protected boolean isQueueEmpty()

Return true if no more elements will be added to the list.

Returns:

true if no more elements will be added to the list.