This version is in HTML, a PDF version can be found in Chapter 8 of Volume 2: Software Architecture
Authors: Steve Neuendorffer
Christopher Brooks
Ankush Varma
Shuvra S. Bhattacharyya
The basic infrastructure is implemented in the copernicus.kernel package. The Copernicus class contains a main function suitable for invocation from the command line. The GeneratorAttribute class represents code generation parameters that can be persistently added to a model for unusual configurations of a code generator. The KernelMain class is a base class from which classes for various code generators can be derived. Instances of these subclasses are instantiated and executed in a code generation job.
The Copernicus class itself is invoked to begin code generation from a model. If invoked from the command line, it reads command line arguments to determine various code generation options and an MoML file to load a model from. If invoked via various static methods, code generation options are assumed to be passed in through a GeneratorAttribute. Default code generation options are specified in the Generator.xml file. One of the code generation parameters determines the code generator to execute. The class representing the code generator is loaded through reflection and invoked through the KernelMain base class.
The KernelMain base class provides default behavior for most code generators. It performs static analysis, such as type resolution and scheduling by invoking the Manager.preinitalizeAndResolveTypes() method and then passes control to the specific code generator.
codeGenerator: The code generator to run.
codeGeneratorClassName: The class that is instantiated to execute a particular code generator. This class is expected to be a subclass of ptolemy.copernicus.kernel.KernelMain.
compile: If true, compile the generated code. The default is true.
show: If true, then show the generated code. The default is true.
run: If true, then run the generated code. The default is true.
ptII: The location of the Ptolemy II classes. The default is the value of the ptolemy.ptII.dir Java system property
ptIIUserDirectory: The top level directory to write the code in. The default is the value of the ptII parameter. The code will appear in 'ptIIUserDirectory/targetpath'.
targetPackage: The package to generate code in. The default is the model name
targetPath: The path relative to the ptIIUserDirectory to generate code in. The default is the "cg" subdirectory of the particular code generator.
outputDirectory: The directory that code will be generated in. By default this is the targetPath parameter appended to the ptIIUserDirectory path.
modelPath: The path to the model, including the .xml extension. The modelPath parameter is converted to a URL internally before use.
compileOptions: User supplied arguments to be passed to the code generator. Defaults to the empty string.
javaClassPath: The Java class path, converted to a string.
runCommandTemplateFile: The template file that contains the command to run the generated code.
runOptions: User supplied arguments to be passed to the command that will run the generated code. Defaults to the empty string.
sootDir: The directory that contains the soot jar files. Defaults to the value of the ptII parameter + "/lib"
sootClasses: The location of sootclasses.jar, jasminclasses.jar and the Java system jar (usually rt.jar). The necessaryClassPath parameter may end up duplicating some of the elements of this parameter.
watchDogTimeout: The number of milliseconds that code generation will run for. Defaults to 720000, which is 12 minutes. The watchdog is used to prevent the code generator and the generated code from hanging the nightly build.
output: The filename to redirect the standard output stream of the code generator to. This is used, for example, in the nightly build to provide easily parseable error messages. If the value is not set, then the output will not be redirected.
The Java code generator operates in several phases, and the output of each phase is a partially specialized model. The output from the intermediate phases can be generated by setting the snapshots parameter to be true. The first snapshot consists of self-contained code specialized to the domains in the model. The second snapshot is additionally specialized to the parameter values in the model, while the third is specialized to the structure of the model. The fourth snapshot eliminates all references to Ptolemy named objects in the model, resulting in self-contained code without component interfaces. The final generated code has also been specialized for data types and contains no references to Ptolemy tokens.
One of the goals of the Java code generator was to avoid separate specifications for simulation and code generation wherever possible. The Java code generator operates by transforming Java actor specifications (actor classes) and on Java data type specifications (token classes). In most cases, new actor and token classes will be leveraged transparently by the code generator. Unfortunately, domain specifications are not as easily reused and the Java code generator contains "re-implementations" of domains for code generation. This allows for more efficient code to be generated, at the expense of duplicating aspects of existing Director and Receiver code, and making it more difficult for new domains to be implemented in code generation.
In order for existing actor code to be leveraged by the code generator, it assumes that the code is written according to the Ptolemy style for writing actors. This style assumes naming conventions for the public fields of an actor class that refer to parameters and ports of the actor. The code generator also assumes that the ports and parameters of an actor are created in the class constructor and not modified later. Some actors do not fit these constraints and cannot be used directly in the code generator. Such actor classes cannot be used directly by the code generator, although in some cases we have been able to have the code generator deal specially with such actors. In other cases, the actor class fits the constraints but cannot be effectively specialized using generic techniques. Such actors can also be dealt with specially by the code generator to more effectively generate code.
The Java code generator consists of a large number of individual transformation steps, which will not be described here. These transformation steps are implemented by classes extending the SceneTransformer class, or the BodyTransformer class. Two key points of extensibility are provided for generating domain code and for generating actor code to replace unspecializable actor classes.
Code generation for specific domains is handled by various implementations of the DomainCodeGenerator interface. An implementation of this interface is responsible for generating domain interaction code for a particular composite actor in the hierarchy, including code to invoke the methods of various actors and domain-specific communication structures. Currently, the following domains are handled:
Synchronous Dataflow (SDF): The SDF implementation transforms the SDF schedule into Java code that invokes the actors in a model. Fixed size arrays are generated for communication buffers dedicated to each relation in the model, and communication methods are replaced with circularly indexed addressing into the communication buffers.
Hybrid Systems (HS): The hybrid systems director deals with modal models. Currently, only the subset that is useful in modal SDF models is implemented.
Giotto: The Giotto implementation interfaces directly with the Java output of the Giotto compiler. It generates classes with static methods used for communication by the Giotto compiler and generates a .giotto file that describes the classes implementing the various Giotto tasks. The Giotto compiler compiles this file into a class that implements the Giotto task scheduling model.
Code Generation for actors is handled various implementations of the AtomicActorCreator interface. An implementation of this interface generates a self-contained class for a particular actor. The default implementation of this interface, the GenericAtomicActorCreator class, simply copies the existing actor specification code. Other implementations of the interface deal with generating code for specific actors. Currently the following actors are handled specially:
Expression: The standard implementation of this actor builds a parse tree for the expression and traverses the parse tree to evaluate it at run time. This actor is handled specifically by the ExpressionCreator class for two reasons. Primarily, the parse tree is convenient way of representing an arbitrary expression, but much simpler code can be generated for a specific expression. Secondarily, the parse tree complicates other transformations that specialize communication between actors and data types.
FSMActor: The standard implementation of this actor builds parse trees for every expression in a model, and suffers from the same drawbacks as the Expression actor. The FSMActor also attempts to deal with run-time modifications of the finite-state machine in an efficient manner, which is not necessary in generated code. This actor is handled specifically by the FSMCreator class.
Many actors are not handled specifically, but should be. Here is a short list:
MathFunction: This actor creates and deletes its ports based on a parameter value. In generated code will likely not happen (since the parameter value is not likely to change), but the GenericAtomicActorCreator is not smart enough to deal with ports that are not created in the constructor. It is likely easiest to handle this actor by handling it specially and checking that the parameter value does not change using reconfiguration analysis.
TypeTest: This actor tests the type system, but has no run-time behavior. It is problematic because it iterates over all of the actors in a model, which is currently not supported by the code generation mechanism. It could probably be checked statically and ignored in generated code.
RecordAssembler and RecordDisassembler: These actors iterator over their input and output ports to construct a record. They could either be dealt with specially, or the code generator could be improved to unroll iterators over ports.
ExpressionToToken and ExpressionReader: These actors operate in a similar way to the expression actor, except that the expression is received from an input port. Because of this, code generation will not work. It is not clear how to make this actor work nicely with type specialization.
The code generated from the Java code generator is a set of self-contained Java .class files with a command-line interface. A makefile is automatically generated with a large number of rules for manipulating the generated code. The makefile rules are:
runJava: Run the generated code.
compareAll: Run a series of comparisons between the simulation model, the generated code, and the obfuscated version of the generated code comparing code size, execution speed, and memory usage.
treeShake: Generate a self-contained .jar file containing only code necessary for the generated code. This rule uses reachable method information gained through static analysis in the code generator, if possible.
treeShakeByRunning: Generate a self-contained .jar file containing only code necessary for the generated code. This rule executes the generated code and extracts information from the virtual machine about which classes were loaded at runtime.
runTreeShake: Run the generated code from the self-contained .jar file.
profileTreeShake: Run the generated code from the self-contained .jar file with profiling options to report runtime memory usage. An average of several runs is reported.
treeShakeWithoutCodegen: Generate a self-contained .jar file containing only code necessary for executing the simulation model. This rule executes the generated code and extracts information from the virtual machine about which classes were loaded at runtime.
runTreeShakeWithoutCodegen: Run the original simulation model from the self-contained .jar file.
profileTreeShakeWithoutCodegen: Run the original simulation model from the self-contained .jar file with profiling options to report runtime memory usage. An average of several runs is reported.
obfuscate: Run the Jode obfuscator on the generated code, to minimize the size of the generated .jar file.
runObfuscate: Run the obfuscated version of the generated code.
profileObfuscate: Run the obfuscated version of the generated code from the self-contained .jar file with profiling options to report runtime memory usage. An average of several runs is reported.
gcj: Compile the generated code into a native executable using gcj. Note: this will likely only work for simple models, as the gcj standard Java libraries are far from complete.
.We use the copernicus shell script, located at $PTII/bin/copernicus to run the Java code generator:
$PTII/bin/copernicus -codeGenerator java $PTII/ptolemy/domains/sdf/demo/OrthogonalCom/OrthogonalCom.xml
The above command will generate voluminous output and eventually create .class files in $PTII/ptolemy/copernicus/java/cg/OrthogonalCom and run the generated code.
The copernicus script also generates a makefile in the output directory that contain rules to perform operations like treeshaking and obfuscation. Treeshaking is an optimization where we run the model, note what .class files are loaded and then place those .class files in a jar file. Treeshaking is not perfect, since if the model is running from the jar file and later throws an exception the error handlers and other .class files might not be present. Obfuscation is an optimization that shortens class and method names so as to decrease the size of the jar file.
We use Jode to obfuscate the code. Jode is available from http://jode.sourceforge.net/. Unfortunately, Jode is distributed under the GNU General Public License (GPL), so we do not include it in the Ptolemy release. To set up Jode:
cd $PTII ./configure
Then go back to the output directory and run make:
$PTII/ptolemy/copernicus/java/cg/OrthogonalCom make compareAll
The truly observant will have noticed that the various versions of the model ran very quickly and that it is difficult to compare the time performance of the different versions. The solution is to increase the number of iterations so that we can see differences in time performance. $PTII/ptolemy/copernicus/kernel/KernelMain.java describes how to set the iterations:
* If the director is an SDF director, then the number of * iterations is handled specially. If the director is an SDF * director and a parameter called "copernicus_iterations" is * present, then the value of that parameter is used as the * number of iterations. If the director is an SDF director, and * there is no "copernicus_iterations" parameter but the * "ptolemy.ptII.copernicusIterations" Java property is set, then * the value of that property is used as the number of * iterations.
So, we can either edit the model and add a copernicus_iterations parameter, or else we can run copernicus with the ptolemy.ptII.copernicusIterations property set. In this example, we set the property and rerun copernicus
export USERJAVAPROPERTIES=-Dptolemy.ptII.copernicusIterations=100 $PTII/bin/copernicus -codeGenerator java $PTII/ptolemy/domains/sdf/demo/OrthogonalCom/OrthogonalCom.xml
Then, we cd to the generated directory and re run the comparison:
cd $PTII/ptolemy/copernicus/java/cg/OrthogonalCom make compareAll
The results is that the different versions of the models run for 1000 iterations and we can compare the times:
Note that these results are not particularly rigorous, see [112] for a more formal analysis.
Within the Ptolemy II framework, the C code generator takes the class files generated by copernicus.java as input.The C code generator can also be used as a stand-alone Java-to-C compiler to generate C code for arbitrary Java programs.
The Soot framework is used to create a Call Graph of the application. This is a graph with methods as the nodes, and calls from one method to another as directed edges.
At first glance, it seems that the transitive closure of the methods in the main class should represent all methods that can be called. However, this is not so, because the first time the field or method of a class is referenced, its class initialization method is also invoked, and this can reference other methods or fields in turn.
The method call graph also contains an edge from a method to every possible target of method calls in it. The number of such targets can be large for polymorphic method calls. A more sophisticated analysis can trim the method call graph by removing some of the edges corresponding to polymorphic invocations.
We use Soot's Variable Type Analysis (VTA) to perform this call graph trimming. This analysis computes the possible runtime types of each variable using a reaching type analysis, and uses this information to remove spurious edges.
From the analysis mentioned above, the set of all possible required classes, methods and fields (collectively grouped as entities) can be statically computed. We use a set of rules to determine which classes are required.
Interfaces are treated as classes. A worklist-based algorithm can be used to add to the set of required entities until no additional entities can be found by application of these rules. Together, rules 2, 3 and 4 encapsulate all possible dependencies between entities. This makes the set of required entities self-contained.
The further limitations of the current implementation are:
The dashed arrows in the UML diagram represent the coarse-grained code flow in copernicus.c. The entry class is JavaToC when used in stand-alone mode, and Main when copernicus.c is used as a ptolemy code-generation back-end.
JavaToC reads in a Java class file and implicitly converts it to the Soot Jimple format. Then it calls RequiredFileGenerator, MakeFileGenerator and MainFileGenerator.
RequiredFileGenerator uses CallGraphPruner to compute the set of required classes, methods and fields. Then it calls ClassFileGenerator, HeaderFileGenerator and StubFileGenerator on each required class.
ClassFileGenerator creates the .c file containing all the function definitions. Each of these function definitions is created by MethodCodeGenerator. MethodCodeGenerator calls CSwitch on each Jimple statement to find its C equivalent.
HeaderFileGenerator creates the .h file corresponding to the class. This consists of a class-specific C structure for the class (created by ClassStructureGenerator), an instance-specific C structure for the class (created by InstanceStructureGenerator) and various function declarations. MethodListGenerator is the class that "understands" inheritance to create the lists of constructors, inherited methods, new methods, private methods, etc.
StubFileGenerator creates a small "stub" of prototype declarations useful for breaking circular dependencies between classes.
MakeFileGenerator creates a makefile proving rules for compiling the generated C code into an executable.
MainFileGenerator creates a file that contains the C "main" method, which performs initialization functions, wraps the command-line arguments into the C equivalent of a Java string array and passes them to the "java" main method.
In addition, CNames converts Java names into unique legal C names, InterfaceLookupGenerator takes care of resolving interface method invocations, FileHandler provides file I/O utilities, Options stores configuration information, Context handles useful global information, NativeMethodGenerator handles native methods and OverriddenMethodGenerator allows user-defined code to override the compiler.
java -classpath $classpath ptolemy.copernicus.c.JavaToC $classpath <className>
(note that the classpath has to be specified twice)
Move the xml model to c/test/simple
java ptolemy.copernicus.kernel.Copernicus -codeGenerator c <model>.xml
Java requires certain native methods, which are methods implemented in platform-dependent code, typically written in another programming language such as C. The C code generator allows the user to specify C code for the body of any native method. At compile-time, this is integrated with the generated C code, allowing any C native methods to be fully supported. To do this:
It is also possible to override the C code generator and write custom C code for a given method instead. To do this:
Note that the term overridden in this context does not refer to methods that are overridden through inheritance in Java classes.
To "turn off" code generation for a method, override it without creating code for it in runtime/over_bodies. For an entire class or package, list it in OverriddenMethodGenerator.isOverriddenClass(). This will generate methods with blank bodies and trivial return statements which will return 0 or NULL.
CAVEAT: Make sure that the returned values are not used. Referencing a NULL pointer will cause the executable to throw a segmentation fault.
The Applet code generator takes a model and creates HTML files for use as a web based applet.
The applet generator reads template files that end in .in from $PTII/ptolemy/copernicus/applet substitutes keywords and writes out the files in the destination directory. Users may modify the template files to match their local setup
Making an applet available via the web is somewhat complex because the Java Plugin has two sections, one for Netscape, the other for Internet Explorer, so changes to the htm files must be replicated in both sections. The codebase and the location of the jar files also add to the problems.
If a model is named MyModel, and the user selects foo.bar as the package, then saving the model as an applet will create a directory called $PTII/foo/bar/MyModel and create the following files for that model:
make demo will run appletviewer on the HTML files
An HTML file containing the code necessary to MyModel.xml
The OrthogonalCom model generates prints output to standard out, so when this model is run as an applet, the output will appear in the Java Plugin console. Instead, we generate an applet for the Butterfly model, which will generate display a nice plot. Note that the Butterfly model uses the Expression actor so that while we cannot use deep code generation on the Butterfly actor, we can generate an applet for this model.
cd $PTII/ptolemy/domains/sdf/demo/Butterfly $PTII/bin/copernicus -codeGenerator applet Butterfly.xml
The HTML can be found in $PTII/ptolemy/copernicus/applet/cg/Butterfly.
Usually, one wants to put an applet on a website. Ptolemy applets require jar files for the runtime environment, so the applet code generator will copy the necessary jar files if the value of the ptIIUserDirectory parameter is outside the $PTII directory.
cd $PTII make install
mkdir c:/tmp/ptIIapplet/Butterfly
cd $PTII/ptolemy/domains/sdf/demo/Butterfly $PTII/bin/copernicus -codeGenerator applet -ptIIUserDirectory \ c:/tmp/ptIIapplet -targetPath Butterfly Butterfly.xml
Note that the copernicus command should be typed in on one line.
cd $PTII make install
mkdir c:/tmp/ptIIapplet/Butterfly
c:/tmp/ptapplet
Note that the directory must already exist. If it does not exist, then the default directory will automatically be used.
$modelName
Butterfly
Under Web Start, you may need to add classes to the necessaryClasses parameter so that the necessaryClassPath parameter will get updated with the appropriate jar files and passed to the subprocess that invokes the applet code generator. The reason this is necessary is because Web Start is invoked using a special class loader that accesses separate jar files in the Web Start cache. The applet code generator does not have direct access to the Web Start class loader, so we tell it what classes we need so that they can be added to the class path.