SimGrid
3.9.90
Versatile Simulation of Distributed Systems
|
Check online for our specific Simgrid-Ruby documentation.
Simgrid-java is a java API that let you use Simgrid MSG API in your favorite language (java). Without it, you would be forced to use C or one of the other bindings provided.
MSG was the first distributed programming environment provided within SimGrid. While almost realistic, it remains quite simple. This describes the Java bindings to this interface.
The javadoc is accessible here
You should use MSG if you want to study some heuristics for a given problem you don't really want to implement. SimGrid-java let you use MSG while coding in Java. So if your need is MSG + Java, you're in the right section!
To make a long story short, it's a JNI binding, so it implies that:
Finally, it implies also that your program can crash for 3 main reasons:
C crashes mainly for 2 reasons:
To use java with Simgrid you have to install some dependencies:
openjdk7
or sun-java6-jdk
(with libgcj10-dev
or another version of gcj). For maximal performance and scalability, use a coroutine-enabled JVM (see How to use the coroutines context factory).Then build Simgrid with the Java bindings enabled:
If cmake complains that jni could not be found, you need to tell it where JNI header files are located. the following command should tell you:
If you have several version of jni installed (as in the example above), you need to check the version of java that is used by default on your machine (using javac -version), and pick the right one. Then set the JAVA_INCLUDE_PATH
environment variable to the right path (note that we remove the filename jni.h
from that path), and relaunch cmake.
To execute the examples you need to add the path where you installed the generated libSG_java
and libsimgrid
libraries into the LD_LIBRARY_PATH
.
Be careful on Mac, this variable is called DYLD_LIBRARY_PATH
and not LD_LIBRARY_PATH
.
If you want to make these settings permanent even after a reboot, you need to add the export lines into your ~/.bashrc
file, or equivalent.
There is two main motivations to use the coroutine variant of SimGrid Java bindings: it's about 5 times faster than the default thread-based context factory, and the amount of runnable processes is then only limited by the amount of RAM that you have. The drawbacks are that it requires a specific and rather experimental JVM to run, and that this context factory itself remains a bit experimental so far.
You need to get a patched JVM from here (many thanks to Lukas Stadler for this work!).
You can either get a prebuilt binary, or recompile your own JVM. Make sure to get a coro-simple version, as we don't need to serialize nor migrate stacks in SimGrid. You should be able to follow the README.txt
that you'll get in the repository, but here is how we did it, just in case. The instructions are given for a debian or Ubuntu box, but I think you should manage to convert it to your system quite easily. Finally, if you're really stuck, you can get the version compiled by Jonathan Rouzaud-Cornabas from his web page. This version is known to work with SimGrid for sure! http://graal.ens-lyon.fr/~jrouzaud/files/corosimple-linux-amd64-20120914.tgz
Configure the mercurial extensions: Edit ~/.hgrc and paste the following lines. Don't forget to change the /path/to/forest.py to point to where you just downloaded the source.
Forest extension is needed to download the openjdk source code and patches while the mq line is needed to apply the patches. The username is needed at the step "preparing the sources", not sure why.
SimGrid Java will automatically switch to the coroutine context factory if your JVM support it, so you will just need to execute your simulation with the correct JVM. The selected context factory gets displayed automatically.
Note that you may have to adjust the "coro.stacksPerThread" configuration option to run large simulations. The default is 100 and you want to increase it to run more processes.
If you reach the point where the creation of new simulated processes fail with the message "Can't create coroutine object", you may need to increase the relevant system limit with the following command.
The full story is that each coroutine requires two memory maps, and that Linux puts a limit on the total amount of memory maps that each process can manage (by default, this limit is often at 65535). Since the JVM needs a few dozen of such maps on its own (three maps per dynamic library – check /proc/the_pid/maps
if you don't believe it), this is enough to create over 30,000 simulated processes. But to go futher, that limit must be modified.
If you want to make this change permanent on your machine, edit your /etc/sysctl.conf
file. Otherwise, you have to redo it by calling sysctl after each reboot.
Most of Simgrid modules require a good level in C programming, since simgrid is used to be as standard C library. Sometime users prefer using some kind of “easy scripts” or a language easier to code with, for their works, which avoid dealing with C errors, and sometime an important gain of time. Besides Java Binding, Lua and Ruby bindings are available since version 3.4 of Simgrid for MSG Module, and we are currenlty working on bindings for other modules.
Lua is a lightweight, reflective, imperative and functional programming language, designed as a scripting language with extensible semantics as a primary goal (see official web site here).
Lua is a fast, portable and powerful script language, quite simple to use for developpers. it combines procedural features with powerful data description facilities, by using a simple, yet powerful, mechanism of tables. Lua has a relatively simple C API compared to other scripting languages, and accordingly it provides a robust, easy to use it.
Actually, the use of lua in Simgrid is quite simple, you have just to follow the same steps as coding with C in Simgird :
You can also exchange data between Process using lua. for that, you have to deal with lua task as a table, since lua is based itself on a mechanism of tables, so you can exchange any kind of data (tables, matrix, strings,…) between process via tasks.
task = simgrid.Task.new("data_task",task_comp,task_comm); task['matrix'] = my_matrix; task['table'] = my_table; task['message'] = "Hello from (Lua || Simgrid ) !! " … simgrid.Task.send(task,alias)After creating task, we associate to it various kind of data with a specific key (string in this case) to distinguish between data variables. The receiver will use this key to access easily to datas.
task = simgrid.Task.recv(alias); sender_matrix = task['matrix']; sender_table = task['table']; sender_message = task['message'] ...Note that in lua, both sender and receiver share the same lua task. So that the receiver could joint data directly on the received task without sending it back. You can find a complet example (matrix multiplication case) in the file example/lua/mult_matrix.lua.
maybe you wonder if there is a way to bypass the XML files, and describe your platform directly from the code, with lua bindings it's Possible !! how ? We provide some additional (tricky?) functions in lua that allows you to set up your own platform without using the XML files ( this can be useful for large platforms, so a simple for loop will avoid you to deal with an annoying XML File ;) )
simgrid.AS.new{id="AS0",mode="Full"};
simgrid.Host.new{id="Tremblay",power=98095000}; simgrid.Host.new{id="Jupiter",power=76296000}; simgrid.Host.new{id="Fafard",power=76296000}; simgrid.Host.new{id="Ginette",power=48492000}; simgrid.Host.new{id="Bourassa",power=48492000};we use simgrid.Host.new{id=id_host,power=power_host} to instanciate our hosts.
for i=0,11 do simgrid.Link.new{id=i,bandwidth=252750+ i*768,latency=0.000270544+i*0.087}; -- some crazy values ;) endwe used simgrid.Link.new{id=link_id,bandwidth=bw,latency=lat} with a simple for loop to create all links we need (much easier than XML hein ?)
-- simgrid.Route.new(src_id,des_id,links_nb,links_list) simgrid.Route.new("Tremblay","Jupiter",1,{"1"}); simgrid.Route.new("Tremblay","Fafard",6,{"0","1","2","3","4","8"}); simgrid.Route.new("Tremblay","Ginette",3,{"3","4","5"}); simgrid.Route.new("Tremblay","Bourassa",7,{"0","1","3","2","4","6","7"}); simgrid.Route.new("Jupiter","Tremblay",1,{"1"}); simgrid.Route.new("Jupiter","Fafard",7,{"0","1","2","3","4","8","9"}); simgrid.Route.new("Jupiter","Ginette",4,{"3","4","5","9"}); simgrid.Route.new("Jupiter","Bourassa",8,{"0","1","2","3","4","6","7","9"}); ...for each host you have to specify which route to choose to access to the rest of hosts connected in the grid.
simgrid.register_platform();Don't forget to register your platform, that SURF callbacks starts their work ;)
simgrid.Host.setFunction("Tremblay","Master",4,{"20","550000000","1000000","4"}); simgrid.Host.setFunction("Bourassa","Slave",1,{"0"}); simgrid.Host.setFunction("Jupiter","Slave",1,{"1"}); simgrid.Host.setFunction("Fafard","Slave",1,{"2"}); simgrid.Host.setFunction("Ginette","Slave",1,{"3"});you don't need to use a deployment XML file, thanks to simgrid.Host.setFunction(host_id,function,args_number,args_list) you can associate functions for each host with arguments if needed .
simgrid.register_application();Yes, Here too you have to register your application before running the simulation.
the full example is distributed in the file examples/lua/master_slave_bypass.lua
Simulation of a master-slave application using lua bindings
as described in the C native master/Slave example, this function has to be assigned to a msg_process_t that will behave as the master.
Lua style arguments (...) in for the master are interpreted as:
Tasks are dumbly sent in a round-robin style.
This function has to be assigned to a msg_process_t that has to behave as a slave. This function keeps waiting for tasks and executes them as it receives them.
in this section the core of the simulation which start by including the simgrid lib for bindings : require "simgrid"
Its arguments are:
Simulation of a master-slave application using lua bindings, Bypassing the XML parser
as described in the C native master/Slave example, this function has to be assigned to a msg_process_t that will behave as the master.
Lua style arguments (...) in for the master are interpreted as:
Tasks are dumbly sent in a round-robin style.
This function has to be assigned to a msg_process_t that has to behave as a slave. This function keeps waiting for tasks and executes them as it receives them.
in this section the core of the simulation which start by including the simgrid lib for bindings, then create the resources we need to set up our environment bypassing the XML parser. : require "simgrid"
we can also bypass the XML deployment file, and associate functions for each of defined hosts.