This mode is very appropriate if you are running a few LARGE MIPs/LPs, as it avoids serialization overhead. If, on the other hand, you are running many small MIPs/LPs in parallel, it makes more sense to run in client/server mode. The tradeoff point depends on the size of your MIP/LP in terms of serialization cost, and the number of MIPs you run in parallel.
In the zip file, there is an ExampleSimple.java -- look at how it can call SolverClient() with no parameters.PART 1. Setting up the Server for running in Client/Server Mode
(CPLEX 8.0 or higher users)
This mode is very appropriate if
you are running many small MIPs/LPs in parallel as it takes advantage
of working on many parts of your sub-problem at once. If, on the
other hand, you are running a few LARGE MIPs/LPs, it makes more sense
to run in unifed mode (instructions above). The tradeoff point
depends on the size of your MIP/LP in terms of serialization cost,
and the number of MIPs you run in parallel. (If you don't run any
MIPs/LPs in parallel, it always makes sense to run in unified mode.)
Verify that you have the same JDK version running on both the client and the servers. This may not be required, but is probably a good idea. JOpt has been tested and should be compatible with JDK 1.4 and 1.5.
Install Apache Ant 1.6.2 or higher.
If you are doing Java
programming and you don't use Ant, then you are in for a treat. Ant
is the de facto standard buildfile / makefile replacement and will
make your life
easier. See http://ant.apache.org/
You can check that you have a recent version installed in your path by
typing:
% ant
-version
Apache Ant version 1.6.2 compiled on September 28
2004
Install CPLEX.
The JOpt server install is actually just a mkdir jopt and then unzip the jopt zip file into this directory. Jopt comes pre-compiled and is in lib/jopt.jar. (The zip file also contains the logging config file, and example, the JOpt source code, this documentation, and the redistribution license file.)
Above the directory where you installed JOpt, create a symbolic
link to your cplex installation directory. (eg: ln -s
/usr/ilog/cplex81 cplexdir). Unless you change build.xml to teach it
otherwise, JOpt running in server-mode assumes that the directory
structure looks something like:
.\cplexdir
(you made this symbolic link
above)
.\cplexdir\bin\i86_linux2_glibc2.3_gcc3.2 (ships with CPLEX
8.1)
.\jopt (you mkdir'd this above)
.\jopt\jopt.jar (you
unzip'd this above)
You are now done with the server side installation. To test, you should run:
ant runcplex
and you should see:
Buildfile:
build.xml
runcplex:
[java] MAIN SolverServer: Binding server to port: 2000
If you don't want to bind to port 2000, change the line in build.xml:
<property name="port" value="2000"/>
That's it! Now read the section “Setting up the Client” Below
PART 2. Setting up the Client for running in Client/Server Mode (CPLEX 8.0 or higher users)
If you followed the instructions for setting up the server, you are basically there. The zip file comes packaged with two example files in the root directory, ExampleSimple.java and ExampleSimpleWrapper.java. To test them, in bash, you would type exactly:
%
javac -d . -classpath lib/jopt.jar ExampleSimple.java
% java
-classpath .:lib/jopt.jar
edu.harvard.econcs.jopt.example.ExampleSimple
(Note that it is important that your classpath include both the JOpt directory (. in the above example) and the jopt.jar file) and you should see:
Usage: java edu.harvard.econcs.jopt.example.ExampleSimple ServerHostName ServerPortNumber
For instance, if you are running your server on drdoom.eecs.harvard.edu on port 2000, you would type:
% java -cp .:lib/jopt.jar edu.harvard.econcs.jopt.example.ExampleSimple drdoom 2000
and
you should see:
Vars:
y
x
Constraints:
Constraint
{[1.0*y] == 2.0}
Constraint {[1.0*x, -2.0*y] <= 5.0}
Objective
Function: Max[ 2.0*x + 2.0*y ]
Constraint {[1.0*y] ==
2.0}
objective: 22.0
x is 9.0
y is 2.0
The file edu/harvard/econcs/util/log.config controls the level of logging (screen output) that you will see while running JOpt, in both server-mode and client-mode. The default settings provide very little screen feedback. Increasing the JOpt debugging output, while helpful for debugging, will cause your program to run slowly.
If you are using CPLEX, note that CPLEX does it's own logging in addition to JOpt's screen output. This can be controlled using Jopt's MIP.setSolveParam(MIP_DISPLAY) and related control variables.
JOpt logging just uses Java 1.4 logging, but you can piggy back your own logging code in your own programming that can use what we've already written. In your file MyClass.java, add the line:
private static Log log = new Log(MyClass.class);
Then, after importing import edu.harvard.econcs.util.Log, you can make calls like:
log.debug(“This is a debug message”);
In a complicated program, picking the right level of message printing will be useful to you, since you can control what level of message gets printed by adding a line to the bottom of file log.config in local directory edu/harvard/econcs/util/ For instance, if MyClass is in package edu.harvard.work, you would add the line:
edu.harvard.work.MyClass = DEBUG
if you wanted to see all messages that were DEBUG severity or worse. Note that this overrides the default value that is listed in .level in that same log.config file.
Possible log levels (from most severe to least severe) are:
ERROR
WARNING
MAIN
INFO
TRACE
DEBUG
TRIVIAL
As a final performance note, if the debug output is in a speed-critical part of your code, it is worthwhile to write your code in this fashion:
if (log.isDebugEnabled())
log.debug("Skipping term: " + term);
Q. Help!
A. Read this FAQ, then go to
https://list.eecs.harvard.edu/mailman/listinfo/jopt-users
LOGGING
Q: I get an error like:
Can't read log config stream
Jun 23, 2005 11:48:24 AM edu.harvard.econcs.util.Log info
INFO: Contacting Server for remote solver: econcs.eecs.harvard.edu:2001
A: You need to include Jopt directory in the classpath (so it can find log.config, which lives in the edu.harvard.econcs.util directory)
SPEED
Q. Let's say that I'm writing a single large MIP that is 128MB or so. Do I really need to serialize this over the wire, and re-serialize it before CPLEX solves it?
A. No! (This is what the “Unified Mode” is all about. See above instrunctions.) In summary, here is how to embed CPLEX calls into your client program. In effect, you eliminate the need for a server, and your client also runs the server code, which calls CPLEX. Rather than calling new SolverClient(host, port), just call new SolverClient() in your code. But, in order to do this, you also need to:
1. Help CPLEX by setting your ILOG_LICENSE_FILE enviornment variable.
2. include cplex.jar (ships with cplex) [and jopt.jar, of course] in your classpath.
2. provide Java with a pointer to the cplex .so file (Linux) or dll file (windows)
For example, if running csh on a machine where CPLEX is installed
in /usr/ilog and jopt.jar is in your current directory:
setenv
CLASSPATH .:jopt.jar:/usr/ilog/cplex81/lib/cplex.jar
followed
by:
java
-Djava.library.path=/usr/ilog/cplex81/bin/i86_linux2_glibc2.3_gcc3.2
-Xmx1024m YourClassFileHere
Q. Why is JOpt running slowly when I call solve? (My machine is pegged at 100%.)
A. Do you have debugging on? Debugging eats up a lot of local processor on a solve call for large MIPs, as it spits out the MIP it is trying to solve on the console screen. Look at log.config in your local directory edu/harvard/econcs/util/ and look for the line that starts “.level=”. This should be MAIN. Moreover, if you have a debugging function that is very expensive, you should wrap it with a call to log.isXXXEnabled. For instance, a call to log.trace(EXPENSIVE ARGUMENTS) should be wrapped with log.isTraceEnabled()
A. Are you running in client-server mode? Is it serializing a huge MIP? If so, consider unified mode.
A. Look at your process overhead -- is it the solver or JOpt that is the bottleneck? If it is the solver, not much we can do.
Q. Running in client-server mode, why is JOpt running slowly when I call solve? (My machine is not pegged.)
A. Does the server-side have debugging on? Debugging eats up a lot of server processor on a solve call for large MIPs, as it spits out all sorts of progress information to the console screen. Look at log.config in your remote directory edu/harvard/econcs/util/ and look for the line that starts “.level=”. This should be MAIN. If you did not set up the server-side component, ask the person who did to check this.
A. Look at your process overhead -- is it the solver or JOpt that is the bottleneck? If it is the solver, not much we can do.
Q. Could it be serialization time / transfer time that is making my program run slowly?
A. Yes, it is possible, especially if your client and server have a poor communications link, or if the MIP is 10s of megabytes in size or larger. But if you haven't already turned off debugging (see previous questions), try that first. Communicating a MIP problem with 100's of constraints and variables can usually be measured in ms, or a few seconds.
LPs/MIPs are transferred from your program to the server over Java RMI. This arrangement makes efficient use out of these server machines and doesn't let any one user hog a solver license. Our experience has been that for SMALL/MEDIUM problems, the Java RMI time is small compared to the solve time, and if you are using co-racked blades, the RMI serialization / transfer / deserialization time for both the client to server and server back to CPLEX is actually dominated by the time it takes to load CPLEX with the problem! For LARGE problems, Java RMI serialization sucks and you should run in unified mode.
MEMORY
Q. What memory things should I know about?
A. In your own launching of Java and in the build.xml file on the server side, you may need to specify a -Xmx parameter to avoid a java.lang.OutOfMemoryError for large problems. Run java -X for more information about this error. The default JOpt Server settings is 1GB.
SPECIAL FEATURES OF JOPT
Q. Special Features: What server parameters can I control?
A. In theory, this depends on whether you are using CPLEX, lpSolve, or some other backend solver. In practice, right now you can only edit parameters with CPLEX, and these parameters are a subset of what we thought was useful from the Cplex documentation. (See the Appendix in the CPLEX documentation. A copy for 8.1 for use in Harvard is at eecs.harvard.edu/~jeffsh/_cplexdoc
Q. Special Features: Seeding values: How do I know that seeding values is working?
A. When using CPLEX, you should see:
[java] MIP start values provide initial solution.
but you know that the starting values weren't accepted if you see:
[java] Warning: MIP start values ignored because of non-optimal
[java] subproblem optimization.
UNIFIED MODE
Q. When I run the server, I see the following:
java.lang.UnsatisfiedLinkError:
no cplexXX in java.library.path
java.library.path must point to
the directory containing the CPLEX shared library
try invoking
java with java
-Djava.library.path=...
java.lang.UnsatisfiedLinkError:
CPXopenCPLEX
at ilog.cplex.Cplex.CPXopenCPLEX(Native Method)
at
ilog.cplex.CplexI.init(CplexI.java:3601)
at
ilog.cplex.CplexI.<init>(CplexI.java:287)
at
ilog.cplex.IloCplex.<init>(IloCplex.java:8556)
at
edu.harvard.econcs.jopt.solver.impl.CPlexMIPSolver.solve(CPlexMIPSolver.java:40)
at
(etc)
A. Either you haven't -Djava.library.path'd correctly, or this is likely a CPLEX installation issue. Make sure that you have installed more than just the static CPLEX library during CPLEX installation. Then, make sure you are running Java with something like: -Djava.library.path=C:\ILOG\cplex81\bin\msvc6
CLIENT/SERVER MODE
Q. When
running the test program, it seems to hang when making a client
connection.
A. If
your server is multi-homed, this can occur. Sun's view of the world
for RMI (used by JOpt) is that every multi-homed host has a 'most
public' IP address. This is somewhat bogus in real life. One
workaround is to edit build.xml to have a
-Djava.rmi.server.hostname=REAL-IP-ADDRESS.
We may have a
cleaner workaround in the future.
Q. In client-server mode, why does my client get a:
ERROR SolverClient: Can't contact server
java.rmi.ConnectException: Connection refused to host: MACHINE NAME HERE; nested exception is:
java.net.ConnectException: Connection refused: connect
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:574)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:185)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
A. This means that the server, MACHINE NAME, isn't running, or isn't listening on the port that you expect. Are you sure the server is running? Are you sure you are contacting it on the right port?
Q. Client/Server: I just starting using JOpt again, and I know the server is running -- in fact, it has been running for a long while without being used. When the client calls solve now, it seems to hang. What is going on?
A. Restart the server. Not sure why this is, but guessing it's a RMI registry timeout issue.
Q. I am seeing weird behavior when running the client in Eclipse.
A. We think we fixed these, but Eclipse seems to use a different internal compiler that whatever you have installed, which screws you up if you don't have SerialUID since it can generate different SerialUID.
Q. Why do I get a:
ERROR SolverClient: Can't contact server
java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.lang.ClassNotFoundException: edu.harvard.econcs.jopt.solver.server.SolverServer_Stub (no security manager: RMI class loader disabled)
A. You should never see the “no security manager” message anymore. This message used to mean that your JOpt client jar is out of date with the server jar. But we think we fixed that.
COMPILING JOPT
Q. When compiling JOpt from scratch, and I run RMIC, I get a message that RMIC can't be run.
A. This can occur if you didn't install the Sun JDK but only installed the Sun JRE. (For instance, on Windows: Is rmic.exe on your machine? Is it where java.exe lives? Are you sure? Do you have multiple java.exe's but only a few rmic.exes, indicating that you have installed several copies of the JDK/JRE in the past but that you are using the wrong one?)
Q. When compiling JOpt from scratch, and I run RMIC, I get a message like:
java.lang.UnsupportedClassVersionError: sun/rmi/rmic/Main (Unsupported major.minor version 49.0)
A. Make sure your rmic.exe and java.exe are from the same version. This can occur if you compile with 1.5 and rmic with 1.4, for instance.