Super Prev Next

Dynamic linking


Super Prev Next

Intro


Super Prev Next

Operating system support for dynamic linking

In a traditional developer toolkit that includes a static compiler and linker, e.g. a C++ toolkit, static libraries are linked together with the application code and all references between them are resolved at link time. Contemporary operating systems provide support for dynamic link (or shared) libraries. Unlike static libraries, they are linked with executables at run time. If an executable contains direct references to methods and/or fields from a DLL, linking is performed by the system loader when the executable starts. This is called load time linking. Some applications, however, postpone the linking process to a later phase so that the application’s code loads DLLs and resolves references using the system API. This is called run time linking. The executable and a set of loaded DLLs form a so called working set of an operating system process.

Typically, working sets of different processes have common components, such as DLLs implementing the system API functions. These components are shared between processes, that is, only one copy of code and constant data from a DLL is loaded into memory. This implies the following important benefits for native applications employing the dynamic link model:


Super Prev Next

What about Java applications?

The question is whether you the Java developer can benefit from dynamic linking supported by the target operating systems? If you are reading this manual, the answer is YES because you most probably have a copy of Excelsior JET JVM installed on your system.

First of all, the Java 2 platform classes are pre-compiled into a set of DLLs when your create a JET profile. Therefore, they will be shared between the processes established by JET-compiled executables, just like the DLLs providing the OS API are shared between conventional native applications. This effectively means that you already have a JVM capable of sharing constant data (such as reflection information) and code (e.g. methods of Java 2 platform classes) between several running Java applications.

As for your application’s code and third-party Java APIs not included in the Java 2 platform, you have two possibilities of using dynamic libraries:

The next Sections describe both techniques in details.


Super Prev Next

Invocation DLLs

To create a DLL callable from applications written in another language, you can use the JET Control Panel: click Invocation DLL on the Welcome Screen (see Starting Excelsior JET Control Panel) and select the desired jars on the Start page (see Step 1: Describing an application).

Using Invocation DLLs is a bit tricky. Like any other JVM, Excelsior JET executes Java code in a special isolated context to correctly support exception handling, garbage collection, and so on. That is why Java methods cannot be directly invoked from a foreign environment. Instead, you have to use the standard Java 2 platform APIs, specifically the Invocation API and Java Native Interface (JNI). See samples/Invocation in your JET installation directory for detailed examples.


Super Prev Next

Multi-component applications

Note: Before you begin, learn about multi-app executables (see Multi-app executables) to decide if you really need to create a multi-component application. For example, if you plan to compile two executables sharing common libraries and both executables are always deployed together, it would be much simpler to build a single multi-app executable rather than two executables and DLL.

Currently, Excelsior JET Control Panel does not support creating multi-component applications so you have to use the command line interface to the JET compiler. However, the Control Panel can assist you at the first steps of creating a multi-component application. Please, read this entire section carefully so as to correctly setup project files for DLLs/executables and avoid problems with compilation and running multi-component applications.


Super Prev Next

Building multi-component applications

This section contains step-by-step instructions for creating multi-component applications. Please, note that all the steps described below are obligatory.

It is supposed that you have a set of jar files containing all class files and resources necessary to your application. If your application includes some standalone classes/resources residing in a directory, you have to create a jar file from this directory’s contents. This is important because the JET compiler can automatically pack all resources from jar files to the resulting DLLs and executables. Moreover, it is much easier to write a JET project file if all classes and resources are packed to jars.


Super Prev Next

STEP 1. Consistency check

First of all, create a project to compile all the jars into a single EXE using the JET Control Panel, then proceed to the Step 3.

If there are consistency warnings, click the Class View buttons and inspect the Warnings tab below. If it says there are duplicated classes, fix the project by removing duplicated classes from your jar files, and repeat the project creation process until there are no more class duplications in the compilation set.

If warnings about unresolved import dependencies are displayed, you can ignore them at this step, provided the application works under the HotSpot JVM flawlessly.

Finally, go through the remaining pages to set up the project, compile it and make sure that your application works properly.

Note: The created project file cannot be used for subsequent compilation of DLLs and EXEs.


Super Prev Next

STEP 2. Defining multiple compilation sets

On this step, you define the contents of DLLs and executable(s) being created. Specifically, you break down the compilation set, checked for consistency as decried above, into several compilation sets, each to be compiled into a separate component (EXE or DLL). The breakdown is not arbitrary as it should obey the following rules:

So be ready that the model "one jar — one DLL" cannot be used if the jars have cyclic interdependencies.

The JET Control Panel can assist you in defining multiple compilation sets as follows:

  1. Open the JET Control Panel and click Invocation DLL on the Welcome Screen.
  2. On the page Start, check the "Enable Manual setting" box and four new buttons will appear on the screen. Using the upper button, add one or more jars to the compilation set.
  3. Go to the page Classpath.
  4. Check that NO warnings about unresolved import dependencies are displayed except those persisted since the Consistency Check step.
  5. If there are such warnings, add the respective jars until all imported classes are resolved.
  6. At this point, you have a compilation set (let us denote it CS1) consisting of one or more jars that will be compiled into the first DLL.
  7. Go to the New page and add few more jars. Again, check that no unresolved import dependencies are displayed. The files you added constitute the compilation set for the second DLL, let us name it CS2.
  8. Repeat the process for the remaining jars, thus defining compilation sets CS3,...,CSk, CSexe for other components.

    The compilation set CSexe, including the jar with the main class of your application has to be defined last as it will be compiled to the executable. The other compilation sets will be compiled to DLLs.

    Actually, you can define several compilation sets that reference CS1,...,CSk to create several executables with common DLLs.

Let us consider a typical example of multiple compilation sets. Suppose you have two applications using a Java Optional Package, such as JavaHelp. You may define three compilation sets: the first one contains the jars of JavaHelp and the others — jar files of your applications. This breakdown implies that you can create a DLL for the JavaHelp API and two executables using that DLL.


Super Prev Next

STEP 3. Creating the build environment /Example project files cited in this Section can be found in samples/MultiComponent in your JET installation directory./

Create an empty directory called, for instance, MultiCompBuild. In the directory, create JET projects (plain text files with the extension .prj) - one project per each component. For all but the last compilation sets defined on STEP 2, create projects named, say, CS_1.prj,..,CS_k.prj after the following template:

    -OUTPUTNAME=DLL-name    (without extension)
    -GENDLL+
    -CLASSABSENCE=HANDLE
    -IGNOREMEMBERABSENCE+
   
    -LOOKUP=*.jar=path-to-jars
   
    !uses path-to-project-file-1
    !uses path-to-project-file-2
   
    %jar files from this compilation set
    !CLASSPATHENTRY jar-1
        -PACK=NONCOMPILED
        -OPTIMIZE=ALL
        -PROTECT=ALL
    !END

You have to edit only the -OUTPUTNAME=, -LOOKUP=*.jar= , !CLASSPATHENTRY and !uses settings. Note that you have to add !uses directives (see The !uses directive) only to the projects that depend on other projects.

For the compilation set corresponding to the main executable, create a project file called, for instance, CS_EXE.prj, after the following template:

    -OUTPUTNAME=EXE-name    (without extension)
    -CLASSABSENCE=HANDLE
    -IGNOREMEMBERABSENCE+
   
    -LOOKUP=*.jar=path-to-jars
    -MAIN=main-class
   
    !uses CS_1.prj
    !uses CS_2.prj
       .  .  .
    !uses CS_k.prj
   
    %jar files from this compilation set
    !CLASSPATHENTRY jar-1-for-exe
        -PACK=NONCOMPILED
        -OPTIMIZE=ALL
        -PROTECT=ALL
    !END
   
    !CLASSPATHENTRY jar-2-for-exe
        -PACK=NONCOMPILED
        -OPTIMIZE=ALL
        -PROTECT=ALL
    !END
       .  .  .

In this template, you have to edit the -OUTPUTNAME=, -LOOKUP=*.jar=, -MAIN= and !CLASSPATHENTRY settings.


Super Prev Next

STEP 4. Compilation

Set MultiCompBuild as the current directory and compile these projects using the commands

   jc =p CS_1.prj
   ...
   jc =p CS_k.prj

   jc =p CS_EXE.prj

Now you can run the resulting executable.

Note that this particular order of compilation is important to transparently resolve references between the components being compiled. This is exactly the order in which the compilation sets were defined on STEP 2.


Super Prev Next

STEP 5. Re-compilation

If you change the jars from CSexe compilation set, it is enough to recompile only the executable with the command

   jc =p CS_EXE.prj

However, if jars compiled into one of the DLLs were changed, it is necessary to recompile that DLL as well as the components that were built after it, including the executable. Finally, if modifications of DLL’s jars change the import relationships (e.g. new classes are referenced), you may have to repeat the whole process starting from STEP 1. Fortunately, that is a rare case if you create DLLs from third-party jars which do not change.


Super Prev Next

Using multiple components

As it was said above, DLLs can be mapped to the process via load time and run time linking. This Section describes how JET-compiled applications employ both ways of linking.


Super Prev Next

Load time linking

If one component directly uses classes from another component, this always implies load time linking /!uses directive (see The !uses directive) also forces load-time linking for the imported component./ . Examples of direct uses are creating a class instance via the new operator, extending a class from another component, accessing static members, etc. However, if one component employs only the Reflection API to use classes from another components, load time linking does not occur because there are no direct references between the components. A typical example is loading classes via the Class.forName() method and passing the obtained class objects to the methods from the java.lang.reflect package.

Note that Class.forName() and the Reflection API also work for all classes from the components linked to the process at load time.


Super Prev Next

Run time linking

JET-compiled applications can load DLLs on demand in a portable way, that is, without using the operating system API. The Class.forName() method serves for this purpose so that looking for a class from a pre-compiled but not yet loaded DLL results in loading that DLL at run time.

Suppose component B has to load component A at run time. To enable run-time linking, two pre-conditions should be met:

  1. B does not reference A directly as described in the previous Section
  2. Class-to-DLL mapping for A should be specified when compiling B

The mapping allows the Class.forName() method to succeed for classes precompiled into DLLs loaded at run time. You need to tell the JET runtime in which DLL to look for each class or package. You may do that by adding the following options to the JETVMPROP equation or the JETVMPROP environment variable:

    -dll:class-name:dll-name    or
    -dll:class-prefix*:dll-name

where

class-name

is a fully qualified name of a class, such as com.MyCompany.MyClass

class-prefix

is a class name prefix followed by the “*” character:

    -dll:com.MyCompany.*:MyDll.dll

This syntax is useful when you compile an entire package into a DLL.

dll-name

is the name of the DLL containing the class class-name or classes whose fully qualified names begin with class-prefix. If dll-name contains spaces, enclose it in double quotes:

    -dll:com.MyCompany.*:"C:\My App\DLLs\MyDll.dll"

Note that if you do not specify a pathname, the DLL will be sought using the conventional Windows DLL search algorithm , but that may slow down application execution. If you specify an incomplete pathname, such as “DLLs\MyDll.dll”, that pathname will be appended to the name of the current directory and then to the name of each directory along the PATH.

You can specify more than one -dll option. If the name of the class being loaded matches more than one class-prefix, the class will be sought sequentially until it is found or there are no more matching prefixes.

Examples:

Project file:

    % MYAPP.PRJ
    -JETVMPROP=-dll:com.My.*:.\private.dll -dll:com.*:common.dll

Batch file:

    REM RUNMYAPP.BAT
    SET JETVMPROP=-dll:com.MyCompany.*:.\plugins\myplugin.dll
    MyApp.exe


Super Prev Next

Run time compilation

If some classes cannot be pre-compiled, for example, those of third-party plug-ins to your application, the JET Runtime enables Mixed Compilation Model.

In this case, the bytecodes for the classes are sought in accordance with the conventional search algorithm used by the Java 2 platform. Then, the bytecodes are loaded and compiled at run time.


Super Prev Next

Optimizing layout of DLLs in memory

The operating system does not load the entire executables into memory on startup, but loads their code and data on demand in 4KB pages. This reduces physical memory consumption and may improve startup time /See also Chapter Startup time optimization/ . However, if the executable could not be loaded at the base address specified in its header due to unavailability of sufficient virtual address space at that location, the operating system has to relocate the executable’s image to ensure proper operation. This causes more pages to be loaded into memory and slows down application startup. When building a multicomponent (EXE plus one or more DLLs) application, it is recommended to adjust DLL base addresses so that they do not overlap each other and the EXE file.

By default, all executables (EXEs and DLLs) produced by the JET compiler use the same image base address, which is 0x400000. You change the default base address by setting the IMAGEBASE equation:

    -imagebase=0x10000000

A considerable portion of the 4GB address space is however already occupied:

Address range Occupied by
above 0x400000 EXE
below 0x10000000 JET Run-Time DLLs
0x6D000000 and above JRE native method DLLs
0x70000000-0x78000000 system DLLs

Microsoft recommends to base DLLs from the top of the address range down, instead of from the bottom up, to avoid collision with dynamic memory blocks allocated by the application.

The final thing to take into account is that a base address is rounded up to a next 64K multiple.

So the algorithm to determine a proper DLL layout for your application can be as follows:

For each of the DLLs constituting your application, determine its image size using a third party PE dump utility or Explorer Quick View (section “Image Optional Header”, item “Size of Image”). You may also use the size of the DLL file as a safe approximation. Round that number up to a multiple of 0x10000, and subtract it from the base address of the previous DLL (this shall be 0x6D000000 for the first DLL.) The result shall be the DLL base address. Specify it in the IMAGEBASE equation in that DLL’s project file, e.g.:

    -imagebase=0x6CC00000

Note: If your application uses Java Extensions or third party Java libraries with native method DLLs, or accesses conventional DLLs through JNI wrappers, you have to take base addresses and image sizes of all those DLLs into consideration when placing your DLLs in the virtual address space.