Add jar libraries to the final jar file

When using Netbeans to create Java projects, you might be using extra library files (jar files), for example, twitter4j-core-2.2.3.jar. The default Netbeans packaging excludes these library files when you build the project. This will result in missing class files when you run the jar file. To prevent this we can add the jar files itself as part of the final application. For this do as follows:

1. Add the library jar files into a folder within the project, say /lib

2. When you use the option “Add JAR/Folder” in Netbeans, make sure you use ‘Relative path’ option instead of Absolute path.

3. In the “build.xml” file present in the project folder, add the following:

<target name="-post-jar">

   <jar jarfile="dist/final_dist.jar">

      <zipfileset src="${dist.jar}" excludes="META-INF/*"/>

      <zipfileset src="lib/library1.jar" excludes="META-INF/*"/>

      <zipfileset src="lib/library2.jar" excludes="META-INF/*"/>

      <manifest><attribute name="Main-Class" value="packageName.Main"/></manifest>

    </jar>

</target>

Here ‘final_dist’ is the name of the final jar file that includes the library jars.

Make sure to set the value of ‘packageName.Main’ appropriately to the main class file of your main jar file i.e replace ‘packageName’ and ‘Main’ as it is in your project

Making ‘make’ work

‘make’ is used to produce output files from several input files, although it is more usually used to compile programs.

This is a very short note of ‘make’ utility, mostly made for my own reference :). The best way to study is to look at makefiles! ‘Beginning Linux Programming’ by Neil Matthew and Richard Stones is a good book to start linux programming. It also mentions ‘make’ utility.

‘make’ searches for a file ‘makefile’ or ‘Makefile’. To use another filename with make use -f option.

make -f Makefile4

Makefile Syntax

Makefile is a set of Dependencies and Rules.

Dependencies: It consist of two parts – Target file and the set of file on which target depends i.e. prerequisites.

Rules: This consist of the command that must be executed to obtain the Target file from the prerequisites.

Syntax is:

Target_name:<space or tab>Prerequisite1 [Prerequisite2 …]

<tab>Command_to_Execute

Important: The Rule should be on a line that begins with TAB.

Comments begin with #

Example:

myapp: main.o 2.o 3.o
    gcc -o myapp main.o 2.o 3.o

main.o: main.c a.h
    gcc -c main.c

2.o: 2.c a.h b.h
    gcc -c 2.c

3.o: 3.c b.h c.h
    gcc -c 3.c

Here myapp is the final executable. It depends on main.o, 2.o and 3.o.

main.o in turn depends on main.c and a.h. Similarly others.

Usually a Target called ‘all’ is used (as the first target). This is handy if more than one output files are required.

By default ‘make’ tries to fulfill the first ‘Target’. So here ‘make’ tries to obtain first target ‘myapp’. To obtain ‘myapp’, it requires main.o, 2.o and 3.o. So then it tries to obtain mao=in.o and so on.

We can also specify a specific target:

make main.o

Macros in Makefile

Defined by MACRONAME=value

The value of a macro is accessed by $(MACRONAME) or ${MACRONAME}. Some also support $MACRONAME.

$(MACRONAME) is replaced with the value of the macro. ‘value’ can be empty also.

Commonly used macro names:

CC                – Compiler

INCLUDE  – Directory in which include files are present

CFLAGS     – Compiler flags/options

Example:

all: myapp
CC=gcc
CFLAGS= -g -Wall -ansi
# dot specifies current directory
INCLUDE=.

myapp: main.o 2.o
    $(CC) -o myapp main.o 2.o

main.o: main.c a.h
    $(CC) -I$(INCLUDE) $(CFLAGS) -c main.c

2.o: 2.c a.h b.h
    $(CC) -I$(INCLUDE) $(CFLAGS) -c 2.c

Here $(CC) -I$(INCLUDE) $(CFLAGS) -c main.c is expanded to gcc -I. -g -Wall -ansi -c main.c

Predefined Macros:

$?              – List of prerequisites changed more recently than the target

$@            – Name of current target

$<             – Name of current prerequisite (Example – main.c)

$*              – Name of current prerequisite without suffix (Example – main for main.c)

Two other useful characters:

– (minus sign)     – Ignore errors. For example, -mkdir ignores any error if the directory already exists.

@                   – Tells ‘make’ not to print the command to output. We may use echo to print instructions.

Other Targets

I have always wondered what is ‘make clean‘ and ‘make install‘ that we type while compiling programs from source. Its only now that I understood that ‘clean’ and ‘install’ are just other targets. So ‘make clean’ and ‘make install’ tries to achieve ‘clean’ and ‘install’ targets.

Example:

...
...
clean:
    -rm main.o 2.o 3.o

install: myapp
    @if [ -d $(INSTDIR) ]; \
    then \
        cp myapp $(INSTDIR);\
        chmod a+x $(INSTDIR)/myapp;\
        echo "Installed in $(INSTDIR)";\
    else \
        echo "Error: $(INSTDIR) does not exist";\
    fi
...
...

‘clean’ has no dependencies. So it is executed whenever the target ‘clean’ is specified. The command ‘rm’ is preceded by ‘-‘ to ignore the errors. Similarly we have used ‘@’ before ‘if’.

Note: ‘[‘ is short for command ‘test‘. It is used for checking files. For more info, ‘man test‘

Built-in Rules

‘make’ has several Built-in rules. You can see them by typing ‘make -p‘.

So makefile can simply be

main.o: main.c a.h
2.o: 2.c a.h b.h

The built-in rules are applied by looking at the file extensions(suffixes). We can add new suffix rules.

Syntax:

<old_suffix>.<new_suffix>:

Example for converting .cpp to .o files:

.SUFFIXES: .cpp
.cpp.o:
    $(CC) -xc++ -I($INCLUDE) $(CFLAGS) -c $< 

Libraries and ‘make’

‘make’ has built-in rules to create and append to library(archive files created with ‘ar’ command). To refer to file in the library use following syntax:

libMyCol(foo.o)

This refers to file foo.o as stored in library libMyCol.a.

Example:

...
MYLIB = libMyCol.a
...
...
myapp: main.o $(MYLIB)
    $(CC) -o myapp main.o $(MYLIB)
...
...
$(MYLIB): $(MYLIB)(2.o) $(MYLIB)(3.o)
main.o: main.c a.h
2.o: 2.c a.h b.h
3.o: 3.c b.h c.h
...
...
clean:
    -rm main.o 2.o 3.o

install: myapp
    @if [ -d $(INSTDIR) ]; \
    then \
        cp myapp $(INSTDIR);\
        chmod a+x $(INSTDIR)/myapp;\
        echo "Installed in $(INSTDIR)";\
    else \
        echo "Error: $(INSTDIR) does not exist";\
    fi
...

A Useful Option

Want to create dependency list easily? Use -MM option with gcc. It produces a list which can be almost copy-pasted for use in a makefile!

Example:

gcc -MM main.c 2.c 3.c

A quick introduction to GCC

GCC – GNU Compiler Collection
From GCC manual – When you invoke GCC, it normally does preprocessing, compilation, assembly and linking.

Compile and Link a simple program:

gcc -o hello.out hello.c

This creates an executable ‘hello.out’ from file ‘hello.c’
-o specifies the output file.

Specify Language:

Create a C file and save it as ‘hello.cpp’.

gcc -o hello.out hello.cpp

This results in an error as gcc tries to perform C++ compilation(assumed from the extension .cpp).
We can specify that the file is a C file as follows:

gcc -x c -o hello.out hello.cpp

-x specifies the language.

Compile Only(No Linking):

gcc -c hello.c

-c tells gcc to compile ONLY. This creates ‘hello.o’, which is an object file.

Compiling and Linking multiple files:

File: hello.c

#include<stdio.h>

long myabs(long);
long factorial(long);

int main()
{
 printf("Hello World..\n");
 printf("Absolute(-3) = %d\nFactorial(5) = %d\n",myabs(-3),factorial(5));
 return 0;
}

File: myabs.c

long myabs(long a)
{
 if(a<0) return -a;
 return a;
}

File: myfac.c

long factorial(long no)
{
 if(no<=1) return 1;
 else return no*factorial(no-1);
}

METHOD 1: Use source files directly:
Make executable hello.out from 3 files by

 gcc -o hello.out hello.c myabs.c myfac.c

METHOD 2: Use a source file with one or more object files:
Compile one file

 gcc -c myabs.c

Now object code of myabs.c is obtained. Use this in usual gcc command:

 gcc -o hello.out hello.c myabs.o myfac.c

Or you can compile remaining files also and use their object files:

 gcc -o hello.out hello.c myabs.o myfac.o
 gcc -o hello.out hello.o myabs.o myfac.o

Create Library:

The above methods are suitable if the number of source files are really small. But if there are large number of source files, then its difficult to specify them each and every time you compile. To make this easy, you can create library.
Library is a collection of object files. Create library with ‘ar’ command, which is short for ‘archive’.
So first create object files (.o files). With above example we have two object files – myabs.o and myfac.o. Now create library:

 ar cr libMyCol.a myabs.o myfac.o

Now a library libMyCol.a is created. To see the contents of a library use ‘ar t’:

 ar t libMyCol.a

It will show the present contents (myabs.o and myfac.o).

Compile using the library:

 gcc -o hello.out hello.c libMyCol.a

OR

 gcc -o hello.out hello.c -L. -lMyCol

-Ldir specifies the direcory in which library files are present. Here we use current directory, so we use a ‘dot’ as directory.
-llibrary specifies the library name. library should only contain library name – extensions and preceeding ‘lib’ is omitted. So here ‘libMyCol.a’ is represented by -lMyCol   . This is why library names should start with ‘lib’.

Create Shared Library:

To create shared libraries, object files must be created with -fPIC.
From gcc manual :
-fpic
Generate position-independent code (PIC) suitable for use in a shared library, if supported for the target machine.  Such code accesses all constant addresses through a global offset table (GOT). The dynamic loader resolves the GOT entries when the program starts (the dynamic loader is not part of GCC; it is part of the operating system).  If the GOT size for the linked executable exceeds a machine-specific maximum size, you get an error message from the linker indicating that -fpic does not work; in that case, recompile with -fPIC instead.

Here we use -fPIC
Before continuing delete all files except the three C files – it looks nice to have a clean directory 🙂

STEP 1: Create object files as follows:

 gcc -c -fPIC myabs.c
 gcc -c -fPIC myfac.c

STEP 2: Now we create shared library or shared object(.so) file from the two .o files created above.

gcc -shared -o libMyCol.so myabs.o myfac.o

The shared library created is libMyCol.so. You can delete the .o files as they are no longer needed.

The Shared Library can be linked either Statically or Dynamically.

Static Linking of Shared Library:

METHOD 1: Give full path to .so file

 gcc -o hello.out hello.c ./libMyCol.so

METHOD 2: Use LD_LIBRARY_PATH environment variable

First set LD_LIBRARY_PATH variable to current directory (dot). Then export it. These two steps are shown below:

LD_LIBRARY_PATH=.
 export LD_LIBRARY_PATH

Now you can perform the final compilation and linking as follows:

 gcc -o hello.out hello.c libMyCol.so

Dynamic Linking of Shared Library:

To do dynamic linking, you need to modify the source file ‘hello.c’ as follows:

File: hello.c

#include<stdio.h>
#include<dlfcn.h>

typedef long (*FUN_PTR)(long);

int main()
{
 void *handle = dlopen("libMyCol.so",RTLD_LAZY);
 if(handle==0)
 {
  printf("ERROR: Failed to load the library\n");
  return -1;
 }
 FUN_PTR fptr1 = dlsym(handle,"myabs");
 FUN_PTR fptr2 = dlsym(handle,"factorial");
 if(fptr1==0 || fptr2==0)
 {
  printf("ERROR: Cannot retrieve function address\n");
  return -1;
 }
 long tp = (*fptr1)(-3);
 printf("Hello World..\n");
 printf("Absolute(-3) = %d",tp);
 tp = (*fptr2)(5);
 printf("\nFactorial(5) = %d\n",tp);
 dlclose(handle);
 return 0;
}

If you haven’t set LD_LIBRARY_PATH variable and exported it as mentioned above, then do it now.
Now perform dynamic linking by

 gcc -o hello.out hello.c -ldl

dl is the dynamic linking library. -ldl asks gcc to use this library.

Dynamic Linking library provides 4 functions – dlopen, dlclose, dlsym and dlerror. These are in dlfcn.h.
dlopen is used to open a .so file. It returns a handle to the file.
FUN_PTR is a function pointer. fptr1 has address of function myabs().
dlsym takes two parameters – handle and function name.
dlclose is used to close the .so file.

Thanks to Praseed.P for his session on GCC at ILUG Cochin. It had been very informative and this post is a result of that session.