Faster Java Serialization

Copyright (C) 2006 by Leonardo "Bubble" Mesquita

Overview | Documentation | Project page | Downloads | FAQ | Benchmarks

Frequently Asked Questions

• Why make a faster serialization mechanism for Java?

  That is actually a very good question. There is a common misconception that the default Java serialization mechanism is slow. In fact, it was this very misconception that prompted me to start this project in the first place.

  During the project development, I studied the default mechanism deeply, and came to the conclusion that, where Java serialization is slow, it is usually due to a lack of understanding of its mechanism and poor use of the Object*Stream classes.

  Using the default mechanism properly, Java serialization is fast enough for most applications. Then why go ahead and try to make it faster?

  Java Serialization was designed in a very early stage of the Java technology development history. The mechanism is essentially the same since version 1.1. While performance was one of its design goals, there where many others, some even conflicting with performance (such as security and the ability to support "evolving classes"), making it a nice trade-off between the many supported features, but not excelling in any partcular one. It was implemented over the Java stream concept, and does not integrate well with Java NIO, which came much later.

  One of the things that further hinders the performance of Java serialization is the need to use indirect means to access the values of the serialized object's fields. Most projects counter that by performing custom serialization. As the number of classes that need to be serialized grows, it rapidly becomes a major code maintenance issue. Some develop code generators to help with the tiresome task of writing custom serialization code.

  I just went one step further: runtime code generation of custom serialization code!

• How does it work?

  The idea is very simple: when first serializing an object of a given class, this class is inspected through reflection to generate code that serializes its declared fields. While it may be slow when first serializing objects of a given class, due to the penalty of compiling code at runtime, it is irrelevant for projects that serialize objects of the same class many times over.

  A great effort is made to access object fields directly. We avoid injecting code into user classes, so what we do is to create a new class on the same package of the serialized object's class. That gives us direct access to every package, protected and public fields. For private fields, we have no option but to resort to reflection, if we are to keep our hands off instrumenting user classes.

  Besides avoiding the reflection penalty, we don't support "class evolution", meaning that the user is responsible for ensuring the same version of the serialized classes are available on both ends of the serialization path. This makes this project more suitable for applications that serialize objects over a network, and not for applications that serialize objects to files, since files could be read much later, with different class versions.

  Also, objects are serialized directly to a NIO ByteBuffer. It should supposedly¹ increase performance, since it allows objects to be serialized directly to system or device buffers, as described by java.nio.ByteBuffer#allocateDirect and java.nio.FileChannel#map.

  Objects are serialized through a "serialization context". It has this name because the serialized form of a given object can differ depending on the "context", i.e., what objects have been serialized before, and so on. The concept is not very different from the stream concept, except that it is not actually attached to some other data stream. In order to de-serialize objects, one should use a "deserialization context", reading objects in the same order as they were written.

  During our tests, we also discovered a major performance bottleneck. In order to allow serialization of object graphs, including many references to the same objects and even cycles, and to reduce the size of the serialized data, we had to keep "seen" objects in a HashMap (actually an IdentityHashMap), so we could remember them later and replace them with references. It just so happens that, when sending a large number of objects, the HashMap has to grow, and that operation totally cripples the performance. Default Java serialization suffers from the same problem. We even tried many different HashMap implementations, but none of them was good enogh at "growing".

  Because of that, I created a "light serialization context", which is mostly the same as the "serialization context", except that it does not keep track of "seen objects". It should not be used for object graphs that contain cycles, but it ought to improve the performance of many applications that don't have that requirement.

  ¹It seems the performance of NIO isn't as good as expected yet. Check out this post and this other post regarding NIO performance

• How fast is it?

  Check out the benchmarks below.

Benchmarks

Back to FAQ

The test is located in class com.bubble.serializer.benchmark.MultipleTest. This class, along with the test objects in package com.bubble.serializer.objects.*, can be found in the project's SVN repository.

The test was run on the following platform: