Java’s New FMA: Renaissance Or Decay?
The Java community has been buzzing recently as this JEP took the spotlight.
This JEP proposes to deprecate and remove the memory access and allocation methods from theUnsafe class, and let's just say that, for the most part, it was not well received…
After I got over the initial shock, I was relieved to discover Java is not leaving us high and dry, and there are alternatives; let's see how they stack up.
GitHub: https://github.com/tomerr90/UnsafeVSFMA
What is Unsafe?
There are no free lunches, and Java is no exception.
That wonderful, automatic memory management (i.e. garbage collection) that we enjoy so much comes at a cost. In fact, various costs.
One of them is that each object in Java has an overhead of quite a few bytes dedicated to what is called an “Object Header” that is needed for garbage collection (amongst others), which is not needed in languages without automatic memory management like C++, for example.
However, there is a way to tell Java “trust me, Im an engineer” and allocate what is called “off-heap” memory.
Off-heap memory lives outside of the Java memory model, which means you are responsible to manually allocate and free the memory used there, and by doing so risk all kinds of memory issues, like leaks.
In exchange, however, you get some REALLY fast memory access methods, and you can do things that are otherwise impossible within the realm of the Java memory model.
What‘s all the fuss about?
You might be reading this and thinking “OK, so they decided to remove some niche API that most Java devs haven’t even heard about, big deal…” Well, it is!
Unsafe is indeed not a class that most Java devs directly use, but make no mistake, you are very likely using it indirectly.
Unsafe is used in practically every high performance library / use case in Java, e.g. Netty, Spark, Avro, Kryo and many more, and they all depend on it to give you those sweet benchmark numbers.
Enter Foreign Memory Access (FMA)
Working with Unsafe is like playing with a loaded shotgun (The Unsafe Class: Unsafe at Any Speed). You better know what you are doing.
Which is why this API was never really meant to be used outside of the Java language core libs, but it somehow managed to find its way into some hands eager to squeeze every last ounce of performance from the JVM and the rest is history.
Recently, the folks at Oracle decided to use the APIs created as part of Project Panama (JEP-424) to replace Unsafe with a safer, officially supported and standardized API.
As of Java 21 Project Panama is still in preview, so take things with a grain of salt, lets get into it!
Results!
We are going to look at the throughput (so higher is better) for the following:
- Reading a bunch of primitives (On/Off Heap)
- Writing a bunch of primitives (On/Off Heap)
Running on my Mac:
- Java 21
- M1 Max CPU
- 64GB RAM
Benchmark Mode Cnt Score Error Units
FMASerDe.fmaRead thrpt 25 919.405 ± 2.279 ops/us
FMASerDe.fmaWrite thrpt 25 1069.929 ± 2.972 ops/us
FMASerDe.unsafeRead thrpt 25 919.617 ± 3.035 ops/us
FMASerDe.unsafeWrite thrpt 25 751.566 ± 2.949 ops/us
FMASerDeOffHeap.fmaRead thrpt 25 489.600 ± 3.042 ops/us
FMASerDeOffHeap.fmaWrite thrpt 25 450.694 ± 1.346 ops/us
FMASerDeOffHeap.unsafeRead thrpt 25 174.658 ± 0.839 ops/us
FMASerDeOffHeap.unsafeWrite thrpt 25 122.167 ± 0.855 ops/usSame read speed, while write is 42% faster with FMA!
Impressive! FMA is almost 3 times faster for Off Heap accesses!
Off heap data structures
Since allocate/free are expensive operations on or off heap, off heap data structures usually dont allocate or free every time you insert/remove from them, they allocate big chunks of memory and use offsets within these chunks to “allocate” / “free” memory for the arbitrary sized pieces of data you insert/remove from the data structure.
In the new API, an Arena object is used to allocate memory and each chunk of memory you allocate is accessed using a MemorySegment that can not be freed, all memory allocated from an Arena is freed together when you close it.
That means our ability to free memory is less granular than with Unsafe which lets you free individual chunks, on the other hand, it makes it easier to avoid memory leaks.
Conclusions
The new FMA seems to pack a punch! It’s ~3 times faster than Unsafe for Off Heap and same speed reads with 42% faster writes for On Heap.
Other than free being less granular, which I think (I might be wrong) is rarely used in off heap data structures, the new API seems to maintain the same capabilities as Unsafe while providing a slightly more convenient API.
Overall this looks to be a big win for Java! So to the Java devs out there, you can rest easy, and for the people who worked on this project, kudos!
That’s it from me!
Thank you for tuning in. Until next time, stay efficient!
