Java loops speed benchmark in JDK 11

Sometimes ago I stumbled upon a bunch of articles mainly written in 2014 or 2015 which compare different loops speed in Java. While the articles are usually well documented, I found they are quite dated. So from the technology point of view, the results are obsolete. Many things have changed since then. For instance, CPUs have seen the massive upgrade and three versions of JDK have released.

So to have a more realistic view of Java lambda speed, I decided to do a similar experiment but this time with JDK 11. The aim obviously is to figure out whether the lambda for-each andParallel Stream can outperform the good old for-loop,iterator, or enhanced for-loop.

I’m interested to know whether the lambda functions can utilize the modern CPU improvements efficiently to beat the good old loops. And that is because I’m passionate to use functional features of Java in my code heavily. But often I receive criticisms/objections quoting the articles that lambda functions perform worse.

For this experiment, I will run a benchmark on five different type of loops as follow:

for-loop
enhanced for-loop
iterator
for-each lambda
parallel stream for-each lambda

To simplify the test I chose to use ArrayList of String as my dataset in four different sizes: 1K, 10K, 100K, and 1M.

I ran the experiment on ten iterations on a 2018 Macbook pro machine equipped with Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz, and 16GB RAM with OpenJDK 11.0.1.

I have run the experiment using the code below (bear the code cleanness):

import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.UUID;
import java.util.stream.IntStream;

public class LoopBenchmark {

    private static final int THOUSAND = 1000;

    private static final int TEN_THOUSAND = 10000;

    private static final int HUNDRED_THOUSAND = 100000;

    private static final int ONE_MILLION = 1000000;

    public static void main(String[] args) {
        List<String> thousandItems = new ArrayList<>();
        List<String> tenThousandItems = new ArrayList<>();
        List<String> hundredThousandItems = new ArrayList<>();
        List<String> oneMillionItems = new ArrayList<>();
        List<String> results = new ArrayList<>();

        IntStream.range(0, THOUSAND).forEach(x -> thousandItems.add(UUID.randomUUID().toString()));

        IntStream.range(0, TEN_THOUSAND).forEach(x -> tenThousandItems.add(UUID.randomUUID().toString()));

        IntStream.range(0, HUNDRED_THOUSAND).forEach(x -> hundredThousandItems.add(UUID.randomUUID().toString()));

        IntStream.range(0, ONE_MILLION).forEach(x -> oneMillionItems.add(UUID.randomUUID().toString()));

        IntStream.range(0, 10).forEach(x -> {
            // for-loop old school 1K
            Instant start = Instant.now();

            for (int i = 0; i < thousandItems.size(); i++) {
                System.out.println(thousandItems.get(i));
            }

            Instant end = Instant.now();
            Duration timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional for-loop 1K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // for-loop old school 10K
            start = Instant.now();

            for (int i = 0; i < tenThousandItems.size(); i++) {
                System.out.println(tenThousandItems.get(i));
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional for-loop 10K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // for-loop old school 100K

            start = Instant.now();

            for (int i = 0; i < hundredThousandItems.size(); i++) {
                System.out.println(hundredThousandItems.get(i));
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional for-loop 100K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // for-loop old school 1M

            start = Instant.now();

            for (int i = 0; i < oneMillionItems.size(); i++) {
                System.out.println(oneMillionItems.get(i));
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional for-loop 1M: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // iterator 1K
            Iterator<String> iterator = thousandItems.iterator();

            start = Instant.now();

            while (iterator.hasNext()) {
                System.out.println(iterator.next());
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional iterator 1K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // iterator 10K
            iterator = tenThousandItems.iterator();

            start = Instant.now();

            while (iterator.hasNext()) {
                System.out.println(iterator.next());
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional iterator 10K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // iterator 100K
            iterator = hundredThousandItems.iterator();

            start = Instant.now();

            while (iterator.hasNext()) {
                System.out.println(iterator.next());
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional iterator 100K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // iterator 1M
            iterator = oneMillionItems.iterator();

            start = Instant.now();

            while (iterator.hasNext()) {
                System.out.println(iterator.next());
            }

            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("Traditional iterator 1M: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // 1K for-loop object
            start = Instant.now();
            for (String item : thousandItems) {
                System.out.println(item);
            }
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-loop object 1K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // 10K for-loop object
            start = Instant.now();
            for (String item : tenThousandItems) {
                System.out.println(item);
            }
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-loop object 10K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 100K for-loop object
            start = Instant.now();
            for (String item : hundredThousandItems) {
                System.out.println(item);
            }
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-loop object 100K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 1M for-loop object
            start = Instant.now();
            for (String item : oneMillionItems) {
                System.out.println(item);
            }
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-loop object 1M: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // 1K for-each lambda
            start = Instant.now();
            thousandItems.forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-each lambda 1K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 10K for-each lambda
            start = Instant.now();
            tenThousandItems.forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-each lambda 10K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 100K for-each lambda
            start = Instant.now();
            hundredThousandItems.forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-each lambda 100K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 1M for-each lambda
            start = Instant.now();
            oneMillionItems.forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("for-each lambda 1M: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));


            // 1K parallel stream for-each lambda
            start = Instant.now();
            thousandItems.parallelStream().forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("parallel stream for-each lambda 1K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 10K parallel stream for-each lambda
            start = Instant.now();
            tenThousandItems.parallelStream().forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("parallel stream for-each lambda 10K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 100K parallel stream for-each lambda
            start = Instant.now();
            hundredThousandItems.parallelStream().forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("parallel stream for-each lambda 100K: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));

            // 1M parallel stream for-each lambda
            start = Instant.now();
            oneMillionItems.parallelStream().forEach(System.out::println);
            end = Instant.now();
            timeElapsed = Duration.between(start, end);
            results.add(String.format("parallel stream for-each lambda 1M: %s milliseconds - experiment %s", timeElapsed.toMillis(), x));
        });

        Collections.sort(results);
        results.forEach(System.out::println);

    }
}

Results analysis

In general, each loop performs differently in different datasets. So coming with a single conclusion is quite tricky. What I can see clearly is parallel streamperforms worst in almost all experiment with exception to 1K dataset. I explain about it later.

Anyhow, in the following section, I discuss the benchmark result for each dataset.

The 1K dataset

As you can see in the below picture, enhanced for-loop outperforms all other loops. Surprisingly, lambda for-eachperforms worst by a large margin, which followed by parallel-stream.

One thing to note is even 5 milliseconds difference does not make anything significantly slower. I believe the 1K benchmark is not good enough to draw any conclusion from it.

The 10K dataset

By contrast to the previous dataset, for-each lambda had a very good performance this time. It outperforms the traditional for-loop and enhanced for-loop. And parallel-stream performs worst by a high margin.

The 100K dataset

This time the winner is again enhanced for-loop. Lambda for-each takes the second place with almost ten milliseconds difference. And the worst performer is again parallel-stream.

The 1M dataset

iterator has the upper hand which followed by enhanced for-loop. And as usual parallel-stream occupied the worst performer by a large margin.

Verdict

Even though lambda/functional loops didn’t perform better in any case than conventional loops, it does not imply that we should avoid using them. This is especially true about for-each. The performance difference is so negligible (at least in JDK 11). In my honest opinion the features provided by for-each like filter, map, … along the way outweighs this performance tradeoff. Not to mention the code clarity and readability.

Hence, I rather stick to my habit of using Java functional features.

Of course parallel stream is the exception here. As you can see in almost all datasets parallel stream performs worst with a large margin. The reason behind it is because parallel stream starts fork-join thread pool each time. And this imposes significant overhead to the process. Yet, you should never avoid using it. Like any other feature, it should be used in appropriate scenarios and not abuse it.