Get started with Spring Cloud Netflix Hystrix

Circuit breaking is one of the necessity of the microservice architecture where services communicate with each other over network. The mean of this communication could be different (API call, messaging, etc.). Since the network is prone to failure and services do not have visibility over the status (healthiness, availability, load) of each other having circuit breaker in place makes the intercommunication much easier and more reliable. In addition to that, it brings some degree of resilience and fault-tolerance to the entire ecosystem. In this article, I demonstrate how to use Spring Cloud Netflix Hystrix circuit breaker by providing a very simple example.

Netflix Hystrix is a circuit breaker library created by Netflix which later open sourced. As of now Netflix does not add the new feature to it. In other words, Netflix Hystrix is in the maintenance mode. However, the guys in the Spring.io created a library on top of Hystrix which is know as Spring Cloud Netflix Hystrix. And it is actively maintained. That’s what we are going to use in this example.

It is worth mentioning that there are tons of circuit breaker libraries and frameworks are available. And some provide many exotic features and also are reactive. However, I decided to use Hystrix because,

Hystrix is still the most popular circuit breaker for JVM languages
No-frills does one job and does it well
Requires zero configuration
Easy to use
Beginner friendly

All the above reasons convinced me to use Hystrix for this example. Once you understood the concept of circuit breaking by having a real example, then switching between different libraries should not be a problem.

Real life scenario

Let’s assume we have a weather API (Eris) that its result depends on multiple other services. In other words, the Eris API aggregates results from different services (Geo location, IP address, and weather forecast). And the order of calling the dependent services are important. This means if any dependent service fails anywhere in the chain call, the entire request that came to Eris should fail as well. However, there is no need to wait for timeout or response error to switch to a fallback solution or fail gracefully. Instead we can use Hystrix to close the circuit (trip) of the faulty service.

In the Eris service, the most unstable service is IP service (IP API) which translates a user IP address to a relative latitude, longitude. To prevent instability of this service affects Eris API, we use Hystrix to short circuit calling to this service whenever one of the two conditions happen:

Response took more than 1000 ms
30 percent of calls to the IP service fails

And to keep the Eris responsive we switch to another service (eXTReMe-IP-Lookup) as a fallback solution which is less accurate but still usable.

To have a better understanding on the entire architecture have a look at the following diagram:

Implementing Spring Cloud Netflix Hystrix in Eris

In this section, I explain how I implemented Hystrix in Eris. Of course, you can take this implementation and apply to any service you wish.

Adding Spring Cloud Netflix Hystrix dependency

The first step is to add the Hystrix dependency to the project as follows:

<dependencies>
        ....
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
            <version>2.1.1.RELEASE</version>
        </dependency>
        ....
</dependencies>
....
....
<dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Greenwich.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
</dependencyManagement>

Here, I used Spring Cloud version Greenwich.RELEASE and spring-cloud-starter-netflix-hystrix version 2.1.1. You use any version that suits your needs. However, keep in mind that your Spring or Spring Boot version should be compatible with the Spring Cloud version.

Then we need to annotate the class that has main method inside with @EnableCircuitBreaker annotation.

Fallback implementation

Now that we have sorted out the Hystrix depencies, we need to code our fallback solution (eXTReMe-IP-Lookup integration) that is used when IP API fails. The integration of this part is no difference than any other services and so far has nothing to do with Hystrix. The only concern is to have a method exposed as public which Hystrix can call. As we have:

package com.madadipouya.eris.integration.fallbacks.iplookup;

import com.madadipouya.eris.integration.ipapi.remote.response.IpApiResponse;

@FunctionalInterface
public interface ExtremeIpLookupIntegration {

    /**
     * Retrieves information related to a given IP Address
     * by calling @see <a href="extreme-ip-lookup.com/json/">Extreme IP Lookip API</a> service
     *
     * @param ipAddress IP Address that needed to extract related information
     * @return {@link IpApiResponse} contains information about the IP,
     * most importantly latitude and longitude
     */
    IpApiResponse getCoordinates(String ipAddress);
}

package com.madadipouya.eris.integration.fallbacks.iplookup.impl;

import com.madadipouya.eris.integration.fallbacks.iplookup.ExtremeIpLookupIntegration;
import com.madadipouya.eris.integration.ipapi.remote.response.IpApiResponse;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import static com.madadipouya.eris.configuration.CacheConfiguration.EXTREME_IP_LOOKUP_CACHE;

@Service("extremeIpLookupIntegration")
public class DefaultExtremeIpLookupIntegration implements ExtremeIpLookupIntegration {

    private static final String API_URL = "http://extreme-ip-lookup.com/json/%s";

    private final RestTemplate restTemplate;

    public DefaultExtremeIpLookupIntegration(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    @Cacheable(EXTREME_IP_LOOKUP_CACHE)
    @Override
    public IpApiResponse getCoordinates(String ipAddress) {
        return restTemplate.getForObject(String.format(API_URL, ipAddress), IpApiResponse.class);
    }
}

Applying Hystrix

The last step is to use Hystrix in anywhere we can IP API and instruct the Hystrix to use the fallback under the aforementioned conditions.

If you have paid attention you know this is the only place where we really deal with Hystrix and all the prior steps are just a preparation. To use Hystrix we ought to annotate the method that calls IP API with @HystrixCommand annotation and apply the conditions in @HystrixProperty. In our case the responsible class that calls IP API is DefaultIpGeoLocation.java with the following implementation,

package com.madadipouya.eris.service.ipgeolocation.impl;

import com.madadipouya.eris.integration.fallbacks.iplookup.ExtremeIpLookupIntegration;
import com.madadipouya.eris.integration.ipapi.IpApiIntegration;
import com.madadipouya.eris.service.ipgeolocation.IpGeoLocation;
import com.madadipouya.eris.service.ipgeolocation.model.Coordinates;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;

import javax.servlet.http.HttpServletRequest;

import static com.madadipouya.eris.util.BeanUtils.copyProperties;
import static org.apache.commons.lang3.StringUtils.isBlank;
import static org.apache.commons.lang3.StringUtils.split;
import static org.apache.commons.lang3.StringUtils.trim;

@Service("ipGeoLocation")
public class DefaultIpGeoLocation implements IpGeoLocation {

    private final IpApiIntegration ipApiIntegration;

    private final ExtremeIpLookupIntegration extremeIpLookupIntegration;

    public DefaultIpGeoLocation(IpApiIntegration ipApiIntegration, ExtremeIpLookupIntegration extremeIpLookupIntegration) {
        this.ipApiIntegration = ipApiIntegration;
        this.extremeIpLookupIntegration = extremeIpLookupIntegration;
    }

    @Override
    public Coordinates getCoordinates(HttpServletRequest request) {
        return getCoordinates(getRequestIpAddress(request));
    }

    @HystrixCommand(fallbackMethod = "doCallFallbackService", commandProperties = {
            @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1000"),
            @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "30")})
    @Override
    public Coordinates getCoordinates(String ipAddress) {
        return copyProperties(ipApiIntegration.getCoordinatesFromIp(ipAddress), new Coordinates());
    }

    @Override
    public String getRequestIpAddress(HttpServletRequest request) {
        String ipAddress = getRequestIpAddressSimple(request);
        return ipAddress.contains(",") ? trim(split(trim(ipAddress), ",")[1]) : ipAddress;
    }

    @Override
    public String getRequestIpAddressSimple(HttpServletRequest request) {
        // Support Reverse Proxy
        String ipAddress = request.getHeader("X-FORWARDED-FOR");
        return isBlank(ipAddress) ? request.getRemoteAddr() : ipAddress;
    }

    @SuppressWarnings("unused")
    private Coordinates doCallFallbackService(String ipAddress) {
        return copyProperties(extremeIpLookupIntegration.getCoordinates(ipAddress), new Coordinates());
    }
}

As you can see we annotated getCoordinates method with @HystrixCommand which whenever it fails it calls the fallback method, doCallFallbackService. Another implementation could be just to close the circuit and return empty or null result.

Of course we can move the conditions to configuration but here for simplicity sake I just hard coded.

That is all for the implementation. You can find the real implementation of Spring Cloud Netflix Hystrix in Eris service at the following commits on GitHub: