Microservices With Spring Cloud

This post is to introduce Spring Cloud and Microservices. In this post, I will go over the some general concepts and patterns of Microservices. Please also see Martin Fowler’s 2014 article that sets the foundation for Microservices.

Note in this post, clients = client applications = service consumers.

Microservice Architecture

Monolithic Architecture: Single deployable software artifact.

Microservices Definition:

Microservice are a bunch of small, loosely coupled, distributed services.

Microservices allow us to take a large application and decompose it into easy-to-manage components with narrowly defined responsibilities.

A microservice architecture has the following characteristics:

Application logic is broken down into small-grained components with well-defined boundaries of responsibility.
Each component has a small domain of responsibility and is completely independent of one another. Also, a microservice should be reusable across multiple applications.
Each instance of the service should be repeatable (have same configs), and services should be easily scalable.
Microservices use lightweight communication protocols such as HTTP and JSON to exchange data between the service consumer and service provider.
The underlying technical implementation of the service is irrelevant because the applications always communicate with a technology-neutral protocol ( e.g. JSON). This means an application can be built with multiple languages and technologies.
Small development teams, and each team completely owns their service code and infrastructure.

We changed the way we build applications because:

Complexity has gone way up
Customers want faster delivery
Customers expect the app to be available all the time
Performance & Scalability

Small, Simple, and Decoupled Services = Scalable, Resilient, and Flexible Applications

The Cloud

IaaS (Infrastructure as a Service)

PaaS (Platform as a Service)

SaaS (Software as a Service)

FaaS (Functions as a Service)

CaaS (Container as a Service)

Difference among the three:

When you eat a meal, you could:

IaaS: Buy a pre-made meal, then heat up in a microwave

PaaS: Get delivery

SaaS: Eat at a restaurant

With a SaaS model, you’re a passive consumer of the service, and have no input on the technology selection or any accountability to maintain the infrastructure for the application.

In order to have the most control, we use IaaS for deployment with Docker containers.

Note: AWS EC2 is IaaS, while Elastic Beanstalk is PaaS.

Reasons for choosing IaaS:

Have the most control over the services
Massive horizontal scalability (quickly start service instances)
High redundancy through geographic distributions (AWS Multi AZs)
IaaS over PaaS: More work, but portable across multiple cloud providers (Packaged as Docker containers)

Microservice Patterns

Core development

Routing

Client resiliency

Security

Logging and tracing

Build & deploy

Core Development

Service granularity

What is the right level of responsibility the service should have?

Communication protocols (JSON)

How your client and service communicate data back and forth?

Interface design

How will you expose your service endpoints to clients?

Configuration management

How your services manage their application-specific configuration, so that the code and configuration are independent entities?

Event processing / Async messaging (Stream)

How can you use events to communicate state and data changes between services?

Spring Cloud Stream enables us to integrate lightweight messaging processing into the microservice.

Routing

Service discovery & service routing are key parts of any large-scale microservice application.

How do I get my client’s service request to a specific instance?

Routing deals with how a client application discovers the service location, and can be routed over it. Hence we need to abstract away the physical IP, and have a single entry point for service calls, so that we can consistently enforce security and content policies for all service calls.

Service discovery (Eureka)

Abstract away the physical location of the service from the client. New microservice instances can be added to scale up, and unhealthy service instances can be transparently removed from the service. Hence client app can find them having the service location hard-coded into the application.

Service routing (Zuul)

Gateway provides the microservice client a single logical URL to talk to, and acts as a policy enforcement point for things like authorization, authentication, and content checking.

Client Resiliency

Client-side load balancing (Ribbon)

How do you cache the location of your service instances on the service client, so that calls to multiple instances of a microservice are load balanced to all the healthy instances of that microservice?

Circuit breakers pattern (Hystrix)

How do you prevent a client from continuing to call a service that’s failing or suffering performance problems? We want failing microservice calls to fail fast so that the calling client can quickly respond and take an appropriate action.

Fallback pattern (Hystrix)

When a service call fails, how do you provide a “plug-in” mechanism that will allow the service client to carry out its work through alternative means other than the microservice being called?

Bulkhead pattern (Hystrix)

Microservice applications use multiple distributed resources to carry out their work. How do you compartmentalize these calls so that the mis-behavior of one service call doesn’t negatively impact the rest of the application?

Security

Authentication (OAuth2)

How do you determine the service client calling the service is who they claim they are?

Authorization(OAuth2)

How do you determine whether the service client calling a microservice is allowed to undertake the action they’re trying to undertake?

Credential management and propagation: (OAuth2 & JWT)

How do you prevent a service client from constantly having to present their credentials for service calls involved in a transaction?

Using a token-based security scheme, we can implement service authentication and authorization without passing around client credentials.

Logging & Tracing

Downside of microservices: Much more difficult to debug & trace what’s going on within the services.

A well-thought-out logging and tracing strategy makes debugging transactions across multiple services manageable.

Log correlation (Sleuth)

All service log entries have a correlation ID that ties the log entry to a single transaction.

With Sleuth, we can integrate unique tracking identifiers into the HTTP calls & message channels.

Log aggregation (Logstash & ElasticSearch)

An aggregation mechanism collects all of the logs from all the services instances. As data comes into a central data store, it is indexed and stored in a searchable format. We use correlation ID to assist in searching aggregated logs.

Microservice transaction tracing (Kibana)

The DevOps team can query the log data to find individual transactions. They should also be able to visualize the flow of all the services involved in a transaction.

Build & Deploy

Each instance of a microservice should be identical to all its other instances.

Immutable Infrastructure

Once a service is deployed, the infrastructure it’s running on should never be touched again by human hands.

We want the deployed microservice and its server to be one atomic artifact that’s deployed as a whole between environments.

Build and deployment pipeline (Jenkins)

How do you create a repeatable build and deployment process that emphasizes one-button builds and deployment to any environment in your organization?

Infrastructure as code (Docker)

How do you treat the provisioning of your services as code that can be executed and managed under source control?

When the microservice is compiled and packaged, we immediately bake and provision a virtual server or container image with the microservice installed on it.

Immutable servers (Docker)

Once a microservice image is created, how do you ensure that it’s never changed after it has been deployed?

Phoenix servers (Docker & Jenkins)

The longer a server is running, the more opportunity for configuration drift. How do you ensure that servers that run microservices get torn down on a regular basis and recreated off an immutable image?

Spring Cloud

Spring Cloud is a collection of open source technologies from companies such as Netflix and HashiCorp that have been “wrapped” with Spring annotations to significantly simplify the setup and configuration of these services.

No industry standards exist for microservices yet.

Microservices take a principle-based approach and align with the concepts of REST and JSON .

Spring Cloud Config

Spring Cloud Config handles the management of application configuration data through a centralized service, so your application configuration data (particularly your environment specific configuration data) is cleanly separated from your deployed microservice. Spring Cloud can integrate with:

Git
Consul (open-source service discovery)
Eureka

Spring Cloud Service Discovery

With Spring Cloud service discovery, we can abstract away the physical location ( IP and/or server name) of where your servers are deployed from the clients. Service consumers invoke business logic for the servers through a logical name rather than a physical location.

Example

A normal controller:

@RestController
@RequestMapping(value="old")
public class OldController {

    @GetMapping(value="/{first}/{last}")
    public String hello(@PathVariable("first") String first, @PathVariable("last") String last){
        return "Hello " + first + " " + last;
    }
}

Upgraded controller with service discovery, circuit breaker, bulkhead & client-side load-balancing.

@RestController

// Enable it to use Hystrix & Ribbon
@EnableCircuitBreaker

// Register with Eureka service discovery agent
@EnableEurekaClient
public class SampleController {
    
    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate(); }
    
    @Autowired
    public RestTemplate restTemplate;

    @HystrixCommand(threadPoolKey = "cloudThreadPool")
    public String helloRemoteServiceCall(String first, String last) {
        
        ResponseEntity<String> restExchange = restTemplate.exchange(
            "http://logical-service-id/name/[ca]{first}/{last}",
            HttpMethod.GET, null, String.class, first, last);
        
        return restExchange.getBody();
    }

    @GetMapping(value="/{first}/{last}")
    public String hello(@PathVariable("first") String first, @PathVariable("last") String last){
        return helloRemoteServiceCall(first, last);
    }
}

Debug: Could not autowire RestTemplate:

First need to create the Bean with @Bean, then autowire it.

@HystrixCommand:

This annotation create a thread pool called cloudThreadPool that’s managed by Hystrix. All calls to helloRemoteServiceCall will only occur in this thread pool, and will be isolated from other remote services.
Circuit Breaker: Anytime helloRemoteServiceCall is called, it won’t be directly invoked. Instead, the method will be delegated to a thread pool managed by Hystrix. If the call takes too long, Hystrix will interrupt it.

@EnableEurekaClient & RestTemplate:

@EnableEurekaClient tells Spring Boot that we’re going to use a modified RestTemplate class when we make REST service calls. This RestTemplate class allows us to pass in a logical service ID for the service we’re trying to invoke.
RestTemplate will contact the Eureka service and look up the physical location of the service instances.
RestTemplate uses Netflix’s Ribbon. Ribbon will retrieve a list of all physical endpoints associated with a service. Every time the service is called by the client, it “round-robins” the call to the different service instances on the client without having to go through a centralized load balancer.
By eliminating a centralized load balancer and moving it to the client, we eliminate another single point of failure in the application infrastructure.

Ribbon (client-side load balancing) replaces the need of using a centralized load balancer.