Overview

Architecture

JLupin Platform provides distributed software load balancers to ensure scale-out architecture of the whole environment (including platform components). At least one instance of JLupin Software Load Balancer is present on Main Server and every servlet and native microservice that is configured to communicate with other microservices on the platform.

Notice that every request between microservices on JLupin Platform goes through two instance of JLupin Software Balancer:

• the first one, located on the microservice that creates a request (1)
• the second one, included in Main Server, chosen by the previous balancer (2).

This cascade configuration of load balancers gives opportunity to provide the highest reliability of the request's processing. Even after sending the request from the first balancer and the sudden failure of microservice on the target node, the second balancer, located on that node, is still able to redirect the traffic to another node. This protects the communication against microservice failures between service discovery iterations.

All instances of JLupin Software Balancers performs periodic service discovery to update their service repository used in the load balancing process (to get to know more about the service repository & discovery go to this chapter). The algorithms used by load balancers in the process of choosing a target node is descried in next chapters. After the target node is chosen, the load balancer tries to establish a connection and sent the request. If it is not succeeded it tries singleRequestRepeatsAmount (the parameter in main.yml, discussed below) times to find another node from NODE PEERS list (in main.yml, discussed below) to handle the request. The singleRequestRepeatsAmount is set to 3 by default and could be overwritten by repeatsAmount while JLupin Software Load Balancer is created (see details in JavaDocs - JLupinClientUtil specification).

The instance located on Main Server performs an additional task - it periodically checks the health of controlled microservices, basic on the results it determines their status and update its internal service repository. More detailed description can be found in the next chapter.

Load balancing

There are two built-in load balancing algorithms:

• PRIORITY - the default algorithm for inner microservice communication (including Main Server to microservices relation).
• ROUND_ROBIN - the default algorithm for external clients that uses jlupin-client-assembly library to communicate with JLupin environment.

You can change the default algorithm during load balancer initialization (in code) by dividing this task into two steps:

• create JLupinLoadBalancer using generateLoadBalancer method in JLupinClientUtil class, where jLupinLoadBalancerAlgorithmType input parameter can be set accordingly. Then
• create JLupinDelegator using previousle created JLupinLoadBalancer as one of the input parameter.

NOTE that algorithm for Main Server to microservices communication is hardcoded as PRIORITY type due to architectural assumptions and cannot be changed.

If you would like to get to know more about custom load balancers configuration please refer to Java Docs.

PRIORITY balancing

The priority balancer always takes the shortest connection path to determine the target node. In practice, it means that priority balancer always first looks for the target microservice on the same node. This algorithm is presented on the following figure on the example of communication between WebApp1 and Microservice A microservice.

The communication schema in normal operation:

The communication schema in case of Microservice A failure:

ROUND_ROBIN balancing

The round robin balancer tries to equally distribute traffic across available nodes. It takes the first node from the list and after changeServerIntervalInMillis milliseconds (input parameter of JLupinDelegator available in JLupinClientUtil class) switches traffic to the next one, as show on the following pictures.

The communication schema the initial state of balancing:

The communication schema after changeServerIntervalInMillis milliseconds:

Configuration

Overview

JLupin Load Balancer is configured by parameters located in main.yml file as well as some sections provided by microservice configuration files (servlet_configuration.yml / configuration.yml) as shown on the following picture.

Detailed description of those parameters can be found below (main.yml) and in chapters about native microservices and servlet microservices

Configuration files (main.yml)

[...]
  SERVICE_REPOSITORY_SETTINGS:
    howOftenCheckingMicroservicesInMillis: 5000
    externalNodePeersStartCheckDelayInMillis: 5000
    SOCKET_PROPERTIES_TO_CONNECT_WITH_PEERS:
      connectionTimeoutInMillis: 1000
      readTimeoutInMillis: 90000
      isKeepAlive: false
      isOOBInline: false
      isTcpNoDelay: false
      isReuseAddress: false
      sendBufferSize: 0
      receiveBufferSize: 0
      soLinger: 0
      trafficClass: 0
  LOAD_BALANCER_SETTINGS:
    JRMC_PROTOCOL_SETTINGS:
      ALGORITHM:
        type: priority
      PROPERTIES:
        changeServerIntervalInMillis: 5000
        repeatsAmount: 3
    QUEUE_PROTOCOL_SETTINGS:
      ALGORITHM:
        type: priority
      PROPERTIES:
        changeServerIntervalInMillis: 5000
        repeatsAmount: 3
[...]

Section 'SERVICE_REPOSITORY_SETTINGS'

Subsection 'SOCKET_PROPERTIES_TO_CONNECT_WITH_PEERS'

This subsection refers to load balancers located on the microservices and Main Server, it describes connections' parameters initialized by this balancers to other jBricks:

Section 'LOAD_BALANCER_SETTINGS'

This section defines configuration for two protocols: JLRMC and Queue. Each of them has same conifguration described below.

Section 'NODE PEERS'

[...]
NODE_PEERS:
  NODE_1:
    ip: 'localhost'
    jlrmcPort: 9090
    queuePort: 9095
    transmissionPort: 9096
    informationPort: 9097
  #NODE_2:
    #ip: '127.0.0.1'
    #jlrmcPort: 19090
    #queuePort: 19095
    #transmissionPort: 19096
    #informationPort: 19097
[...]

This section is the list of other Main Servers (jBricks) that the Main Server and its microservices will connect to. This section could be managed manually by editing main.yml, but more effective is doing it by local console (node peer add|remove) or JLupin Control Center, where jBricks are managed in zones.

Each entry is described by the following set of parameters.

This list is automatically inherited by load balancers of microservices managed by the Main Server.

The default entry in this section is 'localhost' (127.0.0.1) which enables communication within local jBrick.