Overview
Architecture
JLupin Platform provides distributed software load balancers to ensure scale-out architecture of the whole environment (including platform components). At least one instance of JLupin Software Load Balancer is present on Main Server and every servlet and native microservice that is configured to communicate with other microservices on the platform.
Notice that every request between microservices on JLupin Platform goes through two instance of JLupin Software Balancer:
- the first one, located on the microservice that creates a request (1)
- the second one, included in Main Server, chosen by the previous balancer (2).
This cascade configuration of load balancers gives opportunity to provide the highest reliability of the request's processing. Even after sending the request from the first balancer and the sudden failure of microservice on the target node, the second balancer, located on that node, is still able to redirect the traffic to another node. This protects the communication against microservice failures between service discovery iterations.
All instances of JLupin Software Balancers performs periodic service discovery to update their service repository used in the load balancing process (to get to know more about the service repository & discovery go to this chapter). The algorithms used by load balancers in the process of choosing a target node is descried in next chapters. After the target node is chosen, the load balancer tries to establish a connection and sent the request. If it is not succeeded it tries singleRequestRepeatsAmount
(the parameter in main.yml
, discussed below) times to find another node from NODE PEERS
list (in main.yml
, discussed below) to handle the request. The singleRequestRepeatsAmount
is set to 3 by default and could be overwritten by repeatsAmount
while JLupin Software Load Balancer is created (see details in JavaDocs - JLupinClientUtil specification).
The instance located on Main Server performs an additional task - it periodically checks the health of controlled microservices, basic on the results it determines their status and update its internal service repository. More detailed description can be found in the next chapter.
Load balancing
There are two built-in load balancing algorithms:
- PRIORITY - the default algorithm for inner microservice communication (including Main Server to microservices relation).
- ROUND_ROBIN - the default algorithm for external clients that uses jlupin-client-assembly library to communicate with JLupin environment.
You can change the default algorithm during load balancer initialization (in code) by dividing this task into two steps:
- create
JLupinLoadBalancer
usinggenerateLoadBalancer
method inJLupinClientUtil
class, wherejLupinLoadBalancerAlgorithmType
input parameter can be set accordingly. Then - create
JLupinDelegator
using previousle createdJLupinLoadBalancer
as one of the input parameter.
NOTE that algorithm for Main Server to microservices communication is hardcoded as PRIORITY type due to architectural assumptions and cannot be changed.
If you would like to get to know more about custom load balancers configuration please refer to Java Docs.
PRIORITY balancing
The priority balancer always takes the shortest connection path to determine the target node. In practice, it means that priority balancer always first looks for the target microservice on the same node. This algorithm is presented on the following figure on the example of communication between WebApp1
and Microservice A
microservice.
The communication schema in normal operation:
The communication schema in case of Microservice A
failure:
ROUND_ROBIN balancing
The round robin balancer tries to equally distribute traffic across available nodes. It takes the first node from the list and after changeServerIntervalInMillis
milliseconds (input parameter of JLupinDelegator
available in JLupinClientUtil
class) switches traffic to the next one, as show on the following pictures.
The communication schema the initial state of balancing:
The communication schema after changeServerIntervalInMillis
milliseconds:
Configuration
Overview
JLupin Load Balancer is configured by parameters located in main.yml file as well as some sections provided by microservice configuration files (servlet_configuration.yml / configuration.yml) as shown on the following picture.
Detailed description of those parameters can be found below (main.yml) and in chapters about native microservices and servlet microservices
Configuration files (main.yml)
[...]
SERVICE_REPOSITORY_SETTINGS:
howOftenCheckingMicroservicesInMillis: 5000
externalNodePeersStartCheckDelayInMillis: 5000
SOCKET_PROPERTIES_TO_CONNECT_WITH_PEERS:
connectionTimeoutInMillis: 1000
readTimeoutInMillis: 90000
isKeepAlive: false
isOOBInline: false
isTcpNoDelay: false
isReuseAddress: false
sendBufferSize: 0
receiveBufferSize: 0
soLinger: 0
trafficClass: 0
LOAD_BALANCER_SETTINGS:
JRMC_PROTOCOL_SETTINGS:
ALGORITHM:
type: priority
PROPERTIES:
changeServerIntervalInMillis: 5000
repeatsAmount: 3
QUEUE_PROTOCOL_SETTINGS:
ALGORITHM:
type: priority
PROPERTIES:
changeServerIntervalInMillis: 5000
repeatsAmount: 3
[...]
Section 'SERVICE_REPOSITORY_SETTINGS'
Parameter | Description |
---|---|
singleRequestRepeatsAmount | The number of attempts that load balancer will take to process the request to microservice |
howOftenCheckingMicroservicesInMillis | The health checking period and service discovery process for Main Server's load balancer, expressed in milliseconds |
externalHealthCheckPeriod | This parameter SHOULD be set to health checking period (expressed in milliseconds) of the external balancer (if it is used) to keep servlet microservices' restart online. |
externalNodePeersStartCheckDelayInMillis | The delay expressed in milliseconds aftre wich load balancer starts checking peers and collecting infromation from them (zone). |
Parameter | Description |
---|---|
howOftenCheckingMicroservicesInMillis | The health checking period and service discovery process for Main Server's load balancer, expressed in milliseconds |
externalNodePeersStartCheckDelayInMillis | The delay expressed in milliseconds aftre wich load balancer starts checking peers and collecting infromation from them (zone). |
Subsection 'SOCKET_PROPERTIES_TO_CONNECT_WITH_PEERS'
This subsection refers to load balancers located on the microservices and Main Server, it describes connections' parameters initialized by this balancers to other jBricks:
Parameter | Description |
---|---|
connectionTimeoutInMillis | The maximum time in milliseconds that a client's socket is waiting for connection is established, after this timeout the socket is closed. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
readTimeoutInMillis | The maximum time in milliseconds that a client's socket is waiting for the response, after this timeout the socket is closed. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
isKeepAlive | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
isOOBInline | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
isTcpNoDelay | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
isReuseAddress | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
sendBufferSize | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
receiveBufferSize | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
soLinger | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
trafficClass | One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION ! |
Section 'LOAD_BALANCER_SETTINGS'
This section defines configuration for two protocols: JLRMC and Queue. Each of them has same conifguration described below.
Parameter | Description |
---|---|
ALGORITHM -> type | Type of algorithm used by load balancer. One of: priority , roundRobin , localhost . |
PROPERTIES -> changeServerIntervalInMillis | This is parameter used by roundRobin algorithm. It defines how often change server. |
PROPERTIES -> repeatsAmount | This parameter defines how many times balancer tries to execute request. |
Section 'NODE PEERS'
[...]
NODE_PEERS:
NODE_1:
ip: 'localhost'
jlrmcPort: 9090
queuePort: 9095
transmissionPort: 9096
informationPort: 9097
#NODE_2:
#ip: '127.0.0.1'
#jlrmcPort: 19090
#queuePort: 19095
#transmissionPort: 19096
#informationPort: 19097
[...]
This section is the list of other Main Servers (jBricks) that the Main Server and its microservices will connect to. This section could be managed manually by editing main.yml, but more effective is doing it by local console (node peer add|remove
) or JLupin Control Center, where jBricks are managed in zones.
Each entry is described by the following set of parameters.
Parameter | Description |
---|---|
ip | IP address of Main Server. |
jlrmcPort | The port number of Main Server's JLRMC entry port. |
queuePort | The port number of Main Server's QUEUE entry port. |
transmissionPort | The port number of Main Server's TRANSMISSION entry port. |
informationPort | The port number of Main Server's INFORMATION entry port. |
This list is automatically inherited by load balancers of microservices managed by the Main Server.
The default entry in this section is 'localhost' (127.0.0.1) which enables communication within local jBrick.
RATE & DISCUSS (0)