Software load balancers are the software for network traffic routing management for particular nodes. Their usage completely replaces hardware load balancers that require high costs and changes in equipment/network infrastructure. In its distribution JLupin includes a group of software load balancers. Each of them offers a defined algorithm that is responsible for determination of destination of a given query that must go to a given node on which JLupin Next Server is installed and activated. Load balancers provided by JLupin are not a separate component activated somewhere between the client and the server. They are part of the client application written in JAVA language.

It is illustrated by the following figure:

Load balancers of JLupin support code calling on the side of the server via JLupin Java Remote Object Invocation. To use load balancers on the side of the client and to call functions on the side of the server, files described in Maven Dependencies section should be placed on the side of the client (e.g. inside war files that are then activated on Tomcat servers). And now, let’s use an example based on which we will discuss particular implementations of load balancers. Before reading below information, it is recommended to read Create Microservice and Microservice Invocation sections.

The below example shows a calling of "digestService" service described in Create Microservice section. Every load balancer provided by JLupin implements the below interface:

package com.jlupin.interfaces.client.balancer;

import com.jlupin.interfaces.client.balancer.exception.JLupinBalancerException;
import com.jlupin.interfaces.client.balancer.exception.JLupinBalancerNotConnectedException;
import com.jlupin.interfaces.common.to.JLupinInputParameter;
import com.jlupin.interfaces.common.to.JLupinOutputParameter;

public interface JLupinBalancer {
       JLupinOutputParameter sendByBalanceAlgorithm(JLupinInputParameter jLupinInputParameter) throws JLupinBalancerException, JLupinBalancerNotConnectedException;
       void after()  throws JLupinBalancerException;
       void before() throws JLupinBalancerException;
}

Let’s start from the basic implementation: JLupinDefaultAlgSocketBalancerImpl is the easiest implementation of balance algorithm. Traffic is routed in the closed cycle to the every next server node. If connection fails, other available JLupin Next Server nodes will be searched for. In the cycle of calling to another available server, algorithm omits servers that were marked as unavailable by a loop that controls available servers.

Examples of use:

package com.jlupin.test.invoke;

import com.jlupin.impl.client.balancer.JLupinDefaultAlgSocketBalancerImpl;
import com.jlupin.impl.client.delegator.JLupinDefaultBalanceProxyDelegatorImpl;
import com.jlupin.impl.client.proxy.producer.JLupinDefaultProxyObjectProducerImpl;
import com.jlupin.impl.client.util.JLupinClientSocketConfiguration;
import com.jlupin.impl.logger.impl.simple.stdout.JLupinSimpleSystemOutLoggerImpl;
import com.jlupin.impl.serialize.JLupinFSTSerializerImpl;
import com.jlupin.interfaces.client.balancer.JLupinBalancer;
import com.jlupin.interfaces.client.delegator.JLupinDelegator;
import com.jlupin.interfaces.client.proxy.producer.JLupinProxyObjectProducer;
import com.jlupin.interfaces.common.to.JLupinInputParameter;
import com.jlupin.interfaces.logger.JLupinLogger;
import com.jlupin.interfaces.logger.enums.LogMode;
import com.jlupin.interfaces.serialize.JLupinSerializer;
import com.jlupin.sampleapp.paramsfunction.service.DigestService;
import org.junit.Before;

import java.net.InetSocketAddress;

public class InvokeByJLupinDefaultAlgSocketBalancerImplTest {

    private JLupinLogger jLupinLogger;
    private JLupinProxyObjectProducer jLupinProxyObjectProducer;

    @Before
    public void before() {
        JLupinSimpleSystemOutLoggerImpl.initializeWithLogMode(LogMode.DEBUG);
        jLupinLogger     = JLupinSimpleSystemOutLoggerImpl.getInstance();

        JLupinSerializer jLupinSerializer = JLupinFSTSerializerImpl.getInstance();

        JLupinClientSocketConfiguration clientSocketConfServer_1 = new JLupinClientSocketConfiguration();

        clientSocketConfServer_1.setSocketAddressToConnectServer(new InetSocketAddress("198.162.8.3", 8083));
        clientSocketConfServer_1.setConnectionTimeout(1000);
        clientSocketConfServer_1.setReadTimeout(30000);

        JLupinClientSocketConfiguration clientSocketConfServer_2 = new JLupinClientSocketConfiguration();

        clientSocketConfServer_2.setSocketAddressToConnectServer(new InetSocketAddress("198.162.9.3", 8083));
        clientSocketConfServer_2.setConnectionTimeout(1000);
        clientSocketConfServer_2.setReadTimeout(30000);

        JLupinBalancer jLupinBalancer = new JLupinDefaultAlgSocketBalancerImpl(2, jLupinLogger,jLupinSerializer,1000, clientSocketConfServer_1,clientSocketConfServer_2);
        JLupinDelegator jLupinDelegator = new JLupinDefaultBalanceProxyDelegatorImpl(jLupinBalancer);

        jLupinProxyObjectProducer = new JLupinDefaultProxyObjectProducerImpl("sampleMicroservice","testClientApplication",jLupinDelegator, jLupinLogger);
    }

    @org.junit.Test
    public void test() throws Throwable {
        DigestService digestService = produceDigestService();
        String digest = digestService.getMD5Digest("name", "surname");
        jLupinLogger.info("digest:" + digest);
    }

    private DigestService produceDigestService() throws Throwable {
        JLupinInputParameter jLupinInputParameter = new JLupinInputParameter();
        jLupinInputParameter.setServiceName("digestService");
        return jLupinProxyObjectProducer.produceObject(DigestService.class, jLupinInputParameter);
    }

}

Let’s take a look at before() method. For this example, we assume that we have two nodes at 198.162.8.3 and 198.162.9.3. We prepare two objects clientSocketConfServer_1 and clientSocketConfServer_2 in which we provide the details of connections for every node: ip, port, readTimeout, connectionTimeout. Let’s take a look at JLupinClientSocketConfiguration class fields – all are available by getters and setters:

public class JLupinClientSocketConfiguration {

    private SocketAddress socketAddressToConnectServer;
    private SocketAddress socketAddressToBindLocalAddress;
    private Proxy proxy;
    private SocketImpl socketImpl;
    private boolean       isKeepAlive       = false;
    private boolean       isOOBInline       = false;
    private boolean       isTcpNoDelay      = false;
    private boolean       isReuseAddress    = false;
    private int           sendBufferSize    = 0;
    private int           receiveBufferSize = 0;
    private int           soLinger          = 0;
    private int           readTimeout       = 30000;
    private int           trafficClass      = 0;
    private int           connectionTimeout = 1000;

In the next step we build JlupinDefaultAlgSocketBalancerImpl object. Let’s take a look at the constructor:

public JLupinDefaultAlgSocketBalancerImpl(int howManyTryToConnectServerByRequest, JLupinLogger jLupinLogger,
                                          JLupinSerializer jLupinSerializer,  int controlIntervalInMillis, JLupinClientSocketConfiguration...jLupinClientSocketConfigurations)

howManyTryToConnectServerByRequest – it defines number of repetitions made by the algorithm establishing active connections to the server. Precisely, it will be the following value: number of nodes * howManyTryToConnectServerByRequest in the worst scenario before a request is rejected due to unavailable servers.
jLupinLogger – logger
jLupinSerializer – serializer
controlIntervalInMillis – defines interval based on which available server nodes are checked by JLupinBaseBalancer
jLupinClientSocketConfigurations – table of JLupinClientSocketConfigurations objects defining details of connection to a given server node.

Another load balancer implementation is as follows: JLupinOutputParameterAnalyzerAlgSocketBalancerImpl – extends JLupinDefaultAlgSocketBalancerImpl class and is based on its algorithm of server index determination to which a request should be directed. It however has a mechanism that allows analysis of JLupinOutputParameter, and in case of unavailability of a given application (confirmed by the particular main server of a given node), it allows routing the traffic to the first node on which a given application answers properly.

The below example presents how to use it, before() method again:

@Before
public void before() {
    JLupinSimpleSystemOutLoggerImpl.initializeWithLogMode(LogMode.DEBUG);
    jLupinLogger     = JLupinSimpleSystemOutLoggerImpl.getInstance();

    JLupinSerializer jLupinSerializer = JLupinFSTSerializerImpl.getInstance();

    JLupinClientSocketConfiguration clientSocketConfServer_1 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_1.setSocketAddressToConnectServer(new InetSocketAddress("198.162.8.3", 8083));
    clientSocketConfServer_1.setConnectionTimeout(1000);
    clientSocketConfServer_1.setReadTimeout(30000);

    JLupinClientSocketConfiguration clientSocketConfServer_2 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_2.setSocketAddressToConnectServer(new InetSocketAddress("198.162.9.3", 8083));
    clientSocketConfServer_2.setConnectionTimeout(1000);
    clientSocketConfServer_2.setReadTimeout(30000);

    JLupinBalancer jLupinBalancer = new JLupinOutputParameterAnalyzerAlgSocketBalancerImpl(2, jLupinLogger,jLupinSerializer,1000, clientSocketConfServer_1,clientSocketConfServer_2);
    JLupinDelegator jLupinDelegator = new JLupinDefaultBalanceProxyDelegatorImpl(jLupinBalancer);

    jLupinProxyObjectProducer = new JLupinDefaultProxyObjectProducerImpl("sampleMicroservice","testClientApplication",jLupinDelegator, jLupinLogger);
}

As shown, only load balancer class was replaced – all other parameters seem to be unchanged.

Let’s move on to another load balancer implementation:

JLupinPrioritySocketBalancerImpl – this load balancer implementation allows reducing network traffic between many JLupin nodes. This means that the traffic will always be routed to the first provided address at the time of its unavailability (e.g. as a result of overload, failure). Traffic will be routed to another node from configuration, before() method shows it again:

@Before
public void before() {
    JLupinSimpleSystemOutLoggerImpl.initializeWithLogMode(LogMode.DEBUG);
    jLupinLogger     = JLupinSimpleSystemOutLoggerImpl.getInstance();

    JLupinSerializer jLupinSerializer = JLupinFSTSerializerImpl.getInstance();

    JLupinClientSocketConfiguration clientSocketConfServer_1 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_1.setSocketAddressToConnectServer(new InetSocketAddress("198.162.8.3", 8083));
    clientSocketConfServer_1.setConnectionTimeout(1000);
    clientSocketConfServer_1.setReadTimeout(30000);

    JLupinClientSocketConfiguration clientSocketConfServer_2 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_2.setSocketAddressToConnectServer(new InetSocketAddress("198.162.9.3", 8083));
    clientSocketConfServer_2.setConnectionTimeout(1000);
    clientSocketConfServer_2.setReadTimeout(30000);

    JLupinBalancer jLupinBalancer = new JLupinPrioritySocketBalancerImpl(2, jLupinLogger,jLupinSerializer,1000, clientSocketConfServer_1,clientSocketConfServer_2);
    JLupinDelegator jLupinDelegator = new JLupinDefaultBalanceProxyDelegatorImpl(jLupinBalancer);

    jLupinProxyObjectProducer = new JLupinDefaultProxyObjectProducerImpl("sampleMicroservice","testClientApplication",jLupinDelegator, jLupinLogger);
}

As shown, only load balancer class was replaced – all other parameters seem to be unchanged.

Let’s move on to another load balancer implementation:

JLupinHealthCheckingSocketBalancerImpl extends JLupinOutputParameterAnalyzerAlgSocketBalancerImpl class, therefore it has the functionality of all above mentioned balancers. However, the main function of this balancer is to assess parameters defining overload of a given server node and the traffic management in the most optimal way from the point of view of overload of a given node and network traffic. Contrary to previous ones, this balancer does not check algorithms but only main server. Particular microservice is a separate operational system process (separate JVM). Let’s check an example – before() method again:

@Before
public void before() {
    JLupinSimpleSystemOutLoggerImpl.initializeWithLogMode(LogMode.DEBUG);
    jLupinLogger     = JLupinSimpleSystemOutLoggerImpl.getInstance();

    JLupinSerializer jLupinSerializer = JLupinFSTSerializerImpl.getInstance();

    JLupinClientSocketConfiguration clientSocketConfServer_1 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_1.setSocketAddressToConnectServer(new InetSocketAddress("198.162.8.3", 8083));
    clientSocketConfServer_1.setConnectionTimeout(1000);
    clientSocketConfServer_1.setReadTimeout(30000);

    JLupinClientSocketConfiguration clientSocketConfServer_2 = new JLupinClientSocketConfiguration();

    clientSocketConfServer_2.setSocketAddressToConnectServer(new InetSocketAddress("198.162.9.3", 8083));
    clientSocketConfServer_2.setConnectionTimeout(1000);
    clientSocketConfServer_2.setReadTimeout(30000);

    JLupinBalancer jLupinBalancer = new JLupinHealthCheckingSocketBalancerImpl(2, jLupinLogger,jLupinSerializer,
                                        "sampleMicroservice",1000, clientSocketConfServer_1,clientSocketConfServer_2);
    JLupinDelegator jLupinDelegator = new JLupinDefaultBalanceProxyDelegatorImpl(jLupinBalancer);

    jLupinProxyObjectProducer = new JLupinDefaultProxyObjectProducerImpl("sampleMicroservice","testClientApplication",jLupinDelegator, jLupinLogger);
}

As shown, JLupinHealthCheckingSocketBalancerImpl class constructor differs from other constructors presented before with only one variable, which is microserviceName defining the name of microservice that should be examined by a balancer with regard to its efficiency.

Important note – algorithm based on JLupinHealthCheckingSocketBalancerImpl requires additional configuration and creation of the service on the side of a given microservice that will provide information to the load balancer on microservice actual overload.

Here’s the instruction concerning activation of health checking on the side of a given microservice:

  1. Download jlupin-service-healthchecking-1.2.0.jar
  2. Put downloaded jar into microservice (you need to health check – in best way put and configure health checking in all microservices directory serverdirectory/application/microservice_name
  3. Add Spring configuration file (which is part of jar above) to application configuration JAVA file in public JLupinApplicationContainer getJLupinApplicationContainer() method .

Example show below.

@Override
public JLupinApplicationContainer getJLupinApplicationContainer() {

       JLupinSingleApplicationContainerForLocalProcessImpl jLupinSingleApplicationContainerForLocalProcess =
                                                            new JLupinSingleApplicationContainerForLocalProcessImpl(
                                                                new String[] {"classpath:spring-services.xml","classpath:spring-health-checking-services.xml"});
       return jLupinSingleApplicationContainerForLocalProcess;
}
  1. Restart application by command on remote console: appRestart $application_name

If health checking module is installed on a given microservice, via external network devices to (e.g. external load balancers) we can ask a microservice about its status. Invoke Health Checking by HTTP-XML Web Service Entry Point. Set in client tier address to WebService Entry Point (in default released with server JLupinWebserviceMultiProcessEntryPointImpl and default listen on port 8082). Example correct address to Web Service Entry Point: http://localhost:8082/jLupinNextServerWebService/JLupinWebservice

Message format must be as follow:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ent="http://entrypoint.impl.jlupin.org/">
   <soapenv:Header/>
   <soapenv:Body>
      <ent:jLupinService>
         <jLupinInputParameter>
         <applicationName>firstSampleApplication</applicationName>
           <locale>en</locale>
            <methodName>check</methodName>
              <privilegeList>
               <privilegeName></privilegeName>
            </privilegeList>
            <requestId>req1</requestId>
            <serviceName>healthCheckingService</serviceName>
            <sessionId>sess1</sessionId>
            <user>test_user</user>
            <sequenceName>sampleParamArrayXmlInOutSequence</sequenceName>
         </jLupinInputParameter>
     </ent:jLupinService>
   </soapenv:Body>
</soapenv:Envelope>

Health checking response:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
  <env:Header/>
   <env:Body>
     <jlns:jLupinServiceResponse xmlns:jlns="http://entrypoint.impl.jlupin.org/">
            <jLupinOutputParameter>
                <executedServiceError>false</executedServiceError>
                <result>
                 <![CDATA[<?xml version="1.0" encoding="UTF-8"?><java version="1.8.0_45" class="java.beans.XMLDecoder">
                 <object class="com.jlupin.interfaces.common.to.JLupinServiceDTO" id="JLupinServiceDTO0">
                  <void property="objectMap">
                   <void method="put">
                    <string>total_memory</string>
                    <string>253755392</string>
                   </void>
                   <void method="put">
                    <string>max_threads</string>
                    <string>64</string>
                   </void>
                   <void method="put">
                    <string>free_memory</string>
                    <string>127367088</string>
                   </void>
                   <void method="put">
                    <string>active_threads</string>
                    <string>1</string>
                   </void>
                   <void method="put">
                    <string>max_memory</string>
                    <string>253755392</string>
                   </void>
                  </void>
                 </object>]]>
                </result>
            </jLupinOutputParameter>
        </jlns:jLupinServiceResponse>
    </env:Body>
</env:Envelope>

Parameters:
total_memory, free_memory, max_memory are given in bytes

Invoke Health Checking by HTTP-JSON Web Service entry point:

URL: http://localhost:8083/jLupinNextServerRestService/JLupinRestservice

Request:

{
  "jLupinInputParameter": {
    "applicationName":"firstSampleApplication",
    "locale": "en",
    "methodName": "check",
    "privilegeList": [ "privilege_1", "privilege_2" ],
    "requestId": "request_1",
    "serviceName": "healthCheckingService",
    "sequenceName":"sampleParamArrayJsonInOutSequence",
    "sessionId": "session_1",
    "user": "test_user",
  }
}

Response:

{"jLupinOutputParameter":
  {"result": "{"createdDTOTime":"2016-03-24 16:02:49",
               "objectMap":{"total_memory":"253755392","max_threads":"64",
               "free_memory":"121771136","active_threads":"1","max_memory":"253755392"}}",
   "executedServiceError": false
  }
}