Java Concurrency with Load Balancer Simulation
Load balancer are the proxy systems that resides in front of a network and distributes the network requests across multiple servers. The aim of using load balancing is sharing the amount of requests between replica servers and making the overall processing more efficient, so that non of the servers would be overwhelmed trying to deal with many requests.
I find it as a handy example to practice java concurrency and synchronisation. I will share an implementation of LoadBalancer simulator thats should provide an IP address from the IP address pool when a client hits. Imagine that we have 10 identical servers running our application and we want to distribute the load to among them for each client tries to reach. So load balancer should provide addresses fairly to the clients who want to access our app.
The LoadBalancer is represented by an abstract class with getIp() abstract method.
public abstract class LoadBalancer {
final List <String> ipList;
public LoadBalancer(List <String> ipList) {
this.ipList = Collections.unmodifiableList(ipList);
}
abstract String getIp();
}We can implement different algorithms in the load balancer to regarding the choosing IP addresses. For instance it can choose an IP by:
- Picking randomly from the pool
- Using
RoundRobinscheduling algorithm which provides resources to in an ordered way - Using a weight per resource with
WeightedRoundRobinalgorithm
Random Load Balancing
public class RandomLoadBalancer extends LoadBalancer {
public RandomLoadBalancer(List <String> ipList) {
super(ipList);
}
@Override
public String getIp() {
Random random = new Random();
return ipList.get(random.nextInt(ipList.size()));
}
}It simply chooses next random IP from the pool and retrieves to the client. We can consider this implementation as thread safe, because any concurrent get call makes only read operation from the list. So no locking is needed.
Round Robin Load Balancing
public class RoundRobinLoadBalancer extends LoadBalancer {
private int counter = 0;
private final ReentrantLock lock;
public RoundRobinLoadBalancer(List <String> list) {
super(list);
lock = new ReentrantLock();
}
@Override
public String getIp() {
lock.lock();
try {
String ip = ipList.get(counter);
counter += 1;
if (counter == ipList.size()) {
counter = 0;
}
return ip;
} finally {
lock.unlock();
}
}
}This class also extends the LoadBalancer and implements the getIp() method. The method simply chooses the next IP that counter points, so that it gives in an order as round robin requires. This is a simplified approach of RR but I prefer to stick to the subject.
In this implementation we needed a shared counter variable that can be updated by multiple threads, so we used ReentrantLock. It will guarantee that no thread can update the counter when one is in the critical section.
Weighted Round Robin Load Balancing
In Weighted RR each resource should have a so-called weight, that designates the portion of capacity. So if we have 2 servers with same weight, say 10, both will receive same load. But having 2 servers with the weights 5 and 10, then we would expect the second one to be accessed twice more than first one. So we can consider it as ratio.
This implementation needs a map for IP-weight coupling. And our abstract class LoadBalancer accepts List in the constructor. So I tried to comply with SOLID principles and I also did not want to repeat myself implementing the same getIp method for both RRs.
public class WeightedRoundRobinLoadBalancer extends RoundRobinLoadBalancer {
public WeightedRoundRobinLoadBalancer(Map <String, Integer> ipMap) {
super(
ipMap.keySet()
.stream()
.map(ip -> {
List<String> tempList = new LinkedList<>();
for (int i=0; i<ipMap.get(ip); i++) {
tempList.add(ip);
}
return tempList;
})
.flatMap(Collection::stream)
.collect(Collectors.toList())
);
}
}So this class extends the RoundRobinLoadBalancer to use the same getIp method. And also it delivers an IP list to its super constructor by converting it from map. This conversion may look odd, but it adds weight times IP to the pool so we make sure that WRR will be applied.
I have a Client class to make requests to LoadBalancers to make concurrent requests with parallel streams.
public class Client {
public static void main(String[] args) {
int NUM_OF_REQUESTS = 15;
Client client = new Client();
ArrayList <String> ipPool = new ArrayList <>();
ipPool.add("192.168.0.1");
ipPool.add("192.168.0.2");
ipPool.add("192.168.0.3");
ipPool.add("192.168.0.4");
ipPool.add("192.168.0.5");
Map <String, Integer> ipPoolWeighted = new HashMap<>();
ipPoolWeighted.put("192.168.0.1", 6);
ipPoolWeighted.put("192.168.0.2", 6);
ipPoolWeighted.put("192.168.0.3", 3);
client.printNextTurn("Random");
LoadBalancer random = new RandomLoadBalancer(ipPool);
client.simulateConcurrentClientRequest(random, NUM_OF_REQUESTS);
client.printNextTurn("Round-Robin");
LoadBalancer roundRobbin = new RoundRobinLoadBalancer(ipPool);
client.simulateConcurrentClientRequest(roundRobbin, NUM_OF_REQUESTS);
client.printNextTurn("Weighted-Round-Robin");
LoadBalancer weightedRoundRobin = new WeightedRoundRobinLoadBalancer(ipPoolWeighted);
client.simulateConcurrentClientRequest(weightedRoundRobin, NUM_OF_REQUESTS);
System.out.println("Main exits");
}
private void simulateConcurrentClientRequest(LoadBalancer loadBalancer, int numOfCalls) {
IntStream
.range(0, numOfCalls)
.parallel()
.forEach(i ->
System.out.println(
"IP: " + loadBalancer.getIp()
+ " --- Request from Client: " + i
+ " --- [Thread: " + Thread.currentThread().getName() + "]")
);
}
private void printNextTurn(String name) {
System.out.println("---");
System.out.println("Clients starts to send requests to " + name + " Load Balancer");
System.out.println("---");
}
}If we look at the results (making 15 calls):
- Random:
---
Clients starts to send requests to Random Load Balancer
---
IP: 192.168.0.3 --- Request from Client: 9 --- [Thread: main]
IP: 192.168.0.2 --- Request from Client: 10 --- [Thread: main]
IP: 192.168.0.5 --- Request from Client: 8 --- [Thread: main]
IP: 192.168.0.1 --- Request from Client: 14 --- [Thread: ForkJoinPool.commonPool-worker-9]
IP: 192.168.0.2 --- Request from Client: 13 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.1 --- Request from Client: 11 --- [Thread: ForkJoinPool.commonPool-worker-9]
IP: 192.168.0.4 --- Request from Client: 7 --- [Thread: main]
IP: 192.168.0.2 --- Request from Client: 0 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.3 --- Request from Client: 12 --- [Thread: ForkJoinPool.commonPool-worker-23]
IP: 192.168.0.1 --- Request from Client: 5 --- [Thread: ForkJoinPool.commonPool-worker-13]
IP: 192.168.0.5 --- Request from Client: 3 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.3 --- Request from Client: 4 --- [Thread: ForkJoinPool.commonPool-worker-19]
IP: 192.168.0.2 --- Request from Client: 6 --- [Thread: ForkJoinPool.commonPool-worker-9]
IP: 192.168.0.1 --- Request from Client: 2 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.4 --- Request from Client: 1 --- [Thread: ForkJoinPool.commonPool-worker-27]- Round Robin: we can observe that each thread gets ordered IPs
---
Clients starts to send requests to Round-Robin Load Balancer
---
IP: 192.168.0.1 --- Request from Client: 9 --- [Thread: main]
IP: 192.168.0.2 --- Request from Client: 3 --- [Thread: ForkJoinPool.commonPool-worker-21]
IP: 192.168.0.3 --- Request from Client: 5 --- [Thread: ForkJoinPool.commonPool-worker-21]
IP: 192.168.0.4 --- Request from Client: 7 --- [Thread: ForkJoinPool.commonPool-worker-21]
IP: 192.168.0.5 --- Request from Client: 10 --- [Thread: ForkJoinPool.commonPool-worker-3]
IP: 192.168.0.1 --- Request from Client: 6 --- [Thread: ForkJoinPool.commonPool-worker-7]
IP: 192.168.0.3 --- Request from Client: 2 --- [Thread: ForkJoinPool.commonPool-worker-21]
IP: 192.168.0.2 --- Request from Client: 13 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.3 --- Request from Client: 1 --- [Thread: ForkJoinPool.commonPool-worker-19]
IP: 192.168.0.5 --- Request from Client: 4 --- [Thread: ForkJoinPool.commonPool-worker-27]
IP: 192.168.0.4 --- Request from Client: 14 --- [Thread: ForkJoinPool.commonPool-worker-23]
IP: 192.168.0.2 --- Request from Client: 8 --- [Thread: ForkJoinPool.commonPool-worker-9]
IP: 192.168.0.1 --- Request from Client: 12 --- [Thread: ForkJoinPool.commonPool-worker-13]
IP: 192.168.0.5 --- Request from Client: 11 --- [Thread: ForkJoinPool.commonPool-worker-17]
IP: 192.168.0.4 --- Request from Client: 0 --- [Thread: ForkJoinPool.commonPool-worker-31]- Weighted Round Robbin: We can observe that the IPs ending with 1 and 2 gets called 6 times, and 3 gets called 3 times, proportional to their weights
---
Clients starts to send requests to Weighted-Round-Robin Load Balancer
---
IP: 192.168.0.2 --- Request from Client: 9 --- [Thread: main]
IP: 192.168.0.2 --- Request from Client: 10 --- [Thread: ForkJoinPool.commonPool-worker-27]
IP: 192.168.0.1 --- Request from Client: 3 --- [Thread: ForkJoinPool.commonPool-worker-7]
IP: 192.168.0.1 --- Request from Client: 6 --- [Thread: ForkJoinPool.commonPool-worker-3]
IP: 192.168.0.2 --- Request from Client: 13 --- [Thread: ForkJoinPool.commonPool-worker-17]
IP: 192.168.0.1 --- Request from Client: 14 --- [Thread: ForkJoinPool.commonPool-worker-5]
IP: 192.168.0.2 --- Request from Client: 4 --- [Thread: ForkJoinPool.commonPool-worker-31]
IP: 192.168.0.2 --- Request from Client: 11 --- [Thread: ForkJoinPool.commonPool-worker-19]
IP: 192.168.0.2 --- Request from Client: 7 --- [Thread: ForkJoinPool.commonPool-worker-23]
IP: 192.168.0.3 --- Request from Client: 2 --- [Thread: ForkJoinPool.commonPool-worker-3]
IP: 192.168.0.3 --- Request from Client: 5 --- [Thread: ForkJoinPool.commonPool-worker-7]
IP: 192.168.0.3 --- Request from Client: 1 --- [Thread: ForkJoinPool.commonPool-worker-21]
IP: 192.168.0.1 --- Request from Client: 0 --- [Thread: ForkJoinPool.commonPool-worker-27]
IP: 192.168.0.1 --- Request from Client: 8 --- [Thread: ForkJoinPool.commonPool-worker-13]
IP: 192.168.0.1 --- Request from Client: 12 --- [Thread: ForkJoinPool.commonPool-worker-9]
