4

I have created a one java backend service in vertx java. I used httpClient(io.vertx.core.http.HttpClient) with connection pooling enabled to connect to external services. I am excepting throughput of 50. For each request to my service I need to connect to external service. Average response time for my service is 4 seconds and external service is around 3 seconds.

Now my questions are

  1. How can I derive maxPoolSize and maxWaitQueueSize values for HttpClient?
  2. What is the impact maxPoolSize and maxWaitQueueSize values on memory and cpu?
  3. What are the maximum values I can set to maxPoolSize and maxWaitQueueSize?
  4. Should I also use setPipelining option of HttpClient?
Vaibhav Raut
  • 472
  • 3
  • 12

1 Answers1

7

First, beware that maxPoolSize applies to each destination. So if you want different pool sizes, create different HttpClient for your backends.

Then, unless you work in a constrained environment, I would recommend to leave maxWaitQueueSize to the default, which is -1 (unbounded). The size of the queue in memory should be relativeley small given the load you're expecting.

To determine the pool size value you can use Little's law. To support a 50 req/sec throughput with an average 3s service time, you need a pool of 150 connections.

The maximum value you can set for maxPoolSize depends on how your system is configured. In particular, you need to configure the max number of open file descriptors.

For your use case, I believe you should avoid enabling pipeling. First not all HTTP server support it correctly. Second if the service time varies between 0-3 secs, a response might be hold by the backend because previous requests in the pipeline are not handled yet (head-of-line blocking).

tsegismont
  • 8,591
  • 1
  • 17
  • 27