Batch requests

Last modified by Ashterix on 2024/05/16 18:58
Content

Introduction

Batch requests to the API are an approach that allows you to combine multiple requests into one. Instead of sending numerous separate requests to the server, the client can send a single request containing multiple operations. The server receives this request, processes all the operations, and returns a single response that includes the results of all the requests.

Let's imagine we are developing a backend API for an online store and we already have a method ProductService.getInfo for the entity Product. The task on the frontend is to implement the logic of adding products to the cart and updating product information upon request (for example, when opening the cart, we need to update information about the products the user added: availability, prices, descriptions, etc.). 

Scenario that 100% of developers follow

A task is set for the backend to create a separate API method that returns a collection of objects by an array of identifiers. This requires the working time of a backend developer, a tester, and a frontend developer since the new method must be covered by unit tests and additional scenarios for regression testing. This must be done despite having a method that can return one product by its id.

Scenario that no one follows

Send a separate API request for each product to get the current information. If there are 10 products in the cart, this means 10 HTTP requests. This creates significant network load and increases waiting time for the user, which is an unacceptable option.

Alternative using batch requests

If we have the ability to send and process batch requests, we can significantly simplify the implementation of business logic. We can use a batch request to combine all these requests into one. We send a single request to the API containing all the product identifiers, and the server returns information about all the products in one response. Thus, we get:

  1. Universality: Using batch requests allows us to avoid creating specialized methods for each case. This simplifies the API and reduces the amount of required code.
    2. Convenience for the client: The client can independently determine which requests to combine, making the API more flexible and convenient to use.
    3. Reduction of backend complexity: Instead of writing and maintaining specialized methods for different scenarios, we can use a universal approach with batch requests, simplifying backend architecture.

This approach provides more efficient use of resources and improves performance, reducing the load on the network and servers, and enhances the user experience.

Of course, in high-load systems, such a scenario may be inefficient, but there are other optimization options that we will consider in other articles.

Example

Consider the request example I mentioned above.

Request
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[
   {
     "id":"example_1",
     "method":"ProductService.getInfo",
     "params":{
        "productId": 345234
      }
   },
   {
     "id":"example_2",
     "method":"ProductService.getInfo",
     "params":{
        "productId": 994234
      }
   }
]

Important!!! Always specify unique request ids in batch requests, as responses may be returned in a different order.

Response
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[
   {
     "id":"example_2",
     "result": {
        "id": 994234,
        "name": "product 2",
        "price": 11.99,
        "balance": 28
      }
   },
   {
     "id":"example_1",
     "result": {
        "id": 345234,
        "name": "product 1",
        "price": 99.99,
        "balance": 14
      }
   }
]

Advantages of Batch Requests

  1. Simplifying the backend: Instead of implementing specific logic on the backend each time, it can be transferred to the level of building client requests.
    2. Reducing the number of network requests: Instead of sending multiple HTTP requests, which can create network and server load, only one request is used. This significantly reduces network and server resource load.
    3. Response time: The RPC Server will return the result of all requests, regardless of their number, in the time required to process the longest request in the batch, as all requests are processed in parallel.
    4. Reducing latency: Executing multiple requests in one connection can reduce overall latency as less time is spent on establishing and closing connections.
    5. Optimizing client and server resource use: Fewer requests are processed simultaneously, reducing server load and allowing it to handle more requests. The client also uses fewer resources, which can be critical for mobile and limited devices.
    6. Improved transaction management: When multiple operations need to be performed atomically (all or none), a batch request allows for easier implementation, as all operations are executed within one request.

Preparing for Use

Batch request functionality is available immediately and does not require additional settings from the developer.

Request Order

The order of filling the batch does not matter because on the RPC server all requests are executed in parallel, and you will receive a response at the speed of the longest request in the list.

If an error occurs in one of the requests in the list, it does not affect the processing of other requests in the list.

Dependent Requests

The batch request mechanism allows creating dependent requests. This is a request where one or more parameters depend on the response from another request in the list.

Imagine that we need to get the user's email address by their identifier from the frontend and then send a message to this address. We already have the methods UserService.getInfo and Messenger.sendEmail. In a classical scenario, this would be either two requests or a separate method on the backend that sends an email by user id.

Using the JsonRpcBundle library from UFO-Tech, you can create batch requests where one request can depend on another and use its response to execute. In the context of the example, we get the user's information in one request, and the second request uses the received email to send a message.

Request
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[
   {
     "id":"example_1",
     "method":"UserService.getInfo",
     "params":{
        "userId": 141
      }
   },
   {
     "id":"example_2",
     "method":"Messenger.sendEmail",
     "params":{
        "email": "@FROM:example_1(email)",
        "subject": "Welcome!",
        "message": "Thank you for joining our service!"
      }
   }
]
Response
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[
   {
     "id":"example_1",
     "result": {
        "id": 141,
        "name": "John",
        "email": "john.dou@example.com",
        "status": 1
      }
   },
   {
     "id":"example_2",
     "result": "Email for 'john.dou@example.com' sent"
   }
]

In this example:

  1. The first request retrieves user information by id 141, including their email.
    2. The second request sends an email to the received email using data from the first request.

The request order does not matter! The RPC Server determines the dependencies of the requests and queues them in the required order.

Dependent requests are executed sequentially, so the response to the batch request will be returned after the dependent requests are completed.

Advantages of Using Dependent Requests

  1. Convenience and efficiency: Dependent requests allow performing complex operations with minimal effort, ensuring the correct order of request execution.
    2. Reducing the number of network requests: All requests are combined into one, reducing network load.
    3. Flexibility: The ability to create complex request scenarios without additional settings or changes on the backend.

How It Works

The batch request processing mechanism works asynchronously.

When receiving an array of requests, the RPC Server creates a queue of Symfony Process, i.e., it runs CLI commands that are processed asynchronously. In a while loop, the state of the processes is checked, and if a result is obtained, it is added to the response array. If there is no response before the timeout expires, an error is returned indicating that the request was not processed.

Algorithm

  • The batch request is split into individual requests, each of which is added to the queue.
  • The queue is checked in a loop for the presence of objects.
  • For each object in the queue, it is checked whether the process has finished.
  • If the process has finished, the result is added to the response array, and the process is removed from the queue.
  • If the process has not finished and the timeout has not expired, the loop continues.
  • If the timeout expires before receiving the result, an error is returned for that specific request indicating that it was not processed.

To increase the timeout for a batch request, you can specify an additional service parameter $rpc.timeout in the request parameters, which is the maximum number of seconds to wait for a response from the process. By default, the timeout value is 10 seconds.

Request
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[
   {
     "id":"example_1",
     "method":"ExampleApi.fastMethod",
     "params":{
        "someParam": "someValue"
      }
   },
   {
     "id":"example_2",
     "method":"ExampleApi.longMethod",
     "params":{
        "someParam": "someValue",
        "$rpc.timeout": 30
      }
   }
]

This allows you to adjust the duration of waiting for results for methods that might take longer to execute, which can be important for processing certain requests.