Pagination in API – why and how?

Introduction

Pagination is splitting our data lists returned to the user from API into smaller parts. We can see that in many cases – i.e. changing pages of search results on Amazon or scrolling Facebook. What are benefits of pagination? Below you can see couple of them:

• reducing network traffic (users with poor network connection have better experience using app),
• making application more stable and production usable (Java apps can throw Out of Memory Error if the heap size exceeds Xmx value - it can happen when you load too much data from DB),
• reducing load of data processing and as a result reducing cost of maintenance our app - especially when hosting in cloud,
• app can handle more requests in the same period of time (cause faster responses),
• reducing number of returned objects which are not required for user.

Let’s get some background: we have a to-do Java&Spring app which provides basic functions:

• creating and updating task,
• changing status from “open” to “done”,
• deleting task,
• getting lists of open and done tasks.

Last functionality is connected with objects lists so that’s the point where pagination can be used. Why we should do that? In open tasks we loading data sorted by priority to get the most important things at the top because they are first planned to do. Do we need to return all open tasks at once or better wait for user signal (scrolling) indicating need of next 10 tasks? Similar question about history of tasks – it can be very long list.

Example implementation

Adding pagination to existing code is usually not so hard if code is well-organized. I will do that on example of mentioned to-do app.

First step is to create possibility to control of data partitioning. To do that we can add two params to our API – startIndex and quantity. Quantity describes amount of objects per page, startIndex is a position of first object on specific page in all db sorted data.

Old endpoints definitions:

import java.util.List;
 
import lombok.RequiredArgsConstructor;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
 
import com.example.todoapp.domain.model.TaskDto;
import com.example.todoapp.domain.service.TaskService;
 
@RestController
@RequiredArgsConstructor
@RequestMapping(TaskController.PATH)
public class TaskController {
    public static final String PATH = "task";
 
    private final TaskService taskService;
 
    @GetMapping(value = "/open", produces = MediaType.APPLICATION_JSON_VALUE)
    private List<TaskDto> getMyOpenTasks() {
        return taskService.getMyOpenTasks();
    }
 
    @GetMapping(value = "/done", produces = MediaType.APPLICATION_JSON_VALUE)
    private List<TaskDto> getMyDoneTasks() {
        return taskService.getMyDoneTasks();
    }
}

New endpoints definitions:

import java.util.List;
 
import lombok.RequiredArgsConstructor;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
 
import com.example.todoapp.domain.model.TaskDto;
import com.example.todoapp.domain.service.TaskService;
 
@RestController
@RequiredArgsConstructor
@RequestMapping(TaskController.PATH)
public class TaskController {
    public static final String PATH = "task";
 
    private final TaskService taskService;
 
    @GetMapping(value = "/open", produces = MediaType.APPLICATION_JSON_VALUE)
    private List<TaskDto> getMyOpenTasks(@RequestParam Long startIndex,
                                         @RequestParam int quantity) {
        return taskService.getMyOpenTasks(startIndex, quantity);
    }
 
    @GetMapping(value = "/done", produces = MediaType.APPLICATION_JSON_VALUE)
    private List<TaskDto> getMyDoneTasks(@RequestParam Long startIndex,
                                         @RequestParam int quantity) {
        return taskService.getMyDoneTasks(startIndex, quantity);
    }
}

Pass params throw service to repo layer. If you are using Postgres like me, you can add pagination by below changes:

Old repo methods:

import java.util.List;
 
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
 
import com.example.todoapp.domain.entity.Task;
 
@Repository
public interface TaskRepository extends JpaRepository<Task, Long> {
 
    @Query(nativeQuery = true, value = "select * " +
            "from task " +
            "where user_id = :userId " +
            "  and status = :status " +
            "order by priority desc, id;")
    List<Task> findTasksByUserIdAndStatusOrderByPriorityDesc(@Param("userId") String actualUserId,
                                                             @Param("status") String status);
}

New repo methods:

import java.util.List;
 
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
 
import com.example.todoapp.domain.entity.Task;
 
@Repository
public interface TaskRepository extends JpaRepository<Task, Long> {
 
    @Query(nativeQuery = true, value = "select * " +
            "from task " +
            "where user_id = :userId " +
            "  and status = :status " +
            "order by priority desc, id " +
            "limit :quantity offset :startIndex ;")
    List<Task> getMyTasksByStatus(@Param("userId") String actualUserId,
                                  @Param("status") String status,
                                  @Param("startIndex") Long startIndex,
                                  @Param("quantity") int quantity);
}

Now old endpoints calls changes from:

http://localhost:8081/task/done

To:

http://localhost:8081/task/done?startIndex=0&quantity=10

GitHub demo can be found here.