Pagination of Grouped Rows with Client-side pagination and Data Accumulation using Livewire

Pagination of grouped rows that span across pages can get complicated in server-side pagination. Client-side pagination on the other hand offers easy logic in page movement, but has a bottleneck in processing entire datasets.

In Hoarding Order with Livewire we explore a "Hoarding" or "Data Accumulation" approach in client-side pagination, easily implemented with Livewire!

Instead of waiting for an entire dataset to get downloaded, our table receives an initial set(with next page allowance) and periodically updates and piles up its data with batches fetched in the background.

Pagination is client-side(on accumulated data) and so there is no server-response lag in movement between table pages, nor is there complication when it comes to displaying rows in a group!

This client-side pagination + data accumulation approach is implemented with:

1. A client-side pagination logic
2. Livewire:poll to silently fetch new data batches in the background
3. Livewire's event mechanism to update the table's accumulated data with each data batch received from polling

Following Along

You can check out our full repository here as reference. Make sure you run the migrations!

If you're opting out of the repository route, you'll need a Laravel project that has both Livewire and Tailwind configured. Make sure you have a groupable dataset with you so you can see the magic of our approach in maintaining grouped data order across pages. Run php artisan make:livewire article-table to create your Livewire component, and you're free to dive in below.

Livewire Data Accumulation

With Livewire, we can easily set up a table component with public attributes keeping track of the list of items, accumulating data over time with their ordering intact.

We can then simply rely on displaying indices specific to a page and not worry about grouping logic during movement between pages.

Below, we'll set up our Livewire controller's public attributes and functionalities to retrieve and accumulate ordered data. Then on the last step, we'll update our Livewire view to display and refresh our accumulated data with events and polling.

Controller

Let's make some changes to our /app/http/Livewire/ArticleTable.php:

# /App/Http/Livewire/ArticleTable.php

class ArticleTable extends Component
{

    // List that accumulates data over time
    public $dataRows;
    
    // This is total number of data rows for reference, 
    // Also used as a reference to stop polling once reached
    public $totalRows;

    // Used for querying next batch of data to retrieve
    public $pagination;
    public $lastNsId;

    // Override this to initialize our table 
    public function mount()
    {
        $this->pagination = 10;
        $this->initializeData();
    }

Polling Accumulation

For smooth sailing for our users, we'll set up a client-side paginated table that allows table interaction to be lag-free.

We'll initially query the necessary rows to fit the first page in the client's table—with a pinch of allowance. This should be a fast enough query that takes less than a second to retrieve from the database, and the size of the data returned shouldn't be big enough to cause a bottleneck in the page loading.

What makes this initial data retrieval especial is the extra number of rows it provides from what is initially displayed. We get double the first page display, so there is an allowance of available data for next page intearction.

/**
 * Initially let's get double the first page data 
 * to have a smooth next page feel 
 * while we wait for the first poll result
 */
public function initializeData()
{
    $noneSubList = $this->getBaseQuery()
    ->limit($this->pagination*2)
    ->get();

    $this->addListToData( $noneSubList );
}

/**
 * Gets the base query
 */
public function getBaseQuery()
{
    // Quickly refresh the $totalRows every time we check with the db
    $this->totalRows = Article::count();

    // Return none-Sub rows to avoid duplicates in our $dataRows list
    return Article::whereNull('lead_article_id');
}

Then in order to get more data into the table, Livewire from the frontend can quietly keep adding items to the table through polling one of the controller's public functions: nextPageData.

/**
 * For every next page, 
 * we'll get data after our last reference
 */
public function nextPageData()
{
  $noneSubList = $this->getBaseQuery()
    ->where('id','>',$this->lastNsId)
    ->limit($this->pagination*10)
    ->get();
  
  $this->addListToData( $noneSubList );
}

Add in our core functionality for ordering our data retrieved: get possible sub rows for the data result, merge data inclusive of their sub rows in proper ordering to our $dataRows.

 /**
 * 1. Get possible Sub rows for the list of data retrieved in nextPageData or initializeData
 * 2. Merge list of data inclusive of their possible Sub rows, in proper ordering, to the accumulated $dataRows 
 * 3. Update the $lastNsId reference for our nextPage functionality 
 */
public function addListToData($noneSubList)
{
    $subList = $this->getSubRows($noneSubList);
    foreach( $noneSubList as $item ){
        $this->dataRows[] = $item;
        $this->lastNsId   = $item->id;
        foreach( $subList as $subItem){
            if( $subItem->lead_article_id == $item->id ){
                $this->dataRows[] = $subItem;
            }
        }
    }
}


/**
 * Get the Sub rows for the given none-Sub list
 */
private function getSubRows($noneSubList)
{
    $idList = [];
    foreach($noneSubList as $item){
        $idList[] = $item->id;
    }

    return Article::whereIn('lead_article_id', $idList)->get();
}

View

Once we have our Livewire controller set up, let's bring in some color to our Livewire-component view, /app/resources/views/livewire/article-table.blade.php:

<table>
  <thead>...</thead> 
  {{-- wire:ignore helps to not reload this tbody for every update done on our $dataRows --}}
  <tbody id="tbody" wire:ignore></tbody>
</table>
<nav role="navigation" aria-label="Pagination Navigation" class="flex justify-between" >
    <button onclick="prevPage()">Prev</button>
    <button onclick="nextPage()">Next</button>
</nav>

What makes our setup pretty cool is that pagination will be done strictly client-side. This means user interaction is available only on data the UI has access to—meaning from the JavaScript side of things—and consequently, a lag-free pagination experience for our users!

Client-Side Pagination

To display our data, we initialize all variables we need to keep track of from the JavaScript side. This includes a reference to our table element, some default pagination details, and finally a myData variable to easily access the data we received from $dataRows.

Go ahead and add in a <script> section to our Livewire-component view in /app/resources/views/livewire/article-table.blade.php:

<script>
  // Reference to table element
  var mTable   = document.getElementById("myTable");
  // Transfer $dataRows to a JavaScript variable for easy use
  var myData   = JSON.parse('<?php echo json_encode($dataRows) ?>');
  // Default page for our users
  var page     = 1;
  var startRow = 0;
  // Let's update our table element with data
  refreshPage();

Then set up a quick JavaScript function that will display myData rows in our tbody based on the current page.

function refreshPage()
{
    // Let's clear some air 
    document.getElementById("tbody").innerHTML = '';
    
    // Determine which index/row to start the page with
    startRow = calculatePageStartRow(page);
   
    // Add rows to the tbody
    for(let row=startRow; row<myData.length && row<startRow+10; row++){
        let item = myData[row];
        var rowTable = mTable.getElementsByTagName('tbody')[0].insertRow(-1);
        
        // Coloring scheme to differentiate Sub rows
        if(item['lead_article_id']!=null){
            rowTable.className = "pl-10 bg-gray-200";
            var className = "pl-10";   
        }else
            var className = ""; 

        var cell1 = rowTable.insertCell(0);
        var cell2 = rowTable.insertCell(1);
        var cell3 = rowTable.insertCell(2);
        var cell4 = rowTable.insertCell(3);

        cell1.innerHTML = '<div class="py-3 '+className+' px-6 flex items-center">' + item['url'] + '</div>';
        cell2.innerHTML = '<div class="py-3 '+className+' px-6 flex items-center">' + item['source'] + '</div>';
        cell3.innerHTML = '<div class="py-3 '+className+' px-6 flex items-center">' + item['id'] + '</div>';
        cell4.innerHTML = '<div class="py-3 '+className+' px-6 flex items-center">' + item['lead_article_id'] + '</div>';
    }
}

Along with functionality to allow movement from one page to the next and back:

function nextPage()
{   
    if( calculatePageStartRow( page+1 ) < myData.length ){
        page = page+1;
        refreshPage();
    }
}

function prevPage()
{
    if( page > 1 ){
        page = page-1;
        refreshPage();
    }
}

function calculatePageStartRow( mPage )
{
    return (mPage*10)-10;
}

Discreetly add in our accumulation of remaining data with the help of Livewire's magical polling feature that can eventually stop once we've reached maximum rows:

@if( count($dataRows) < $totalRows )
    <div wire:poll.5s>
        Loading more data... 
        {{ $this->nextPageData() }}
        {{ $this->dispatchBrowserEvent('data-updated', ['newData' => $dataRows]); }}
    </div>
@endif

And finally, in response to the dispatchBrowserEvent above, let's create a JavaScript listener to refresh our myData list and re-render our table rows—just in case the current page still has available slots for rows to show.

window.addEventListener('data-updated', event => {
   myData = event.detail.newData;
   refreshPage();
});
 

And that's it! We're done, in less than 300 lines of logic!

Of course, there is always room for improvement, never think there isn't! So checkout my article on data accumulation tips "Offloading Data Baggage ", plus a well-improved repository and demo app quickly flown with Fly.io!