Data Variable Limited to 20 Records

I have created a Data Variable with brings in a Collection of Data Records through the REST API from
a Firestore collection. The Firestore collection that I’m connected to has 39 records, however the Data Variable is only being populated with the first 20 (confirmed with the debugger).

Is there a setting or option that I’m missing that is limiting the number of items being loaded? or is this a limitation of AppGyver or Firestore?

At least AppGyver has no such limit :thinking: but no idea about Firestore…

@Geoff_Harcombe
How many records are returned in the data resource test tab? To me it sounds like the Firestore result may have some pagination etc. applied to it.

The data resource test tab only returns 20 of the 39 records.

Ok, this pretty much means your query to Firestore is only returning 20 records. I unfortunately don’t have enough experience with Firestore to have a solution for that :neutral_face:

Thanks @Akseli_Virtanen for the hint about pagination.

It appears that Firestore by default only returns 20 records using the “list” method in their API. However, included in the response is a nextPageToken that can be added to a new call to get the next 20 results and so on. Though I’m not sure how to implement this in AppGyver such that when the bottom of the list is reached more records are called from Firestore.

Alternatively, Firestore also allows you to set the maximum number of documents to return using the “pageSize” query parameter. If this set to a number higher than 20, then more than 20 results are returned.

2 Likes

This is similar to AirTable, which returns batches of records. If you search on the forum for AirTable, you will see ideas that will help you with pagination for other DBs as well.

1 Like

You’re a life saver! I was totally mystified initially. Adding the query parameter solved the problem :slight_smile:

2 Likes