I implemented a data source with Airtable which was pretty straightforward. I’m running into a problem as Airtable will return a max of 100 records at a time. If the table has more than 100 records, Airtable will return an offset value that you have to pass back to the API to get the next 100.
There’s not really a way to do this with a data source that I can see. The GET in Airtable returns an object like:
{
records: array of the table schema,
offset: the value to pass back to the API in the next call to get the next 100 records
}
I’ve set the Response Key Path to “records” to get the data. How can I access the “offset” value?
The REST API direct integration doesn’t support that at the moment, unfortunately.
The easiest way to get around this is to use the HTTP request node instead, so you get your hands on the whole response – you can still use Set data variable, you would just set it to the formula outputs["HTTP request"].resBody.records, and access the offset via outputs["HTTP request"].resBody.offset.
If I want to fetch entries of an airtable base that exceeds 100 in a list item,
as you said that I should use the http request node to request data and I did and verified by an alert that data is pulled.
but can you please guide me that how to populate the list item with this data?
If you know how many records you need, a naive way is to just string together multiple HTTP request nodes with different offset values, and finally merge their response outputs into one array with a formula like
Alternatively you can build a loop with e.g. a page variable for fetchMore that is set to true if the API returns info about the next page and then serves as an IF condition clause to determine if more data should be fetched or the loop finished. Each set of fetched data would then be added to a page/data variable array with the CONCAT function before checking if more data should be fetched still.
Thanks @Harri_Sarsa it worked !! Now I am able to fetch more than 100 items,
but there is another issue that by adding Http request logic, the app got very slow. I can not tap any component.
any help?
The first thing to look is the looping logic — could you be triggering the loop twice in every iteration so you’d have an exponentially growing load? Can you post a screenshot of your logic?
If you remove the wire going straight to the Delay node from the HTTP request, you should be good – now if appVars.offset is not empty, every execution of HTTP request causes exponentially growing number of loops.
The other part is that, if you repeat the records (the app variable) on page, after each Set app variable all will be rendered from scratch. This might make your app slow if you have a lot of components to repeat. I’d consider adding some pagination at this point as huge amount of items take some time to render.
(Also, asking just to be safe: Have you checked that the offset you’re using actually increases after each iteration?)
I’ve problems with concat. If I don’t use concat (Just to try) data variable is assigned to response record. If i use concat (also without iterations) it gives me null values. It seems there is an error, but can’t understand wich.
Any idea?
I have done what you said as follows: 1 - I have cut the line from Flow function #4 (If condition) and instead connect directly back to HTTP Request 2 - And also I have convert offset variable from app to page
But the same problem exists
And also there is new problem that the text data which I want to fetch are showing in from app preview portal in laptop but in android appgyver app the data is invisible
And I have checked that the offset increases and the data is iterating
Hi Viraj, this may not be the most elegant way, but it works for me.
I start populating my client list (I’m retrieving a list of clients from Airtable) by calling a ‘HTTP request’ flow function when my client page is mounted. If there are more than 100 records (Airtable’s limit) or if you’ve specified a pageSize less than the records available, the response will contain an offset value.
I created a page variable called client_offset, which I assign the offset value to (if it exists).
I then have a button at the bottom of my client list, which becomes visible when my client_offset variable is not empty. Clicking on the button calls a new HTTP request, which includes the offset value. The URL for my HTTP request (button) is as follows:
