Large list and cannot limit API get collection requests

Hi all,

I have a large list of about 7000 items with each list item having about 7 objects linked to it.

The problem is I cannot limit the items from the API because they all appear on a search and if I do that the search will not find the relevant items as only 50/100 were fetched. How can I go about getting this done?

Should I save all the items on the client-side storage? Will that increase the speed because at this stage it is freezing due to the number of items in the list.

The size of the list is 40mb.

Any help or guidance will be appreciated.

Hi there!

That’s a lot data! However handling it in memory should still be possible.
I believe the slowest points will be downloading it from the API and rendering it in a repeat etc.

I recommend downloading it and then storing it Client-side storage to avoid having to fetch it repeatedly. Be sure to also disable to automatic refreshing from data variable logic canvas as we don’t want to be downloading 40MB of data every 5 seconds :sweat_smile:

The filtering and limiting I’d do before attempting to use the data in a “repeat with” as 7000 item repeat does not sound optimal neither in terms of ux or performance.

Let me know how it goes!

Do you have control over the APIs? If you do, you could consider having a search API that will allow you to reduce the amount of data returned to your client app based on the search criteria sent to the Search API.

Also, if you have control of the APIs and need to have the search/filtering done on the client side, you could “lighten” the amount of data returned for the 7,000 items so that you aren’t waiting on 40mb to load to the client app. You could have it send just the data needed to support the search by the user.

Good luck!