Search & Filter a dataset

I’m trying to create an app that allows the user to search and filter based on multiple criteria. My initial data contains about 2,300 records (other datasets I have contain millions). I tried listing all 2,300 in a repeating list initially but it slowed the app considerably, so I’m trying to only pull the data as needed based on search criteria. I’ve been able to get search working for stock ticker where the data does not show until the ticker is entered. But I need to be able to search on more than just the Ticker.

I’d like to be able to search the dataset by the following categories (fields):

  1. Stock ticker
  2. Company name
  3. Author
  4. Date range

Is there a way to conduct all of these various searches on one table of data being pulled from an API without showing that data until it is searched? As I said, I’ve set up search on Ticker and I also have a filter by Author and that works. But because I’m only pulling search data, the filter only filters the search results. For the filter, what I’d really prefer is to be able to use the dropdown to filter by all Authors, but the authors don’t show in the dropdown until the ticker is searched.

So what I’d like to do is also have these other ways to search by these other fields, and then include a few different filters on top of the search results.

Also worth noting I’m using Supabase as my backend and the database is PostgreSQL. Any help is appreciated!!!

Hi! There are several different ways of doing this. I would probably go for the following.

  1. On the global canvas, fetch all the 2,300 data records and store them in an app variable
  2. On the page where you want to display some of the data, you can set into a page/data variable the data from the app variable which you want to show. You can in the logic determine that this setting is only done after the filters are sufficient.

I would also recommend for you to use RecyclerView with our 2.X runtime (currently available in beta, soon also otherwise). It should help with the performance issues.

1 Like

I have the data available via API data variable. How do I fetch and store to app variable?

Hi! So on the global canvas, from the app launch event, attach a get record collection flow function and then a set app variable flow function where you set the result of the get record collection to the app variable.

1 Like

Hi im trying to implement this way that you explain, but im having trouble storing the object list from get record collection in the app variable, how is it possible to set a data collection in an app variable?
should the app variable be a list or plain text?

Hi! You need to configure the app variable to be a list of objects with the same exact schema as the data collection has if you want to do it properly. I sometimes cheat and just define the app variable as list of objects (deleting even the id field of the schema) and then just set the list of objects via formula. I recommend setting the schema though, as it will help you later on in the app. If you don’t set the schema, the app variable won’t have one, so you always have to use it and its fields via formulas and remember all the property names correctly etc.

1 Like

Nice, i will try it out, thank you very much @Mevi

Hi Mevi. First thank you for your tutos they have been a huge help for me. :smiley:
Sorry to dig this old post. :sweat_smile:
Can you be more step by step to explain what you said in the previous message.
I have about 2500 document in a collection and each document have 14 fields. I would like to create a search page on either one or more of the 14 fields.
Not sure if i’m specific enough let me know if you need more info .

1 Like

Hi! I won’t go into super specifics, but once you have the (app) variable with all the data, you can use a formula like SELECT and SOME to choose only the records that match your search query. Something like SELECT(appVars.myData, SOME(VALUES(record), CONTAINS(item, pageVars.searchQuery))) for example. This is however a very heavy operation as it goes through all the fields of all of your data records, so I would recommend limiting it down to three fields at the most, and using this sparingly - e.g. Only when the query is more than three characters, and with a debounce.

1 Like

Ok thank you very much
Well it worked but putting all my datas in a variable make me reach the limit of Firebase too fast. Do you think there is a way to pick the right data directly from the database without importing all of them in the device?

Check out this thread for example – you can filter straight in the query if you prefer :slight_smile:

1 Like

Thanks you very much Mevi. :slight_smile: