Not able to search big Database

Hello!

I try to make a database app with appgyver. Everything works well with my small demo database, I can search for the name of my chemical, and the app shows only the records with the name from the search field. The Page looks like this:



It works really well with my demo database. But I also want to search through my real database, with 37000 different chemicals. If I just use the search formula with my real database, no entries are shown. Some troubleshooting I did:

<> Both databases are visible at the Data Configurator/Ressource ID
<> The content of the paragraph which is repeated is adjusted to the new variable name
<> I tried to recode the whole search page, one time with my demo database, one time with my big database, It only works with the demo database.

I don’t know if its important, but here are the fields of my big database:

I have no clue what is wrong with my app. It is really strange for me that only one database works, and the bigger one doesn’t work, although I use the same code.

Thank you for your help,
Michael

37000 i think its too many for the app to handle, you need to set up some basic filtering in the backend so you can fetch some reasonable number of data and then you can make further frontend filtering.
Maybe you can make the user give some basic chemical characteristic from a dropdown list (which you have in your data fields and you can filter based on that) that can help you get the right data.

I hope it helps

You should never be querying that many records on the client side. You always want that to be handled on the back end. What is your back end?

I use a firebase database as backend.

Does anyone have a tip for me, regarding my problem?

Hello
Did you manage to get a a solution? I’m very interested.
Thank you

If you use firestore, there is filtering option, built in appgyver composer.

Hello Dimos!
As far as I know you can only filter string type variables in the Data variable section with condition type “equal” and “inequal”. But this is not useful for me, since I also want to search for parts of the string, for example the search text “car” should also give “big green car” as a result, but this is not possible with the “equal” condition.

yeah, in this case you need to use full search capability. there are plugins for that in firebase.
look in the tab for plugins in firebase

Thank you for your answer! Do you mean there are plugins in appgyver for this problem or in firestore? Do you have more information for me how to implement these plugins?

And another question: Is it possible to store a whole record collection from firebase to an On-device data variable that is accessible from every page of my app?

Like @Dimos_Vamvourellis points out, the full text search is not capability of firestore. You can start here to dig into it:

https://firebase.google.com/docs/firestore/solutions/search?hl=en

For myself I did not follow up on this feature and decided, that the app will only rely on receiving filtered data by using Query. Therefore I can’t tell how to implement the Full Text Search.
For a big database like yours, the filtering should to be done on the backend.

Regarding your question about the data variable: you can save the data into an App Variable.

1 Like

Okay thanks!
To your last sentence: How do I store a full record collection in an App Variable? It seems possible to store the collection in a list of objects, but how do I store the data from appgyver in this list of objects at every start of my app?

There a several ways to implement it. The first that comes to my mind is to

  • prepare an App variable
  • enter the logic canvas of the data variable, that receives the collection in the first place.
  • add a flow function ‘set app variable’ to copy the collection into the app variable.