Limiting Firebase record reads to keep costs down

#1

I have an idea to keep firebase costs down, but I’m curious what you all think its the best way to implement it.

Let’s say you have 200 records in a firestore database.
When you mount the page you GET all of those records and set to a data variable (pretty standard).
If then you were to update a record, unfortunately you need to GET and set again to refresh the list. That’s another 200 reads. Not a huge deal, but you only get 50,000 a day for free. So it could be a big deal if you have large databases and lots of users. If figure I can only have about 10 users without exceeding that limit. Then it gets real expensive, real fast.

Option 1:
GET all records from Data on every page mount. Set to a separate client local Data Variable.
Show the local data in the repeating list.
When you update a record, update both the local and the Firestore data sets.
Unfortunately if you do this, you need to pull the data every time you open that page. I don’t think it keeps the Data Variable across pages.

Which leads me to option 2
Option 2:
GET all records once. Set to an App Variable list of objects .
Show the App Variable list in the repeating list.
When you update a record, update both the App Variable and the Firestore data.
You’d have to do some real gymnastics to make this work, but you’d only need to pull all the records once per session.
Do you think there would be a performance impact in keeping a reasonably sized list in app.var memory rather than in a data variable?

Option 3:
GET all records on page mount. Set to a Data Variable as is typical.
Show the Firestore data in the repeating list.
When you update a record, update both just the Data Variable record and just the actual Firestore data record. That way don’t have to get the full set of records again. I don’t know how to change just one record in the Data Variable without changine the firestore data and have the list refresh. I also don’t know if you can update a record without getting a full response (and so the 200 reads again). So this doesn’t seem to work.

Option 4
I learn how pagination and indexes work and have the user refresh as necessary. But without either pull to refresh or infinite lists this isn’t great either.

There must be a better way to do this than the standard. At the scale I want to run my app, the costs of the way the firestore connector operates would make the whole project unfeasible.

1 Like
#2

i had the same thought but:

for option 4 there is a pull to refresh trigger

For firestore an other cleaver way of making some kind of infinite page, is to put three (or as many as y like) copies of your list component in the page with buttons in between with visibility function for the next list as well as the fetch request and using three (or more) different requests of the same data resource but with incrementing pagination numbers, you can make some kind of infinite list this way (but with ending).
This can make it that only if the user goes to the end and wants more content you make a new request and only then make more reads. Also this way is smart, because it makes the previous content visible and its not like pressing a next page button.

as for option 1 its a smart way of not making an extra read, i use it for ex in liking system so when the write request was successful just change the frontend too

Option 2,3 doesnt sound so promising and not so cleaver because some times its impossible fetching all the data for numbers lime 2000 documents and also there is no need to ‘pay’ for all these reads cause there are so many good ways of quering

Also take a look in firebase Realtime database, even though it seems more difficult it isnt (takes a few hours to make everything work) and from what i understand, if you have many documents but ‘small’ in size Realtime is the one to use because you dont pay by read right but by the bytes down/uploaded.

#3

This is a really great reply. Thank you for your help!
I also found this to help with option 2.

Combine that with your comments about queuing or paging and this might work.

My records are very small. I’m going to check out the Realtime database option. I’m hopeful that this will work!

1 Like