Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large list performance #561

Closed
tiagocorreiaitcenter opened this issue Nov 11, 2019 · 9 comments
Closed

Large list performance #561

tiagocorreiaitcenter opened this issue Nov 11, 2019 · 9 comments

Comments

@tiagocorreiaitcenter
Copy link

Hello, im trying to implement a FlatList with a large amount of data, between 5-20k of items, and I can't stop thinking if there's no better way of handling it other than waiting for the entire data to load since its a long process? some logic like, .skip and .limit methods, since FlatLists let us know we are reaching the end of the list for example, not sure if you have a better idea of how to implement such scenarios but would like to know how you work with it, thanks

@tiagocorreiaitcenter
Copy link
Author

Also sorting plays a big role on this list, its a contacts list, im not sure how we sould implement the sorting since it should based on the database data

@radex
Copy link
Collaborator

radex commented Nov 12, 2019

See: #217

🍉 has no built-in support for this case. The easiest option for you is to add your own column and query based on a range of values in that column to implement chunking/limiting of the list. The better option is to start making steps towards implementing proper DB-level sorting and limiting in 🍉 itself. Look for "limit" and "sorting" in issues to find discussions about this

@tiagocorreiaitcenter
Copy link
Author

@radex Thanks will pay attention to it, I was thinking about adding a number to know the order and use stuff like greater than or something like that, but I Have a small issue, I can change a single contact name, and the stored order would be useless sadly, will try to see if I can come around with an idea

@vishgoel007
Copy link

vishgoel007 commented Nov 13, 2019

@tiagocorreiaitcenter I was facing the same performance issue and i wasted so many days in improving the performance of flat list. Finally i end up using recyclerlistview instead of flat list and it solved my issue.

Hope it helps.

@tiagocorreiaitcenter
Copy link
Author

@vishgoel007 The main issue it's actually fetch the entire data from database its a large chunk of data

@Kenneth-KT
Copy link
Contributor

Take a look at this PR #622
It implements SQL ORDER BY, LIMIT and OFFSET by Q.sortBy(sortColumn, sortOrder), Q.take(count) and Q.skip(count) query operators.

@awgv
Copy link

awgv commented Jun 27, 2020

@radex Hello. Considering that I know nothing about internal workings of WatermelonDB or LokiJS, and that I was thinking about the problem a bit too much, I thought it would be best to ask for the second opinion here; there’re many ways to store tree structures in relational databases (like files organized in folders, so unlike the OP, not flat) that provide different—but often comparable—performance of CRUD operations under various circumstances—do you think these different approaches would cause the same effects if applied to WatermelonDB with the LokiJS adapter?

I don’t have an immediate problem, was just wondering if I should be investing time in a more appropriate database structure instead of using something simple like the adjacency list for my tree, which might or might not bite me in the future.

@radex
Copy link
Collaborator

radex commented Jul 1, 2020

@awgv No idea, I didn't have a use case for tree structures. Worth experimenting with -- and please do share your findings!

@radex
Copy link
Collaborator

radex commented Oct 5, 2020

Closing this as there is now a possibility for sorting and limiting queries to support this.

@radex radex closed this as completed Oct 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants