Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Domain transformations in batches for less memory use #5218

Merged
merged 13 commits into from
Mar 22, 2021
Merged

[ENH] Domain transformations in batches for less memory use #5218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
064120e
Benchmark domain conversions
markotoplak Oct 1, 2020
81ef4a0
benchmark: add class benchmark
markotoplak Jan 25, 2021
3bbd6ed
Remove callbacks from benchmarks
markotoplak Jan 5, 2021
e1942f1
Table.from_table: refactor caching
markotoplak Jan 28, 2021
b34a95b
refactor Table.from_table and its get_columns
markotoplak Jan 28, 2021
c9894a9
Table.from_table: domain conversion in batches
markotoplak Jan 28, 2021
80f38c4
Transformation avoids creating new domains for each transform
markotoplak Jan 5, 2021
32eec7b
_optimize_indices: optimize single column selection
markotoplak Jan 5, 2021
004847e
assure_column_dense: handle common case fast
markotoplak Jan 28, 2021
14908fb
from_table slicing test: do not block from_table_rows calls
markotoplak Jan 26, 2021
9273896
partwise from_table, adopt tests for smallest part size = 10
markotoplak Jan 28, 2021
7d8e1fd
Table.from_table: set batch size to 5000
markotoplak Jan 28, 2021
ebf5e47
map instead of list comprehension in _idcache_*
markotoplak Mar 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading