Skip to content

Commit

Permalink
Clean up config names
Browse files Browse the repository at this point in the history
  • Loading branch information
alamb committed Aug 2, 2023
1 parent f5019c9 commit dda1c81
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 55 deletions.
23 changes: 12 additions & 11 deletions datafusion/common/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -236,21 +236,22 @@ config_namespace! {
/// Defaults to the number of CPU cores on the system
pub planning_concurrency: usize, default = num_cpus::get()

/// How much memory is set aside, when using spillable sorts,
/// to ensure in-memory merge can occur. This setting has no
/// effect of the sort can not spill (there is no
/// `DiskManager` configured)
/// How much memory is set aside, for each spillable sort, to
/// ensure an in-memory merge can occur. This setting has no
/// if the sort can not spill (there is no `DiskManager`
/// configured)
///
/// Data is first sorted when spilling from memory to disk
/// when available memory is exhausted. However, this pre-sort
/// requires additional memory. To avoid trying to allocate once
/// memory is exhausted, DataFusion will set aside this many bytes.
/// As part of spilling to disk, in memory data must be sorted
/// / merged before writing the file. This in-memory
/// sort/merge requires memory as well, so To avoid allocating
/// once memory is exhausted, DataFusion sets aside this
/// many bytes before.
pub sort_spill_reservation_bytes: usize, default = 10 * 1024 * 1024

/// Below what size should data be concatenated and sorted in
/// a single RecordBatch rather than sorted in batches and merged.
/// When sorting, below what size should data be concatenated
/// and sorted in a single RecordBatch rather than sorted in
/// batches and merged.
pub sort_in_place_threshold_bytes: usize, default = 1 * 1024 * 1024

}
}

Expand Down
Loading

0 comments on commit dda1c81

Please sign in to comment.