Skip to content

Commit

Permalink
Minor: Improve documentation for PartitionStream and `StreamingTabl…
Browse files Browse the repository at this point in the history
…eExec` (apache#8035)

* Minor: Improve documentation for  `PartitionStream` and `StreamingTableExec`

* fmt

* fmt
  • Loading branch information
alamb authored Nov 3, 2023
1 parent 8c42d94 commit 2906a24
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions datafusion/physical-plan/src/streaming.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.

//! Execution plan for streaming [`PartitionStream`]
//! Generic plans for deferred execution: [`StreamingTableExec`] and [`PartitionStream`]

use std::any::Any;
use std::sync::Arc;
Expand All @@ -35,6 +35,10 @@ use futures::stream::StreamExt;
use log::debug;

/// A partition that can be converted into a [`SendableRecordBatchStream`]
///
/// Combined with [`StreamingTableExec`], you can use this trait to implement
/// [`ExecutionPlan`] for a custom source with less boiler plate than
/// implementing `ExecutionPlan` directly for many use cases.
pub trait PartitionStream: Send + Sync {
/// Returns the schema of this partition
fn schema(&self) -> &SchemaRef;
Expand All @@ -43,7 +47,10 @@ pub trait PartitionStream: Send + Sync {
fn execute(&self, ctx: Arc<TaskContext>) -> SendableRecordBatchStream;
}

/// An [`ExecutionPlan`] for [`PartitionStream`]
/// An [`ExecutionPlan`] for one or more [`PartitionStream`]s.
///
/// If your source can be represented as one or more [`PartitionStream`]s, you can
/// use this struct to implement [`ExecutionPlan`].
pub struct StreamingTableExec {
partitions: Vec<Arc<dyn PartitionStream>>,
projection: Option<Arc<[usize]>>,
Expand Down

0 comments on commit 2906a24

Please sign in to comment.