Skip to content

Commit

Permalink
feat: add more docs (#82)
Browse files Browse the repository at this point in the history
  • Loading branch information
ethe authored Oct 26, 2024
1 parent 7233ffe commit 3e56c53
Show file tree
Hide file tree
Showing 7 changed files with 124 additions and 35 deletions.
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
# Fusio

`fusio` provides [Read](https://github.com/tonbo-io/fusio/blob/main/fusio/src/lib.rs#L63) and [Write](https://github.com/tonbo-io/fusio/blob/main/fusio/src/lib.rs#L50) traits to operate on multiple storage backends (e.g., local disk, Amazon S3) across various asynchronous runtimes—both poll-based ([tokio](https://github.com/tokio-rs/tokio)) and completion-based ([tokio-uring](https://github.com/tokio-rs/tokio-uring), [monoio](https://github.com/bytedance/monoio))—with:
<p align="center">
<a href="https://crates.io/crates/fusio">
<img alt="crates.io" src="https://img.shields.io/crates/v/fusio">
</a>

<a href="https://docs.rs/fusio/latest/fusio/">
<img alt="docs.rs" src="https://img.shields.io/docsrs/fusio">
</a>
</p>

`fusio` provides [Read](https://docs.rs/fusio/latest/fusio/trait.Read.html) and [Write](https://docs.rs/fusio/latest/fusio/trait.Write.html) traits to operate on multiple storage backends (e.g., local disk, Amazon S3) across various asynchronous runtimes—both poll-based ([tokio](https://github.com/tokio-rs/tokio)) and completion-based ([tokio-uring](https://github.com/tokio-rs/tokio-uring), [monoio](https://github.com/bytedance/monoio))—with:
- lean: binary size is at least 14× smaller than others.
- minimal-cost abstraction: compared to bare storage backends, trait definitions allow dispatching file operations without extra overhead.
- extensible: exposes traits to support implementing storage backends as third-party crates.
Expand Down
26 changes: 21 additions & 5 deletions fusio/src/buf/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Buffer abstraction for I/O operations.
mod slice;

use std::ops::{Bound, RangeBounds};
Expand All @@ -6,22 +8,34 @@ pub use slice::*;

use crate::MaybeSend;

/// # Safety
/// Completion-based I/O operations require the buffer to be pinned.
#[cfg(not(feature = "completion-based"))]
pub unsafe trait MaybeOwned {}
pub unsafe trait MaybeOwned {
//! A trait for determining whether the buffer is owned or borrowed.
//! Poll-based I/O operations require the buffer to be borrowed, while completion-based I/O
//! operations require the buffer to be owned. This trait provides a way to abstract over
//! the ownership of the buffer. Users could switch between poll-based and completion-based
//! I/O operations at compile-time by enabling or disabling the `completion-based` feature.
//!
//! # Safety
//! Do not implement this trait manually.
}
#[cfg(not(feature = "completion-based"))]
unsafe impl<T> MaybeOwned for T {}

/// # Safety
/// Completion-based I/O operations require the buffer to be pinned.
#[cfg(feature = "completion-based")]
pub unsafe trait MaybeOwned: 'static {}

#[cfg(feature = "completion-based")]
unsafe impl<T: 'static> MaybeOwned for T {}

pub trait IoBuf: Unpin + Sized + MaybeOwned + MaybeSend {
//! A poll-based I/O and completion-based I/O buffer compatible buffer.
//! The [`IoBuf`] trait is implemented by buffer types that can be used with [`crate::Read`].
//! Fusio has already implemented this trait for common buffer types
//! like `Vec<u8>`, `&[u8]`, `&mut [u8]`, `bytes::Bytes`, `bytes::BytesMut`, every buffer type
//! may be not be able to be used in all async runtimes, fusio provides compile-time safety to
//! ensure which buffer types are compatible with the async runtime.
fn as_ptr(&self) -> *const u8;

fn bytes_init(&self) -> usize;
Expand Down Expand Up @@ -60,6 +74,8 @@ pub trait IoBuf: Unpin + Sized + MaybeOwned + MaybeSend {
}

pub trait IoBufMut: IoBuf {
//! Mutable version of [`IoBuf`] which is used with [`crate::Write`].
fn as_mut_ptr(&mut self) -> *mut u8;

fn as_slice_mut(&mut self) -> &mut [u8] {
Expand Down
10 changes: 10 additions & 0 deletions fusio/src/dynamic/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Dyn compatible(object safety) version of [`Read`], [`Write`] and others.
#[cfg(feature = "fs")]
pub mod fs;

Expand All @@ -16,6 +18,11 @@ pub trait MaybeSendFuture: Future + MaybeSend {}
impl<F> MaybeSendFuture for F where F: Future + MaybeSend {}

pub trait DynWrite: MaybeSend {
//! Dyn compatible(object safety) version of [`Write`].
//! All implementations of [`Write`] has already implemented this trait.
//! Also, all implementations of [`DynWrite`] has already implemented [`Write`].
//! User should not use this trait directly.
fn write_all(
&mut self,
buf: Slice,
Expand Down Expand Up @@ -44,6 +51,9 @@ impl<W: Write> DynWrite for W {
}

pub trait DynRead: MaybeSend + MaybeSync {
//! Dyn compatible(object safety) version of [`Read`].
//! Same as [`DynWrite`].
fn read_exact_at(
&mut self,
buf: SliceMut,
Expand Down
5 changes: 5 additions & 0 deletions fusio/src/fs/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
//! This module contains the `Fs` trait, which is used to abstract file system operations across
//! different file systems.
mod options;

use std::future::Future;
Expand All @@ -14,6 +17,8 @@ pub struct FileMeta {
}

pub trait Fs: MaybeSend + MaybeSync {
//! This trait is used to abstract file system operations across different file systems.
type File: Read + Write + MaybeSend + 'static;

fn open(&self, path: &Path) -> impl Future<Output = Result<Self::File, Error>> {
Expand Down
2 changes: 2 additions & 0 deletions fusio/src/impls/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Implementations of the traits in the `fusio` crate.
pub mod buffered;
pub mod disk;
pub mod remotes;
Expand Down
102 changes: 73 additions & 29 deletions fusio/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,51 @@
mod buf;
//! Fusio is a library that provides a unified IO interface for different IO backends.
//! # Example
//! ```no_run
//! use fusio::{Error, IoBuf, IoBufMut, Read, Write};
//!
//! async fn write_without_runtime_awareness<F, B, BM>(
//! file: &mut F,
//! write_buf: B,
//! read_buf: BM,
//! ) -> (Result<(), Error>, B, BM)
//! where
//! F: Read + Write,
//! B: IoBuf,
//! BM: IoBufMut,
//! {
//! let (result, write_buf) = file.write_all(write_buf).await;
//! if result.is_err() {
//! return (result, write_buf, read_buf);
//! }
//!
//! file.close().await.unwrap();
//!
//! let (result, read_buf) = file.read_exact_at(read_buf, 0).await;
//! if result.is_err() {
//! return (result.map(|_| ()), write_buf, read_buf);
//! }
//!
//! (Ok(()), write_buf, read_buf)
//! }
//!
//! #[tokio::main]
//! async fn main() {
//! #[cfg(feature = "tokio")]
//! {
//! use tokio::fs::File;
//!
//! let mut file = File::open("foo.txt").await.unwrap();
//! let write_buf = "hello, world".as_bytes();
//! let mut read_buf = [0; 12];
//! let (result, _, read_buf) =
//! write_without_runtime_awareness(&mut file, write_buf, &mut read_buf[..]).await;
//! result.unwrap();
//! assert_eq!(&read_buf, b"hello, world");
//! }
//! }
//! ```
pub mod buf;
#[cfg(feature = "dyn")]
pub mod dynamic;
mod error;
Expand All @@ -19,10 +66,11 @@ pub use impls::*;

#[cfg(not(feature = "no-send"))]
pub unsafe trait MaybeSend: Send {
//! Considering lots of runtimes does not require `Send` for [`std::future::Future`] and
//! [`futures::stream::Stream`], we provide a trait to represent the future or stream that
//! may not require `Send`. Users could switch the feature `no-send` at compile-time to
//! disable the `Send` bound for [`std::future::Future`] and [`futures::stream::Stream`].
//! Considering lots of runtimes does not require [`std::marker::Send`] for
//! [`std::future::Future`] and [`futures_core::stream::Stream`], we provide a trait to
//! represent the future or stream that may not require [`std::marker::Send`]. Users could
//! switch the feature `no-send` at compile-time to disable the [`std::marker::Send`] bound
//! for [`std::future::Future`] and [`futures_core::stream::Stream`].
//!
//! # Safety
//! Do not implement it directly.
Expand All @@ -31,15 +79,7 @@ pub unsafe trait MaybeSend: Send {
/// # Safety
/// Do not implement it directly
#[cfg(feature = "no-send")]
pub unsafe trait MaybeSend {
//! Considering lots of runtimes does not require [`Send`] for [`std::future::Future`] and
//! [`futures::stream::Stream`], we provide a trait to represent the future or stream that
//! may not require `Send`. Users could switch the feature `no-send` at compile-time to
//! disable the `Send` bound for [`std::future::Future`] and [`futures::stream::Stream`].
//!
//! # Safety
//! Do not implement it directly
}
pub unsafe trait MaybeSend {}

#[cfg(not(feature = "no-send"))]
unsafe impl<T: Send> MaybeSend for T {}
Expand All @@ -48,19 +88,14 @@ unsafe impl<T> MaybeSend for T {}

#[cfg(not(feature = "no-send"))]
pub unsafe trait MaybeSync: Sync {
//! Same as [`MaybeSend`], but for [`Sync`].
//! Same as [`MaybeSend`], but for [`std::marker::Sync`].
//!
//! # Safety
//! Do not implement it directly
//! Do not implement it directly.
}

#[cfg(feature = "no-send")]
pub unsafe trait MaybeSync {
//! Same as [`MaybeSend`], but for [`Sync`].
//!
//! # Safety
//! Do not implement it directly
}
pub unsafe trait MaybeSync {}

#[cfg(not(feature = "no-send"))]
unsafe impl<T: Sync> MaybeSync for T {}
Expand All @@ -73,16 +108,20 @@ pub trait Write: MaybeSend {
//! because completion-based IO requires the buffer to be pinned and should be safe to
//! cancellation.
//!
//! [`Write`] represents "truncate to 0 then append all" semantics,
//! which means the file will be truncated to 0 at first,
//! and each write operation will be appended to the end of the file.
//! [`Write`] represents "sequential write all and overwrite" semantics,
//! which means each buffer will be written to the file sequentially and overwrite the previous
//! file when closed.
//!
//! Contents are not be garanteed to be written to the file until the [`Write::close`] method is
//! called, [`Write::flush`] may be used to flush the data to the file in some
//! implementations, but not all implementations will do so.
//!
//! Whether the operation is successful or not, the buffer will be returned,
//! [`fusio`] promises that the returned buffer will be the same as the input buffer.
//! fusio promises that the returned buffer will be the same as the input buffer.
//!
//! # Dyn Compatibility
//! This trait is not dyn compatible.
//! If you want to use [`Write`] trait in a dynamic way, you could use [`DynWrite`] trait.
fn write_all<B: IoBuf>(
&mut self,
Expand All @@ -101,13 +140,18 @@ pub trait Read: MaybeSend + MaybeSync {
//! because completion-based IO requires the buffer to be pinned and should be safe to
//! cancellation.
//!
//! [`Read`] represents "read exactly at" semantics,
//! which means the read operation will start at the specified position.
//! [`Read`] represents "random exactly read" semantics,
//! which means the read operation will start at the specified position, and the buffer will be
//! exactly filled with the data read.
//!
//! The buffer will be returned with the result, whether the operation is successful or not,
//! [`fusio`] promises that the returned buffer will be the same as the input buffer.
//! fusio promises that the returned buffer will be the same as the input buffer.
//!
//! If you want sequential reading, try [`SeqRead`].
//!
//! # Dyn Compatibility
//! This trait is not dyn compatible.
//! If you want to use [`Read`] trait in a dynamic way, you could use [`DynRead`] trait.
fn read_exact_at<B: IoBufMut>(
&mut self,
Expand Down
2 changes: 2 additions & 0 deletions fusio/src/path/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! A path abstraction that can be used to represent paths in a cloud-agnostic way.
use std::{fmt::Formatter, path::PathBuf};

use itertools::Itertools;
Expand Down

0 comments on commit 3e56c53

Please sign in to comment.