COPY TO api #30
COPY TO api #30
Conversation
| let query = PgList::from_pg(rewritten_queries) | ||
| .pop() | ||
| .expect("rewritten query is empty"); | ||
|
|
There was a problem hiding this comment.
minor: can probably capture the below in something like execute_query_into_destreceiver
| if copy_stmt.attlist.is_null() { | ||
| let mut col_ref = | ||
| unsafe { PgBox::<pg_sys::ColumnRef>::alloc_node(pg_sys::NodeTag::T_ColumnRef) }; | ||
| let a_star = unsafe { PgBox::<pg_sys::A_Star>::alloc_node(pg_sys::NodeTag::T_A_Star) }; |
There was a problem hiding this comment.
could use some more comments on the query we're constructing
|
|
||
| use crate::parquet_copy_hook::copy_utils::{copy_has_relation, copy_lock_mode, copy_relation_oid}; | ||
|
|
||
| pub(crate) fn execute_copy_to_with_dest_receiver( |
There was a problem hiding this comment.
could probably use a comment on what cleverness is going on here / how we can do COPY TO in a custom format with this one weird trick
| per_copy_context: MemoryContext, | ||
| } | ||
|
|
||
| static mut PARQUET_WRITER_CONTEXT_STACK: Vec<ParquetWriterContext> = vec![]; |
There was a problem hiding this comment.
kind of neat, could use a comment on why we have this :)
|
|
||
| let codec = parquet_dest.codec; | ||
|
|
||
| // create parquet writer context and push it to the stack |
There was a problem hiding this comment.
general remark: I think comments on why we do something are a lot more useful than comments on what we do, since code itself is almost always clear on the latter, but not always on the former.
We internally execute COPY TO command via parquet dest receiver to write parquet file from the table or query.
aykut-bozkurt
force-pushed
the
copy-hook-utils
branch
from
3f87cc1
to
3627d3d
Compare
aykut-bozkurt
force-pushed
the
copy-to-api
branch
from
762cf82
to
b71a602
Compare
|
Addressed at |
Add COPY TO parquet api, which internally executes COPY TO command via parquet dest receiver to write parquet file from the table or query.