Skip to content

Commit

Permalink
Add new plans for spark 3.5
Browse files Browse the repository at this point in the history
  • Loading branch information
parthchandra committed Jan 11, 2025
1 parent 89eaad5 commit 8c51644
Show file tree
Hide file tree
Showing 540 changed files with 89,067 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
== Physical Plan ==
* ColumnarToRow (40)
+- CometTakeOrderedAndProject (39)
+- CometProject (38)
+- CometBroadcastHashJoin (37)
:- CometProject (33)
: +- CometBroadcastHashJoin (32)
: :- CometProject (27)
: : +- CometBroadcastHashJoin (26)
: : :- CometFilter (12)
: : : +- CometHashAggregate (11)
: : : +- CometExchange (10)
: : : +- CometHashAggregate (9)
: : : +- CometProject (8)
: : : +- CometBroadcastHashJoin (7)
: : : :- CometFilter (2)
: : : : +- CometNativeScan: `spark_catalog`.`default`.`store_returns` (1)
: : : +- CometBroadcastExchange (6)
: : : +- CometProject (5)
: : : +- CometFilter (4)
: : : +- CometNativeScan: `spark_catalog`.`default`.`date_dim` (3)
: : +- CometBroadcastExchange (25)
: : +- CometFilter (24)
: : +- CometHashAggregate (23)
: : +- CometExchange (22)
: : +- CometHashAggregate (21)
: : +- CometHashAggregate (20)
: : +- CometExchange (19)
: : +- CometHashAggregate (18)
: : +- CometProject (17)
: : +- CometBroadcastHashJoin (16)
: : :- CometFilter (14)
: : : +- CometNativeScan: `spark_catalog`.`default`.`store_returns` (13)
: : +- ReusedExchange (15)
: +- CometBroadcastExchange (31)
: +- CometProject (30)
: +- CometFilter (29)
: +- CometNativeScan: `spark_catalog`.`default`.`store` (28)
+- CometBroadcastExchange (36)
+- CometFilter (35)
+- CometNativeScan: `spark_catalog`.`default`.`customer` (34)


(1) CometNativeScan: `spark_catalog`.`default`.`store_returns`
Output [4]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3, sr_returned_date_sk#4]
Arguments: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3, sr_returned_date_sk#4]

(2) CometFilter
Input [4]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3, sr_returned_date_sk#4]
Condition : (isnotnull(sr_store_sk#2) AND isnotnull(sr_customer_sk#1))

(3) CometNativeScan: `spark_catalog`.`default`.`date_dim`
Output [2]: [d_date_sk#5, d_year#6]
Arguments: [d_date_sk#5, d_year#6]

(4) CometFilter
Input [2]: [d_date_sk#5, d_year#6]
Condition : ((isnotnull(d_year#6) AND (d_year#6 = 2000)) AND isnotnull(d_date_sk#5))

(5) CometProject
Input [2]: [d_date_sk#5, d_year#6]
Arguments: [d_date_sk#5], [d_date_sk#5]

(6) CometBroadcastExchange
Input [1]: [d_date_sk#5]
Arguments: [d_date_sk#5]

(7) CometBroadcastHashJoin
Left output [4]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3, sr_returned_date_sk#4]
Right output [1]: [d_date_sk#5]
Arguments: [sr_returned_date_sk#4], [d_date_sk#5], Inner, BuildRight

(8) CometProject
Input [5]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3, sr_returned_date_sk#4, d_date_sk#5]
Arguments: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3], [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3]

(9) CometHashAggregate
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3]
Keys [2]: [sr_customer_sk#1, sr_store_sk#2]
Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#3))]

(10) CometExchange
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#7]
Arguments: hashpartitioning(sr_customer_sk#1, sr_store_sk#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]

(11) CometHashAggregate
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#7]
Keys [2]: [sr_customer_sk#1, sr_store_sk#2]
Functions [1]: [sum(UnscaledValue(sr_return_amt#3))]

(12) CometFilter
Input [3]: [ctr_customer_sk#8, ctr_store_sk#9, ctr_total_return#10]
Condition : isnotnull(ctr_total_return#10)

(13) CometNativeScan: `spark_catalog`.`default`.`store_returns`
Output [4]: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13, sr_returned_date_sk#14]
Arguments: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13, sr_returned_date_sk#14]

(14) CometFilter
Input [4]: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13, sr_returned_date_sk#14]
Condition : isnotnull(sr_store_sk#12)

(15) ReusedExchange [Reuses operator id: 6]
Output [1]: [d_date_sk#15]

(16) CometBroadcastHashJoin
Left output [4]: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13, sr_returned_date_sk#14]
Right output [1]: [d_date_sk#15]
Arguments: [sr_returned_date_sk#14], [d_date_sk#15], Inner, BuildRight

(17) CometProject
Input [5]: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13, sr_returned_date_sk#14, d_date_sk#15]
Arguments: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13], [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13]

(18) CometHashAggregate
Input [3]: [sr_customer_sk#11, sr_store_sk#12, sr_return_amt#13]
Keys [2]: [sr_customer_sk#11, sr_store_sk#12]
Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#13))]

(19) CometExchange
Input [3]: [sr_customer_sk#11, sr_store_sk#12, sum#16]
Arguments: hashpartitioning(sr_customer_sk#11, sr_store_sk#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]

(20) CometHashAggregate
Input [3]: [sr_customer_sk#11, sr_store_sk#12, sum#16]
Keys [2]: [sr_customer_sk#11, sr_store_sk#12]
Functions [1]: [sum(UnscaledValue(sr_return_amt#13))]

(21) CometHashAggregate
Input [2]: [ctr_store_sk#17, ctr_total_return#18]
Keys [1]: [ctr_store_sk#17]
Functions [1]: [partial_avg(ctr_total_return#18)]

(22) CometExchange
Input [3]: [ctr_store_sk#17, sum#19, count#20]
Arguments: hashpartitioning(ctr_store_sk#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]

(23) CometHashAggregate
Input [3]: [ctr_store_sk#17, sum#19, count#20]
Keys [1]: [ctr_store_sk#17]
Functions [1]: [avg(ctr_total_return#18)]

(24) CometFilter
Input [2]: [(avg(ctr_total_return) * 1.2)#21, ctr_store_sk#17]
Condition : isnotnull((avg(ctr_total_return) * 1.2)#21)

(25) CometBroadcastExchange
Input [2]: [(avg(ctr_total_return) * 1.2)#21, ctr_store_sk#17]
Arguments: [(avg(ctr_total_return) * 1.2)#21, ctr_store_sk#17]

(26) CometBroadcastHashJoin
Left output [3]: [ctr_customer_sk#8, ctr_store_sk#9, ctr_total_return#10]
Right output [2]: [(avg(ctr_total_return) * 1.2)#21, ctr_store_sk#17]
Arguments: [ctr_store_sk#9], [ctr_store_sk#17], Inner, (cast(ctr_total_return#10 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#21), BuildRight

(27) CometProject
Input [5]: [ctr_customer_sk#8, ctr_store_sk#9, ctr_total_return#10, (avg(ctr_total_return) * 1.2)#21, ctr_store_sk#17]
Arguments: [ctr_customer_sk#8, ctr_store_sk#9], [ctr_customer_sk#8, ctr_store_sk#9]

(28) CometNativeScan: `spark_catalog`.`default`.`store`
Output [2]: [s_store_sk#22, s_state#23]
Arguments: [s_store_sk#22, s_state#23]

(29) CometFilter
Input [2]: [s_store_sk#22, s_state#23]
Condition : ((isnotnull(s_state#23) AND (s_state#23 = TN)) AND isnotnull(s_store_sk#22))

(30) CometProject
Input [2]: [s_store_sk#22, s_state#23]
Arguments: [s_store_sk#22], [s_store_sk#22]

(31) CometBroadcastExchange
Input [1]: [s_store_sk#22]
Arguments: [s_store_sk#22]

(32) CometBroadcastHashJoin
Left output [2]: [ctr_customer_sk#8, ctr_store_sk#9]
Right output [1]: [s_store_sk#22]
Arguments: [ctr_store_sk#9], [s_store_sk#22], Inner, BuildRight

(33) CometProject
Input [3]: [ctr_customer_sk#8, ctr_store_sk#9, s_store_sk#22]
Arguments: [ctr_customer_sk#8], [ctr_customer_sk#8]

(34) CometNativeScan: `spark_catalog`.`default`.`customer`
Output [2]: [c_customer_sk#24, c_customer_id#25]
Arguments: [c_customer_sk#24, c_customer_id#25]

(35) CometFilter
Input [2]: [c_customer_sk#24, c_customer_id#25]
Condition : isnotnull(c_customer_sk#24)

(36) CometBroadcastExchange
Input [2]: [c_customer_sk#24, c_customer_id#25]
Arguments: [c_customer_sk#24, c_customer_id#25]

(37) CometBroadcastHashJoin
Left output [1]: [ctr_customer_sk#8]
Right output [2]: [c_customer_sk#24, c_customer_id#25]
Arguments: [ctr_customer_sk#8], [c_customer_sk#24], Inner, BuildRight

(38) CometProject
Input [3]: [ctr_customer_sk#8, c_customer_sk#24, c_customer_id#25]
Arguments: [c_customer_id#25], [c_customer_id#25]

(39) CometTakeOrderedAndProject
Input [1]: [c_customer_id#25]
Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#25 ASC NULLS FIRST], output=[c_customer_id#25]), [c_customer_id#25], 100, [c_customer_id#25 ASC NULLS FIRST], [c_customer_id#25]

(40) ColumnarToRow [codegen id : 1]
Input [1]: [c_customer_id#25]

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometTakeOrderedAndProject [c_customer_id]
CometProject [c_customer_id]
CometBroadcastHashJoin [ctr_customer_sk,c_customer_sk,c_customer_id]
CometProject [ctr_customer_sk]
CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,s_store_sk]
CometProject [ctr_customer_sk,ctr_store_sk]
CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_store_sk]
CometFilter [ctr_customer_sk,ctr_store_sk,ctr_total_return]
CometHashAggregate [ctr_customer_sk,ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum,sum(UnscaledValue(sr_return_amt))]
CometExchange [sr_customer_sk,sr_store_sk] #1
CometHashAggregate [sr_customer_sk,sr_store_sk,sum,sr_return_amt]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
CometNativeScan: `spark_catalog`.`default`.`store_returns` [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
CometBroadcastExchange [d_date_sk] #2
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year]
CometNativeScan: `spark_catalog`.`default`.`date_dim` [d_date_sk,d_year]
CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_store_sk] #3
CometFilter [(avg(ctr_total_return) * 1.2),ctr_store_sk]
CometHashAggregate [(avg(ctr_total_return) * 1.2),ctr_store_sk,sum,count,avg(ctr_total_return)]
CometExchange [ctr_store_sk] #4
CometHashAggregate [ctr_store_sk,sum,count,ctr_total_return]
CometHashAggregate [ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum,sum(UnscaledValue(sr_return_amt))]
CometExchange [sr_customer_sk,sr_store_sk] #5
CometHashAggregate [sr_customer_sk,sr_store_sk,sum,sr_return_amt]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
CometNativeScan: `spark_catalog`.`default`.`store_returns` [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
ReusedExchange [d_date_sk] #2
CometBroadcastExchange [s_store_sk] #6
CometProject [s_store_sk]
CometFilter [s_store_sk,s_state]
CometNativeScan: `spark_catalog`.`default`.`store` [s_store_sk,s_state]
CometBroadcastExchange [c_customer_sk,c_customer_id] #7
CometFilter [c_customer_sk,c_customer_id]
CometNativeScan: `spark_catalog`.`default`.`customer` [c_customer_sk,c_customer_id]
Loading

0 comments on commit 8c51644

Please sign in to comment.