Skip to content

Latest commit

 

History

History
582 lines (298 loc) · 32.1 KB

CHANGELOG.md

File metadata and controls

582 lines (298 loc) · 32.1 KB
Raw

Release Notes

0.45.0 (2024-10-27)

⚠ BREAKING CHANGES

  • spark: Spark TimestampNTZType is now emitted as Substrait PrecisionTimestamp
  • spark: Spark TimestampType is now emitted as Substrait PrecisionTimestampTZ

feat(core): added support for Expression.EmptyMapLiteral

Features

  • spark: add Window support (#307) (b3f61a2)
  • spark: additional type and literal support (#311) (513a049)
  • spark: bitwise functions (#309) (b8ccd8b)
  • spark: convert VirtualTableScan to LocalRelation (#312) (3f2cc1e)
  • spark: enable upper/lower/concat/coalesce string functions (#308) (fc8a764)

Bug Fixes

  • propagate sorts in aggregate function invocation proto->rel (#313) (75ebac2)
  • spark: nullability of output columns of expand relation (#310) (6413e55)

0.44.0 (2024-10-20)

Features

  • spark: map EqualNullSafe fn to is_not_distinct_from (#306) (357cc01)
  • spark: support conversion of boolean types (#305) (d5452ad)

0.43.0 (2024-10-13)

Features

  • spark: support for FetchRel offset field (#296) (d22c07f)

0.42.0 (2024-10-06)

⚠ BREAKING CHANGES

  • EXCEPT ALL and INTERSECT ALL now output different SetOps

Features

Miscellaneous Chores

0.41.0 (2024-09-29)

Features

  • add ExpandRel support to core and Spark (#295) (32fea18)

0.40.0 (2024-09-22)

⚠ BREAKING CHANGES

  • IntervalDay now has "subsecond" and "precision" fields instead of "microseconds". Old protobufs should be still read correctly.

Features

  • support new IntervalCompound and updated IntervalDay types (#288) (e24ce6f)

0.39.0 (2024-09-01)

Features

  • isthmus: injectable TypeConverter for window fn converters (#291) (d625648)

0.38.1 (2024-08-18)

Bug Fixes

  • core: output nullability of IfThen depends on all possible outputs (a0ca17b)
  • core: wrong type derivation for ConsistentPartitionWindow (#286) (60575b3)

0.38.0 (2024-08-11)

Features

  • make ProtoRelConverter utility methods protected (#285) (3864710)

0.37.0 (2024-07-21)

⚠ BREAKING CHANGES

  • AdvancedExtension#getOptimization() has been removed. Use getOptimizations() instead.

Features

  • literal support for precision timestamp types (#283) (94996f9)
  • validate VirtualTableScan field names with schema (#284) (0f8514a)

Miscellaneous Chores

0.36.0 (2024-07-14)

⚠ BREAKING CHANGES

  • Expression#options now returns List
  • ProtoAggregateFunctionConverter#from(AggregateFunction) now returns AggregateFunctionInvocation

Bug Fixes

  • include FunctionOptions when converting functions (#278) (e574913)

0.35.0 (2024-06-30)

Features

  • deprecate Timestamp and TimestampTZ visit functions (#273) (8a8253e)
  • introduce substrait-spark module (#271) (8537dca)

0.34.0 (2024-06-23)

⚠ BREAKING CHANGES

  • getDfsNames() has been removed from VirtualTableScan
  • getInitialSchema() not longer has a default implementation in VirtualTableScan

Bug Fixes

  • set VirtualTableScan schema explicitly (#272) (f1192cf)

0.33.0 (2024-06-16)

Features

  • isthmus: support for PrecisionTimestamp conversions (#262) (e726904)

Bug Fixes

0.32.0 (2024-06-04)

⚠ BREAKING CHANGES

  • Substrait FP32 is now mapped to Calcite REAL instead of FLOAT
  • Calcite FLOAT is now mapped to Substrait FP64 instead of FP32

In Calcite, the Sql Type Names DOUBLE and FLOAT correspond to FP64, and REAL corresponds to FP32

Bug Fixes

  • account for struct fields in VirtualTableScan check (#255) (3bbcf82)
  • map Calcite REAL to Substrait FP32 (#261) (37331c2)

0.31.0 (2024-05-05)

⚠ BREAKING CHANGES

  • isthumus: CLI related functionality is now in the io.substrait.isthmus.cli package

Features

0.30.0 (2024-04-28)

⚠ BREAKING CHANGES

  • ParameterizedTypeVisitor has new visit methods
  • TypeExpressionVisitor has new visit methods
  • TypeVisitor has new visit methods
  • BaseProtoTypes has new visit methods

Bug Fixes

  • handle FetchRels with only offset set (#252) (b334e1d)

Miscellaneous Chores

0.29.1 (2024-03-31)

Bug Fixes

  • correct function compound names for IntervalDay and IntervalYear #240 (#242) (856331b)

0.29.0 (2024-03-17)

⚠ BREAKING CHANGES

  • isthmus: method ExpressionCreator.cast(Type, Expression) has been removed

Features

0.28.1 (2024-03-10)

Bug Fixes

  • use coercive function matcher before least restrictive matcher (#237) (e7aa8ff)

0.28.0 (2024-02-25)

Features

  • isthmus: add WindowRelFunctionConverter (#234) (a5e1a21)
  • isthmus: improve signature matching for functions with wildcard arguments (#226) (ec1887c)

Bug Fixes

  • disable task caches in the gradle build (#233) (7879f2b)
  • isthmus: allow for conversion of plans containing Calcite SqlAggFunctions (#230) (0bdac49), closes #180

0.27.0 (2024-02-18)

⚠ BREAKING CHANGES

  • ExpressionVisitor now has a visit(Expression.EmptyListLiteral) method
  • LiteralConstructorConverter constructor now requires a TypeConverter

Features

  • add support for empty list literals (#227) (2a98e3c)
  • pojo: add POJO representation and converters for ConsistentPartitionWindowRel (#231) (f148bbb)
  • support for user-defined type literals (#232) (ca8187f)

0.26.0 (2024-02-11)

Features

Bug Fixes

  • update bad URLs in the release readme (#225) (aad2739)

0.25.0 (2024-01-21)

⚠ BREAKING CHANGES

  • isthmus: signatures for aggregate building utils have changed

  • feat: additional builder methods for arithmetic aggregate functions

  • feat: sortField builder method

  • feat: grouping builder method

  • feat: add, subtract, multiply, divide and negate methods for builder

  • refactor: extract row matching assertions to PlanTestBase

  • feat(isthmus): improved Calcite support for Substrait Aggregate rels

  • refactor: builder functions for aggregates and aggregate functions now consume and return Aggregate.Measure instead of AggregateFunctionInvocation

Features

  • enable conversion of SQL expressions to Substrait ExtendedExpressions (#191) (750220e)
  • improved error messages for Substrait conversion failures (#221) (#222) (8c70245)
  • isthmus: improved Calcite support for Substrait Aggregate rels (#214) (1689c93)

0.24.0 (2024-01-07)

Features

  • introduce DefaultExtensionCatalog (#217) (35f9b62)
  • isthmus: additional output formats for cli (#216) (6e62f46)

0.23.0 (2023-12-18)

Features

0.22.0 (2023-11-26)

Features

  • isthmus: support inPredicate expression (#205) (133ab83)

0.21.0 (2023-11-19)

Features

0.20.0 (2023-11-07)

⚠ BREAKING CHANGES

  • RelCopyOnWriteVisitor now extends RelVisitor and has generic type parameter

Features

Bug Fixes

  • map switch expression to a Calcite CASE statement (#189) (b938573)

0.19.0 (2023-10-29)

Features

0.18.0 (2023-09-24)

⚠ BREAKING CHANGES

  • StrLiteral is no longer converted to CHAR()

fix(isthmus): convert BinaryLiteral to VARBINARY

  • BinaryLiteral is no longer converted to BINARY)
  • calcite: Isthmus no longer uses Calcite built-in MAX, MIN, SUM, SUM0 and AVG functions
  • calcite: removed REQUIRED and NULLABLE fields from Type interface

Features

  • calcite: dedicated Substrait MAX, MIN, SUM, SUM0 and AVG functions (#180) (477b63e)
  • extend literal conversion support (#183) (6e82f39)

Bug Fixes

  • support any? type syntax in function extensions (#184) (16e5604)

0.17.0 (2023-09-17)

⚠ BREAKING CHANGES

  • EnumArgument, TypeArgument and ValueArgument are now abstract
  • visit over core substrait types (#178)

Features

  • calcite: support reading in list and map literals (#177) (e8a2645)
  • use Immutables for Argument subclasses (#179) (edfc65f)
  • visit over core substrait types (#178) (6bee452)

Bug Fixes

  • calcite: use upperBound util when converting upperBound (#176) (b90432d)

0.16.0 (2023-09-06)

⚠ BREAKING CHANGES

    • windowFunction expression creator now requires window bound type parameter

  • the WindowBound POJO representation has been reworked to use visitation and more closely match the spec

  • ExpressionRexConverter now requires a WindowFunctionConverter

    • feat: convert Substrait window functions to Calcite RexOvers (#172) (7618bb8), closes #172

0.15.0 (2023-08-20)

⚠ BREAKING CHANGES

  • pojos: various public functions that took the AggregateFunction.AggregationInvocation proto now take the POJO equivalent Expression.AggregationInvocation.

Features

  • implement fractional second intervals (#167) (68aa7c4)
  • support reading Substrait plans with Window Functions (#165) (93c6db5)

Code Refactoring

  • pojos: avoid using raw proto enums in POJO layer (#164) (41c8400)

0.14.1 (2023-08-06)

Bug Fixes

  • handle custom extensions through expressions (#161) (af91dc3)

0.14.0 (2023-07-23)

Features

  • Add SingleOrList support to the Isthmus converter (#159) (297c535)

0.13.0 (2023-07-16)

Features

  • add missing getters to ValueArgument (#158) (46f0b9f)

Bug Fixes

  • left/right/outer joins have nullable fields (#157) (b987058)

0.12.1 (2023-06-18)

Bug Fixes

  • extension merging should include types (#152) (5718537)

0.12.0 (2023-06-06)

⚠ BREAKING CHANGES

  • TypeConverter no longer uses static methods
  • SimpleExtension.MAPPER has been replaced with SimpleExtension.objectMapper(String namespace)

Features

  • handle user-defined types in Isthmus (#149) (7d7acf8)
  • support IfThen translation from Substrait to Calcite (#151) (f505c23)

0.11.0 (2023-05-28)

⚠ BREAKING CHANGES

  • rename and move extension associated code (#148)

Code Refactoring

  • rename and move extension associated code (#148) (6f29d32)

0.10.0 (2023-05-15)

Features

0.9.0 (2023-04-30)

Features

  • support for custom scalar and aggregate fns (#140) (8bd599a)

0.8.0 (2023-04-09)

⚠ BREAKING CHANGES

  • Use commonExtension to indicate that these extensions are associated with the RelCommon message
  • Use relExtension to indicate that these extensions are associated directly with Rels

Features

0.7.0 (2023-03-12)

Features

0.6.0 (2023-03-05)

Features

  • add FailureBehavior enum to Cast immutable (#115) (fd7cd5f)

Bug Fixes

  • incorrect mapping of floating point + and - ops (#131) (963c72f)

0.5.0 (2023-02-05)

Features

Bug Fixes

0.4.0 (2023-01-29)

Features

  • registered upper & lower in function mapping (#103) (c4f94db)

0.3.0 (2023-01-19)

⚠ BREAKING CHANGES

  • SubstraitRelNodeConverter constructor has changed

Features

  • Set RelBuilder directly in SubstraitRelNodeConverter (#117) (5c84515)

0.2.0 (2022-12-04)

Features

  • support parsing of SQL queries with APPLY (#106) (4c81833)

0.1.4 (2022-11-30)

Bug Fixes

  • open, close and promote from staging to maven (#112) (565e340)

0.1.3 (2022-11-25)

Bug Fixes

0.1.2 (2022-11-25)

Bug Fixes

0.1.1 (2022-11-25)

Bug Fixes

0.1.0 (2022-11-25)

Features