- Implement multi-processing to improve extraction performance #20
- Handle non-union Avro types correctly during sync #18
- Only send activate version if we sent records
- Only filter files on the bookmark if the bookmark is in the files list
- Sort the files explicitly using the dump id and part number instead of using string sorting
- #16
- Handle case where there are no
incremental=false
dumps #14
- Check for a version bookmark and if it is not present then do a full table resync #12
- Updated how we read manifest files and how we choose what files to sync. Added a version bookmark and tests. #11
- Add logging of the dump id / manifest id, files, and incremental value when full sync is requested based on heap's documentation of incremental #9
- Add logging around Activate Version Messages
- Release as open beta
- Use
filter_data_by_metadata()
instead oftransform()
to speed up record throughput #7