Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Corner case it #82

Open
wants to merge 100 commits into
base: main
Choose a base branch
from
Open

Corner case it #82

wants to merge 100 commits into from

Conversation

pawankashyapollion
Copy link
Collaborator

No description provided.

taherkl and others added 30 commits January 23, 2025 16:21
@taherkl @pawankashyapollion
Dml integration (#53)
f101d40
@pawankashyapollion
Added extensive UT
8706ad2 
Added extensive UT
@pawankashyapollion
Cassandra pr bug fixes (#57)
25bef48
@pawankashyapollion
Cassandra Consolidate Unit Test case and Regression testing fixes (#58)
d1526f5 
* Added Mapping fixes

* Added Spoltles fixes

* Added Consolidated fixes

* Added TODO

* Addess Data and Time
@pawankashyapollion
Cassandra pr bug fixes (#64)
75da86c
@akashthawaitcc @pawankashyapollion
Added load test for the Spanner to cassandra flow
d6e25e2
@pawankashyapollion
Added IT for Spanner to Cassandra SOurce DB
5bd8ec5
@pawankashyapollion
Added All Premitive datatypes
79f6a66
@pawankashyapollion
Added Error Category
16b0ce4
@pawankashyapollion
Squashed commit of the following:
757c7f2 
commit b7747f9
Author: nasir19noor <[email protected]>
Date:   Sun Jan 19 16:20:37 2025 +0700

    revert back java-pr.yml

commit a4d4184
Author: nasir19noor <[email protected]>
Date:   Sun Jan 19 00:34:02 2025 +0700

    test all

commit 6397cb7
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 23:50:54 2025 +0700

    build & Unit testing add -pl v2/spanner-to-sourcedb

commit 9fd29c4
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 23:18:53 2025 +0700

    build & Unit testing

commit 08a4bed
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 23:06:23 2025 +0700

    build test

commit 1444644
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 17:03:18 2025 +0700

    IT and LT

commit c65882c
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 16:25:21 2025 +0700

    test Integration Testing revert to initial code

commit 4485e9c
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 14:53:41 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbIT

commit 6f0241d
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 14:43:02 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbInterleaveMultiShardIT change hotsIP

commit ed8a496
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 14:05:53 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbInterleaveMultiShardIT increase timeout

commit e0f613d
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 13:32:31 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbInterleaveMultiShardIT

commit ae7a17d
Author: nasir19noor <[email protected]>
Date:   Sat Jan 18 12:24:00 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbDatatypeIT

commit 5f474a2
Author: nasir19noor <[email protected]>
Date:   Fri Jan 17 21:59:31 2025 +0700

    test Integration Testing -Dtest=SpannerToSourceDbCustomShardIT

commit 80d258a
Author: nasir19noor <[email protected]>
Date:   Fri Jan 17 14:18:33 2025 +0700

    test Integration Testing

commit 8b06229
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 23:27:36 2025 +0700

    update bigquery value

commit 8ac600b
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 23:21:41 2025 +0700

    update bigquery value

commit e900037
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 22:16:48 2025 +0700

    revert back BigQuery conf

commit 543c73a
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 22:05:56 2025 +0700

    modify export bigquery

commit f89e3de
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 21:23:19 2025 +0700

    solve bigquery tabl

commit 3551c59
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 20:28:11 2025 +0700

    solve bigquery issue

commit eac6cca
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 19:39:35 2025 +0700

    decrease streaming data

commit 51a4e2e
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 17:03:36 2025 +0700

    load test: add specific module:spanner-to-cassandra 2

commit dfcb9de
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 16:46:27 2025 +0700

    load test: add specific module:spanner-to-cassandra

commit 2229233
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 16:07:38 2025 +0700

    Load Test: revert back SpannerToSourceDbCustomShardLT.java

commit 5eab9b6
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 16:03:07 2025 +0700

    Load Test: reduce dataflow jobs and workder node 2 modify format

commit 29a071f
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 15:39:59 2025 +0700

    Load Test: reduce dataflow jobs and workder node

commit 964cc52
Author: taherkl <[email protected]>
Date:   Thu Jan 16 11:41:07 2025 +0530

    update load test with lower config

commit dfc315b
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 12:35:32 2025 +0700

    modify load test

commit cedd703
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 08:55:51 2025 +0700

    IT use command form google(modified)

commit 7ba3c38
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 08:10:38 2025 +0700

    IT use command form google(modified)

commit a1dcae7
Author: nasir19noor <[email protected]>
Date:   Thu Jan 16 00:08:45 2025 +0700

    use IT command from Google

commit 3c8b375
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 23:45:16 2025 +0700

    modify run it test main.go

commit 93e3d21
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 23:41:32 2025 +0700

    modify host IP

commit 70481f2
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 23:32:05 2025 +0700

    modify IT to initial

commit bbb61cd
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 23:27:47 2025 +0700

    modify IT to initial

commit 40d23bf
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 23:24:47 2025 +0700

    modify IT to initial

commit c6334db
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 22:47:45 2025 +0700

    modify IT command 4

commit a0b6ded
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 22:33:49 2025 +0700

    modify IT command 3

commit dbfec41
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 22:00:12 2025 +0700

    modify IT command 2

commit b6f5761
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 21:43:30 2025 +0700

    modify IT command

commit 409200b
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 21:16:55 2025 +0700

    exclude build and unit test

commit c19e1a0
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 20:46:29 2025 +0700

    exclude tags perf on load test

commit 1e0bf31
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 19:36:13 2025 +0700

    exclude build dependency

commit bf1c3d7
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 18:01:33 2025 +0700

    modify the format

commit 783f00f
Author: nasir19noor <[email protected]>
Date:   Wed Jan 15 17:08:30 2025 +0700

    modify it test main.go

commit b019935
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 16:50:27 2025 +0700

    revert back it-flags.go

commit f10b6ce
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 16:28:59 2025 +0700

    add spanner-demo instance id

commit 49942e7
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 16:20:08 2025 +0700

    temp - exclude unit test dependency

commit c6278b5
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 14:58:49 2025 +0700

    include mysql cred

commit c5b8103
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 08:21:35 2025 +0700

    exclude unnecessary parameter, only spanner is requird

commit ca1ea25
Author: Nasir Noor <[email protected]>
Date:   Wed Jan 15 00:30:30 2025 +0700

    change spanner instance id spanner-demo

commit 1db2c82
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 22:27:26 2025 +0700

    change spanner instance id

commit bea9422
Author: nasir19noor <[email protected]>
Date:   Tue Jan 14 21:02:47 2025 +0700

    remove comment

commit 362443f
Author: nasir19noor <[email protected]>
Date:   Tue Jan 14 19:01:45 2025 +0700

    test codecov

commit 54907b0
Author: nasir19noor <[email protected]>
Date:   Tue Jan 14 18:54:17 2025 +0700

    remove backup file

commit b1543cf
Author: nasir19noor <[email protected]>
Date:   Tue Jan 14 18:50:59 2025 +0700

    remove backup file

commit 09dd1e1
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 17:53:45 2025 +0700

    modify project, bucket in IT

commit 8ec0f2a
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 15:48:13 2025 +0700

    v2/spanner-to-sourcedb project for IT

commit 63449d9
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 14:29:43 2025 +0700

    v2/spanner-to-sourcedb project

commit 603d28f
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 12:37:48 2025 +0700

    spanner-to-sourcedb project

commit f786250
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 12:23:05 2025 +0700

    handle only spanner-to-sourcedb project

commit 220d8fb
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 11:30:33 2025 +0700

    update Run IT Smoke tests ./cicd/run-it-smoke-tests \

commit 4bb4028
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 10:40:49 2025 +0700

    add v2/spanner-to-sourcedb/.mvn

commit 69463e6
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 10:08:14 2025 +0700

    update integration test work only spanner-to-sourcedb

commit 6cedb8a
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 08:02:22 2025 +0700

    remove modules googlecloud-to-googleclou

commit 552a171
Author: Nasir Noor <[email protected]>
Date:   Tue Jan 14 07:50:47 2025 +0700

    remove modules keep spanner-to-sourcedb

commit f8a7f5b
Author: nasir19noor <[email protected]>
Date:   Fri Jan 10 17:06:33 2025 +0700

    exclude v1

commit 0e35d9e
Author: nasir19noor <[email protected]>
Date:   Fri Jan 10 15:52:05 2025 +0700

    revert back UT and IT

commit 20371fb
Author: nasir19noor <[email protected]>
Date:   Fri Jan 10 14:14:33 2025 +0700

    modify integration smoke test steps to use own bucket, project

commit 47ae923
Author: nasir19noor <[email protected]>
Date:   Fri Jan 10 08:53:57 2025 +0700

    revert back surefire version

commit 954d5ec
Author: nasir19noor <[email protected]>
Date:   Thu Jan 9 20:33:31 2025 +0700

    remove execution block v1/pom.xml

commit 38747bf
Author: nasir19noor <[email protected]>
Date:   Thu Jan 9 16:33:52 2025 +0700

    modify surefire version

commit d38a591
Author: nasir19noor <[email protected]>
Date:   Thu Jan 9 13:46:15 2025 +0700

    remove java-pr copy.yml

commit bf2ec2b
Author: nasir19noor <[email protected]>
Date:   Thu Jan 9 13:06:50 2025 +0700

    update workflow to work only for spanner-to-sourcedb

commit b0d7724
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 23:08:27 2025 +0700

    revert back

commit 0ff10d6
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 16:19:26 2025 +0700

    surefire plugin testing

commit 6232d27
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 16:12:21 2025 +0700

    surefire plugin testing

commit 6ed8195
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 15:14:15 2025 +0700

    revert back main.go

commit 9a0c4a2
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 15:10:42 2025 +0700

    modify cicd/cmd/run-unit-test/main.go for testing

commit db75fa5
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 12:27:39 2025 +0700

    revert back

commit b85f4ce
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 09:30:59 2025 +0700

    increase memory for run unit test stage

commit f5a6599
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 08:23:30 2025 +0700

    revert back

commit 0708902
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 08:15:31 2025 +0700

    revert back run-unit-test

commit 62965d2
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 08:12:20 2025 +0700

    modify run-unit-test line 45

commit 8d6e777
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 08:10:14 2025 +0700

    modify run-unit-test line 43

commit 284603c
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 08:06:29 2025 +0700

    modify run-unit-test

commit 394c477
Author: nasir19noor <[email protected]>
Date:   Wed Jan 8 07:49:29 2025 +0700

    modify maven surefire plugin

commit d87ccfc
Author: nasir19noor <[email protected]>
Date:   Tue Jan 7 23:27:38 2025 +0700

    modify maven registry

commit 7076172
Author: taherkl <[email protected]>
Date:   Tue Jan 7 18:34:18 2025 +0530

    actions change

commit ef5ae8d
Merge: ae3037a 23be7bc
Author: Taher Lakdawala <[email protected]>
Date:   Tue Jan 7 11:23:23 2025 +0530

    Merge pull request #48 from GoogleCloudPlatform/main

    Metadata config and pipeline options (GoogleCloudPlatform#2081)

commit ae3037a
Merge: 0d140e6 0a9fcec
Author: Taher Lakdawala <[email protected]>
Date:   Tue Jan 7 11:05:26 2025 +0530

    Merge pull request #46 from GoogleCloudPlatform/main

    Sync main branch

commit 0d140e6
Merge: 407a593 5b0c2cd
Author: Taher Lakdawala <[email protected]>
Date:   Thu Jan 2 15:19:25 2025 +0530

    Merge pull request #40 from GoogleCloudPlatform/main

    Sync main branch

commit 407a593
Merge: e53e8e4 587fc05
Author: Taher Lakdawala <[email protected]>
Date:   Mon Dec 30 10:54:16 2024 +0530

    Merge pull request #36 from GoogleCloudPlatform/main

    Sync main branch

commit e53e8e4
Merge: f618128 177a218
Author: Akash Thawait <[email protected]>
Date:   Thu Dec 26 10:54:05 2024 +0530

    Merge pull request #32 from GoogleCloudPlatform/main

    Sync main branch

commit f618128
Merge: 04bac39 8e7ae4f
Author: Akash Thawait <[email protected]>
Date:   Mon Dec 23 16:38:26 2024 +0530

    Merge pull request #29 from GoogleCloudPlatform/main

    Use [self-hosted, it] for prepare java cache workflow (GoogleCloudPlatform#2080)

commit 04bac39
Merge: 5aa21dc edee6a7
Author: Taher Lakdawala <[email protected]>
Date:   Fri Dec 20 17:51:12 2024 +0530

    Merge pull request #28 from GoogleCloudPlatform/main

    Sync main branch

commit 5aa21dc
Merge: 7512825 1bae9ad
Author: Taher Lakdawala <[email protected]>
Date:   Wed Dec 18 18:18:41 2024 +0530

    Merge pull request #26 from GoogleCloudPlatform/main

    tests: Adding Forward Migration Tests (GoogleCloudPlatform#2001)

commit 7512825
Merge: 71a6477 8479c8a
Author: Taher Lakdawala <[email protected]>
Date:   Tue Dec 17 17:59:51 2024 +0530

    Merge pull request #24 from GoogleCloudPlatform/main

    Sync main branch

commit 71a6477
Merge: d314c70 cd28d01
Author: Taher Lakdawala <[email protected]>
Date:   Mon Dec 16 18:33:54 2024 +0530

    Merge pull request #20 from GoogleCloudPlatform/main

    Sync main branch

commit d314c70
Merge: 15b621d 91b5be3
Author: Taher Lakdawala <[email protected]>
Date:   Mon Dec 9 12:02:07 2024 +0530

    Merge pull request #9 from GoogleCloudPlatform/main

    Sync master

commit 15b621d
Merge: 1bef5ff b68fe21
Author: taherkl <[email protected]>
Date:   Wed Nov 27 20:19:54 2024 +0530

    Merge remote-tracking branch 'upstream/main'
@pawankashyapollion
Added Dependecy Fixes
b7c675b
@pawankashyapollion
Added Depedency
4591f3f
@pawankashyapollion
exclude driver
b240efa
@pawankashyapollion
Removed Unwanted LOG Integration
488f7c2
@pawankashyapollion
Added Template Class Fixes
e8788de
@akashthawaitcc @pawankashyapollion
Updated command on java-pr.yml
f2f5d20
@pawankashyapollion
reverse merge with main and load Resourfce
fddc24a
@pawankashyapollion
Added Logger
02ae150
@pawankashyapollion
Added keyspace Name issue fixex
9c69aba
@pawankashyapollion
Removed Static Conatiner Code
c4ca7c4
@pawankashyapollion
Added Custom Class
d6fe940
@pawankashyapollion
Added Missing Code
0decc30
@pawankashyapollion
Added Dummy commit
b41f09e
@pawankashyapollion
Added Source type
8d6ede3
@pawankashyapollion
Added Logger
3d41b6b
@pawankashyapollion
Added Logger
6d4f7db
@pawankashyapollion
Added More Logs
a1d58fe
@pawankashyapollion
Ignore MYSQL IT
ae8ee3c
@pawankashyapollion
Added Table name fixes
102cf3d
@taherkl @pawankashyapollion
Dml integration (#53)
750c1d1
pawankashyapollion and others added 29 commits January 24, 2025 13:35
@pawankashyapollion
Fix tst case
266d05e
@pawankashyapollion
Added System.out.println(row.getFormattedContents());
cc4c278
@pawankashyapollion
Added LT fixes
be6f3d2
@pawankashyapollion
Added Load test Issue fixes
822dddb
@akashthawaitcc
removed other load test execution
ce91b54
@akashthawaitcc
fixed mvn loadtest command
e98c9d9
@akashthawaitcc
run only loadtest
5dd7fb8
@akashthawaitcc
fix: workflows file
cd248bf
@akashthawaitcc
fixed: spotless
0645ce8
@akashthawaitcc
fix schema name
bd357b2
@akashthawaitcc
fix code.
4d86c3a
@akashthawaitcc
fix: cassandra resource manager
819c1be
@akashthawaitcc
changed to cassandra resource manager
0298ac1
@akashthawaitcc
removed static containter
5550cbf
@akashthawaitcc
removed setting up keyspace name
c23b529
@akashthawaitcc
fix: spotless issue
186a8ea
@akashthawaitcc
added code to create keyspace
1f583d5
@akashthawaitcc
fixed code
277aac0
@akashthawaitcc
fixed issue
1d72277
@akashthawaitcc
make cassandra host static
a3a6206
@akashthawaitcc
make port static
5a2ff3f
@akashthawaitcc
removed static port
7434d59
@akashthawaitcc
changed load test numbers
4004d29
@pawankashyapollion
Code Refectored
a883ed5
@pawankashyapollion
Handle Dependecny Fixes
207f92c
@pawankashyapollion
Enabled Load Test
ef3dafd
@pawankashyapollion
DUMMY commit
aedb39d
@pawankashyapollion
Enabled UT dependecy
b6fc780
@pawankashyapollion
Workflow Fixes
1f3aae1
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 41.66667% with 7 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@e3f4369). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...v2/templates/dbutils/dml/CassandraTypeHandler.java 37.50% 3 Missing and 2 partials ⚠️
...cloud/teleport/v2/templates/SpannerToSourceDb.java 0.00% 2 Missing ⚠️
Additional details and impacted files 
@@           Coverage Diff           @@
##             main      #82   +/-   ##
=======================================
  Coverage        ?   51.20%           
  Complexity      ?      734           
=======================================
  Files           ?      221           
  Lines           ?    12186           
  Branches        ?     1212           
=======================================
  Hits            ?     6240           
  Misses          ?     5524           
  Partials        ?      422
Components Coverage Δ
spanner-templates 71.15% <0.00%> (?)
spanner-import-export ∅ <0.00%> (?)
spanner-live-forward-migration 81.84% <0.00%> (?)
spanner-live-reverse-replication 77.91% <0.00%> (?)
spanner-bulk-migration 82.21% <0.00%> (?)
Files with missing lines Coverage Δ
...2/templates/dbutils/dml/CassandraDMLGenerator.java 90.55% <100.00%> (ø)
...leport/v2/templates/transforms/SourceWriterFn.java 85.21% <ø> (ø)
...2/templates/utils/CassandraSourceSchemaReader.java 90.90% <ø> (ø)
...cloud/teleport/v2/templates/SpannerToSourceDb.java 0.00% <0.00%> (ø)
...v2/templates/dbutils/dml/CassandraTypeHandler.java 79.48% <37.50%> (ø)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@pawankashyapollion @codecov-commenter @taherkl @akashthawaitcc @fbiville @nasir19noor