Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] catalog_alias Field in CleanRoomCollaborator Not Mapped Correctly #846

Open
markdegroat opened this issue Dec 30, 2024 · 0 comments

Comments

@markdegroat
Copy link

Description
The databricks-sdk does not correctly map the collaborator_alias field in the CleanRoomCollaborator class to the required catalog_alias field for the Databricks Clean Room API. This causes the following error when attempting to create a clean room:

Missing required fields: clean_room.remote_detailed_info.collaborators.catalog_alias, clean_room.remote_detailed_info.collaborators.catalog_alias

This issue prevents the SDK from being used to create clean rooms without manual intervention or subclassing.

Reproduction

  1. Attempt to create a clean room using the SDK:
from databricks.sdk import WorkspaceClient
from databricks.sdk.service.cleanrooms import CleanRoom, CleanRoomRemoteDetail, CleanRoomCollaborator

client = WorkspaceClient()

creator = CleanRoomCollaborator(
    collaborator_alias="creator_catalog",
    global_metastore_id="aws:us-west-2:creator-metastore-id"
)

collaborators = [
    creator,
    CleanRoomCollaborator(
        collaborator_alias="collaborator_catalog",
        global_metastore_id="aws:us-east-2:collaborator-metastore-id"
    )
]

remote_detail = CleanRoomRemoteDetail(
    cloud_vendor="aws",
    region="us-west-2",
    collaborators=collaborators
)

clean_room = CleanRoom(
    owner="[email protected]",
    name="test_clean_room",
    remote_detailed_info=remote_detail
)

response = client.clean_rooms.create(clean_room=clean_room)
  1. Observe the error:
Missing required fields: clean_room.remote_detailed_info.collaborators.catalog_alias

Expected behavior
The CleanRoomCollaborator class should map collaborator_alias to the API-required catalog_alias field during serialization.

Is it a regression?
Did this work in a previous version of the SDK? If so, which versions did you try?
No, this is a new feature.

Debug Logs
The SDK logs helpful debugging information when debug logging is enabled. Set the log level to debug by adding logging.basicConfig(level=logging.DEBUG) to your program, and include the logs here.
Not necessary for this I don't believe but can add more information if needed.

Other Information

  • SDK Version: 0.40.0

Additional context

Workaround:
Subclass the CleanRoomCollaborator class to correctly map the collaborator_alias field to catalog_alias. The workaround implementation looks like this:

class PatchedCleanRoomCollaborator(CleanRoomCollaborator):
    def as_dict(self):
        """Override to map 'collaborator_alias' to 'catalog_alias'."""
        body = super().as_dict()
        if 'collaborator_alias' in body:
            body['catalog_alias'] = body['collaborator_alias']
        return body

# Updated usage
creator = PatchedCleanRoomCollaborator(
    global_metastore_id="aws:us-west-2:creator-metastore-id",
    collaborator_alias="creator_catalog"
)

collaborators = [
    creator,
    PatchedCleanRoomCollaborator(
        global_metastore_id="aws:us-east-2:collaborator-metastore-id",
        collaborator_alias="collaborator_catalog"
    )
]

remote_detail = CleanRoomRemoteDetail(
    cloud_vendor="aws",
    region="us-west-2",
    collaborators=collaborators
)

clean_room = CleanRoom(
    owner="[email protected]",
    name="test_clean_room",
    remote_detailed_info=remote_detail
)

response = client.clean_rooms.create(clean_room=clean_room)

I'm currently using this workaround and confirmed this successfully creates a clean room utilizing the SDK.

Proposed Fix:
Update the SDK to automatically map the collaborator_alias field to catalog_alias during serialization in the CleanRoomCollaborator class.
Ensure SDK tests verify compatibility with the latest API requirements for clean room collaborators.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant