Write struct with binary column failed with Json error: Binary is not supported by JSON

[flxliu]

Environment

Delta-rs version:
'0.17.1'

Binding:
Python

Environment:

• Cloud provider:
• OS: Linux 20.04.1-Ubuntu
• Other:

---

Bug

What happened:
I have a struct column, which has a nest binary column. writing this strut failed.

It's easy to repro by using duckdb

import duckdb
import deltalake

result = duckdb.sql("select {'a': NULL::BLOB}").arrow()
deltalake.write_deltalake("a-rs", result)

output:
Exception: Json error: Binary is not supported by JSON

The file is generated though, it looks the parquet file itself is correct when querying it from duckdb.

What you expected to happen:
create the delta table correctly.

How to reproduce it:

run the code above

More details:

I tried to read the generated delta table using a = deltalake.DeltaTable('a-rs'). It throws error "Exception: Json error: Binary is not supported by JSON". Which seems to be #2302

[omkar-foss]

This issue is related to #2854, so I should be able resolve both these in PR #3146.

[omkar-foss]

take