Switch to Pydantic v2

[tcompa]

This is an issue to track pydantic-related updates in fractal-tasks-core dependencies, and to discuss strategies for a future v1->v2 migration. Note that this can happen at multiple levels, e.g. we could migrate the dev tools first and the models later. TBD.

First update:

• As of the (upcoming) napari 0.4.19 release, it should be possible to install pydantic v2 in the same environment (see Pydantic 2 compatibility using pydantic.v1  napari/napari#6358 or Compatibility with pydantic versions >= 2 napari/napari#6350).

[tcompa]

This issue was explored in #737. Porting the code base to Pydantic V2 was mostly successful (and it mainly consists in updating validators), but we were not satisfied with the new use of anyOf for Optional arguments/attributes.

This is a common issue, and possible workarounds exist (e.g. #737 (comment), or the recent activity in pydantic/pydantic#7161).

We could give it a second try, by adopting one of those workarounds and checking how the outcome looks both in fractal-tasks-core and in other existing packages (ref #757), while keeping an eye on possibly more standard solutions in Pydantic.

[tcompa]

Closed with #793