Think about reality-simulation matching

[David-Estevez]

Once we have a simulation, we will need to match the real garment to the simulation.

[David-Estevez]

It might be interesting to check out Volumetric GANs if a GAN-approach is selected for this issue: tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow

https://ge.in.tum.de/publications/tempogan/

[David-Estevez]

3D machine learning state of the art, useful for this issue: https://github.com/timzhang642/3D-Machine-Learning

[David-Estevez]

In these works they say something about tracking constraints (might be useful):

• Adaptive Cloth Simulations | Two Minute Papers #14
• Physics-inspired Upsampling for Cloth Simulation in Games

[David-Estevez]

Tracking Deformable Objects with Point Clouds - John Schulman, Alex Lee, Jonathan Ho, Pieter Abbeel

Abstract

We introduce an algorithm for tracking deformable objects from a sequence of point clouds. The proposed tracking algorithm is based on a probabilistic generative model that incorporates observations of the point cloud and the physical properties of the tracked object and its environment. We propose a modified expectation maximization algorithm to perform maximum a posteriori estimation to update the state estimate at each time step. Our modification makes it practical to perform the inference through calls to a physics simulation engine. This is significant because (i) it allows for the use of highly optimized physics simulation engines for the core computations of our tracking algorithm, and (ii) it makes it possible to naturally, and efficiently, account for physical constraints imposed by collisions, grasping actions, and material properties in the observation updates. Even in the presence of the relatively large occlusions that occur during manipulation tasks, our algorithm is able to robustly track a variety of types of deformable objects, including ones that are one-dimensional, such as ropes; two- dimensional, such as cloth; and three-dimensional, such as sponges. Our implementation can track these objects in real time.