Questions about test of scene reconstruction

[TLB-MISS]

Thank you for providing good work!

When inferring with 3D-FRONT data, results are only at the object level.
As in the paper, how can I get the scene reconstruction result considering the spatial relationship between each object in the scene?

Additionally, I wonder why occ is needed. I think the occ serves as a template, so doesn't that eliminate the need for PIFu here?

[HaolinLiu97]

Hi, for the scene reconstruction results, you can put the results at the location that is predicted by the object reconstruction module. You can directly run the demo to obtain a scene reconstruction results. Can you specify more about 'why occ is needed'? The training of PIFu requires occ. Does it means that some other reconstruction method can be used instead of using PIFu based method?

[TLB-MISS]

@UncleMEDM
Thank you for kind reply! Let me be more specific.

You can directly run the demo to obtain a scene reconstruction results.

Oh, Sorry for my misunderstanding.

Can you specify more about 'why occ is needed'? The training of PIFu requires occ.

Of course, PIFu train requires occ. I wondered why the test needed that data.
The code below or demo will not run(error occurred) if the occ is not in place.
python main.py --mode test --config ./configs/test_instPIFu.yaml

Thank you!

[HaolinLiu97]

Hi, the testing actually does not need the occ. But I reused the dataloader that is used during the training. I will add a flag in the dataloader so that it can disable loading occ.

[TLB-MISS]

Thank you!

I want to inference web image by using the InstPIFu trained on the 3D FRONT dataset. However, I'm not sure how InstPIFu can be applied for web images. When preprocessing a 3D FRONT dataset, it seems to assume that there is a file called "desc.json" by default. However, these files do not exist for web images.

Is there any way to inference InstPIFu from web RGB image?(If we need depth, we can assume we can get depth.)

[HaolinLiu97]

Hi, I will consider to add the function of testing arbitary images, but it may need several days to write the codes. I may update the codes in next week.

[TLB-MISS]

@UncleMEDM

I would really appreciate it if you could update the code!
Thank you again for kindly accepting my suggestions.

[TLB-MISS]

Hi! @UncleMEDM

Sorry, but could you tell me when the code will be updated?

[miiddb]

Hello @UncleMEDM, can you at least list the steps for testing arbitary images?

[HaolinLiu97]

Sorry for the late reply. Actually there are only a few things need to be done.
First, run faster rcnn or some other advanced 2d object detection framework to generate some proposals with object class.
Second, run the 3d detection model as I provided to obtain 3d bounding box for each 2d proposal.
Third, run the object reconstruction.