-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
195 lines (172 loc) · 8.56 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<title>AISDiff</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- <base href="/"> -->
<!-- <link rel="apple-touch-icon" href="apple-touch-icon.png"> -->
<!-- <link rel="icon" type="image/png" href="icon.png"> -->
<!-- Place favicon.ico in the root directory -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.css">
<link rel="stylesheet" href="css/app.css">
<link rel="stylesheet" href="css/bootstrap.min.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.5.3/clipboard.min.js"></script>
<script src="js/app.js"></script>
</head>
<body>
<div class="container" id="main">
<div class="row">
<h1 class="col-md-12 text-center">
Amodal Instance Segmentation with </br> Diffusion Shape Prior Estimation</br>
<small>
ACCV 2024
</small>
</h1>
</div>
<div class="row">
<div class="col-md-12 text-center">
<ul class="list-inline">
<li>
<a href="https://trqminh.github.io/">
Minh Tran
</a>
</br>University of Arkansas
</li>
<li>
<a href="https://vhvkhoa.github.io/">
Khoa Vo
</a>
</br>University of Arkansas
</li>
<li>
<a href="https://www.linkedin.com/in/tringn/">
Tri Nguyen
</a>
</br>Coupang, Inc.
</li>
<li>
<a href="https://www.uark-aicv.github.io/">
Ngan Le
</a>
</br>University of Arkansas
</li>
</ul>
</div>
</div>
<div class="row">
<div class="col-md-8 col-md-offset-2 text-center">
<ul class="nav nav-pills nav-justified">
<li>
<a href="https://arxiv.org/pdf/2409.18256">
<image src="paper.png" height="120px"><br>
<h4><strong>Paper</strong></h4>
</a>
</li>
<!-- <li>
<a href="https://youtu.be/jFubeqxJO6U">
<image src="youtube_icon.png" height="120px"><br>
<h4><strong>Technical Video</strong></h4>
</a>
</li> -->
<!-- <li>
<a href="https://github.com/UARK-AICV/AISDiff">
<image src="github_icon.png" height="120px"><br>
<h4><strong>Code</strong></h4>
</a>
</li> -->
</ul>
</div>
</div>
<!-- <div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Example Results
</h3>
<img src="teaser_noisy.gif" style="max-height: 20em; max-width:32%">
<img src="teaser_kpn.png" style="max-height: 20em; max-width:32%">
<img src="teaser_bpn.png" style="max-height: 20em; max-width:32%">
</div>
</div> -->
<div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Abstract
</h3>
<p class="text-justify">
Amodal Instance Segmentation (AIS) presents an intriguing challenge, including the segmentation prediction of both visible and occluded parts of objects within images. Previous methods have often relied on shape prior information gleaned from training data to enhance amodal segmentation. However, these approaches are susceptible to overfitting and disregard object category details. Recent advancements highlight the potential of conditioned diffusion models, pretrained on extensive datasets, to generate images from latent space. Drawing inspiration from this, we propose AISDiff with a Diffusion Shape Prior Estimation (DiffSP) module. AISDiff begins with the prediction of the visible segmentation mask and object category, alongside occlusion-aware processing through the prediction of occluding masks. Subsequently, these elements are inputted into our DiffSP module to infer the shape prior of the object. DiffSP utilizes conditioned diffusion models pretrained on extensive datasets to extract rich visual features for shape prior estimation. Additionally, we introduce the Shape Prior Amodal Predictor, which utilizes attention-based feature maps from the shape prior to refine amodal segmentation. Experiments across various AIS benchmarks demonstrate the effectiveness of our AISDiff.
</p>
<image src="pipeline2/pipeline2-1.png" class="img-responsive" alt="overview"><br>
</div>
</div>
<div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Proposed Modules
</h3>
<img src="diffusion_shapeprior/diffusion_shapeprior-1.png" style="max-height: 30em; max-width:62%">
<img src="amodal_predictor/amodal_predictor-1.png" style="max-height: 30em; max-width:34%">
<p class="text-justify">
<b>Left:</b> Overall process of Diffusion Shape Prior Estimation (DiffSP),<b>Right:</b> Overall design of Shape Prior Amodal Predictor.
</p>
</div>
</div>
<div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Qualitative Results
</h3>
<image src="quali_aisdiff.png" class="img-responsive" alt="overview"><br>
<p class="text-justify">
Qualitative results of AISDiff. Left to right: Input RoI, Visible masks, Occluding masks, Amodal masks.
</p>
</div>
</div>
<!-- <div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Technical Video
</h3>
<div class="text-center">
<div style="position:relative;padding-top:56.25%;">
<iframe src="https://www.youtube.com/embed/jFubeqxJO6U" allowfullscreen style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe>
</div>
</div>
</div>
</div> -->
<div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
BibTeX
</h3>
<div class="form-group col-md-10 col-md-offset-1">
<textarea id="bibtex" class="form-control" readonly>
@article{tran2024amodal,
title={Amodal Instance Segmentation with Diffusion Shape Prior Estimation},
author={Tran, Minh and Vo, Khoa and Nguyen, Tri and Le, Ngan},
journal={arXiv preprint arXiv:2409.18256},
year={2024}
}
</textarea>
</div>
</div>
</div>
<div class="row">
<div class="col-md-8 col-md-offset-2">
<h3>
Acknowledgements
</h3>
This work is sponsored by the National Science Foundation (NSF) under Award No OIA-1946391. The website template was borrowed from <a href="http://mgharbi.com/">Michaël Gharbi</a>.
</p>
</div>
</div>
</div>
</body>
</html>