-
Notifications
You must be signed in to change notification settings - Fork 0
/
faq.html
290 lines (263 loc) · 18.6 KB
/
faq.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
<!DOCTYPE html>
<html lang="en">
<head>
<!-- jquery refactor link -->
<script src="https://code.jquery.com/jquery-1.10.2.js"></script>
</head>
<body>
<!-- ======= Nav Bar Section & Head Tag files imported from another file ======= -->
<div id="nav-placeholder"></div>
<script>
$.get("navBar.html", function(data){
$("#nav-placeholder").replaceWith(data);
});
</script>
<!-- ======= Frequently Asked Questions Section ======= -->
<section id="faq" class="faq section-bg2">
<div class="container" data-aos="fade-in">
<div class="section-title">
<h3>Frequently Asked Questions</h3>
<p>For more info, see our <a href="#" class="link" target="_blank">research paper</a>
and <a href="#" class="link" target="_blank">open-source dataset.</a></p>
</div>
<button class="accordion"> <i class="bx bx-help-circle"></i> What are Terms and Conditions? </button>
<div class="panel">
<p>Terms and Conditions (T&C) are the <span class="emphasis"> legal agreements </span> between a service provider and people who wise to
use that service. A Terms and Condtions documentation often includes information such as use of the content (copyright) and limitation
of liability and discaimers.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> What is ClearTerms? </button>
<div class="panel">
<p>ClearTerms helps internet users better understand the policies they are implicitly agreeing to.
It uses a crowdsourcing model to highlight the terms that users find most valuable.
This helps online shoppers be more informed, and <span class="emphasis">helps consumer rights advocates
and public policy makers </span>better understand the ecommerce ecosystem.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> Why did you create ClearTerms? </button>
<div class="panel">
<p>Almost every web site has a Terms and Conditions, and they are often too long and too difficult to read. Researchers Aleecia McDonald and Lorrie Cranor calculated that
it would take a person <a href="https://www.theatlantic.com/technology/archive/2012/03/reading-the-privacy-policies-you-encounter-in-a-year-would-take-76-work-days/253851/" class="link" target="_blank">76 work days</a>
to read all the Terms & Conditions they encounter in a year. Extrapolated over the U.S. population,
it would cost <span class="emphasis">$781 billion dollars.</span> Another infographic from Reddit compares
the length of Terms & Conditions from popular sites to <a href="https://old.reddit.com/r/coolguides/comments/g5uarl/nobody_got_time_for_that/" class="link" target="_blank">classic works of literature.</a>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> What kinds of surprising statements have you found in Terms & Conditions? </button>
<div class="panel">
<p><a href="https://www.facebook.com/terms.php" class="link" target="_blank">Facebook</a> asks for
"permission to use your name, profile picture, and information about your actions with ads and
sponsored content." Airbnb is upfront about <a href="https://www.airbnb.com/terms" class="link" target="_blank">making it difficult to delete your account.</a>
Airlines such as JetBlue and Alaska Airlines often have <a href="https://millionmilesecrets.com/guides/cancellation-policies-of-major-us-airlines-online-travel-agencies/" class="link" target="_blank"></a>complicated cancellation policy.</a>
Microsoft, Sony and AT&T banned users from <a href="https://business.time.com/2012/08/28/7-surprising-things-lurking-in-online-terms-of-service-agreements/" class="link" target="_blank">filing class action lawsuits.</a>
Instagram also reserves the right to <a href="https://business.time.com/2012/08/28/7-surprising-things-lurking-in-online-terms-of-service-agreements/" class="link" target="_blank">use and publily display your photos.</a>
Individuals such as Jesse Willims, a Canadian businessman, was able to make a fortune on the Web using negative option billing and automatic subscription plan following a
<a href="https://www.theatlantic.com/magazine/archive/2014/01/the-dark-lord-of-the-internet/355726/" class="link" target="_blank">"free trial". </a>
These examples aren't the only ones - a quick Google search will turn up plenty of stories about terms like these.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> How exactly does ClearTerms work? </button>
<div class="panel">
<p>ClearTerms uses a Machine Learning model to automatically extract important statements from websites’ T&C documents.
To build the model, we collected 1,551 statements from 27 e-commerce websites, and hired 3,462 crowd workers to
manually label which ones are important to know.
<br><br>To make the labeling process easier and more robust, we employed a
method called <span class="emphasis">pairwise comparison</span>, in which crowd workers select between two
statements at a time. This pairwise comparison method not only makes the labeling process easier for crowd workers
but also generates more reliable and accurate results. We collected in total 203,068 pairwise comparison results.
<br><br>The comparison results are then fed into a <span class="emphasis">statistical model</span>
that will produce the ranking of statements, and finally, we used that result to train our model.
Our machine learning model achieved at best 92.7% balanced accuracy, 91.6% recall, and 89.2% precision.
<br><br>
<a href="#" class="link" target="_blank">See our research paper for more details</a> <br>
<a href="#" class="link" target="_blank">Explore our open-source data to learn more about ClearTerms</a>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> But everyone has different definitions of “importance,” right? </button>
<div class="panel">
<p>Not as much as you’d think. In our research, we found <span class="emphasis">high voting consistency.</span>
The voting variances of the most important & least important statements are significantly lower than the middle statements.
In other words, <span class="emphasis">the most important and least important statements are highly agreed upon.</span>
</br>
</br>
Specifically, our result showned that crowd workers had a high agreement when performing pairwise comparison tasks with a mean percent
agreement score of 0.72 (more the 4 out of 6 different workers voted a the same when being asked to choose the statement that they
think is more important for them to know). All comparisons were at least agreed by 3 out 6 crowd workers, 79.3% of the comparisons were
agreed by at least 4 workers, 41.8% were agreed by at least 5 workers, and 13.4% received unanimous agreement.
<br><br>
<a href="#" class="link" target="_blank">See our research paper for more details</a> <br>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> How was the model trained and tested? </button>
<div class="panel">
<p>
We trained the model with a Support Vector Machine (SVM) and a Grid Search on parameters C
(penalty parameter of the error term), kernel (which kernel to use) and gamma (the kernel coefficient).
The model was tested for accuracy via <span class="emphasis">5-fold cross-validation.</span> Ultimately, the model reached 86.8% accuracy,
93.2% recall, and 82.0% precision in detecting user-labelled important clauses.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> What are some interesting research findings from analyzing Terms and Conditions? </button>
<div class="panel">
<p>
We used <a href="https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test" class="link" target="_blank">Pearson’s chi-squared test</a>
to find the top 20 words most correlated with important statements in our data set. This plot suggests consumers place
high value on potential <span class="emphasis">financial loss</span>, with terms like “fee”, “price”, and “damages”.
Many words are also related to return/cancellation policies, e,g, “cancel”, “terminate”, and “original”, as are words relating
to shipping policies, e.g. “distribute”, “days”, and “shipping”.
</br>
</br>
We also explored the corrlation between readability and importance of Terms and Conditions using
<a href="https://en.wikipedia.org/wiki/Flesch–Kincaid_readability_tests" class="link" target="_blank">Flesch Reading Ease score</a> and
<a href="https://en.wikipedia.org/wiki/Flesch–Kincaid_readability_tests" class="link" target="_blank">Kendall rank corrlation coefficient </a>.
We discovered that <span class="emphasis"> there is at least medium association between the readability and the importance perceived by consumers of a statement. </span>
Most statements that had a readability score score greater than 80 - 7th grade (fairly easy to read) were considered unimportant by
consumers, while nearly half of the statement with a readability score < 20 - college graduate (very difficult to read) were in the top 25%
of importance rankings. </br>
</br>
<a href="#" class="link" target="_blank">See our research paper for more details</a> <br>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> What statement types do consumers seem to care most about? </button>
<div class="panel">
<p>
We found that statements relating to <span class="emphasis">product/services</span> - specifically those related to
returning, cancellation, and shipping policies - appeared the highest number of times, comprising a third of the
top 5 statements. For example, one site stipulates: “Unless you cancel within 14 days from today, you will be
automatically charged the full purchase fee ($88.92) 14 days from today and enrolled in our auto-ship program.”
<br><br><a href="https://www.neimanmarcus.com/c/Assistance/Privacy-Policy-Terms-of-Use-cat33940739" class="link" target="_blank">
Neiman Marcus</a> informs consumers of their “...ongoing commitment to ensure that your shopping experience...is protected,
we will prosecute all unauthorized or fraudulent transactions to the fullest extent allowed by law.”
This statement type frequency suggests that consumers place a high value on <span class="emphasis">money-related issues</span>,
which corresponds to our chi-square test results in the previous section.
<br><br> The second most frequent statement type relates to the company’s ability to <span class="emphasis">control content
generated and uploaded by the user.</span> For example, <a href="https://sgsg.samsung.com/service/agreement.php" class="link" target="_blank">Samsung</a>
states their right that “Any Submission may be used by Samsung without restriction for any purpose whatsoever, including,
without limitation, reproduction, disclosure, transmission, publication, broadcast or posting...”. Other high ranking statements
refer to the company’s right to delete user content, or to even fine users for bad reviews on review websites.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> What are the limitations of ClearTerms’ method for analyzing Terms and Conditions? </button>
<div class="panel">
<p>
We're not perfect! Some of our limitations include:
<ul>
<li>The model only works for <span class="emphasis">e-commerce</span> sites right now.</li>
<li>Though pairwise comparison lowers the cognitive load for crowd workers, pulling individual
statements out from the full document may lose some context.</li>
<li>Terms and Conditions are filled with complex jargon that is sometimes hard to understand
in the first place. It's hard for people to understand a <span class="emphasis">"worldwide license"
to use content you generate"</span> or phrases like <span class="emphasis">"in perpetutity."</span>
</li>
<li>The model produces false positive predictions with statements that contained some sensitive
keywords (e.g. "purchase", "price", "right" etc.) but did not necessarily convey important imformation.
This is due to the nature of our algorithm which largely depends on word embedding.
</li>
<li>One tradeoff with our approach is with our choice of granularity, namely individual sentences. There
may be case that some sentences refer to others, or cannot be easily understood without more context.
Those cases are not accounted for in our model.
</li>
<li>We achknowledge the potntial for bias in our model since we had a long tail distribution of contributions
from MTurkers.</li>
</ul>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> How is ClearTerms different from TOS;DR? </button>
<div class="panel">
<p>
Much like ClearTerms, <a href="https://tosdr.org/" class="link" target="_blank">TOS;DR</a> aims to analyze
the terms of service and privacy policies of major Internet sites and services. While TOS;DR relies fully
on crowd workers to do this, ClearTerms uses a natural language model to rank important statements.
There's pros and cons to both, but this allows us to <span class="emphasis">analyze more policies</span>,
and makes it easier for us to keep our analyses <span class="emphasis">up to date</span> as web sites
update their Terms and Conditions.
<br><br>We're not the only ones using ML to look at online activity! There are more projects going on
at <a href="https://www.isri.cmu.edu/" class="link" target="_blank">CMU</a>, including those in
<a href="https://www.isri.cmu.edu/people/core-faculty/sadeh-norman.html" class="link" target="_blank">Norman Sadeh's</a> lab.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> How can I help with ClearTerms? </button>
<div class="panel">
<p>
Volunteers can help by participating in <span class="emphasis">pairwise comparison</span> -
by comparing individual statements on <a href="crowd.html">this page</a>.
<br><br>If you have skills in machine learning or data science, you can check out our <a href="#" class="link" target="_blank">open-source data</a>
and look for better ways of improving our models. For example, some Terms and Conditions are highly similar
in text, probably because of templates. We're also looking into classifying the
<span class="emphasis">category</span> of the statement too. If you have any ideas, <a href="about-us.html"></a>let us know!</a>
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> Who is the team behind ClearTerms? </button>
<div class="panel">
<!-- <p>
This project was led by <a href="http://www.cs.cmu.edu/~jasonh/" target="_blank">Jason Hong</a>.
Annabel Sun and Bruce Xingyu Liu did the bulk of the work in developing
the method, collecting data from the crowd, and analyzing
the results. This web site was designed and built by
Raymond Li, Wen Shan Jiang, and Stacy Kellner. </br>
</br>
There have also been many other students who have helped
with data analysis or refining our models, including Angelia,
Arpita Agrawal, Chaiwut Chaianuchittrakul, Lu Chen, Jonathan Dinu, Jineet Doshi,
Shawn Hanna, Brandon Jiang, Jennifer Kong, Siddharth Nair, Rajat Pandey,
Alex Sciuto, Kunal Sekhri, Seoyeon Sim, Sarah Shy, Weijia Sun.
</p> -->
<p>
Click <a href="about-us.html">here</a> to view the team behind the Clearterms project.
</p>
</div>
<!-- End F.A.Q Item-->
<button class="accordion"> <i class="bx bx-help-circle"></i> Who funded ClearTerms? </button>
<div class="panel">
<p>
This research was funded in part by the National Science Foundation
(CNS-1422018). The views and conclusions contained in this web site
are those of the authors and should not be interpreted as representing
the official policies, either expressly or implied, of the National
Science Foundation or the U.S. Government.
</p>
</div>
<!-- End F.A.Q Item-->
</div>
</section><!-- End Frequently Asked Questions Section -->
<!-- ======= Footer & JS files + Firebase imported from another file ======= -->
<div id="footer-placeholder"></div>
<script>
$.get("footer.html", function(data){
$("#footer-placeholder").replaceWith(data);
});
</script>
<!-- Top of page arrow -->
<a href="#" class="back-to-top"><i class="icofont-simple-up"></i></a>
<!-- Vendor JS Files -->
<script src="assets/vendor/jquery/jquery.min.js"></script>
<script src="assets/vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<script src="assets/vendor/jquery.easing/jquery.easing.min.js"></script>
<script src="assets/vendor/php-email-form/validate.js"></script>
<script src="assets/vendor/isotope-layout/isotope.pkgd.min.js"></script>
<script src="assets/vendor/venobox/venobox.min.js"></script>
<script src="assets/vendor/aos/aos.js"></script>
<!-- Template Main JS File -->
<script src="assets/js/main.js"></script>
<!-- FIREBASE -->
<!-- The core Firebase JS SDK is always required and must be listed first -->
<script src="https://www.gstatic.com/firebasejs/7.14.1/firebase-app.js"></script>
<!-- Add SDKs for Firebase products that you want to use
https://firebase.google.com/docs/web/setup#available-libraries -->
<script src="https://www.gstatic.com/firebasejs/7.14.1/firebase-database.js"></script>
<!-- END FIREBASE -->
</body>
</html>