-
Notifications
You must be signed in to change notification settings - Fork 112
/
README.rst
357 lines (244 loc) · 10.1 KB
/
README.rst
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
.. raw:: html
<!--
Copyright 2012-2018 Lionheart Software LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
|image0| |image1|
|Version| |Version| |License| |Versions|
Bottlenose is a thin, well-tested, maintained, and powerful Python
wrapper over the Amazon Product Advertising API. There is practically no
overhead, and no magic (unless you add it yourself).
Before you get started, make sure you have both Amazon Product
Advertising and AWS accounts. ``AWS_ACCESS_KEY_ID``,
``AWS_SECRET_ACCESS_KEY`` and ``AWS_ASSOCIATE_TAG`` are all from your
Amazon Associate Account.
Features
--------
- [x] Compatible with Python versions 2.4 and up
- [x] Support for AU, BR, CA, CN, DE, ES, FR, IN, IT, JP, MX, UK, and
US Amazon Product Advertising API endpoints
- [x] No requirements, except simplejson for Python versions before 2.6
- [x] Configurable query parsing
- [x] Configurable throttling for batches of queries
- [x] Configurable query caching
- [x] Configurable error handling and retries
Usage
-----
`pip <https://pip.pypa.io/en/stable/installing/>`__
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
pip install bottlenose
or
::
python3 -m pip install bottlenose
Then, using your ``AWS_ACCESS_KEY_ID``, ``AWS_SECRET_ACCESS_KEY``, and
``AWS_ASSOCIATE_TAG``:
.. code:: python
import bottlenose
amazon = bottlenose.Amazon(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ASSOCIATE_TAG)
response = amazon.ItemLookup(ItemId="B007OZNUCE")
You can then parse the ``response`` output to view item information.
Troubleshooting
---------------
- If you need help or would like to ask a general question, use `Stack
Overflow <http://stackoverflow.com/questions/tagged/bottlenose>`__.
Apply the ‘bottlenose’ tag to your question to get help faster.
- If you found a bug or have a feature request, open an issue.
- If you want to contribute, submit a pull request. If it’s a big
change, please open an issue first to discuss implementation.
Advanced Usage
--------------
1. Available Search Methods
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Region Endpoint
'''''''''''''''
The default Region is the US (``webservices.amazon.com``). To specify a
different endpoint simply set the Region parameter with the request. For
example to specify the French endpoint (``webservices.amazon.fr``) set
the Region parameter to ‘FR’:
.. code:: python
amazon = bottlenose.Amazon(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ASSOCIATE_TAG, Region='FR')
Supported values for the Region parameter are CA, CN, DE, ES, FR, IN,
IT, JP, UK, and US (default).
Your Amazon Product Advertising account (AWS_ASSOCIATE_TAG) mut exist
for the given endpoint or you’ll get an HTTP 400 error (‘Bad Request’).
Search for a Specific Item
''''''''''''''''''''''''''
.. code:: python
response = amazon.ItemLookup(ItemId="B007OZNUCE")
Search for Items by Keywords
''''''''''''''''''''''''''''
.. code:: python
response = amazon.ItemSearch(Keywords="Kindle 3G", SearchIndex="All")
Search for Images for an item
'''''''''''''''''''''''''''''
.. code:: python
response = amazon.ItemLookup(ItemId="1449372422", ResponseGroup="Images")
Search for Similar Items
''''''''''''''''''''''''
.. code:: python
response = amazon.SimilarityLookup(ItemId="B007OZNUCE")
2. Available Shopping Related Methods
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Required
''''''''
.. code:: python
amazon = bottlenose.Amazon(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ASSOCIATE_TAG)
Create a cart
'''''''''''''
.. code:: python
response = amazon.CartCreate(...)
Adding to a cart
''''''''''''''''
.. code:: python
response = amazon.CartAdd(CartId, ...)
Get a cart by ID
''''''''''''''''
.. code:: python
response = amazon.CartGet(CartId, ...)
Modifying a cart
''''''''''''''''
.. code:: python
response = amazon.CartModify(ASIN, CartId,...)
Clearing a cart
'''''''''''''''
.. code:: python
response = amazon.CartClear(CartId, ...)
3. Sample Code
^^^^^^^^^^^^^^
.. code:: python
import bottlenose
amazon = bottlenose.Amazon(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ASSOCIATE_TAG)
response = amazon.ItemLookup(ItemId="0596520999", ResponseGroup="Images",
SearchIndex="Books", IdType="ISBN")
print(response)
# <?xml version="1.0" ?><ItemLookupResponse xmlns="http://webservices.amazon...
Here is another example.
.. code:: python
response = amazon.ItemSearch(Keywords="Kindle 3G", SearchIndex="All")
# <?xml version="1.0" ?><ItemSearchResponse xmlns="http://webservices.amazon...
Bottlenose can also read your credentials from the environment
automatically; just set ``$AWS_ACCESS_KEY_ID``,
``$AWS_SECRET_ACCESS_KEY`` and ``$AWS_ASSOCIATE_TAG``.
Any valid API call from the following is supported (in addition to any
others that may be added in the future). Just plug in appropriate
request parameters for the operation you’d like to call, and you’re good
to go.
::
BrowseNodeLookup
CartAdd
CartClear
CartCreate
CartGet
CartModify
ItemLookup
ItemSearch
SimilarityLookup
You can refer here for a full listing of API calls to be made from
Amazon. - `Amazon API Quick Reference
Card <http://s3.amazonaws.com/awsdocs/Associates/latest/prod-adv-api-qrc.pdf>`__
--------------
For more information about these calls, please consult the `Product
Advertising API Developer
Guide <http://docs.aws.amazon.com/AWSECommerceService/latest/DG/Welcome.html>`__.
Parsing
-------
By default, API calls return the response as a raw bytestring. You can
change this with the ``Parser`` constructor argument. The parser is a
callable that takes a single argument, the response as a raw bytestring,
and returns the parsed response in a format of your choice.
For example, to parse responses with BeautifulSoup:
.. code:: python
import bottlenose
from bs4 import BeautifulSoup
amazon = bottlenose.Amazon(
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ASSOCIATE_TAG,
Parser=lambda text: BeautifulSoup(text, 'xml')
)
results = amazon.ItemLookup(ItemId="0198596790", ResponseGroup="SalesRank")
print(results.find('SalesRank').string)
# 168088
Throttling/Batch Mode
---------------------
Amazon strictly limits the query rate on its API (by default, one query
per second per associate tag). If you have a batch of non-urgent
queries, you can use the ``MaxQPS`` argument to limit them to no more
than a certain rate; any faster, and bottlenose will ``sleep()`` until
it is time to make the next API call.
Generally, you want to be just under the query limit, for example:
.. code:: python
amazon = bottlenose.Amazon(MaxQPS=0.9)
If some other code is also querying the API with your associate tag (for
example, a website backend), you’ll want to choose an even lower value
for MaxQPS.
Caching
-------
You can often get a major speedup by caching API queries. Use the
``CacheWriter`` and ``CacheReader`` constructor arguments.
``CacheWriter`` is a callable that takes two arguments, a cache url, and
the raw response (a bytestring). It will only be called after successful
queries.
``CacheReader`` is a callable that takes a single argument, a cache url,
and returns a (cached) raw response, or ``None`` if there is nothing
cached.
The cache url is the actual query URL with authentication information
removed. For example:
::
http://webservices.amazon.com/onca/xml?Keywords=vacuums&Operation=ItemSearch&Region=US&ResponseGroup=SearchBins&SearchIndex=All&Service=AWSECommerceService&Version=2013-08-01
Example code:
.. code:: python
def write_query_to_db(cache_url, data):
...
def read_query_from_db(cache_url):
...
amazon = bottlenose.Amazon(CacheWriter=write_query_to_db,
CacheReader=read_query_from_db)
Note that Amazon’s `Product Advertising API
Agreement <https://affiliate-program.amazon.com/gp/advertising/api/detail/agreement.html>`__
only allows you to cache queries for up to 24 hours.
Error Handling
--------------
Sometimes the Amazon API returns errors; for example, if you have gone
over your query limit, you’ll get a 503. The ``ErrorHandler``
constructor argument gives you a way to keep track of such errors, and
to retry queries when you receive a transient error.
``ErrorHandler`` should be a callable that takes a single argument, a
dictionary with these keys:
- api_url: the actual URL used to call the API
- cache_url: ``api_url`` minus authentication information
- exception: the exception raised (usually an ``HTTPError`` or
``URLError``)
If your ``ErrorHandler`` returns true, the query will be retried. Here’s
some example code that does exponential backoff after throttling:
.. code:: python
import random
import time
from urllib2 import HTTPError
def error_handler(err):
ex = err['exception']
if isinstance(ex, HTTPError) and ex.code == 503:
time.sleep(random.expovariate(0.1))
return True
amazon = bottlenose.Amazon(ErrorHandler=error_handler)
License
-------
Apache License, Version 2.0. See `LICENSE <LICENSE>`__ for details.
.. |image0| image:: meta/repo-banner.png
.. |image1| image:: meta/repo-banner-bottom.png
:target: https://lionheartsw.com/
.. |Version| image:: https://img.shields.io/travis/lionheart/bottlenose.svg?style=flat
:target: https://travis-ci.org/lionheart/bottlenose
.. |Version| image:: https://img.shields.io/pypi/v/bottlenose.svg?style=flat
:target: https://pypi.python.org/pypi/bottlenose
.. |License| image:: https://img.shields.io/pypi/l/bottlenose.svg?style=flat
:target: LICENSE
.. |Versions| image:: https://img.shields.io/pypi/pyversions/bottlenose.svg?style=flat
:target: https://pypi.python.org/pypi/bottlenose