HTTP: 403 Forbidden

[aswinvk28]

Anyone know or remember why the HTTP: 403 Forbidden is showing up on screen for Twitter Search, even though the browser search is fine while executing this bash script.

Parameters used for headers XMLHttpRequest

headers = [
			('Host', "twitter.com"),
			('User-Agent', "Mozilla/5.0 (Windows NT 6.1; Win64; x64)"),
			('Accept', "application/json, text/javascript, */*; q=0.01"),
			('Accept-Language', "de,en-US;q=0.7,en;q=0.3"),
			('X-Requested-With', "XMLHttpRequest"),
			('Referer', url),
			('Connection', "keep-alive")
		]

python Exporter.py --querysearch "europe refugees" --maxtweets 1 --output new.csv

python Exporter.py --querysearch "europe refugees" --maxtweets 1 --output new.csv
Searching...

Twitter weird response. Try to see on browser:  https://twitter.com/search?f=tweets&q=%20europe%20refugees&src=typd&max_position=
Twitter weird response. Try to see on browser: https://twitter.com/search?q=%20europe%20refugees&src=typd
Unexpected error: <class 'urllib.error.HTTPError'>
Done. Output file generated "new.csv".
Traceback (most recent call last):
  File "C:\Users\burse\Documents\Project-Documentation\Perceived-Stress-Questionnaire\GetOldTweets-python\got3\manager\TweetManager.py", line 130, in getJsonReponse
    response = opener.open(url)
  File "C:\Users\burse\anaconda3\envs\meshcnn\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\burse\anaconda3\envs\meshcnn\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\burse\anaconda3\envs\meshcnn\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\burse\anaconda3\envs\meshcnn\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\burse\anaconda3\envs\meshcnn\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Exporter.py", line 70, in main
    got.manager.TweetManager.getTweets(tweetCriteria, receiveBuffer)
  File "C:\Users\burse\Documents\Project-Documentation\Perceived-Stress-Questionnaire\GetOldTweets-python\got3\manager\TweetManager.py", line 21, in getTweets
    json = TweetManager.getJsonReponse(tweetCriteria, refreshCursor, cookieJar, proxy)
  File "C:\Users\burse\Documents\Project-Documentation\Perceived-Stress-Questionnaire\GetOldTweets-python\got3\manager\TweetManager.py", line 136, in getJsonReponse
    sys.exit()
SystemExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Exporter.py", line 79, in <module>
    main(sys.argv[1:])
  File "Exporter.py", line 72, in main
    except arg:
TypeError: catching classes that do not inherit from BaseException is not allowed

[mikaelamcnamara]

Hi @aswinvk28did you get this working in the end?

[anansi]

I am also getting this error for accessing a user profile with the code:

tweetCriteria = got.manager.TweetCriteria().setUsername('barackobama').setMaxTweets(1)
tweets = got.manager.TweetManager.getTweets(tweetCriteria)

Error is:

An error occured during an HTTP request: HTTP Error 403: Forbidden
Try to open in browser: https://twitter.com/search?q=%20from%3Abarackobama&src=typd

Has twitter updated its scraping detection? Please help