requests.get returns 403 while the same url works in browser

Well that’s because default User-Agent of requests is python-requests/2.13.0, and in your case that website don’t like traffic from “non-browsers”, so they try to block such traffic.

>>> import requests
>>> session = requests.Session()
>>> session.headers
{'Connection': 'keep-alive', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'User-Agent': 'python-requests/2.13.0'}

All you need to do is to make the request appear like coming from a browser, so just add an extra header parameter:

import requests

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'} # This is chrome, you can set whatever browser you like
response = requests.get('http://www.rlsnet.ru/search_result.htm?word=%D6%E5%F0%E5%E1%F0%EE%EB%E8%E7%E8%ED', headers=headers)

print response.status_code
print response.url

200 
http://www.rlsnet.ru/search_result.htm?word=%D6%E5%F0%E5%E1%F0%EE%EB%E8%E7%E8%ED

Related Contents:

  1. How to remove curly bracket from the output in python?
  2. What is the best way to remove accents (normalize) in a Python unicode string?
  3. “Unicode Error “unicodeescape” codec can’t decode bytes… Cannot open text files in Python 3 [duplicate]
  4. UnicodeDecodeError: ‘charmap’ codec can’t decode byte X in position Y: character maps to
  5. Normalizing Unicode
  6. How to set sys.stdout encoding in Python 3?
  7. How can I convert surrogate pairs to normal string in Python?
  8. How to correct TypeError: Unicode-objects must be encoded before hashing?
  9. Facebook JSON badly encoded
  10. NameError: global name ‘unicode’ is not defined – in Python 3
  11. Upload Image using POST form data in Python-requests
  12. Using Python Requests: Sessions, Cookies, and POST
  13. Why doesn’t Python recognize my utf-8 encoded source file?
  14. Identifier normalization: Why is the micro sign converted into the Greek letter mu?
  15. Python 3: How to specify stdin encoding
  16. OpenSSL errors in python requests
  17. How to make unicode string with python3
  18. Python 3 print() function with Farsi/Arabic characters [duplicate]
  19. How to pass arguments to a Button command in Tkinter?
  20. UnicodeDecodeError when reading CSV file in Pandas with Python
  21. TypeError: unsupported operand type(s) for -: ‘str’ and ‘int’
  22. What is setup.py?
  23. Django MEDIA_URL and MEDIA_ROOT
  24. u’\ufeff’ in Python string
  25. Remove and Replace Printed items [duplicate]
  26. Python requests. 403 Forbidden
  27. How to fetch a non-ascii url with urlopen?
  28. How to resolve TypeError: can only concatenate str (not “int”) to str [duplicate]
  29. Timeout for python requests.get entire response
  30. How to get the latest file in a folder?
  31. Python – ‘ascii’ codec can’t decode byte
  32. Conditionally fill column values based on another columns value in pandas
  33. Get a list of all the encodings Python can encode to
  34. Python – TypeError: ‘int’ object is not iterable
  35. Python Requests – How to use system ca-certificates (debian/ubuntu)?
  36. Most pythonic way to interleave two strings
  37. How do I format a string using a dictionary in python-3.x?
  38. How to escape curly-brackets in f-strings? [duplicate]
  39. Microsoft Visual C++ 14.0 is required. Get it with “Microsoft Visual C++ Build Tools”: http://landinghub.visualstudio.com/visual-cpp-build-tools [duplicate]
  40. What’s the difference between “update” and “update_idletasks”?
  41. Simplest async/await example possible in Python
  42. building Python from source with zlib support
  43. pandas.io.json.json_normalize with very nested json
  44. Number rows within group in increasing order in a pandas dataframe
  45. What does print()’s `flush` do?
  46. All example concurrent.futures code is failing with “BrokenProcessPool”
  47. Getting value of enum on string conversion
  48. What is the difference between ‘content’ and ‘text’
  49. Updating OptionMenu from List
  50. Python3 project remove __pycache__ folders and .pyc files

Leave a Comment