The error code 10060 means it cannot connect to the remote peer. It might be because of the network problem or mostly your setting issues, such as proxy setting. You could try to connect the same host with other tools(such as ncat) and/or with another PC within your same local network to find out where … Read more
The simplest way to limit number of concurrent connections is to use a thread pool: #!/usr/bin/env python from itertools import izip, repeat from multiprocessing.dummy import Pool # use threads for I/O bound tasks from urllib2 import urlopen def fetch(url_data): try: return url_data[0], urlopen(*url_data).read(), None except EnvironmentError as e: return url_data[0], None, str(e) if __name__==”__main__”: pool … Read more
Looks like name resolution is ultimately handled by socket.create_connection. -> urllib2.urlopen -> httplib.HTTPConnection -> socket.create_connection Though once the “Host:” header has been set, you can resolve the host and pass on the IP address through down to the opener. I’d suggest that you subclass httplib.HTTPConnection, and wrap the connect method to modify self.host before passing … Read more
The problem is that httplib.HTTPConnection uses the socket module’s create_connection helper function which does the DNS request via the usual getaddrinfo method before connecting the socket. The solution is to make your own create_connection function and monkey-patch it into the socket module before importing urllib2, just like we do with the socket class. import socks … Read more
I am pretty sure that urllib’s unquote is the common way of doing this. >>> import urllib >>> urllib.unquote(“myemail%40gmail.com”) ‘myemail@gmail.com’ There’s also unquote_plus: Like unquote(), but also replaces plus signs by spaces, as required for unquoting HTML form values.
When you download a file with urllib or urllib2, you can find out whether a charset header was transmitted: fp = urllib2.urlopen(request) charset = fp.headers.getparam(‘charset’) You can use BeautifulSoup to locate a meta element in the HTML: soup = BeatifulSoup.BeautifulSoup(data) meta = soup.findAll(‘meta’, {‘http-equiv’:lambda v:v.lower()==’content-type’}) If neither is available, browsers typically fall back to user … Read more
You can easily get D by just asking for the current URL. req = urllib2.Request(starturl, datagen, headers) res = urllib2.urlopen(req) finalurl = res.geturl() To deal with the intermediate redirects you’ll probably need to build your own opener, using HTTPRedirectHandler that records the redirects.
From the docs Note that params output from urlencode is encoded to bytes before it is sent to urlopen as data: data = urllib.parse.urlencode(d).encode(“utf-8”) req = urllib.request.Request(url) with urllib.request.urlopen(req,data=data) as f: resp = f.read() print(resp)
Basically what you do is correct. Looking at redmine docs you linked to, it seems that suffix after the dot in the url denotes type of posted data (.json for JSON, .xml for XML), which agrees with the response you get – Processing by AttachmentsController#upload as XML. I guess maybe there’s a bug in docs … Read more
Python 3 from urllib.error import HTTPError Python 2 from urllib2 import HTTPError Just catch HTTPError, handle it, and if it’s not Error 404, simply use raise to re-raise the exception. See the Python tutorial. Here is a complete example for Python 2: import urllib2 from urllib2 import HTTPError try: urllib2.urlopen(“some url”) except HTTPError as err: … Read more