python's urllib2 module connected to the network with a proxy

2011-05-15  来源:本站原创  分类:Python  人气:91 

Today to download some pictures, these pictures were all the wall. To think about using python to download, of course, starting a proxy.

Search, I found that urllib and urllib2 modules are in support of the agent, but to http proxy. And I built with Tunnelier is sock5 agent, then re-use the search python sock5 agents, third-party module that is relevant, called SocksiPy.

After downloading follow the instructions to use, but has been unsuccessful, the sweat, it may be too low perception.

It seems only to find a http proxy, and we should not put forward a nginx http proxy? Toss the next, seemingly without success.

Also think of this blog wrote a sock5 agent of another transfer agent http post, which refers to the software immediately found: Privoxy

Download and install, the final version is

Privoxy 3.0.17

I downloaded this. After installation open the Options-Eidt Main Configuration

Then will use Notepad to open the configuration file, add in the bottom of this file:

forward-socks5 /

Note that the last plane has a point, the best copy.

My Tunnelier open port in 7070 sock5 agent, depending on individual circumstances change.

Save and restart Privoxy.

Http proxy at and then there is, the 8118 is the default port privoxy.

urllib2 to use a proxy:

import urllib2

proxy = '127 .0.0.1:8118 '
opener = urllib2.build_opener (urllib2.ProxyHandler ({'http': proxy}))
urllib2.install_opener (opener)
sContent = urllib2.urlopen (url). read ()
file1 = open (filepath, 'wb')
file1.write (sContent)
file1.close ()

  • python's urllib2 module connected to the network with a proxy 2011-05-15

    Today to download some pictures, these pictures were all the wall. To think about using python to download, of course, starting a proxy. Search, I found that urllib and urllib2 modules are in support of the agent, but to http proxy. And I built with

  • python urllib2 module HTTPBasicAuthHandler certification in HTTPError bug 2010-09-07

    System winxp python version 2.6.6 Use python's urllib2 module to do HTTP Basic Access Authentication certification more than six times thrown HTTPError import urllib2 import os import re import time userAgent="Mozilla/5.0 (Windows; U; Windows NT 5.1;

  • python提示No module named images的解决方法 2013-10-16

    这篇文章主要介绍了python提示No module named images的解决方法,是Python程序设计中经常遇到的问题,本文给出了具有针对性的解决方法,需要的朋友可以参考下 本文讲述了python提示No module named images的解决方法,非常实用!分享给大家供大家参考.具体方法如下: 出现提示:ImportError: No module named images 表示找不到images模块 可将: import images 替换为: import

  • python通过urllib2获取带有中文参数url内容的方法 2013-12-12

    这篇文章主要介绍了python通过urllib2获取带有中文参数url内容的方法,涉及Python中文编码的技巧,具有一定参考借鉴价值,需要的朋友可以参考下 本文实例讲述了python通过urllib2获取带有中文参数url内容的方法.分享给大家供大家参考.具体如下: 对于中文的参数如果不进行编码的话,python的urllib2直接处理会报错,我们可以先将中文转换成utf-8编码,然后使用urllib2.quote方法对参数进行url编码后传递. content = u'你好

  • python使用urllib2实现发送带cookie的请求 2014-04-12

    这篇文章主要介绍了python使用urllib2实现发送带cookie的请求,涉及Python操作cookie的相关技巧,非常具有实用价值,需要的朋友可以参考下 本文实例讲述了python使用urllib2实现发送带cookie的请求.分享给大家供大家参考.具体实现方法如下: import urllib2 opener = urllib2.build_opener() opener.addheaders.append(('Cookie','cookiename=cookievalue')) f

  • python使用urllib2模块获取gravatar头像实例 2014-06-22

    python使用urllib2模块获取gravatar头像的实例,大家参考使用吧 Gravatar注册地址: """`Gravatar <>`_""" # import code for encoding urls and generating md5 hashes import urllib2, hashl

  • Python中urllib2模块的8个使用细节分享 2015-02-03

    这篇文章主要介绍了Python中urllib2模块的8个使用细节分享,本文讲解了Proxy设置.Timeout设置.加入特定Header.Redirect.Cookie.PUT和DELETE方法等内容,需要的朋友可以参考下 Python 标准库中有很多实用的工具类,但是在具体使用时,标准库文档上对使用细节描述的并不清楚,比如 urllib2 这个 HTTP 客户端库.这里总结了一些 urllib2 库的使用细节. 1 Proxy 的设置 urllib2 默认会使用环境变量 http_proxy

  • python中from module import * 的一个坑 2015-01-28

    from module import *把module中的成员全部导到了当前的global namespace,访问起来就比较方便了.当然,python style一般不建议这么做,因为可能引起name conflict. 但还有另外一个问题 - 你以为你修改了某个变量,其实,被from module import *后的那个并没有被更新,非常危险,因为程序有可能还可以正常运行, 只不过结果错了,到了production才被发现就比较惨了. 举个例子: 你定义了一些变量在base模块中: # r

  • python错误:AttributeError: 'module' object has no attribute 'setdefaultencoding'问题的解决方法 2015-04-13

    这篇文章主要介绍了python错误:AttributeError: 'module' object has no attribute 'setdefaultencoding'问题的解决方法,需要的朋友可以参考下 Python的字符集处理实在蛋疼,目前使用UTF-8居多,然后默认使用的字符集是ascii,所以我们需要改成utf-8 查看目前系统字符集 import sys print sys.getdefaultencoding() 执行: 复制代码 代码如下: [[email protected] ~]# pyt

  • Python使用urllib2获取网络资源实例讲解 2015-04-30

    urllib2是Python的一个获取URLs(Uniform Resource Locators)的组件.他以urlopen函数的形式提供了一个非常简单的接口,下面我们用实例讲解他的使用方法 这是具有利用不同协议获取URLs的能力,他同样提供了一个比较复杂的接口来处理一般情况,例如:基础验证,cookies,代理和其他. 它们通过handlers和openers的对象提供. urllib2支持获取不同格式的URLs(在URL的":"前定义的字串,例如:"ftp"是

  • Python学习之module用法 2012-12-13

    Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module,definitions from a module can be imported into other modules or into the main module Python提供了一种可以

  • Python standard library module sys 2010-03-28

    Today to learn the "simple Python Tutorial" Chapter 14, Python standard library. Learning Address: Few characters of the first pass is also wrong, but do not know if sys

  • Radio receiver is connected to the network detection 2010-12-12

    public class ConnectivityReceiver extends BroadcastReceiver { @Override public void onReceive(Context context, Intent intent) { String action = intent.getAction(); if(action.equals(WifiManager.WIFI_STATE_CHANGED_ACTION)) { WifiManager wm = (WifiManag

  • ubuntu dual-card configurations are connected to external network 2011-01-08

    Under today's ubuntu with virtual box installation ubuntuserver10.04, with the external network connections need to install the software, but also need access to the machine, if you want to assign ip within the network also need to configure the brid

  • python通过urllib2爬网页上种子下载示例 2014-12-15

    这篇文章主要介绍了通过urllib2.re模块抓种子下载的示例,需要的朋友可以参考下 通过urllib2.re模块抓种子 思路 1.用程序登录论坛(如果需要登录才能访问的版块) 2.访问指定版块 3.遍历帖子(先取指定页,再遍历页面所有帖子的url) 4.循环访问所有帖子url,从帖子页面代码中取种子下载地址(通过正则表达式或第三方页面解析库) 5.访问种子页面下载种子 import urllib import urllib2 import cookielib import re import

  • [Transfer] Python how to use urllib2 to get network resources 2010-10-29

    urllib2 is a Python for URLs (Uniform Resource Locators) of the components. Function in the form of his urlopen provides a very simple interface, This is the use of different protocols with the ability to access URLs, he also provides a more complex

  • Python module used the information 2010-08-15

    1.os module Packing a different operating system os module common interface, allowing users on different operating systems, you can use the same interface function to return the results of the same structure. returns the current operating sy

  • The python module uuid 2010-12-14

    The python module uuid uuid is a unique identifier, for identification purposes in many areas. python's uuid module is used to generate it. Gossip did not say, python uuid generation method of providing a total of four kinds, namely: 1. From the hard

  • 零基础写python爬虫之使用urllib2组件抓取网页内容 2013-11-19

    文章详细介绍了在python2.5环境下,如何使用urllib2这个python自带的组件进行抓取指定网页内容的,整个过程记录的非常的详细,也很简单,有需要的朋友可以参考下,写出自己的python爬虫 版本号:Python2.7.5,Python3改动较大,各位另寻教程. 所谓网页抓取,就是把URL地址中指定的网络资源从网络流中读取出来,保存到本地. 类似于使用程序模拟IE浏览器的功能,把URL作为HTTP请求的内容发送到服务器端, 然后读取服务器端的响应资源. 在Python中,我们使用url

  • Python中使用urllib2防止302跳转的代码例子 2014-05-01

    这篇文章主要介绍了Python中使用urllib2防止302跳转的代码例子,即避免302跳转的实现,需要的朋友可以参考下 说明:python的urllib2获取网页(urlopen)会自动重定向(301,302).但是,有时候我们需要获取302,301页面的状态信息.就必须获取到转向前的调试信息. 下面代码将可以做到避免302重定向到新的网页 #!/usr/bin/python # -*- coding: utf-8 -*- #encoding=utf-8 #Filename:states_co