设为首页 加入收藏

TOP

爬虫必备—requests(一)
2017-09-30 15:29:49 】 浏览:6121
Tags:爬虫 必备 requests

Requests 是使用 Apache2 Licensed 许可证的 基于Python开发的HTTP 库,其在Python内置模块的基础上进行了高度的封装,从而使得Pythoner进行网络请求时,变得美好了许多,使用Requests可以轻而易举的完成浏览器可有的任何操作。

1. GET请求

 1 # 1、无参数实例
 2   
 3 import requests
 4   
 5 ret = requests.get('https://github.com/timeline.json')
 6   
 7 print(ret.url)
 8 print(ret.text)
 9   
10   
11   
12 # 2、有参数实例
13   
14 import requests
15   
16 payload = {'key1': 'value1', 'key2': 'value2'}
17 ret = requests.get("http://httpbin.org/get", params=payload)
18   
19 print(ret.url)
20 print(ret.text)

2. POST请求

 1 # 1、基本POST实例
 2   
 3 import requests
 4   
 5 payload = {'key1': 'value1', 'key2': 'value2'}
 6 ret = requests.post("http://httpbin.org/post", data=payload)
 7   
 8 print(ret.text)
 9   
10   
11 # 2、发送请求头和数据实例
12   
13 import requests
14 import json
15   
16 url = 'https://api.github.com/some/endpoint'
17 payload = {'some': 'data'}
18 headers = {'content-type': 'application/json'}
19   
20 ret = requests.post(url, data=json.dumps(payload), headers=headers)
21   
22 print(ret.text)
23 print(ret.cookies)

3. 其它请求

 1 requests.get(url, params=None, **kwargs)
 2 requests.post(url, data=None, json=None, **kwargs)
 3 requests.put(url, data=None, **kwargs)
 4 requests.head(url, **kwargs)
 5 requests.delete(url, **kwargs)
 6 requests.patch(url, data=None, **kwargs)
 7 requests.options(url, **kwargs)
 8   
 9 # 以上方法均是在此方法的基础上构建
10 requests.request(method, url, **kwargs)

4. 请求参数

 1 def request(method, url, **kwargs):
 2     """Constructs and sends a :class:`Request <Request>`.
 3 
 4     :param method: method for the new :class:`Request` object.
 5     :param url: URL for the new :class:`Request` object.
 6     :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
 7     :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
 8     :param json: (optional) json data to send in the body of the :class:`Request`.
 9     :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
10     :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
11     :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
12         ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
13         or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
14         defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
15         to add for the file.
16     :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
17     :param timeout: (optional) How long to wait for the server to send data
18         before giving up, as a float, or a :ref:`(connect timeout, read
19         timeout) <timeouts>` tuple.
20     :type timeout: float or tuple
21     :param allow_redirects: (optional) Boolean. Set to True if POST/PUT/DELETE redirect following is allowed.
22     :type allow_redirects: bool
23     :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.
24     :param verify: (optional) whether the SSL cert will be verified. A CA_BUNDLE path can also be provided. Defaults to ``True``.
25     :param stream: (optional) if ``False``, the response content will be immediately downloaded.
26     :param cert: (optional) if String, path to ssl client cert
首页 上一页 1 2 3 4 下一页 尾页 1/4/4
】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
上一篇Python网络爬虫与信息提取(一) 下一篇32、进程池与回调函数

最新文章

热门文章

Hot 文章

Python

C 语言

C++基础

大数据基础

linux编程基础

C/C++面试题目