zhouzhongping/wiki
1
0
Fork 0
Code
920b29a889
wiki/Technology/ProgrammingLanguage/Python/模块/网络处理/requests-HTTP-库.html

452 lines
512 KiB
HTML
Raw Normal View History

Deploy HTML files - 2024-09-06
2024-09-06 10:58:26 +08:00
<!DOCTYPE html>
<html lang="zh"><head><title>requests HTTP 库</title><meta charset="utf-8"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com"/><link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM Plex Mono&amp;family=Noto Serif Simplified Chinese:wght@400;700&amp;family=Source Sans Pro:ital,wght@0,400;0,600;1,400;1,600&amp;display=swap"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta property="og:title" content="requests HTTP 库"/><meta property="og:description" content="requests HTTP 库，基于 urllib 封装。."/><meta property="og:image" content="https://wiki.7wate.com/static/og-image.png"/><meta property="og:width" content="1200"/><meta property="og:height" content="675"/><link rel="icon" href="../../../../../static/icon.png"/><meta name="description" content="requests HTTP 库，基于 urllib 封装。."/><meta name="generator" content="Quartz"/><link href="../../../../../index.css" rel="stylesheet" type="text/css" spa-preserve/><link href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.16.9/katex.min.css" rel="stylesheet" type="text/css" spa-preserve/><script src="../../../../../prescript.js" type="application/javascript" spa-preserve></script><script type="application/javascript" spa-preserve>const fetchData = fetch("../../../../../static/contentIndex.json").then(data => data.json())</script></head><body data-slug="Technology/ProgrammingLanguage/Python/模块/网络处理/requests-HTTP-库"><div id="quartz-root" class="page"><div id="quartz-body"><div class="left sidebar"><h2 class="page-title"><a href="../../../../..">🪴 X·Eden</a></h2><div class="spacer mobile-only"></div><div class="search"><button class="search-button" id="search-button"><p>搜索</p><svg role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 19.9 19.7"><title>Search</title><g class="search-path" fill="none"><path stroke-linecap="square" d="M18.5 18.3l-5.4-5.4"></path><circle cx="8" cy="8" r="7"></circle></g></svg></button><div id="search-container"><div id="search-space"><input autocomplete="off" id="search-bar" name="search" type="text" aria-label="搜索些什么" placeholder="搜索些什么"/><div id="search-layout" data-preview="true"></div></div></div></div><button class="darkmode" id="darkmode"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" id="dayIcon" x="0px" y="0px" viewBox="0 0 35 35" style="enable-background:new 0 0 35 35" xml:space="preserve" aria-label="暗色模式"><title>暗色模式</title><path d="M6,17.5C6,16.672,5.328,16,4.5,16h-3C0.672,16,0,16.672,0,17.5 S0.672,19,1.5,19h3C5.328,19,6,18.328,6,17.5z M7.5,26c-0.414,0-0.789,0.168-1.061,0.439l-2,2C4.168,28.711,4,29.086,4,29.5 C4,30.328,4.671,31,5.5,31c0.414,0,0.789-0.168,1.06-0.44l2-2C8.832,28.289,9,27.914,9,27.5C9,26.672,8.329,26,7.5,26z M17.5,6 C18.329,6,19,5.328,19,4.5v-3C19,0.672,18.329,0,17.5,0S16,0.672,16,1.5v3C16,5.328,16.671,6,17.5,6z M27.5,9 c0.414,0,0.789-0.168,1.06-0.439l2-2C30.832,6.289,31,5.914,31,5.5C31,4.672,30.329,4,29.5,4c-0.414,0-0.789,0.168-1.061,0.44 l-2,2C26.168,6.711,26,7.086,26,7.5C26,8.328,26.671,9,27.5,9z M6.439,8.561C6.711,8.832,7.086,9,7.5,9C8.328,9,9,8.328,9,7.5 c0-0.414-0.168-0.789-0.439-1.061l-2-2C6.289,4.168,5.914,4,5.5,4C4.672,4,4,4.672,4,5.5c0,0.414,0.168,0.789,0.439,1.06 L6.439,8.561z M33.5,16h-3c-0.828,0-1.5,0.672-1.5,1.5s0.672,1.5,1.5,1.5h3c0.828,0,1.5-0.672,1.5-1.5S34.328,16,33.5,16z M28.561,26.439C28.289,26.168,27.914,26,27.5,26c-0.828,0-1.5,0.672-1.5,1.5c0,0.414,0.168,0.789,0.439,1.06l2,2 C28.711,30.832,29.086,31,29.5,31c0.828,0,1.5-0.672,1.5-1.5c0-0.414-0.168-0.789-0.439-1.061L28.561,26.439z M17.5,29 c-0.829,0-1.5,0.672-1.5,1.5v3c0,0.828,0.671,1.5,1.5,1.5s1.5-0.672,1.5-1.5v-3C19,29.672,18.329,29,17.5,29z M17.5,7 C11.71,7,7,11.71,7,17.5S11.71,28,17.5,28S28,23.29,28,17.5S23.29,7,17.5,7z M17.5,25c-4.136,0-7.5-3.364-7.5-7.5 c0-4.136,3.364-7.5,7.5-7.5c4.136,0,7.5,3.364,7.5,7.5C25,21.636,21.636,25,17.5,25z"></path></svg><svg xmlns="http://www.w3
<h2 id="概述">概述<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#概述" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h2>
<p>**Requests 库的目标就是让 HTTP 请求变得简单而又 Pythonic。**它可以显著减少程序员发送 HTTP 请求的工作量。使用 Requests，你不必手动添加查询字符串到 URL 中，或 encode post 数据了。一切都自动完成。</p>
<h3 id="requests-vs-urllib">Requests Vs Urllib<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#requests-vs-urllib" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Python 内置的 urllib 模块也可以发送网络请求，但其 API 不够优雅简洁。与 urllib 相比，Requests 更加 Pythonic，而且更简单易用。</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 使用 urllib 获取一个网页的代码</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.request</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.request.urlopen(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//www.python.org'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">html </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response.read()</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 使用 Requests 获取同一个页面的代码</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//www.python.org'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) </span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">html </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response.text</span></span>
<span data-line> </span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># Requests 允许使用 params 关键字传递参数，数据自动编码</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">payload </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">} </span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//httpbin.org/get'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">params</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">payload)</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 而通过 urllib 则要手动编码</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.parse</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.request</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//httpbin.org/get'</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> +</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> '?'</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> +</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.parse.urlencode(payload)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">resp </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urllib.request.urlopen(url)</span></span></code></pre></figure>
<p>总之，Requests 相对于 urllib 更加简洁易用。</p>
<div class="table-container"><table><thead><tr><th style="text-align:left;">对比项</th><th style="text-align:left;">requests</th><th style="text-align:left;">urllib</th></tr></thead><tbody><tr><td style="text-align:left;">发送请求方法</td><td style="text-align:left;">简洁的 requests.get()/post()</td><td style="text-align:left;">较复杂的 urllib.request.urlopen()</td></tr><tr><td style="text-align:left;">参数传递</td><td style="text-align:left;">自动编码，直接传 dict</td><td style="text-align:left;">需要手动 urlencode</td></tr><tr><td style="text-align:left;">请求头</td><td style="text-align:left;">直接传 dict 作为 headers</td><td style="text-align:left;">通过 Request 类设置</td></tr><tr><td style="text-align:left;">响应内容</td><td style="text-align:left;">多属性访问 text/content/json/raw 等</td><td style="text-align:left;">仅 response.read()</td></tr><tr><td style="text-align:left;">编码支持</td><td style="text-align:left;">自动编码</td><td style="text-align:left;">需要手动编码</td></tr><tr><td style="text-align:left;">连接池</td><td style="text-align:left;">支持连接池和会话</td><td style="text-align:left;">不支持</td></tr><tr><td style="text-align:left;">异常处理</td><td style="text-align:left;">提供多种请求相关异常</td><td style="text-align:left;">仅 urllib.error 异常</td></tr><tr><td style="text-align:left;">证书验证</td><td style="text-align:left;">通过 verify 参数验证 SSL 证书</td><td style="text-align:left;">通过 context 参数验证</td></tr><tr><td style="text-align:left;">代理设置</td><td style="text-align:left;">支持通过 proxies 参数</td><td style="text-align:left;">较复杂的 ProxyHandler</td></tr><tr><td style="text-align:left;">Cookies</td><td style="text-align:left;">提供 cookie 参数</td><td style="text-align:left;">通过 cookielib 模块管理</td></tr><tr><td style="text-align:left;">重定向</td><td style="text-align:left;">自动处理，可通过 max_redirects 配置</td><td style="text-align:left;">需要手动处理</td></tr><tr><td style="text-align:left;">基本认证</td><td style="text-align:left;">通过 auth 参数</td><td style="text-align:left;">通过 HTTPSHandler</td></tr><tr><td style="text-align:left;">流请求</td><td style="text-align:left;">内置支持</td><td style="text-align:left;">需要自定义</td></tr><tr><td style="text-align:left;">异步请求</td><td style="text-align:left;">支持异步模式</td><td style="text-align:left;">不支持异步</td></tr></tbody></table></div>
<h3 id="requests-的关键特性">Requests 的关键特性<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#requests-的关键特性" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<ul>
<li>继承了 urllib 的所有特性</li>
<li>支持 HTTP 连接保持和连接池，提高效率</li>
<li>支持使用 cookie 跟踪会话</li>
<li>支持文件上传</li>
<li>支持自动解码内容</li>
<li>支持国际化的 URL 和 POST 数据自动编码</li>
<li>更加 Pythonic 的 API</li>
<li>连接超时设置</li>
<li>支持 HTTPS 请求，SSL 证书验证</li>
<li>自动解压</li>
<li>流下载</li>
<li>支持基本/摘要式的身份认证</li>
</ul>
<h2 id="基础用法">基础用法<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#基础用法" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h2>
<h3 id="构造请求">构造请求<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#构造请求" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># requests.get 用于获取页面信息，</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//www.example.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) </span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># requests.post 用于提交 POST 请求。</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.post(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//httpbin.org/post'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">data</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> =</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">：</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">})</span></span></code></pre></figure>
<h3 id="获取响应">获取响应<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#获取响应" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 获取响应的内容使用 text 属性</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">html </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response.text</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 获取二进制响应内容使用 content 属性</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">png_data </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response.content</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 获取 JSON 响应使用 json() 方法</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">json_data </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response.json()</span></span></code></pre></figure>
<h3 id="获取响应状态码">获取响应状态码<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#获取响应状态码" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>获取响应状态码，可以检查 response.status_code：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(response.status_code)</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">200</span></span></code></pre></figure>
<p>Requests 还提供了一个内置的状态码查询对象 requests.codes。例如：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(requests.codes.ok) </span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">200</span></span></code></pre></figure>
<h3 id="请求参数">请求参数<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#请求参数" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>向请求中传入参数，有以下几种方法：</p>
<ol>
<li>通过 params 参数传入键值对</li>
</ol>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">payload </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//httpbin.org/get'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">params</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">payload) </span></span></code></pre></figure>
<ol>
<li>通过字典直接作为 params 参数传入</li>
</ol>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">params </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value2'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//httpbin.org/get'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， params)</span></span></code></pre></figure>
<ol>
<li>通过 url 中的查询字符串传递参数</li>
</ol>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//httpbin.org/get?key1=val1&amp;key2=val2'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(url)</span></span></code></pre></figure>
<h3 id="设置请求头">设置请求头<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#设置请求头" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>可以通过 headers 参数设置 HTTP 请求头，例如：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//httpbin.org/get'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">headers </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'user-agent'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'my-app/0.0.1'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(url， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">headers</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">headers)</span></span></code></pre></figure>
<h3 id="响应内容">响应内容<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#响应内容" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>对于响应内容，有多种属性供访问：</p>
<div class="table-container"><table><thead><tr><th style="text-align:left;">属性</th><th style="text-align:left;">说明</th></tr></thead><tbody><tr><td style="text-align:left;">r.text</td><td style="text-align:left;">字符串形式的响应体，会自动解码</td></tr><tr><td style="text-align:left;">r.content</td><td style="text-align:left;">字节形式的响应体，可迭代</td></tr><tr><td style="text-align:left;">r.json()</td><td style="text-align:left;">将 JSON 响应转换为字典</td></tr><tr><td style="text-align:left;">r.raw</td><td style="text-align:left;">原始响应体，需要自行解码</td></tr><tr><td style="text-align:left;">r.encoding</td><td style="text-align:left;">响应体编码方式</td></tr><tr><td style="text-align:left;">r.status_code</td><td style="text-align:left;">HTTP 响应状态码</td></tr><tr><td style="text-align:left;">r.headers</td><td style="text-align:left;">响应头部的字典</td></tr><tr><td style="text-align:left;">r.request</td><td style="text-align:left;">请求的 Request 对象</td></tr><tr><td style="text-align:left;">r.url</td><td style="text-align:left;">请求的 URL</td></tr><tr><td style="text-align:left;">r.history</td><td style="text-align:left;">请求的重定向信息</td></tr></tbody></table></div>
<p>例如：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//api.github.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(r.text) </span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 字符串形式的响应体</span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(r.content) </span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 字节形式的响应体，可迭代 </span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(r.json()) </span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># JSON格式转换为字典 </span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(r.raw) </span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># 返回原始响应体</span></span></code></pre></figure>
<h2 id="高级用法">高级用法<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#高级用法" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h2>
<p>Requests 还提供了很多高级功能，极大地丰富了这一模块的使用场景。</p>
<h3 id="会话维持">会话维持<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#会话维持" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Requests 提供了 session 对象，用于实现会话维持：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">s </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.Session()</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">s.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'http：//httpbin.org/cookies/set/sessioncookie/123456789'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> s.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;http：//httpbin.org/cookies&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(r.text)</span></span>
<span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># '{&quot;cookies&quot;： {&quot;sessioncookie&quot;： &quot;123456789&quot;}}'</span></span></code></pre></figure>
<p><strong>默认的 requests 函数并不会在同一个 session 中保持 cookie</strong>，所以它不会在跨请求保持状态。要保持会话，就需要使用 session 对象。</p>
<h3 id="ssl-证书验证">SSL 证书验证<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#ssl-证书验证" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Requests 可以验证 SSL 证书，你可以指定一个本地证书用作客户端证书，以完成客户端验证：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">resp </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//example.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">verify</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'path/to/certfile'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span></code></pre></figure>
<p>或者你也可以指定一个本地证书作为 CA 证书 BUNDLE，来验证请求的 TLS 服务端证书：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">resp </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//example.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">verify</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'path/to/cacert.pem'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span></code></pre></figure>
<h3 id="代理设置">代理设置<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#代理设置" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>使用代理也很简单：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">proxies </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;http&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;http：//10.10.1.10：3128&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;https&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;http：//10.10.1.10：1080&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;http：//example.org&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">proxies</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">proxies)</span></span></code></pre></figure>
<p>你也可以通过环境变量 HTTP_PROXY 和 HTTPS_PROXY 配置代理。</p>
<h3 id="超时设置">超时设置<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#超时设置" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>通过 timeout 参数，可以告诉 requests 等待服务器响应的超时时间，以秒为单位：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//github.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">timeout</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">0.001</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span></code></pre></figure>
<p>分别为连接超时 connect timeou t 和读取超时 read timeout：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//github.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">timeout</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">3.05</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">10</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">))</span></span></code></pre></figure>
<p>好的，文章继续：</p>
<h3 id="异常处理">异常处理<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#异常处理" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Requests 的异常类型主要分为以下几类：</p>
<ul>
<li><strong>连接异常</strong>：包括 RequestsConnectionError 和 ConnectTimeout，表示与远程服务器的连接发生错误。</li>
<li><strong>超时异常</strong>：RequestsTimeout 表示请求超时。可以分为连接超时和读取超时。</li>
<li><strong>TooManyRedirects</strong>：表示重定向次数超过了最大限制（默认为 30 次）。</li>
<li><strong>HTTP 错误</strong>：HTTPError 表示 HTTP 错误响应，例如 404 或者 500 等。Requests 会自动为其封装异常。</li>
<li><strong>请求异常</strong>：RequestException 是 Requests 库自身的异常基类。</li>
<li><strong>SSL 错误</strong>：SSLError 表示 SSL 证书验证错误。</li>
<li><strong>代理错误</strong>：ProxyError 表示代理连接失败。</li>
<li><strong>数据解析错误</strong>：JSONDecodeError 和 DecodeError 表示响应数据解析错误。</li>
<li><strong>其他</strong>：ConnectionError、InvalidURL 等其他异常。</li>
</ul>
<p>可以通过 try except 语句捕获这些 <a href="../../../../../Tech/programming-language/Python/进阶/异常处理" class="internal alias" data-slug="Tech/programming-language/Python/进阶/异常处理">异常</a>：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">try</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">：</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'https：//httpbin.org/delay/10'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">timeout</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">2</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">except</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.ConnectTimeout：</span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'Connection timed out'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) </span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">except</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.ConnectionError：</span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'Connection error'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span></code></pre></figure>
<p>如果不捕获异常，程序会中断并抛出异常。</p>
<div class="table-container"><table><thead><tr><th style="text-align:left;">异常类型</th><th style="text-align:left;">说明</th></tr></thead><tbody><tr><td style="text-align:left;">RequestsConnectionError</td><td style="text-align:left;">网络连接错误</td></tr><tr><td style="text-align:left;">ConnectTimeout</td><td style="text-align:left;">连接超时错误</td></tr><tr><td style="text-align:left;">RequestsTimeout</td><td style="text-align:left;">请求超时错误</td></tr><tr><td style="text-align:left;">TooManyRedirects</td><td style="text-align:left;">重定向次数超限</td></tr><tr><td style="text-align:left;">HTTPError</td><td style="text-align:left;">HTTP 错误响应</td></tr><tr><td style="text-align:left;">RequestException</td><td style="text-align:left;">请求异常基类</td></tr><tr><td style="text-align:left;">SSLError</td><td style="text-align:left;">SSL 证书验证错误</td></tr><tr><td style="text-align:left;">ProxyError</td><td style="text-align:left;">代理连接错误</td></tr><tr><td style="text-align:left;">JSONDecodeError</td><td style="text-align:left;">JSON 解析错误</td></tr><tr><td style="text-align:left;">ConnectionError</td><td style="text-align:left;">连接错误</td></tr></tbody></table></div>
<h3 id="流式下载">流式下载<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#流式下载" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>对于大文件下载，可以使用流模式节省内存：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">with</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'http：//httpbin.org/stream/100'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">stream</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">True</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">as</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> r：</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> for</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> chunk </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">in</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> r.iter_content(</span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">chunk_size</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">1024</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)： </span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(chunk)</span></span></code></pre></figure>
<p>该模式仅当你在迭代时才会持续下载响应体部分，如果你要多次读取响应，必须使用 r.content 访问内容。</p>
<h3 id="连接重试">连接重试<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#连接重试" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>可以通过设置 retries 参数，让请求在遇到连接错误时自动重试指定次数：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.adapters </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> HTTPAdapter</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">s </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.Session()</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">retries </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> Retry(</span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">total</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">5</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">backoff_factor</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">1</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">status_forcelist</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">[</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">502</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">503</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">504</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">])</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">s.mount(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'http：//'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， HTTPAdapter(</span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">max_retries</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">retries)) </span></span></code></pre></figure>
<p>如果响应状态码是 502/503/504，该请求会重试最多 5 次。</p>
<h2 id="实践技巧">实践技巧<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#实践技巧" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h2>
<h3 id="文件上传">文件上传<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#文件上传" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Requests 使得文件上传变得极其简单：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//httpbin.org/post'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">files </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'file'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">open</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'report.pdf'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'rb'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)}</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.post(url， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">files</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">files)</span></span></code></pre></figure>
<p>我们只需要在传递的字典中设置好文件名和文件对象即可，Requests 会帮你正确编码并发送。</p>
<h3 id="获取图片">获取图片<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#获取图片" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>由于图片也是一种二进制数据，所以获取图片可以这么写：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//images.pexels.com/photos/1562477/pexels-photo-1562477.jpeg'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">r </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.get(url)</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">with</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> open</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'image.jpeg'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'wb'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">as</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> f：</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> f.write(r.content)</span></span></code></pre></figure>
<p>图片内容保存在 r.content 中，我们可以直接 write 到文件。</p>
<h3 id="prepared-request">Prepared Request<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#prepared-request" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>如果要一次构造同一个请求发送多次，可以使用 Prepared Request：</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//httpbin.org/post'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">data </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'key'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">：</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'value'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">headers </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'User-Agent'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">： </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'my-app'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">request </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.Request(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">'POST'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">， url， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">data</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">data， </span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">headers</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">headers)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">prepared_request </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> request.prepare()</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">s </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.Session()</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">response </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> s.send(prepared_request)</span></span></code></pre></figure>
<h2 id="异步-requests">异步 Requests<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#异步-requests" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h2>
<h3 id="基于-gevent-的异步">基于 Gevent 的异步<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#基于-gevent-的异步" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> gevent</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> gevent </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> monkey</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">monkey.patch_all()</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">urls </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> [</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//www.python.org'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//www.mi.com'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> 'https：//www.baidu.com'</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">]</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">jobs </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> [gevent.spawn(requests.get， url) </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">for</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">in</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> urls]</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">gevent.joinall(jobs)</span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">([job.value.text </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">for</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> job </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">in</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> jobs])</span></span></code></pre></figure>
<h3 id="基于-asyncio-的异步">基于 Asyncio 的异步<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#基于-asyncio-的异步" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="python" data-theme="github-light github-dark"><code data-language="python" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> asyncio</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests</span></span>
<span data-line> </span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">async</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> def</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> download_site</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(url， session)：</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> async</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> with</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> session.get(url) </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">as</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> response：</span></span>
<span data-line><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> print</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">f</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;Read </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">{len</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(response.content)</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">}</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> from </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">{</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">url</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">}</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> </span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">async</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> def</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> download_all_sites</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(sites)：</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> async</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> with</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> requests.Session() </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">as</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> session：</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> tasks </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> []</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> for</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> url </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">in</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> sites：</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> task </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> asyncio.ensure_future(download_site(url， session))</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> tasks.append(task)</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> asyncio.gather(</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">*</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">tasks)</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> </span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">if</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> __name__</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> ==</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;__main__&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">：</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> sites </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">=</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> [</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;https：//www.jython.org&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;http：//olympus.realpython.org/dice&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">，</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> ] </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">*</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> 80</span></span>
<span data-line><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> asyncio.run(download_all_sites(sites))</span></span></code></pre></figure></article><hr/><div class="page-footer"></div></div><div class="right sidebar"><div class="graph"><h3>关系图谱</h3><div class="graph-outer"><div id="graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:1,&quot;scale&quot;:1.1,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[],&quot;focusOnHover&quot;:false}"></div><button id="global-graph-icon" aria-label="Global Graph"><svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 55 55" fill="currentColor" xml:space="preserve"><path d="M49,0c-3.309,0-6,2.691-6,6c0,1.035,0.263,2.009,0.726,2.86l-9.829,9.829C32.542,17.634,30.846,17,29,17
s-3.542,0.634-4.898,1.688l-7.669-7.669C16.785,10.424,17,9.74,17,9c0-2.206-1.794-4-4-4S9,6.794,9,9s1.794,4,4,4
c0.74,0,1.424-0.215,2.019-0.567l7.669,7.669C21.634,21.458,21,23.154,21,25s0.634,3.542,1.688,4.897L10.024,42.562
C8.958,41.595,7.549,41,6,41c-3.309,0-6,2.691-6,6s2.691,6,6,6s6-2.691,6-6c0-1.035-0.263-2.009-0.726-2.86l12.829-12.829
c1.106,0.86,2.44,1.436,3.898,1.619v10.16c-2.833,0.478-5,2.942-5,5.91c0,3.309,2.691,6,6,6s6-2.691,6-6c0-2.967-2.167-5.431-5-5.91
v-10.16c1.458-0.183,2.792-0.759,3.898-1.619l7.669,7.669C41.215,39.576,41,40.26,41,41c0,2.206,1.794,4,4,4s4-1.794,4-4
s-1.794-4-4-4c-0.74,0-1.424,0.215-2.019,0.567l-7.669-7.669C36.366,28.542,37,26.846,37,25s-0.634-3.542-1.688-4.897l9.665-9.665
C46.042,11.405,47.451,12,49,12c3.309,0,6-2.691,6-6S52.309,0,49,0z M11,9c0-1.103,0.897-2,2-2s2,0.897,2,2s-0.897,2-2,2
S11,10.103,11,9z M6,51c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S8.206,51,6,51z M33,49c0,2.206-1.794,4-4,4s-4-1.794-4-4
s1.794-4,4-4S33,46.794,33,49z M29,31c-3.309,0-6-2.691-6-6s2.691-6,6-6s6,2.691,6,6S32.309,31,29,31z M47,41c0,1.103-0.897,2-2,2
s-2-0.897-2-2s0.897-2,2-2S47,39.897,47,41z M49,10c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S51.206,10,49,10z"></path></svg></button></div><div id="global-graph-outer"><div id="global-graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:-1,&quot;scale&quot;:0.9,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[],&quot;focusOnHover&quot;:true}"></div></div></div><div class="toc desktop-only"><button type="button" id="toc" class aria-controls="toc-content" aria-expanded="true"><h3>目录</h3><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="fold"><polyline points="6 9 12 15 18 9"></polyline></svg></button><div id="toc-content" class><ul class="overflow"><li class="depth-0"><a href="#概述" data-for="概述">概述</a></li><li class="depth-1"><a href="#requests-vs-urllib" data-for="requests-vs-urllib">Requests Vs Urllib</a></li><li class="depth-1"><a href="#requests-的关键特性" data-for="requests-的关键特性">Requests 的关键特性</a></li><li class="depth-0"><a href="#基础用法" data-for="基础用法">基础用法</a></li><li class="depth-1"><a href="#构造请求" data-for="构造请求">构造请求</a></li><li class="depth-1"><a href="#获取响应" data-for="获取响应">获取响应</a></li><li class="depth-1"><a href="#获取响应状态码" data-for="获取响应状态码">获取响应状态码</a></li><li class="depth-1"><a href="#请求参数" data-for="请求参数">请求参数</a></li><li class="depth-1"><a href="#设置请求头" data-for="设置请求头">设置请求头</a></li><li class="depth-1"><a href="#响应内容" data-for="响应内容">响应内容</a></li><li class="depth-0"><a href="#高级用法" data-for="高级用法">高级用法</a></li><li class="depth-1"><a href="#会话维持" data-for="会话维持">会话维持</a></li><li class="depth-1"><a href="#ssl-证书验证" data-for="ssl-证书验证">SSL 证书验证</a></li><li class="depth-1"><a href="#代理设置" data-for="代理设置">代理设置</a></li><li class="depth-1"><a href="#超时设置" data-for="超时设置">超时设置</a></li><li class="depth-1"><a href="#异常处理" data-for="异常处理">异常处理</a></li><li class="depth-1"><a href="#流式下载" data-for="流式下载">流式下载</a></li><li class="depth-1"><a href="#连接重试" data-for="连接重试">连接重试</a></li><li class="depth-0"><a href="#实践技巧" data-for="实践技巧">实践技巧</a></li><li class="depth-1"><a href="#文件上传" data-for="文件上传">文件上传</a></li><li class="depth-1"><a href="#获取图片" data-for="获取图片">获取图片</a></li><li class="depth-1"><a href="#prepared-request" data-for="prepared-request">Prepared Request</a></li><li class="depth-0"><a href="#异步-requests" data-for="异步-requests">异步 Requests</a></li><li class="depth-1"><a href="#基于-gevent-的异步" data-for="基于-gevent-的异步">基于 Gevent 的异步</a></li><li class="depth-1"><a href="#基于-asyncio-的异步" data-for="基于-asyncio-的异步">基于 Asyncio 的异步</a></li></ul></div></div><div class="explorer mobile-only"><button type="button" id="explorer" data-behavior="collapse" data-collapsed="collapsed" data-savestate="true" data-tree="[{&quot;path&quot;:&quot;Personal&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog/2018&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog/2020&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog/2021&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog/2022&quot;,&quot;collapsed&quot;:true},{&quot;path&quot;:&quot;Personal/Blog/2023&quot;,&quot;collapsed&quot;:true},{&quot;path
</script><script type="module">
let mermaidImport = undefined
document.addEventListener('nav', async () => {
if (document.querySelector("code.mermaid")) {
mermaidImport ||= await import('https://cdnjs.cloudflare.com/ajax/libs/mermaid/10.7.0/mermaid.esm.min.mjs')
const mermaid = mermaidImport.default
const darkMode = document.documentElement.getAttribute('saved-theme') === 'dark'
mermaid.initialize({
startOnLoad: false,
securityLevel: 'loose',
theme: darkMode ? 'dark' : 'default'
})
await mermaid.run({
querySelector: '.mermaid'
})
}
});
</script><script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.16.9/contrib/copy-tex.min.js" type="application/javascript"></script><script src="../../../../../postscript.js" type="module"></script></html>
Copy Permalink