设为首页 加入收藏

TOP

关于StreamReader.ReadToEnd方法
2017-10-13 10:39:33 】 浏览:3073
Tags:关于 StreamReader.ReadToEnd 方法

    以前写抓取网页的代码喜欢用ReadToEnd,因为简单省事,后来发现,在爬取网页的时候,如果网速很慢,ReadToEnd超时的几率很大。使用Read改写后,超时几率大大减小,完整代码如下:

/// <summary>
/// HttpPost
/// </summary>
public static string HttpPost(string url, string data)
{
    byte[] bArr = ASCIIEncoding.UTF8.GetBytes(data);

    // 设置参数
    HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
    request.CookieContainer = m_Cookie;
    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = bArr.Length;
    request.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)";

    Stream postStream = request.GetRequestStream();
    postStream.Write(bArr, 0, bArr.Length);
    postStream.Close();

    //发送请求并获取相应回应数据
    HttpWebResponse response = request.GetResponse() as HttpWebResponse;
    //直到request.GetResponse()程序才开始向目标网页发送Post请求
    Stream responseStream = response.GetResponseStream();
    //返回结果网页(html)代码
    MemoryStream memoryStream = new MemoryStream();
    bArr = new byte[1024];
    int size = responseStream.Read(bArr, 0, (int)bArr.Length);
    while (size > 0)
    {
        memoryStream.Write(bArr, 0, size);
        size = responseStream.Read(bArr, 0, (int)bArr.Length);
        Thread.Sleep(1);
    }
    string content = Encoding.UTF8.GetString(memoryStream.ToArray());
    return content;
}
View Code

 代码中Thread.Sleep(1);也可以去掉。

】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
上一篇写了一个Windows服务,通过C#模拟.. 下一篇C#窗体控件GroupBox修改边框色

最新文章

热门文章

Hot 文章

Python

C 语言

C++基础

大数据基础

linux编程基础

C/C++面试题目