山寨机中的战斗机！	程序优化工程师到底对IT界有没有贡献

哪位能用工具下载 wowchina 的BBS页面(思归等大人救命)?

楼主binbare（学习·学习·再学习！）2005-06-01 12:52:19 在 .NET技术 / ASP.NET 提问

地址:http://bbs.wowchina.com/bbs/bbs_list.aspx?forum_id=4&page_no=60

太绝了.只能用浏览器打开.用网际快乐或C#的webrequest方法写的都取得错误的页面,不知是何原因,哪位高手能解决? 问题点数：100、回复次数：13Top

1 楼webfactory（jack）回复于 2005-06-01 13:03:47 得分 0

他全是用脚本生成的，所以你的快车或webrequest不行了Top

2 楼hchxxzx（NET？摸到一点门槛）回复于 2005-06-01 13:15:36 得分 0

那是个XML文件.Top

3 楼binbare（学习·学习·再学习！）回复于 2005-06-01 13:24:15 得分 0

有何办法可以拉下来?Top

4 楼LaoDai_Net（『老代』）回复于 2005-06-01 13:24:55 得分 0

告诉你一个笨办法

装一下那个 Full Source 你搜索一下论坛，，有下载的连接

然后复制代码到DW中，，在根据里面地址把图片 CSS 等下载回来，，不过这样要花点时间了

还有个办法，，可以抓图回来Top

5 楼liuqinglq（白菜）回复于 2005-06-01 13:28:37 得分 0

就是XML

另存的结果，和CSDN一样的Top

6 楼Nils（睡瞌睡等机会）回复于 2005-06-01 13:30:21 得分 0

看了. xmlTop

7 楼binbare（学习·学习·再学习！）回复于 2005-06-01 13:35:53 得分 0

抓图的方法肯定是不行了.至于你说的那个IE Full Source ,好像也不行.

的确是生成XML代码的.我要的就是利用程序或工具可以把此类页面给成批量的DOWN下来.至于什么CSS,图页之类的要不要无所谓.重要的就是那个数据,帖子标题,发言人等.Top

8 楼binbare（学习·学习·再学习！）回复于 2005-06-01 14:15:16 得分 0

附上本人写的代码,望高手指点.

private static void Main(string[] args)
{
string localDir = "D:\\WOW";

if (!Directory.Exists(localDir))
{
Directory.CreateDirectory(localDir);
Console.WriteLine("Directory {0} Created!!!", localDir);
}

for (int n = 1; n < 100; n++)
{
string mPage = n.ToString();
string url = "http://bbs.wowchina.com/bbs/bbs_list.aspx?forum_id=4&page_no=" + mPage;

try
{
WebClient wc = new WebClient();
wc.DownloadFile(url, localDir + "\\" + mPage + ".htm");
Console.WriteLine("oh ye~~ : {0} download!!!", localDir + "\\" + mPage + ".htm");
}
catch (WebException e)
{
Console.WriteLine(e.Message);
}
catch (IOException e)
{
Console.WriteLine(e.Message);
}
}

}
Top

9 楼fancyf（凡瑞）回复于 2005-06-01 22:52:20 得分 0

哈哈哈哈~~搞定了
Directory D:\WOW Created!!!
oh ye~~ : D:\WOW\1.htm download!!!
oh ye~~ : D:\WOW\2.htm download!!!
oh ye~~ : D:\WOW\3.htm download!!!
oh ye~~ : D:\WOW\4.htm download!!!
oh ye~~ : D:\WOW\5.htm download!!!
oh ye~~ : D:\WOW\6.htm download!!!
oh ye~~ : D:\WOW\7.htm download!!!
oh ye~~ : D:\WOW\8.htm download!!!
oh ye~~ : D:\WOW\9.htm download!!!
oh ye~~ : D:\WOW\10.htm download!!!
oh ye~~ : D:\WOW\11.htm download!!!
oh ye~~ : D:\WOW\12.htm download!!!
oh ye~~ : D:\WOW\13.htm download!!!
oh ye~~ : D:\WOW\14.htm download!!!
oh ye~~ : D:\WOW\15.htm download!!!
oh ye~~ : D:\WOW\16.htm download!!!
oh ye~~ : D:\WOW\17.htm download!!!
oh ye~~ : D:\WOW\18.htm download!!!
oh ye~~ : D:\WOW\19.htm download!!!
……………………Top

10 楼fancyf（凡瑞）回复于 2005-06-01 22:53:05 得分 100

Here is the code:
public static void Main(string[] args)
{
string localDir = "D:\\WOW";

if (!Directory.Exists(localDir))
{
Directory.CreateDirectory(localDir);
Console.WriteLine("Directory {0} Created!!!", localDir);
}

CookieContainer cookieContainer = new CookieContainer();
Uri bbsRoot = new Uri("http://bbs.wowchina.com");
cookieContainer.Add(bbsRoot, new Cookie("render", "2"));

for (int n = 1; n < 100; n++)
{
string mPage = n.ToString();
string url = "http://bbs.wowchina.com/bbs/bbs_list.aspx?forum_id=4&page_no=" + mPage;
Uri target = new Uri(url);
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(target);
request.AllowAutoRedirect = false;
request.Accept = "*/*";
request.Headers.Add("Accept-Language", "zh-cn");
request.Headers.Add("Accept-Encoding", "gzip, deflate");
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)";

WebProxy myProxy = new WebProxy();
Uri newUri=new Uri("http://128.101.191.244:3127");
myProxy.Address = newUri;
request.Proxy = myProxy;

if (cookieContainer.Count > 0)
{
request.CookieContainer = cookieContainer;
}

string content = null;
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
if (response.Headers["SetCookies"] != null && response.Headers["SetCookies"] != string.Empty)
{
cookieContainer.SetCookies(bbsRoot, response.Headers["SetCookies"]);
}

using(StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.Default, true))
{
content = sr.ReadToEnd();
}

}

StreamWriter sw = new StreamWriter(localDir + "\\" + mPage + ".htm", System.Text.Encoding.Default);
sw.Write(content);
sw.Close();
Console.WriteLine("oh ye~~ : {0} download!!!", localDir + "\\" + mPage + ".htm");
}
catch (WebException e)
{
Console.WriteLine(e.Message);
}
catch (IOException e)
{
Console.WriteLine(e.Message);
}
}
}Top

11 楼sunjian_qi（sonne）回复于 2005-06-01 23:17:53 得分 0

upTop

12 楼binbare（学习·学习·再学习！）回复于 2005-06-02 08:23:13 得分 0

fancyf(凡瑞) ,强...太感谢了.呵.结分.Top

13 楼jinwishing（醒醒吧Child）回复于 2005-06-08 15:23:51 得分 0

强人都是....Top