郭爽的网页设计资料

http://www.google.com/toolbar/ie7/

Internet Explorer 7 on Linux

貌似 tomatolei 2.9 版本可以通过正版验证而装之…

BTW:

funny article

让 wget 忽略 robot.txt

December 26th, 2006

GNU, wget

如果想用wget来抓网页或者镜像站点但是遇到 robot.txt 不允许怎么办？比如这样的 robot.txt

User-agent: *

Disallow: /snapshots/

Disallow: /cgi-bin/

Disallow: /cgi2-bin/

在 $HOME/.wgetrc 中加入

robots = off

一切OK，镜像sina？

wget -c -m -p -q -b http://www.sina.com

-c 是继续传送， -m 是镜像， -p 是下载媒体文件，比如图片， -q 是安静模式， -b 是后台运行

via 让 wget 忽略 robot.txt

This entry was posted on Thursday, January 18th, 2007 at 11:45 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.