Kamo Petrosyan e484c6d05a | 9 years ago | |
---|---|---|
pysitemap | 9 years ago | |
.gitignore | 9 years ago | |
LICENSE | 9 years ago | |
NOTICE | 9 years ago | |
README.md | 9 years ago | |
composer.json | 9 years ago | |
run.py | 9 years ago | |
setup.py | 9 years ago | |
sitemap.xml | 9 years ago |
Sitemap generator
pip install sitemap-generator
Sitemap-generator uses gevent to implement multiprocessing. Install gevent:
pip install gevent
import pysitemap
if __name__=='__main__':
url = 'http://www.example.com/' # url from to crawl
logfile = 'errlog.log' # path to logfile
oformat = 'xml' # output format
crawl = pysitemap.Crawler(url=url, logfile=logfile, oformat=oformat)
crawl.crawl()
import pysitemap
if __name__=='__main__':
url = 'http://www.example.com/' # url from to crawl
logfile = 'errlog.log' # path to logfile
oformat = 'xml' # output format
crawl = pysitemap.Crawler(url=url, logfile=logfile, oformat=oformat)
crawl.crawl(pool_size=10) # 10 parsing processes