Kamo Petrosyan
|
e9b1fe142a
|
version 0.4.1
|
6 years ago |
Kamo Petrosyan
|
3e5209802e
|
mechanize removed
now using requests and lxml.html (both required to install)
links with error code != 200 will be written in file errors.txt in the path ./
|
6 years ago |
Kamo Petrosyan
|
c5057f39ae
|
Merge pull request #3 from mowshon/patch-1
Crawler instance has no attribute 'pool'
|
7 years ago |
mowshon
|
8a7c86588f
|
Crawler instance has no attribute 'pool'
|
7 years ago |
Kamo Petrosyan
|
94ffde6211
|
Merge pull request #1 from ataylor32/code-cleanup
Code cleanup
|
8 years ago |
Adam Taylor
|
3ab04f6da0
|
Code cleanup
|
8 years ago |
Kamo Petrosyan
|
1c55865aa9
|
removes anchor links
|
8 years ago |
Kamo Petrosyan
|
25ca8c0980
|
Create requirements.txt
|
8 years ago |
Kamo Petrosyan
|
e484c6d05a
|
0.3.8
write_txt added
|
9 years ago |
Kamo Petrosyan
|
3c70e32b10
|
0.3.7
|
9 years ago |
Kamo Petrosyan
|
5d2a2729c6
|
0.3.6
|
9 years ago |
Kamo Petrosyan
|
4c42cac12a
|
Update composer.json
|
9 years ago |
Kamo Petrosyan
|
56baa3d521
|
Update composer.json
|
9 years ago |
Kamo Petrosyan
|
4ddd73b091
|
Update composer.json
|
9 years ago |
Kamo Petrosyan
|
838ab1bc29
|
Create composer.json
|
9 years ago |
Kamo Petrosyan
|
a23c8ed5a0
|
fix url
|
9 years ago |
Kamo Petrosyan
|
3ccd456f32
|
v 0.3.5
more faster fast and fastertfast
|
9 years ago |
Kamo Petrosyan
|
5af38e1058
|
Update README.md
|
9 years ago |
Kamo Petrosyan
|
84db7046b8
|
example script changed
|
9 years ago |
Kamo Petrosyan
|
ffce8989f5
|
fix readme
|
9 years ago |
Kamo Petrosyan
|
c8e3c70224
|
Multiprocessing version 0.3.4
|
9 years ago |
Kamo Petrosyan
|
942a22826c
|
Merge remote-tracking branch 'origin/master'
Conflicts:
run.py
modified: .gitignore
|
9 years ago |
Kamo Petrosyan
|
347b4f7380
|
0.2.8
|
9 years ago |
Kamo Petrosyan
|
40ec0454af
|
Update run.py
|
9 years ago |
Kamo Petrosyan
|
328781b95b
|
readme
|
9 years ago |
Kamo Petrosyan
|
06d9116d20
|
using sets
|
9 years ago |
Kamo Petrosyan
|
8cf515e9fe
|
0.2.2
|
9 years ago |
Kamo Petrosyan
|
4babce3c61
|
renamed to sitemap-generator
|
9 years ago |
Kamo Petrosyan
|
708255c5b5
|
Release
|
9 years ago |
Kamo Petrosyan
|
0bf7bc2e5a
|
Create Notice
|
9 years ago |
Kamo Petrosyan
|
54d4d5b0dd
|
Initial commit
|
9 years ago |