Pekka Helenius
|
f4512b36da
|
Remove unnecessary list structure
|
4 years ago |
Pekka Helenius
|
17d60efdec
|
Set image url/data crawling optional
|
4 years ago |
Pekka Helenius
|
750ad03dfe
|
Handle & symbols correctly in tag values
|
4 years ago |
Pekka Helenius
|
2f74f5a849
|
Improve & clean up tags url handling
|
4 years ago |
Pekka Helenius
|
1d0b386535
|
Improve image processing; Support more image data; Generalize tag data fetching operation
|
4 years ago |
Pekka Helenius
|
45340e119c
|
Introduce 'image_root_urls' parameter
|
4 years ago |
Pekka Helenius
|
86e043b609
|
Improve image validator processing
|
4 years ago |
Pekka Helenius
|
9527e65912
|
Add tags only for new images
|
4 years ago |
Pekka Helenius
|
2083a07e3f
|
Add missing image namespace XML header
|
4 years ago |
Pekka Helenius
|
4a93ef1f24
|
Implement mime type checker
|
4 years ago |
Pekka Helenius
|
e65b8b39f2
|
Implement image crawler
|
4 years ago |
Pekka Helenius
|
2c781e0835
|
fix typos
|
4 years ago |
Pekka Helenius
|
2d232c6b09
|
Add v.0.9.3 features
|
4 years ago |
Kamo Petrosyan
|
b7ad3ca04f
|
backend
|
4 years ago |
Kamo Petrosyan
|
8035674f0c
|
backend
|
4 years ago |
Kamo Petrosyan
|
75c4770b12
|
docstring and comments for code
|
4 years ago |
Kamo Petrosyan
|
92919bec22
|
text writer
|
4 years ago |
Kamo Petrosyan
|
e11e289e5f
|
aiohttp and aiofile
|
4 years ago |
Kamo Petrosyan
|
26688aff8a
|
some fisex
|
4 years ago |
Kamo Petrosyan
|
6484c78352
|
From knocker
|
4 years ago |
Kamo Petrosyan
|
41e9fb2526
|
Not functional changes
|
6 years ago |
Kamo Petrosyan
|
5837e0e226
|
version 0.5.2
|
6 years ago |
|
2c4d7c5a11
|
modified: .gitignore
modified: LICENSE
modified: NOTICE
modified: README.md
modified: composer.json
modified: pysitemap/crawler.py
modified: requirements.txt
modified: run.py
modified: setup.py
|
6 years ago |
Adi Eyal
|
1ef89fbd37
|
Fixed #5
|
6 years ago |
Kamo Petrosyan
|
230005a49a
|
V 0.5.0
|
6 years ago |
Kamo Petrosyan
|
71ac413b14
|
Python 3 compatible
|
6 years ago |
Kamo Petrosyan
|
020772659a
|
show_progress function created but not using yet
|
6 years ago |
Kamo Petrosyan
|
3e5209802e
|
mechanize removed
now using requests and lxml.html (both required to install)
links with error code != 200 will be written in file errors.txt in the path ./
|
6 years ago |
mowshon
|
8a7c86588f
|
Crawler instance has no attribute 'pool'
|
7 years ago |
Adam Taylor
|
3ab04f6da0
|
Code cleanup
|
8 years ago |
Kamo Petrosyan
|
1c55865aa9
|
removes anchor links
|
8 years ago |
Kamo Petrosyan
|
e484c6d05a
|
0.3.8
write_txt added
|
9 years ago |
Kamo Petrosyan
|
3c70e32b10
|
0.3.7
|
9 years ago |
Kamo Petrosyan
|
5d2a2729c6
|
0.3.6
|
9 years ago |
Kamo Petrosyan
|
3ccd456f32
|
v 0.3.5
more faster fast and fastertfast
|
9 years ago |
Kamo Petrosyan
|
c8e3c70224
|
Multiprocessing version 0.3.4
|
9 years ago |
Kamo Petrosyan
|
347b4f7380
|
0.2.8
|
9 years ago |
Kamo Petrosyan
|
06d9116d20
|
using sets
|
9 years ago |
Kamo Petrosyan
|
708255c5b5
|
Release
|
9 years ago |