| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@295 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@294 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
in a map allowing them to be serialized
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@293 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
policy
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@292 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
Build-Depends-Indep
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@291 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@290 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@289 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
quoted strings
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@288 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
and add FIXME
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@287 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
point where the previous crawl stopped
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@286 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@285 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
again after crawl
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@284 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@283 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@282 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@281 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@280 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
crawler.postprocess() functions
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@279 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@278 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@277 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
reserialized again
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@276 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@275 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@274 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
updating files (e.g. serialization files)
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@273 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@272 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
site
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@271 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@270 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@269 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
change since the constructor (or serialization)
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@268 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
constructor's default value
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@267 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
deserialized links to avoid duplicates as a link can be deserialized multiple times
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@266 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
could be a partial deserialize that needs more tweaking to the site before the call to crawl()
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@265 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
handle case where encoding is specified as empty string
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@264 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
of encodings
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@263 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@262 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@261 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
invalid scheme names
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@260 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@259 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@258 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
deserializing all crawler state (site and links) to and from a file, this module is not called anywhere yet
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@257 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@256 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@255 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@254 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@253 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@252 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@251 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@250 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
method recallable
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@249 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@248 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
|
|
| |
anchor debugging messages even further
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@247 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|
|
|
|
| |
git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@246 86f53f14-5ff3-0310-afe5-9b438ce3f40c
|