Arthur de Jong

Open Source / Free Software developer

summaryrefslogtreecommitdiffstats
path: root/schemes/http.py
Commit message (Expand)AuthorAgeFilesLines
* add some more debugging information (cache hit or miss)Arthur de Jong2006-01-291-1/+11
* trim empty ports (http://host:/) from URLs and do not cra...Arthur de Jong2005-12-291-1/+1
* catch all relevant exceptions when looking up content-typ...Arthur de Jong2005-12-261-1/+1
* add copyright clarification to specify that generated out...Arthur de Jong2005-12-171-0/+3
* remove trailing : from netloc if it is presentArthur de Jong2005-12-171-0/+3
* add configuration option to disable proxy cachingArthur de Jong2005-09-181-0/+4
* try to extract character encoding from http response and ...Arthur de Jong2005-09-171-0/+8
* support basic authentication for http proxies and some in...Arthur de Jong2005-09-131-5/+11
* set status to result of fetching the document (not an err...Arthur de Jong2005-08-201-1/+2
* move redirect handling code to crawler module, including ...Arthur de Jong2005-08-191-15/+2
* split problems into page problems (parsing errors, wrong ...Arthur de Jong2005-08-191-6/+6
* rename parameter to acceptedtypes to not conflict with mi...Arthur de Jong2005-08-121-2/+2
* also pass mimetypes to scheme modules to only fetch conte...Arthur de Jong2005-08-121-2/+3
* reimplement http module to be a little more generic and c...Arthur de Jong2005-07-301-97/+91
* remove references to email addresses where they are not u...Arthur de Jong2005-07-291-3/+3
* handle socket errors properlyArthur de Jong2005-07-241-1/+6
* fix for incomplete change in r76, now version should not ...Arthur de Jong2005-07-241-1/+1
* integrate versio.py into config.py, clean up config.py re...Arthur de Jong2005-07-231-2/+1
* Mike Meyer -> Mike W. MeyerArthur de Jong2005-07-231-1/+1
* almost complete rewrite of crawling and site state code m...Arthur de Jong2005-07-221-52/+58
* use lower-case URL attribute in Link instead of upper-cas...Arthur de Jong2005-07-171-5/+5
* rework scheme code to use more logical function names, mo...Arthur de Jong2005-07-101-61/+20
* store mtime in link object instead of age in daysArthur de Jong2005-07-101-1/+1
* handle and document proxy settings with environment varia...Arthur de Jong2005-07-031-6/+3
* name webcheck with lower caseArthur de Jong2005-07-031-2/+2
* clean up get_reply() function to uses proper recursion an...Arthur de Jong2005-06-281-23/+16
* change to most recent version of the GPL (FSF address cha...Arthur de Jong2005-06-221-1/+1
* pass reference to Link class to plugins with parameter an...Arthur de Jong2005-06-151-1/+1
* redo output writing using a cleaner debugio and change de...Arthur de Jong2005-06-061-12/+11
* rename linkList to linkMapArthur de Jong2005-04-131-3/+3
* indent with spaces instead of tabs (tabs are evil)Arthur de Jong2005-04-091-5/+5
* rebump loglevel to debugArthur de Jong2005-04-081-1/+1
* remove link part from scheme modulesArthur de Jong2005-04-081-0/+173