Arthur de Jong

Open Source / Free Software developer

summaryrefslogtreecommitdiffstats
path: root/webcheck/db.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove duplicate column definitionArthur de Jong2013-12-151-1/+0
|
* Split functionality into Link.get_or_create()Arthur de Jong2013-12-151-9/+20
| | | | | This splits some common functionality from Link._get_child() and Crawler.get_link() to the new Link.get_or_create() function.
* Rename some functionsArthur de Jong2013-12-151-3/+3
| | | | | This should make some functions clearer and marks internal functions with a leading underscore.
* Move SQLite initialisation to db moduleArthur de Jong2013-12-151-4/+11
|
* Store link and page problems as unicodeArthur de Jong2013-12-021-4/+10
| | | | | | This converts problems to unicode so they can be stored correctly by SQLAlchemy. This amongst other things fixes a problem when the web server returns a status message with non-ASCII characters.
* Optimise count_parents()Arthur de Jong2013-10-061-11/+4
| | | | | | | This combines two queries using a union that already does distinct. This also removes the distinct from the parents() function because it uses a union which is supposed to use distinct already.
* detect self-referencing redirects even with intermediate ↵Devin Bayer2011-11-161-7/+10
| | | | | | pages git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@463 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* implement a MAX_DEPTH configuration option to limit ↵Arthur de Jong2011-11-041-2/+7
| | | | | | crawling based on a patch by Devin Bayer git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@459 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* switch to using the logging frameworkArthur de Jong2011-10-141-4/+8
| | | | git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@457 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* ensure that the database is emptied completely and move ↵Arthur de Jong2011-10-081-0/+19
| | | | | | the code to webcheck.db git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@452 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* move some more initialisation from cmd to crawler and ↵Arthur de Jong2011-10-071-4/+3
| | | | | | make imports of config and debugio consistent git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@447 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* remove unnecessary importsArthur de Jong2011-10-071-1/+1
| | | | git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@443 86f53f14-5ff3-0310-afe5-9b438ce3f40c
* move all the code except the command-line handling to ↵Arthur de Jong2011-09-161-0/+299
the webcheck package and reorganise imports accordingly git-svn-id: http://arthurdejong.org/svn/webcheck/webcheck@435 86f53f14-5ff3-0310-afe5-9b438ce3f40c