| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
| |
This inlines the clean_meta() function and reads the whole JSON file in
memory to greatly reduce the number of function calls that are performed
reading the files list. This is especially noticable in reading the
backup files lists.
This does mean that more memory is used when reading large files.
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
Storing this in SQLite is slow and grows the cache to a huge size. The
approach of reading these files lists may be a bit slower but saves a
lot of space and overhead and removes quite some complexity.
|
|
|
|
| |
When fsck finds an unknown file, it could also be a directory.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This changes the information in the metadata dict to include the file
type in a separate field and limit the mode information to standard
permissions only.
Upon reading files lists from the repository the old format is
automatically converted. This changes local cache file to ensure all
information is re-read (the previous commit also already required this).
|
|
|
|
|
| |
This is apparently needed because at least GNU tar expects the
extractlists to be in the same order as files in the archive itself.
|
|
|
|
|
|
| |
This creates a FileBackend class (from parts of the FileRepository class
that is now renamed to Repository) that implements a simple API that can
also be provided by other backends.
|
|
|
|
|
|
| |
This option ensures that a full restore does not require extractlists.
Note that this is less efficient than with extractlists because existing
archives need to be discarded earlier.
|
|
|
|
|
| |
This handles handling of --block-size values that don't end in a suffix
(fixes 4a2c63c).
|
|
|
|
|
| |
The option removes files from the repository that either are currupt,
unknown or have become redundant.
|
|
|
|
|
|
|
|
| |
This command checks the repository to see if any files are missing or
corrupt and whether backups can still be restored.
This currently only reads meta-data files and only checks archive files
presence.
|
| |
|
|
|
|
| |
This implements a command to restore files from a specific backup.
|
|
|
|
|
| |
This ensures that open files and streams are properly closed when an
exception occurs.
|
|
|
| |
The keyencryption property was removed. This fixes d676905.
|
|
|
|
|
| |
This also replaces catches IOError by catching EnvironmentError which
covers a wider range of errors.
|
|
|
|
|
|
|
|
| |
This allows for specifying a policy for backups to keep. All other
backups will be removed.
This also implements a --no-act option to only print which backups would
be removed.
|
|
|
|
|
| |
This adds the list of archives that have an extractlist to the infofile
so the backup can be checked to see if all needed files are present.
|
|
|
|
|
| |
The checks that the repository contains an archive file when resyncing
the metdata cache from the repository.
|
|
|
|
|
| |
This tries to read the newly written passphrase file before installing
it as a new file to avoid installing a not-decryptable passphrase file.
|
|
|
|
|
|
|
|
| |
This ignores archives that will only be for 40% or less by default and
make the percentage count towards effective use and use relative to
block size.
Also renames --min-files to --archive-min-files.
|
|
|
|
|
| |
This moves functionality related to the metadata cache and repository
files to separate modules.
|
|
|
|
|
| |
This ignores any exceptions when trying to remove files from the
repository.
|
|
|
|
|
| |
This moves a validation to cmdline, supports natural values for
--block-size and does simple validation of the BACKUP argument.
|
| |
|
|
|
|
|
|
| |
When backing up directories containing large files, archives with only
one or two files are created. The --min-files option forces the archives
to always contain at least a set amount of files.
|
|
|
|
|
| |
This tries to avoid filling the SQLite cache and only fills the backup
contents cache when it is needed.
|
| |
|
|
|
|
|
|
|
|
| |
This UUID is used to distinguish repository cache files from each other
which allows running multiple backups without manually specifying
different cache directories.
This also changes the default cache directory to ~/.cache/sloth.
|
| |
|
|
|
|
|
| |
This allows removing backups from the repository along with archives
that have become redundant.
|
|
|
|
| |
WIP: Implement a find command that finds files in archives
|
|
|
|
|
| |
This stores the file list for the backup in the cache and modifies the
ls command to use the cache.
|
|
|
|
|
| |
This adds the information from the backup (stored in info.json) in the
cache so it can be queried easier.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use fetchall() on a cursor because SQLite cannot handle partial reads
from a cursor if the database is being modified in another cursor. This
clearly uses more memory during the backup.
Set the synchronous and journal_mode SQLite pragmas to have better
performance but be less safe. Since it is a cache, the data can be
reconstructed from the repository if needed.
This also uses the connection as a context manager instead of manually
calling commit, changes some of the transactions around to have better
performance and includes a few consistency improvements.
|
|
|
|
| |
The names now better reflect the purpose and contents of the table.
|
|
|
|
| |
Don't use the term snapshot any more and use backup instead.
|
| |
|
| |
|
|
|
|
| |
This reads exclude patterns from a file.
|
|
|
|
|
|
|
| |
This instructs the crawler to skip certain patterns from the backup. It
supports * for matching any part of a file name, ** to also match /,
ending the pattern with / to only match directories and starting the
pattern with / to match the full path.
|
| |
|
|
|
|
|
|
|
|
|
| |
This avoids problems with long extractlists and long lists of archives
where more than SQLite's maximum number of SQL variables were passed.
This also uses database transactions in order to improve performance and
moves some database actions around in the code to make for example
temporary table creation closer to where it is actually used.
|
|
|
|
|
| |
This reads snapshot file list and filters and formats the output to be
like ls.
|
|
|
|
|
| |
This writes an info.json file that contains information on each backup
that was made and shows that information with a backups command.
|
|
|
|
|
|
|
| |
Create indexes (some after crawling which is a minor improvement) in the
tables to improve queries, and use explicit transactions to improve
performance (small improvement). Also, move temporary table creation to
the functions where they are used (instead of global).
|