digup - A Digest Updating Tool

4 min read Original article ↗

Picture of a shovel and digest matching

TalkBox
No posts yet.
GravatarName: Text

Posted on 2009-11-10, last updated 2020-07-25 by Timo Bingmann at Permlink.

Description

digup is a console tool to update md5sum or shasum digest files. It will read existing digest files, check the current directory for new, updated, modified, renamed or deleted files and query the user with a summary of changes. After reviewing the updates, they can be written back to the digest file.

One of the envisioned applications of digup is to update and verify incremental archives like chronological data storages or music collections. By using digup possibly undesired changes or lost files can easily be detected while quickly adding new files.

Another purpose could be to automatically verify the integrity of hard disk copies of archives, as backups to hard disks are becoming increasingly popular. Using a full file digest scan even slowly creeping bad blocks on old hard disks can be detected. By using a crontab entry, this check can be performed unattended and routinely.

In normal operation only touched files with newer modification times are fully read and their digest compared. Optionally a full scan can be performed to test all file contents against their expected digests.

Symbolic links are supported by either following the link and reading the target's digest or by saving only the link target path and verifying it against the old one.

The digest files written by digup are compatible with those generated and read by md5sum and similar programs from the coreutils package. Additional information like file size and modification time or symlink targets are stored on comment lines.

Four digest algorithms are supported: MD5, SHA1, SHA256 and SHA512. The digest file itself is also checksummed using CRC32 against unintentional changes. A fast red-black binary tree is used for the internal file list, allowing fast operation on a large number of files.

Alternatives

digup is targeted at simple digest updating and verification without filtering capabilities or configuration files. For more complex applications like host-based intrusion detection, refer to tools like Tripwire (commercial), AIDE (open-source) or debsums for Debian/Linux.

Downloads

The source code is released under the GNU General Public License v3 (GPL). Some parts of the source tarball are copyrighted under free software licenses by different authors.

A git repository containing all sources and revisions is fetchable by running
git clone https://github.com/bingmann/digup.git

See bottom of this page for older downloads.

Documentation

Please refer to the manpage of digup(1): also available online in HTML and PDF formats.

Screenshots

Screenshot of two digup runs on Linux

ChangeLog

2020-07-25 19:58 - Timo Bingmann - 0.6.57

  • src/digup.c: on Windows replace backslashes in filenames with forward slashes.

2020-01-20 18:19 - Timo Bingmann - 0.6.55

  • src/digup.c: fixed some edge case memory leaks, better packaging, added --version flag.

2020-01-18 20:26 - Timo Bingmann - 0.6.50

  • src/digup.c: minor fixes for portability, update autotools build scripts.

2011-01-30 19:05 - Timo Bingmann - 0.6.40

  • src/digup.c: Adding shortcut option -w, --windows for --modify-window=1 as needed to check files on FAT filesystems.
  • src/digup.c: New feature: --restrict=substring to temporarily ignore all files not matching the substring. Required a new fileinfo state FS_SKIPPED and appropriate commands.

2011-01-13 17:11 - Timo Bingmann - 0.6.40

  • src/digup.c: Fixing problems with 64-bit file sizes on win32 under (mingw32): using _stat64 on mingw now.
  • src/digup.c: Simple variable rename node -> digestiter for clarity.

2011-01-13 17:04 - Timo Bingmann - 0.6.40

  • src/digup.c: Adding check in digest_file to verify that the complete file was read.

2011-01-13 17:01 - Timo Bingmann - 0.6.40

  • src/digup.c: Using intptr_t in rbtree tests to fix warnings about pointer conversion on 64-bit systems.

2010-11-29 18:05 - Timo Bingmann - 0.6.40

  • src/digup.c: Adding new flags --modify-window inspired by rsync: allow modification time deltas larger than zero e.g. for backuping on FAT filesystems.
  • src/digup.c: Adding new option --exclude-marker=file. If file is found in a directory during the recursive scan, the directory itself and all subdirectories are skipped.

2010-10-03 11:28 - Timo Bingmann - 0.6.30

  • src/digup.c: Applying contributed patch which fixes segfault on amd64 due to va_start(), va_end() use pattern.

2010-08-20 10:49 - Timo Bingmann - 0.6.27

  • src/digup.c: Fixed string allocation bug causing a segfault when using -t followed by -f parameters.

2010-03-20 14:27 - Timo Bingmann - 0.6.27

  • src/digup.c: Changing size variable from ssize_t to long long to enable support for large files.

2009-11-10 14:57 - Timo Bingmann - 0.6.23

  • Initial release of 0.6.23

Older Downloads

digup 0.6.55 released 2020-01-20
Source Code: digup-0.6.55.tar.gz digup-0.6.55.tar.gz (181 KiB)
Binaries: Generic 32-bit and 64-bit Linux binaries
digup-0.6.55-linux.tar.gz digup-0.6.55-linux.tar.gz (94.0 KiB)
Windows 95/98/Me/2000/NT/XP/Vista/7/10 stand-alone 32-bit binary
digup-0.6.55-win32.zip digup-0.6.55-win32.zip (55.7 KiB)

digup 0.6.50 released 2020-01-18
Source Code: digup-0.6.50.tar.gz digup-0.6.50.tar.gz (173 KiB)
Binaries: Generic 32-bit and 64-bit Linux binaries
digup-0.6.50-linux.tar.gz digup-0.6.50-linux.tar.gz (93.7 KiB)
Windows 95/98/Me/2000/NT/XP/Vista/7/10 stand-alone 32-bit binary
digup-0.6.50-win32.zip digup-0.6.50-win32.zip (55.6 KiB)

digup 0.6.40 released 2011-01-30
Source Code: digup-0.6.40.tar.bz2 digup-0.6.40.tar.bz2 (127 KiB)
Binaries: Generic 32-bit and 64-bit static Linux binaries
digup-0.6.40-linux-static.tar.gz digup-0.6.40-linux-static.tar.gz (303 KiB)
Windows 95/98/Me/2000/NT/XP/Vista/7/10 stand-alone 32-bit binary
digup-0.6.40-win32.zip digup-0.6.40-win32.zip (50.8 KiB)