mod_digest
The mod_digest module offers functionality for calculating the hash (or digest) value of files. This is particularly useful when verifying the integrity of files. This functionality is used by the following custom FTP commands:
XCRC
MD5/XMD5
XSHA
XSHA1
XSHA256
XSHA512
HASH
Depending on the file size and the hash function, it takes a fair amount of CPU and IO resources to calculate the result. Therefore decide wisely where to enable the features and set the DigestMaxSize configuration directive appropriately.
This module was compiled and tested against ProFTPD 1.3.3 Installation instructions are discussed here.
The most current version of mod_digest is distributed with the ProFTPD source code.
Please contact TJ Saunders <tj at castaglia.org> with any questions, concerns, or suggestions regarding this module.
2016-01-09: Thanks to Mathias Berchtold <mb at smartftp.com> for his original mod_digest, upon which this version is based.
The DigestAlgorithms directive configures the enabled digest algorithms. If no DigestAlgorithms directive is configured, then all supported digest algorithms are enabled.
DigestAlgorithms
Enabled digest algorithms are announced/discovered via the FEAT response. The following algorithms are currently supported by mod_digest:
FEAT
crc32
md5
XMD5
sha1
sha256
sha512
The mod_digest module will cache the results of any checksum command, on a per-file basis. This improves performance, and reduces computational overhead. To disable this caching for any reason, use this directive:
# Disable checksum caching DigestCache off
The DigestCache directive can also be used to configure/tune the max-size of the in-memory cache. Note that once the maximum cache size is reached, any checksum FTP commands will be temporarily refused:
DigestCache
# Use a smaller cache size DigestCache size 100
# Retain cached entries longer DigestCache maxAge 60s
If on is used, mod_digest will use the default max-size of 10000:
DigestCache on
The default digest algorithm that the mod_digest module uses, for e.g. opportunistic digesting of file transfers, is SHA1. For selecting a different default algorithm, use the DigestDefaultAlgorithm directive:
DigestDefaultAlgorithm
# Use MD5 rather than SHA1 as the default algorithm DigestDefaultAlgorithm md5
Note that the DigestAlgorithms directive takes precedence; if the DigestDefaultAlgorithm is not included in the DigestAlgorithms, the default algorithm setting will be ignored.
<Directory>
.ftpaccess
The DigestEnable directive can be used to block or prevent checksumming/digests on files in the configured <Directory>. This can be very useful for preventing checksumming of files located on network-mounted filesystems, for example.
DigestEnable
The DigestEngine directive enables or disables the handling of the checksum-related FTP commands by mod_digest, i.e.:
DigestEngine
The DigestMaxSize directive configures the maximum number of bytes a single hash command is allowed to read from a file. If the number of bytes to be read from the file is greater than the configured number the server will refuse that command.
DigestMaxSize
If no DigestMaxSize directive is configured, then there is no limit. It is highly recommended to set an upper limit.
Example:
# Limit hashing to 1GB of data DigestMaxSize 1 GB
<VirtualHost>
<Global>
The DigestOptions directive is used to configure various optional behavior of mod_digest.
DigestOptions
The currently implemented options are:
NoTransferCache
The mod_digest module will automatically calculate and cache the results of any transferred file, on a per-file basis. This is done assuming that many FTP clients will want to verify the integrity of the file just uploaded/downloaded. This improves performance, and reduces computational overhead. To disable this caching for any reason, use this option. Not recommended.
Note: The NoTransferCache option is automatically enabled when using ProFTPD versions before 1.3.6rc2, due to bugs/missing support in the older versions.
$ ./configure --enable-openssl --with-modules=mod_digest
$ ./configure --enable-dso --enable-openssl --with-shared=mod_digest
$ make $ make install
Alternatively, if your proftpd was compiled with DSO support, you can use the prxs tool to build mod_digest as a shared module:
prxs
$ prxs -c -i -d mod_digest.c
<IfModule mod_digest.c> # Set a limit on file sizes that can be digested DigestMaxSize 1 GB </IfModule>
Recording Uploaded/Downloaded File Checksums One particular use case that comes up is whether the mod_digest can be used to record the digests ("checksums") of uploaded/downloaded files in e.g. a SQL database. The answer is "yes", with some caveats.
First, here is a configuration excerpt showing show such functionality might be implemented, using mod_digest and mod_sql:
mod_sql
<IfModule mod_digest.c> </IfModule> <IfModule mod_sql.c> ... SQLNamedQuery log-file-checksum FREEFORM "INSERT INTO file_checksums (user, file, algo, checksum) VALUES ('%u', '%f', '%{note:mod_digest.algo}', '%{note:mod_digest.digest}')" SQLLog RETR,STOR log-file-checksum ... </IfModule>
%{note:...}
SQLLog
LogFormat
Name of the digest algorithm used, e.g. "SHA1".
Calculated digest of the file as a hex-encoded lowercase string.
Now, the caveats with this technique:
STOR
RETR
APPE
REST
RETR/STOR
UseSendfile
--with-modules
$ ./configure --with-modules=mod_sql:mod_sql_mysql:mod_digest ...
LoadModule
LoadModule mod_sql.c LoadModule mod_sql_mysql.c LoadModule mod_digest.c