One Hat Cyber Team

View File Name : Testing.html

<!DOCTYPE html>
<html>
<head>
<title>ProFTPD: Testsuite</title>
</head>

<body bgcolor=white>

<hr>
<center><h2><b>ProFTPD: Testsuite</b></h2></center>
<hr>

<p>
The proftpd source code is accompanied by a testsuite, for testing various
features and functionality.  At present, the testsuite is divided into two
main parts: the API-level tests, and the integration tests.  The API tests
are written in C, and exercise the various APIs that proftpd provides to
module developers.  The integration tests are currently written in Perl, and
seek to check a compiled <code>proftpd</code>'s run-time behavior.

<p>
In order to run the testsuite, you must configure your proftpd build so that
the testsuite is prepared.  You do this by using the <code>--enable-tests</code>
configure option, along with your other build options, <i>e.g.</i>:
<pre>
  $ ./configure --enable-tests ...
</pre>

<p>
<b>API Tests</b><br>
The API tests use the <code>libcheck</code> library; see:
<pre>
  <a href="http://check.sourceforge.net/">http://check.sourceforge.net/</a>
</pre>
This library must be installed on your machine in order to run the API tests.
When the <code>--enable-tests</code> configure option is used, proftpd's
configure script will check for libcheck, and will fail unless the library
is present.  If no <code>libcheck</code> is found and you've used
<code>--enable-tests</code>, you will see:
<pre>
    checking for tcase_create in -lcheck... no
    configure: error: libcheck support, required for tests, not present -- aborting
</pre>
Note that, if necessary, the <code>--with-includes</code> and
<code>--with-libraries</code> configure options can be used to specify the
locations of the <code>libcheck</code> headers and libraries.

<p>
To run the testsuite, use the <code>make check</code> target:
<pre>
  $ ./configure --enable-tests ...
  $ make
    ...
  $ make check
    ...
  $ ./api-tests
  Running suite(s): pool
   array
   str
   sets
   timers
   table
   var
   event
   env
   version
   feat
   netaddr
   netacl
   class
   regexp
   expr
   scoreboard
   modules
  100%: Checks: 140, Failures: 0, Errors: 0
</pre>
The <code>make check</code> will also go on to run the integration tests,
if the API tests all pass.

<p>
If one of the API tests fails, you will see an error message like:
<pre>
  99%: Checks: 140, Failures: 1, Errors: 0
  api/env.c:53:F:base:env_get_test:0: Failed to handle null arguments
  -------------------------------------------------
   FAILED 1 test

   Please send email to:

     proftp-devel@lists.sourceforge.net

   containing the `api-tests.log' file (in the tests/ directory)
   and the output from running `proftpd -V'
  -------------------------------------------------
</pre>
Please do follow those instructions; you will be helping in proftpd
development!

<p>
<b>Expected Failures</b><br>
Unfortunately, due to variations in system configurations, it's possible
that some of the API tests will fail, without it necessarily being an
indication of a regression or problem.

<p>
Most notably, some of the most brittle tests make assumptions about the
host's <code>/etc/hosts</code> file, particularly the list of names
associated with the address <code>127.0.0.1</code>.  If the
<code>/etc/hosts</code> file has a name other than "localhost" or
"localhost.localdomain" immediately after the 127.0.0.1 address, <i>e.g.</i>:
<pre>
  127.0.0.1	some.other.name localhost
</pre>
Then the API tests may fail with errors like this:
<pre>
  98%: Checks: 152, Failures: 2, Errors: 0
  api/netaddr.c:344:F:base:netaddr_get_dnsstr_test:0: Expected 'localhost or localhost.localdomain', got '<i>some.other.name</i>'
  api/netacl.c:535:F:base:netacl_match_test:0: Failed to positively match ACL to addr: No such file or directory
</pre>
If this is the case, it is an <i>expected failure</i>; please do not report
the issue as a bug.  (Unfortunately, I have not been able to think of a good
solution/workaround for these situations other than ignoring the test failure,
but I would rather not do that if possible.)

<p>
<b>Integration Tests</b><br>
The current integration tests are written in Perl, and rely on the
<code>Test-Unit</code> Perl package.  Specifically, you <b>must</b> use
<code>Test-Unit-0.14</code>, <b>not</b> <code>Test-Unit-0.25</code>.  Not sure
which version of <code>Test-Unit</code> you have, if at all?  Run the following
command:
<pre>
  $ perl -MTest::Unit -e 'print $Test::Unit::VERSION, "\n";'
  0.14
</pre>

<p>
To run the integration tests manually, use the <code>tests.pl</code> script:
<pre>
  $ cd tests/
  $ perl tests.pl
  t/logins.....................ok                                              
  t/commands/user..............ok                                              
  t/commands/pass..............ok                                              
  t/commands/pwd...............ok                                              
  t/commands/cwd...............ok                                              
  t/commands/cdup..............ok                                              
  t/commands/syst..............ok                                              
  t/commands/type..............ok                                              
  t/commands/mkd...............ok                                              
  t/commands/rmd...............ok                                              
  t/commands/dele..............ok                                              
  t/commands/mdtm..............ok                                              
  t/commands/size..............ok                                              
  t/commands/mode..............ok                                              
  t/commands/stru..............ok                                              
  t/commands/allo..............ok                                              
  t/commands/noop..............ok                                              
  t/commands/feat..............ok                                              
  t/commands/help..............ok                                              
  t/commands/quit..............ok                                              
  t/commands/rnfr..............ok                                              
  t/commands/rnto..............ok                                              
  t/commands/rest..............ok                                              
  t/commands/pasv..............ok                                              
  t/commands/epsv..............ok                                              
  t/commands/port..............ok                                              
  t/commands/eprt..............ok                                              
  t/commands/nlst..............ok                                              
  t/commands/list..............ok                                              
  t/commands/retr..............ok                                              
  t/commands/stor..............ok                                              
  t/commands/appe..............ok                                              
  t/config/displayconnect......ok                                              
  t/config/displaylogin........ok                                              
  t/config/maxloginattempts....ok                                              
  t/config/serverident.........ok                                              
  All tests successful.
  Files=37, Tests=141, 160 wallclock secs ( 4.33 cusr +  4.74 csys =  9.07 CPU)
</pre>

<p>
Some of the integration require root privileges in order to perform the
test properly, <i>e.g.</i> for
<code><Anonymous></code>/<code>DefaultRoot</code> logins, <i>etc</i>.
If you do <i>not</i> run `make check' as root, those tests which require
root privileges are automatically skipped.  Thus for doing a full regression,
run the integration tests as the root user.

<p>
<b>Adding New Tests</b><br>
The following information is for those who are interested in adding new
tests, updating existing tests, or otherwise helping with the testsuite
development.

<p>
The API tests driver, <code>api-tests</code>, honors the
<code>PR_TEST_SUITE</code> environment variable.  This is useful for running
just the test cases associated with a particular suite, such as <i>env</i> or
<i>pool</i>.  Only <b>one</b> suite can be specified using the
<code>PR_TEST_SUITE</code> environment variable:
<pre>
  $ make check
  $ cd tests/
  $ PR_TEST_SUITE=pool ./api-tests
  Running suite(s): pool
  100%: Checks: 5, Failures: 0, Errors: 0
</pre>
This way, when you are adding to an existing test case or adding a new test
case to an existing suite, you need not run the entire API testsuite in order
to run your changes.

<p><a name="FAQ">
<b>Frequently Asked Questions</b><br>
<p>
<font color=red>Question</font>: I have installed the <code>check</code>
library, and I am using the <code>--enable-tests</code> configure option, but
the configure command fails with:
<pre>
  checking check.h usability... yes
  checking check.h presence... yes
  checking for check.h... yes
  checking for tcase_create in -lcheck... no
  configure: error: libcheck support, required for tests, not present -- aborting
</pre>
What am I missing?<br>
<p>
<font color=blue>Answer</font>: To properly answer this, we would need to look
in the log file generated by the configure script, <i>i.e.</i> the
<code>config.log</code> file.

I suspect that looking in that <code>config.log</code> file, you would see
something like this:
<pre>
  configure:20568: checking for tcase_create in -lcheck
  configure:20603: gcc -o conftest -g2 -g -O2 -fdebug-prefix-map=/home/hille/devel/proftp/pftp_build/proftpd-dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -fno-omit-frame-pointer -Werror=implicit-function-declaration -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include/mysql -I/usr/include/postgresql -L$(top_srcdir)/lib -Wl,-z,relro -rdynamic conftest.c -lcheck  -ldl  >&5
  /usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `tcase_create':
  (.text+0x1ce): undefined reference to `floor'
  /usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `tcase_set_timeout':
  (.text+0x513): undefined reference to `floor'
  /usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `check_get_clockid':
  (.text+0xc1d): undefined reference to `timer_create'
  (.text+0xc48): undefined reference to `timer_delete'
  /usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check_pack.o): In function `ppack':
  (.text+0x5b1): undefined reference to `__pthread_register_cancel'
  (.text+0x5fc): undefined reference to `__pthread_unregister_cancel'
  collect2: error: ld returned 1 exit status
</pre>

Newer versions of the <code>check</code> library have dependencies on other
libraries, which need to be declared explicitly.  Thus to address these
dependency linker errors, you would need to re-run your configure command and
provide those linker dependencies:
<pre>
  $ make clean
  $ ./configure LIBS="-lm -lrt -pthread" ...
</pre>
<i>Note</i> that even after doing the above, you <i>still</i> may have the
configure command fail to detect the <code>check</code> library.  There are
reports that using <code>check</code> 0.10.0 and later <i>also</i> require the
<code>subunit</code> library, and thus you may need to use:
<pre>
  $ make clean
  $ ./configure LIBS="-lsubunit -lm -lrt -pthread" ...
</pre>

<p>
<hr>
<font size=2><b><i>
© Copyright 2017-2018 The ProFTPD Project<br>
 All Rights Reserved<br>
</i></b></font>
<hr>

</body>
</html>