Skip to main content

Command Line

Command Line

doctest works perfectly well without any command-line options – but for more control, a bunch are available.

Query Flags - After printing the result, the program exits without executing any test cases (and if the framework is integrated into a client codebase that provides its own main() entry point – after calling run() on the doctest::Context object, the program should check the result of the shouldExit() method and exit if needed – this is up to the user to decide).

Int/String Options - They require a value after the "=" sign – no spaces! For example: --order-by=rand.

Bool Options - They expect 1/yes/on/true or 0/no/off/false after the = sign – but they can also be used like flags and the =value part can be skipped – then true is assumed.

Filters - Comma-separated lists of wildcards used to match values – where * means "match any sequence" and ? means "match any single character".

To pass patterns with spaces, use "" like this: --test-case="*no sound*,Fuzzily Named Test Number?". Patterns containing commas or backslashes can be escaped with \ (e.g.: --test-case=this\,test\,has\,commas\,and\,a\\backslash\,followed\,by\,a\,comma). If a backslash is not followed by either \``` or ,, it remains as-is, e.g., --test-case="Test if \ works correctly". Note: Your shell may also use ``\ for escaping, so \ may actually be consumed by the shell rather than doctest.

If the user provides a main() function, all options can also be set (defaulted/overridden) using code.

Query FlagsDescription
-?     --help -hPrint a help message listing all these flags/options
-v     --versionPrint the version of the doctest framework
-c     --countPrint the number of test cases matching the current filters (see below)
-ltc --list-test-casesList all test cases by name that match the current filters (see below)
-lts --list-test-suitesList all test suites by name that have at least one test case matching the current filters (see below)
-lr --list-reportersList all registered reporters
Int/String Options<hr>
-tc   --test-case=<filters>Filter test cases based on their names. By default, all test cases are matched, but if a value is specified for this filter (e.g., --test-case=*math*,*sound*), only test cases matching at least one pattern in the comma-separated list of wildcards will be executed/counted/listed
-tce --test-case-exclude=<filters>Same as the -test-case=<filters> option, but if any pattern in the comma-separated list of values matches – the test case is skipped
-sf   --source-file=<filters>Same as --test-case=<filters>, but filter based on the file where the test case is written
-sfe --source-file-exclude=<filters>Same as --test-case-exclude=<filters>, but filter based on the file where the test case is written
-ts   --test-suite=<filters>Same as --test-case=<filters>, but filter based on the test suite the test case belongs to
-tse --test-suite-exclude=<filters>Same as --test-case-exclude=<filters>, but filter based on the test suite the test case belongs to
-sc   --subcase=<filters>Same as --test-case=<filters>, but filter subcases by name. Test cases are not filtered (they have to be executed to discover subcases), so you may want to use this together with --test-case=<filters>.
-sce --subcase-exclude=<filters>Same as --test-case-exclude=<filters>, but filter based on subcase names
-r --reporters=<filters>List of reporters to use (defaults to console)
-o   --out=<string>Output file name
-ob   --order-by=<string>Test cases will be sorted by the file they are in / the test suite they are in / their name / randomly before execution. Possible values for <string> are file/suite/name/rand/none. Defaults to file. Note: The order generated by the "file", "suite", and "name" options is compiler-dependent and may vary depending on the compiler used.
-rs   --rand-seed=<int>Seed for random ordering
-f     --first=<int>The first test case to execute among those passing the current filters – for range-based execution – see example python script
-l     --last=<int>The last test case to execute among those passing the current filters – for range-based execution – see example python script
-aa   --abort-after=<int>After a number of failed assertions, the test framework will stop executing test cases/assertions. The default value is 0, which means not stopping at all. Note that the framework uses exceptions to stop the current test case regardless of the assertion level (CHECK/REQUIRE) – so be careful with assertions in destructors...
-scfl --subcase-filter-levels=<int>Apply subcase filters only to the first <int> levels of nested subcases, and run only subcases nested deeper. The default value is a very high number, meaning filter any subcase
Bool Options<hr>
-s     --success=<bool>Include successful assertions in the output
-cs   --case-sensitive=<bool>Make filter matching case-sensitive (applies to test case names, suite names, source file names, and subcase names)
-e     --exit=<bool>Exit after tests complete – only meaningful if the client provides a main() entry point – the program should check the shouldExit() method after calling run() on the "doctest::Context" object and exit if needed – this is up to the user. The idea is to be able to execute only the tests in the client program and not continue with its execution
-d   --duration=<bool>Print the time (in seconds) taken by each test case
-m   --minimal=<bool>Print only failed tests
-q   --quiet=<bool>Do not print any output
-nt   --no-throw=<bool>Skip exception-related assertion checks
-ne   --no-exitcode=<bool>Always return a successful exit code – even if test cases fail
-nr   --no-run=<bool>Skip all runtime doctest operations (except test registration that happens before the program enters main()). Useful if the test framework is integrated into a client codebase that provides a main() entry point and the user wants to skip running the tests and just use the program
-ni   --no-intro=<bool>Omit the framework introduction in the output
-nv   --no-version=<bool>Omit the framework version in the output
-nc   --no-colors=<bool>Disable colors in the output
-fc   --force-colors=<bool>Force color usage even if a TTY cannot be detected
-nb   --no-breaks=<bool>Disable breakpoints in the debugger when an assertion fails
-ns   --no-skip=<bool>Do not skip test cases marked as skipped with decorators
-gfl --gnu-file-line=<bool>Use :n: instead of (n): for line numbers in output (GNU mode is commonly used for Linux tools/IDEs with : separators)
-npf --no-path-filenames=<bool>When printing file names, the path will be removed from the output – useful if you want to get identical output from the test framework across different environments
-nln --no-line-numbers=<bool>When printing source locations, line numbers in the output will be replaced with 0 – useful if you want the test framework to have identical output even if test positions in source files change
-ndo --no-debug-output=<bool>Disable output in the debug console when a debugger is attached
                                                                 

All flags/options also come with prefixed versions (with --dt- in front by default) – for example --version can also be used with --dt-version or --dt-v.

The default prefix is --dt-, but it can be changed by setting the DOCTEST_CONFIG_OPTIONS_PREFIX define.

All unprefixed versions listed here can be disabled using the DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS define.

This is done to easily interoperate with client command-line option handling when the test framework is integrated into a client codebase – all doctest-related flags/options can be prefixed so no conflicts occur, allowing users to exclude everything starting with --dt- from their option parsing.

If there is no option to exclude things starting with --dt-, the dt_removed helper class may be useful to filter them out:

#define DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS
#define DOCTEST_CONFIG_IMPLEMENT
#include "doctest.h"

class dt_removed {
std::vector<const char*> vec;
public:
dt_removed(const char** argv_in) {
for(; *argv_in; ++argv_in)
if(strncmp(*argv_in, "--dt-", strlen("--dt-")) != 0)
vec.push_back(*argv_in);
vec.push_back(NULL);
}

int argc() { return static_cast<int>(vec.size()) - 1; }
const char** argv() { return &vec[0]; } // Note: non-const char **:
};

int program(int argc, const char** argv);

int main(int argc, const char** argv) {
doctest::Context context(argc, argv);
int test_result = context.run(); // run queries, or run tests unless --no-run

if(context.shouldExit()) // honor query flags and --exit
return test_result;

dt_removed args(argv);
int app_result = program(args.argc(), args.argv());

return test_result + app_result; // combine the 2 results
}

int program(int argc, const char** argv) {
printf("Program: %d arguments received:\n", argc - 1);
while(*++argv)
printf("'%s'\n", *argv);
return EXIT_SUCCESS;
}

When run like this:

program.exe --dt-test-case=math* --my-option -s --dt-no-breaks

It will output this:

Program: 2 arguments received:
'--my-option'
'-s'