Add ability to filter subcases with filters - fixes #57

- the filters are "--subcase=" and "--subcase-exclude="
- there is also the option "--subcase-filter-levels=<int>" which affects which levels of nested subcases should check the filters

8 files changed, 57 insertions, 21 deletions

diff --git a/doc/markdown/commandline.md b/doc/markdown/commandline.md
index de23e5b..6385e27 100644
--- a/doc/markdown/commandline.md
+++ b/doc/markdown/commandline.md
@@ -2,7 +2,7 @@
 
 **doctest** works quite nicely without any command line options at all - but for more control a bunch are available.
 
-**Query flags** - after the result is printed the program quits without executing any test cases (and if the framework is integrated into a client codebase which [**supplies it's own ```main()``` entry point**](main.md) - the program should check the result of ```doctest::Context::shouldExit()``` after calling ```doctest::Context::run()``` and should exit - this is left up to the user).
+**Query flags** - after the result is printed the program quits without executing any test cases (and if the framework is integrated into a client codebase which [**supplies it's own ```main()``` entry point**](main.md) - the program should check the result of ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user).
 
 **Int/String options** - they require a value after the ```=``` sign - without spaces! For example: ```--order-by=rand```.
 
@@ -27,15 +27,18 @@ All the options can also be set with code (defaults/overrides) if the user [**su
 | ```-sfe``` ```--source-file-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the file in which test cases are written |
 | ```-ts``` &nbsp; ```--test-suite=<filters>``` | Same as ```--test-case=<filters>``` but filters based on the test suite in which test cases are in |
 | ```-tse``` ```--test-suite-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the test suite in which test cases are in |
+| ```-sc``` &nbsp; ```--subcase=<filters>``` | Same as ```--test-case=<filters>``` but filters subcases based on their names |
+| ```-sce``` ```--subcase-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters subcases based on their names |
 | ```-ob``` &nbsp; ```--order-by=<string>``` | Test cases will be sorted before being executed either by **the file in which they are** / **the test suite they are in** / **their name** / **random**. The possible values of ```<string>``` are ```file```/```suite```/```name```/```rand```. The default is ```file``` |
 | ```-rs``` &nbsp; ```--rand-seed=<int>``` | The seed for random ordering |
 | ```-f``` &nbsp;&nbsp;&nbsp; ```--first=<int>``` | The **first** test case to execute which passes the current filters - for range-based execution - see [**the example**](../../examples/range_based_execution/) (the **run.py** script) |
 | ```-l``` &nbsp;&nbsp;&nbsp; ```--last=<int>``` | The **last** test case to execute which passes the current filters - for range-based execution - see [**the example**](../../examples/range_based_execution/) (the **run.py** script) |
 | ```-aa``` &nbsp; ```--abort-after=<int>``` | The testing framework will stop executing test cases/assertions after this many failed assertions. The default is 0 which means don't stop at all |
+| ```-scfl``` &nbsp; ```--subcase-filter-levels=<int>``` | Apply subcase filters only for the first ```<int>``` levels of nested subcases and just run the ones nested deeper. Default is a very high number which means *filter any subcase* |
 | **Bool Options** | <hr> |
 | ```-s``` &nbsp;&nbsp;&nbsp; ```--success=<bool>``` | To include successful assertions in the output |
 | ```-cs``` &nbsp; ```--case-sensitive=<bool>``` | Filters being treated as case sensitive |
-| ```-e``` &nbsp;&nbsp;&nbsp; ```--exit=<bool>``` | Exits after the tests finish - this is meaningful only when the client has [**provided the ```main()``` entry point**](main.md)  - the program should check ```doctest::Context::shouldExit()``` after calling ```doctest::Context::run()``` and should exit - this is left up to the user. The idea is to be able to execute just the tests in a client program and to not continue with it's execution |
+| ```-e``` &nbsp;&nbsp;&nbsp; ```--exit=<bool>``` | Exits after the tests finish - this is meaningful only when the client has [**provided the ```main()``` entry point**](main.md)  - the program should check the ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user. The idea is to be able to execute just the tests in a client program and to not continue with it's execution |
 | ```-nt``` &nbsp; ```--no-throw=<bool>``` | Skips [**exceptions-related assertion**](assertions.md#exceptions) checks |
 | ```-ne``` &nbsp; ```--no-exitcode=<bool>``` | Always returns a successful exit code - even if a test case has failed |
 | ```-nr``` &nbsp; ```--no-run=<bool>``` | Skips all runtime **doctest** operations (except the test registering which happens before the program enters ```main()```). This is useful if the testing framework is integrated into a client codebase which has [**provided the ```main()``` entry point**](main.md) and the user wants to skip running the tests and just use the program |
diff --git a/doc/markdown/features.md b/doc/markdown/features.md
index 1cbfb00..1d674c2 100644
--- a/doc/markdown/features.md
+++ b/doc/markdown/features.md
@@ -65,6 +65,7 @@ The library can be used like any other if you don't like the idea of mixing prod
 - can be used without exceptions and rtti - checkout [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](configuration.md#doctest_config_no_exceptions)
 - powerful [**command line**](commandline.md) with lots of options
 - tests can be [**filtered**](commandline.md) based on their name/file/test suite using wildcards
+- can [**filter**](commandline.md) subcases using wildcards and by specifying the nesting levels for which those filters should work
 - failures can (optionally) break into the debugger on Windows and Mac
 - integration with the output window of Visual Studio for failing tests
 - a ```main()``` can be provided when implementing the library with the [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](main.md#doctest_config_implement_with_main) identifier
diff --git a/doc/markdown/main.md b/doc/markdown/main.md
index 58d3095..c7e8e7e 100644
--- a/doc/markdown/main.md
+++ b/doc/markdown/main.md
@@ -11,7 +11,7 @@ This should be done in exactly one source file and is even a good idea to do thi
 
 However if you need more control - want to set options with code to the execution context or want to integrate the framework in your production code - then the default ```main()``` just won't do the job. In that case use [**```DOCTEST_CONFIG_IMPLEMENT```**](configuration.md#doctest_config_implement).
 
-All the [**command line**](commandline.md) options can be set like this (flags cannot because it wouldn't make sense). Filters can only be appended or cleared with ```Context::clearFilters()``` - the user cannot remove a specific filter with code. 
+All the [**command line**](commandline.md) options can be set like this (flags cannot because it wouldn't make sense). Filters can only be appended or cleared with the ```addFilter()``` or ```clearFilters()``` method of a ```doctest::Context``` object - the user cannot remove a specific filter with code.
 
 ```c++
 #define DOCTEST_CONFIG_IMPLEMENT
diff --git a/doc/markdown/roadmap.md b/doc/markdown/roadmap.md
index e2a39e8..2f194d2 100644
--- a/doc/markdown/roadmap.md
+++ b/doc/markdown/roadmap.md
@@ -10,9 +10,6 @@ Planned features for future releases - order may change.
 
 ### For 1.2:
 
-- running tests a [few times](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md#repeating-the-tests)
-- ability to filter not just TEST_CASE names but also SUBCASE names (and maybe tags when they are introduced)
-
 - adding contextual info to asserts (logging) - with an ```INFO```/```CONTEXT``` /```TRACEPOINT``` macro (also look at [this](https://github.com/philsquared/Catch/issues/601))
 - add ```ERROR```/```FAIL``` macros (also ```ADD_FAILURE_AT(file, line);``` and extend the asserts to have ```_AT``` variants)
 - Parametric test cases (Value/Type-parameterized tests) - https://github.com/onqtam/doctest/issues/38
@@ -44,6 +41,7 @@ Planned features for future releases - order may change.
 
 ### For 1.4:
 
+- running tests a [few times](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md#repeating-the-tests)
 - runtime performance
     - the set holding all registered tests should use a specialized allocator to minimize program startup time
     - lazily stringify expressions - only when needed
diff --git a/doc/markdown/testcases.md b/doc/markdown/testcases.md
index 27f5f5e..c68ff19 100644
--- a/doc/markdown/testcases.md
+++ b/doc/markdown/testcases.md
@@ -13,6 +13,8 @@ _test name_ and _subcase name_ are free form, quoted, strings. Test names don't
 
 For examples see the [Tutorial](tutorial.md)
 
+Test cases and subcases can be filtered through the use of the [**command line**](commandline.md)
+
 ## BDD-style test cases
 
 In addition to **doctest**'s take on the classic style of test cases, **doctest** supports an alternative syntax that allow tests to be written as "executable specifications" (one of the early goals of [Behaviour Driven Development](http://dannorth.net/introducing-bdd/)). This set of macros map on to ```TEST_CASE```s and ```SUBCASE```s, with a little internal support to make them smoother to work with.
diff --git a/doctest/doctest.h b/doctest/doctest.h
index 83edf41..180b57d 100644
--- a/doctest/doctest.h
+++ b/doctest/doctest.h
@@ -2040,7 +2040,8 @@ namespace detail
         unsigned first; // the first (matching) test to be executed
         unsigned last;  // the last (matching) test to be executed
 
-        int  abort_after;    // stop tests after this many failed assertions
+        int  abort_after;           // stop tests after this many failed assertions
+        int  subcase_filter_levels; // apply the subcase filters for the first N levels
         bool success;        // include successful assertions in output
         bool case_sensitive; // if filtering should be case sensitive
         bool exit;           // if the program should be exited after the tests are ran/whatever
@@ -2080,7 +2081,7 @@ namespace detail
         }
 
         ContextState()
-                : filters(6) // 6 different filters total
+                : filters(8) // 8 different filters total
         {
             resetRunData();
         }
@@ -2142,9 +2143,7 @@ int String::compare(const char* other, bool no_case) const {
 }
 
 int String::compare(const String& other, bool no_case) const {
-    if(no_case)
-        return detail::stricmp(m_str, other.m_str);
-    return strcmp(m_str, other.m_str);
+    return compare(other.c_str(), no_case);
 }
 
 std::ostream& operator<<(std::ostream& stream, const String& in) {
@@ -2524,6 +2523,14 @@ namespace detail
         if(s->subcasesPassed.count(m_signature) != 0)
             return;
 
+        // check subcase filters
+        if(s->subcasesCurrentLevel < s->subcase_filter_levels) {
+            if(!matchesAny(m_signature.m_name, s->filters[6], 1, s->case_sensitive))
+                return;
+            if(matchesAny(m_signature.m_name, s->filters[7], 0, s->case_sensitive))
+                return;
+        }
+
         // if a Subcase on the same level has already been entered
         if(s->subcasesEnteredLevels.count(s->subcasesCurrentLevel) != 0) {
             s->subcasesHasSkipped = true;
@@ -3268,6 +3275,8 @@ namespace detail
         printf(" -sfe, --source-file-exclude=<filters> filters OUT tests by their file\n");
         printf(" -ts,  --test-suite=<filters>          filters     tests by their test suite\n");
         printf(" -tse, --test-suite-exclude=<filters>  filters OUT tests by their test suite\n");
+        printf(" -sc,  --subcase=<filters>             filters     subcases by their name\n");
+        printf(" -sce, --subcase-exclude=<filters>     filters OUT subcases by their name\n");
         printf(" -ob,  --order-by=<string>             how the tests should be ordered\n");
         printf("                                       <string> - by [file/suite/name/rand]\n");
         printf(" -rs,  --rand-seed=<int>               seed for random ordering\n");
@@ -3275,7 +3284,8 @@ namespace detail
         printf("                                       execute - for range-based execution\n");
         printf(" -l,   --last=<int>                    the last test passing the filters to\n");
         printf("                                       execute - for range-based execution\n");
-        printf(" -aa,  --abort-after=<int>             stop after <int> failed assertions\n\n");
+        printf(" -aa,  --abort-after=<int>             stop after <int> failed assertions\n");
+        printf(" -scfl,--subcase-filter-levels=<int>   apply filters for the first <int> levels\n\n");
         DOCTEST_PRINTF_COLORED("[doctest] ", Color::Cyan);
         printf("Bool options - can be used like flags and true is assumed. Available:\n\n");
         printf(" -s,   --success=<bool>                include successful assertions in output\n");
@@ -3288,7 +3298,7 @@ namespace detail
         printf(" -nc,  --no-colors=<bool>              disables colors in output\n");
         printf(" -fc,  --force-colors=<bool>           use colors even when not in a tty\n");
         printf(" -nb,  --no-breaks=<bool>              disables breakpoints in debuggers\n");
-        printf(" -npf, --no-path-filenames=<bool>      only filenames and no paths in output\n\n");
+        printf(" -npf, --no-path-filenames=<bool>      only filenames and no paths in output\n");
         printf(" -nln, --no-line-numbers=<bool>        0 instead of real line numbers in output\n\n");
         // ==================================================================================== << 79
 
@@ -3325,6 +3335,10 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) {
     parseCommaSepArgs(argc, argv, "dt-tc=",                 p->filters[4]);
     parseCommaSepArgs(argc, argv, "dt-test-case-exclude=",  p->filters[5]);
     parseCommaSepArgs(argc, argv, "dt-tce=",                p->filters[5]);
+    parseCommaSepArgs(argc, argv, "dt-subcase=",            p->filters[6]);
+    parseCommaSepArgs(argc, argv, "dt-sc=",                 p->filters[6]);
+    parseCommaSepArgs(argc, argv, "dt-subcase-exclude=",    p->filters[7]);
+    parseCommaSepArgs(argc, argv, "dt-sce=",                p->filters[7]);
     // clang-format on
 
     int    intRes = 0;
@@ -3360,6 +3374,7 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) {
     DOCTEST_PARSE_INT_OPTION(dt-last, dt-l, last, 0);
 
     DOCTEST_PARSE_INT_OPTION(dt-abort-after, dt-aa, abort_after, 0);
+    DOCTEST_PARSE_INT_OPTION(dt-subcase-filter-levels, dt-scfl, subcase_filter_levels, 2000000000);
 
     DOCTEST_PARSE_AS_BOOL_OR_FLAG(dt-success, dt-s, success, 0);
     DOCTEST_PARSE_AS_BOOL_OR_FLAG(dt-case-sensitive, dt-cs, case_sensitive, 0);
diff --git a/doctest/parts/doctest_impl.h b/doctest/parts/doctest_impl.h
index 6038053..4474608 100644
--- a/doctest/parts/doctest_impl.h
+++ b/doctest/parts/doctest_impl.h
@@ -226,7 +226,8 @@ namespace detail
         unsigned first; // the first (matching) test to be executed
         unsigned last;  // the last (matching) test to be executed
 
-        int  abort_after;    // stop tests after this many failed assertions
+        int  abort_after;           // stop tests after this many failed assertions
+        int  subcase_filter_levels; // apply the subcase filters for the first N levels
         bool success;        // include successful assertions in output
         bool case_sensitive; // if filtering should be case sensitive
         bool exit;           // if the program should be exited after the tests are ran/whatever
@@ -266,7 +267,7 @@ namespace detail
         }
 
         ContextState()
-                : filters(6) // 6 different filters total
+                : filters(8) // 8 different filters total
         {
             resetRunData();
         }
@@ -328,9 +329,7 @@ int String::compare(const char* other, bool no_case) const {
 }
 
 int String::compare(const String& other, bool no_case) const {
-    if(no_case)
-        return detail::stricmp(m_str, other.m_str);
-    return strcmp(m_str, other.m_str);
+    return compare(other.c_str(), no_case);
 }
 
 std::ostream& operator<<(std::ostream& stream, const String& in) {
@@ -710,6 +709,14 @@ namespace detail
         if(s->subcasesPassed.count(m_signature) != 0)
             return;
 
+        // check subcase filters
+        if(s->subcasesCurrentLevel < s->subcase_filter_levels) {
+            if(!matchesAny(m_signature.m_name, s->filters[6], 1, s->case_sensitive))
+                return;
+            if(matchesAny(m_signature.m_name, s->filters[7], 0, s->case_sensitive))
+                return;
+        }
+
         // if a Subcase on the same level has already been entered
         if(s->subcasesEnteredLevels.count(s->subcasesCurrentLevel) != 0) {
             s->subcasesHasSkipped = true;
@@ -1454,6 +1461,8 @@ namespace detail
         printf(" -sfe, --source-file-exclude=<filters> filters OUT tests by their file\n");
         printf(" -ts,  --test-suite=<filters>          filters     tests by their test suite\n");
         printf(" -tse, --test-suite-exclude=<filters>  filters OUT tests by their test suite\n");
+        printf(" -sc,  --subcase=<filters>             filters     subcases by their name\n");
+        printf(" -sce, --subcase-exclude=<filters>     filters OUT subcases by their name\n");
         printf(" -ob,  --order-by=<string>             how the tests should be ordered\n");
         printf("                                       <string> - by [file/suite/name/rand]\n");
         printf(" -rs,  --rand-seed=<int>               seed for random ordering\n");
@@ -1461,7 +1470,8 @@ namespace detail
         printf("                                       execute - for range-based execution\n");
         printf(" -l,   --last=<int>                    the last test passing the filters to\n");
         printf("                                       execute - for range-based execution\n");
-        printf(" -aa,  --abort-after=<int>             stop after <int> failed assertions\n\n");
+        printf(" -aa,  --abort-after=<int>             stop after <int> failed assertions\n");
+        printf(" -scfl,--subcase-filter-levels=<int>   apply filters for the first <int> levels\n\n");
         DOCTEST_PRINTF_COLORED("[doctest] ", Color::Cyan);
         printf("Bool options - can be used like flags and true is assumed. Available:\n\n");
         printf(" -s,   --success=<bool>                include successful assertions in output\n");
@@ -1474,7 +1484,7 @@ namespace detail
         printf(" -nc,  --no-colors=<bool>              disables colors in output\n");
         printf(" -fc,  --force-colors=<bool>           use colors even when not in a tty\n");
         printf(" -nb,  --no-breaks=<bool>              disables breakpoints in debuggers\n");
-        printf(" -npf, --no-path-filenames=<bool>      only filenames and no paths in output\n\n");
+        printf(" -npf, --no-path-filenames=<bool>      only filenames and no paths in output\n");
         printf(" -nln, --no-line-numbers=<bool>        0 instead of real line numbers in output\n\n");
         // ==================================================================================== << 79
 
@@ -1511,6 +1521,10 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) {
     parseCommaSepArgs(argc, argv, "dt-tc=",                 p->filters[4]);
     parseCommaSepArgs(argc, argv, "dt-test-case-exclude=",  p->filters[5]);
     parseCommaSepArgs(argc, argv, "dt-tce=",                p->filters[5]);
+    parseCommaSepArgs(argc, argv, "dt-subcase=",            p->filters[6]);
+    parseCommaSepArgs(argc, argv, "dt-sc=",                 p->filters[6]);
+    parseCommaSepArgs(argc, argv, "dt-subcase-exclude=",    p->filters[7]);
+    parseCommaSepArgs(argc, argv, "dt-sce=",                p->filters[7]);
     // clang-format on
 
     int    intRes = 0;
@@ -1546,6 +1560,7 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) {
     DOCTEST_PARSE_INT_OPTION(dt-last, dt-l, last, 0);
 
     DOCTEST_PARSE_INT_OPTION(dt-abort-after, dt-aa, abort_after, 0);
+    DOCTEST_PARSE_INT_OPTION(dt-subcase-filter-levels, dt-scfl, subcase_filter_levels, 2000000000);
 
     DOCTEST_PARSE_AS_BOOL_OR_FLAG(dt-success, dt-s, success, 0);
     DOCTEST_PARSE_AS_BOOL_OR_FLAG(dt-case-sensitive, dt-cs, case_sensitive, 0);
diff --git a/scripts/code_coverage_source/test_output/help.txt b/scripts/code_coverage_source/test_output/help.txt
index e2cbd59..05600fa 100644
--- a/scripts/code_coverage_source/test_output/help.txt
+++ b/scripts/code_coverage_source/test_output/help.txt
@@ -23,6 +23,8 @@
  -sfe, --source-file-exclude=<filters> filters OUT tests by their file
  -ts,  --test-suite=<filters>          filters     tests by their test suite
  -tse, --test-suite-exclude=<filters>  filters OUT tests by their test suite
+ -sc,  --subcase=<filters>             filters     subcases by their name
+ -sce, --subcase-exclude=<filters>     filters OUT subcases by their name
  -ob,  --order-by=<string>             how the tests should be ordered
                                        <string> - by [file/suite/name/rand]
  -rs,  --rand-seed=<int>               seed for random ordering
@@ -31,6 +33,7 @@
  -l,   --last=<int>                    the last test passing the filters to
                                        execute - for range-based execution
  -aa,  --abort-after=<int>             stop after <int> failed assertions
+ -scfl,--subcase-filter-levels=<int>   apply filters for the first <int> levels
 
 [doctest] Bool options - can be used like flags and true is assumed. Available:
 
@@ -45,7 +48,6 @@
  -fc,  --force-colors=<bool>           use colors even when not in a tty
  -nb,  --no-breaks=<bool>              disables breakpoints in debuggers
  -npf, --no-path-filenames=<bool>      only filenames and no paths in output
-
  -nln, --no-line-numbers=<bool>        0 instead of real line numbers in output
 
 [doctest] for more information visit the project documentation