Regex++, regular expression library.

(version 2.00, 01 April 1999)

Copyright (c) 1998-9
Dr John Maddock

Permission to use, copy, modify, distribute and sell this software
and its documentation for any purpose is hereby granted without fee,
provided that the above copyright notice appear in all copies and
that both that copyright notice and this permission notice appear
in supporting documentation.  Dr John Maddock makes no representations
about the suitability of this software for any purpose.  
It is provided "as is" without express or implied warranty.

 

Contents

    1. Implementation notes
    2. Thread safety
    3. Localisation
    4. Demo Applications
    5. Header Files
    6. Redistributables

 


Introduction

The library consists of three parts: a low level template library for compiling, matching and searching regular expressions in <regex>. A high level class that encapsulates the lower level API's and hides the user from template code at the expense of less versatility, and a POSIX API emulation for those that require regular C function API's, both of the latter in <cregex>.

The low level template code is based around the template class reg_expression<charT, traits, Allocator>, that is used to encapsulate compiled regular expressions. There are two typedefs of reg_expression<> for easy access to the narrow and wide character versions:

typedef reg_expression<char> regex;
typedef reg_expression<wchar_t> wregex;

In order to match a regular expression, the library provides the algorithm query_match. Searching is performed with the algorithm reg_search, and grepping (i.e. finding all matches in a string) with reg_grep. In each case what matched is reported in an instance of the template class reg_match<iterator, Allocator>, as before there are typedefs of this class for the two most common cases:

typedef reg_match<const char*, regex::alloc_type> cmatch;
typedef reg_match<const wchar_t*, wregex::alloc_type> wcmatch;

However note that the algorithms are not restricted to searching regular C-strings, any bi-directional iterator type can be searched, allowing for the possibility of seamlessly searching almost any kind of data.

The low level code also supports search and replace operations: the algorithm reg_format takes the result of a match and a format string, and produces a new string by merging the two. The algorithm reg_merge, takes an expression, a text, and a format string, and transforms the text into a new string by replacing matches with the result of reg_format called on the match and the format string - you can think of this as a version of reg_grep that does search and replace.

The class RegEx is a high level encapsulation of the lower level template code - it provides a simplified interface for those that don't need the full power of the library, and supports only narrow characters, and the "extended" regular expression syntax.

The POSIX API functions: regcomp, regexec, regfree and regerror, are available in both narrow character and Unicode versions, and are provided for those who need compatability with these API's.

Finally, note that the library now has run-time localisation support, and recognises the full POSIX regular expression syntax - including advanced features like multi-character collating elements and equivalence classes - as well as providing compatibility with other regular expression libraries including GNU and BSD4 regex packages, and to a more limited extent perl 5.

 

Installation and Configuration Options

When you extract the library from its zip file, you must preserve its internal directory structure (for example by using the -d option when extracting). If you didn't do that when extracting, then you'd better stop reading this, delete the files you just extracted, and try again!

Currently the library will automatically detect and configure itself for Borland, Microsoft and EGCS compilers only. The library will also detect the HP, SGI, Rogue Wave, or Microsoft STL implementations. Alternatively you can define JM_NO_STL and use the library with no underlying STL. If the STL type is detected, then the library will attempt to extract suitable compiler configuration options from the STL used. Otherwise the library will assume that the compiler is fully compliant with the latest draft standard, unless various options are defined to depreciate features not implemented by your compiler, these are documented in <jm_cfg.h>, do not edit this file directly, if you want to add permanent configuration options add them to <jm_opt.h> which is provided for this purpose - this will allow you to keep your configuration options between library versions by retaining <jm_opt.h>. I realise that the library needs a configure script for Unix builds, no doubt at some point I'll get around to writing one.

The library will encase all code inside namespace __JM unless the macro JM_NO_NAMESPACES is defined, __JM is a macro that defaults to "jm", but can be defined on the command line to be whatever you want. You can also use the macro __JM_STD to refer to the namespace enclosing your standard template library.

Unlike some other template libraries, this library consists of a mixture of template code (in the headers) and static code and data (in cpp files). Consequently it is necessary to build the library's support code into a library or archive file before you can use it, instructions for specific platforms are as follows:

Borland C++:

Open up a console window and change to the <regex++>\lib directory.

Select the appropriate batch file (for example bc5.bat for C++ 5, bcb1.bat for Builder1, bcb3.bat for Builder 3 etc).

Invoke the batch file passing the full path to your version of make on the command line, for example:

Bc5 c:\bc5\bin\make

Note that the full path to make is required if you have more than one version of make on your system (as happens if you have more than one version of Borland's tools installed).

The build process will build a variety of .lib and .dll files (the exact number depends upon the version of Borland's tools you are using), and you should see a message at the end indicating success. The .lib files will be copied to <BCROOT>/lib and the dll's to <BCROOT>/bin, where <BCROOT> corresponds to the install path of your Borland C++ tools.

Finally when you use regex++ it is only necessary for you to add <regex++>\include to your list of include directories for that project. It is not necessary for you to manually add a .lib file to the project; the headers will automatically select the correct .lib file for your build mode and tell the linker to include it. There is one caveat however: the library can not tell the difference between Borland C++ 5.02, and Borland C++ Builder 1, consequently if you use the library with C++ Builder 1 then you must define the pre-processor symbol JM_USE_VCL in order to ensure that the correct link libraries are selected.

 

Microsoft Visual C++ 5/6

You need either version 5 (service pack 3) or version 6 of MSVC to build this library.

Open up a command prompt, which has the necessary MSVC environment variables defined (for example by using the batch file Vcvars32.bat installed by the Visual Studio installation), and change to the <regex++>\lib directory.

Execute the vc6.bat file - at the end you should have six .lib files and two dll files, copies of these will have been placed in your <msvc>\lib and <msvc>\bin directories.

Finally when you use regex++ it is only necessary for you to add <regex++>\include to your list of include directories for that project. It is not necessary for you to manually add a .lib file to the project; the headers will automatically select the correct .lib file for your build mode and tell the linker to include it.

 

EGCS

There is currently no configure script, however there is a makefile for win32 versions of the egcs compiler. With minor modification this should be suitable for egcs on other platforms. From the command prompt change to the <regex++>/lib directory and type:

make -f egcs.mak

At the end of the build process you should have a libregex++.a archive file. When you build projects that use regex++, you will need to add <regex++>/include to your list of include paths and add regex++ to your list of library files.

 

Other compilers:

You need to build all the .cpp files in the <regex++>/src directory into a library or archive file. Note that the code is designed to work with compilers that support automatic merging of duplicate template instances - lack of support for this is still a big handicap to making any template code truly portable. If your compiler does not support this then you may be better off adding the .cpp files directly to your project on a bespoke basis.

 


Template Class and Algorithm Reference

class regbase

#include <regex>

Class regbase provides error handling and other template independent functions, it is the base class for reg_expression:

class JM_IX_DECL regbase
{
protected:
public:
   enum flag_type
   {
      escape_in_lists = 1,                          // '\' special inside [...]
      char_classes = escape_in_lists << 1,          // [[:CLASS:]] allowed
      intervals = char_classes << 1,                // {x,y} allowed
      limited_ops = intervals << 1,                 // all of + ? and | are normal characters
      newline_alt = limited_ops << 1,               // \n is the same as |
      bk_plus_qm = newline_alt << 1,                // uses \+ and \?
      bk_braces = bk_plus_qm << 1,                  // uses \{ and \}
      bk_parens = bk_braces << 1,                   // uses \( and \)
      bk_refs = bk_parens << 1,                     // \d allowed
      bk_vbar = bk_refs << 1,                       // uses \|
      use_except = bk_vbar << 1,                    // exception on error
      failbit = use_except << 1,                    // error flag
      literal = failbit << 1,                       // all characters are literals
      icase = literal << 1,                         // characters are matched regardless of case
      nocollate = icase << 1,                  // don't use locale specific collation
      basic = char_classes | intervals | limited_ops | bk_braces | bk_parens | bk_refs,
      extended = char_classes | intervals | bk_refs,
      normal = escape_in_lists | char_classes | intervals | bk_refs | nocollate,
   };

   unsigned int RE_CALL error_code()const;
   void RE_CALL fail(unsigned int err);
   unsigned int RE_CALL flags()const
#ifdef RE_LOCALE_CPP
   __JM_STD::string RE_CALL errmsg()const;
   __JM_STD::locale RE_CALL imbue(const __JM_STD::locale& l);
   const __JM_STD::locale& RE_CALL locale()const;
#else
   const char* RE_CALL errmsg()const;
#endif
};

 

The enumerated type regbase::flag_type determines the syntax rules for regular expression compilation, the various flags have the following effects:

 

regbase::escape_in_lists

Allows the use of the escape "\" character in sets of characters, for example [\]] represents the set of characters containing only "]". If this flag is not set then "\" is an ordinary character inside sets.

 

 

regbase::char_classes

When this bit is set, character classes [:classname:] are allowed inside character set declarations, for example "[[:word:]]" represents the set of all characters that belong to the character class "word".

 

 

regbase:: intervals

When this bit is set, repetition intervals are allowed, for example "a{2,4}" represents a repeat of between 2 and 4 letter a's.

 

 

regbase:: limited_ops

When this bit is set all of "+", "?" and "|" are ordinary characters in all situations.

 

 

regbase:: newline_alt

When this bit is set, then the newline character "\n" has the same effect as the alternation operator "|".

 

 

regbase:: bk_plus_qm

When this bit is set then "\+" represents the one or more repetition operator and "\?" represents the zero or one repetition operator. When this bit is not set then "+" and "?" are used instead.

 

 

regbase:: bk_braces

When this bit is set then "\{" and "\}" are used for bounded repetitions and "{" and "}" are normal characters. This is the opposite of default behaviour.

 

 

regbase:: bk_parens

When this bit is set then "\(" and "\)" are used to group sub-expressions and "(" and ")" are ordinary characters, this is the opposite of default behaviour.

 

 

regbase:: bk_refs

When this bit is set then back references are allowed.

 

 

regbase:: bk_vbar

When this bit is set then "\|" represents the alternation operator and "|" is an ordinary character. This is the opposite of default behaviour.

 

 

regbase:: use_except

When this bit is set then a bad_expression exception will be thrown on error.

 

 

regbase:: failbit

This bit is set on error, if regbase::use_except is not set, then this bit should be checked to see if a regular expression is valid before usage.

 

 

regbase::literal

All characters in the string are treated as literals, there are no special characters or escape sequences.

 

 

regbase::icase

All characters in the string are matched regardless of case.

 

 

regbase::nocollate

Locale specific collation is disabled when dealing with ranges in character set declarations For example when this bit is set the expression [a-c] would match the characters a, b and c only regardless of locale, where as when this is not set , then [a-c] matches any character which collates in the range a to c.

 

 

regbase::basic

Equivalent to the POSIX basic regular expression syntax: char_classes | intervals | limited_ops | bk_braces | bk_parens | bk_refs.

 

 

Regbase::extended

Equivalent to the POSIX extended regular expression syntax: char_classes | intervals | bk_refs.

 

 

regbase::normal

This is the default setting, and represents how most people expect the library to behave. Equivalent to the POSIX extended syntax, but with locale specific collation disabled, and escape characters inside set declarations enabled: regbase::escape_in_lists | regbase::char_classes | regbase::intervals | regbase::bk_refs | regbase::nocollate.

 

 

Class regbase contains the following public member functions:

 

reg_errcode_t regbase::error_code();

Returns the error code associated with the current regular expression or zero if no error occurred, the code is one of the reg_errcode_t enumerated values.

 

 

void fail(reg_errcode_t err);

For use by derived classes, sets the current error code and throws an bad_expression exception if regbase::use_except is set.

 

 

unsigned int flags()const;

Returns the current flags used to compile the expression, can be used to check for the presence of regbase::failbit.

 

 

const char* errmsg()const;

Returns a user readable string associated with the current error value.

 

 

__JM_STD::locale imbue(const __JM_STD::locale& l);

Available only if RE_LOCALE_CPP is defined to enable the C++ locale. Imbues the expression with a copy of the locale l, invalidates any existing compiled expression.

 

 

const __JM_STD::locale& locale()const

Available only if RE_LOCALE_CPP is defined to enable the C++ locale. Returns the expressions current locale.

 

 

Error codes are returned as one of the reg_errcode_t enumerated types:

 

REG_NOERROR

No error occurred.

 

 

REG_NOMATCH

No match was found.

 

 

REG_BADPAT

The expression was invalid.

 

 

REG_ECOLLATE

An invalid collating name was encountered.

 

 

REG_ECTYPE

An invalid character class name was encountered.

 

 

REG_EESCAPE

A trailing escape character was encountered.

 

 

REG_ESUBREG

An invalid back-reference.

 

 

REG_EBRACK

A "(" was encountered with no matching ")".

 

 

REG_EPAREN

"(" and ")" do not balance.

 

 

REG_EBRACE

"{" was encountered with no matching "}".

 

 

REG_BADBR

The contents of a bounded repetition were invalid.

 

 

REG_ERANGE

The endpoint of a range was invalid.

 

 

REG_ESPACE

Ran out of memory trying to compile the expression.

 

 

REG_BADRPT

A repetition operator was preceded by a sub-expression that can not be repeated.

 

 

REG_EEND

Unexpected end of expression.

 

 

REG_ESIZE

The expression too big.

 

 

REG_ERPAREN

Unmatched right parenthesis.

 

 

REG_EMPTY

An empty expression or sub-expression was encountered.

 

 

REG_E_MEMORY

Ran out of memory trying to compile the expression.

 

 

REG_E_UNKNOWN

An unknown error occurred.

 

 

All error codes are in the range REG_NOERROR to REG_E_UNKNOWN, with REG_NOERROR always taking the value zero.


Class bad_expression

#include <regex>

An instance of bad_expression is thrown when the flag regbase::use_except is set and a bad regular expression is encountered.

class JM_IX_DECL bad_expression : public __JM_STD::exception
{
public:
#ifdef RE_LOCALE_CPP
bad_expression(const __JM_STD::string& s) : code(s) {}
#else
bad_expression(unsigned int err) : code(err) {}
#endif
   bad_expression(const bad_expression& e) : __JM_STD::exception(e), code(e.code) {}
   bad_expression& operator=(const bad_expression& e);
   virtual const char* what()const throw();
};

 N.B. if JM_NO_EXCEPTION_H is defined during program compilation then bad_expression has no base classes, this allows this library to be used independently of the C++ standard library. In addition if the header defines JM_NO_EXCEPTIONS then class bad_expression is not defined, and no exceptions will be thrown by the library, in this case the flag regbase::use_except has no effect.


Class reg_expression

#include <regex>

The template class reg_expression encapsulates regular expression parsing and compilation. The class derives from class regbase and takes three template parameters:

charT: determines the character type, i.e. either char or wchar_t.

traits: determines the behaviour of the character type, for example whether character matching is case sensitive or not, and which character class names are recognised. A default traits class are provided: char_regex_traits<charT>. The traits class char_regex_traits_i<charT> is provided for backward compatibility only.

Allocator: the allocator class used to allocate memory by the class.

For ease of use there are two typedefs that define the two standard reg_expression instances, unless you want to use custom allocators, you won't need to use anything other than these:

typedef reg_expression<char, char_regex_traits<char>, JM_DEF_ALLOC(char)> regex;
typedef reg_expression<wchar_t, char_regex_traits<wchar_t>, JM_DEF_ALLOC(wchar_t)> wregex;

template <class charT, class traits JM_TRICKY_DEFAULT_PARAM(char_regex_traits<charT>), class Allocator JM_DEF_ALLOC_PARAM(charT) >
class reg_expression : public regbase
{
public:
   // typedefs:
   typedef Allocator alloc_type;
   typedef typename REBIND_TYPE(charT, alloc_type)::size_type size_type;
   typedef charT value_type;
   typedef charT char_type;
   typedef traits traits_type;
   typedef typename traits_type::size_type traits_size_type;
   typedef typename traits_type::uchar_type traits_uchar_type;

private:

public:
   unsigned int RE_CALL set_expression(const charT* p, const charT* end, unsigned f = regbase::normal);
   unsigned int RE_CALL set_expression(const charT* p, unsigned f = regbase::normal) ;
   reg_expression(const Allocator& a = Allocator());
   reg_expression(const charT* p, unsigned f = regbase::normal, const Allocator& a = Allocator());
   reg_expression(const charT* p1, const charT* p2, unsigned f = regbase::normal, const Allocator& a = Allocator());
   reg_expression(const charT* p, size_type len, unsigned f, const Allocator& a = Allocator());
   reg_expression(const reg_expression&);
   ~reg_expression();
   reg_expression& RE_CALL operator=(const reg_expression&);

   bool RE_CALL operator==(const reg_expression&);
   bool RE_CALL operator<(const reg_expression&);

   unsigned RE_CALL mark_count()const ;
   alloc_type RE_CALL allocator()const;
   const charT* RE_CALL expression()const ;
};

 

Class reg_expression has the following public member functions:

 

unsigned int RE_CALL set_expression(const charT* p, const charT* end, unsigned f = regbase::normal);

 Compiles the regular expression denoted by the two iterators p and end, using the flags specified by f to determine the regular expression syntax. See class regbase for allowable flag values. This variant of set_expression allows the regular expression string to contain nulls. Returns the error code on failure, or zero on success.

 

 

unsigned int RE_CALL set_expression(const charT* p, unsigned f = regbase::normal);

 Compiles the regular expression contained in the null terminated string p, using the flags specified by f to determine the regular expression syntax. See class regbase for allowable flag values. Returns the error code on failure, or zero on success.

 

 

reg_expression(Allocator a = Allocator());

 Constructs a default instance of reg_expression without any expression.

 

 

reg_expression(charT* p, unsigned f = regbase::normal, Allocator a = Allocator());

 Constructs an instance of reg_expression from the expression denoted by the null terminated string p, using the flags f to determine regular expression syntax. See class regbase for allowable flag values.

 

 

reg_expression(charT* p1, charT* p2, unsigned f = regbase::normal, Allocator a = Allocator());

 Constructs an instance of reg_expression from the expression denoted by pair of iterators p1 and p2, using the flags f to determine regular expression syntax. See class regbase for allowable flag values.

 

 

reg_expression(charT* p, size_type len, unsigned f, Allocator a = Allocator());

 Constructs an instance of reg_expression from the expression denoted by the string p of length len, using the flags f to determine regular expression syntax. See class regbase for allowable flag values.

 

 

reg_expression(const reg_expression&);

Copy constructor - copies an existing regular expression.

 

 

reg_expression& RE_CALL operator=(const reg_expression&);

Copies an existing regular expression.

 

 

unsigned RE_CALL mark_count()const ;

Returns the number of sub-expressions in the compiled regular expression.

 

 

alloc_type RE_CALL allocator()const;

Returns a copy of the objects allocator.

 

 

const charT* RE_CALL expression()const ;

Returns a pointer to a copy of the original expression string.

 

Footnote: version 1 of this library did not allow instances of reg_expression<> to be copied, version 2 now supports standard copy constructors and assignments, so that you can insert expressions into standard library containers. Note however that regular expression copying is a relatively slow operation.

 


Class char_regex_traits

#include <regex>

The definition of the default traits class has changed substantially to support both new language features, and new localisation features. In addition version 1 of the library used a separate traits class for case insensitive matching, this version uses the same traits class for both case sensitive and case insensitive matching. For compatibility the old char_regex_traits_i<> template is still present, but new code should use only char_regex_traits<>, and set the regbase::icase flag when compiling to turn on case insensitive matching.

The purpose of the traits class is to make it easier to customise the behaviour of reg_expression and the associated matching algorithms. Custom traits classes can handle special character sets or define additional character classes, for example one could define [[:kanji:]] as the set of all (Unicode) kanji characters. The following definition defines the interface that all traits classes must provide.

template <class charT>
class char_regex_traits
{
public:
   typedef charT char_type;
   //
   // uchar_type is the same size as char_type
   // but must be unsigned:
   typedef charT uchar_type;
   //
   // size_type is normally the same as charT
   // but could be unsigned int to improve performance
   // of narrow character types, NB must be unsigned:
   typedef unsigned size_type;

   // length:
   // returns the length of a null terminated string
   // can be left unimplimented for non-character types.
   static size_t length(const char_type* );

   // syntax_type
   // returns the syntax type of a given charT
   // translates customised syntax to a unified enum.
   static unsigned int syntax_type(size_type c);

   // translate:
   //
   static charT RE_CALL translate(charT c, bool icase
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // transform:
   //
   // converts a string into a sort key for locale dependant
   // character ranges.
   static void RE_CALL transform(re_str<charT>& out, const re_str<charT>& in
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // transform_primary:
   //
   // converts a string into a primary sort key for locale dependant
   // equivalence classes.
   static void RE_CALL transform_primary(re_str<charT>& out, const re_str<charT>& in
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // is_separator
   // returns true if c is a newline character
   static bool RE_CALL is_separator(charT c);

   // is_combining
   // returns true if the character is a unicode
   // combining character
   static bool RE_CALL is_combining(charT c);

   // is_class
   // returns true if the character is a member
   // of the specified character class
   static bool RE_CALL is_class(charT c, unsigned int f
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // toi
   // converts c to integer
   static int RE_CALL toi(charT c
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // toi
   // converts multi-character value to int
   // updating first as required
   static int RE_CALL toi(const charT*& first, const charT* last, int radix
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // lookup_classname
   // parses a class declaration of the form [:class:]
   // On entry first points to the first character of the class name.
   // 
   static unsigned int RE_CALL lookup_classname(const charT* first, const charT* last
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

   // lookup_collatename
   // parses a collating element declaration of the form [.collating_name.]
   // On entry first points to the first character of the collating element name.
   // 
   static bool RE_CALL lookup_collatename(re_str<charT>& s, const charT* first, const charT* last
   #ifdef RE_LOCALE_CPP
   , const __JM_STD::locale&
   #endif
   );

};

 


Class reg_match

#include <regex>

Class reg_match is used for reporting what matched a regular expression, it is passed to the matching algorithms query_match and reg_search, and is used by reg_grep to notify the callback function (or function object) what matched.

//
// class reg_match
// encapsulates reg_match_base, does a deep copy rather than
// reference counting to ensure thread safety when copying
// other reg_match instances

template <class iterator, class Allocator>
class reg_match : public reg_match_base<iterator, Allocator>
{
public:
   struct pair
   {
      iterator first;
      iterator second;
      bool matched;
   };

   reg_match(const Allocator& a = Allocator())
      : reg_match_base<iterator, Allocator>(a){}

   reg_match(const reg_match_base<iterator, Allocator>& m)
      : reg_match_base<iterator, Allocator>(m){}

   reg_match& operator=(const reg_match_base<iterator, Allocator>& m);

   reg_match(const reg_match& m);
   reg_match& operator=(const reg_match& m);

   const pair& RE_CALL operator[](int n) const;
   Allocator RE_CALL allocator()const;
   size_t RE_CALL length()const;
   unsigned int RE_CALL line()const;
   iterator RE_CALL line_start()const;
   size_type RE_CALL size()const;
};

 

Class reg_match and the public base class reg_match_base have the following public member functions:

 

reg_match(Allocator a = Allocator());

Constructs an instance of reg_match, using allocator instance a.

 

 

const pair& operator[](size_type n) const;

Returns what matched, item 0 represents the whole string, item 1 the first sub-expression and so on.

 

 

Allocator& allocator()const;

Returns the allocator used by the class.

 

 

size_t length();

Returns the length of the matched string, in effect this is equivalent to operator[](0).second - operator[](0).first.

 

 

unsigned int RE_CALL line()const;

Returns the index of the line on which the match occurred, indices start with 1, not zero. Equivalent to the number of newline characters prior to operator[](0).first plus one.

 

 

iterator RE_CALL line_start()const;

Returns an iterator denoting the start of the line on which the match occurred.

 

 

size_type RE_CALL size()const;

Returns how many sub-expressions are present in the match, including sub-expression zero (the whole match).

 

 

The operator[] member function needs further explanation: it returns a const reference to an object of type reg_match<>::pair, which has the following public members:

 

first

An iterator denoting the position of the start of the match.

 

 

second

An iterator denoting the position of the end of the match.

 

 

matched

A Boolean value denoting whether this sub-expression participated in the match.

 

Operator[] takes an integer as an argument that denotes the sub-expression for which to return information, the argument can take the following special values:

 

-2

Returns everything from the end of the match, to the end of the input string, equivalent to $' in perl. If this is a null string, then:

first = second

And

matched = false.

 

 

-1

Returns everything from the start of the input string (or the end of the last match if this is a grep operation), to the start of this match. Equivalent to $` in perl. If this is a null string, then:

first = second

And

matched = false.

 

 

0

Returns the whole of what matched, equivalent to $& in perl. The matched parameter is always true.

 

 

0 < N < size()

Returns what matched sub-expression N, if this sub-expression did not participate in the match then

matched =false

otherwise:

matched =true.

 

 

N < -2 or N >= size()

Represents an out-of range non-existent sub-expression. Returns a "null" match in which

first = last

And

matched = false.

 

The other public member functions are for the internal use of the matching algorithms only. Class reg_match_base<> is a reference counted class for internal use only, while reg_match<> always performs deep-copies to ensure thread safety. Note that as well as being parameterised for an allocator, reg_match<> also takes an iterator type, this allows any pair of iterators to be searched for a given regular expression, provided the iterators have at least bi-directional properties.


Algorithm query_match

#include <regex>

The algorithm query_match determines whether a given regular expression matches a given sequence denoted by a pair of iterators, the algorithm is defined as follows:

template <class iterator, class Allocator, class charT, class traits, class Allocator2>
bool query_match(iterator first, 
                 iterator last, 
                 reg_match<iterator, Allocator>& m, 
                 const reg_expression<charT, traits, Allocator2>& e, 
                 unsigned flags = match_default);

The parameters are as follows:

 

iterator first

Denotes the start of the range to be matched.

 

 

iterator last

Denotes the end of the range to be matched.

 

 

reg_match<iterator, Allocator>& m

An instance of reg_match in which what matched will be reported. On exit if a match occurred then m[0] denotes the whole of the string that matched, m[0].first must be equal to first, m[0].second will be less than or equal to last. m[1] denotes the first subexpression m[2] the second subexpression and so on. If no match occurred then m[0].first = m[0].second = last.

 

 

const reg_expression<charT, traits, Allocator2>& e

Contains the regular expression to be matched.

 

 

unsigned flags = match_default

Determines the semantics used for matching, a combination of one or more match_flags enumerators.

 

 

query_match returns false if no match occurs or true if it does. A match only occurs if it starts at first and finishes on or before last. Example: the following example processes an ftp response:

#include <stdlib.h>
#include <regex>
#include <string>
#include <iostream>

regex expression("^([0-9]+)(\\-| |$)(.*)$");

// process_ftp:
// on success returns the ftp response code, and fills
// msg with the ftp response message.
int process_ftp(const char* response, std::string* msg)
{
   cmatch what;
   if(query_match(response, response + strlen(response), what, expression))
   {
      // what[0] contains the whole string
      // what[1] contains the response code
      // what[2] contains the separator character
      // what[3] contains the text message.
      if(msg)
         msg->assign(what[3].first, what[3].second);
      return atoi(what[1].first);
   }
   // failure did not match
   if(msg)
      msg->erase();
   return -1;
}

The value of the flags parameter passed to the algorithm must be a combination of one or more of the following values:

 

match_default

The default value, indicates that first represents the start of a line, the start of a buffer, and (possibly) the start of a word. Also implies that last represents the end of a line, the end of the buffer and (possibly) the end of a word. Implies that a dot sub-expression "." will match both the newline character and a null.

 

 

match_not_bol

When this flag is set then first does not represent the start of a new line.

 

 

match_not_eol

When this flag is set then last does not represent the end of a line.

 

 

match_not_bob

When this flag is set then first is not the beginning of a buffer.

 

 

match_not_eob

When this flag is set then last does not represent the end of a buffer.

 

 

match_not_bow

When this flag is set then first can never match the start of a word.

 

 

match_not_eow

When this flag is set then last can never match the end of a word.

 

 

match_not_dot_newline

When this flag is set then a dot expression "." can not match the newline character.

 

 

match_not_dot_null

When this flag is set then a dot expression "." can not match a null character.

 

 

match_prev_avail

When this flag is set, then *--first is a valid expression and the flags match_not_bol and match_not_bow have no effect, since the value of the previous character can be used to check these.

 

 

match_any

When this flag is set, then the first string matched is returned, rather than the longest possible match. This flag can significantly reduce the time taken to find a match, but what matches is undefined.

 

 

match_not_null

When this flag is set, then the expression will never match a null string.

 

 

match_continuous

When this flags is set, then during a grep operation, each successive match must start from where the previous match finished.

 

 


Algorithm reg_search

 #include <regex>

The algorithm reg_search will search a range denoted by a pair of iterators for a given regular expression. The algorithm uses various heuristics to reduce the search time by only checking for a match if a match could conceivably start at that position. The algorithm is defined as follows:

template <class iterator, class Allocator, class charT, class traits, class Allocator2>
bool reg_search(iterator first, 
                iterator last, 
                reg_match<iterator, Allocator>& m, 
                const reg_expression<charT, traits, Allocator2>& e, 
                unsigned flags = match_default); 

The parameters are as follows:

 

iterator first

The starting position of the range to search.

 

 

iterator last

The ending position of the range to search.

 

 

reg_match<iterator, Allocator>& m

An instance of reg_match in which what matched will be reported. On exit if a match occurred then m[0] denotes the whole of the string that matched, m[0].first and m[0].second will be less than or equal to last. m[1] denotes the first sub-expression m[2] the second sub-expression and so on. If no match occurred then m[0].first = m[0].second = last.

 

 

const reg_expression<charT, traits, Allocator2>& e

The regular expression to search for.

 

 

unsigned flags = match_default

The flags that determine what gets matched, a combination of one or more match_flags enumerators.

 

 

Example: the following example, takes the contents of a file in the form of a string, and searches for all the C++ class declarations in the file. The code will work regardless of the way that std::string is implemented, for example it could easily be modified to work with the SGI rope class, which uses a non-contiguous storage strategy.

#include <string>
#include <map>
#include <regex>

// purpose:
// takes the contents of a file in the form of a string
// and searches for all the C++ class definitions, storing
// their locations in a map of strings/int's

typedef std::map<std::string, int, std::less<std::string> > map_type;

regex expression("^(template[[:space:]]*<[^;:{]+>[[:space:]]*)?(class|struct)[[:space:]]*(\\<\\w+\\>([[:blank:]]*\\([^)]*\\))?[[:space:]]*)*(\\<\\w*\\>)[[:space:]]*(<[^;:{]+>[[:space:]]*)?(\\{|:[^;\\{()]*\\{)");

void IndexClasses(map_type& m, const std::string& file)
{
   std::string::const_iterator start, end;
   start = file.begin();
   end = file.end();   
   reg_match<std::string::const_iterator> what;
   unsigned int flags = match_default;
   while(reg_search(start, end, what, expression, flags))   
   {
      // what[0] contains the whole string
      // what[5] contains the class name.
      // what[6] contains the template specialisation if any.
      // add class name and position to map:
      m[std::string(what[5].first, what[5].second) + std::string(what[6].first, what[6].second)] = 
               what[5].first - file.begin();      
      // update search position:
      start = what[0].second;      
      // update flags:
      flags |= match_prev_avail;      
      flags &= ~match_not_bob;   
   }
}

 


Algorithm reg_grep

#include <regex>

 Reg_grep allows you to search through an iterator range and locate all the (non-overlapping) matches with a given regular expression. The function is declared as:

template <class Predicate, class iterator, class charT, class traits, class Allocator>
unsigned int reg_grep(Predicate foo, iterator first, iterator last, const reg_expression<charT, traits, Allocator>& e, unsigned flags = match_default)

The parameters have the following meanings:

 

foo

A predicate function object or function pointer, see below for more information.

 

 

first

The start of the range to search.

 

 

last

The end of the range to search.

 

 

e

The regular expression to search for.

 

 

flags

The flags that determine how matching is carried out, one of the match_flags enumerators.

 

 

 The algorithm finds all of the non-overlapping matches of the expression e, for each match it fills a reg_match<iterator, Allocator> structure, which contains information on what matched, and calls the predicate foo, passing the reg_match<iterator, Allocator> as a single argument. If the predicate returns true, then the grep operation continues, otherwise it terminates without searching for further matches.

The general form of the predicate is:

struct grep_predicate
{
   grep_predicate& operator*() { return *this; }
   bool operator()(const reg_match<iterator_type, expression_type::alloc_type>& m);
};

For example the regular expression "a*b" would find one match in the string "aaaaab" and two in the string "aaabb".

Remember this algorithm can be used for a lot more than implementing a version of grep, the predicate can be and do anything that you want, grep utilities would output the results to the screen, another program could index a file based on a regular expression and store a set of bookmarks in a list, or a text file conversion utility would output to file. The results of one reg_grep can even be chained into another reg_grep to create recursive parsers.

Footnote: the format of reg_grep has changed compared to version 1 of the library, existing code can not use this version of reg_grep, a function reg_grep_old is provided with the same syntax as the old version, you can either use this version in existing code, or port to the new version by changing your existing output iterator to a predicate. 

Example: convert the example from reg_search to use reg_grep instead:

#include <string>
#include <map>
#include <regex>

// IndexClasses:
// takes the contents of a file in the form of a string
// and searches for all the C++ class definitions, storing
// their locations in a map of strings/int's

typedef std::map<std::string, int, std::less<std::string> > map_type;

regex expression("^(template[[:space:]]*<[^;:{]+>[[:space:]]*)?"
                 "(class|struct)[[:space:]]*(\\<\\w+\\>([[:blank:]]*\\([^)]*\\))?[[:space:]]*)*(\\<\\w*\\>)"
                 "[[:space:]]*(<[^;:{]+>[[:space:]]*)?(\\{|:[^;\\{()]*\\{)");

class IndexClassesPred
{
   map_type& m;
   std::string::const_iterator base;
public:
   IndexClassesPred(map_type& a, std::string::const_iterator b) : m(a), base(b) {}
   bool operator()(const reg_match<std::string::const_iterator, regex::alloc_type>& what)
   {
      // what[0] contains the whole string
      // what[5] contains the class name.
      // what[6] contains the template specialisation if any.
      // add class name and position to map:
      m[std::string(what[5].first, what[5].second) + std::string(what[6].first, what[6].second)] = 
               what[5].first - base;
      return true;
   }
};

void IndexClasses(map_type& m, const std::string& file)
{
   std::string::const_iterator start, end;
   start = file.begin();
   end = file.end();
   reg_grep(IndexClassesPred(m, start), start, end, expression, match_default);

}

Example: Use reg_grep to call a global callback function:

#include <string>
#include <map>
#include <regex>

// purpose:
// takes the contents of a file in the form of a string
// and searches for all the C++ class definitions, storing
// their locations in a map of strings/int's

typedef std::map<std::string, int, std::less<std::string> > map_type;

regex expression("^(template[[:space:]]*<[^;:{]+>[[:space:]]*)?(class|struct)[[:space:]]*(\\<\\w+\\>([[:blank:]]*\\([^)]*\\))?[[:space:]]*)*(\\<\\w*\\>)[[:space:]]*(<[^;:{]+>[[:space:]]*)?(\\{|:[^;\\{()]*\\{)");
map_type class_index;
std::string::const_iterator base;

bool grep_callback(const reg_match<std::string::const_iterator, regex::alloc_type>& what)
{
   // what[0] contains the whole string
   // what[5] contains the class name.
   // what[6] contains the template specialisation if any.
   // add class name and position to map:
   class_index[std::string(what[5].first, what[5].second) + std::string(what[6].first, what[6].second)] = 
               what[5].first - base;
   return true;
}

void IndexClasses(const std::string& file)
{
   std::string::const_iterator start, end;
   start = file.begin();
   end = file.end();
   base = start;
   reg_grep(grep_callback, start, end, expression, match_default);
}

 

Example use reg_grep to call a class member function, use the standard library adapters mem_fun and bind1st to convert the member function into a predicate:

#include <string>
#include <map>
#include <regex>
#include <functional>

// purpose:
// takes the contents of a file in the form of a string
// and searches for all the C++ class definitions, storing
// their locations in a map of strings/int's

typedef std::map<std::string, int, std::less<std::string> > map_type;

class class_index
{
   jm::regex expression;
   map_type index;
   std::string::const_iterator base;

   bool grep_callback(reg_match<std::string::const_iterator, regex::alloc_type> what);
public:
   void IndexClasses(const std::string& file);
   class_index()
      : index(),
        expression("^(template[[:space:]]*<[^;:{]+>[[:space:]]*)?"
                   "(class|struct)[[:space:]]*(\\<\\w+\\>([[:blank:]]*\\([^)]*\\))?"
                   "[[:space:]]*)*(\\<\\w*\\>)[[:space:]]*(<[^;:{]+>[[:space:]]*)?"
                   "(\\{|:[^;\\{()]*\\{)"
                   ){}
};

bool class_index::grep_callback(reg_match<std::string::const_iterator, regex::alloc_type> what)
{
   // what[0] contains the whole string
   // what[5] contains the class name.
   // what[6] contains the template specialisation if any.
   // add class name and position to map:
   index[std::string(what[5].first, what[5].second) + std::string(what[6].first, what[6].second)] =
               what[5].first - base;
   return true;
}

void class_index::IndexClasses(const std::string& file)
{
   std::string::const_iterator start, end;
   start = file.begin();
   end = file.end();
   base = start;
   reg_grep(std::bind1st(std::mem_fun(&class_index::grep_callback), this),
            start,
            end,
            expression,
            match_default);
}

 

Finally, C++ Builder users can use C++ Builder's closure type as a callback argument:

#include <string>
#include <map>
#include <regex>
#include <functional>

// purpose:
// takes the contents of a file in the form of a string
// and searches for all the C++ class definitions, storing
// their locations in a map of strings/int's

typedef std::map<std::string, int, std::less<std::string> > map_type;

class class_index
{
   jm::regex expression;
   map_type index;
   std::string::const_iterator base;
   typedef reg_match<std::string::const_iterator, regex::alloc_type> arg_type;

   bool grep_callback(const reg_match<std::string::const_iterator, regex::alloc_type>& what);
public:
   typedef bool (__closure* grep_callback_type)(const arg_type&);
   void IndexClasses(const std::string& file);
   class_index()
      : index(),
        expression("^(template[[:space:]]*<[^;:{]+>[[:space:]]*)?"
                   "(class|struct)[[:space:]]*(\\<\\w+\\>([[:blank:]]*\\([^)]*\\))?"
                   "[[:space:]]*)*(\\<\\w*\\>)[[:space:]]*(<[^;:{]+>[[:space:]]*)?"
                   "(\\{|:[^;\\{()]*\\{)"
                   ){}
};

bool class_index::grep_callback(const reg_match<std::string::const_iterator, regex::alloc_type>& what)
{
   // what[0] contains the whole string
   // what[5] contains the class name.
   // what[6] contains the template specialisation if any.
   // add class name and position to map:
   index[std::string(what[5].first, what[5].second) + std::string(what[6].first, what[6].second)] =
               what[5].first - base;
   return true;
}

void class_index::IndexClasses(const std::string& file)
{
   std::string::const_iterator start, end;
   start = file.begin();
   end = file.end();
   base = start;
   class_index::grep_callback_type cl = &(this->grep_callback);
   reg_grep(cl,
            start,
            end,
            expression,
            match_default);
}

 


 

Algorithm reg_format

#include <regex>

The algorithm reg_format takes the results of a match and creates a new string based upon a format string, reg_format can be used for search and replace operations:

template <class OutputIterator, class iterator, class Allocator, class charT>
OutputIterator RE_CALL reg_format(OutputIterator out,
                          const reg_match<iterator, Allocator>& m,
                          const charT* fmt
#ifdef RE_LOCALE_CPP
                         , __JM_STD::locale locale_inst = __JM_STD::locale()
#endif
                         )

 

Parameters are passed to the function are as follows:

 

OutputIterator out

An output iterator type, the output string is sent to this iterator. Typically this would be a std::ostream_iterator.

 

 

const reg_match<iterator, Allocator>& m

An instance of reg_match<> obtained from one of the matching algorithms above, and denoting what matched.

 

 

const charT* fmt

A format string that determines how the match is transformed into the new string.

 

 

__JM_STD::locale locale_inst

An optional locale, to use for the transformation, this parameter is only available when the preprocessor symbol RE_LOCALE_CPP is defined.

 

 

The format string uses a perl-like syntax: see format strings.


Algorithm reg_merge

#include <regex>

The algorithm reg_merge is a combination of reg_grep and reg_format. That is it greps through the string finding all the matches to the regular expression, for each match it then calls reg_format to format the string and sends the result to the output iterator. Sections of text that do not match are copied to the output unchanged only if the parameter copy is true.

template <class OutputIterator, class iterator, class traits, class Allocator, class charT>
OutputIterator RE_CALL reg_merge(OutputIterator out, 
                         iterator first,
                         iterator last,
                         const reg_expression<charT, traits, Allocator>& e, 
                         const charT* fmt, 
                         bool copy = true, 
                         unsigned int flags = match_default)

 

The parameters of reg_merge take the following values:

 

OutputIterator out

An output iterator type, the output string is sent to this iterator. Typically this would be a std::ostream_iterator.

 

 

iterator first

The start of the range of text to grep.

 

 

iterator last

The end of the range of text to grep.

 

 

const reg_expression<charT, traits, Allocator>& e

The expression to search for.

 

 

const charT* fmt

The format string to be applied to sections of text that match.

 

 

bool copy = true

Determines whether or not sections of text which do not match the expression are copied to output.

 

 

unsigned int flags = match_default

Flags which determine how the expression is matched - see match_flags.

 

Example: the following example takes C/C++ source code as input, and outputs syntax highlighted HTML code, note that the code has been abridged slightly - see snip4.cpp for full details.

#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <iterator>
#include <regex>
#include <fileiter.h>

// purpose:
// takes the contents of a file and transform to
// syntax highlighted code in html format

regex e1, e2;
extern const char* expression_text;
extern const char* format_string;
extern const char* pre_expression;
extern const char* pre_format;
extern const char* header_text;
extern const char* footer_text;

int main(int argc, const char** argv)
{
   e1.set_expression(expression_text);
   e2.set_expression(pre_expression);
   for(int i = 1; i < argc; ++i)
   {
      std::cout << "Processing file " << argv[i] << std::endl;
      mapfile in(argv[i]);
      std::string out_name(std::string(argv[i]) + std::string(".htm"));
      std::ofstream os(out_name.c_str());
      os << header_text;
      // strip '<' and '>' first by outputting to a
      // temporary string stream
      std::ostringstream t(std::ios::out | std::ios::binary);
      std::ostream_iterator<char, char> oi(t);
      reg_merge(oi, in.begin(), in.end(), e2, pre_format, true);
      // then output to final output stream
      // adding syntax highlighting:
      std::string s(t.str());
      std::ostream_iterator<char, char> out(os);
      reg_merge(out, s.begin(), s.end(), e1, format_string, true);
      os << footer_text;
   }
   return 0;
}

extern const char* pre_expression = "(<)|(>)|\\r";
extern const char* pre_format = "(?1<)(?2>)";


const char* expression_text = 
      // preprocessor directives: index 1
      "(^[[:blank:]]*#([^\\n]*\\\\[[:space:]]+)*[^\\n]*)|"
      // comment: index 3
      "(//[^\\n]*|/\\*([^*]|\\*+[^*/])*\\*+/)|"
      // literals: index 5
      "\\<([+-]?((0x[[:xdigit:]]+)|(([[:digit:]]*\\.)?[[:digit:]]+([eE][+-]?[[:digit:]]+)?))u?((int(8|16|32|64))|L)?)\\>|"
      // string literals: index 14
      "('([^\\\\']|\\\\.)*'|\"([^\\\\\"]|\\\\.)*\")|"
      // keywords: index 17
      "\\<(__asm|__cdecl|__declspec|__export|__far16|__fastcall|__fortran|__import"
      "|__pascal|__rtti|__stdcall|_asm|_cdecl|__except|_export|_far16|_fastcall"
      "|__finally|_fortran|_import|_pascal|_stdcall|__thread|__try|asm|auto|bool"
      "|break|case|catch|cdecl|char|class|const|const_cast|continue|default|delete"
      "|do|double|dynamic_cast|else|enum|explicit|extern|false|float|for|friend|goto"
      "|if|inline|int|long|mutable|namespace|new|operator|pascal|private|protected"
      "|public|register|reinterpret_cast|return|short|signed|sizeof|static|static_cast"
      "|struct|switch|template|this|throw|true|try|typedef|typeid|typename|union|unsigned"
      "|using|virtual|void|volatile|wchar_t|while)\\>"
      ;

const char* format_string = "(?1<font color=\"#008040\">$&</font>)"
                            "(?3<I><font color=\"#000080\">$&</font></I>)"
                            "(?5<font color=\"#0000A0\">$&</font>)"
                            "(?14<font color=\"#0000FF\">$&</font>)"
                            "(?17<B>$&</B>)";

const char* header_text = "<HTML>\n<HEAD>\n"
                          "<META HTTP-EQUIV=\"Content-Type\" CONTENT=\"text/html; charset=windows-1252\">\n"
                          "</HEAD>\n"
                          "<BODY LINK=\"#0000ff\" VLINK=\"#800080\" BGCOLOR=\"#ffff99\">\n"
                          "<PARA> </PARA>\n<PRE>";

const char* footer_text = "</PRE>\n</BODY>\n\n";

 


Class RegEx

#include <cregex>

The class RegEx provides a high level simplified interface to the regular expression library, this class only handles narrow character strings, and regular expressions always follow the "normal" syntax - that is the same as the standard POSIX extended syntax, but with locale specific collation disabled, and escape characters inside character set declarations are allowed.

typedef bool (*GrepCallback)(const RegEx& expression);
typedef bool (*GrepFileCallback)(const char* file, const RegEx& expression);
typedef bool (*FindFilesCallback)(const char* file);

class  RegEx
{
public:
   RegEx();
   RegEx(const RegEx& o);
   ~RegEx();
   RegEx(const char* c, bool icase = false);
   RegEx(const __JM_STD::string& s, bool icase = false);
   RegEx& operator=(const RegEx& o);
   RegEx& operator=(const char* p);
   RegEx& operator=(const __JM_STD::string& s);
   unsigned int SetExpression(const char* p, bool icase = false);
   unsigned int SetExpression(const __JM_STD::string& s, bool icase = false);
   __JM_STD::string Expression()const;
   //
   // now matching operators:
   //
   bool Match(const char* p, unsigned int flags = match_default);
   bool Match(const __JM_STD::string& s, unsigned int flags = match_default) ;
   bool Search(const char* p, unsigned int flags = match_default);
   bool Search(const __JM_STD::string& s, unsigned int flags = match_default) ;
   unsigned int Grep(GrepCallback cb, const char* p, unsigned int flags = match_default);
   unsigned int Grep(GrepCallback cb, const __JM_STD::string& s, unsigned int flags = match_default);
   unsigned int Grep(__JM_STD::vector<__JM_STD::string>& v, const char* p, unsigned int flags = match_default);
   unsigned int Grep(__JM_STD::vector<__JM_STD::string>& v, const __JM_STD::string& s, unsigned int flags = match_default);
   unsigned int Grep(__JM_STD::vector<unsigned int>& v, const char* p, unsigned int flags = match_default);
   unsigned int Grep(__JM_STD::vector<unsigned int>& v, const __JM_STD::string& s, unsigned int flags = match_default);
   unsigned int GrepFiles(GrepFileCallback cb, const char* files, bool recurse = false, unsigned int flags = match_default);
   unsigned int GrepFiles(GrepFileCallback cb, const __JM_STD::string& files, bool recurse = false, unsigned int flags = match_default);
   unsigned int FindFiles(FindFilesCallback cb, const char* files, bool recurse = false, unsigned int flags = match_default);
   unsigned int FindFiles(FindFilesCallback cb, const __JM_STD::string& files, bool recurse = false, unsigned int flags = match_default);
   //
   // now operators for returning what matched in more detail:
   //
   unsigned int Position(int i = 0)const;
   unsigned int Length(int i = 0)const;
   __JM_STD::string What(int i)const;
   __JM_STD::string operator[](int i)const ;
};

 

Member functions for class RegEx are defined as follows:

 

RegEx();

Default constructor, constructs an instance of RegEx without any valid expression.

 

 

RegEx(const RegEx& o);

Copy constructor, all the properties of parameter o are copied.

 

 

RegEx(const char* c, bool icase = false);

Constructs an instance of RegEx, setting the expression to c, if icase is true then matching is insensitive to case, otherwise it is sensitive to case. Throws bad_expression on failure.

 

 

RegEx(const __JM_STD::string& s, bool icase = false);

Constructs an instance of RegEx, setting the expression to s, if icase is true then matching is insensitive to case, otherwise it is sensitive to case. Throws bad_expression on failure.

 

 

RegEx& operator=(const RegEx& o);

Default assignment operator.

 

 

RegEx& operator=(const char* p);

Assignment operator, equivalent to calling SetExpression(p, false). Throws bad_expression on failure.

 

 

RegEx& operator=(const __JM_STD::string& s);

Assignment operator, equivalent to calling SetExpression(s, false). Throws bad_expression on failure.

 

 

unsigned int SetExpression(const char* p, bool icase = false);

Sets the current expression to p, if icase is true then matching is insensitive to case, otherwise it is sensitive to case. Throws bad_expression on failure.

 

 

unsigned int SetExpression(const __JM_STD::string& s, bool icase = false);

Sets the current expression to s, if icase is true then matching is insensitive to case, otherwise it is sensitive to case. Throws bad_expression on failure.

 

 

__JM_STD::string Expression()const;

Returns a copy of the current regular expression.

 

 

bool Match(const char* p, unsigned int flags = match_default);

Attempts to match the current expression against the text p using the match flags flags - see match flags. Returns true if the match succeeds.

 

 

bool Match(const __JM_STD::string& s, unsigned int flags = match_default) ;

Attempts to match the current expression against the text s using the match flags flags - see match flags. Returns true if the match succeeds.

 

 

bool Search(const char* p, unsigned int flags = match_default);

Attempts to find a match for the current expression somewhere in the text p using the match flags flags - see match flags. Returns true if the match succeeds.

 

 

bool Search(const __JM_STD::string& s, unsigned int flags = match_default) ;

Attempts to find a match for the current expression somewhere in the text s using the match flags flags - see match flags. Returns true if the match succeeds.

 

 

unsigned int Grep(GrepCallback cb, const char* p, unsigned int flags = match_default);

Finds all matches of the current expression in the text p using the match flags flags - see match flags. For each match found calls the call-back function cb as: cb(*this);

If at any stage the call-back function returns false then the grep operation terminates, otherwise continues until no further matches are found. Returns the number of matches found.

 

 

unsigned int Grep(GrepCallback cb, const __JM_STD::string& s, unsigned int flags = match_default);

Finds all matches of the current expression in the text s using the match flags flags - see match flags. For each match found calls the call-back function cb as: cb(*this);

If at any stage the call-back function returns false then the grep operation terminates, otherwise continues until no further matches are found. Returns the number of matches found.

 

 

unsigned int Grep(__JM_STD::vector<__JM_STD::string>& v, const char* p, unsigned int flags = match_default);

Finds all matches of the current expression in the text p using the match flags flags - see match flags. For each match pushes a copy of what matched onto v. Returns the number of matches found.

 

 

unsigned int Grep(__JM_STD::vector<__JM_STD::string>& v, const __JM_STD::string& s, unsigned int flags = match_default);

Finds all matches of the current expression in the text s using the match flags flags - see match flags. For each match pushes a copy of what matched onto v. Returns the number of matches found.

 

 

unsigned int Grep(__JM_STD::vector<unsigned int>& v, const char* p, unsigned int flags = match_default);

Finds all matches of the current expression in the text p using the match flags flags - see match flags. For each match pushes the starting index of what matched onto v. Returns the number of matches found.

 

 

unsigned int Grep(__JM_STD::vector<unsigned int>& v, const __JM_STD::string& s, unsigned int flags = match_default);

Finds all matches of the current expression in the text s using the match flags flags - see match flags. For each match pushes the starting index of what matched onto v. Returns the number of matches found.

 

 

unsigned int GrepFiles(GrepFileCallback cb, const char* files, bool recurse = false, unsigned int flags = match_default);

Finds all matches of the current expression in the files files using the match flags flags - see match flags. For each match calls the call-back function cb. If the call-back returns false then the algorithm skips that file and moves on to the next. The parameter files can include wild card characters '*' and '?', if the parameter recurse is true then searches sub-directories for matching file names. Returns the total number of matches found.

 

 

unsigned int GrepFiles(GrepFileCallback cb, const __JM_STD::string& files, bool recurse = false, unsigned int flags = match_default);

Finds all matches of the current expression in the files files using the match flags flags - see match flags. For each match calls the call-back function cb. If the call-back returns false then the algorithm skips that file and moves on to the next. The parameter files can include wild card characters '*' and '?', if the parameter recurse is true then searches sub-directories for matching file names. Returns the total number of matches found.

 

 

unsigned int FindFiles(FindFilesCallback cb, const char* files, bool recurse = false, unsigned int flags = match_default);

As GrepFiles, but finds no more than one match per file, no information on where the match occurred is passed to the call-back - just the file name. Returns the total number of files that contain a match to the current expression.

 

 

unsigned int FindFiles(FindFilesCallback cb, const __JM_STD::string& files, bool recurse = false, unsigned int flags = match_default);

As GrepFiles, but finds no more than one match per file, no information on where the match occurred is passed to the call-back - just the file name. Returns the total number of files that contain a match to the current expression.

 

 

unsigned int Position(int i = 0)const;

Returns the position of what matched sub-expression i. If i = 0 then returns the position of the whole match. Returns -1 if the supplied index is invalid, or if the specified sub-expression did not participate in the match.

 

 

unsigned int Length(int i = 0)const;

Returns the length of what matched sub-expression i. If i = 0 then returns the length of the whole match. Returns -1 if the supplied index is invalid, or if the specified sub-expression did not participate in the match.

 

 

__JM_STD::string What(int i)const;

Returns a copy of what matched sub-expression i. If i = 0 then returns a copy of the whole match. Returns a null string if the index is invalid or if the specified sub-expression did not participate in a match.

 

 

__JM_STD::string operator[](int i)const ;

Returns what(i);

Can be used to simplify access to sub-expression matches, and make usage more perl-like.

 

 


POSIX compatibility library

#include <cregex>

The following functions are available for users who need a POSIX compatible C library, they are available in both Unicode and narrow character versions, the standard POSIX API names are macros that expand to one version or the other depending upon whether UNICODE is defined or not.

The functions are defined as:

extern "C" {
int regcompA(regex_tA*, const char*, int);
unsigned int regerrorA(int, const regex_tA*, char*, unsigned int);
int regexecA(const regex_tA*, const char*, unsigned int, regmatch_t*, int);
void regfreeA(regex_tA*);

int regcompW(regex_tW*, const wchar_t*, int);
unsigned int regerrorW(int, const regex_tW*, wchar_t*, unsigned int);
int regexecW(const regex_tW*, const wchar_t*, unsigned int, regmatch_t*, int);
void regfreeW(regex_tW*);

#ifdef UNICODE
#define regcomp regcompW
#define regerror regerrorW
#define regexec regexecW
#define regfree regfreeW
#define regex_t regex_tW
#else
#define regcomp regcompA
#define regerror regerrorA
#define regexec regexecA
#define regfree regfreeA
#define regex_t regex_tA
#endif
}

All the functions operate on structure regex_t, which exposes two public members:

unsigned int re_nsub this is filled in by regcomp and indicates the number of sub-expressions contained in the regular expression.

const TCHAR* re_endp points to the end of the expression to compile when the flag REG_PEND is set.

Footnote: regex_t is actually a typedef - it is either regex_tA or regex_tW depending upon whether UNICODE is defined or not, TCHAR is either char or wchar_t again depending upon the macro UNICODE.

 

regcomp takes a pointer to a regex_t, a pointer to the expression to compile and a flags parameter which can be a combination of:

 

REG_EXTENDED

Compiles modern regular expressions. Equivalent to regbase::char_classes | regbase::intervals | regbase::bk_refs.

 

 

REG_BASIC

Compiles basic (obsolete) regular expression syntax. Equivalent to regbase::char_classes | regbase::intervals | regbase::limited_ops | regbase::bk_braces | regbase::bk_parens | regbase::bk_refs.

 

 

REG_NOSPEC

All characters are ordinary, the expression is a literal string.

 

 

REG_ICASE

Compiles for matching that ignores character case.

 

 

REG_NOSUB

Has no effect in this library.

 

 

REG_NEWLINE

When this flag is set a dot does not match the newline character.

 

 

REG_PEND

When this flag is set the re_endp parameter of the regex_t structure must point to the end of the regular expression to compile.

 

 

regerror takes the following parameters, it maps an error code to a human readable string:

 

int code

The error code.

 

 

const regex_t* e

The regular expression (can be null).

 

 

char* buf

The buffer to fill in with the error message.

 

 

unsigned int buf_size

The length of buf.

 

If the error code is OR'ed with REG_ITOA then the message that results is the printable name of the code rather than a message, for example "REG_BADPAT". If the code is REG_ATIO then e must not be null and e->re_pend must point to the printable name of an error code, the return value is then the value of the error code. For any other value of code, the return value is the number of characters in the error message, if the return value is greater than or equal to buf_size then regerror will have to be called again with a larger buffer.

 

regexec finds the first occurrence of expression e within string buf. If len is non-zero then *m is filled in with what matched the regular expression, m[0] contains what matched the whole string, m[1] the first sub-expression etc, see regmatch_t in the header file declaration for more details. The eflags parameter can be a combination of:

 

REG_NOTBOL

Parameter buf does not represent the start of a line.

 

 

REG_NOTEOL

Parameter buf does not terminate at the end of a line.

 

 

REG_STARTEND

The string searched starts at buf + pmatch[0].rm_so and ends at buf + pmatch[0].rm_eo.

 

 

Finally regfree frees all the memory that was allocated by regcomp.

Footnote: this is an abridged reference to the POSIX API functions, it is provided for compatibility with other libraries, rather than an API to be used in new code (unless you need access from a language other than C++). This version of these functions should also happily co-exist with other versions, as the names used are macros that expand to the actual function names.


Regular expression syntax

This section covers the regular expression syntax used by this library, this is a programmers guide, the actual syntax presented to your program's users will depend upon the flags used during expression compilation.

Literals

All characters are literals except: ".", "*", "?", "+", "(", ")", "{", "}", "[", "]", "^" and "$". These characters are literals when preceded by a "\". A literal is a character that matches itself, or matches the result of traits_type::translate(), where traits_type is the traits template parameter to class reg_expression.

 

Wildcard

The dot character "." matches any single character except : when match_not_dot_null is passed to the matching algorithms, the dot does not match a null character; when match_not_dot_newline is passed to the matching algorithms, then the dot does not match a newline character.

 

Repeats

A repeat is an expression that is repeated an arbitrary number of times. An expression followed by "*" can be repeated any number of times including zero. An expression followed by "+" can be repeated any number of times, but at least once, if the expression is compiled with the flag regbase::bk_plus_qm then "+" is an ordinary character and "\+" represents a repeat of once or more. An expression followed by "?" may be repeated zero or one times only, if the expression is compiled with the flag regbase::bk_plus_qm then "?" is an ordinary character and "\?" represents the repeat zero or once operator. When it is necessary to specify the minimum and maximum number of repeats explicitly, the bounds operator "{}" may be used, thus "a{2}" is the letter "a" repeated exactly twice, "a{2,4}" represents the letter "a" repeated between 2 and 4 times, and "a{2,}" represents the letter "a" repeated at least twice with no upper limit. Note that there must be no white-space inside the {}, and there is no upper limit on the values of the lower and upper bounds. When the expression is compiled with the flag regbase::bk_braces then "{" and "}" are ordinary characters and "\{" and "\}" are used to delimit bounds instead. All repeat expressions refer to the shortest possible previous sub-expression: a single character; a character set, or a sub-expression grouped with "()" for example.

Examples:

"ba*" will match all of "b", "ba", "baaa" etc.

"ba+" will match "ba" or "baaaa" for example but not "b".

"ba?" will match "b" or "ba".

"ba{2,4}" will match "baa", "baaa" and "baaaa".

 

Parenthesis

Parentheses serve two purposes, to group items together into a sub-expression, and to mark what generated the match. For example the expression "(ab)*" would match all of the string "ababab". The matching algorithms query_match and reg_search each take an instance of reg_match that reports what caused the match, on exit from these functions the reg_match contains information both on what the whole expression matched and on what each sub-expression matched. In the example above reg_match[1] would contain a pair of iterators denoting the final "ab" of the matching string. It is permissible for sub-expressions to match null strings. If a sub-expression takes no part in a match - for example if it is part of an alternative that is not taken - then both of the iterators that are returned for that sub-expression point to the end of the input string, and the matched parameter for that sub-expression is false. Sub-expressions are indexed from left to right starting from 1, sub-expression 0 is the whole expression.

 

Alternatives

Alternatives occur when the expression can match either one sub-expression or another, each alternative is separated by a "|", or a "\|" if the flag regbase::bk_vbar is set, or by a newline character if the flag regbase::newline_alt is set. Each alternative is the largest possible previous sub-expression; this is the opposite behaviour from repetition operators.

Examples:

"a(b|c)" could match "ab" or "ac".

"abc|def" could match "abc" or "def".

 

Sets

A set is a set of characters that can match any single character that is a member of the set. Sets are delimited by "[" and "]" and can contain literals, character ranges, character classes, collating elements and equivalence classes. Set declarations that start with "^" contain the compliment of the elements that follow.

Examples:

Character literals:

"[abc]" will match either of "a", "b", or "c".

"[^abc] will match any character other than "a", "b", or "c".

Character ranges:

"[a-z]" will match any character in the range "a" to "z".

"[^A-Z]" will match any character other than those in the range "A" to "Z".

Note that character ranges are highly locale dependent: they match any character that collates between the endpoints of the range, ranges will only behave according to ASCII rules when the default "C" locale is in effect. For example if the library is compiled with the Win32 localisation model, then [a-z] will match the ASCII characters a-z, and also 'A', 'B' etc, but not 'Z' which collates just after 'z'. This locale specific behaviour can be disabled by specifying regbase::nocollate when compiling, this is the default behaviour when using regbase::normal, and forces ranges to collate according to ASCII character code.

Character classes are denoted using the syntax "[:classname:]" within a set declaration, for example "[[:space:]]" is the set of all whitespace characters. Character classes are only available if the flag regbase::char_classes is set. The available character classes are:

 

alnum

Any alpha numeric character.

 

 

alpha

Any alphabetical character a-z and A-Z. Other characters may also be included depending upon the locale.

 

 

blank

Any blank character, either a space or a tab.

 

 

cntrl

Any control character.

 

 

digit

Any digit 0-9.

 

 

graph

Any graphical character.

 

 

lower

Any lower case character a-z. Other characters may also be included depending upon the locale.

 

 

print

Any printable character.

 

 

punct

Any punctuation character.

 

 

space

Any whitespace character.

 

 

upper

Any upper case character A-Z. Other characters may also be included depending upon the locale.

 

 

xdigit

Any hexadecimal digit character, 0-9, a-f and A-F.

 

 

word

Any word character - all alphanumeric characters plus the underscore.

 

 

unicode

Any character whose code is greater than 255, this applies to the wide character traits classes only.

 

There are some shortcuts that can be used in place of the character classes, provided the flag regbase::escape_in_lists is set then you can use:

\w in place of [:word:]

\s in place of [:space:]

\d in place of [:digit:]

\l in place of [:lower:]

\u in place of [:upper:]

 

Collating elements take the general form [.tagname.] inside a set declaration, where tagname is either a single character, or a name of a collating element, for example [[.a.]] is equivalent to [a], and [[.comma.]] is equivalent to [,]. The library supports all the standard POSIX collating element names, and in addition the following digraphs: "ae", "ch", "ll", "ss", "nj", "dz", "lj", each in lower, upper and title case variations. Multi-character collating elements can result in the set matching more than one character, for example [[.ae.]] would match two characters, but note that [^[.ae.]] would only match one character.

 

Equivalence classes take the general form [=tagname=] inside a set declaration, where tagname is either a single character, or a name of a collating element, and matches any character that is a member of the same primary equivalence class as the collating element [.tagname.]. An equivalence class is a set of characters that collate the same, a primary equivalence class is a set of characters whose primary sort key are all the same (for example strings are typically collated by character, then by accent, and then by case; the primary sort key then relates to the character, the secondary to the accentation, and the tertiary to the case). If there is no equivalence class corresponding to tagname, then [=tagname=] is exactly the same as [.tagname.]. Unfortunately there is no locale independent method of obtaining the primary sort key for a character, except under Win32. For other operating systems the library will "guess" the primary sort key from the full sort key (obtained from strxfrm), so equivalence classes are probably best considered broken under any operating system other than Win32.

 

To include a literal "-" in a set declaration then: make it the first character after the opening "[" or "[^", the endpoint of a range, a collating element, or if the flag regbase::escape_in_lists is set then precede with an escape character as in "[\-]". To include a literal "[" or "]" or "^" in a set then make them the endpoint of a range, a collating element, or precede with an escape character if the flag regbase::escape_in_lists is set.

 

Line anchors

An anchor is something that matches the null string at the start or end of a line: "^" matches the null string at the start of a line, "$" matches the null string at the end of a line.

 

Back references

A back reference is a reference to a previous sub-expression that has already been matched, the reference is to what the sub-expression matched, not to the expression itself. A back reference consists of the escape character "\" followed by a digit "1" to "9", "\1" refers to the first sub-expression, "\2" to the second etc. For example the expression "(.*)\1" matches any string that is repeated about its mid-point for example "abcabc" or "xyzxyz". A back reference to a sub-expression that did not participate in any match, matches the null string: NB this is different to some other regular expression matchers. Back references are only available if the expression is compiled with the flag regbase::bk_refs set.

 

Characters by code

This is an extension to the algorithm that is not available in other libraries, it consists of the escape character followed by the digit "0" followed by the octal character code. For example "\023" represents the character whose octal code is 23. Where ambiguity could occur use parentheses to break the expression up: "\0103" represents the character whose code is 103, "(\010)3 represents the character 10 followed by "3". To match characters by their hexadecimal code, use \x followed by a string of hexadecimal digits, optionally enclosed inside {}, for example \xf0 or \x{aff}, notice the latter example is a Unicode character.

 

Word operators

The following operators are provided for compatibility with the GNU regular expression library.

"\w" matches any single character that is a member of the "word" character class, this is identical to the expression "[[:word:]]".

"\W" matches any single character that is not a member of the "word" character class, this is identical to the expression "[^[:word:]]".

"\<" matches the null string at the start of a word.

"\>" matches the null string at the end of the word.

"\b" matches the null string at either the start or the end of a word.

"\B" matches a null string within a word.

The start of the sequence passed to the matching algorithms is considered to be a potential start of a word unless the flag match_not_bow is set. The end of the sequence passed to the matching algorithms is considered to be a potential end of a word unless the flag match_not_eow is set.

 

Buffer operators

The following operators are provide for compatibility with the GNU regular expression library, and Perl regular expressions:

"\`" matches the start of a buffer.

"\A" matches the start of the buffer.

"\'" matches the end of a buffer.

"\z" matches the end of a buffer.

"\Z" matches the end of a buffer, or possibly one or more new line characters followed by the end of the buffer.

A buffer is considered to consist of the whole sequence passed to the matching algorithms, unless the flags match_not_bob or match_not_eob are set.

 

Escape operator

The escape character "\" has several meanings.

Inside a set declaration the escape character is a normal character unless the flag regbase::escape_in_lists is set in which case whatever follows the escape is a literal character regardless of its normal meaning.

The escape operator may introduce an operator for example: back references, or a word operator.

The escape operator may make the following character normal, for example "\*" represents a literal "*" rather than the repeat operator.

 

Single character escape sequences

The following escape sequences are aliases for single characters:

 

Escape sequence

Character code

Meaning

 

 

\a

0x07

Bell character.

 

 

\f

0x08

Form feed.

 

 

\n

0x0C

Newline character.

 

 

\r

0x0D

Carriage return.

 

 

\t

0x09

Tab character.

 

 

\v

0x0B

Vertical tab.

 

 

\e

0x1B

ASCII Escape character.

 

 

\0dd

0dd

An octal character code, where dd is one or more octal digits.

 

 

\xXX

0xXX

A hexadecimal character code, where XX is one or more hexadecimal digits.

 

 

\x{XX}

0xXX

A hexadecimal character code, where XX is one or more hexadecimal digits, optionally a unicode character.

 

 

\cZ

z-@

An ASCII escape sequence control-Z, where Z is any ASCII character greater than or equal to the character code for '@'.

 

 

Miscellaneous escape sequences:

The following are provided mostly for perl compatibility, but note that there are some differences in the meanings of \l \L \u and \U:

 

\w

Equivalent to [[:word:]].

 

 

\W

Equivalent to [^[:word:]].

 

 

\s

Equivalent to [[:space:]].

 

 

\S

Equivalent to [^[:space:]].

 

 

\d

Equivalent to [[:digit:]].

 

 

\D

Equivalent to [^[:digit:]].

 

 

\l

Equivalent to [[:lower:]].

 

 

\L

Equivalent to [^[:lower:]].

 

 

\u

Equivalent to [[:upper:]].

 

 

\U

Equivalent to [^[:upper:]].

 

 

\C

Any single character, equivalent to '.'.

 

 

\X

Match any Unicode combining character sequence, for example "a\x 0301" (a letter a with an accute).

 

 

\Q

The begin quote operator, everything that follows is treated as a literal character until a \E end quote operator is found.

 

 

\E

The end quote operator, terminates a sequence begun with \Q.

 

 

What gets matched?

The regular expression library will match the first possible matching string, if more than one string starting at a given location can match then it matches the longest possible string, unless the flag match_any is set, in which case the first match encountered is returned. Use of the match_any option can reduce the time taken to find the match - but is only useful if the user is less concerned about what matched - for example it would not be suitable for search and replace operations. In cases where their are multiple possible matches all starting at the same location, and all of the same length, then the match chosen is the one with the longest first sub-expression, if that is the same for two or more matches, then the second sub-expression will be examined and so on.


Format String Syntax

Format strings are used by the algorithms reg_format and reg_merge, and are used to transform one string into another.

In format strings, all characters are treated as literals except: ()$\?:

To use any of these as literals you must prefix them with the escape character \

The following special sequences are recognised:

 

Grouping:

Use the parenthesis characters ( and ) to group sub-expressions within the format string, use \( and \) to represent literal '(' and ')'.

 

Sub-expression expansions:

The following perl like expressions expand to a particular matched sub-expression:

 

$`

Expands to all the text from the end of the previous match to the start of the current match, if there was no previous match in the current operation, then everything from the start of the input string to the start of the match.

 

 

$'

Expands to all the text from the end of the match to the end of the input string.

 

 

$&

Expands to all of the current match.

 

 

$0

Expands to all of the current match.

 

 

$N

Expands to the text which matched sub-expression N.

 

 

Conditional expressions:

Conditional expressions allow two different format strings to be selected dependent upon whether a sub-expression participated in the match or not:

?Ntrue_expression:false_expression

Executes true_expression if sub-expression N participated in the match, otherwise executes false_expression.

Example: suppose we search for "(while)|(for)" then the format string "?1WHILE:FOR" would output what matched, but in upper case.

 

Escape sequences:

The following escape sequences are also allowed:

 

\a

The bell character.

 

 

\f

The form feed character.

 

 

\n

The newline character.

 

 

\r

The carriage return character.

 

 

\t

The tab character.

 

 

\v

A vertical tab character.

 

 

\x

A hexadecimal character - for example \x0D.

 

 

\x{}

A possible unicode hexadecimal character - for example \x{1A0}

 

 

\cx

The ASCII escape character x, for example \c@ is equivalent to escape-@.

 

 

\e

The ASCII escape character.

 

 

\dd

An octal character constant, for example \10.

 

 


Appendix 1: Implementation notes

This is the second full release, see changes.txt for a full list of changes from the previous version. There are no known functionality bugs except that POSIX style equivalence classes are only guaranteed correct if the Win32 localisation model is used (the default for Win32 builds of the library).

There are some aspects of the code that C++ puritans will consider to be poor style, in particular the use of goto in some of the algorithms. The code could be cleaned up, by changing to a recursive implementation, although it is likely to be slower in that case.

The performance of the algorithms should be satisfactory in most cases. For example the times taken to match the ftp response expression "^([0-9]+)(\-| |$)(.*)$" against the string "100- this is a line of ftp response which contains a message string" are: BSD implementation 450 micro seconds, GNU implementation 271 micro seconds, regex++ 127 micro seconds (Pentium P90, Win32 console app under MS Windows 95).

However it should be noted that there are some "pathological" expressions which may require exponential time for matching; these all involve nested repetition operators, for example attempting to match the expression "(a*a)*b" against N letter a's requires time proportional to 2N. These expressions can (almost) always be rewritten in such a way as to avoid the problem, for example "(a*a)*b" could be rewritten as "a*b" which requires only time linearly proportional to N to solve. In the general case, non-nested repeat expressions require time proportional to N2, however if the clauses are mutually exclusive then they can be matched in linear time - this is the case with "a*b", for each character the matcher will either match an "a" or a "b" or fail, where as with "a*a" the matcher can't tell which branch to take (the first "a" or the second) and so has to try both. Be careful how you write your regular expressions and avoid nested repeats if you can! New to this version, some previously pathological cases have been fixed - in particular searching for expressions which contain leading repeats and/or leading literal strings should be much faster than before. Literal strings are now searched for using the Knuth/Morris/Pratt algorithm (this is used in preference to the Boyer/More algorithm because it allows the tracking of newline characters).

Some aspects of the POSIX regular expression syntax are implementation defined:


Appendix 2: Thread safety

Class reg_expression<> and its typedefs regex and wregex are thread safe, in that compiled regular expressions can safely be shared between threads. The matching algorithms query_match, reg_search, reg_grep, reg_format and reg_merge are all reentrant and thread safe. Class reg_match is now thread safe, in that the results of a match can be safely copied from one thread to another (for example one thread may find matches and push reg_match instances onto a queue, while another thread pops them off the other end), otherwise use a separate instance of reg_match per thread.

The POSIX API functions are all reentrant and thread safe, regular expressions compiled with regcomp can also be shared between threads.

The class RegEx is only thread safe if each thread gets its own RegEx instance (apartment threading) - this is a consequence of RegEx handling both compiling and matching regular expressions.

Finally note that changing the global locale invalidates all compiled regular expressions, therefore calling set_locale from one thread while another uses regular expressions will produce an unpredictable results.


Appendix 3: Localisation

 Regex++ provides extensive support for run-time localisation, the localisation model used can be split into two parts: front-end and back-end.

Front-end localisation deals with everything which the user sees - error messages, and the regular expression syntax itself. For example a French application could change [[:word:]] to [[:mot:]] and \w to \m. Modifying the front end locale requires active support from the developer, by providing the library with a message catalogue to load, containing the localised strings. Front-end locale is affected by the LC_MESSAGES category only.

Back-end localisation deals with everything that occurs after the expression has been parsed - in other words everything that the user does not see or interact with directly. It deals with case conversion, collation, and character class membership. The back-end locale does not require any intervention from the developer - the library will acquire all the information it requires for the current locale from the underlying operating system / run time library. This means that if the program user does not interact with regular expressions directly - for example if the expressions are embedded in your C++ code - then no explicit localisation is required, as the library will take care of everything for you. For example embedding the expression [[:word:]]+ in your code will always match a whole word, if the program is run on a machine with, for example, a Greek locale, the it will still match a whole word, but in Greek characters rather than Latin ones. The back-end locale is affected by the LC_TYPE and LC_COLLATE categories.

There are three separate localisation mechanisms supported by regex++:

Win32 localisation model.

This is the default model when the library is compiled under Win32. When this model is in effect there is a single global locale as defined by the user's control panel settings, and returned by GetUserDefaultLCID. All the settings used by regex++ are acquired directly from the operating system bypassing the C run time library. Front-end localisation requires a resource dll, containing a string table with the user defined strings. The library exports the symbol:

const char* regex_message_catalogue = 0;

which needs to be set to point to a string containing the name of the resource dll, before your code compiles any regular expressions.

This model does not currently support thread specific locales (via SetThreadLocale under Windows NT), the library provides full Unicode support under NT, under Windows 9x the library degrades gracefully - characters 0 to 255 are supported, the remainder are treated as "unknown" graphic characters.

C localisation model.

This is the default model when the library is compiled under an operating system other than Win32, Win32 users can force this model to take effect by defining the preprocessor symbol RE_LOCALE_C. When this model is in effect there is a single global locale, as set by setlocale. All settings are acquired from your run time library, consequently Unicode support is dependent upon your run time library implimentation. Front end localisation requires a POSIX message catalogue, the library exports the symbol:

const char* regex_message_catalogue = 0;

which needs to be set to point to a string containing the name of the message catalogue, before your code compiles any regular expressions. If your run time library does not support POSIX message catalogues, then you can either provide your own implementation of <nl_types.h> or define JM_NO_CAT to disable front-end localisation via message catalogues.

Note that calling setlocale invalidates all compiled regular expressions, calling setlocale(LC_ALL, "C") will make this library behave equivalent to most traditional regular expression libraries including version 1 of this library.

C++ localisation model.

This model is only in effect if the library is built with the preprocessor symbol RE_LOCALE_CPP defined. When this model is in effect each instance of reg_expression<> has its own instance of std::locale, class reg_expression<> also has a member function imbue which allows the locale for the expression to be set on a per-instance basis. Front end localisation requires a POSIX message catalogue, which will be loaded via the std::messages facet of the expression's locale, the library exports the symbol:

const char* regex_message_catalogue = 0;

which needs to be set to point to a string containing the name of the message catalogue, before your code compiles any regular expressions.

Note that calling reg_expression<>::imbue will invalidate any expression currently compiled in that instance of reg_expression<>. This model is the one which closest fits the ethos of the C++ standard library, however it is the model which will produce the slowest code, and which is the least well supported by current standard library implimentations, for example I have yet to find an implementation of std::locale which support either message catalogues, or locales other than "C" or "POSIX".

Finally note that if you build the library with a non-default localisation model, then the appropriate preprocessor symbol (RE_LOCALE_C or RE_LOCALE_CPP) must be defined both when you build the support library, and when you include <regex> or <cregex> in your code. The best way to ensure this is to add the #define to <jm/jm_opt.h>.

Providing a message catalogue: 

In order to localise the front end of the library, you need to provide the library with the appropriate message strings contained either in a resource dll's string table (Win32 model), or a POSIX message catalogue (C or C++ models). In the latter case the messages must appear in message set zero of the catalogue. The messages and their id's are as follows:

 

Message id

Meaning

Default value

 

 

101

The character used to start a sub-expression.

"("

 

 

102

The character used to end a sub-expression declaration.

")"

 

 

103

The character used to denote an end of line assertion.

"$"

 

 

104

The character used to denote the start of line assertion.

"^"

 

 

105

The character used to denote the "match any character expression".

"."

 

 

106

The match zero or more times repetition operator.

"*"

 

 

107

The match one or more repetition operator.

"+"

 

 

108

The match zero or one repetition operator.

"?"

 

 

109

The character set opening character.

"["

 

 

110

The character set closing character.

"]"

 

 

111

The alternation operator.

"|"

 

 

112

The escape character.

"\\"

 

 

113

The hash character (not currently used).

"#"

 

 

114

The range operator.

"-"

 

 

115

The repetition operator opening character.

"{"

 

 

116

The repetition operator closing character.

"}"

 

 

117

The digit characters.

"0123456789"

 

 

118

The character which when preceded by an escape character represents the word boundary assertion.

"b"

 

 

119

The character which when preceded by an escape character represents the non-word boundary assertion.

"B"

 

 

120

The character which when preceded by an escape character represents the word-start boundary assertion.

"<"

 

 

121

The character which when preceded by an escape character represents the word-end boundary assertion.

">"

 

 

122

The character which when preceded by an escape character represents any word character.

"w"

 

 

123

The character which when preceded by an escape character represents a non-word character.

"W"

 

 

124

The character which when preceded by an escape character represents a start of buffer assertion.

"`A"

 

 

125

The character which when preceded by an escape character represents an end of buffer assertion.

"'z"

 

 

126

The newline character.

"\n"

 

 

127

The comma separator.

","

 

 

128

The character which when preceded by an escape character represents the bell character.

"a"

 

 

129

The character which when preceded by an escape character represents the form feed character.

"f"

 

 

130

The character which when preceded by an escape character represents the newline character.

"n"

 

 

131

The character which when preceded by an escape character represents the carriage return character.

"r"

 

 

132

The character which when preceded by an escape character represents the tab character.

"t"

 

 

133

The character which when preceded by an escape character represents the vertical tab character.

"v"

 

 

134

The character which when preceded by an escape character represents the start of a hexadecimal character constant.

"x"

 

 

135

The character which when preceded by an escape character represents the start of an ASCII escape character.

"c"

 

 

136

The colon character.

":"

 

 

137

The equals character.

"="

 

 

138

The character which when preceded by an escape character represents the ASCII escape character.

"e"

 

 

139

The character which when preceded by an escape character represents any lower case character.

"l"

 

 

140

The character which when preceded by an escape character represents any non-lower case character.

"L"

 

 

141

The character which when preceded by an escape character represents any upper case character.

"u"

 

 

142

The character which when preceded by an escape character represents any non-upper case character.

"U"

 

 

143

The character which when preceded by an escape character represents any space character.

"s"

 

 

144

The character which when preceded by an escape character represents any non-space character.

"S"

 

 

145

The character which when preceded by an escape character represents any digit character.

"d"

 

 

146

The character which when preceded by an escape character represents any non-digit character.

"D"

 

 

147

The character which when preceded by an escape character represents the end quote operator.

"E"

 

 

148

The character which when preceded by an escape character represents the start quote operator.

"Q"

 

 

149

The character which when preceded by an escape character represents a Unicode combining character sequence.

"X"

 

 

150

The character which when preceded by an escape character represents any single character.

"C"

 

 

151

The character which when preceded by an escape character represents end of buffer operator.

"Z"

 

 

152

The character which when preceded by an escape character represents the continuation assertion.

"G"

 

 

Custom error messages are loaded as follows:

 

Message ID

Error message ID

Default string

 

 

201

REG_NOMATCH

"No match"

 

 

202

REG_BADPAT

"Invalid regular expression"

 

 

203

REG_ECOLLATE

"Invalid collation character"

 

 

204

REG_ECTYPE

"Invalid character class name"

 

 

205

REG_EESCAPE

"Trailing backslash"

 

 

206

REG_ESUBREG

"Invalid back reference"

 

 

207

REG_EBRACK

"Unmatched [ or [^"

 

 

208

REG_EPAREN

"Unmatched ( or \\("

 

 

209

REG_EBRACE

"Unmatched \\{"

 

 

210

REG_BADBR

"Invalid content of \\{\\}"

 

 

211

REG_ERANGE

"Invalid range end"

 

 

212

REG_ESPACE

"Memory exhausted"

 

 

213

REG_BADRPT

"Invalid preceding regular expression"

 

 

214

REG_EEND

"Premature end of regular expression"

 

 

215

REG_ESIZE

"Regular expression too big"

 

 

216

REG_ERPAREN

"Unmatched ) or \\)"

 

 

217

REG_EMPTY

"Empty expression"

 

 

218

REG_E_MEMORY

"Out of memory"

 

 

219

REG_E_UNKNOWN

"Unknown error"

 

 

Custom character class names are loaded as followed:

 

Message ID

Description

Equivalent default class name

 

 

300

The character class name for alpha-numeric characters.

"alnum"

 

 

301

The character class name for alphabetic characters.

"alpha"

 

 

302

The character class name for control characters.

"cntrl"

 

 

303

The character class name for digit characters.

"digit"

 

 

304

The character class name for graphics characters.

"graph"

 

 

305

The character class name for lower case characters.

"lower"

 

 

306

The character class name for printable characters.

"print"

 

 

307

The character class name for punctuation characters.

"punct"

 

 

308

The character class name for space characters.

"space"

 

 

309

The character class name for upper case characters.

"upper"

 

 

310

The character class name for hexadecimal characters.

"xdigit"

 

 

311

The character class name for blank characters.

"blank"

 

 

312

The character class name for word characters.

"word"

 

 

313

The character class name for Unicode characters.

"unicode"

 

 

Finally, custom collating element names are loaded starting from message id 400, and terminating when the first load thereafter fails. Each message looks something like: "tagname string" where tagname is the name used inside [[.tagname.]] and string is the actual text of the collating element. Note that the value of collating element [[.zero.]] is used for the conversion of strings to numbers - if you replace this with another value then that will be used for string parsing - for example use the Unicode character 0x0660 for [[.zero.]] if you want to use Unicode Arabic-Indic digits in your regular expressions in place of Latin digits.

Note that the POSIX defined names for character classes and collating elements are always available - even if custom names are defined, in contrast, custom error messages, and custom syntax messages replace the default ones.


Appendix 4: Demo Applications

There are three demo applications that ship with this library, they all come with makefiles for Borland, Microsoft and EGCS compilers, otherwise you will have to create your own makefiles.

regress.exe:

A regression test application that gives the matching/searching algorithms a full workout. The presence of this program is your guarantee that the library will behave as claimed - at least as far as those items tested are concerned - if anyone spots anything that isn't being tested I'd be glad to hear about it.

Files: parse.cpp, regress.cpp, tests.cpp.

 

jgrep.exe

A simple grep implimentation, run with no command line options to find out its usage. Look at fileiter.cpp/.h and the mapfile class to see an example of a "smart" bidirectional iterator that can be used with regex++ or any other STL algorithm.

Files: jgrep.cpp, main.cpp.

 

timer.exe

A simple interactive expression matching application, the results of all matches are timed, allowing the programmer to optimise their regular expressions where performance is critical.

Files: timer.cpp.


Appendix 5: Header Files

There are two main headers used by this library: <regex> provides full access to the entire library, while <cregex> provides access to just the high level class RegEx, and the POSIX API functions. As an alternative you can use <regex.h> or <jm/regex.h> in place of <regex>, and <cregex.h> or <jm/cregex.h> in place of <cregex>. The use of the <jm/header.h> syntax is recommended for those who already have a version of regex.h on their system, which they do not wish to replace, or which they wish to use in parallel with this library.


Appendix 6: Redistributables

 If you are using Microsoft or Borland C++ and link to a dll version of the run time library, then you will also link to one of the dll versions of regex++. While these dll's are redistributable, there are no "standard" versions, so when installing on the users PC, you should place these in a directory private to your application, and not in the PC's directory path. Note that if you link to a static version of your run time library, then you will also link to a static version of regex++ and no dll's will need to be distributed. The possible regex++ dll's are as follows:

 

Development Tool

Run Time Library

Regex++ Dll

 
 

Microsoft Visual C++ 5/6

Msvcp60.dll and msvcrt.dll

Mre200l.dll

 
 

Microsoft Visual C++ 5/6

Msvcp60d.dll and msvcrtd.dll

Mre200dl.dll

 
 

Borland C++ 5.02

Cw3230.dll

b2re200l.dll

 
 

Borland C++ 5.02

Cw3230mt.dll

B2re200lm.dll

 
 

Borland C++ Builder 1

Cp3230mt.dll

B2re200lv.dll

 
 

Borland C++ Builder 3

Cw3240.dll

B3re200l.dll

 
 

Borland C++ Builder 3

Cw3240mt.dll

B3re200lm.dll

 
 

Borland C++ Builder 3

Cp3240mt.dll and vcl35.bpl

B3re200lv.dll

 
 

Borland C++ Builder 4

Cw3245.dll

B4re200l.dll

 
 

Borland C++ Builder 4

Cw3245mt.dll

B4re200lm.dll

 
 

Borland C++ Builder 4

Cp3245mt.dll and vcl40.bpl

B4re200lv.dll

 

 


Further Information

The author can be contacted at John_Maddock@compuserve.com, the home page for this library is at http://ourworld.compuserve.com/homepages/John_Maddock/regexpp.htm, you can obtain the latest official release from this site, anything else is a mirror.

I am indebted to Robert Sedgewick's "Algorithms in C++" for forcing me to think about algorithms and their performance. The following people have all contributed useful comments or fixes: Edward Diener, Robert Dunn, Max Leung, Rob Ward and Yuval Yosef. I am also grateful to the manuals supplied with the Henry Spencer, Perl and GNU regular expression libraries - wherever possible I have tried to maintain compatibility with these libraries and with the POSIX standard - the code however is entirely my own, including any bugs! I can absolutely guarantee that I will not fix any bugs I don't know about, so if you have any comments or spot any bugs, please get in touch.

Useful further information can be found at:

The Open Unix Specification contains a wealth of useful material, including the regular expression syntax, and specifications for <regex.h> and <nl_types.h>.

The Pattern Matching Pointers site is a "must visit" resource for anyone interested in pattern matching.

Glimpse and Agrep, use a simplified regular expression syntax to achieve faster search times.

Udi Manber and Ricardo Baeza-Yates both have a selection of useful pattern matching papers available from their respective web sites.

Copyright Dr John Maddock 1998-9 all rights reserved.