Our Moral Obligation to Grant Asylum to Refugees from Central America

Ben Key:

November 28, 2018

Recently, the Trump administration has done everything it could to make it more difficult to seek asylum in The United States of America.

These actions are morally reprehensible.

To begin with, these people are fleeing truly terrible conditions that include violence, organized crime, extremely low incomes, and truly unbearable living conditions. They did not decide to travel over 2,000 miles on a whim. They are fleeing clear and present danger.

These descriptions of what the refugees are fleeing should be enough to convince anyone with an ounce of compassion to welcome these refugees from Central America with open arms and grant them asylum in The United States of America, especially if they claim to be Christian.

When a foreigner resides among you in your land, do not mistreat them. The foreigner residing among you must be treated as your native-born. Love them as yourself, for you were foreigners in Egypt. (Leviticus 19:33-34)

Compassion, empathy, and common decency are not the only reasons why we should be welcoming these refugees instead of turning them away. There are numerous sources that provide overwhelming evidence that these refugees are fleeing conditions that were created by actions directly taken by the government of The United States of America over the past 100 years.

We helped create the issues that caused the refugee crisis. We have a moral obligation to provide those who are fleeing the conditions we created a place of safety where they can live in peace. We have a moral responsibility to make right our wrongs!

Do not be deceived: God cannot be mocked. A man reaps what he sows. (Galatians 6:7)

Advertisements

Using Character or String Constants in a Template function

Using Character or String Constants in a Template function

Ben Key:

October 1, 2013; November 10, 2018

 


This article provides several techniques that may be used when creating template functions in which the character type is a template parameter and you must use character or string constants. It also provides the source code for several template functions and macros that may be used to implement one of the proposed solutions.

Problem Description

There are various data types that may be used to represent characters in C++. The most common of these are char and wchar_t. It often is necessary to write code that is capable of handling either type of character.

One alternative is to simply implement the function once for each character type that needs to be supported. There are obvious problems with this approach. The first problem is that this approach causes unnecessary code duplication. It also opens up the possibility that the different implementations of the function will diverge over time as changes are made in one implementation, for example in order to fix bugs, but not in the other implementations of the function.

Consider the following example.

The CommandLineToArgVector function parses a command line string such as that which might be returned by the GetCommandLine function. This function depends on a number of character constants: specifically NULCHAR (’\0’), SPACECHAR (’ ‘), TABCHAR (’), DQUOTECHAR (’”‘), and SLASHCHAR (’\’).

In order to support characters of both char and wchar_t it is necessary to implement the function twice as follows.


inline size_t CommandLineToArgVector(
    const char* commandLine,
    std::vector<std::string>& arg_vector)
{
    arg_vector.clear();
    /* Code omitted. */
    return static_cast<size_t>(arg_vector.size());
}

inline size_t CommandLineToArgVector(
    const wchar_t* commandLine,
    std::vector<std::wstring>& arg_vector)
{
    arg_vector.clear();
    /* Code omitted. */
    return static_cast<size_t>(arg_vector.size());
}

The obvious solution is to implement this as a template function that accepts the character type as a template parameter as follows.


template<typename CharType>
size_t CommandLineToArgVector(
    const CharType* commandLine,
    std::vector< std::basic_string<CharType> >& arg_vector)
{
    arg_vector.clear();
    /* Code omitted. */
    return static_cast<size_t>(arg_vector.size());
}

The only thing that prevents us from doing that is the existence of the character constants. How do you represent the constants so that they are can be of either data type?

Possible Solutions

As is often the case, there are many possible solutions to this problem. This article discusses several of them.

Widen

One possible solution is to make use of std::ctype::widen as follows.


template <typename CharType>
CharType widenChar(
    const char ch, const std::locale& loc = std::locale())
{
    const auto& cType = std::use_facet<std::ctype<CharType>>(loc);
    return cType.widen(ch);
}

This same technique can be extended for use with string literals as follows.


template<typename CharType>
std::basic_string<CharType> widenString(
    const char* str, const std::locale& loc = std::locale())
{
    std::basic_string<CharType> ret;
    if (str == nullptr || str[0] == 0)
    {
        return ret;
    }
    const auto& cType = std::use_facet<std::ctype<CharType>>(loc);
    auto srcLen = std::strlen(str);
    auto bufferSize = srcLen + 32;
    auto tmpPtr = yekneb::make_unique<CharType[]>(bufferSize);
    auto tmp = tmpPtr.get();
    cType.widen(str, str + srcLen, tmp);
    ret = tmp;
    return ret;
}

Then in the template function you can use the character constants in a character type neutral way as follows.


const CharType NULCHAR = widenChar<CharType>('\0');
const CharType SPACECHAR = widenChar<CharType>(' ');
const CharType TABCHAR =  widenChar<CharType>('\t');
const CharType DQUOTECHAR = widenChar<CharType>('\"');
const CharType SLASHCHAR =  widenChar<CharType>('\\');

The problem is that this requires a multibyte to wide character conversion for the wchar_t case. This may not be much of an issue for functions that are not used very often and are not performance critical. However the cost of the multibyte to wide character conversion would not be acceptable in performance critical situations.

The question is how to solve this problem without these performance issues.

Algorithm Traits

Another option is to create an algorithm traits class that includes functions that return each of the constants as follows.


template <typename CharType>
struct CommandLineToArgVector_Traits
{
    CharType NULCHAR();
    CharType SPACECHAR();
    CharType TABCHAR();
    CharType DQUOTECHAR();
    CharType SLASHCHAR();
    CharType* STRING();
};

template<>
struct CommandLineToArgVector_Traits<char>
{
    char NULCHAR()
    {
        return '\0';
    }
    char SPACECHAR()
    {
        return ' ';
    }
    char TABCHAR()
    {
        return '\t';
    }
    char DQUOTECHAR()
    {
        return '\"';
    }
    char SLASHCHAR()
    {
        return '\\';
    }
    char* STRING()
    {
        return "String";
    }
};

template<>
struct CommandLineToArgVector_Traits<wchar_t>
{
    wchar_t NULCHAR()
    {
        return L'\0';
    }
    wchar_t SPACECHAR()
    {
        return L' ';
    }
    wchar_t TABCHAR()
    {
        return L'\t';
    }
    wchar_t DQUOTECHAR()
    {
        return L'\"';
    }
    wchar_t SLASHCHAR()
    {
        return L'\\';
    }
    wchar_t* STRING()
    {
        return L"String";
    }
};

As you can see, it is a simple matter to incorporate string constants in the algorithm traits structure.

The algorithm traits structure can then be used as follows in your template function.


CommandLineToArgVector_Traits<CharType> traits;
const CharType NULCHAR = traits.NULCHAR();
const CharType SPACECHAR = traits.SPACECHAR();
const CharType TABCHAR =  traits.TABCHAR();
const CharType DQUOTECHAR = traits.DQUOTECHAR();
const CharType SLASHCHAR =  traits.SLASHCHAR();
const CharType* STRING = traits.STRING();

This solution does not have the same performance issues that the widenChar solution does but these performance gains come at the cost of a significant increase in the complexity of the code. In addition, there is now a need to maintain two implementations of the Algorithm Traits structure, which means that we are right back where we started from, though admittedly maintaining two implementations of the Algorithm Traits structure is a lot less work than maintaining two implementations of the algorithm.

It would be ideal to develop a solution that has the convenience and simplicity of the widenChar solution without the performance issues. The question is how.

Preprocessor and Template Magic

Fortunately it is possible to solve this problem using some preprocessor and template magic. I found this solution in the Stack Overflow article How to express a string literal within a template parameterized by the type of the characters used to represent the literal.

The solution is as follows.


template<typename CharType>
CharType CharConstantOfType(const char c, const wchar_t w);

template<>
char CharConstantOfType<char>(const char c, const wchar_t /*w*/)
{
    return c;
}
template<>
wchar_t CharConstantOfType<wchar_t>(const char /*c*/, const wchar_t w)
{
    return w;
}

template<typename CharType>
const CharType* StringConstantOfType(const char* c, const wchar_t* w);

template<>
const char* StringConstantOfType<char>(const char* c, const wchar_t* /*w*/)
{
    return c;
}
template<>
const wchar_t* StringConstantOfType<wchar_t>(const char* /*c*/, const wchar_t* w)
{
    return w;
}

#define _TOWSTRING(x) L##x
#define TOWSTRING(x) _TOWSTRING(x)
#define CHAR_CONSTANT(TYPE, STRING) CharConstantOfType<TYPE>(STRING, TOWSTRING(STRING))
#define STRING_CONSTANT(TYPE, STRING) StringConstantOfType<TYPE>(STRING, TOWSTRING(STRING))

Then in the template function you can use the character constants in a character type neutral way as follows.


const CharType NULCHAR = CHAR_CONSTANT(CharType, '\0');
const CharType SPACECHAR = CHAR_CONSTANT(CharType, ' ');
const CharType TABCHAR =  CHAR_CONSTANT(CharType, '\t');
const CharType DQUOTECHAR = CHAR_CONSTANT(CharType, '\"');
const CharType SLASHCHAR =  CHAR_CONSTANT(CharType, '\\');
const CharType* STRING = STRING_CONSTANT(CharType, "String");

Abracadabra. The problem is solved without any code duplication or performance issues.

Article Source Code

The source code for this article can be found on SullivanAndKey.com. The relevant files may be found at the following locations.

  • widen.h: Contains the source of widenChar and widenString.
  • ConstantOfType.h: Contains the source of CharConstantOfType and StringConstantOfType.
  • CmdLineToArgv.h: The header file for the CommandLineToArgVector function.
  • CmdLineToArgv.cpp: The source code for the CommandLineToArgVector function.

 

Cross Platform Conversion Between string and wstring

Cross Platform Conversion Between string and wstring

Ben Key

October 31, 2013; November 09, 2018

 


This article describes a cross platform method of converting between a STL string and a STL wstring. The technique described in this article does not make use of any external libraries. Nor does it make use of any operating system specific APIs. It uses only features that are part of the standard template library.

Problem Description

The Standard Template Library provides the basic_string template class to represents a sequence of characters. It supports all the usual operations of a sequence and standard string operations such as search and concatenation.

There are two common specializations of the basic_string template class. They are string, which is a typedef for basic_string<char>, and wstring, which is a typedef for basic_string<wchar_t>. In addition there are two specializations of the basic_string template class that are new to C++11. The new specializations of the basic_string template class are u16string, which is a typedef for basic_string<char16_t>, and u32string, which is a typedef for basic_string<char32_t>.

A question that is often asked on the Internet is how to convert between string and wstring. This may be necessary if calling an API function that is designed for one type of string from code that uses a different type of string. For example your code might use wstring objects and you may need to call an API function that uses string objects. In this case you will need to convert between a wstring to a string before calling the API function.

Unfortunately the Standard Template Library does not provide a simple means of doing this conversion. As a result, this is a commonly asked question on the Internet. A Google search for convert between string and wstring returns about 76,300 results. Unfortunately many of the answers that are provided are incorrect.

The Incorrect Answer

The most common incorrect answer found on the Internet can be found in the article Convert std::string to std::wstring on Mijalko.com. The solution provided in that article is as follows.


// std::string -> std::wstring
std::string s("string");
std::wstring ws;
ws.assign(s.begin(), s.end());

// std::wstring -> std::string
std::wstring ws(L"wstring");
std::string s;
s.assign(ws.begin(), ws.end());

The problem is that this solution compiles and runs and for the simple string constants used in this example code, appears to yield the correct results. Many sub standard computer programmers will assume that since it works for these very simple strings they have found the correct solution. Then they translate their application to other languages besides English and they are extremely surprised to discover that their application is riddled with bugs that only affect their non English speaking customers!

The problem is that this solution only works correctly for ASCII characters in the range of 0 to 127. If your strings contain even a single character with a numerical value greater than 127, this simple solution will yield incorrect results. In other words, this simple solution will yield incorrect results if your strings contain any characters other than a through z, A through Z, the numerals 0 through 9, and a few punctuation marks.

This means that any application that uses this technique will not support Chinese. It will not even properly support Spanish since the Spanish alphabet contains several characters that are outside of the ASCII character set such as ch (ce hache), ll (elle), and ñ (eñe).

The Correct Solution

This article describes a cross platform solution for this problem that has the following characteristics.

The Functions


namespace yekneb
{

template<typename Target, typename Source>
inline Target string_cast(const Source& source)
{
    return source;
}

template<>
inline std::wstring string_cast(const std::string& source)
{
    std::locale loc;
    return ::yekneb::detail::string_cast::s2w(source, loc);
}

template<>
inline std::string string_cast(const std::wstring& source)
{
    std::locale loc;
    return ::yekneb::detail::string_cast::w2s(source, loc);
}

}

Three additional functions are provided that allow the user to specify the locale parameter.

The functions are used as follows.


std::wstring wIn(L"Hello World! It is truly a wonderful day to be alive.");
std::string sIn("Hello World! It is truly a wonderful day to be alive.");
std::string sOut = yekneb::string_cast<std::string>(wIn);
std::wstring wOut = yekneb::string_cast<std::wstring>(sIn);

The details

The functions s2w and w2s are implemented using the following features of the STL.

The implementation of these functions is as follows.


namespace yekneb
{

namespace detail
{
namespace string_cast
{

inline std::string w2s(const std::wstring& ws, const std::locale& loc)
{
    typedef std::codecvt<wchar_t, char, std::mbstate_t> converter_type;
    typedef std::ctype<wchar_t> wchar_facet;
    std::string return_value;
    if (ws.empty())
    {
        return "";
    }
    const wchar_t* from = ws.c_str();
    size_t len = ws.length();
    size_t converterMaxLength = 6;
    size_t vectorSize = ((len + 6) * converterMaxLength);
    if (std::has_facet<converter_type>(loc))
    {
        const converter_type& converter = std::use_facet<converter_type>(loc);
        if (converter.always_noconv())
        {
            converterMaxLength = converter.max_length();
            if (converterMaxLength != 6)
            {
                vectorSize = ((len + 6) * converterMaxLength);
            }
            std::mbstate_t state;
            const wchar_t* from_next = nullptr;
            std::vector<char> to(vectorSize, 0);
            std::vector<char>::pointer toPtr = to.data();
            std::vector<char>::pointer to_next = nullptr;
            const converter_type::result result = converter.out(
                state, from, from + len, from_next,
                toPtr, toPtr + vectorSize, to_next);
            if (
              (converter_type::ok == result || converter_type::noconv == result)
              && 0 != toPtr[0]
              )
            {
              return_value.assign(toPtr, to_next);
            }
        }
    }
    if (return_value.empty() && std::has_facet<wchar_facet>(loc))
    {
        std::vector<char> to(vectorSize, 0);
        std::vector<char>::pointer toPtr = to.data();
        const wchar_facet& facet = std::use_facet<wchar_facet>(loc);
        if (facet.narrow(from, from + len, '?', toPtr) != 0)
        {
            return_value = toPtr;
        }
    }
    return return_value;
}

inline std::wstring s2w(const std::string& s, const std::locale& loc)
{
    typedef std::ctype<wchar_t> wchar_facet;
    std::wstring return_value;
    if (s.empty())
    {
        return L"";
    }
    if (std::has_facet<wchar_facet>(loc))
    {
        std::vector<wchar_t> to(s.size() + 2, 0);
        std::vector<wchar_t>::pointer toPtr = to.data();
        const wchar_facet& facet = std::use_facet<wchar_facet>(loc);
        if (facet.widen(s.c_str(), s.c_str() + s.size(), toPtr) != 0)
        {
            return_value = to.data();
        }
    }
    return return_value;
}

}
}

}

A GNU/Linux Specific Bug

The above functions function well on Microsoft Windows and MAC OS X. However they do not work as well on GNU/Linux. Specifically the w2s function will fail with one of two errors, depending on whether debugging is enabled.

If debugging is enabled the error will be a double free or corruption error that occurs when the output buffer is being deallocated. The error occurs do to the fact that the call to converter.out will cause the output buffer to be corrupted and far more bytes will be written to the output buffer than the number of bytes that were allocated for the buffer, even if you over allocate the output buffer by a power of 2.

If debugging is not enabled the error will be as follows.

../iconv/loop.c:448: internal_utf8_loop_single: Assertion `inptr - bytebuf > (state->__count & 7)' failed.

These bugs bug only occur when the active locale uses UTF-8.

I have been unable to resolve this issue using just the STL. However, the Boost C++ Libraries found at www.boost.org provide an acceptable solution, the boost::locale::conv::utf_to_utf function. If using Boost is an option for you this problem can be resolved as follows.

First, add the IsUTF8 function to the ::yekneb::detail::string_cast namespace. This function may be used to determine whether or not a given locale uses UTF-8 and thus whether or not the w2s function should use the boost::locale::conv::utf_to_utf function.

The source code for the IsUTF8 function is as follows.


inline bool IsUTF8(const std::locale &loc)
{
    std::string locName = loc.name();
    if (! locName.empty() && std::string::npos != locName.find("UTF-8"))
    {
        return true;
    }
    return false;
}

Then simply add the following code to the w2s function just after the if (ws.empty()) code block.


if (IsUTF8(loc))
{
    return_value = boost::locale::conv::utf_to_utf<char>(ws);
    if (! return_value.empty())
    {
        return return_value;
    }
}

For consistency a similar code block should be added to the s2w function as well.

Article Source Code

The complete source code for this article can be found on SullivanAndKey.com in the header file StringCast.h. You can also see the code in action on ideone.com.


 

Variable Expansion in Strings

Variable Expansion in Strings

Ben Key:

December 6, 2013; November 7, 2018

 


A common task in C and C++ is to build a string out of a template string containing variable placeholders, often called format specifiers, and additional data. This article describes several options that are available for solving this problem and introduces two versions of an ExpandVars function as an alternative to those solutions.

Problem Description

The simplest description of the problem can be best described via an expansion of the classic Hello World program that is so often the first program one learns to write when learning a new programming language.

The simplest implementation of Hello World in C++ is as follows.


#include <iostream>
 
int main()
{
    std::cout << "Hello, world!\n";
    return 0;
}

Source: Variable Expansion in Strings – Example 1


What if you wanted to modify this program to first ask you for your name and then display a more personal greeting? One way to do this is as follows.


#include <iostream>
#include <string>
 
int main()
{
    std::cout << "What is your name?\n";
    std::string name;
    std::getline(std::cin, name);
    std::cout << "Hello, " << name << "!\n";
    return 0;
}

Source: Variable Expansion in Strings – Example 2


The problem with this approach is that it is not very extensible. It can also be very unwieldy when you have multiple variables that you need to print out.

For example, imagine you have the following person structure and you want to display a message containing all of the fields in the person structure.


struct person
{
    std::string firstName;
    std::string middleName;
    std::string lastName;
    std::string streetAddress1;
    std::string streetAddress2;
    std::string city;
    std::string state;
    std::string zip;
};

Source: Variable Expansion in Strings – Example 3


One possible explanation of a function to display a message containing all of the fields in the person structure is as follows.


void PrintPersonWithStream(const person& p)
{
    std::cout
        << "First Name: " << p.firstName << "\n"
        << "Middle Name: " << p.middleName << "\n"
        << "Last Name: " << p.lastName << "\n"
        << "Street Address 1: " << p.streetAddress1 << "\n"
        << "Street Address 2: " << p.streetAddress2 << "\n"
        << "City: " << p.city << "\n"
        << "State: " << p.state << "\n"
        << "Zip: " << p.zip << "\n";
}

Source: Variable Expansion in Strings – Example 3


As you can see, this function is rather unwieldy. It would be far simpler to be able to write something like the following.


void PrintPerson(const person& p)
{
    const char FormatString[] = R"(
First Name: {FormatSpecifier}
Middle Name: {FormatSpecifier}
Last Name: {FormatSpecifier}
Street Address 1: {FormatSpecifier}
Street Address 2: {FormatSpecifier}
City: {FormatSpecifier}
State: {FormatSpecifier}
Zip: {FormatSpecifier}
)";
    std::string message = SomeFormatFunction(
        FormatString,
        p.firstName, p.middleName, p.lastName,
        p.streetAddress1, p.streetAddress2,
        p.city, p.state, p.zip);
    std::cout << message << "\n";
}

In the above example {FormatSpecifier} will be replaced with a bit of text that causes the text of the appropriate variable to be inserted at the appropriate place in the final string. It will vary depending on the solution you use.

The benefit of this type of solution is that it is far less verbose and it is far less work to change the order of variables in the final output and to add a variable.

You might ask why being able to change the order of variable counts. A simple answer is if your program supports several languages and you need to change the order of items such as dates to account for standards used by a given language.

Using the Standard C Library Functions printf or sprintf

One option is to simply make use of the Standard C library functions printf, or if you need to store the output in a string, sprintf. The printf function writes formatted data to stdout. The sprintf function writes formatted data to a string. These functions can be used as follows.

First, add the following function to the person structure.


static std::string GetPrintfFormatString()
{
    static const char FormatString[] = R"(
First Name: %s
Middle Name: %s
Last Name: %s
Street Address 1: %s
Street Address 2: %s
City: %s
State: %s
Zip: %s
)";
    return FormatString;
}

Source: Variable Expansion in Strings – Example 4


Then the functions can be defined as follows.


void PrintPersonWithPrintf(const person& p)
{
    ::printf(
        p.GetPrintfFormatString().c_str(),
        p.firstName.c_str(), p.middleName.c_str(), p.lastName.c_str(),
        p.streetAddress1.c_str(), p.streetAddress2.c_str(),
        p.city.c_str(), p.state.c_str(), p.zip.c_str());
}
 
void PrintPersonWithSPrintf(const person& p)
{
    std::string formatString = p.GetPrintfFormatString();
    size_t outputLen = formatString.length() + p.firstName.length()
        + p.middleName.length() + p.lastName.length()
        + p.streetAddress1.length() + p.streetAddress2.length()
        + p.city.length() + p.state.length() + p.zip.length()
        + 20;
    std::vector<char> buffer(outputLen, 0);
    ::sprintf(
        buffer.data(), formatString.c_str(),
        p.firstName.c_str(), p.middleName.c_str(), p.lastName.c_str(),
        p.streetAddress1.c_str(), p.streetAddress2.c_str(),
        p.city.c_str(), p.state.c_str(), p.zip.c_str());
    std::string out = buffer.data();
    std::cout << out;
}

Source: Variable Expansion in Strings – Example 4


Limitations of sprintf

One limitation of using the sprintf function is that it is not very flexible for international applications. Often the order of words differ from one language to another. One often discussed example is a time and date string.

For example, in the United States date strings are written as {Month}/{Day}/{Year} while in France date strings are written as {Day}/{Month}/{Year} and in Japan date strings are written as {Year}/{Day}/{Month}. There are many other instance in which word order varies from language to language. For more information refer to the Word order article on Wikipedia, [The origin and evolution of word order][], and The Typology of the Word Order of Languages.

One problem with the sprintf function is that it is not possible to change the order of words in the final output by simply changing the order of words in the format string. That is due to the fact that the order of parameters in the code would need to be changed as well.

One solution to this problem is the use of positional specifiers for format strings.

Positional Specifiers for Format Strings

POSIX compatible systems implement an extension to the printf family of functions to add support for positional specifiers for format strings. This extension allows the conversion specifier character % to be is replaced by the sequence “%n$”, where n is a decimal integer in the range [1, {NL_ARGMAX}], giving the position of the argument in the argument list. For more information see the following resources.

The problem for this solution is that this is not universally supported. For example, on Microsoft Windows, the printf family of functions does not support positional specifiers for format strings. Instead this functionality is supported in the printf_p family of functions: see printf_p Positional Parameters and _sprintf_p, _sprintf_p_l, _swprintf_p, _swprintf_p_l. This makes writing cross platform code unnecessarily difficult.

The following code demonstrates the use of positional specifiers for format strings to write a function that will properly format a date string for the United States, France, and Japan.


std::string GetDateFormatString(const std::string& langCode)
{
    if (
       0 == langCode.compare(0, 2, "en")
       || 0 == langCode.compare(0, 2, "EN")
       )
    {
        return std::string("%1$i/%2$i/%3$i");
    }
    else if (
       0 == langCode.compare(0, 2, "fr")
       || 0 == langCode.compare(0, 2, "FR")
       )
    {
        return std::string("%2$i/%1$i/%3$i");
    }
    else if (
       0 == langCode.compare(0, 2, "ja")
       || 0 == langCode.compare(0, 2, "JA")
       || 0 == langCode.compare(0, 2, "jp")
       || 0 == langCode.compare(0, 2, "JP")
       )
    {
        return std::string("%3$i/%2$i/%1$i");
    }
    return std::string("%1$i/%2$i/%3$i");
}
 
std::string GetDateString(
    const std::string& langCode,
    int month, int day, int year)
{
    std::string fmt = GetDateFormatString(langCode);
    std::array<char, 32> buffer;
    buffer.fill(0);
#if defined(_WIN32)
    ::_sprintf_p(buffer.data(), 32, fmt.c_str(), month, day, year);
#else
    ::sprintf(buffer.data(), fmt.c_str(), month, day, year);
#endif
    std::string ret = buffer.data();
    return ret;
}

Source: Variable Expansion in Strings – Example 5


Note that the there is one major drawback of the above GetDateString function, the presence of that nasty #if/#else/#endif block. This is far from ideal. Unfortunately, due to the fact that the _sprintf_p function expects an additional sizeOfBuffer parameter. Therefore you cannot simply do the following.


#if !defined(_WIN32)
#  define _sprintf_p sprintf
#endif

std::string GetDateString(
    const std::string &langCode,
    int month, int day, int year)
{
    std::string fmt = GetDateFormatString(langCode);
    std::array<char, 32> buffer;
    buffer.fill(0);
    // This will not compile on non Windows systems due to the extra parameter.
    _sprintf_p(buffer.data(), 32, fmt.c_str(), month, day, year);
    std::string ret = buffer.data();
    return ret;
}

The following will work as an acceptable alternative, however.


#if defined(_WIN32)
#  define sprintfp _sprintf_p
#else
#  define sprintfp snprintf
#endif

std::string GetDateString(
    const std::string &langCode,
    int month, int day, int year)
{
    std::string fmt = GetDateFormatString(langCode);
    std::array<char, 32> buffer;
    buffer.fill(0);
    sprintfp(buffer.data(), 32, fmt.c_str(), month, day, year);
    std::string ret = buffer.data();
    return ret;
}

Source: Variable Expansion in Strings – Example 6


This leaves one problem that all of the solutions I have discussed so far unsolved. This function uses C style strings. That is, the first parameter of _sprintf_p is expected to be a pre-allocated char array. It does not natively make use of the C++ basic_string class.

The Boost Format library

The Boost C++ Libraries are a collection of free peer-reviewed portable C++ source libraries that work well with the C++ Standard Library and enhance the capabilities of the C++ Standard Library. In fact, some of the features of the C++ Standard Library were first implemented in the Boost C++ Libraries and the Boost C++ Libraries are designed so that they are suitable for eventual standardization.

One of the components of Boost is The Boost Format library. The Boost home page describes The Boost Format library as follows.

The format library provides a class for formatting arguments according to a format-string, as does printf, but with two major differences:

  • format sends the arguments to an internal stream, and so is entirely type-safe and naturally supports all user-defined types.
  • The ellipsis (…) can not be used correctly in the strongly typed context of format, and thus the function call with arbitrary arguments is replaced by successive calls to an argument feeding operator%

The format specification strings used by the Boost Format library use the Unix98 Open-group printf precise syntax. Further information on the format specification strings used by the Boost Format library can be found in the Boost printf format specifications section of the Boost Format library documentation. Note that these are essentially the same format specification strings that are used by the _sprintf_p function. As a result, the GetDateFormatString function can be used with The Boost Format library.

The following function shows how this can be done.


std::string GetDateStringBoost(
    const std::string &langCode, int month, int day, int year)
{
    std::string fmt = GetDateFormatString(langCode);
    std::string ret = boost::str(boost::format(fmt) % month % day % year);
    return ret;
}

Source: Variable Expansion in Strings – Example 7


Self Documenting Format Specification Strings

One problem with the printf style format specification strings is that they require some form of supporting documentation to indicate which part of the format specification string corresponds to which variable. For example, in order for the GetDateFormatString function to be considered complete, a comment should be added to specify that the %1$ component corresponds to the month, the %2$ component corresponds to the day, and the %3$ component corresponds to the year.

It would be idea if this documentation was an inherent part of the format specification string. Consider the following syntax for a format string: “\((month)/\)(day)/$(year).” In this string their is no need for supporting documentation to indicate the meaning of each component of the format specification string.

This technique is commonly referred to as String interpolation or variable interpolation, variable substitution, or variable expansion. Some programming languages have this functionality built in.

For example, the Python programming language supports the Literal String Interpolation feature since Python 3.6. This makes the following possible.


apples = 4
print(f"I have {apples} apples")

Another example is in the C# programming language. C# 6 added the interpolated string feature.


string name = "Mark";
var date = DateTime.Now;
Console.WriteLine($"Hello, {name}! Today is {date.DayOfWeek}, it's {date:HH:mm} now.");

The ExpandVars Function

The YekNeb C++ Code snippets library provides two versions of the ExpandVars function, which provides string interpolation functionality for C++. One version of the function uses nothing beyond the STL. Another version of the function uses the Boost Xpressive library. Both versions of the function return a string in which the variables are expanded based on the values specified in either an environment map or the environment variables. The following formats are supported for variable names.

  • %VarName%
  • %(VarName)
  • %[VarName]
  • %{VarName}
  • $(VarName)
  • $[VarName]
  • ${VarName}
  • #(VarName)
  • #[VarName]
  • #{VarName}

Bash style variable names in the form of $VarName are not supported.

The variable names used by the ExpandVars function may contain word characters, space characters, the ( character, and the ) character. Note that if the variable includes either the ( character or the ) character you should not use the %(VarName) or $(VarName) syntax.

The following is a simplified version of the STL only ExpandVars function.


bool FindVariableString(
    const std::string &str,
    const std::string::size_type pos,
    std::string::size_type &beginVarStringPos,
    std::string::size_type &endVarStringPos,
    std::string::size_type &beginVarNamePos,
    std::string::size_type &endVarNamePos)
{
    const char *TestString = "%$#";
    const char PercentSign = '%';
    const char LeftParenthesis = '(';
    const char LeftSquareBracket = '[';
    const char LeftCurlyBracket = '{';
    const char RightParenthesis = ')';
    const char RightSquareBracket = ']';
    const char RightCurlyBracket = '}';
    beginVarStringPos = std::string::npos;
    endVarStringPos = std::string::npos;
    beginVarNamePos = std::string::npos;
    endVarNamePos = std::string::npos;
    if (str.empty())
    {
        return false;
    }
    beginVarStringPos = str.find_first_of(TestString, pos);
    if (std::string::npos == beginVarStringPos)
    {
        return false;
    }
    if (beginVarStringPos >= str.length() - 1)
    {
        return false;
    }
    char ch = str[beginVarStringPos];
    char ch1 = str[beginVarStringPos + 1];
    if (
       PercentSign == ch
       && LeftParenthesis != ch1 && LeftSquareBracket != ch1
       && LeftCurlyBracket != ch1
       )
    {
        beginVarNamePos = beginVarStringPos + 1;
        endVarStringPos = str.find(PercentSign, beginVarNamePos);
        if (std::string::npos == endVarStringPos)
        {
            return false;
        }
    }
    else if (
       LeftParenthesis != ch1 && LeftSquareBracket != ch1
       && LeftCurlyBracket != ch1
       )
    {
        return false;
    }
    else
    {
        beginVarNamePos = beginVarStringPos + 2;
        char closeChar = 0;
        if (LeftParenthesis == ch1)
        {
            closeChar = RightParenthesis;
        }
        else if (LeftSquareBracket == ch1)
        {
            closeChar = RightSquareBracket;
        }
        else if (LeftCurlyBracket == ch1)
        {
            closeChar = RightCurlyBracket;
        }
        endVarStringPos = str.find(closeChar, beginVarNamePos);
        if (std::string::npos == endVarStringPos)
        {
            return false;
        }
    }
    endVarNamePos = endVarStringPos - 1;
    return true;
}
 
bool StringContainsVariableStrings(const std::string &str)
{
    std::string::size_type beginVarStringPos = 0;
    std::string::size_type endVarStringPos = 0;
    std::string::size_type beginVarNamePos = 0;
    std::string::size_type endVarNamePos = 0;
    bool ret = FindVariableString(str, 0, beginVarStringPos, endVarStringPos, beginVarNamePos, endVarNamePos);
    return ret;
}
 
std::string GetVariableValue(
    const std::string &varName,
    const std::map<std::string, std::string> &env,
    bool &fromEnvMap, bool &valueContainsVariableStrings)
{
    typedef std::map<std::string, std::string> my_map;
    fromEnvMap = false;
    valueContainsVariableStrings = false;
    std::string ret;
    my_map::const_iterator itFind = env.find(varName);
    if (itFind != env.end())
    {
        ret = (*itFind).second;
        if (!ret.empty())
        {
            fromEnvMap = true;
            valueContainsVariableStrings = StringContainsVariableStrings(ret);
        }
    }
    if (ret.empty())
    {
        ret = ::getenv(varName.c_str());
    }
    return ret;
}
 
std::string ExpandVars(
    const std::string &original,
    const std::map<std::string, std::string> &env)
{
    std::string ret = original;
    if (original.empty())
    {
        return ret;
    }
    bool foundVar = false;
    std::string::size_type pos = 0;
    do
    {
        std::string::size_type beginVarStringPos = 0;
        std::string::size_type endVarStringPos = 0;
        std::string::size_type beginVarNamePos = 0;
        std::string::size_type endVarNamePos = 0;
        foundVar = FindVariableString(ret, pos, beginVarStringPos, endVarStringPos, beginVarNamePos, endVarNamePos);
        if (foundVar)
        {
            std::string::size_type varStringLen = endVarStringPos - beginVarStringPos + 1;
            std::string varString = ret.substr(beginVarStringPos, varStringLen);
            std::string::size_type varNameLen = endVarNamePos - beginVarNamePos + 1;
            std::string varName = ret.substr(beginVarNamePos, varNameLen);
            bool fromEnvMap;
            bool valueContainsVariableStrings;
            std::string value = GetVariableValue(varName, env, fromEnvMap, valueContainsVariableStrings);
            if (!value.empty())
            {
                ret = ret.replace(beginVarStringPos, varStringLen, value);
                pos = beginVarStringPos;
            }
            else
            {
                pos = endVarStringPos + 1;
            }
        }
    } while (foundVar);
    return ret;
}

Source: Variable Expansion in Strings – Example 8

The following code demonstrates the use of the ExpandVars function.

std::string GetDateFormatStringExpandVars(const std::string& langCode)
{
    if (
       0 == langCode.compare(0, 2, "en")
       || 0 == langCode.compare(0, 2, "EN")
       )
    {
        return std::string("${month}/${day}/${year}");
    }
    else if (
       0 == langCode.compare(0, 2, "fr")
       || 0 == langCode.compare(0, 2, "FR")
       )
    {
        return std::string("${day}/${month}/${year}");
    }
    else if (
       0 == langCode.compare(0, 2, "ja")
       || 0 == langCode.compare(0, 2, "JA")
       || 0 == langCode.compare(0, 2, "jp")
       || 0 == langCode.compare(0, 2, "JP")
       )
    {
        return std::string("${year}/${day}/${month}");
    }
    return std::string("${month}/${day}/${year}");
}
 
std::string GetDateStringExpandVars(
    const std::string &langCode, int month, int day, int year)
{
    std::string fmt = GetDateFormatStringExpandVars(langCode);
    std::map<std::string,std::string> env{
        {"month", std::to_string(month)},
        {"day", std::to_string(day)},
        {"year", std::to_string(year)}
    };
    std::string ret = ExpandVars(fmt, env);
    return ret;
}

Source: Variable Expansion in Strings – Example 8


The following is a simplified version of a version of the ExpandVars function that uses the Boost Xpressive regex_replace function.


::boost::xpressive::sregex GetRegex()
{
    namespace xpr = ::boost::xpressive;
    xpr::sregex ret =
        "%" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> '%'
        | "%(" >> (xpr::s1 = +(xpr::_w | xpr::_s)) >> ')'
        | "%[" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> ']'
        | "%{" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> '}'
        | "$(" >> (xpr::s1 = +(xpr::_w | xpr::_s)) >> ')'
        | "$[" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> ']'
        | "${" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> '}'
        | "#(" >> (xpr::s1 = +(xpr::_w | xpr::_s)) >> ')'
        | "#[" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> ']'
        | "#{" >> (xpr::s1 = +(xpr::_w | xpr::_s | "(" | ")")) >> '}';
    return ret;
}

struct string_formatter
{
    typedef std::map<std::string, std::string> env_map;
    env_map env;
    mutable bool valueContainsVariables;
    string_formatter()
    {
        valueContainsVariables = false;
    }
    template<typename Out>
    Out operator()(::boost::xpressive::smatch const& what, Out out) const
    {
        bool fromEnvMap;
        bool valueContainsVariableStrings;
        std::string value = GetVariableValue(
            what.str(1), env, fromEnvMap, valueContainsVariableStrings);
        if (fromEnvMap && !value.empty() && valueContainsVariableStrings)
        {
            valueContainsVariables = true;
        }
        if (value.empty())
        {
            value = what[0];
        }
        if (!value.empty())
        {
            out = std::copy(value.begin(), value.end(), out);
        }
        return out;
    }
};

std::string ExpandVarsR(
    const std::string &original,
    const std::map<std::string, std::string> &env)
{
    std::string ret = original;
    if (original.empty())
    {
        return ret;
    }
    string_formatter fmt;
    fmt.env = env;
    fmt.valueContainsVariables = false;
    ::boost::xpressive::sregex envar = GetRegex();
    ret = ::boost::xpressive::regex_replace(original, envar, fmt);
    if (fmt.valueContainsVariables)
    {
        std::string newValue;
        std::string prevValue = ret;
        do
        {
            fmt.valueContainsVariables = false;
            newValue = ::boost::xpressive::regex_replace(prevValue, envar, fmt);
            if (0 == prevValue.compare(newValue))
            {
                break;
            }
            prevValue.erase();
            prevValue = newValue;
        }
        while (fmt.valueContainsVariables);
        if (0 != ret.compare(newValue))
        {
            ret = newValue;
        }
    }
    return ret;
}

Source: Variable Expansion in Strings – Example 9


The source code of the full version of the STL only implementation of the ExpandVars function can be found in ExpandVars.h and ExpandVars.cpp. The source code of the Boost implementation of the ExpandVars function can be found in boost/ExpandVars.h and boost/ExpandVars.cpp.