I want to catch an exception, and rethrow with a more specific error message:

Dictionary<string, string> m_dictionary;

try
{
     var value = m_dictionary["key"];
     ...
} 
catch ( KeyNotFoundException e )
{
     throw new KeyNotFoundException( "Error message.", e );
}

But Unity will display the original exception’s error message in the console. It gives this:

KeyNotFoundException: The given key was not present in the dictionary.

Instead of this:

KeyNotFoundException: Error message.

Is there any way to get this behavior without throwing away the call stack information of the original exception?

As Bonfire said the innermost exception is the most important and usually the most detailed exception. If an exception is catched by a more general system somewhere up the callstack, it usually don’t know what exactly happend. They usually just add a more general exception as outer exception and keep the inner exception.

If you actually handle the exception properly and you want to provide a more detailed exception, you might want to remove the inner exception and just throw your customized exception.

However This approach is actually bad design, especially for realtime applications. A try-catch block adds quite a bit of overhead. Exceptions should never be used as a way of flowcontrol. If an exception can be avoided you should go that route.

In the case of a dictionary the method which does this is TryGetValue

string value;
if (m_dictionary.TryGetValue("key", out value))
{
    // key was found, value has been filled.
}
else
{
    // key wasn't found, take further actions.
    // Either fill "value" manually if possible or throw an exception if there's no way around.
}

Finally some information on when to use exceptions and when not.