Vector3 question: How to know right axis when using pinch punch gestures to scale object in virtual reality?

I have an object that can be scaled in virtual reality using Vive touch controllers. The issue: the pinch punch gestures for scaling relate to the world, but the object to scale is already rotated. How do I know along which axis the user is intending to scale?

Here’s a sample video of how it should work, with the sample code to achieve this below: - YouTube

As it shows, only one axis (the biggest stretch) is meant to be scaled a time. However, in this video example the cube was carefully rotated for the code to work. Rotated anyway else, and the wrong scaling axis will be applied. How would I go about this? And to clarify, I would like to only change the localScale, not permanently apply the scaling to a parent transform. Thanks for any help!

// had been trying eulerAngles as well as localEulerAngles
Vector3 originalAngles = (Vector3)transformToHandle.eulerAngles;

// trying to reset object's rotation
transformToHandle.Rotate(-originalAngles);

/*
// (an alternative was to rotate the pinch punch points along their center)
Vector3 centerToBePivot = Vector3.Lerp(startPos1, startPos2, 0.5f);
Vector3 anglesToRotate = -Universe.lastTransformHandled.eulerAngles;
startPos1 = RotatePointAroundPivot(startPos1, centerToBePivot, anglesToRotate);
endPos1 = RotatePointAroundPivot(endPos1, centerToBePivot, anglesToRotate);
startPos2 = RotatePointAroundPivot(startPos2, centerToBePivot, anglesToRotate);
endPos2 = RotatePointAroundPivot(endPos2, centerToBePivot, anglesToRotate);
*/

// preparing some values
Vector3 scale = transformToHandle.localScale;

Vector3 startPos1 = positionEmptyClickStarted;
Vector3 endPos1 = transform.position;
Vector3 startPos2 = otherDotScript.positionEmptyClickStarted;
Vector3 endPos2 = otherDot.transform.position;

float leftRightDistanceStart = Vector3.Distance(startPos1, startPos2);
float leftRightDistanceEnd = Vector3.Distance(endPos1, endPos2);

float growth = leftRightDistanceEnd - leftRightDistanceStart;

// along which axis do we want to stretch? Pick biggest
float xStretch = Mathf.Abs(startPos1.x - endPos1.x) + Mathf.Abs(startPos2.x - endPos2.x);
float yStretch = Mathf.Abs(startPos1.y - endPos1.y) + Mathf.Abs(startPos2.y - endPos2.y);
float zStretch = Mathf.Abs(startPos1.z - endPos1.z) + Mathf.Abs(startPos2.z - endPos2.z);

// apply the scaling now
if ( xStretch >= yStretch && xStretch >= zStretch ) {
    scale.x = scaleAtStart.x + growth;
}
else if ( yStretch >= xStretch && yStretch >= zStretch ) {
    scale.y = scaleAtStart.y + growth;
}
else if ( zStretch >= xStretch && zStretch >= yStretch ) {
    scale.z = scaleAtStart.z + growth;
}

// ensuring scaling won't go into negative or too small
float min = 0.01f;
if (scale.x < min) { scale.x = min; }
if (scale.y < min) { scale.y = min; }
if (scale.z < min) { scale.z = min; }

// reapply scale and rotate back to how it was
transformToHandle.localScale = scale;

transformToHandle.eulerAngles = originalAngles;

So it turns out the problem in above code was that for the alternative, outcommented try, this

Vector3 anglesToRotate = -transformToHandle.eulerAngles;

… needed to be this …

Quaternion inverse = Quaternion.Inverse(transformToHandle.rotation);
Vector3 anglesToRotate = inverse.eulerAngles;

Then it all works.