Need equivalent of Transform.InverseTransformPoint() but without using transforms

I want to be able to do the equivalent of this:

var worldpos = Vector3(0,0,2);
var invtPoint = transform.InverseTransformPoint( worldpos );

But without using a transform - so instead:

var pos = transform.position;
var rot = transform.rotation;
var worldpos = Vector3(0,0,2);
var invtPoint = // ???

This previous answer kindly showed me a simple way to calculate the other way around (TransformPoint):

var tPoint = rotation * localpos + position;

… but now I want to convert from world to local, rather than local to world. Thanks in advance!

The transform from local to global is equivalent to: scale, rotate, translate in that order. To do the inverse transform you just process it backwards. This means -translate, rotate.inverse, 1/scale. so:

Vector3 t = transform.position;
Quaternion r = transform.rotation;
Vector3 s = transform.localScale;
Vector3 sInv = new Vector3(1 / s.x, 1 / s.y, 1 / s.z);

Vector3 q = Vector3.Scale(sInv, (Quaternion.Inverse(r) * (p - t)));
float err = (q - transform.InverseTransformPoint(p)).magnitude;
Debug.Log(err);