How to smooth transform scaling with 2d sprite

Basically I have a sprite of a cube that reacts to the loudness of the audio playing in the scene. I want to make this a scaling smooth. I looked at .Lerp but that doesn’t really work. Any help on this would be greatly appreciated.

public AudioSource audioSource;
public float updateStep = 0.1f;
public int sampleDataLength = 1024;

private float currentUpdateTime = 0f;

public float clipLoudness;
public float temp;
private float[] clipSampleData;

public GameObject sprite;
public float sizeFactor = 1;

public float minSize = 0;
public float maxSize = 500;

public void Awake()
    clipSampleData = new float[sampleDataLength];

public void Update()
    currentUpdateTime += Time.deltaTime;
    if (currentUpdateTime >= updateStep)
        currentUpdateTime = 0f;
        audioSource.clip.GetData(clipSampleData, audioSource.timeSamples);
        clipLoudness = 0f;
        foreach (var sample in clipSampleData)
            clipLoudness += Mathf.Abs(sample);

        clipLoudness /= sampleDataLength;

        clipLoudness *= sizeFactor;
        clipLoudness = Mathf.Clamp(clipLoudness, minSize, maxSize);
        sprite.transform.localScale = new Vector2 (7, clipLoudness);

Maybe you can make smoothdamp work with your project? Unity - Scripting API: Mathf.SmoothDamp