Lerping back and forth flickering

I am using lerp to move a texture back and forth between two points. My problem is that it does the first lerp just fine, but once it reaches it’s destination it is supposed to reverse and come back. Instead it flickers between start and end points, ie instantly arriving at the point and moving back again. Here is the code:

            Vector3 startpos = new Vector3(indicator.startPoint.x,indicator.startPoint.y,0);
			Vector3 endpos = new Vector3(indicator.animationLimit.x,indicator.animationLimit.y,0);
			Vector3 pos = new Vector3(indicator.Container.x,indicator.Container.y,0);

			if(direction == 1){
				pos = Vector3.Lerp(startpos,endpos, Time.time);
			}
			if(direction == -1){
				pos = Vector3.Lerp(endpos,startpos, Time.time);
			}

			if(pos == startpos || pos == endpos){
				direction *= -1;
			}

So after reaching the end point, it instantly returns to start point and then end and so on. direction instantly changing between 1 and -1. I’m sure this is an idiotic mistake, i’ve already made a few today, but i just can’t see it.

i tried ping pong gave even worse results, figured out what i was doing wrong though, made use of my own timer that used += time.deltatime and reset to 0 upon reaching either target.