Remove lag when dragging object (touch based iphone), possible?

Just trying to get an object to follow the finger when draging around the screen.
Currently it works pretty badly as the actual object trails heavily after the fingers position.
This is when I actually run it on the iPhone by Building and run.

Supersimple scene, just a plane, a ball and four cubes. The object sometimes trails as much as up to an inch (couple of centimeters).

Using the remote shows the ball on my Mac (of course) and it “looks” a lot snappier there (I pay no attention to the iPhone in this case as I understand it streams video that is pretty laggy).

My code: (this.beamShortSide is the object I’m trying to drag around)

void FixedUpdate()
	foreach (Touch touch in Input.touches)
		Ray ray = Camera.main.ScreenPointToRay(touch.position);
		int layerMask = 1 << Layer.BaseSurface;
		RaycastHit hitInfo = new RaycastHit();

		if (Physics.Raycast(ray, out hitInfo, Mathf.Infinity, layerMask))
			Vector3 hitpos = new Vector3(hitInfo.point.x, 0.1f, hitInfo.point.z);
			this.beamShortSide.transform.position = hitpos;

Is it even possible to make an object track your finger really close? Should I stop trying to find a solution?

You should move all logic into Update() or LateUpdate() instead, doing it in the physic time step is what causes the delay. Are you sure you need a for-loop for all touches? If the player isn’t able to interact with the beam with multiple touches you’re better off just checking Input.GetTouch(0). You should also cache the components (for instance all transforms and make them into variables).