Android Swipe In 3D World

I am working on an AR application using Unity3D and the Vuforia SDK for Android. The way the application works is a when the designated image(a frame marker in our case) is recognized by the camera, a 3D island is rendered at that spot. Currently I am able to detect when/which objects are touched on the model by raycasting. I also am able to successfully detect a swipe using this code:

if (Input.touchCount > 0) 
{

        	Touch touch = Input.touches[0];

        	switch (touch.phase) 
			{
            	case TouchPhase.Began:
				
                	couldBeSwipe = true;

                	startPos = touch.position;

                	startTime = Time.time;

                break;

            
            	case TouchPhase.Moved:

                	if (Mathf.Abs(touch.position.y - startPos.y) > comfortZoneY) 
					{

                    	couldBeSwipe = false;

                	}
					//track points here for raycast if it is swipe
                break;
            

            	case TouchPhase.Stationary:

                	couldBeSwipe = false;

                break;

            

            	case TouchPhase.Ended:

                	float swipeTime = Time.time - startTime;

                	float swipeDist = (touch.position - startPos).magnitude;
					swipeCounter.guiText.text = "Touch phase ended. cbs: --" + couldBeSwipe;
                
                	if (couldBeSwipe && (swipeTime < maxSwipeTime) && (swipeDist > minSwipeDist)) 
					{

                    	// It's a swiiiiiiiiiiiipe!

                    	float swipeDirection = Mathf.Sign(touch.position.y - startPos.y);
	
                    	// Do something here in reaction to the swipe.
						swipeCounter.IncrementCounter();
                	}
				
                break;
				
				
        	}
			
			touchInfo.SetTouchInfo
				(Time.time-startTime,(touch.position-startPos).magnitude,Mathf.Abs (touch.position.y-startPos.y));
		}

Thanks to andeeeee for the logic. But I want to have some interaction in the 3D world based on the swipe on the screen. I.E. If the user swipes over unoccluded enemies, they die. My first thought was to track all the points in the moved TouchPhase, and then if it is a swipe raycast into all those points and kill any enemy that is hit. Is there a better way to do this? What is the best approach?

Thanks for the help!

I am not sure
But if your raycast is in OnMouseDown then store the gameobject
and also do raycast in OnMouseUp and compare both gameobjects
if both gameobjects are same then go further i.e kill the enemies.

Note: before it kills the enemies just check whether it is swiped or staionary