Issues when using Rigidbody force to Move/Carry Objects

Hi all, I am having some issues when changing my Object Carrying Method from a Static Locked position attached to the player to a more fluent movement that also has physics so carried objects cannot phase through walls etc. The issue I am facing while using this is that the object if placed under the Player and then picked up it levitates the player up into the sky, is there a way that while an object is being carried it ignores only the players collider, while still being able to crash into walls and other objects?


The other issue I am having with this is that the object never seems to settle at the carry position, it always overshoots it’s mark and is constantly ‘bouncing’ back and forth around the carrying position.


The coding I am using to move the object toward its position while being carried is as follows:

void moveObject()
    {
        if (isSmallOBJ)
        {
            if (transform.position != smallDest.position)
            {
                if(distance < 0.05f)
                {
                    Vector3 f = smallDest.position - transform.position;
                    f = f.normalized;
                    f = f * lowForce;
                    rig.AddForce(f);
                    distance = Vector3.Distance(transform.position, smallDest.position);
                }
                else if (distance < 0.5f)
                {
                    Vector3 f = smallDest.position - transform.position;
                    f = f.normalized;
                    f = f * midForce;
                    rig.AddForce(f);
                    distance = Vector3.Distance(transform.position, smallDest.position);
                }
                else if (distance < 5f)
                {
                    Vector3 f = smallDest.position - transform.position;
                    f = f.normalized;
                    f = f * highForce;
                    rig.AddForce(f);
                    distance = Vector3.Distance(transform.position, smallDest.position);
                }
                else
                {
                    dropObject();
                }
            }
        }
        else
        {
            if (distance < 0.05f)
            {
                Vector3 f = dest.position - transform.position;
                f = f.normalized;
                f = f * lowForce;
                rig.AddForce(f);
                distance = Vector3.Distance(transform.position, dest.position);
            }
            else if (distance < 0.5f)
            {
                Vector3 f = dest.position - transform.position;
                f = f.normalized;
                f = f * midForce;
                rig.AddForce(f);
                distance = Vector3.Distance(transform.position, dest.position);
            }
            else if (distance < 5)
            {
                Vector3 f = dest.position - transform.position;
                f = f.normalized;
                f = f * highForce;
                rig.AddForce(f);
                distance = Vector3.Distance(transform.position, dest.position);
            }
            else
            {
                dropObject();
            }
        }
    }

Showcasing the Issues

Youtube Video

UnityEngine has a function:

    Physics.IgnoreCollision(colliderPlayer, colliderObject, true);

to disable collision of these two colliders, and:

    Physics.IgnoreCollision(colliderPlayer, colliderObject, false);

to enable collision of these two colliders.
_
When you pick up an object, you need to use the first function for all players and all colliders of the lifted object, and the second function - when throwing the object.
_
The moveObject() function code can be simplified:

void moveObject()
{
    if(isSmallOBJ)
    {
        if (transform.position != smallDest.position)
        {
            if (distance < 5)
            {
                rig.velocity = Vector3.zero;
                rig.MovePosition(Vector3.Lerp(rig.position, smallDest.position, Time.deltaTime * 15));
            }
            else
            {
                dropObject();
            }
        }
    }
    else
    {
        if (distance < 5f)
        {
            rig.velocity = Vector3.zero;
            rig.MovePosition(Vector3.Lerp(rig.position, dest.position, Time.deltaTime * 15));
        }
        else
        {
            dropObject();
        }
    }
}

In this code, the object does not tremble in different directions.
_
Also, the MovePosition() function takes into account physics, so the interaction with other objects will be correct.