Please help with collision (top down shooter 2d),Help with collision 2d top down

It’s just really weird. When the player hits a wall and you let go of the move key, they will get pushed back, but if you keep the key pressed down, the player goes through the wall. I am very new to unity (my first project)and I really just have no Idea what is happening.
Here is a recording of what happens(please download I don’t know how to upload videos:

and here is my code to move the player (and rotate them)

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class PlayerController : MonoBehaviour
{
    public float moveSpeed = 5.0f;

    void Update()
    {
        //code to move player
        float xmove = Input.GetAxis("Horizontal") * moveSpeed;
        float ymove = Input.GetAxis("Vertical") * moveSpeed;
        transform.rotation = Quaternion.Euler(new Vector3(0, 0, 0));
        xmove *= Time.deltaTime;
        ymove *= Time.deltaTime;
        transform.Translate(xmove, ymove, 0);

        //code to rotate to mouse
        Vector3 mousePos = Input.mousePosition;
        mousePos.z = 0;

        Vector3 objectPos = Camera.main.WorldToScreenPoint(transform.position);
        mousePos.x = mousePos.x - objectPos.x;
        mousePos.y = mousePos.y - objectPos.y;

        float angle = Mathf.Atan2(mousePos.y, mousePos.x) * Mathf.Rad2Deg;
        angle -= 90;
        transform.rotation = Quaternion.Euler(new Vector3(0, 0, angle));
    }
}

Just so you know, the player object has a rigidbody2d(gravity set to 0) and a boxcollider2d
and all the walls have just a boxcollider2d (grass does nothing that is there for show)
I haven’t ticked isTrigger or anything. Thanks for your help in advance.

@TheThree13 Have you found any solution yet? I’m facing a similar issue.