UI ScrollRect is bouncing back to Origin

In my Inventory-UI I want to create a scrollable List, but I noticed some Issues.
My List was always bouncing back, so i tested the ScrollView with an Image.

The Canvas Scaler is set to “Scale With Screen Size”.
When I’m trying to scroll, the Image will bounce back to its Origin. The Image is larger than the Scroll-View.

When I set the Canvas Scaler back to Default (Constant Pixel Size), it is scrolling without bouncing back.

Why is it bouncing back to the Origin of the Image? Is there a solution without setting the Canvas Scaler to Constant Pixel Size?

I recreated the Canvas with a Resolution of 1280x720 instead of 1920*1080 and repositioned everything.
now it seems to work correctly.

This problem is still unsolved as of Unity 2020.3.20f1. Tried workarounds but failed.