How to specify minsdk for Android plugin after Unity 2020

After I update my project to Unity 2020, I got following error message when building APK.
The min sdk version cannot be specified in the AndroidManifest.xml file. You have to remove it.
I located uses-sdk android:minSdkVersion property in the plug-in AndroidManifest.xml and I can build again after I removed this property. But I would like to know is there a proper way to specify the minimum required SDK version for plug-in after Unity 2020? I tried adding:

apply plugin: 'com.android.library'

android {
    compileSdkVersion "android-16"
}

to build.gradle under Android plugin folder and it seems Unity did pick it up but result in a build fail.

build.gradle file is the correct place to specify minSdkVersion. Note that you have specified the compileSdkVersion, not minSdkVersion.