Codeless IAP not working

I have a couple of products successfully linked to the unity IAP system, but when I press any one of them, the “Purchased Failed” object shows up. What is happening?

I got this to work by setting up an alpha release and downloading the app from google play as an early access app. Unity has the docs here: Unity - Manual: Configuring for Google Play Store
It turns out that the IAP will only work if the game is published onto the google play or apple store.