I am using SQLite in a project and every time I open a connection with the database I take advantage of the IDisposable interface with a using statement. But even though I do that unity still keeps a handle to the database open .The handles stay open until I close Unity . Here is an example of how I am accessing the SQLLite db:
After a doing a couple operations like the above I can see that Unity has many handles open to the database file by using the Process Explorer.

I would love to here some ideas as to why this would be happening. As a side note I am using dapper as an ORM.
First, I’ll say now that I am not an sqllite user, but I’ve worked with a few databases (MS SQL, MySql, and SimpleDatabase), in game development, with my own server core and with UE. Currently working with MySql in Unity 2018. But I know db behaviors and how to work with them.
SqlLite tends to keep connections open in order to speed things up. You can try using the Close() from the created connection, which MAY work. From my understanding of SQlLite though, the catch is that SqlLite can still maintain the connection if it thinks it will need it to keep transaction speeds up. But from my own personal experience, it is best to open a connection, perform the transaction or transactions needed, and then close the connection. In this case, to the file rather than a database engine.
I hope that helps.