You can on a LAN, but on the internet, it’s more complex, and made easier with the master server.
I will give a roundown on how IPs work over the internet, shall I? 
Basically, (You probably know this already) every router and internet connection in each person’s home has a local network (LAN) and a router that connects to the outside world. (The internet) To do this, the router’s job is to grab requests from the LAN computers, and send it out through a different IP. (Your public IP address)
Consider this tree diagram:
Internet
| (Public IP applies here)
Router
| (LAN IP applies here)
Client Computers
Here, our client may want to, say connect to google.com. (Classic example, I know!
) What happens is that when the computer does this, it sends the request to the router over the network. Then the router translates the local IP to the public one your ISP assigned to it. (I find it’s always assigned the same IP every time) this Public IP is used to then fetch the data from the server, then once received, pass it to the LAN computer asking for it.
But what if you want to connect one device to another remotely? Well, you would think only one Local IP would do. Not so, as they, in your case exist on two different networks! To access such device, both of them need to be aware of their connected router’s public IP. (Let’s say one of them uses 203.189.3.232, and the other uses 122.150.181.242) to connect to one of them, the device that want’s to connect need sto query this to get to the device.
Here is where the weird stuff comes in: Once a device has made a request, sure, it can be received by the router however you like, but then there’s the problem of which device to forward the request to!
The fix: in any type of networking software, including games, the server being communicated to usually uses both an IP address (usually a local one, with a router providing the public one for remote access) and a port. The port is the key here, as the port is where the data is forwarded through. To tell the router which computer will take the request, you use port forwarding on the router, and the router takes care of the rest.
Also, since a public IP can be prone to change, it is also recommended one uses a domain name and a DNS service to host the public IP. no-ip.com has a good free service. what you do is set up an A-type DNS record, and have it point to your public IP. that way, any device can grab the URL, query the public IP from the address table, and then use the newly acquired IP to connect to the remote server.
For your case, it is as simple as either deploying a master server (Which is actually not all that simple for beginners) or using Unity’s. either way, just use that to connect your two devices from separate networks.
That is all for now.
EDIT: I will also note that technically, in theory, one could run the unity master server executables themselves and use them to make their own server, but in practice, it is pretty unwieldy, and a much better idea is to hand such duties to a remote server on the internet.