IMO no, because if they could take over your IP address they could already obtain a domain-validated certificate which is arguably a lot more valuable than an IP address certificate.
If an attacker manages to gain ownership of an IP address, and gets a Let's Encrypt certificate for that IP address, the certificate will show up in Certificate Transparency logs. In that way, if people are watching, the attack will become visible fairly quickly.
At least in France and I think in a lot of other countries, you still get a dedicated IP for your connection, so yes you could receive inbound traffic.
Just the IP will most of the time be dynamic, and you might have your IP changing regularly.
Update: Had less time to post than I realized, hence the terse reply.
Meant to say those solutions are in addition to Lets Encrypt. An X509 certificate is an X509 certificate, regardless if its for https, imaps, or smtps. If you're distributing your stuff across multiple hosts or containers, then it makes sense to use some sort of automation, configuration management, or certificate management/distribution system.
Related 6-Day and IP Address Certificates Are Generally Available (506 points, 2 months ago, 281 comments) https://news.ycombinator.com/item?id=46647491