I know you chose to use nginx fastcgi instead, but I have had great luck with Varnish too. For instance, I run a server on Cloudways that I have been running for over a year and I did not use Varnish with it at first. I left it disabled. Once I enabled Varnish, it sped up things so much faster. I backed up and copied the server over to RunCloud, but it doesn't load as fast. I am not taking advantage of any of the other caching layers on Cloudways (Redis, Memcached, etc.) and I am only using their Apache + Nginx + MariaDB setup with Varnish.
You could make it easier for us by writing a tutorial like this:
For some reason, Varnish delivers everything faster to the customers.
I ran it for awhile without Varnish and it was about the same speed as the RunCloud clone I setup.
I believe that Varnish works better for some people depending on their type of WP install and that it should be an option.