What are best practices for configuring a server for WordPress sites?

I hired a sysadmin to set up a VPS server for me and, unfortunately, it looks like things were not set up correctly. When trying to install and update plugins, I run into permissions errors all the time. WP Super Cache is the main issue as it causing my readers to run into 502 errors. Currently, my site does not load pagination (no Page 2, Page 3, etc..).