Type
Go for lesser-known small companies that have a track record of high reputation, reliability and fast support. Justified Image Grid proudly uses MDDhosting since 2011, long before development has started. Their 24/7/365 ticket-based support's response time is insane, it's always just a couple of minutes to half an hour maximum. They are positioned in the middle of the USA, with their speedy servers placed in the Handy networks data center in Denver, Colorado. They offer a discount for new users and regular promotions for existing customers in form of a credit balance system with deposit bonus.
Location
Decide the physical location of your hosting, it's also important. Does the majority of your target audience reside in the same country as you? Consider which is more important to you, the server placed at a central location or fast route between you and the server? If your server is on another continent, then copying lots of small files or merely navigating on the FTP will be slow. Similarly, serving lots of small files such as thumbnails may come with delays. The reason for that is the high ping - round trip time it takes to communicate. In case neither are you or your audience is in the USA, there is even less reason to go for large US hosting companies. Choose a local company with a small user base. Your speeds shall be lightning fast for both you and your visitors. You can always use a CDN or the free Cloudflare to speed things up for international visitors.
Avoid
- Popular hosting companies and their cheap shared hosting packages. It may sound harsh, but these include Bluehost, Dreamhost, Hostgator, GoDaddy (by the way they bought Media Temple but continue to operate independently). With possibly millions of customers (gained through aggressive affiliate marketing), more and more sites need to share resources provided by a finite number of servers. They start to introduce superfluous security measures, connection rate limiting, tight memory and resource limits, to protect themselves and neighbor users. This makes your site prone to errors and a bumpy overall experience. To give you a metaphor, assuming the pricing was similar, where would you live? In New York City's projects or in a private gated community in Los Angeles...
- Any company that claims unlimited speed, disk space, monthly bandwidth. It's much better to use the service within your known and realistic limits than to be penalized for hidden "fair use" limits. The mindset behind "unlimited" is that not all of the customers would use all of the allocated resources anyway. There are quite a few things that can be truly unlimited though, such as add-on domains, email addresses, MySQL databases, FTP accounts etc.
- Packages labelled WordPress optimized or similar, especially WP Engine. It's usually marketing nonsense, WordPress doesn't need anything special except a solid foundation and a modern environment.