Improving a Hosting Environment: 1. Web server OS
Windows has one BIG thing going for it - we're already a Windows shop. We have Windows servers, we currently run Microsoft SQL Server and we're comfortable with the OS. It doesn't do things quite the way we'd like, but at least we KNOW most of the problems we're living with. Also, by using Windows 2003 server Web edition and having a Software Provider License Agreement the cost really isn't much of a consideration.
That said, there are a few immediate strikes against Windows.
- Backup - We've had the hardest time finding a reliable, inexpensive (say sub $100 per server) backup solution (we tried rsynch on Windows, but it was just too funky for us). We have something that works, but I don't love it.
- Reliability - While it's come a long way from the bad old days, so many windows patches require a reboot that I've gotten pretty used to seeing a few minutes of downtime at 3am EST when out automated updates are run.
- Performance - There is still enough overhead in Windows that it is naturally less performant than a well tuned headless Linux setup. For us, not the biggest issue, but something we bear in mind.
- Security - I have an acquaintance who does security consulting. When I suggested securing a Windows server, he just laughed. Of course you can create passably secure Windows machines (and they're probable more secure than a general Linux distro with a n00b sys-admin) but for security Windows isn't the best choice. Also, with the Windows equivalent of software like Tripwire costing serious money, trying to get a PCI compliant setup using Windows servers is a real challenge.
- Skills - There are plenty of great windows sys-admins, but Linux gets the Guru's. It's like the idea of using Ruby for programming compared to Java - today irrespective of the benefits of the language, the people who have a years solid experience in Ruby are much more likely to be l33t than the average Java coder.
There are some downsides to Linux OS's.
- No Hand Holding - To do anything useful with a Linux box (GNOME/KDE notwithstanding), you have to learn to love the command line. One of the reasons I moved to a Mac was to get comfortable with the terminal, but there's no question that a command line isn't the optimal UI for a casual user.
- No Security Support - No windows firewall, no automatic updates. However flawed, at least Windows helps the casual sys-admin to have some degree of security. With Linux you really need advice from an experienced sys-admin.
- Too Many Options - Try to get a definitive answer - Debian vs. RHE - which is best? What packages do you REALLY need to install? What's the best package manager and the best repository to point them to? I just want a working server and I'd rather not have to learn the nuances - I just want it to work.right.first.time.no.fuss. For this, the world of Linux is not my friend.
The High Cost of Free Software
On balance, for us Linux will be the more expensive OS. The sys-admin costs for a handful of Linux boxes from someone we trust with all of our clients data and uptime will definitely exceed the licensing fee for a few Microsoft machines. However, because of the security and backup options available on Linux plus the easy ability to script everything, we've decided to eat the cost and increase our budget to move to the open source alternative.
Thoughts?
RHEL always worked right out of the box and more importantly, Adobe supports it. That should narrow down your choices of what to install very quickly. CentOS isn't Adobe supported, but it's been commented more than once by Stephen Erat as being the RHEL-alternative. CentOS is RHEL compiled by the community because they don't want to pay for RHEL support. CF installed on it with zero issues and the installer issues I had in beta went away when they fixed stuff for RHEL.
As for Updates / security support, one would hope that you have an actual firewall hardware rather than putting security on the responsibility of the box. That being said, ipchains/iptables isn't that difficult to figure out and is very easy to work with. Software updates? type: 'yum update' or apt-get update or any of the other favorite updaters and you're good to go. Just be careful what you're really updating.
Bottom line is that it is do-able to get off of the Windows OS for your servers. However much time that you put into it is what you're going to get out of it. I've been very happy with my setup and it is low maintenance for me, which was important.
You probably should have done the colo and data center topics first - because picking the right data center (and staff) can reduce some of these concerns. When we did hosting - I made a few decisions up front:
- no purchasing hardware - we leased hardware from the data center - and they were responsible for it - so when the drive controller failed - they had spare servers and parts on hand to fix it quickly - things we would not have been able to afford to have sitting on a shelf if we were responsible for hardware.
- OS - I went with Windows simply because no one else had any *nix experience and I didn't want that responsibility to fall solely on my shoulders. Running Windows allowed other people to remote in and do basic things without bothering me if I was on vacation, sick, etc.
- dedicated firewall - we ran a heavily locked down firewall - and again - it was maintained by the data center who had experts on hand to modify rules, etc.
- backups - again - data center - they had all kinds of redundant, off-site backup and storage - we could not replicate without spending huge amounts of money.
- reliability - again - data center - 99.9999% uptime, redundant backbones, hardware, etc.
- security support - most distros now do include 'automated updates' - Debian through apt and RH through yum I think. Of course with any updates - you need to roll them out and test first - I used to have a development server setup in our office on which I installed all patches first to test.
I don't think you mentioned monitoring yet. That's another important item if you have multiple bits of hardware to keep an eye on. I've used Nagios in the past and it worked great.
I think your logic is sound. I don't think many people look at the big picture issue with switching OS's at this level.
I would put forth that a properly patched and secured Windows box is not a huge security risk. That being said, we still keep ours behind a Cisco firewall.
I think that with some of your other topics in this series, you can mitigate some of the issues of Windows. (If you NLB cluster, then you can achieve zero downtime for users during rebooting updates.)
All that said, I'm looking forwards to porting to Linux.
@Jim, now-now!!! Causing trouble again?! Thanks for the info on your setup - makes perfect sense. I'll be covering each of those decisions in a little more detail over the next few days. Thanks for reminding about monitoring - I have a simple solution but will add it to my list of decisions.
@Terrence, Glad you're enjoying this! I tend to agree that the downsides of Windows can be mitigated with best practices, but all other things being equal I'd rather have less downsides to mitigate, so while I think there is a greater "cost" to Linux (at least for us as switchers), I think it is the right choice for us - at least for our web servers.
Only issue I have with any linux box is sendmail. Just about every default install comes with sendmail. That crap has been turned off and qmail was installed and I found a nice resource online to setup qmail easily - http://www.shupp.org/toaster/.
Not sure what Apache scripts he's talking about, just a base install of Apache is already about 10 times more secure than IIS has ever been. One thing you're forgetting is that there's a HUGE HUGE HUGE resource online for either platform. Type nearly any question or error into google and someone has something to say about it.
Thanks for the input - and the guide - that'll be really useful! I'm not forgetting about the online resources, it's just that either I need to pay someone else to track them down or I need to spend time (which I then can't bill) tracking them down, so either way it costs "money". It is great to see all the online resources that are available now - no question it makes life way easier when trying to figure something out.
Focus on the 20 percent of the business that's going to make you money, and leave the hosting to the experts in hosting. We took care of our own stuff for many years here, and since moving to Rackspace and leasing everything from them, life on the sysadmin side of things has been a lot brighter. You can do what you know how and ask their fanatical support for help with the rest.
Interestingly enough, I used Rackspace back when I had a dot-com in Houston, but their prices don't make sense for where we are as a company today, although I might revisit that down the line.
All great stuff guys - thanks!
That's why I want to go with someone who knows what they are doing. =)
"prices don't make sense for where we are as a company today"
Not sure what the pricing scheme is now vs then and what is too much, but we've found them cheaper than doing it in-house. I don't want to get into specifics here, but feel free to shoot me an email if you want (or I imagine you could tell if it changed by checking if you wanted...)
We buy our servers (pizza boxes) from old stock that is sold out the back door of HP. The prices are about 25% of new servers and come with manufacturer warranties ~AUD1000 per box with dual core processors and 2Gb ram and 2x 72Gb hard drives. Co-location is a _lot_ cheaper than dedicated and the reliability of servers these days are amazing (I've had no failures in six years!) Supermicro, interestingly enough are the fastest servers we have and we've never had a failure with them in six years.
As far as stress of management, there really isn't much stress if you set your environment up properly and spend a couple of weeks reading blogs and articles as well as the info on the microsoft website.
Here's some tips from my experiences:
1) get a good firewall with spam and content filtering : like watchguard firebox. (anyone can manage one of those)
2) get a couple of good managed (gigabit) switches with vlans (with web-based management)
3) run internal network and external network on each machine
4) if you can't find a good backup program, get datacentre backup
5) put your data on a different hard disk to your system
Make sure you check out Windows Server 2008 as well. There's hundreds of new features and new management tools that make life a lot easier and if you're running blue dragon it integrates into the windows server admin. It's also secure out of the box. (that's for the linux g33ks) and you can chose to install windows or not .
We chose the windows platform on the basis of management and staff. We have also deployed virtually using vmware therefore we have standard installs ready to roll if we need more servers or one goes belly up. we keep data and setting all mapped to the data drive for almost instant recovery from failures. we also use windows server standard using our SPLA
Many thanks for all of the great info here! I'm going to roll this into our thinking and will post something next week with our conclusions. Thanks again!
Out of interest, do you use VMWare (freebie) or their GSX product? Any thoughts on choosing between the two?