Washing Machine Syndrome

washing machineAt some point in your life, it happens. The washing machine stops spinning out or simply dies. What a major inconvenience to have wet and soapy clothes. You may have to go buy some underwear for the next couple of days or try to find a laundromat. It costs as much to repair as it does to buy a replacement. If you’re fortunate, you whip out the credit card on the weekend and try not to get sucked into your favorite color or cool front loader with the matching new dryer. Depending upon your stage in life, you may be forced to hit the garage sales and get a friend with a truck to pick one up.

Regardless, the old washing machine is removed and the new one plugged in and screwed to the hot and cold water. You’re back in business and wash away without another thought until it happens again in another 6-8 years. The problem is that many companies take the same approach with their servers and it’s not as simple as replacing the washing machine.

Anyone in business can tell you, there is significant risk and lots of small and massive failures. Owners have a lot of moving pieces to understand and must have the courage to make decisions like that old Kenny Rogers song – count your money, be ready for the cards you are dealt,  and know when to walk away or run. These same people wouldn’t dare try to not pay payroll tax for a couple of months (which can never be escaped), but regularly risk it all to run 12 – 36 months out of warranty on servers running the business.

You see I have a unique ability to tell the future about when a server will die. Well, actually it’s simple. Servers are generally warranted for  3 years with some limited options for upfront 4 or 5 years. That means there are no parts or extremely limited spare refurbished parts throughout the world at warranty end – thank you lean manufacturing. So that means if you have only one power supply that shorts out, it’s 5-10 business days to have a new server shipped plus emergency recovery because complex hardware and software just isn’t like plugging in a new washing machine. Bad RAM, that old type is probably not available. If you have RAID and one drive dies and the machine is still running, try 5 times the cost of a new one with 5 times the capacity wasted to replace.

I’ve heard it all: “That’s the best server we ever bought” or “We just got that server in 2005″. This is generally begun or ended with some explicative or irrational statement about so much was spent, it should last for 20 years. Then, wait for it, the new equipment won’t run any of the old software and the new software requires other new software.

Treating your infrastructure like a washing machine just costs the business half to a full month’s expenses and revenues, plus lost goodwill with employees, customers, and suppliers – along with bad press and loss of new potential sales. But its IT or that support firm’s fault right? We’ll just fire those bastards. After all, IT has no budget authority and those consultants were crazy and the recruiter of the day with PC Magazine says anybody can do this stuff.

Ding. Ding. School is in. While it may feel good to fire someone in this situation, how about trying to prevent it in the future? Fact: your equipment is only as good as the warranty. Often it’s only like $200 – $400 more to get that 4 hour 4 year warranty part replacement from Dell. Then a hardware problem is fixed quickly versus doing everything by hand, waiting 10 business days for a new server and weeks to workout restoration with the waterfall effect of upgraded software.

Back in the day like 10 years ago, the answer was to buy two of each server or a spare parts kit for each server. The cost was exorbitant and the software configuration overly complex and rarely worked. Today, you stop repeating the washing machine syndrome by:

1) Cloud Computing – Get rid of most of your servers. Escape the cost of hardware, software, maintenance, and upgrades. Along with less cost, a big chunk of your disaster recovery is also resolved.

2) Virtualization – What servers are left should be virtual with reduced equipment cost and the side benefit of running on older versions of software on the virtual servers longer.

3) Managed Services – You’ve outsourced everything from the coffee service to payroll because those services do it better and for less cost. Why not shrink or eliminate your IT staff for the same benefits? The good ones manage cloud computing and virtualization, as well as keeping you current for anything on-premise.

Know Requirements

The most common mistake made in IT is not knowing the requirements to implement technology. However, the real problem usually starts with not understanding business needs and/or having some kind of staunch bias.

Take a website for example. Most people review various sites they like and have it built using whatever technology has the best pitch. Today there is no point in having a website without knowing how customers will find it or a clear call to action when they do. Gone are the days of the spaghetti PHP code and custom content management. Business people should be well versed in common Search Engine Optimization (SEO) and feel comfortable using any WYSIWYG editor for simple updates.

Now that Windows 7 is becoming prevalent, many are finding that third-party anti-virus products no longer function or must be upgraded. The old habit of throwing an anti-virus console on a domain controller is not supported by Microsoft and nearly all security products now require a server of their own.

2010 is a big transition year for 32bit to 64bit architecture. While most server applications are now 64bit, workstations and remote desktop services should remain 32bit to match the prevalent applications. Otherwise, applications will actually run much slower due to the conversion called thunking.

The dirty secret of virtualization is applications like Exchange and SQL  are only supported under HyperV. Novices often try to get away with only one host and don’t provision enough memory or network cards to support the load, much less consider storage for data or failover.

There is much confusion over hosting versus cloud computing. Moving servers to hosted rack space is a good solution for things like e-commerce sites. However, it doesn’t fit for general network and line of business applications as slow and error prone VPNs are often required with costly charges for bandwidth, rack space, server maintenance, and backup. On the other hand cloud computing offers things like Exchange online for $5 per month per user.

Knowing requirements allows management to make informed business decisions concerning technology while saving time and money.

Remove IT Conflict

Your IT Department likely has contempt for you or best case is just indifferent. The reason why is that any feedback you give is negative and what measurements do you really use to judge IT performance?

There are always the same 3 plays for someone in IT:

1) Grow a department.
2) Fly below the radar and keep status quo.
3) Learn something new and move on in 2-3 years.

The first play goes like this: upgrade the infrastructure often starting with cabling/bandwidth, implement some new technology, get some licensing, add desktop and application people, and then rule the fiefdom. The second play is to latch onto some technology while avoiding change and guarding legacy knowledge until termination or retirement. The third play is often the most confusing for businesses as all seems well until the player suddenly turns in their resignation one day. The really scary thing is that each type of player may wake up suddenly and decide they want their own business, wreaking havoc on themselves and unsuspecting customers before understanding they have little chance of success or happiness with a hyper-competitive market and limited business/sales/marketing/financial knowledge.

No scenario described above really has any advantages for business. So it’s time to take control and recognize your situation. After all, IT is your number 4 business cost after salaries, rent, and taxes/insurance. The following are a few general tips:

  1. Know the cost of IT personnel. You should pay student interns no more than $10-$15 per hour and the rarely occasional one-man-show “consultant” only $30-$35 per hour. Ignore Salary Surveys and look up the real wages reported by the state for IT positions. Demand a relevant Bachelor Degree and current certification or adjust earnings down. Check references and ask for a copy of a W-2 to verify previous salary. Much of these same things should be done when picking a vendor.
  2. Measure IT to reduce cost or increase sales. Don’t have IT report to Accounting. Accounting has no understanding in this area and should be focused on business valuation, shortening accounts receivable, increasing credit lines, and reducing debt/taxes. Ideally, try to tie a portion of compensation to reducing cash outlay over time or increasing revenue. Also, there should be some built-in retention so things like bonuses are lost on an early exit. Know that you get what you measure and you should have an easy and regular way to review monthly to quarterly.
  3. Focus on business and not technology. Your IT staff would much rather help improve the business than do mundane tasks or risk livelihood implementing new technology they don’t know. Instead of having IT struggle for time to update and maintain the environment or before hiring that next IT staff person, consider managed services for better support at less cost. Ensure business continuity, lessen IT staff involvement, and reduce storage and tape costs with online backup. Spend less for hardware with virtualization or maybe eliminate some portion of IT infrastructure with cloud computing.

Like anything in business, you’ll find there are some things you need to stop doing and others that have been completely ignored. For certain, you can keep blindly spending money or embrace your IT and move the business forward.

Don’t Lose a Quarter

It’s year-end and time to reflect on your performance. When I ask most people how they measure IT, there is always either a blank stare or a 15 minute Obama-like answer. The tendency is to get caught up in the holiday rush, have a much-needed break with family and friends, and then get steam rolled by year-end/year begin just to realize in Q2 you’ve made little progress and generally do nothing until something major breaks.

Here are 3 simple things you should do now, so you’re not in the same predicament at the end of 2010:

  1. Draw a picture of what you have on the left and what it should look like on the right. Don’t worry about too much detail like IP addresses, but rather versions of OS and warranty expiration dates  of equipment. List a few problem bullets in the upper left and wins in the upper right above the changes in the environment you show on the right like removal/migration of systems. Whenever IT is a topic throughout the year, refer to the picture.
  2. Create a spreadsheet with columns for the next 5 years and list down the side software licenses and other categories like servers and workstations. Tip – You’ll usually want to buy new workstations for a third of your fleet each year so there is not a huge cash outlay every 3 years. It’s common to have a detail page for each category. License agreements are usually 1-3 years and warranty on equipment is generally 3 years. Now you have a tool for no surprises and a format to quickly add  total cost of ownership for new business requirements.
  3. Make a list of projects. Then rank them and schedule by quarter.

With these 3 things, manage the BUSINESS of IT by dollars, date, and informed knowledge of issues.