Author’s Note: I am currently a MSP Partner with Calyptix Security, and have been selling their product for about a year now. I am exceedingly happy with my decision to work with Calyptix, and for the sake of transparency thought I should make this point clear to readers that are new to my blog, so that any perceived bias would not be misconstrued.

Harry from SMB Nation posted a request yesterday for a comparison between Calyptix and Untangle, with SonicWALL, Watchguard and Napera thrown in for good measure. Having been in this space for a number of years, and having worked for different companies that sold and supported a wide range of products, I feel like I can make a worthwhile contribution to this particular question.

Over the past five years I’ve had the chance to support products from the following vendors (Yes, I know Barracuda isn’t a UTM company, but they are often used in comparisons):

• Astaro – Astaro Secuirty Gateway Software and Appliances (Versions 5-7)
• Barracuda Networks – Spam Firewall (Model 200 and 300), Web Filter (Model 210, 310 and 410)
• Cisco – PIX 501, PIX515E, ASA 5510
• Fortinet – FortiGate 60/60B, FortiGate-100A, FortiGate-200A
• Juniper – SSG5, SSG140
• ServGate – SSG100, PointForce, EdgeForce M30
• SonicWALL – TZ150, TZ170, TZ180, 1260 PRO, 2040 PRO, 3060 PRO, 4060 PRO
• Watchguard – SOHO6, Both Edge and Edge e-Series, Core X550e, Core 700 and Core 1000

I feel this leads me to be able to have a well informed opinion on the performance, reliability, and overall quality of devices in the UTM space from a consultant’s point of view. I’ve worked with them all, seen the bugs, seen the places where they fall flat, seen the places they shine, and over this period had just about every vendor commit some form of unforgivable sin or simply not live up to the fact that there is an issue with their product that needs to be corrected. Some of the most repeatable and common issues that I could bring up with my past experiences with the major players:

• Poor Support – Most all the big vendors have outsourced, call-back support. Current record awaiting a return phone call is two weeks. When a company has a single point of contact for support via phone and you wait on hold for twenty-plus minutes only to be dropped into a voicemail and told someone should call you back within four hours, how do you explain to your customer that is down and needs a replacement box that they are just going to have to wait. The other half of the time the support staff simply doesn’t know the features of the boxes they support.
• Long Development Cycle – You’ve identified a definite bug with the product you use. After many hours on the phone with the outsourced technical support and finally being moved to Tier 2, they agree that it is a problem with the box. Great, it’ll be fixed in the next software release. Problem, next software release won’t be for two months. What am I supposed to do now?
• Level of Detachment – Most of these companies selling UTM Appliances and Firewalls are huge. Unless you sell hundreds of thousands of dollars of their equipment a year, you are simply another number to them. They don’t care about you or your level of success. You probably never even deal with anyone from the company because you do all your purchases through Ingram or TechData, et al. You could probably call whoever is your Account Representative with your vendor and they wouldn’t have a clue as to who you are.
• Lack of Influence – These vendors all have finely crafted development timelines meant to follow the current trends and anticipated movements in the security space. Your request for a new minor feature will most likely fall on deaf ears, or at the most, end up in the next major build of the product planned for sometime in Q3 of next year. If it’s a major feature change or upgrade, unless more people get onboard with you to request it, you just may simply never see the change.
• Price – Depending on configuration and features, some of these boxes can be horribly expensive to meet the needs of a SMB. There was a Title Company we did a quote on an Astaro for one time that ended up being almost $10K, and that was before the hardware to run it on, which would have brought the total up to about $12K. There simply had to be something better out there that didn’t run at such a high cost point. The fact that the pricing options for a lot of these products can require a complete PHD in discrete mathematics to figure out.

These were all factors in my decision process when I was looking at a provider for my managed firewall service when I was starting my company. I wanted something that was reliable, stable, progressive, and fixed the problems that I had with the other vendors I worked with in the past. There was a long process of researching various vendors for my new solution, including the ones listed above as well as:

• Calyptix Security
• Cyberoam
• Endian
• eSoft
• GTA
• Smoothwall
• Untangle
• XRoads Networks
• ZyXEL

In the end I selected the Calyptix AccessEnforcer as my vendor for my company, but not after a long exhaustive search, lots of reading, lots of comparisons and plenty of hours on the phone with sales representatives. If you are interested in reading more of the final parts of why I chose Calyptix as my vendor you can read about it in this post here.

Being that this is a response to a question about Calyptix versus Untangle, I also need to hit on my reasons on why Untangle wasn’t chosen as my preferred vendor. Untangle seemed interesting enough, but not very much a standout in any matter when I first started looking into it. Personally, I felt initially that Untangle was just another Smoothwall, as they had their free product, but it you pay them money you can get support and more features. To make things even more muddled, that puts them in the same boat as Astaro and Endian as well. That’s four companies all with very similar models of a free version (although Astaro doesn’t advertise it) that you can then add commercial add-ons to gain more features and support.

When I installed and ran Untangle in a test environment on a Dell Optiplex, I really can’t say I had any problems and things went smooth enough, pretty much the same as most of the other Linux based firewall distros I’ve used in the past. Though the biggest issue I had was once I loaded the interface and the Java GUI kicked in. Instant flashbacks to the pain that was the old Symantec firewalls, which to this day I rate as the worst firewalls ever built. In this day and age I simply find a Java interface for something that can easily be done via a simple web interface completely unacceptable. This was an instant black mark against Untangle in my book, and something that pretty much kept them out of the running for a winning spot in my book. I still ran the box for a week, and can’t really come up with any complaints for the box outside of some tweaking and trying to get to know it.

The short version on the product itself was that I simply didn’t feel the synergy. I didn’t enjoy administrating it, and didn’t feel that I could be enthusiastic about selling it. The other part of the coin, relating to the company itself was a bit more interesting of a story.

Like all the other non-major players, I did a lot of googling and reading to find out as much about the companies themselves as possible, and see if I could find any horror stories or praise pieces that weren’t directly linked to the company itself. It really seemed that almost all the work I did researching the company all ended up pointing back to their forum or wiki. The one thing that really got me was the fact that they had been working under their old moniker (Metavize) for years selling an appliance and then one day suddenly changed models, secured a bunch of funding and said we are doing things this way now.

Granted, I have to give them credit for contributing to the OSS community, but the fact that on more than one occasion I’ve seen them compare themselves to SugarCRM, which I find troubling. I’ve had many involvements with Sugar in the past and currently, and none of them have been anything approaching good. In fact, just about everyone I know that has used and paid for Sugar has ended up switching to a new product within two years, with most of them moving to SalesForce. If you are going to relate yourself to a product, I’d make sure that it is one that is thought of more highly than SugarCRM.

Their pricing structure is another part that put me off, as it pretty much fits in with the overly complex formulas of some of the other larger vendors. I don’t want to have to pick from this or that, and figure out pricing. I just wanted something simple. I want this model and it’s this price (which I love about the Calyptix model, One Box. One Price). Also, their pricing complete neglects to add-in the actual hardware that the box is going to run on, and the preconfigured boxes are unacceptable to me, as there is no reason they have to be selling a 2.5U unit. There is no reason these boxes couldn’t be at least SFF or 1U units. Maybe I’m overly picky, but a good 1U server that I would find acceptable would probably end up running a minimum of $1500 on top of all that, and it’s now another vendor to manage for support. I want the vendor to be responsible for the whole setup so that there is no possible chance of them coming back saying “Oh, we don’t support that hardware”. Their HCL is extremely short, which extremely limits my options for how to build a server for a client. Which makes the chance that the unsupported hardware conversation even greater.

The one real gripe I have with Untangle, that really has come up post my decision to not use them, has been their blog and the testing methodologies used in their two fight clubs that they’ve done so far. First the Virus fight club, where they tested stand-alone scanners against UTM appliances (apples against oranges) and most recently the porn filtering capabilities. Neither test specifies exactly how the boxes were configured (as I noted in a previous post, the SonicWALL’s GAV seems dependant on blocking packers for full effectiveness), and both completely leave their own product out of the mix. If you are going to perform that sort of comparison testing, you have to use your own product; otherwise it looks like you are simply trying to pass off a test as Snake Oil. Granted if their product won, it would look biased, but that’s a lot better than sitting out a test where you are critical of your competitors.

I’ve actually been working on redoing both their Fight Club tests on my own and have almost completed the virus testing, and been slowly working on prepping (and verifying) the porn test. I tweaked the virus test to only include UTM firewalls, versus using some desktop applications and some UTM boxes. So far I’m happy to say that the Calyptix AccessEnforcer passed the virus test with a full 100% pass rate and it also blocked a number of them even before the Web Filter got them via the IPS.

So if I didn’t choose Untangle, then why did I choose to partner with Calyptix? Well for starters, their business model of “One Box. One Price.” was what got me in the door and moved the ball forward. From that starting point it became a number of different points including:

• Flexibility - The flexibility that they had in how they presented their offering. They actually built a program for MSPs to use to sell their product. No huge cash outlay up front to get a box in a client’s location.
• The Technology - OpenBSD is simply secure and rock solid. Too many people pass over it for Linux in this environment and I feel like it’s a mistake. pf is vastly superior to iptables that most of the Linux based systems use. The fact that they wrote their own inspection algorithm from scratch wasn’t a bad thing either.
• Simplicity - The box is simple to administer. Most of the features that the box incorporates has all the needlessly complex configuration details taken care of automatically. This drastically cuts down on setup time, and the amount of hours per month required to manage the box effectively.
• The people - Lawrence and his crew are a great team with tons of talent. They know what they are doing. They write good code, and they focus very heavily on helping their partners have the best product they can build.
• Quality - I have yet to see them release a half-working or broken feature just to get it out there. They’d rather take the extra time and have it work right the first time versus putting something on the box that is going to cause problems.
• Belief - Everyone that works there truly believes they are making a great product for the SMB space, and you can tell it every time you talk to them. It shows in everything they say and do, and is one of the factors I believe will lead them towards long term success.

Since partnering with Calyptix, I’ve been introduced to lots of interesting people in the SMB community, and had plenty of cross-referral opportunities passed through them for various projects. I truly enjoy getting together with them whenever they are in town (or close by), and am hoping to make time one of these days to make the trek to Charlotte and give them a visit.

I’ve had plenty of early looks at the next generation of the Calyptix AccessEnforcer and everyday am more and more pleased that I chose to work with the team at Calyptix to provide security to my clients. With the features in the next release, and the GUI enhancement, and the general improvements being made daily to the box I’m going to be very excited what the SMB market will have to say about the product when it reaches the 2.0 milestone.

Now, I have not had enough interactions with the people at Untangle, or enough time running an Untangle server to say that you shouldn’t try it or even partner with them. My entire point was that I did not feel a synergy between Untangle and myself when I looked into their offering to feel that I should sell their product. The people at Untangle could be the greatest people in the world, but I haven’t met any of them (which surprises me, do these guys go to any SMB conferences?). I’m always open to talking to new people, and seeing if there are new ideas out there I haven’t thought of, but I’m proud of my choice to work with Calyptix and being a part of making their company a success.

I’d love to hear more from people that have used Untangle or are Untangle partners and why you chose to use their product and what other products you evaluated in your decision making process. Or even more broadly, have there been any people out there in the last year or so that switched firewall vendors to use a newer or different UTM Firewall solution?