Secure Mobile Unified Communications with Cisco Jabber
Secure Mobile Unified Communications with Cisco Jabber
You can have your entire Cisco desk phone as an app on your mobile device. Imagine having the ability to make phone calls, access voice mail, make video calls, see co-worker presence and conduct chat all through an encrypted connection back to your office all the time.
That’s what Adcap’s Secure Mobile Unified Communication provides, and we would be happy to talk to you about how to upgrade your existing Cisco phone service to enable it.
The Adcap Secure Mobile Unified Communications System is composed of different elements working together in harmony.
Connect to the cloud and the datacenter faster by using the new Intelligent WAN (iWAN) system from Cisco. Adcap engineers have the expertise and experience to enable this for your organization today. See links below the video for more information.
Cisco iWAN is a combination of technologies from Cisco, and design expertise from Adcap engineers, that provides faster and more reliable remote site WAN connectivity using business class DSL, cable, and 4G connections.
Information Technology systems keep getting more complex every year, and keeping the network running reliable only gets more and more important, especially as organizations rely on both public and private cloud services to operate their business.
Adcap’s PremierNet Annual Support Agreement provides the assurance and assistance your IT team needs to keep their complex network, unified communications, security and data systems running reliable. It’s immediate access to a team of technical specialist experienced in supporting these advanced systems.
Find out more about the support solutions, and how easy it is to get started with Adcap PremierNet IT Support Services.
In the first year of our business while we were still a startup company I hired a sales rep. He was our first sales guy. He said, “Can I be the VP of Sales?” I said, “Sure!” That was just one of the many dumb things I’ve done in my life, and later when I had other sales people but I didn’t want him to manage them I had to demote him. That was an uncomfortable situation, but it had to be done.
A few years later we had grown enough that I needed some managers, but I really had no idea of what the different titles meant. I knew what titles meant when i was a junior officer in the Navy, but that was no help. Every other work experience I had was just with individual contributors and managers. I tried to research it, but it was all very confusing to me.
After more than a decade owning and running our business, I figure I’d write down my thoughts and observations. I’m sure its wrong in some cases, but it’s the guidelines that I use.
Individual contributors are some of the most vital people in organization. They’re the sales people that bring in business, the technicians and engineers who do the work, the project managers that make things happen, as well as the finance and admin people who send the bills and collect the money.
Managers have to first be outstanding individual contributors. Then they have to gain the skill set of a teacher and leader of people. Manager should understand the role of the group of people and be able to optimize the efforts of the individual contributors.
Directors have the skill set above and beyond managers to be able to take action to achieve goals for the good of the company. They should be able to see when something is done correctly or incorrectly and take independent action to improve their own work as well as the work of the managers and independent contributors who report to them.
Vice presidents have the skill set above and beyond directors to be able to take vague orders from executives, create a plan of action, and make things happen. They should be experienced in both the industry and the company. They are the backbone of the organization, are difficult to replace, and almost indispensable.
Executives are either owners or report directly to the owners of the organization. They bear the responsibility for the success as well as the culture of the organization. The commitments that they make to customers, vendors, and employees are given with the backing of the company itself.
Members of the board are important because they are able to take an outside view of the organization and to ask the difficult questions of the executives and VP’s regarding strategy and execution.
The owners of the organization are directly and financially responsible for the success or failure of the organization. If the company makes money they make money and if a company loses money they lose money. The actions of owners may not make sense to employees of the company but in many cases they have a completely different perspective of what is going on.
In a small company the owners, members of the board, and executives are often the same people. Hopefully they have a decent working relationship and use their skills and expertise together to keep the company profitable and headed in a positive direction.
For many years my belief regarding the title of the person depender on the number of direct reports the person had. But over time I’ve come to believe that it’s more in line with how much responsibility for the success of the business that person has. For some managers, directors, or VP’s, that means they have a lot of people reporting to them. For others, they might have few direct reports, but be very influential in the direction of the business.
If there is a key engineer, salesperson, or financial person whose skills are unique and above and beyond everyone else’s, it doesn’t hurt to give that person a title to show how much the company appreciates their contributions overall. Just watch out for giving out too many titles like director, VP, or executive, else they become meaningless.
And when hiring a new person or promoting someone, its always safer to give them a lower level title and then promote them after they prove themselves capable and loyal to the organization.
Finally, this flows into an observation I’ve had over the years. Some people who work for an organization are rapid learners, and their skill sets and abilities increase over time. The managers have to recognize this and reward that person through raises, promotions, and recognition, or else the person will get dissatisfied and leave.
And that sales rep that I trained in the industry then demoted? He went to go work for a competitor and did very well there.
In many role playing games there is the ability to have your character level up. After a certain amount of experience, study, or achievement, the character advances a level. This level advancement confers new abilities such as improved strength, increased dexterity, more wisdom, or a higher level of intelligence. As these games are supposed to be a reflection of life, it made me think, what am i doing to level up in my life?
Many of us live the same day over and over, kind of like the movie the Groundhog Day. Go to work, eat, take the kids to their activities and help them with their homework. Those are not bad things, they are just the things that we all have to do in order to make it through each day and come out at the end of it slightly better financially and one day older.
But what about leveling up in life? Should I be satisfied with the status quo? Should I save the exciting experiences I have for my once a year travel vacation? We all have 24 hours in a day, so how is it that some people can continue to grow and improve while others don’t? How do I level up in my life?
I think it’s important to focus on the activities the help you level up. This is a combination of study, activity, and new, sometimes uncomfortable, experience. It can certainly come serendipitously, but I think that it’s more effective to have some kind of a leveling up plan and then following it.
I haven’t ever been much for setting very detailed goals, I kind of more generally know what I want to do, and make sure I move in that direction, being open to short cuts and detours along the way, and making sure I continue to progress in the direction I want to go.
Sometimes, though, it is necessary to set a goal and plan for doing a specific difficult activity at a specific time in the future. By doing that, the preparation toward doing the activity creates the study and task list necessary to reach that big goal.
For example, I know I want to be better at building electronic things, so I have set a short term goal of getting my technician ham radio license. Longer term, I want to have a basement electronics workshop and the ability to make more complicated stuff. I am planning on taking that ham radio test this month. By doing this, I gain knowledge, experience, and a certification, all which are aspects of leveling up.
A friend of mine is planning on doing well at an upcoming athletic competition. This competition is on a specific date, and since she wants to do really well, she has put together a specific training plan to let her reach her goal. These are the types of things you need to do to get better.
My point is it is not the daily repeating activity that you do that gives you the opportunity to level up in life. It’s specific plans, study, preparation, accomplishments, experience, and certifications that do. Make sure you set aside time to level up in your life.
Information Technology Solution Sales Methods That Work
Information Technology Solution Sales Methods That Work
The Information Technology solution sale is a tricky business. It is definitely a complex sale, with many different decision makers, multiple competitors, and a continuously changing mix of capabilities, cost, and value. Throw in disruptive technology change on a regular basis that can entirely upend business models, and it all leads to one of the most challenging sales environments out there.
In that type of environment, there is the opportunity to create a solution for a customer that can make a significant positive impact on their business. If a customer can upgrade their systems and reap big benefits, most realize it is worth paying a solution provider to get that done for them, especially before their competitors do. That leads to solutions that have decent profit in them, which leads to well-compensated salespeople. The IT Solution Provider industry has some of the best salespeople of any field out there. If that is something you are interested in, here are a few methods that work for me.
First, it’s a technology field! You have to stay up to speed on the technology in your field of concentration, at least to the level of having a basic conversation about the benefits of the products and your company. On top of that, it is important to recognize opportunity, get the customer interested in your company, then communicating that opportunity back to the design engineers. There’s also a split between the type of conversation that leads to profitable business, and the kind that can generate interest.
I always like to start talking to the techies, because if you don’t, then when you talk to the managers who have the budget, the techies always find a way to cut your legs out from under you. To talk to the techies, it helps to have something new and interesting to discuss. That’s why it’s important to learn the basics, the history, and always be learning the new technology.
As a baseline, I define an IT Solution sale as composed of the following, all which require expert engineers:
Assessment of existing system.
Identification of needs and issues
Design of system upgrade or new system installation
Software installation, configuration, deployment.
Professional installation and configuration services
Training and system documentation
This tends to be an expensive investment for the customer, so there has to be a real need for the upgrade. To make profitable sales, you have to address needs. And if you can identify a need that is currently causing pain, even better. Unfortunately, if you are just talking technology that the customer implements themselves, it does not make very much money. In order to afford engineers that can put together a complex solution, the IT sales rep has to design the proposal to be profitable on multiple fronts. From a standpoint of profitability, the different elements usually balance out something like this:
Hardware — 20%
Software — 25%
Implementation — 35%
Training — 5%
Recurring services — 15%
If you are just selling hardware, you have to sell five times as much as if you sell compete solutions! It is possible to do, but it is much better to look for complete solutions to sell. Besides, most customers want the benefits provided by the complete solutions, sometimes they just don’t know it at the start of the engagement. And if you are selling a solution, it is better to focus on items that have not become commoditized, because it is much harder to make money on them.
For example, when personal computers first came out, providers would sell a full solution around the PC’s, and would be able to make large profits on them. Nowadays it is almost not even worth selling the systems, because the profit margins are in the low single digits. It is much better to focus on new or specialized systems. Unfortunately it is really difficult to get in to talk to techies if your initial conversation is about a complete solution. You have to craft a few different entry points. The ones that work for me tend to involve a combination of:
We have a new technology to introduce.
We have a new product using new technology that can make things better for you.
My organizations has recently successfully implemented a new product/technology at a similar type customer.
In this technology area, we have knowledge, expertise, and experience. We have good local engineers proficient in this technology that can help you out.
We are a leading solution provider for the vendor that makes this technology.
Do you have a few minutes to meet to talk about how this technology can help make things better for you and your organization?
Once I get a meeting set up, I like to go through a very specific set of steps to create an opportunity:
Talk about why we are there and that we will discuss the new technology.
Create authority. I have to show the customer that I represent a company that is expert and experienced in the field.
Frame the discussion. I want to bound it to areas that we are good at. Those are broad areas, but there are a lot of things in IT we don’t do, and it is important for the customer to know that up front.
Talk at a high level about the different complete solutions we are really good at. Who knows, they might be interested in some of the other stuff we do.
Give a highlight of the new technology, and how it fits in to the overall scheme of the complete technology solutions we provide.
Then it starts to become complex. During the next ten minutes to two hours of the discussion, I weave in and out of different topics, all with an end goal in mind. The different discussions encompass:
How we take care of our customers with our team.
Why we are good at what we do.
Success stories of technology solutions implementations we have done for our customers.
Finding out the customers needs and desires.
Finding out the size, scope, complexity, budget cycles, project plans, and anything else relevant to how we can assist the customer.
More information on the new technology, and how it may or may not apply to the customer’s operations.
There are many danger zones in this conversation, though. Be careful to avoid these types of actions:
Identifying a customer need, then honing in on it and starting to sell to address the need immediately. This is a way to get a small sale and miss the big opportunity.
Bag-diving. The customer says they are thinking of buying something, and saying we can sell you that, let me give you a quote. Very annoying.
Asking open ended questions that gets the customer talking about something that we completely can’t address. That is a waste of everyone’s time.
Talk about things that are coming in the future. That’s nice, but it does not help the customer to address needs that they have today.
Tell the customer they should do things. They don’t need some outsider coming in and telling them what to do! It is more productive to talk about how you know of other organizations that have had similar issues, and mention some ways they addressed those issues.
Asking about problems or issues. How cheeky! Much better to ask if there are concerns or areas that they are looking to improve in the future.
Acting like a supplicant. You are part of a leading technology organization and you have a multitude of value you can bring to the customer. Act like you have value.
Act like a tourist instead of a salesperson. You are there to represent your company and find out a need that the customer has so you can sell them something that helps them out. If you don’t do that the customer will lose respect for you.
Is is very important to make this a useful back and forth discussion. The customer is usually interested in finding out how new technology has been successfully implemented at similar organizations, and how other organizations solve problems that they might have. They are happy to provide information about their organization and its goals in return. Customers have their areas of expertise, and the salespeople at IT solution vendors have theirs. Remember the advantages you have in the discussion. You work for an organization that:
Stays close to vendors to determine their strategy and the direction of the industry.
Sees lots of different customers who implement things both successfully and unsuccessfully.
Can bring experienced engineering resources to bear who have a track record of success.
Has a track record of success and stands behind the work they do.
Can be blamed if something goes wrong…
The next part is critical, though. You need to elevate the discussion from technology and products to solving a need. There are usually plans, concerns, and issues. In your discussion with the customer, you have to both identify a concern and be able to present enough information about your organization that demonstrates to the customer that you represent the people that can address the need. When talking about those needs, you have to put yourself in the customer’s shoes and figure out what motivates them, and at what level it motivates them. IT people have a few basic things they need to accomplish:
Keep the systems running.
Don’t lose data.
Make sure the company does not have security breaches or outages that are bad publicity.
Above and beyond that, they have other needs. Some are company needs, some are personal:
Improve systems operations and help their company be more competitive.
Update systems that are getting old or are underpowered.
Operate advanced systems at the forefront of the industry.
Do something that improves their resume and can lead to a raise or better job.
Most IT professionals do not put saving money at the top of their list, so it is not usually a fruitful line of discussion.
Assuming you are able to identify a need that your organizations can solve, then it is critically important to show how a complete IT Solution is the way to address the problem. Just selling some hardware or software and having the customer install it themselves is not a complete solution. Addressing the entire need with a proven package that you can bring to them is.
Even if they are interested, however, they still need to have a way to pay for it. That is where budgeting comes in. As one of my previous sales managers told me, a basic criteria for a prospect is that they have money.
Sometimes you get lucky, and you meet with a customer that has already decided they are going to buy something that your company is good at. Then you just have to show them why you all are the best, and why they should get the IT solution from you instead of their existing suppliers. This is where a good salesperson can make a big difference. It is also why repeatedly doing an excellent job for an existing customer is much easier than trying to find new customers.
If there is not an existing budget, then you can try to create budget out of nothing. Unfortunately, trying to create new a new budget allocation from scratch is a multi-year endeavor. A better tack is to find ways that existing budget expenditures can be redirected into a more useful way. If you can help the customer figure out how to get more bang for their buck, especially if it is by doing something shown to work in other places already, then that would be useful.
Sometimes, believe it or not, the customer might not have any needs at all that you can address! That happens occasionally. If so, I recommend you don’t try to force it. Tell them you will stay in touch and continue to bring them information about new solutions and successes that similar customers have had, and tell them you would like a chance to be their supplier of choice for something new that comes up in the future.
Furthermore, if the customer has show interest in any of the solutions that you discussed, but needs to wait until the next budget year, that is OK. Offer to provide them a budgetary price on the system so they have an accurate price to use in their annual budgeting exercise. It is important for your customers to have those types of things in hand, because it shows that they are prepared in case end of year money comes up, or something else unexpected happens.
If you do provide them that budgetary price, you may as well provide a financing quote for it that shows them what it would cost on a monthly basis. Sometimes when looked at on a monthly basis, especially when compared to other expenses, it looks like something that is realistic and desirable.
Hopefully, though, there is a need you can address. In the classification of needs and solutions, it is worth figuring out if you are offering the customer a vitamin or a painkiller. If the need you have identified is for a future project, or it is a planned replacement or upgrade of a system, it is a vitamin. Vitamins are kind of hard to sell, and it takes awhile. If, however, you have the ability to make their pain go away, then rejoice!
There is almost nothing more rewarding than having a substantive discussion with a customer, learning about them, explaining what your company does, then realizing that your organization has the capability to offer them something that will make things immediately better for them.
That’s just the tip of the iceberg regarding IT Solution sales methods, but it should be enough to get an aspiring account manager to get started. There’s lots of other tips, like do your studying and quote preparations at night, and visit customers during the day. But those fall into the category of more general sales, and there are plenty of books out there on that.
If you are able to become an effective IT solution provider sales rep, you will have practically guaranteed employability. It is a difficult industry to become successful in, but there is no other industry that changes so rapidly and provides so many different ways of bringing solutions to your customers.
Customer Focused Technology Solution Stacks from Your IT Solutions Expert
Customer Focused Technology Solution Stacks from Your IT Solutions Expert
It used to be so easy to tell people what we did. Cisco made switches and routers, and that’s what our customers purchased and installed. So we called ourselves a Cisco reseller. Then Cisco added wireless, security, and voice systems, and we called ourselves a Value Added Reseller. Fast forward a decade, and neither of those terms describe what we do anymore.
Being a reseller or even a value added reseller is no longer good enough. Over the last decade, our engineers have been designing, building, installing, testing, tweaking, improving, and supporting complex systems of hardware and software into well-tuned systems.
A better description of Adcap Network Systems is a customer focused Information Technology Solutions Provider. The systems we deploy at our customers we call Adcap Technology Solution Stacks. These combinations of hardware and software address specific operations needs of our customers.
The common thread running through these stacks is that they address the real technology operational needs of our customers. Adcap places the customer requirements first in our development. Our account team develops a close working relationship with our customers and brings their issues, concerns, and desires back to our engineers.
Our engineers are really good at designing, deploying, and supporting these systems. We are always talking to customers about what they need, talking to Cisco Systems and our other vendors about the products they are developing, and staying up to date on how the industry, technology, and products are changing. We modify our Technology Solution Stacks over time, and create new ones as well.
Adcap Network Systems Technology Solution Stacks:
Complete Data Network Systems
Enterprise Communications Systems
Video Education Systems
What’s so good about these stacks? We design them from the beginning to be the type of system we would want our best friend or family member to buy. They work well and address real operational needs. They can be upgraded or expanded in the future as requirements change. They are designed to have minimal points of failure. There is no extra complexity involved, and are installed based on the manufacturer’s recommendations, so they can be supported by other people.
We can’t just ship these stacks to you and have them work. They need to be deployed and integrated into a currently working environment. If we replace your phone system, your users should be able to know the system is going to work and have everything set up before they switch over to the new system. If we set up a new server, storage, and disaster recovery system, it needs to be tested and work well before we are done. Our engineers and project managers have the experience to do this properly.
Adcap Professional Services Elements of the Technology Solution Stacks:
Support from Adcap PremierNet
Part of Adcap’s advantage is our ability to rapidly bring current technology into operation. For example, we started deploying business VoIP phone systems using the money saving SIP technology at our customers in 2006. It requires the ability to rapidly learn, test new configurations in our development lab, and figure out the benefits and drawbacks to the new technology, but in most cases it is well worth doing.
Product Elements of the Technology Solution Stacks:
Servers, storage, hypervisors, operating systems from multiple manufacturers.
Cisco voice and video infrastructure hardware, software, and endpoints.
Circuits for Internet, voice, private wide area networks.
Cloud Application Services for voice, video, network, security, computing.
Many times our customers need just a portion of the stack, and we are of course able to address that requirement as well. Sometimes another organization has done an incomplete deployment of a similar system. We are happy to help bring it up to the level that an Adcap system has.
Adcap focuses on specific sizes and types of customers. Our Technology Solution Stacks are optimized for mid-size organizations. We have many customers of the following types:
Higher Education Institutions
State and Local Government
K-12 School Districts
Manufacturing and Distribution
Professional Service Organizations
The Adcap Technology Solution Stacks are created, developed, tested and optimized by our experienced and industry leading engineers and focused on addressing real customer needs. Our account services teams work closely with our customers to identify those needs and show how these solutions will make substantial and noticeable operational improvements.
So don’t call us a reseller or a Value Added Reseller. Adcap Network Systems is an Information Technology Solutions Provider. We design, deploy, and support Technology Solutions Stacks that enable our customers to operate more effectively and reliably. This is our true value.
Adcap Is No Longer a Reseller of Information Technology Products
Adcap Is No Longer a Reseller of Information Technology Products
Guest post from Christine Versluis:
Adcap is no longer a reseller of IT Products!
Why do I say that?
Let me start with an analogy. When I was in the elevator industry, I worked with a lot of construction companies. Are construction companies called resellers? They could be. They buy pre-fabricated and pre-formed metal, glass, wood, stone, and plastic components, mark them up, and resell them to their customer.
Of course they install the products they resell and they produce a visible end product. But they define themselves by what they produce, not by their activity of reselling raw materials. They are construction companies.
If a construction company resold their materials without assessment, design, installation, project management, and support, they would be known as a building supply company. The quality of the pre-fabricated materials in the construction industry is an important part of the building process, and the materials account for the majority of the cost in the project, but construction companies are not resellers.
If Adcap just resold information technology systems without assessment, design, project management, installation or support, we would be just a reseller. We would not even be a very good reseller. We don’t have an online catalog, we don’t stock products, and we don’t provide tracking information for every shipment.
CDW is an excellent reseller, a leader in the industry. So are Insight, Tiger Direct, and NewEgg. Amazon is an excellent reseller. Anyone who wants to buy information technology hardware can go to the websites of those companies, compare prices, place an order with a credit card, and have the product shipped with next day service. Adcap does not do that, because that is not where our value is.
We also don’t buy, stock, and resell used information technology equipment. There are companies that do that as well, and if people want to buy out of date equipment and put it in place themselves, there are many companies ready and willing to supply them the used gear.
We are not even a Value Added Reseller, even though that is how the Information Technology industry likes to describe companies that resell and install servers and networks. Adcap is not defined by the products we resell, but by the complete technology based solutions we integrate into our customers’ existing working environment.
Adcap Network Systems is a customer focused, complex information technology solutions company. The systems that we design, deploy, and support provide solutions to problems that our customers need to address. Our value is in our engineering and project management. Our reputation is based on our expertise and the success of our deployments. Our customers are happy when we solve their problems in a rapid, pleasant, and cost effective manner.
Adcap Network Systems is an Information Technology Solutions Provider.
We work closely with our customers and vendors, and over time we have created well-honed combinations of products and services that we integrate and support into our customers existing operations. We call these systems Adcap Technology Solution Stacks (see link for more complete explanation).
Complete Data Network Systems
Enterprise Communications Systems
Video Education Systems
So to repeat, Adcap is no longer a reseller of Information Technology Products.
Adcap Network Systems is a customer focused, Information Technology Solutions Provider. We design, deploy, and support technology systems that enable our customers to operate more effectively and reliably. A reseller just can’t do what we do.
The metric system, on the face of it, seems sensible. When you want to go to the next bigger unit of measurement, just increase things by a factor of 10. It is apparently so sensible, students always ask the question, why does America not convert to metric? The teachers and professors shake their heads at the stupidity of the American common man, and move on the next lesson.
It is the metric system that’s stupid. Imperial measurement units are for people that need to get things done. It’s all because of the basic number that the systems are built around. Pick up a ruler, and that’s all you need to use to demonstrate how superior the American system is. Quick, in both systems, tell me what the major unity of measurement is. American — 1 foot, or 12 inches. Metric — 1 meter, or 100 centimeters. As you can see, that’s already a problem. What student is going to lug around a 1 meter ruler? So we have to make the metric basic unit of measurement a decimeter.
But let’s look into this a little more closely. Something that people do all the time with measurements is to subdivide them. So, divide the basic unit of measurement by, 2,3,4, and see what you get.
Metric 10/2 = 5 10/3 = 3.333333 10/4 = 2.5
American 12/2 = 6 12/3 = 4 12/4 = 3
Hmmm…that’s interesting. American units provide us even numbers when divided. Every American tradesman knows this, of course. Ask an American carpenter to switch to metric and he’ll laugh in your face.
How about big units of measurement? Do you remember how many meters in a kilometer? That’s easy, 1000! How many feet in a mile? 5280, which is a little tougher. Where the heck did 5280 feet in a mile come from, that’s a tough number to remember. Well, let’s get into some factoring action, which tells us what numbers the big number can be easily divided by:
Looks like you can evenly subdivide a mile in a whole bunch of different ways. Need to know how many feet in 1/11 of a mile? Bam — it’s 480. Beat that, metric system.
OK, enough with the land, how about when you are cruising out on the open ocean. When I was on a submarine, we had to navigate all over the world. For that we use latitude and longitude. The people who thought up latitude and longitude were smart. There are 360 degrees of longitude — that’s a beautiful number!
360 is 6 times 60. And 60 is five times better than 12. Twelve is the basic unit of measurements for the Imperial system. But if you use 60, you can evenly divide it by 2,3,4 and 5! So, using 360 degrees for longitude was just as smart as using 24 hours in a day — actually 15 times smarter, but who’s counting?
Back to open ocean navigating. When using smaller measurements than the full degree of latitude, it gets subdivided into 60 minutes. One minute of latitude is also known as a nautical mile. That makes it easier to navigate using nautical charts, which show latitude and longitude. Speed on water is measured in knots, which nautical miles per hour.
At the equator, the length of one degree of latitude and longitude are the same, 69 miles. On American submarines, we need to have something more easily divisible than a nautical mile when judging tactical distances, so we use yards. I would expect Russian and Chinese submarines use meters. Even there, we have an advantage, because:
Metric 1 nautical mile = 1852 meters
American 1 nautical mile = 2024 yards
It’s a heck of a lot easier to approximate a nautical mile as 2000 yards, with a 1% error, than 2000 meters, with a 13% error.
The Metric system does have an advantage in the area of power. A Watt is a heck of a lot more usable than horsepower. So, which does America use? Both, of course! Our car and boat engines are measured in horsepower, and our electricity is measured in Watts. Who says Americans can’t be sensible?
So there you go. Just like time is measured in 24 hours, subdivided into 60 minutes and seconds, and there are 360 degrees on a circle, the American system of measurement is based on 12, which is a beautiful number. The metric system is based around the number 10, which is the number of toes on your feet, and is not a beautiful number. In fact, the whole world would be better off if we switched to a base 12 numbering system, instead of base 10. But that is a discussion for a different day.
Nexenta High Availability Cluster on Cisco UCS — Failover Demonstration
Nexenta High Availability Cluster on Cisco UCS — Failover Demonstration
One of the lessons I learned while operating a nuclear power plant in the middle of the Pacific Ocean, underwater, was that it is important to have systems that don’t have a single point of failure. I’m sure many people in Information Technology feel the same way, and given their choice, want their systems to be highly available.
The NexentaStor HA Cluster plugin can be a bit of work to set up (see previous post), but it is definitely worth the effort. I had the good fortune to go through some validation testing with some talented Nexenta engineers using the system that I set up in the Adcap lab, so I got to learn the ins and outs of the High Availability setup. The clustered high availability demo system is circled in green in the picture below.
The controllers are Cisco C240 servers (on loan from Intel) and the JBOD’s are from DataOn Storage. I discussed the build and initial setup in my previous post Nexenta High Availability ZFS Storage Systems Using Cisco and DataOn. The Cisco UCS B series blade server system is circled in blue, and the Cisco Nexus 5548, 5596, and 2000 are circled in orange — they are used in the IOMeter testing of failover.
I made some improvements since the first build of the system. The 10GbE NIC’s are Intel X520’s and are operating at full speed with jumbo frames, all the hard drive bays are filled, and the sTec SSD’s and ZeusRAM are in full use.
This is one configuration of the Adcap SwiftStor product. The validation testing involved stress tests, equipment tests, and failover tests for any number of different hardware and software faults. After the validation was done, I rebuilt the cluster then took a few videos and screenshots demonstrating failover of CIFS, NFS, and iSCSI.
The HA Cluster shares volumes between two controllers, but the volumes have to be created first. I found it easiest to create the volumes all on one of the controllers first, then share them out from there. If the cluster is removed, the volumes revert back to the controller on which they were active at the time, and are exported. You would just have to go to Import Volume to get them back up and running with no loss of data.
I set up two volumes. One with twenty 3 TB drives, set up into four RAIDZ1 groups, called Large_Volumes. The other a set of ten Mirrors, with two 400GB SSD’s for L2ARC (read cache) and two 8GB ZeusRAM SSD’s for Synchronous Log cache.
Next we establish the cluster. Even through it can all be done with the GUI, I had better success establishing the cluster using the command line. The command line provides a few more helpful hints and seems to have a better flow. I found that when I used the GUI I would go back and forth between the two controllers, which causes issues. I recommend using the command line to create the cluster, if only because it forces you to set up everything from one controller.
As shown below, the Mirrors are shared out on a Virtual IP address (VIP) of 10.124.12.204, and is currently managed by the C240-HA-A controller. The Large_Volumes are shared on VIP 10.124.12.214, and currently managed by the C240-HA-B controller. Heartbeats are done on both the network and the hard drive side.
This is an Active-Active configuration where some of the volumes are shared out by one controller, and some by the other. This permits full use of the processors, memory, SAS channels, and network of both controllers, while providing High Availability in case of a hardware or software issue.
After creating the volumes and sharing them out using the HA Cluster feature, and using different IP addresses for each volume, everything else that is created is done as part of the HA system, which means the configuration changes made on one controller are also updated on the other.
Each volume is then shared out using both CIFS and NFS. A CIFS password was set, and NFS version changed to version 3 instead of the default of 4. In an enterprise setup, the CIFS would be tied into Active Directory, and permissions set up properly on both CIFS and NFS. For the demo setup, I left if wide open.
By clicking on the individual volumes, the mount point for the CIFS and NFS shared are shown. CIFS is easy to mount from a Windows and Mac client. NFS is a pain in the butt with Windows and Mac, and I did not feel like messing around with it too much, so I set up an Ubuntu Linux Virtual Machine to test NFS.
Accessing the CIFS from Windows was easiest by just typing in the IP address in the Windows preferred \10.124.12.204 format, entering the username of SMB and the password typed in earlier. Both the shares were available on both the IP addresses, as shown below.
So, on to the failover testing. This is a video that shows the failover of the cluster and how CIFS stays up and running during the transition of the volume from one controller to the other. During the video I use the manual failover feature of the Nexenta HA control. This has the same effect as if a heartbeat detection of a failure causes a transition of control.
There is a finite amount of time for the transfer, as can be seen by the network pings in the video. The Windows machines maintains the mapping and access to the shares during the transition, but the in-progress file transfer has to be restarted.
So, I did the same thing with a linux box using NFS for a file transfer. The problem was, I set the Linux box up on the Cisco UCS, and connected it to the storage network at 10Gbps. No matter how big of a file I transferred, it would take less than 10 seconds. At some point maybe I can find a 30GB file and transfer that, or set up Linux laptop and transfer over wireless so it slows things down. Until then, take my word for it that NFS is resilient.
For a more practical demonstration of the resilience of NFS, I set up three IOMeter virtual test servers on the Cisco UCS. I set up their storage as 200GB virtual hard drives using the NFS share on the Nexenta HA cluster. Then I ran a 120 second performance test, and failed the volume over from one controller to the other in the middle of the test.
This is a picture of the results. Obviously the IOPS in the graph are an average, because there was no storage activity while the VIP was unreachable. It is nice to see that things come back up instead of the Virtual Machine puking out.
This is a video demonstrating how NFS shares stay up during the failover. It shows the Nexenta HA interface, the IOMeter interface, and a continuous ping during the testing.
Then I did the same thing with iSCSI. This was a little trickier, because when I did the scan of the iSCSI from the VMware server, it found both iSCSI targets on the Nexenta box, and the iSCSI target for the Mirros was found on the IP address for the Large Volumes, which is not what I wanted. I found that the solution for this is to set up specific mapping of the iSCSI targets to volumes on the Nexenta box.
I had first set up a LUN, or ZVol as it is known, for the Performance Mirrors
The iSCSI targets had been created when the Virtual IP addresses were created during the Cluster setup, so they were already there.
The Cluster setup had also created Target Portal Groups, providing a way to separate out the volumes and IP addresses within the iSCSI realm.
However, it was necessary for me to create a Target group that put the iSCSI target into a specific group.
At that point I could map a specific LUN to a specific iSCSI initiator.
After this, when I rescanned the iSCSI targets from the VMware virtual machine, the correct iSCSI name was matched to the correct LUN and IP address. Even though this seems a little bit complex, this system is necessary when there are multiple LUN’s, initiators, and targets.
I set up three new IOMeter virtual machines with their storage defined on the LUN’s through the iSCSI connection. Then I did the same exact test that I had done with NFS using the iSCSI connection.
This is a picture of the results. Just like on NFS, it shows that there is a loss of connectivity in the middle of the test, and then it recovers.
This is a video. It is a little longer, because I also show a little bit of the iSCSI setup on the VMware setup.
This is a really solid implementation of High Availability Clustering. The setup of the system is straightforward, the tools to use it are powerful, and the failover works well. By having each controller be the primary manager for a set of volumes a true Active-Active configuration is enabled. Both controllers are able to use the capabilities of the processors, memory, SAS controllers, and network connections to full effect.
I have four more failover videos for those of you who would like to see the actual time it takes to failover in real world cases where equipment fails or is disconnected.
In the first video I use the Cisco Integrated Management Controller to do a hard reset of the active controller.
The power cycle is done while running a continuous ping, monitoring the Nexenta High Availability Management GUI, and running both and iSCSI and NFS test using IOMeter.
In the second video I go to the test lab and pull one of the SAS cables from the active controller. The backup controller for the volume figures out that there is a problem and takes control of the cluster. The previously active controller does a controlled reboot.
The SAS cable pull is done while running a continuous ping, monitoring the Nexenta High Availability Management GUI, and running both and iSCSI and NFS test using IOMeter.
In the third video I pull first one then the other of the 10GbE network cables from the active controller. One of the cables is jammed in tight, so I have a hard time pulling it out. But after I do, the backup controller for the volume figures out that there is a problem and takes control of the cluster. The previously active controller just hangs out until the network is restored.
The network cable pull is done while running a continuous ping, monitoring the Nexenta High Availability Management GUI, and running both and iSCSI and NFS test using IOMeter.
In the fourth video I pull both of the power cables from the active controller. The backup controller for the volume figures out that there is a problem and takes control of the cluster. The previously active controller does a controlled reboot.
The power cable pull is done while running a continuous ping, monitoring the Nexenta High Availability Management GUI, and running both and iSCSI and NFS test using IOMeter. Unfortunately I was a little fast on pulling the power so the results just show what happens after failover.