Podcast Episode 9: 5 Learnings from the Azure Expert Managed Services Certification Audit

Ira Bell: Today, Casey Ryan, Director of Consulting Operations talks with Ryan McDonald, VP of Managed Services, and Jon Clemenson, Operations Manager, all from 10th Magnitude. In this discussion, they talk about Azure’s Expert Managed Services Certification Azure’s Expert Managed Services Certification, which is a prestigious designation given by Microsoft. Those who mature beyond technical agility and organizational agility to become digital disruptors are leading the charge in this type of digital transformation. Now onto the podcast.

Casey Ryan: Welcome, my name is Casey Ryan, Director of Consulting Operations at 10th Magnitude. I’m here with Ryan McDonald and Jon Clemenson. I’ll let them introduce themselves. Ryan?

Ryan McDonald: Hey Casey, yeah this is Ryan McDonald, VP of Managed Services with 10th Magnitude. I’ve been with 10th Magnitude for a little bit over 2 years. Responsible for our Managed Services organization. And I’ve been working in Azure for about the last 5 years.

Casey Ryan: Cool. Jon?

Jon Clemenson: Hey Casey. Jon Clemenson, Operations Manager at 10th Magnitude. Help run the day to day engineering team on our support side. Been with 10th Magnitude for coming up on 2 years now.

Casey Ryan: Awesome. And, ah again Casey Ryan, Director of Consulting Operations at 10th. I’m on the Professional Service side. I have been here about 2 years. Today, we’re going to be talking at length about this Agility Quadrant concept, where we’ve been talking at length about the Agility Quadrant concept as it relates to a customer’s journey towards digital transformation. One of the components of which 10th Magnitude brings to the table is management of Cloud-based workloads, under 10th Magnitude’s Managed Service practice. Recently we were awarded the title of Azure Expert Managed Service Provider by Microsoft. It’s a huge accomplishment for us. Ryan, I was wondering if you can tell us a little about that Azure Expert MSP program.

Ryan McDonald: Sure. It was announced in July of 2018 at Inspire, which is Microsoft’s partner conference. In the way that they’ve structured this is really to have it be focused on their top MSP’s who have passed a rigorous on site audit to prove their kind of IT service management competencies. And really what you have to demonstrate as part of the audit, is that you can deliver consistent, you know, repeatable managed services on Azure.

Ryan McDonald: There’s an application process to even be accepted into the program. You have to be like a Gold Level Partner, for example, you have to be a CSP Cloud Solution Provider with Microsoft. You’ve got to have some offers in some of their market places. There’s requirements on certified staff and having customer references. So there’s a pretty high bar just to be entered in and then you’ve got to go through the certification process.

Casey Ryan: Cool. Today we’re going to talk about the top 5 learnings that 10th Magnitude had while it pursued this Azure Expert MSP audit. And going through that process. Ah, in a spoiler alert – we got it. So we found out, I think late last year of 2018. But so we’re speaking from some good experience here. It was a really cool process in the top 5 that we’re going to share with you, rather than run through the whole thing, is really just the big learnings we got out of it and kind of how we, how we went through the audit process. Jon, do you want to … what I thought would be cool is if we each gave just kind of our backgrounds. Because we all have different backgrounds, in audits and kind of frameworks and things like that, from our experience. Jon, why don’t we start with you. Tell us a little bit about your background.

Jon Clemenson: Yeah definitely. When I first heard about the Azure Expert MSP audit, I had two things kind of run through my mind. One, I was really excited about it because I knew, you know, that the outcome was going to be great. But at the same time that was kind of tinged with a healthy bit of fear. Which, I think is good when you go through any audit, to kind of have that … a little bit of you know, nervousness going through it so that kind of puts the right focus on it. You know over the last 10 years I’ve been though a dozen or so audits, centering around the NIST framework and network accreditation. So, you know all the nuts and bolts of those audits are a little bit different than this one. I think there was definitely some high level overlap, you know, that helped us succeed in MSP audit.

Casey Ryan: Nice. I found out by talking to Ryan about the MSP Audit, never heard of it. Coming from the professional services side I don’t think that’s too unusual and just how new it was to Microsoft, even. But quickly jumped in and wanted to help out. I love audits, I don’t know why. I’ve just always been entertained by walking through them. They’re great learning experiences for a company. My background, I’ve previously managed SaaS infrastructure and SaaS operations and things like that so, besides going through SOC 2, Type 1 and Type 2 attestations and then holding accountable, our vendors for their SOC 1’s. Things like that. I’ve also been through a number of customer audits of us. I’ve gone through a few of those here, at 10th Magnitude since joining, but this was by far the most extensive one and formal I think I’ve been through.

Ryan McDonald: Yeah Casey, my background is really similar to yours, with a SaaS background and doing the kind of typical you know, yearly SOC 2’s that a lot of SaaS providers do so, again yeah, I mean I think for me you know, this was very aligned to the high level pieces that you would see in a standard audit, whether it’s DIST or SOC 2. So, it was exciting.

Casey Ryan: Awesome. Let’s just jump into the top 5 learnings from the audit. So, Ryan you want to kick us off with the first one, or number 5?

Ryan McDonald: Yeah, number 5 was really the comprehensiveness of the audit. You know, this was a big audit for us and after we passed in December, as we were you know, reflecting back on that, we went and looked at all of the team members that contributed to the success of the audit and when we counted it up, we had 28 total team members. Almost, you know, a quarter of our company that participated in preparing for the audit. The audit was you know, 2 days on site and you know we had 13 presenters. Individual presenters that presented evidence in front of the audit. So, it was definitely comprehensive.

Casey Ryan: Yeah, just to add to that, I think it … one of the things we found was it wasn’t solely focused on services provided by managed services. So 10th Magnitude has traditionally, kind of historically been a professional services company. Really Ryan has started a managed service area, I think 2 plus years ago, and so we’ve been really able to develop and mature that area of the business. But what the audit focused on was really the full life cycle of cloud services that 10th Magnitude provide so everything from the initial, like how do you assess, how do you help the customer design, how do you help them realize opportunity as moving to Azure. The build and migrate portions, the optimization portions, where we really help them take advantage of Azure services and then the ongoing management and ensuring that it’s up, operational, functioning at a high quality, in that they can feel good about their ongoing workload support in Azure.

Jon Clemenson: Yeah definitely, kind of speaking to the comprehensiveness, there were really 3 sections. One that Ryan talked about earlier, with the prerequisites that kind of proved we got over that initial hurdle. And then followed up by the pre-audit which was a pretty brutal 8 hour phone screen. But it was great for us because it gave us a chance to sit down with an auditor, a different auditor that would do the onsite audit. But, you know to sit down with them and pick their brain and really make sure we understood the audit requirements and were you know, fully prepared. So, fast forward several weeks after the pre-audit, you know, to the 2 day onsite audit where we all came together and just the logistics of it alone were incredible.

Jon Clemenson: So kind of rolling from that into our number 4 learning, which was prep time for the audit. It really was a kind of a job in and of itself. You know the 3 of us, Ryan, Casey and myself kind of drove the bus to co-ordinate the, you know the 28 total participants and just working through the requirements and the, you know, the 40 page documentation and collecting the documentation and collecting our processes and piecing those together to make sure that we were telling the right story for the auditor. And that’s another piece of it as well that … it’s not, you’re not just telling a story in front of an auditor, you know they dive deep. They want to see the tools, they want to see the technology, they want to walk through, you know, basically each and every process that we use to provide our service to our customers.

Casey Ryan: Yeah, I think ah some of … one of the other things I think we did that during this prep time, our number 4, was we did dry runs. I thought that was really a great way to just test out and tweak and discover areas where we maybe were light. Those were internal dry runs, ah mainly between the 3 of us and then we’d bring in others to just confirm that they have the evidence, they knew what to show, they knew what they were speaking to. Things like that. And then another part of what I remember you were doing during that process Jon, was you were talking to and validating some of our assumptions with the gentleman who ran the pre-audit. I thought he was a great … I thought that pre-audit day was amazing. We learned a ton. It was a little scary but I thought he became a great resource in kind of prepping for what the audit would be. And just validating some assumptions we would have to make during that kind of prep time process.

Casey Ryan: Ultimately there were 64 controls across all the Microsoft services, assessment and design. Security and governance, SLA’s, customer SAT and continual improvement. We had to run through and find customer examples. We had to find evidence of those customer examples and then be … and then we had to find the people that ran them in order to be able to speak to those examples and evidence. One of the kind of the missteps we made in the beginning was, we had a lot of people from our Professional Services org, so they’re billable people. And so we’re trying optimize their time and have them record or pre-record videos of them walking through guides. We tried to validate if that was a good … if that was an acceptable way to provide evidence and we were told it was not. To what Jon said earlier, the auditors wanted to be able to interact with the people. They wanted to be able to ask questions, they wanted to be able to steer it down a path. And I … honestly, I thought that was a better outcome.

Casey Ryan: It ended up being something we were able to maneuver and get people aligned to do all the presentations.

Ryan McDonald: Yeah, Casey, one of the things that stood out for me and I know it was an area of focus for us as we prepped is… the audit had several different categories of requirements and so it was category zero through four or three. And the category zero items you had to be able to provide evidence for, on the day, or you failed the audit. You know there were some strict guidelines around hey if you had a category one item and maybe you didn’t have all of the evidence that auditor was looking for, that you could provide that within you know, 48 hours or something. But for us, you know prepping on those category zero items, which were really core to the essence of the audit was certainly a big takeaway for me.

Casey Ryan: Cool. Moving into our number 3 learning. Largely the audit and everything about it was based on ITIL and ISO processes. Ryan, do you want to give us some more background on what you saw there?

Ryan McDonald: Yeah definitely. So, you know when we looked at the controls that we were being audited against, I mean you could tell that Microsoft, to their credit, really looked at the best practices across ITIL and ISO and built the audit around those so, you know it was relatively straight forward I think for all of us, with our backgrounds in audits previously, but it was good to see that Microsoft was taking the best of kind of IT service management and ISO and incorporating it into the audit.

Jon Clemenson: Along with that, we really focused a lot of time on one of the larger portions of the audit, not surprisingly. Which was cloud operations and service management. So, that’s kind of my wheelhouse and it covered several sub sections, operational support and service desk procedures and how do we escalate and how do we collect metrics and what are those metrics. And how do we train our support engineers and you know, what is the incident management process look like in service and problem and incident management tools that we and how do we use them and what is… you know what are those processes. Or how do those processes flow. So, this is again this was an area that we poured a lot of time and effort into for the audit. Because we really wanted to make sure that we were both telling and demonstrating to the auditor the story that we you know, provide to our customers.

Jon Clemenson: So we wanted to show it from the engineers side so we dove into the tools and we brought up an incident, you know, a priority one incident, that we worked for one of our customers. And we stepped through that process from the engineers side, you know what do we do to make sure that we’re giving consistent service you know, and then we showed the follow-up, we kind of closed the loop on that priority one and showed you know, what the customer experience was you know, what tools does the customer have to interact with us. And how do they ensure that they are getting top tier of service. So, that was really important to us to show during the audit.

Casey Ryan: Yeah, just to wrap that number 3 up, I think to stress that these… the processes are not… don’t have to be overly complex right? The best part about ITIL and ISO is just that they’re documented. You start off simple and then you make them available, you train people on them, you use them, you track the evidence and then you improve over time. And naturally they’ll advance as you use them, you start to look at things and say that doesn’t work, we need to improve it. But at least you have this base of, base of written knowledge to move from. So I thought this was a huge… again, we did a ton of work here, I thought it was a huge maturity step for us. One of the biggest realizations, or biggest aha’s at least for me was this cloud management platform requirement. Ryan, I know you have a lot to say about that, so why don’t you take that one.

Ryan McDonald: Yeah, that was number 2 on our list and so you know, Cloud Management Platform or CMP is you know, there’s a number of tools out there that would fit into this category and you know, broadly speaking they help you manage cloud and a lot of them are focused on you know, multi cloud management. And you know, Microsoft had a requirement for Cloud Management Platform in this audit. So I think they did that to really align to Gartner’s view for CMP’s which is kind of detailed in the, in their Magic quadrant for public cloud infrastructure managed services providers. In that they lay out that hyper scale you know, managed services provider need to have a CMP in order to effectively you know, manage their customers and scale. So for us you know we were using a lot of the platform native tools initially and as we looked at the audit, realizing that we had to have a Cloud Management Platform was something new for us and you know, fast forward to the ending you know, we’ve really found a lot of benefits in having the Cloud Management Platform in place.

Jon Clemenson: Yeah, that’s a good point being our first step out of platform native tools. For us you know, on the engineering team it was a change to our process. And ah it really is worked out really well because it gives us a way to interact with our Azure subscriptions at a broader scale. So, for example we often get requests from customers, you know that may need inventory reports that are kind of have really special requirements that are specific to their environment. And the CMP gives us the ability to kind of quickly wrangle that information and give it to our customers in a format that they’re used to working with. So that was a key takeaway for us.

Ryan McDonald: Yeah, the other thing that you know as a pure play Azure managed services provider, we don’t manage other clouds or on premise infrastructure but the CMP you know, gives us the ability to bring data from other clouds into it so if one of our customers wants to bring that data in and see all their data in kind of that single view, they have that while we, you know, we use the CMP to manage their Azure infrastructure.

Casey Ryan: That brings us to our number one learning through this whole process, which was the focus on continuous improvement. Realistically, across this whole thing with Microsoft and the auditors really asked us to do, was show that we weren’t just using what we had but we were improving it and evolving it to meet our customer needs, to give good service and to make our lives easier and to be able to scale. So, the big thing was having to show that throughout every piece of the evidence that we had done. And I think that, that tone like taking that tone, rather than what can be done as a defensive, like you can come across as defensive in audits. Taking the tone of continuous improvement and learning, which is really what the opportunity of an audit is. Really made it tremendously easier, made the relationship with the auditor, the relationship with Microsoft just a much better place to conduct and take part of the audit as we went through it.

Casey Ryan: I think we came out of it… that approach, I think we came out of it with a ton of maturity, a ton of learning, a really good vision for where we wanted go. And I think that showed in a lot of what kind of the data we collected, the analysis we did. Having things like capacity management, automation, records management and just a good cultural focus on this continuous improvement.

Ryan McDonald: Yeah, the only thing I’ll add to that Casey was you know, our auditor was great and she gave us some you know, suggestions or opportunities for improvement as she saw our processes. Because she had a deep background in audit and looking at companies and you know, looking for excellence in service processes. So that was great to actually get recommendations that were valid for us to consider for future enhancement.

Casey Ryan: Awesome, that wraps up our top 5 learnings through the Azure Expert Managed Service Provider audit and ah, we’ll just… what we’re going to do is just wrap up. I wanted to give a chance for Ryan to talk about why, why would someone use managed services for a cloud, why would someone look at using an Azure Expert MSP provider as opposed to doing it themselves. Ryan?

Ryan McDonald: Yeah, thanks Casey. So you know, look I mean if… as you can see from everything we’ve talked about here, the Azure Expert MSP program is really an extensive program and if you are looking at management of cloud and realize you don’t have the skill sets then you know, looking at an Azure Expert MSP is a great place to start. You know, obviously they’re going to bring deep Azure expertise you know, that’s proven by the audit you know, 10th Magnitude I think is a little bit special in that we are 100% focused on Azure and that’s really all we do. We also differentiate ourselves with USA based support.

Ryan McDonald: The other thing that I think in terms of management when looking at possible alternatives is really having someone take a proactive approach and making sure they’re looking for optimizations, way to improve your infrastructure, way to save money. But ultimately the value for our customers is really around piece of mind. Knowing that they have experts there 24 by 7 to support them. But probably more importantly is that they get to use their precious internal resources to focus on you know, revenue generating activities that are usually focused around applications and data that they have created or that they manage and so, you know, their expertise focused on application and data is where they really see a lot of benefit, while they let us manage the infrastructure.

Casey Ryan: Well thank you very much for you time Ryan and Jon. It’s been a pleasure to work with you and I hope we can have you as guests on our podcasts again soon.

Ryan McDonald: Thanks Casey.

Jon Clemenson: Thanks Casey.

Ira Bell: Thanks for listening to the Art of Digital Disruption. At 10th Magnitude we’re proud to create the path for organizations to stay competitive and disrupt their industries. For more information on how to accelerate your cloud journey and get a clean bill of health as you continue your digital transformation into the cloud, visit www.10thmagnitude.com/azure-cloud-health-check to get started on your Azure Cloud Health Check Azure Cloud Health Check today.

By |2019-10-28T16:47:06+00:00April 25th, 2019|

Leave A Comment