- Azure Sentinel—A real-world example - Tue, Oct 12 2021
- Deploying Windows Hello for Business - Wed, Aug 4 2021
- Azure Purview: Data governance for on-premises, multicloud, and SaaS data - Wed, Feb 17 2021
These interviews have become a bit of a tradition, with Paul talking to Ben here and here over the years.
Ben Armstrong and Paul Schnackenburg
DevOps
PS: I saw your session on containers in Windows Server yesterday, it was great.
BA: I did have to smile. I always look at the feedback, and I pay a lot of attention to it, so I did my introduction to containers session. At the beginning of a session, I ask the room: How many developers? How many IT Pro people? Because there’s a different message for each of them when it comes to containers.
I suspected that this would be the case, it’s a very ops crowd here. I asked: How many developers? I don’t see any hands. I ask, How many ops? Like a sea of hands. I’m going pivot this heavily towards the ops crowd. One person in the feedback was said I was very condescending towards developers. Well, yes I was a little because you all told me there were no developers here, and I was trying to build rapport with all the ops people in the room. Because I don’t know if you noticed, but ops people are condescending towards developers.
PS: Subconsciously, I think a lot of ops guys are getting a little bit worried because of the DevOps movement. When you came to Ignite five years ago we would have 60-70% IT Pro oriented sessions, now it's 60%+ developer sessions.
BA: Yes. The frank reality of it is that when you look at the move towards DevOps, it’s certainly easier for the developers to start doing ops than it is for the ops to start doing dev.
Windows Server 2016
PS: Let's talk about Windows Server 2016, which has now been released. When you were here in Australia last time it was in Tech Preview (TP).
BA: I believe it was TP 3 back then.
PS: I think so. Do you have any large deployments you can talk about?
BA: Yes, we do have a number of large deployments. I don’t know if we have any case studies that we can talk about yet.
Now that I think about this, last week I got an e-mail report from the team that’s been working on on-site customer deployment with shielded virtual machines, so I know of at least three very large companies in the finance and government sectors who have deployed shielded virtual machines and were successful with it.
PS: Cool.
BA: I can’t name them for legal reasons, sorry.
Shielded VMs
PS: So when will we see shielded virtual machines in Azure?
BA: That’s something we really want to do, and it’s something we’re working hard towards. I can never commit to timelines with it, though. The fun, and kind of the fascinating thing, is that we build these core technologies and then we have to see how it is adopted. It seems like 30% of the challenge is getting the technology working for a standard customer, and then 70% of the challenge is trying to figure out how to make this work at a scale of millions in a reliable, repeatable way. We have a number of features where we’re working actively with Azure to figure out how can we do this.
There are two things that I find really fascinating about working to get these features out to Azure. The first one is the sorts of challenges that you hit at the millions-of-computer scale that you won’t hit anywhere else. The second thing is management of the features, on a daily in-the-trenches basis, as an engineering person, this is super frustrating. But, at the end of the day, when I take a step back and think about what’s right for the business, I’m really thankful that they do this, and that is the leadership of Azure is really hard core about the fundamentals.
They're really hard core about performance. They care about uptime. They care about the customer promises. And so, if we can’t go in there and explain how a feature is going to be reliable at large scale, if we don't have the data to show that it's not going to impact reliability in any way at scale of millions of hosts, their answer is: no, we’re doing it yet. Go do more homework, please. So, there are number of features in Hyper-V that are in that category, unfortunately. I would love to be able to say, yes, it’s coming out in the next six months, but I can’t.
Large deployments of Hyper-V
PS: Well that makes sense. So apart from Azure, large deployments of Hyper-V? It can be any version. In my session yesterday, I had somebody with 140 Hyper-V servers. That was the biggest one in the crowd.
BA: When we talk about large deployments, there are actually three audiences and customers that I look at: enterprises, hosters and branch office / retail.
There’s this fascinating trend, and that is that largely thanks to virtualization, what we’ve thought about as the traditional large data centers has shrunk down to a fraction of what they used to be. I talk to enterprises today who ten years ago would’ve had like 200, 300 servers. And over the last ten years, they’ve grown their server workload by a factor of ten, and they now have 20 host servers.
As someone who’s been in the industry for a while, you have to occasionally take a step back and do a reality check. About 18 months ago I was talking to the team about scale and I was saying that if you set up a four-node Hyper-V cluster on modern hardware from your OEM of choice, you can probably quite comfortably run 400, 500, 600 virtual machines on that. And 600 virtual machines is a lot of virtual machines.
We’re seeing these enterprise customers who’re running huge amounts of workloads but actually doing it on really small amounts of hardware, and so we’re seeing like a lot of traction in the eight-node space.
Back to your question, where we’re seeing large deployments. One is the people who’re regional hosting providers, and we have a number of case studies where we talk about different hosters around the world that are using Hyper-V.
But, even in the hosting space it’s fascinating because you see the dwindling host effect. These days if I run into a customer who say they have 150 hosts. That kind of sends a red flag to me where I see that they were in the early wave of virtualization, and they're not doing things very efficiently. To put this in context, I’ve met with hosters in Europe, where they’re serving 15,000 virtual machines out of one rack.
PS: Wow!
BA: Yes if you do the math, one rack can hold 20 servers in there with lots of cores and RAM so 15,000 divided by 20, that's 750 VMs per server. On modern hardware that’s entirely possible.
But if you’re looking for number of hosts, where we have the huge deployments is for the customers who’re doing branch office. So we have a case study with these folks, and their main IT guy is here at Ignite. I’ve had a chat to him. They have 1,500 Hyper-V servers. But it’s two in every shop. Two Hyper-V servers running the back end. I mean, we have a lot of retail, where they’re doing that, and those ones give us the crazy large host numbers.
PS: They only run a couple VMs on those hosts.
BA: And that’s why I say actually I find it more interesting to talk about how many VMs people are running.
PS: All right, so the follow-up question then. What’s the highest density you see? How many hundreds of VMs do people fit on to a single host?
BA: I rarely see more than 100.
BA: This is something that we always track, that at any point in time, there’s a density/hardware cost sweet spot. It tends move around. Right now, when I see the people who’re really focused on the bottom line, what we tend to see is 2-processor 8-core hyper-threading system that gives you a total of 32 cores to play with, and with 256 gig of RAM connected over multiple 10-gig links to some form of shared storage.
And they can handily run 100 low-usage virtual machines.
Storage Spaces Direct
PS: So let's talk about Storage Spaces Direct.
BA: Yes.
PS: For a branch office it seems simple, two hosts with SSD and HDD and then you have your shared storage. You don’t need to buy any external storage of any kind. It’s still going to be fairly costly however because you have to buy Windows Server Data Center edition, rather than Standard edition.
BA: I have kind of a two-part answer to that. The first part is I do want to be very clear that our aspiration for Storage Spaces Direct is we want it to be everywhere, doing everything for all people. We are messaging towards the smaller deployments right now largely because it’s a version 1 technology, we think all our work is good, but it’s also unproven. We’re looking for these opportunities where we can start to get more experience around that.
I remain thankful I’ve worked at Microsoft for 13 years now, and I’ve always managed to be on the periphery of licensing discussions.
We do want to make sure that our customers feel like they’re getting a good deal with Microsoft, we always want to be a very cost-efficient offering. On the other hand, we also all like getting paid, and so making money is important. So, in the case of Storage Spaces Direct, a large part of the decision to put it in Data Center, there were really two main things that fed that decision. The first one is this is something that we’ve done a lot of engineering on over the last few years, even though it's only coming out now. We’ve invested a lot of our resource in bringing this to market, and we have to show that that was time well spent.
The second one is that from a customer's point of view, where they’re spending huge amounts of money on traditional storage, we think it’s appropriate for us to say: Hey, we can get rid of that cost, but you have to pay us a bit more in the process.
User feedback
PS: That makes sense. What's the user feedback been so far on Hyper-V in Windows Server 2016?
BA: One of the things that I absolutely love is Windows Server user voice. I have got to tell you, there have been a couple of things that have come up on Windows server user voice, where someone puts it up there and it gets like a couple of hundreds votes and I, as the program manager, can say yes, I’ve always wanted to fix that, but I never had the customer data to say that we should.
The team all know I’m the most devious, terrible person in the world, so what I’ll do is I’ll go in and say in this release, we should make it a priority to fix the top three issues on Windows Server user voice for Hyper-V, not mentioning what they are. I just get people to agree to that philosophy, and then, when they see what the top three are, they’re like, Ben, this is just like your list of stuff you’ve wanted to do for years.
PS: There is a common misconception that because you can’t use VHDX files in Azure, it' still running on Windows 2008 R2 Hyper-V?
BA: No, that's not true, Azure is running Windows Server 2012 R2 and upgrading to Windows Server 2016. It's just that some features aren't enabled.
PS: Scott Guthrie mentioned in his unplugged talk, that they have upgraded 100,000 servers already.
BA: The whole VHD thing, once again comes back to the joys of dealing with Azure-style scale.
It’s simply like the storage infrastructure they have for supporting millions of hosts, tens of millions of VMs, is truly mind-blowing. But it’s also something that is highly customized and something that the Azure team has built just for their environment. And it is currently very optimized towards the VHD format. We’ve been working with them for quite a while to get to VHDX but there are challenges with that.
Microsoft Azure Stack
PS: Next I want to talk about Microsoft Azure Stack. I’ve had a couple of interesting conversations this week with people who’ve said, oh yes, Azure Stack is going to be the way forward on premises and certainly, from Microsoft’s messaging, that seems to be the case.
BA: Yes.
PS: But there are some limitations, and I’m not sure that people realize this because if you’re using Azure Pack today or just ordinary Hyper-Y you've got dynamic memory. You've got shared VHDX. You can do guest clustering easily. You can do live migration. Well, as far as I understand it, none of things are picked up in Azure Stack because you can’t do it in Azure, so it won’t be in Azure Stack.
BA: This is a point of a lot of discussion inside Microsoft right now. When we look at, when we look at what people are doing on premises today, 95% of what we see people doing on premises today is just standard IaaS cloud computing—spinning up VMs. Our assertion is that at some point in the future, you aren’t going to be running IaaS virtual machines. You’re going to be running cloud apps, and you’re either going to be running them in a public cloud or on your private cloud.
And we’re going to get to a world where actually most enterprises are microservices-based. And if you believe that that is the future, (and there are people who don’t), but if you do, then Azure Stack is exactly what you need because Azure Stack is Azure in your data center. Now a couple of the interesting questions that come up, which all come around like so what about the customers who haven’t started on that journey. Now what about the IaaS customers today?
And that’s a really good question, and I mean, from our point of view, certainly from the Azure Stack team point of view, can you do IaaS on Azure Stack? Absolutely. Are they focusing on making that a fantastic offer? No. Their top priority is delivering the cloud platform for Azure app development.
PS: Okay.
BA: And IaaS is possible, but that’s not the selling point. I don’t think you’d find anyone in Microsoft who would go and say if all you care about is IaaS all day long, then Azure Stack is going to be the bee's knees. And there are a lot of open questions about how do we position that, how do we deal with that. This gets argued back and forth.
There are people who are asking what investments we need to continue to make in System Center so that as customers are going through this transition, they have an offering. From my experience, talking to customers about Azure Stack and their plans for Azure Stack; the two observations I would make are that the customers that are very much in the mindset of really looking forward to Azure Stack because we’ve started dipping our toe into developing applications cloud style and we see that that’s the way of the future and that’s how we want to be doing everything.
The second observation is where customers say we have a bunch of already deployed infrastructure aptly running on a System Centre–based cloud, and that’s not going anywhere soon. But we’re really looking forward to starting up an Azure Stack cloud so that we can start moving to the new model while we continue to have the old infrastructure, in parallel.
PS: So it’s more of IaaS versus PaaS?
BA: Absolutely.
PS: Thank you. That actually makes it a lot clearer.
BA: Our statement has always been that Azure Stack is Azure consistent, and that comes with both the good and the bad. Over the last year, I’ve had the opportunity to help a number of companies of different sizes transition their workloads to Azure. And the thing that is really obvious to me is that the companies that have the best experience, the companies that get the most savings, the companies that come away at the end of the day—going like, man, this is great; we’re really happy that we went through that—are the companies who at the beginning of that process go, okay, here are the high-level services and functions we need. How do we rebuild infrastructure to do that efficiently on Azure? The companies that don’t have great experience are the ones who just say; we got this infrastructure. We want to just lift and shift that to Azure.
And the one that really frustrates me, and I see this so many times, is I talk to the companies who are great Microsoft customers doing great things with private cloud. And they sit there and go like, Yes, we’re going to be private cloud forever because every time we sit down and do the math's of figuring out what it would look like to do a lift and shift with no architecture change from on premise to Azure, it’s stupid. Azure would cost us more. And it’s like, yes, doing a lift and shift to Azure with no architecture consideration or change is stupid. That doesn’t mean Azure is stupid. It means, it’s time to think about a new architecture.
Subscribe to 4sysops newsletter!
PS: Thanks very much for your time Ben.
