Q&A: Virginia CIO Nelson Moe on Scaling Up IT in a Hurry

STATETECH:  Before the pandemic, roughly how many state workers were working from home, and where are you guys at right now? 

MOE: Remote work for a vast majority of the commonwealth was something that was about work-family balance, but not a strategic aspect. And specifically, we have a virtual private network infrastructure that we’ve upgraded. But before this, we baselined it. The weekend before everybody started working home, the peak usage was about 3,000 concurrent users at any one time, and then about 66,000 devices on the network. So, some people did work from home and needed a VPN connection, but it was not near the peak it is today. And the maximum we were able to support at any one time was 5,000 concurrent connections. We never really got to that. We were always able to support the demand with existing infrastructure. 

After the COVID-19 events and the executive orders started happening, the demand for VPN technologies and other ones we have generated since then was dramatically increased almost overnight. In fact, today we’re seeing about 11,000 concurrent connections at any one time. And so, from a 3,000 peak to an almost 11,000 peak is amazing. And separately, we’ve changed our capacity to support those connections from 5,000 to 35,000. This was just traditional VPN. And so, we’re very comfortable with that.

As a #StateLocal #IT leader, which #ITInfrastructure or support solutions have you invested in most to enable #telework?

— StateTech Magazine (@StateTech) June 3, 2020

In addition to that, we’ve been able to respond with other technologies. The company is called Zscaler, and they have basically an application layer VPN that is born in the cloud. We went from our on-premises VPN, which is the traditional VPN, to adding an application layer VPN, which agencies like [the Virginia Department of Transportation] love. It’s a little easier to use and just as secure. 

We were getting requests by the Department of Social Services because they have to connect to entities that do not have Commonwealth of Virginia accounts. And so, we have to set up virtual desktops for them. That technology is not brand-new, but we just could never come up with a business case to do it. Only a couple of people wanted it. Now, we have a huge demand for it. So, we’re able to jump in and roll out virtual desktops, and we’re using Microsoft’s Azure virtual desktop platform to meet those needs. 

STATETECH: How has VITA shifted in terms of how it works with the other state agencies? 

MOE: Virginia has positioned itself uniquely to provide an extremely adaptable service set. The phrase you use is “CIO as a broker,” providing services across a large number of things. I don’t have to own it; I just have to provide the contract to it. We have a supplier model, an integrated supplier model, and a contract structure that allows for a significant amount of flexibility. We’ve been moving toward the cloud for a number of years and have the ability to add services within the platform and also outside of it. In fact, we were able to adapt much more quickly with our service model in rolling out technologies — the VPN in the cloud, the virtual desktop, the laptops and cloud services — in our model now than we would have ever been able to in our previous model. 

It’s been two years now that we’ve been on this journey. We’re seeing the fruits of being able to have a contract and service structure like a service catalog that is very adaptable to adding services. 

READ MORE: Find out how state CIOs think their operations will evolve. 

STATETECH: What has been the most challenging aspect of this from your perspective? How have you been able to address potential challenges?

MOE: For the initial week or so, we really had to roll out and expand the capacity as fast as we could. After about two weeks, that was solved. Now, it’s being able to meet the demand on the number of people who want to get connections. Now that we’re through that, the call center times are getting much, much better. The VPN requests are trailing off. I think the initial flood was what happened in every state. We were able to support our unemployment insurance agency as far as processing claims. They went from 2,000 claims to 600,000 claims in a matter of a month. And they were able to scale because of what we were able to provide them. And that was amazing. So, we were able to work through those challenges. 

The other challenge that we’ve been able to work through: A couple of agencies, their buildings got closed because they had to clean them. So, we had to set up a few disaster recovery sites on the fly overnight. We did that for the Department of Medical Assistance Services — they had to move out of their building. But we’ve been able to work through it and we’ve been able to manage it in a very secure manner. One of the lessons learned, which other states have said, is to not compromise cybersecurity for this type of thing. And we’ve been able to do that. 

STATETECH: What do you anticipate are going to be the long-term consequences, either from a continuity of operations perspective or just a state workforce cultural perspective?

MOE: The new working arrangement — what that looks like going forward, I really don’t know. But the conventional thoughts I’ve seen kicked around are that there’d be some expectation of working from home because of a virus event. 

People realize they can get their work done, they can minimize their cost structure and their physical footprint from a rent perspective. And it helps with disaster recovery because you’re not all in one spot. 

There’s a significant number of human resources policies that have to be fine-tuned. Onboarding is difficult. Creating a culture is more difficult, at least from my perspective, but it can be done. And we’ve had an all-hands meeting for my staff. It went really well. You want to see more interaction, but in the absence of physical contact in one big room, I think it’s the new norm, or at least part of the new norm. 

As we start bringing people back in, we will do it slowly and measuredly. We would maintain the social distancing, maintain the best practices and make sure people are safe and felt safe. And we communicate all that, how that’s being done. And, of course, we follow the best practices put out by the administration.

Check out this page for more coverage from the NASCIO Midyear 2020 conference, and follow us on Twitter at @StateTech, or the official conference Twitter account, @NASCIO,  and join the conversation using the hashtag #NASCIO20.