War Games and Vendor Management Best Practices

This is the third and final installment of our podcast series on War Games... gaming what-if scenarios with key vendor partners before something bad happens.

To listen checkout seprio.com/podcast.

In our first podcast, we discussed War Games at a high level. In our second podcast, we went into more detail around specifically how to run a war game. A common theme throughout both podcasts was using war games as part of vendor management best practices. In our latest episode, we delve into this topic to determine how war games can be used to better select a vendor, draft a more effective agreement, or improve an existing vendor relationship.

Here’s a peek into our conversation with special guest, Caren Shiozaki, and host, Pat Bohnenkamp.

Pat: What part of vendor management is important to you as a CIO and how do War Games fit into this?

Caren: I take a 5-point approach to vendor management.

  1. Not every vendor is going to have the same impact on your company so you really can’t manage them in the same way. The first thing I like to do is take our companies primary business objectives and then make sure that each vendor aligns in some way to that objective.
  2. Classify the vendors based on the products and services they provide and the level of impact they will have on the company. Are they falling in the strategic realm, tactical realm, or somewhere in the middle? Even within these buckets. I might have different tiers depending on what we’re trying to accomplish.
  3. Make sure you have the right structure in which you would manage the vendor. This includes building a team of people that would manage the vendor relationship. You want to design those performance management processes as well.
  4. Determine how you’ll manage vendor risk which has to fit into the overall risk management framework for your company and any corporate governance that the company has established that you have to fall in line with. It’s important to do a risk profile of every vendor you’re dealing with.
  5. Apply different performance based metrics to the relationship which you should have already considered when you developed the contract and should be constantly adjusted based on changes with the vendor and your company.

I would say war games would provide extremely useful input into point 4, vendor risk, and point 5, performance based metrics.

New Call-to-action

Pat: Can you talk specifically about how war games fit into vendor risk management and performance metrics? What specific scenarios would it be appropriate to run a war game?

Caren: The information that you get out of a war game can tell you whether you need to tweak the way you’re managing the relationship with a vendor. Depending on the nature of your relationship the vendors may participate in either a tactical or strategic game.

Let’s say I’m going to run a game that focuses on business continuity management. Those tend to be more tactical so I’d want all the vendors that would help me with business continuity to be involved.

If I’m running a game to test the company’s ability to deal with a data breach, that’s more strategic in nature. I’d figure out based on the framework which strategic vendors would have an impact on our ability to deal with a data breach and ask them to participate in the game.

Pat: In our previous podcast, you mentioned “tabletop” exercises. Can you explain what “tabletop” exercises are and how they differ from war games?

Caren: A tabletop exercise is similar to a war game, but is a bit less formal and conducted on a smaller scale. It will include a smaller group of people that are all directly involved in the scenario that’s being played out. The group would sit around a table with any supporting procedural documentation and talk through a given scenario.

A war game, on the other hand, is far more complex, requires more planning, and includes people from across the spectrum organized into teams that play a very specific role in the game with specific scripts. A war game requires the development of a playbook which involves a lot of research and data. The team will use the rules, data and the procedures that are designed to depict an actual or assumed real life situation.

War games can be costly where as tabletop exercises can cost little to nothing.

Pat: Another topic from our last conversation was around design bias, the idea that whoever designs the playbook may skew the direction of the games. Can you talk about that and how we may avoid that?

Caren: I really like to hire an objective, 3rd party consultant to run the whole game otherwise design bias can creep in. That’s especially dangerous if you’re trying to run a game to test scenarios for a particular service that a vendor is a key factor in. You can’t turn over the design of the war game to the vendor because they may, unintentionally, make their product or service look stellar through the course of the game. However, I would expect the vendor to be included in the team that provides input to the consultant who is designing the game.

Pat: How might we use the results of a war game in order to select a vendor during an RFP process?

Caren: I don’t necessarily see war games having a role in vendor selection given the complexity and cost of conducting them. I would ask the vendor what sort of experience they have had with war games with other clients and, if they had participated in other war games, I would follow up on their references to see how they performed. I wouldn’t necessarily hold it against them if they hadn’t conducted war games in house, especially if they’re a smaller vendor. But it bodes well if they’ve shown a willingness to participate in war games in the past or are open to the idea in the future.

Pat: How can war games be used to draft a more affective agreement or tweak the contract based on the results? For example, SLAs strengthened, termination rights tied more closely to areas of weakness, or limitation on liability tweaked to provide for greater vendor liability for a breach that a war game suggests is more likely than another.

Caren: Absolutely. In the war game, you’re going to see how different people play their roles in an assumed crisis, outside of their comfort zone. Remember, in a war game you might be assigned to a team that does not represent a role that you play in real life, but it gives you insight into how individuals view that counter role they’re playing. This can give you some indication about how well a vendor may be working with your team. After the games, you want to sit down and go over the results with your vendors and discuss any concerns or talk about areas where they actually excelled. You may want to increase your engagement with that vendor as a result. It might also mean you have to re-calibrate your risk assessment with that vendor.

You also want to keep in mind that the war game is pulling in people with different perspectives and even different vendors. This is a really good test of how things will operate in a multi-vendor environment. Cooperation doesn’t happen automatically so this can give you an idea where contract terms and relationship definitions may need to be tweaked.

Don’t miss this…

Listen to the full podcast to hear what happened to Caren when she served a bunch of Texans fruit and yogurt for a breakfast meeting.

This topic is the last in a 3-part series on War Games with our special guest Caren Shiozaki. To listen to all episodes, visit seprio.com/podcast.

If you’d like to explore the value of war gaming in your environment, visit seprio.com/wargames.

Protecting business priorities when partnering with vendors grows more complicated every day. Because of constraints in resources and expertise, many teams find themselves at a disadvantage managing vendor evaluations, negotiating terms, and securing fair financial value. Overcome those constraints, reduce risk and preserve fiscal health.


Seprio... Protect your business priorities. Contract better.