search menu icon-carat-right cmu-wordmark

CERT Tactical Response and Analysis Challenge Tests Cybersecurity Skills

CERT Tactical Response and Analysis Challenge Tests Cybersecurity Skills
November 17, 2009 • Media Coverage

November 17, 2009—Throughout the first day of competition, Poland and Australia were jockeying for the lead, but at the end of the two-day challenge, it was Australia in first place among the 29 competing teams from 20 countries. No, it wasn't the Olympics or World Cup;the five-person Australia 1 team took first place in the Tactical Response and Analysis Challenge (TRAC) conducted by the SEI's CERT Program as part of the weeklong International Cyber Defense Workshop (ICDW), which concluded November 13.

The ICDW is sponsored by the Office of the Assistant Secretary of Defense for Networks and Information Integration (NII) and hosted by the University of Nebraska Omaha. CERT is one of several organizations providing training as part of the virtual workshop. CERT's TRAC is the only portion with competitive scoring. The top five finishers in the TRAC event were Australia, Poland, the Republic of Korea, NATO, and a combined (but geographically separated) team from Germany and France.

TRAC tested the teams' responses to cyber-attack scenarios, or war games for cyber-security, said Chris May, technical manager of the CERT Workforce Development team. May draws upon his experience as an Air Force captain to explain: "Going to a training course doesn't mean you can actually do something in the fog of war;you need experience with situations that test your abilities to do live problem solving. That's why the military uses live-fire exercises that embody the 'train as you fight' goal. You can have the best bombing or shooting range in the world, but the range is only as good as the training scenario that goes with it."

"The TRAC exercises were designed around realistic scenarios to serve as experiential training," said Jeff Mattson, a member of the CERT Workforce Development team. "Participants generally walk away with more benefit because they are hands on. Participants approach the activities in a free-play environment where they have to bring their problem solving skills to bear on the situation. It's more than just doing something they know how to do. They're in a new environment where they have to apply principles they know in that new environment."

The teams are guided through the sessions by quizzes that direct the players toward the information that they must find. "We don't tell them how to find it," said Mattson, "and we've actually seen a lot of different approaches to getting to the same answer."

In the TRAC scenario, internet-based attackers find and exploit vulnerable web-application and database servers of a fictional shipping company and wreak havoc throughout its network. "This is all accomplished in real time while participants are in the environment," said Mattson. "It's hacked while they're watching. The intruders are able to get a root kit, download it, and then they start with some botnet activity trying to infect other machines in that network. So there's a lot of network activity that the participants can find—there's a lot of log file artifacts they can find—so it's a robust exercise."

The first day of the two-day TRAC exercise concentrated on detection, monitoring, and mitigation activities, and the second day focused on computer forensic analysis. The scenarios were created by May and Mattson, CERT staff members Rob Floodeen and Josh Hammerstein, and graduate students in information security from the Carnegie Mellon University Heinz College and Information Networking Institute.

The virtual environment that allows participants all over the world to engage in the challenge is CERT's Exercise Network (XNET). Participants in an XNET training exercise access the environment through a standard web browser. They are presented with an exercise portal from which they can communicate with teammates via a chat client, wiki space, and desktop-sharing capabilities. This portal essentially assembles the team in the same virtual room, so participants can be physically located anywhere in the world and yet experience exercise events and accomplish the training tasks together. The exercise network is isolated from the participants' home networks through controls in place at the portal;however, it also provides for out of band upload and integration of participants' favorite tools and applications.

CERT Program director Richard Pethia explained the significance of the TRAC exercises and the XNET environment: "Building a workforce that has the knowledge, skills, and experience to effectively perform cybersecurity tasks at this level of sophistication is a hard thing, and those skills are needed in a lot of organizations. We've been looking for a cost effective way of delivering the training and, most importantly, the exercises which allow that training to take root. Constructing a real network where people could practice these kinds of skills would be horrendously expensive, especially if you think about duplicating that across all the tens of thousands of organizations that need the training, so this is a very cost effective way to allow teams to build up team skills."

This is the largest test so far of XNET, which was created in 2008, said May. During the ICDW, more than 140 globally distributed participants accessed and controlled over 700 virtual machines running inside the web-based XNET training environment. CERT will conduct a complete review of XNET performance in the execution of the TRAC exercise, but the preliminary results indicate that the exercise came off without any significant glitches.

More information about XNET is available on the CERT website and in the CERT Annual Research Report (see Page 84). Specific inquires can be sent to

Photo caption: Jeff Mattson (left) and Chris May