Software Engineering Institute

October 30, 2014—The SEI's Ian Gorton has been thinking big. Specifically, he's been thinking about the advent of big data and methods for overcoming scalability issues in the development of systems designed to handle it. "Project scope, requirements gathering, technological roadblocks, and access to data become key issues when building big data systems," said Gorton in a recent SEI webinar. "We need to think differently about our solutions and adopt software architectures, methods, and mechanisms that can ensure the scalability that our systems require."

The ability of Gorton and SEI colleague John Klein to think differently about big data challenges has generated a promising line of research at the SEI. This research forms the foundation for a new one-day course from the SEI, "Big Data: Architectures and Technologies." The course is designed for architects and technical stakeholders such as product managers, development managers, and systems engineers involved in the development of big-data applications. It focuses on the relationship among application software, data models, deployment architectures, and how specific technology selection relates to all of these.

"Many of the open-source, big-data solutions currently available provide high performance, but they can be complex and difficult to use," noted Gorton. The new SEI course will explore those challenges and also examine the new and alternative approaches he and Klein have been investigating. "The research and development by internet organizations into solving big data challenges means there now exist numerous technologies available for those building solutions," said Gorton.

Participants in "Big Data: Architectures and Technologies" will learn about the following topics:

• Principles of Scalable Systems
• Introduction to Distributed Systems
• Defining Scalability
• The NoSQL Zoo
• Durability and Consistency
• Distributed Data Processing Frameworks
• Data, Distribution, Deployment – Coupling of Concerns
• Technology Evaluation and Selection

"We want participants in this course to come away with a solid grounding in issues important to the development of big-data systems," said Klein. "Specifically, we will provide them a better understanding of the major elements of big-data software architectures, the different types and major features of NoSQL databases, and patterns for designing data models that support high performance and scalability." Klein also noted that the course will provide an introduction to the SEI's Lightweight Evaluation and Architecture Prototyping for Big Data (LEAP4BD) method for rigorous evaluation of big-data technologies and architecture approaches. LEAP4BD reduces the risks in adopting a NoSQL database management system by ensuring that a thorough evaluation of the solution space is carried out in the shortest possible time and with minimal effort.

The course will be taught by Gorton and Klein. Gorton is a senior member of the technical staff at the SEI where he investigates issues related to the design of large-scale data management and analytics systems and the inherent connections and tensions among software, data, and deployment architectures in cloud-based systems. Before joining the SEI, Gorton was a laboratory fellow in computational sciences and math at Pacific Northwest National Laboratory. He also led the software architecture R&D at National ICT Australia (NICTA) in Sydney, Australia. Gorton is a senior member of the IEEE Computer Society and a fellow of the Australian Computer Society.

John Klein, also a senior member of the SEI technical staff, has more than 20 years of experience developing systems and software. He joined the SEI in 2008. Before joining the SEI, John was a chief architect at Avaya, Inc., where his responsibilities included development of multimodal agents, architectures for communication analytics, and the creation and enhancement of the Customer Interaction Software Product Line architecture. Before that, Klein was a software architect at Quintus, where he designed the first commercially successful multi-channel integrated contact center product. Klein began his professional career at Raytheon, where he developed hardware and software solutions for radar signal processing, multi-spectral image processing, and parallel processing architectures and algorithms.

The SEI is offering three ways to attend the course:

• public, instructor-led offering at an SEI office
• private, instructor-led training at customer sites
• eLearning(Available soon)

To learn more about the new SEI course "Big Data: Architectures and Technologies" and to register, please visit  http://www.sei.cmu.edu/training/p114.cfm.