Right from the Start: Mission Assurance at Program Initiation
Right from the Start: Mission Assurance at Program Initiation
Early introduction of mission assurance injects discipline into the development approach. Aerospace is working to ensure that measurable mission assurance products and deliverables are designed and implemented early in the acquisition lifecycle and clearly spelled out in the contract.
First published May 2013, Crosslink® magazine
A driving principle of acquisition reform in the 1990s was that space systems could be obtained more efficiently through drastic cutting of perceived nonessential activities in system development. On the government side, contract specifications, technical oversight, and independent reviews were curtailed. On the industry side, system engineering and testing were deferred. As a result, inevitable defects in system designs were not detected until late in the development cycle. The resulting rework needed to deliver the intended capability led to long delays and higher costs.
The industry has gradually been recovering from the problems that occurred with this approach. Recovery efforts focused on the effectiveness of mission assurance, as measured by launch and satellite operational success; but adding it after the fact (due to a lack of detailed planning and management) led to unexpected rework that negatively affected cost and schedule. In 2006, Aerospace embarked on an examination of acquisition processes; it became apparent that improvements in both effectiveness and efficiency were possible and necessary. The performance-based view of mission assurance was expanded to include practices and processes focused on meeting cost and schedule objectives. These included the disciplined application of technical and program management principles that collectively contribute to the goal of comprehensive lifecycle success.
Achieving overall lifecycle success begins at program inception. Aerospace is taking innovative actions to increase emphasis on front-end engagement on nearly all Space and Missile Systems Center (SMC) programs. These actions range from early concept development and future architecting to acquisition planning and integrated baseline development, and they rely on Aerospace’s core strengths and resources to provide domain-specific expertise in the identification, prevention, and resolution of complex program executability issues. Key to this work is engagement in the early acquisition phases to balance desired capabilities with a variety of constraints, including cost, schedule, risk, technology readiness, and industrial capacity.
Influence on Early Requirements, Concept, and Design Exploration
The development of a new or upgraded system begins when a warfighter identifies new operational needs. These needs are translated into a set of desired capabilities. Aerospace serves as an active participant in needs assessment and requirements definition. Examples include understanding warfighter needs and translating those needs into functional capabilities, helping define feasible system solutions based on engineering principles and programmatic constraints, and providing alternative architectures that help to shape the next generation of military space.
Before any system acquisition can begin, the government must have an approved acquisition strategy that defines the approach to deliver the required capabilities within the approved budget and schedule. The strategy should define the relationship between the acquisition phases, resources, work efforts, and key program events such as milestone reviews, contract awards, development activities, test and evaluation, production quantities, and operational deployment objectives.
The acquisition strategy assimilates the plans from critical program documents. It starts with the requirements captured in the initial capabilities document, capabilities development document, and capabilities production document, which describe operational performance and contain key performance parameters and system attributes. The capabilities development document is drafted during the technology development phase; it defines measurable and testable capabilities to guide the subsequent engineering and manufacturing development phases.
Other critical documents include the technical requirements document, which translates warfighter requirements into performance-based acquisition requirements. Aerospace is involved in the development of this document— often as the primary author. As such, Aerospace carries the primary responsibility on behalf of the program office for ensuring that all system and technical requirements clearly trace back to user requirements. This requirements tracing task is not a mechanical administrative activity, but rather requires complex understanding of the user’s intent, key performance parameters, key system attributes, concept of operations, state of technology readiness, industrial base capacity, verification approach, sustainment concept, and life-cycle budget. Recent policy requires the major command, as user representative, to certify that the technical requirements document demonstrates clear traceability and acceptable decomposition of user requirements and intent. Therefore, this administrative coordination is actually the culmination of a rigorous systems engineering process for requirements elicitation and functional decomposition.
The technical requirements document is the technical description of a new system the government intends to develop. As such, it is the prime component of a request for proposal (RFP), which invites contractors to submit competitive bids to produce the envisioned system. The RFP describes the government’s acquisition proposal solicitation elements, defines the content of the required documents with specific implementation details, and ensures coordination of workforce and organizational resources involved in expected contractual tasks. Aerospace is recognized as an expert in RFP development and provides orientation and training to government program managers and staff. In fact, Aerospace developed a template to help programs develop RFPs.
The selection of a contractor must be done in strict accordance with federal regulations. The SMC Acquisition Center of Excellence is responsible for providing guidance and assistance to programs in planning and conducting their source selections. Personnel from the center and Aerospace have visibility across all SMC acquisitions and are well versed in current acquisition policy. This expertise, provided in training sessions and documentation reviews, benefits the programs by decreasing the overall evaluation timeline and reducing the likelihood of a sustainable protest. For example, the Acquisition Center of Excellence and Aerospace provide training in how to evaluate technical capability and past performance both prior to and during source selection. They advise source selection personnel in their role and responsibilities and ensure that source selections are conducted in accordance with acquisition policy, the RFP, and the source selection plan. They develop templates for briefings and decision documents, provide technical support and training for source-selection tools, provide a secure facility for conducting source selections, and collect lessons learned to improve future processes and resources.
Aerospace provides unbiased technical support in the preparation and execution of SMC source selections through service on the source selection evaluation team, the source selection advisory committee, and the multifunctional independent review team. One critical area of support is in the technical evaluation of proposals. Aerospace personnel serve as advisors to evaluate specific areas to determine whether and to what degree the bidder meets the stated requirements. These technical advisors assess the risk associated with an approach and determine whether the cost is realistic. Based on experience supporting a variety of SMC source selections, Aerospace provides training and feedback to help source selection teams make high-quality, defendable evaluations and operate in accordance with applicable policy.
For example, for the AEHF 5/6 production contract, Aerospace provided technical and integrated program management expertise to perform affordability analyses and identify cost-avoidance opportunities. This resulted in a prioritized list of cost-avoidance opportunities with detailed rationale supporting the government positions and use as part of the AEHF 5/6 business clearance. The lessons learned from the AEHF 5/6 proposal evaluation will be applied to other upcoming production contracts.
Mission Assurance on Contracts
Aerospace provides technical expertise for competitive source selection or technical evaluation for sole-source contracts. The contract defines the government and contractor partnership, affects each partner’s structure, and defines the program and contract baselines.
Front-end acquisition activities influence a majority of system costs, but represent only a small portion of the effort; in fact, by the end of the preliminary design phase, about two-thirds of the lifecycle cost is committed. The increasing difficulty in changing a design over time directly translates to higher cost and schedule delays, especially when replanning or rebaselining occurs late in the lifecycle. No amount of government oversight at the eleventh hour can overcome
fatal flaws or inadequate test programs that were not addressed early on. Therefore, mission assurance must begin early in the program acquisition lifecycle and continue throughout design, build, launch, and operations. The objective is to arrive at efficient, measurable core mission assurance standards and deliverables that can be placed on contract to provide the government with a positive understanding and appropriate span of control for accessing and mitigating risks in performance, schedule, and cost. Incorporating mission assurance increases knowledge prior to contract negotiation and should lead to higher confidence in achieving overall program lifecycle success.
Aerospace helps to ensure that mission assurance is applied throughout all phases of a program. The level of mission assurance for any point or element is based upon system risk management—the identification, assessment, and prioritization of risks. It is a common thread linking program management, acquisition planning, system engineering, and cost estimation. Aerospace developed and offers a series of hands-on workshops to assist program office personnel in implementing the acquisition development process. As one example of these workshops, experts from the SMC Acquisition Center of Excellence and Aerospace meet with the participants to guide risk-workshop outcomes and develop risk-mitigation plans for identified high-risk items.
Availability of contractor data and key personnel is critical to ensuring that mission assurance is incorporated in the program baseline. Contractual provisions regarding contractor activity, procedures, and reporting systems determine staffing requirements and are generally covered in statements of work, compliance documents, schedules, specifications, contract data requirement lists, and data item descriptions. These provisions must provide for adequate contractor implementation and information transmittal; if the mission assurance provisions are not included in the baseline contract, the government program office must identify and implement contract change mechanisms.
Government efforts have focused on specific ways to foster mission assurance by providing contractors with effective incentives. Good mission assurance processes should become a key discriminator in future source selections, and an approach that effectively and efficiently manages risk of system development should be rewarded. Contracts must delineate the required specifications and standards and show how they are reflected in the contractor’s command media. The adaption of generic practices and processes must maximize the added value (especially in the reduction of cost, schedule, and performance risks) while minimizing the costs of compliance. As affordability trades are considered, the DOD must guard against creating an acquisition environment that unintentionally motivates contractors to cut corners in mission assurance.
Acquisition reform in the 1990s presumed mission assurance would be automatic, but experience showed that it must be explicitly defined and accepted by the government and contractor. Fiscal conditions in subsequent years also highlighted that full risk mitigation was not affordable. Accordingly, the government realized that mission assurance must be tailored. Tailoring mission assurance is the effective and efficient customizing of proven practices to suit a specific situation and level of acceptable risk. Tailoring strategies typically include transferring or deferring the risk to another system element or time period, avoiding the risk, reducing the probability or impact of the risk, or accepting some of the potential consequences of a particular risk. This tailoring requires an iterative exchange between the party that understands the requirements and situation and the party that understands the proven practices. More often than not, there is a chasm between these two groups, so the government typically errs either on the side of caution (which can result in program execution issues) or carelessness (which can result in failures); both induce cost and schedule overruns or constrained alternatives and inflexible architectures. This interplay becomes ever more complex when a variety of concepts are pursued to achieve resiliency and cost efficiency. Examples include the exploitation of space capabilities through commercial and foreign resources as well as space-capability enhancements through lateral exploitation of data from existing sources. Aerospace is often called upon to bridge this chasm of understanding.
One major example of mission assurance tailoring involves testing and evaluation, which is performed largely by prime development contractors using their own facilities. Much of this testing is performed at lower levels of assembly because defects identified at that phase are cheaper to correct. System-level development tests are also typically performed at contractor facilities because of the cost, risk, and time involved in transporting a fully assembled spacecraft to a centralized government facility. While these tests are performed at contractor facilities, they still receive robust government oversight—and in most cases, the government team (including Aerospace) helps to plan them, witness their execution, analyze the results, and troubleshoot any anomalies.
The government defines the testing approach and levies test requirements through the disciplined application of appropriately tailored specifications and standards on development contracts. This has led to a dramatic increase in reliability of space and launch vehicles in the last decade. The Air Force is also working to expand and enhance its test and evaluation workforce and continues to develop and apply “test like you fly” principles to space systems.
New Program Exemplar
The first program to benefit from SMC’s mission assurance baseline is GPS III. Although all SMC acquisition programs have benefited from the renewed emphasis on mission assurance and program executability, GPS III is the first program to fully implement the lessons learned from a decade of relearning and to incorporate mission assurance from
GPS III program management began with a prioritized set of approved and well-understood system requirements along with senior leadership advocacy and DOD Joint Requirements Oversight Council stabilization. GPS III took six years to understand, vet, and decompose requirements through numerous executive-level reviews. The next step was the development and approval of the acquisition strategy, which was vetted through multiple independent program assessments and a five-month-long integrated baseline review. A number of SMC assistance organizations were engaged as well, including the Acquisition Center of Excellence, the Program Management Assistance Group, and the Engineering Assistance Support Team.
Significant risk mitigation was achieved by thorough concept exploration, ensuring that critical technology elements would all reach appropriate readiness levels. The program maintained two prime contractors from requirements definition through system design review. In addition, key risk mitigators—such as a pathfinder and GPS satellite simulator—were built into the master schedule. The workforce was trained to provide capable and consistent government oversight and system integration. The GPS III baseline included a robust integrated master schedule and independent baseline-review process. Extensive use of the Program Management Assistance Group filled critical program office gaps, and joint training and execution was held with contractor cost-accounting models. Lastly, business execution was held on par with technical execution. Realistic cost estimates resulted in a low-risk schedule and 80 percent confidence based on government schedule estimates with mature technologies. The single integrated performance baseline provided visibility of cost and schedule impacts, and the critical-path analysis allowed for proper allocation of resources and early intervention.
Disciplined engineering for GPS III was also of paramount importance. The integration of program segments—space, control, and user—required a strong configuration control board. Directorate and contractor processes were standardized or integrated and included integrated change management, mission-level system engineering plans, and test and evaluation master plans in concert with industrial-base and manufacturing readiness assessments. Emphasis was placed on subcontractor management (e.g., supplier audits), and FFRDCs were engaged as much as possible; for example, Aerospace served as the squadron’s chief engineer. Early compliance with mandatory efforts (such as environmental analysis and information assurance) was performed. Special attention was given to software development, design verification, and satellite modeling. Additionally, fully tailored specifications and standards and lessons learned were on contract.
In the current fiscal environment, the space community must focus on affordability; however, caution should be taken not to sacrifice mission success, but to always build upon lessons learned and apply proven, disciplined engineering practices that are instituted at the start of acquisition planning and execution. A single catastrophic launch or on-orbit failure of a billion-dollar system carries a high price in terms of lost warfighting capability, replacement costs, and national prestige. The inability to develop and field less expensive systems because of high mission assurance costs is also unacceptable. The space community must turn its attention to early collaboration with the requirements community to define and prioritize warfighter needs in accordance with affordable system solutions balanced across capabilities, cost, schedule, and mission assurance. The government also must reestablish its ability to plan and manage programs. Finally, the government must continue to build its system engineering and integration capability and capacity as well as test and evaluation. Aerospace can influence the acquisition front end and serve as the glue holding it all together across the entire lifecycle.
AFFARS, “Informational Guidance for Developing Source Selection Documentation and Conducting Various Activities During a Source Selection,” http://farsite.hill.af.mil/reghtml/regs/far2afmcfars/af_afmc/affars/affarig1toc.htm (as of Feb. 12, 2013).
AFFARS 5315/DFARS 215/FAR 15, “Contracting by Negotiations,” http://www.acq.osd.mil/dpap/dars/dfarspgi/current/index.html; http://www.acquisition.gov/far/current/html/FARTOCP15.html (as of Feb. 12, 2013).
AFI 10-601, “Operational Capability Requirements Development” (July 12, 2010), http://www.e-publishing.af.mil/shared/media/epubs/afi10-601.pdf (as of Feb. 12, 2013).
AFI 63-101, “Acquisition and Sustainment Life Cycle Management” (Apr. 17, 2009), http://www.af.mil/shared/media/epubs/AFI63-101.pdf (as of Feb. 12, 2013).
AFSPCI 10-103, “Capabilities-Based Operational Requirements Guidance” (Sept. 3, 2010), http://static.e-publishing.af.mil/production/1/afspc/publication/afspci10-103/afspci10-103.pdf (as of Feb. 12, 2013).
Federal Acquisition Regulation (FAR) Part 7, “Acquisition Planning,” http://www.acquisition.gov/far/current/html/FARTOCP07.html (as of Feb. 12, 2013).
Guidance Memorandum: “Life Cycle Risk Management” (Nov. 4, 2008), http://www.e-publishing.af.mil/shared/media/epubs/AFMCPAM63-101.pdf (as of Feb. 12, 2013).
MIL-HDBK-520, “Systems Requirements Document Guidance” (Mar. 5, 2010), https://acc.dau.mil/adl/en-US/375563/File/50756/MIL-HDBK-520.pdf (as of Feb. 12, 2013).
Risk Management Guide for DOD Acquisition, 6th ed. (Department of Defense, Aug. 2006), http://www.acq.osd.mil/se/docs/2006-RM-Guide-4Aug06-final-version.pdf (as of Feb. 12, 2013).
About the Authors
Sumner S. Matsunaga
General Manager, Engineering and Integration Division, has expertise in satellite design and development and space and terrestrial communication systems. Since joining Aerospace in 1989, he has supported numerous National Reconnaissance Office and Air Force programs. He has a Ph.D. in electrical engineering from the University of Southern California.
Andy T. Guillen
Systems Director, Engineering and Integration Division, joined Aerospace in 1980 supporting government multiagency and multinational programs. His experience includes system engineering, system acquisition, program management, and structural mechanics. He has a B.S. in mechanical engineering from the Massachusetts Institute of Technology and an M.S. in systems management from the University of Southern California.
Ray G. Bonesteele
Senior Project Leader, Engineering and Integration Division, supports the Acquisition Center of Excellence at the Space and Missile Systems Center, Los Angeles Air Force Base, primarily planning and coordinating independent program assessments. Before joining Aerospace in 2004, he served for 24 years in the Air Force. He has a Ph.D. in meteorology from St. Louis University, specializing in Doppler weather radar detection of severe thunderstorms.
David L. Wang
Director, Engineering and Integration Division, serves as the Program Management Assistance Group technical director for the Space and Missile Systems Center, Los Angeles Air Force Base. Before joining Aerospace in 2007, Wang was an engineering manager at Cisco Systems supporting multiple international partnerships and also worked at Northrop Grumman supporting the SBIRS program. He has a B.S. in engineering and applied sciences and an M.S. in electrical engineering from the California Institute of Technology, and an M.S. and Ph.D. in electrical engineering and computer science from the Massachusetts Institute of Technology.
Back to the Spring 2013 Table of Contents
Go to sidebar: Successful Front-End Engagement