Monday, June 1, 2009

Past Performance Data and the Need to Change the Source Selection Paradigm

GAO-09-374 Federal Contractors: Better Performance Information Needed to Support Agency Contract Award Decisions

The GAO report entitled Better Performance Information Needed to Support Agency Contract Award Decisions (GAO-09-374), lays out a disturbing trend in government oversight and contract management that needs to be corrected on many levels if the government is to increase performance of federal contracts.

As noted in the report, the typical information management problem of federal databases is persistent in the Past Performance Information Retrieval System (PPIRS); garbage in equals garbage out. Past performance data is crucial to evaluation of past performance of course, such as contract terminations for default and subcontract management. However, this data can also be a good indicator of future performance and the ability of contractors to perform at the level required by the government. This data should be specifically documented, relevant, and reliable. However, the GAO report indicates a very low confidence level for PPIRS:

…GAO’s review of PPIRS data for fiscal years 2006 and 2007 indicates that only a small percentage of contracts had a documented performance assessment; in particular, we found little contractor performance information for orders against the General Services Administration’s Multiple Award Schedule…

The lack of standardized, streamlined processes for performance evaluations seemed to be at the heart of the GAO report for the reasoning behind the poor quality and quantity of data:

…Some officials noted that a lack of accountability and lack of system tools and metrics made it difficult for managers to ensure timely performance reports. Variations in evaluation and rating factors have also limited the usefulness of past performance information. Finally, a lack of central oversight and management of PPIRS data has hindered efforts to address these and other shortcomings…

As the Federal Acquisition Regulation (FAR) requires past performance information to be considered as an evaluation factor in certain negotiated competitive procurements (FAR 15.304(c)(3)), agencies have broad latitude in determining the importance of past performance data and its use in evaluations, such as evaluating past contracts in terms of size, scope, complexity, contract type and relative importance to the solicitation at hand. However, many solicitations for which requirements may not be clearly defined (e.g. many current federal acquisitions) and a higher risk of unsuccessful contract performance, past performance, technical capability, and other factors are weighed as more important than cost or price. It is this pursuit of “best value” where government source selections often break down, and where the paradigm shift needs to occur to truly get the performance the government desires and needs.

Since the FAR requires agencies to document and evaluate contractor performance for each contract that exceeds the simplified acquisition threshold upon completion of the work (FAR 42.1502(a)), past performance must be made a higher priority during not only the contract term, but before contract award during the evaluation process. As noted by the GAO:

…Agencies considered past performance information in evaluating contractors for the contract solicitations we reviewed, but many of the officials we spoke with noted that past performance rarely, if ever, was the deciding factor in their contract award decisions. Their reluctance to base award decisions on past performance was due, in part, to their skepticism about the comprehensiveness and reliability of past performance information and difficulty assessing its relevance to specific acquisitions…

Technical and Management capability are often the most important factors in source selections where the government seeks to award based on best value, which is often counterproductive to the realities of the government market. Further, this process actually hinders accountability as performance not only is poorly documented, but also not effectively integrated into proper program management techniques in the execution of government contracts. Getting past performance as the key to contract management oversight would go a long way to improving cost, schedule and performance goals and thus improving acquisition outcomes.

…Contracting officials who viewed past performance as an important evaluation factor noted that basing contract award decisions, in part, on past performance encourages companies to achieve better acquisition outcomes over the long term. For example, according to officials at one Air Force location, an incumbent contractor was not awarded a follow-on contract worth over $1 billion primarily because of poor performance on the prior contract. As a result, the contractor implemented several management and procedural changes to improve its performance on future contracts…

Nonetheless, many contracting officers, the report noted, preferred to rely on other more objective factors such as technical approach or price due to their reluctance to rely more on past performance in making award decisions and obtaining candid past performance information. Reasons given included the fact that contracts are so poorly managed that the government has such difficulty separating issues caused by the contractor from those caused by the government, most notably changing or poorly defined requirements. Other factors included fear of damaging contractor relations, and further fear of contractors challenging negative ratings and thus the “watering of down” of assessments by contracting officials.

Focusing on technical approaches to solicitations has not been successful, and will continue to lead to poor outcomes. Most federal contractors provide high levels of goods and services to the federal government, although the bad ones are rarely or poorly held accountable through poorly documented performance reviews. It is not common to have a company be so exceptionally superior during source selections that they are the clear winner, as the requirements would have to be very unique, along with the goods and services being offered. The reality of source selections is that technical factors are often weighed the most important, followed by past performance, then price. However many times during “best value,” it ultimately leads to price as the differentiators of technical and past performance seldom exist, thus negating best value. Although performance based acquisition is a methodology that would help offset this scenario, this process is poorly planned and executed by the government as well.

Accountability and transparency are the central themes of government contracting reforms by the Obama administration, and improving performance data and making it a more important factor can help play a vital role is reaching these objectives. The GAO made several important and common sense recommendations to help improve the data, including having standardized evaluation factors and rating scales governmentwide for documenting contractor performance, establishing policy for documenting performance-related information not captured systematically across agencies, and having controls and oversight responsibilities for updating PPIRS.

By improving the processes and the incentives to adequately evaluate, monitor, and document past performance information, the government can realize a vast improvement on acquisition outcomes that hold contractors accountable for performance.

No comments:

Post a Comment