Best Practices for Submitting Data and Economic Analysis during Antitrust Investigations
The Bureau of Economics issued “Economics Best Practices” (EBP) in the early 2000s that provided guidance to parties on how best to submit and discuss data. The EBP also provided guidance on the most fruitful ways that the parties could present economic analysis to 51ɫstaff economists during an antitrust investigation. Economic analysis, including empirical analysis of potentially large quantities of data, continues to play a very important role in the Commission’s antitrust enforcement decision-making. The amount of data kept by parties, and the complexity of their data systems, has grown over the intervening period. In addition, econometric analysis of those data by both the parties’ consultants and 51ɫeconomists has become more sophisticated. Thus, we believe it a good time to update the guidance.
Data Submission
The FTC’s Premerger Notification Office guidance, Guide III, provides a sample model Request for Additional Information and Documentary Materials (“Second Request”).[1] It contains specifications regarding data as well as instructions on some specifics for its submission. The discussion below highlights these portions of the model Second Request and augments that document by providing additional guidance.
Although Guide III is directed to HSR-reportable transactions, its discussion of data submissions is broadly applicable to all competition investigations, including non-merger investigations. Therefore, although these Best Practices reference the model Second Request, they apply more broadly to CIDs and other information requests as well.
While the specifics of the data requested varies across investigations, the sample model Second Request indicates the types of data that will be requested. Specification 1. D. requires the submission of a data map for the company. Specification 10 requires the company to identify each electronic database used or maintained by the company in connection with any Relevant Product and for each database to identify its characteristics and properties (subpart a.); compile and submit one or more data sets from the database (subpart b.), and to provide a data dictionary for each data set submitted (subpart c.)
We understand that complying with data requests can be expensive and burdensome. However, the burdens of compliance with data requests can often be significantly reduced if the parties engage staff at the earliest opportunity. We generally want to discuss data issues before we write the Second Request or a CID so we can tailor the requests to the data that are available and relevant to the investigation. This may not always be feasible prior to issuing the Second Request or CID, but even after the Second Request or CID is issued we are interested in negotiating with parties to limit the requests to the data relevant to the investigation.
It is generally very useful for us to have discussions with the personnel of the parties (i.e., business and IT people, economic consultants) most knowledgeable about how the company maintains the relevant data. Thus, we strongly encourage the parties to make such personnel available as early as possible.
One of the goals of early interaction is to determine what data are available, its suitability and its form, to make it easier and faster for the parties to provide the required data and for 51ɫstaff to analyze it.; 51ɫstaff will need to understand, among other things, how the companies’ various files work together, the type of data kept in each data file and how that data is arranged and organized (e.g., the unit of record-keeping, etc.). To the greatest extent possible, data maps[2] and data dictionaries[3] should be provided to facilitate this discussion. 51ɫstaff’s ability to negotiate the data specifications will be limited until the structure and content of the parties’ data holdings can be assessed. Providing samples of the data is very helpful as it allows 51ɫstaff to determine what data is available as well as what may be responsive.
Data submissions should be segregated from document submissions and provided separately as per the instructions in the CID or Second Request. The model Second Request discusses the formats of data submissions, and the size of a data submission and its format should be discussed with staff in advance of submittal of data in all competition investigations.
The model Second Request directs parties not to submit any Sensitive Personally Identifiable Information (“Sensitive PII”) or Sensitive Health Information (“SHI”) prior to discussing the information with 51ɫstaff. With respect to data submissions, it is very important that the parties or their consultants identify any data sets that contain sensitive PII or SHI to 51ɫstaff and segregate those data files from the remainder of the data submission. The 51ɫcannot accept data with Sensitive PII or SHI unless it is responsive to a Specification in a Second Request or CID. If any data set or document responsive to a particular Specification contains unresponsive Sensitive PII or SHI, the parties should redact the unresponsive Sensitive PII or SHI prior to producing the data or document. If submissions containing Sensitive PII or SHI remain necessary, they should be delivered directly to the BE Data Support Center at the address below or through an Accellion link.
BE Data Support Center
51ɫ
600 Pennsylvania Ave., NW
Room H-285
Washington DC, 20580
Interaction on Economic Analysis: What the Bureau of Economics Will Do
Bureau of Economics staff expects to have a dialogue with the parties throughout the investigation on theories of harm and potential justifications for behavior (i.e., efficiencies). In the early stages of an investigation, including before the Second Request or CID issuance, if possible, BE staff will discuss with the parties economic, financial, and data issues, including theories that are being considered (although at such an early stage it should be expected that any discussions of theories would likely be quite general).
Theories may change as the investigation proceeds and additional information is learned. As part of the ongoing dialogue regarding economics, BE staff will endeavor to be as transparent as possible about theories of harm and economic analyses, subject to confidentiality and investigational limitations.
Parties can expect BE staff to:
- Describe, at least in general form (subject to confidentiality and potential litigation issues), results obtained from BE staff’s empirical analyses.
- Review empirical analyses submitted by the parties.
- In some cases, propose empirical analyses BE staff believe might be useful for the parties to conduct.
The parties and their consultants should bear in mind that evaluation of material presented by the parties’ economists may take some time. BE staff will re-engage the parties and their consultants as needed during their evaluation and engage in a more thorough discussion of the analysis when they have completed their evaluation.
Interaction on Economic Analysis: What We Expect from the Parties and Their Consultants
In order to maximize the usefulness of interactions on economic issues, parties should:
- Always inform BE and other 51ɫstaff in advance if they plan to bring economists to meetings with staff or Bureau management and identify those economists.
- Present empirical work as early in the investigation as is practicable. Waiting to present empirical work until later stages of the investigation is not as productive.
- Provide white papers or other analyses sufficiently prior, i.e., several days, to meetings with BE or other 51ɫstaff, including management, to provide staff enough time to absorb the analyses and results. This will encourage a more productive dialogue at the meeting.
- Present empirical work in a manner that provides enough detail so that BE staff can understand the methodology used to conduct the analysis and can review the results of such analyses. Whenever possible, empirical presentations should include:
- confidence intervals around point estimates and/or their standard errors;
- a discussion of potential data endogeneity issues and how they are addressed; and
- multiple robustness checks.
- Provide BE staff with the data used in any analysis presented, both in its original raw form and compiled in the manner used by the parties. Ideally, these data would be submitted prior to any meeting so that staff can raise questions at the earliest possible time for a useful exchange.
- Provide the programs and codes used to compile/transform the data and to calculate the results, indicating the specific versions of the software programs used (e.g., STATA Version 9 or STATA Version 14, etc.).
PowerPoint presentations that only provide a summary of an analysis (without details on the methodology or the specifics of the results) are likely to be too sparse on details to be informative and will generally not be given great weight.
To the extent that the parties wish to present new analyses in meetings with Bureau management and Commissioners, they should provide the data and backup material, including computer programs, to staff in sufficient time for staff to brief Bureau management and Commissioners regarding the new analyses.
Time is of the essence. 51ɫeconomists need time to assess any analyses submitted. If there is insufficient time, generally the submitted analyses cannot be given much, if any, weight in BE staff recommendations or in Bureau management or Commission decisions.
[1] /system/files/attachments/merger-review/may2019_model_second_request_final.pdf (May 2019)
[2] A data map is an organized list, schematic, diagram, or other representation sufficient to show where and how the Company stores all physical and electronic information in its possession and includes, among other items, the physical and logical network topology of the Company’s computer systems.
[3] A data dictionary should include: i) a list of field names and a definition for each field contained in the data set; (ii) the meaning of each code that appears as a field value in the data set; and (iii) the primary key in the data set or table that defines a unique observation. (Specification 10. C of the model 2nd request)