Methodology

Employer Survey Methodology
Data for this year’s report is derived from the United States Energy and Employment Report (USEER). The 2024 USEER methodology relies on the most recently available data from the Bureau of Labor Statistics (BLS) Quarterly Census of Employment and Wages (QCEW, third quarter 2023), the BLS Unemployment Situation Table B-1 monthly reports, together with a detailed supplemental survey of businesses across Massachusetts designed and conducted by BW Research Partnership (BW Research or BW) on behalf of the Department of Energy (DOE). Due to anomalies in BLS QCEW 2023 Q4 motor vehicle and motor vehicle component parts manufacturing employment caused by the September 2023 United Auto Workers (UAW) strike, revised 2023 Q3 and 2022 Q3 data is used. The 2024 USEER includes revised 2022 employment numbers (from the 2023 USEER) for motor vehicle and motor vehicle component parts and any other employment number that relies on an aggregation that includes motor vehicle and motor vehicle component parts employment, including total energy at the national and state level.  The methodology employed for the survey has been used for local, state, and federal energy-related data collection and analysis for more than a decade.
 
The survey uses a stratified sampling plan that is representative by the North American Industry Classification System (NAICS) industry codes, business size, and geography to determine the proportion of businesses that work with specific clean energy-related technologies, as well as the proportion of workers in such businesses that work with the same. Survey results are analyzed and applied to the existing public QCEW data series, constraining the potential universe of energy businesses and employment.
 
The survey was administered by telephone with approximately 28,100 outbound calls in Massachusetts, as well as by web with more than 11,800 emails sent to potential participants across the state. The phone survey was conducted by ReconMR and BW Research. The web instrument was programmed internally, and each respondent was required to use a unique ID in order to prevent duplication.
 
The sample was split into two categories, referred to as the known and unknown universes. The known universe includes businesses that have previously identified as energy-related, either in prior research or some other manner, such as membership in an industry association or participation in government programs. These businesses were surveyed census-style, and their associated business and employment totals were removed from the unknown universe for both sampling and resulting employment calculations and estimates. As performed on an annual basis, BW Research cleaned, deduplicated, added to, and refined its database to reflect churn (companies out of business, moved, no longer in clean energy, etc.), unverified (no answer, answering machine, fast-busy, disconnect, etc.), verified, and other available demographic tags (industry, technology, sub-technology, size, etc.). _¹

In addition to cleaning the original known clean energy database, BW Research also supplemented with industry association contact lists by technology, new companies from the unknown database that took the previous year’s survey, and contact lists from subcontractors. BW Research also appended contact information, including six-digit NAICS codes, contact, employment, and location information.
 
The unknown universe includes hundreds of thousands of businesses in potentially clean energy related NAICS codes across agriculture, mining, utilities, construction, manufacturing, wholesale trade, professional services, and repair and maintenance. Each of these segments and their total reported businesses (within the Bureau of Labor Statistics QCEW) were carefully analyzed by size (employment – provided by the Census Bureau’s County Business Patterns) and state to develop representative clusters for sampling. 
 
In total, 1,088 businesses in Massachusetts participated in the survey effort, with 414 providing full responses to the survey. Since 2015, 4,405 employers have provided full responses to the survey. These responses were used to develop incidence rates among industries, as well as to apportion employment across various industry categories in ways currently not provided by state and federal labor market information agencies. The margin of error is +/- 4.80% for Massachusetts at a 95% confidence interval.
 
Commodity flow data was collected and analyzed for the USEER 2024 report, however, since the industries covered do not transport products that are related to clean energy, they were left out of the analyses for the Massachusetts Clean Energy report. ²
 
All data in the USEER rely on the BLS QCEW data for the end of the third quarter of 2023, and the BLS Unemployment Situation Table B-1 monthly reports through December 2023, except for motor vehicle employment. Employment extrapolations are based off BLS QCEW and survey data, resulting in totals that carry precise decimal values. As a result, some employment totals in the report may sum differently due to rounding. The USEER survey was administered between December 21, 2023, and March 28, 2024, and averaged 18 minutes in length.

Gross State Product Data
Gross State Product (GSP) is presented for both the overall clean energy economy and each of the value chain categories; the data supports the economic index portion of each of the BW indices. The input-output data for GSP is derived from data from the U.S. Bureau of Economic Analysis (BEA) by NAICS code.  GSP is an important measure of economic activity, measuring the value and flow of goods and services produced in the economy. 
 
Each NAICS industry’s Gross State Product is multiplied by the ratio of clean energy businesses to all businesses within the NAICS segment. This produces the Gross State Product contribution of businesses engaged in clean energy activities. To generate the clean energy proportion, this figure is further reduced by multiplying it by the mean reported revenues attributed to clean energy goods and services from the survey. 

Economic Contribution Analysis
BW Research used IMPLAN, an input-output model that traces spending and infrastructure developments through the economy to determine the economic impact of the clean energy jobs in 2023 to the State of Massachusetts. The cumulative effects of the jobs are quantified, and the results are categorized into direct, indirect, and induced effects.Direct effects show the change in the economy associated with overall clean energy jobs, or how the industry experiences the change. Indirect effects include all the backward linkages, or the supply chain responses as a result of clean energy jobs. Induced effects refer to household spending and are the result of workers who are responsible for the direct and indirect effects spending their wages.

Model Input
To develop the economic model in IMPLAN, BW Research identified the clean energy jobs in the State of Massachusetts disaggregated by NAICS code, as calculated for the 2024 Massachusetts Clean Energy Industry Report (i.e. in-scope jobs). All jobs in 2023 were converted from NAICS code to IMPLAN code and added as input to IMPLAN. The study area was set as the State of Massachusetts and the event year was set to 2022 since 2023 IMPLAN data was not yet available.

Model Output
Results from the economic contribution analysis include employment ³ (full- and part-time jobs), labor income, value added, and output. Output is the gross revenue of the industry. Value added is the total output minus the cost of inputs from outside the firm; it is a measure of the contribution to the Gross State Product made by the companies or industries. Labor income includes all forms of employment income, such as employee compensation (wages and benefits) and proprietor income (i.e. payments received by self-employed individuals and unincorporated business owners).

Addressing Supply and Value Chain Double-Counting
One important step in the analysis was to ensure the IMPLAN model, by quantifying direct and indirect jobs, would not double-count the in-scope jobs (i.e. jobs from the 2024 Massachusetts Clean Energy Industry Report data). Since the 2024 Massachusetts Clean Energy Industry Report data includes value chain jobs and IMPLAN also calculates the supply chain employment in the indirect impacts, there could be some double-counting. When using jobs as an input (as we do in our analysis), as compared to sales or expenditures, there is the additional challenge of determining whether the jobs should be considered direct or indirect jobs (i.e., part of the supply chain economic activity). For example, construction jobs entered in IMPLAN have impacts through the entire value chain (e.g., purchasing ENERGY STAR boilers). If the supply chain jobs are entered into IMPLAN as direct jobs and the model also accounts for them as an indirect impact of the new construction jobs, then there is double-counting, and the impacts will be inflated.The challenge faced by using jobs as the economic model input was to determine the number of in-scope energy jobs that should be counted in IMPLAN as direct or indirect jobs, without eliminating activity that was not initially included in the 2024 Massachusetts Clean Energy Industry Report data. To address the double-counting challenge, the research team adopted the following methodology:

Step 1: Run detailed, individual models for each in-scope industry by IMPLAN code
The research team ran detailed models for each in-scope industry by IMPLAN code and analyzed the indirect jobs created by each in-scope industry. By creating individual models for each IMPLAN code, the team gained a better understanding of the jobs created in different indirect industries by each in-scope industry. 
 
Step 2: Compare the number of direct + indirect jobs by industry estimated in IMPLAN with the initial in-scope jobs
This step included looking at the number of direct + indirect jobs by industry and comparing them with the initial in-scope jobs by industry. By doing this, the team analyzed the supply chain jobs that are created by each in-scope industry, which helped adjust the in-scope jobs based on the number of direct and indirect jobs created in IMPLAN.
 
Step 3: Adjust (decrease) the initial in-scope jobs based on the direct + indirect jobs calculated in the IMPLAN model
This step included reducing the in-scope jobs based on the direct + indirect jobs that IMPLAN estimated. For example, if based on the construction in-scope jobs, IMPLAN calculated that x number of indirect jobs were created in wholesale trade, we excluded that x number from the initial in-scope jobs in wholesale trade since they were already accounted for as indirect jobs of construction.
 
This important step addresses the fundamental challenge of this study which is determining the proportion of in-scope jobs that should be considered direct or indirect (supply-chain) jobs. By following this methodology, we avoided double-counting the in-scope jobs that would occur if all of them were considered direct jobs.

Step 4: Re-run the IMPLAN model with the “adjusted” in-scope jobs by industry
After running several individual and collective models, the last step was to re-run the IMPLAN model one more time with the adjusted number of in-scope jobs by industry.
 
Final Output
Direct = “adjusted” in-scope industry jobs by sector to account for the indirect jobs IMPLAN calculates.
Indirect = indirect jobs produced by the model which include in- and out-of-scope industries
Induced = all induced jobs calculated in IMPLAN

Employer Hiring Difficulty
Massachusetts clean energy employer hiring difficulty for the 2024 Massachusetts Clean Energy Industry Report was extrapolated from national hiring difficulty by technology, applied to MassCEC defined technology segments, and averaged with Massachusetts all energy hiring difficulty from the 2024 USEER. The following table includes the MassCEC technology with the corresponding USEER technology, the USEER difficulty percentages applied, the weighted difficulty (by employment in each MassCEC segment), the Massachusetts all energy difficulty from the 2024 USEER State Report, and the final Massachusetts averaged clean energy employer hiring difficulty. This methodology differs from previous year’s analyses, which used extrapolations directly from the Massachusetts USEER database.