-
1,263 (32%) of Eversource leaks are lost. 2,938 (17%) of NationalGrid leaks are lost.What is a "lost leak"?
-
1,263 (32%) of Eversource leaks are lost. 2,938 (17%) of NationalGrid leaks are lost.What areas areaffected by lost leaks?
-
1,263 (32%) of Eversource leaks are lost. 2,938 (17%) of NationalGrid leaks are lost.How did we trackdown lost leaks?
LostLeaks
What do you mean "lost leak"? Learn More
These natural gas leaks weren't repaired, but they vanished from public utility company data between 2014 and 2015.
So what happened to them? Learn More
Some may have been omitted due to clerical error, or repaired via replacing gas mains. It appears that many have simply been deleted from the record. That's 1,263 lost Eversource leaks (32%) and 2,938 lost National Grid leaks (17%).
How can you tell? Our Process
Utilities are required to release data on leaks and repairs. We checked their data, and found that parts of it were missing. Our process, our code, and the data are publically documented.
Has anyone else done work on this?
Yes. The Home Energy Efficiency Team did a manual analysis of the data. Our automated study has reached similar conclusions.
Where Are Lost Leaks?
On the map below, view leak locations
Colored leaks are "lost," while leaks in gray are "found."
"Found" leaks were not listed as repaired, but were reported in the 2015 data.
Gas leaks are reported in Grades 1 to 3, with 1 being the most severe and 3 being the least severe. The radius of the circle is based on how long ago the leak was reported. Newer leaks have smaller circles while older leaks have larger ones .
Click the circles on the map to view more information about specific lost leaks, and click the checkboxes on the left side of the map to view each grade individually.
Leak Legend
Grade 1 Lost: Severe LeakGrade 2 Lost: Moderate Leak
Grade 3 Lost: Mild Leak
All Found Leaks
What causes Natural Gas Leaks?
What is a “lost leak”?
In Boston, natural gas is often transported in old and leak-prone cast iron, wrought iron, or unprotected steel pipes. Natural gas is dangerous, especially when it escapes from pipes into surrounding neighborhoods
Utility companies in Massachusetts have had to report natural gas leaks and repairs since 2014.
We call leaks “lost” if, after being recorded in 2014, they disappeared from the public record for 2015 without having been logged as repaired. We analyzed the data to match repairs to leaks
Between 2014 and 2015, it appears that 2,938 of National Grid's and 1,263 of Eversource's leaks went missing. That is 17% and 32% (respectively) of leaks that were omitted from the public record without any information about their being repaired.
Why? Some leaks may have been omitted due to clerical errors. Others might have been omitted due to standard scheduled replacements to pipes, which would have happened regardless of the presence of a leak, effectively repairing the leaks. These standard updates can't be logged in the public record as repairs, so the leaks remedied by planned pipe replacements would simply be deleted from the public record
However, if scheduled pipe replacements accounted for all (or most) of the missing gas leaks we discovered, we would expect these leaks to be clustered along streets where pipes were replaced between 2014 and 2015. But we don't see an obvious pattern like that. Instead, the map looks like confetti. We have no way of knowing whether anything has been done to address these leaks.
This is a lot of unaccounted for gas. Something is amiss.
- edf.org/leaks-problem Natural Gas Leaks A Problem?
- edf.org/city-snapshots Boston: Snapshot Natural Gas Leaks
Why are we concerned about Natural Gas Leaks?
Gas leaks harm our home and planet
Gas leaks contribute to greenhouse gas buildup. Natural gas leaking from the local distribution system accounts for 60 percent to 100 percent of the region’s emissions of methane, one of the most insidious heat-trapping greenhouse gases.
According to a March 28, 2016 study from BU, roughly 15% of leaks are potentially explosive and 7% of leaks contribute 50% of the Metro Boston Area's methane emissions.
- bu.edu Risky Leaks From Gas Mains
Similarly, the paper 'Fugitive Methane Emissions From Leak-Prone Natural Gas Distribution Infrastructure in Urban Environments' concluded that 15% of leaks surveyed qualified as potentially explosive. All leaks must be addressed, as even small leaks cannot be disregarded as ‘safely leaking.’
- sciencedirect.com Leak-prone gas infrastructure
Natural gas is known to kill trees and shrubs by causing root asphyxiation in the soil, and the methane and carbon monoxide present in natural gas are well known asphyxiants and are dangers to human health.
- stateimpact.npr.org Hidden Culprit For Dead Trees
- extension.umd.edu Natural Gas Injury Trees and Shrubs
The cost is not paid by utilities
Our aging natural gas distribution system short-changes customers.
In 2015, a Harvard-led team estimated that the Boston region loses 15 billion cubic feet of natural gas worth $90 million dollars each year. This means 2.7 percent of the gas that is brought to the Boston region never makes it to customers, but instead escapes into the atmosphere (and soil). These loss rate estimates are more than twice as high as those made by government regulators and utilities.
- seas.harvard.edu Gas infrastructure releases methane
Instead of focusing on repairing all existing leaks, utility companies are focused on building new pipelines. The utility companies can then shift the cost of the new pipelines onto their customers through pipeline tariffs (taxes on all natural gas customers) which subsidize the construction of new natural gas pipelines across the state.
- bostonglobe.com Baker/Healey face off pipeline tariff
Because utility companies have local monopolies, customers have no choice but to pay these higher prices. In the video below, National Grid's director of Gas Pipeline Safety is acknowledging that utility companies charge the consumer for the gas that is lost due to leaks.
- MA Law Section 144
(a) There shall be uniform natural gas leaks classification for all gas companies
(b) (1) Gas companies shall assess a grade to all reported natural gas leaks based on the system provided in this section.
(2) A Grade 1 leak
shall be a leak that represents an existing or probable hazard to persons or property. Grade 1 leaks require repair as immediately as possible and continuous action until the conditions are no longer hazardous. The gas company shall immediately schedule a completion of repairs and the condition shall be kept under continuous surveillance until the hazard or source of the leak is eliminated. Whenever appropriate and feasible, a gas company shall notify the fire department and chief law enforcement officer in each city or town where a Grade 1 leak is identified.
(3) A Grade 2 leak
shall be a leak that is recognized as non-hazardous to persons or property at the time of detection, but justifies scheduled repair based on probable future hazard. The gas company shall repair Grade 2 leaks or replace the main within 12 months from the date the leak was classified. All Grade 2 leaks shall be reevaluated by a gas company at least once every 6 months until eliminated; provided, however, that the frequency of reevaluation shall be determined by the location and magnitude of the leakage condition.
(4) A Grade 3 leak
shall be a leak that is recognized as non-hazardous to persons or property at the time of detection and can be reasonably expected to remain non-hazardous. The gas company shall reevaluate Grade 3 leaks during the next scheduled survey, or within 12 months from the date last evaluated, whichever occurs first, until the leak is eliminated or the main is replaced. A municipal or state public safety official may request a reevaluation of a Grade 3 leak prior to the next scheduled survey, or sooner than 12 months of the date last evaluated, if the official reasonably believes that the Grade 3 leak poses a threat to public safety.
(c) Projects on Public Ways
Upon the undertaking of a significant project on a public way exposing confirmed natural gas infrastructure, and with sufficient notice, a municipality or the commonwealth shall submit written notification of the project to a gas company. The gas company shall survey the project area for the presence of Grade 1 or Grade 2 leaks and set repair and replacement schedules for all known or newly detected Grade 1 or Grade 2 leaks. The gas company shall ensure that any shut off valve in the significant project area has a gate box installed upon it or a reasonable alternative that would otherwise ensure continued public safety and that any critical valve that has not been inspected and tested within the past 12 months is verified to be operational and accessible. The gas company shall provide the repair and replacement schedule of gas leaks to the municipality or the commonwealth
(d) School Zones
Gas companies shall prioritize any pipeline repairs required under this section for gas leaks detected within a school zone. For the purposes of this section, ''school zone'' shall mean on or within 50 feet of the real property comprising a public or private accredited preschool, accredited Head Start facility, elementary, vocational or secondary school.
(e) Gas Company Response and Reporting
As part of the annual service quality standards report required by section 1I, each gas company shall report to the department the location of each Grade 1, Grade 2 and Grade 3 leak existing as of the date of the report, the date each Grade 1, Grade 2 and Grade 3 leak was classified and the dates of repairs performed on each Grade 1, Grade 2 and Grade 3 leak. A gas company shall specify any reclassification of previously identified leaks in its annual report. Gas leak information shall be made available to any municipal or state public safety official upon written request to the department.
(f)
The department shall promulgate regulations necessary to implement the uniform natural gas leak classifications as specified in this section and shall oversee and monitor company response and reporting.
A Data Story
How We Found Lost Leaks
[Step 1]
Obtain data from the State of Massachusetts Office of Energy and Environmental Affairs
We identified records via docket number and downloaded them from public utilities. The data was typically somewhat irregular and in pdf format.
[Step 2]
Parse PDFs using SmallPDF.com and PDFTables Audrey Schulman, HEET Co-founder
Audrey converted the pdfs into excel spreadsheets and cleaned the spreadsheet data by hand. You can some of this data in our repository.
[Step 3]
Convert data to CSV and standardize
Albert Carter converted the excel spreadsheets into CSV format and ran them through a Python program to standardize and sanitize the data. Reported leaks were geolocated using the Google Maps API. The latitude and longitude information was inserted into DataHub.
If you are interested in looking at the raw CSV data. You can view them on DataHub. You can also find our data sanitization code on GitHub
[Step 4]
Identify lost leaks
We compared the lists of 2014 leaks and 2014 repairs. If a repair happened near a pre-existing leak (approximately 100 ft), the leak was counted as repaired and removed from analysis.
Any remaining leaks without a matching repair were classified as unrepaired 2014 leaks.
We cross-referenced our list of unrepaired 2014 leaks with the list of 2015 leaks. We identified leaks appearing in both the 2014 and 2015 data as the same if their record date and address matched. If there was a match, the 2014 unrepaired leak was accounted for and therefore not a lost leak.
The remaining unrepaired 2014 leaks that did not match with any 2015 leaks were identified as lost leaks.
Michael wrote this section of the code. You can find it on GitHub
[Step 5]
Publish the data on DataHub
Each leak was associated with an ID number. On the map, inside each popup is a link to view the data in Datahub.
For example: This is a leak originally reported by Eversource with the id of 6. Please note that the "location_type" for the coordinates is "ROOFTOP" but that the leak is most likely in the street or parking lot around the address.
[Step 6]
Make the Map
Lost leaks were populated onto this map Allie built with Leaflet.js.
Who is Involved?
These are the natural gas utilities we've analyzed
National Grid's 2014 Annual Service Quality Report PDF included the location of each leak existing as of February 2015, the date each leak was reported, the dates of repairs performed on each leak in 2014, and current classification. National Grid is made up of Boston Gas Company, Colonial Gas Company, and Essex Gas Company.
- National Grid's 2014 Annual Service Quality Report
Other (unanalyzed) natural gas utilities in Massachusetts:
About Us
This isn't our day job and we don't get paid to do this.
We hope our work will inform public discussion about natural gas leaks, their dissapearances, and their effects.
Michael Webber
Clustering
Allie S.
Map and Website
Al Carter
Data Wrangling, Clustering, Project Management
Michael Webber
Creative and determined problem solver and builder, committed to excellence in his work and personal interactions.
Michael Webber has experience programming applications and supporting them, either working as part of a team and on his own while travelling across the country and working remotely. He is currently seeking interesting and challenging IT projects.
Follow Michael Webber
Allison Sichling
Web developer interested in interactive experiences, with a BA in Fine Art.
Allison graduated from General Assembly’s full-time Web Development Immersive boot camp in December 2015. Generally passionate about creating things, she enjoys solving problems, learning, collaborating, drawing, painting, sewing, and making art with LEDs.
Follow Allie
Albert Carter
Programmer and research staff member at the MIT CSAIL Big Data Initiative and LivingLab.
Albert Carter has been at MIT for 4 years. Most recently, he worked on the DataHub, a data collaboration and publishing tool.
If you are a reporter with questions about the data, please email arcarter@csail.mit.edu. If you are a data scientist or programmer with questions about sanitization, point identification, or the processed data, please either email arcarter@csail.mit.edu or file a github issue in the appropriate repository.