Beyond the Surface: Data-Driven Solutions for Water Quality 


An award-winning data-driven solution has already been successfully deployed in New Zealand to help improve water quality. Holly Foreman of Auckland Council and Steve Couper of Mott MacDonald examines how Safeswim, a World Health Organisation recognised solution, is leading the way in effective water quality management and strengthening public trust.

Bathing water quality is a critical public issue facing water companies globally. The UK governments’ Storm Overflows Discharge Reduction Plan requires all water companies to demonstrate that there is no local adverse ecological impact from the overflow. Water companies must have improved 75% of storm overflows by 2035, and 100% by 2045. Plus, storm overflows will not be permitted to discharge above an average of 10 rainfall events per year by 2050.

The mass deployment of sensors, including Event Duration Monitoring (EDM), at all UK storm overflows creates opportunities for digital solutions to improve the way water quality is both managed and subsequently communicated to communities.

What is Safeswim?

In 1995, Auckland Council implemented a water quality management programme to provide an assessment based on retrospective water quality sampling. Driven from a need to overcome limitations related to retrospective sampling and delayed reporting, Safeswim (www.safeswim.org.nz) was developed and deployed in the summer of 2017. The data-driven solution enables more proactive predictions by integrating real-time modelling, artificial intelligence, big-data analytics, visualisation, and information dissemination to the community.

The Safeswim programme was also used as a public relations and education tool by the Council and its water utility, Watercare, to surface and drive the public understanding of the ageing infrastructure problem. Like most large urban areas with combined and ageing sewer systems, Auckland suffers from storm overflow, inflow and infiltration (II) and diffuse pollution as a result of surface water runoff. Safeswim provided a platform for robust stakeholder management allowing Auckland Council to improve public awareness and understanding of the trade-offs around the level of infrastructure investment against the associated level of service and the environmental, social and cultural outcomes associated with that level of investment. Providing greater public transparency around the performance of the City’s wastewater and storm water network was a bold move. However, it paid off, building trust and shifting public perception around the complexity of the problem.

The need for a smart solution to deliver a real time decision support system

Safeswim arose from an acknowledgement that the existing programme had the potential to misrepresent the public health risk at bathing beaches. Traditional monitoring approaches relied on retrospective sampling data and delayed reporting, which was both time-consuming and involved practical limitations around when and where samples could be taken. This led to significant potential for under-estimation of the frequency of contamination events, creating a potential “false sense of security” to swimmers.

Safeswim is a web-based digital twin which provides real-time information on the water quality and the associated public health risk of swimming at Auckland's beaches. The tool uses predictive analytics to integrate data from various sources and translates them into simple risk indicators for the public. The platform has improved the accuracy of water quality predictions from less than 20% to greater than 80%.

Scale of programme

Since its creation and delivery by a Mott MacDonald led team in 2017, the programme has continued to develop under the leadership of Auckland Council’s Healthy Waters team with support from supply chain partners. The underlying Mott MacDonald MoataTM platform enables the combination of multiple data sources and selected real-time analytics from across the supply chain. The water quality digital twin system integrates large scale climate data and predictions, wastewater network data with real-time modelling and AI to understand and predict the impacts of wastewater overflows and diffuse run off on receiving waters. It includes comprehensive long-term monitoring of 200 priority sewer overflows allowing Safeswim to predict water quality at 209 beaches across 3,200km of coastline. The platform is accessed by over 400,000 Aucklanders and tourists each year, helping them make informed decisions about safely enjoying bathing waters.

Inputs provide the base for machine learning predictions

MoataTM ingests and processes over eight billion data points daily from multiple sources, including:

  • Real-time network infrastructure performance data – wastewater flow/overflow gauges and SCADA data from pumping stations.
  • Real-time storm overflow monitoring data enhanced with AI for early identification of blockages and forecasting wet weather overflows.
  • Rain fall and rain radar data, and ensemble forecasts.
  • Tide, wind and solar radiation forecasts.
  • Data from nine network hydraulic models and two hydrodynamic models of the harbour.
  • Water quality sampling data and location specific predictive models spanning 209 beaches.
  • Surf lifesaving knowledge related to local physical marine hazards.

This base data is used to drive machine learning algorithms to produce bathing water quality predictions up to three days ahead. Comprehensive alarming enables proactive intervention to prevent minor blockages from causing overflows and to highlight any dry weather overflows.

Integration and analytics make it smart

On their own, each of the data sources only provides part of the puzzle. However, integrated in a digital twin platform with associated analytics, the system has become “smart”.

Figures 1, 2 and 3 set out how the various input data sources, the modelling and analytics developed are layered and integrated onto MoataTMto provide the predictions. The diagrams present an overview around the integration and inter-relationships for the base data and associated analytics that make up the Safeswim predictions.

Figure 1: Data (input) layer

Figure 2: Analytics and logic layer

Figure 3: Learning layer

Data analysis and validation

Two types of model inform the Safeswim water quality predictions. They compare forecasted enterococci concentration against the bathing water standard of 280 colony forming unit (cfu) per 100ml as indicators of environmental contamination1:

  1. A white box model is run every six hours for three day forecasting. It consists of a contaminant load model for the wastewater network and a three-dimensional receiving environment model built using DHI’s MIKE 3 FM for the Waitematā Harbour.
  2. A black box (machine learning) model runs in real-time in MoataTM to reflect the latest observations and weather forecasts available. This consists of a statistical model built from the learnings of historical sampling and rainfall data at each site.

Safeswim’s hydrodynamic and historical sampling learning water quality models are overseen by an independent panel of four public health experts from academia, consultancy, the public health service, and the National Institute of Water and Atmospheric (NIWA) research. Other analytical components are targeted at improving the reliability of the system which is a critical aspect to build public trust. The system uses performance standards published by the United States Geological Survey (USGS) to direct the deployment of predictive models, providing guidelines for model accuracy, specificity and sensitivity. A comprehensive alarm system informs data providers of issues identified with the quality or the availability of their dataset to maintain high data availability throughout the swimming season.

The delivery team

Auckland Council’s Healthy Waters team recognised the need for effective collaboration and proactively sought asset owners and public partners with a willingness to contribute. Stakeholders were open and transparent, sharing access to and information about asset condition and performance.

Healthy Waters drove the supply chain collaboration through matching existing suppliers from their infrastructure project office with diverse experts in water quality science, modelling, big data analytics and data management, communications and web services. They were able to build a dedicated “best for project” delivery team from across six organisations.

Starting initially with a six month deployment for 62 priority sites, the team has worked collaboratively to extend coverage and enhance functionality over the past seven years. Most site configuration is now automated by custom-built templates and before each swimming season additional sites and new data pipelines are considered.

Customer outcomes

The outputs of Safeswim are presented through a purpose-built globally accessible web platform. Predictions of safe swimming locations are depicted by a simple green or red risk indicators driven from the MoataTM digital twin platform.

Figure 4: Safeswim public facing web site - www.safeswim.org

Note: Safeswim predictions of where it will be safe (or not) to swim as a simple green or red risk indicators driven from the MoataTM digital twin platform. This figure highlights the predicted conditions at Point Chevalier on the evening of 10th April 2024. The map shows the current risk level with the predictions (water quality and weather forecasts) shown as changing over the next two days.

The web site provides a risk assessment based on the water quality prediction and the New Zealand public health bathing water guidelines using enterococci as the indicator organism. The risk assessment is forecast for a three-day period alongside the weather, climatic and other local safety and hazard information. Details about the beach environment including photos and information about the facilities are also presented providing an effective public communication tool.

Figure 5 and 6 present an example of the MoataTM web layout along with the data inputs and prediction outputs. They illustrate the back end interface for the Auckland Council team that integrates the data streams, over eight billion data points per day, models and asset information.

Figure 5: Rainfall predictions over the Waitematā Harbour, Watercare sewer trunk mains and the Safeswim monitoring and beach prediction locations.

Note: Rainfall predictions over the Waitematā Harbour, the Watercare sewer trunk mains and the Safeswim monitoring and beach prediction locations. This is the MoataTM platform and presents the back end interface for the Auckland Council team that integrates the data streams (over 8 billion data points per day), models and asset information. Highlighted is the Point Chevalier beach location.

Figure 6: Example of microbiological predictions generated from historic sampling and weather forecast data for Point Chevalier beach in the inner Waitematā Harbour

Note - Example of microbiological predictions generated from historic sampling and weather forecast data for Point Chevalier beach in the inner Waitematā Harbour. The outputs from these predictions are simplified into risk indicators (red and green) are displayed via the public customer facing web portal presented above.Acknowledgements

The Author would like to acknowledge the Safeswim project partners: Auckland Council (Healthy Waters, Watercare, Regulatory and Compliance, Auckland Council’s Research and Evaluation Unit (RIMU)), The Auckland Regional Public Health Service, and Surf Life Saving Northern Region.

The Author would also like to acknowledge the delivery partners: Mott MacDonald, DHI, Translate Digital, Fingermark, Puhoi Stour, Weather Radar.

Ongoing development

The Safeswim programme now drives further development of new analytical capabilities. One current example is further improving blockage identification from wastewater anomaly detection, creating causal network graphs to analyse how rainfall in one part of the network can influence wastewater flow in other areas. Combining this with synthetic training data and a human-in-the loop capability is leading to greatly improved performance for capturing a wide range of anomalies that are difficult to detect using off-the-shelf libraries.

Lessons learned and sector acknowledgement

Developing Safeswim was grounded in the deep understanding by the team of wastewater and storm water networks and through close collaboration with the asset owners. The combination of domain knowledge with digital and data science expertise was crucial to unlocking value from large volumes of sensor and other data, enabling the team to pinpoint blockages and forecast high flows across the network. Safeswim is now referenced as best practice in the World Health Organisation (WHO) Guidelines on Recreational Water Quality and has won multiple awards.

Conclusion

The Safeswim programme is now well established and in its seventh year of operation informing more than 400,000 Auckland community members and visitors each year about the water quality across more than 200 bathing locations. Safeswim has also opened the door to a greater public understanding and acceptance that more needs to be invested in ageing infrastructure if we are to achieve an appropriate balance between public spending and environmental and social outcomes. This has led to increased funding and political support to improve water quality. A Council-initiated water quality targeted rate for the Auckland region is now in place, with over NZ$400 million in additional funding being focused on infrastructure to drive water quality improvements.

Acknowledgements

The Author would like to acknowledge the Safeswim project partners: Auckland Council (Healthy Waters, Watercare, Regulatory and Compliance, Auckland Council’s Research and Evaluation Unit (RIMU)), The Auckland Regional Public Health Service, and Surf Life Saving Northern Region.

The Author would also like to acknowledge the delivery partners: Mott MacDonald, DHI, Translate Digital, Fingermark, Puhoi Stour, Weather Radar.

References

Brown, N., Schollum, A. 2018. Safeswim, Wai Ora and the NPSFM. Water New Zealand’s 2018 Stormwater Conference, Queenstown, New Zealand.

Neale, M. et al. 2018. Safeswim: A Sea Change in Assessing Beach Water Quality Risk. Water New Zealand’s 2018 Stormwater Conference, Queenstown, New Zealand.

Steve Couper, Nick Brown, Dukessa Blackburn-Huettner, Martin Neale (2019) Safeswim – Or Troubled Waters? Auckland’s Smart Customer Engagement Initiative, OzWater Water Conference 2019.

World Health Organization (2021) Guidelines on Recreational Water Quality

Dr Margaret Leonard, Dr Carla Eaton (2021), Recreational Water Quality Guidelines Update, Ministry of Health.

Helsel, D.R., and Hirsch, R.M. (2002) Statistical methods in water resources: U.S. Geological Survey Techniques of Water-Resource Investigations, book 4, chap. A3, last accessed June 2023 at http://pubs.er.usgs.gov/pubs/twri/twri04A3

Francy, D.S., and Darner, R.A.(2006) Procedures for developing models to predict exceedances of recreational water quality standards at coastal beaches: U.S. Geological Survey Techniques and Methods 6–B5, 34 p, last accessed June 2023 at https://pubs.usgs.gov/tm/2006/tm6b5/pdf/tm6B5_web_rev120706.pdf

Share this article

Become a member

Whether you are studying, actively looking to progress your career, or already extensively experienced, our membership will add value and recognition to your achievements. We can actively help you progress throughout your career.

Become a member

View our events

We organise a wide portfolio of UK and international thought leading events, providing an industry recognised forum for debate, CPD and sector networking. These events also support our policy work and inform key initiatives.

View our events