FLOOD-PREPARED: A NOWCASTING SYSTEM FOR REAL-TIME IMPACT ADAPTION TO SURFACE WATER FLOODING IN CITIES

Extreme rainfall events pose an ever increasing threat to cities due to the potential for surface water flooding resulting in damage to properties and major disruption of transport systems. Modern sensor networks offer enormous potential for the real-time monitoring of urban systems and potentially allow improved situational awareness of impeding hazards and their impacts such as flooding. However, monitoring in itself is not enough if we are to be able to adapt in in real-time to hazards. Systems are required that allow analytics and models, that feed of real-time observations, to make predictions of impacts and suggest adaption options ahead of the hazard event. The Flood-PREPARED project is developing a system for real-time adaption to surface water flooding. The system comprises of advanced spatiotemporal models of rainfall, surface water flooding and road traffic impacts. These models are linked and orchestrated within into a Big Data workflow that allows events to be simulated using emerging rainfall data recorded by a short range weather radar. This approach allows nowcasting to be undertaken where predictions of surface water inundation and impacts on the road network can be predicted ahead of the rainfall event reaching the city; thus providing the ability for an improved adaptive response to the actual event.


INTRODUCTION
There is an increased awareness of the risks faced by cities from surface water flooding due to intense rainfall events (Kendon et al. 2014). For example, in the UK alone 2 million people live within a 1-in-200 year pluvial flood zone; a figure expected to increase to 3.2 million by 2050 with population growth and climate change (Houston et al. 2011;Committee on Climate Change, 2016). Surface flooding in cities is primarily a result of high intensity rainfall that can lead to severe disruption, damage to property and infrastructure, and potentially loss of life. For example, pluvial floods in Istanbul in 2009 resulted in 30+ deaths and in excess of $70million in damage, and annually in the UK it is estimated that the cost of surface water flooding could be as high as £2.2bn (DEFRA, 2011).
The accurate spatial prediction of surface water flooding within cities remains a significant challenge; due to the highly dynamic temporal and spatial variability of intense rainfall. This is further compounded by the spatially complex and heterogeneous configuration of urban conurbations in terms of their buildings, roads, kerbs, drainage networks and topography, resulting in complex surface water run-off patterns and urban hydrology. Alongside the difficulty of predicting urban surface water flooding from intense rainfall is the challenge of understanding the subsequent impacts of this in real-time (Cole et al. 2013). Traditionally, impact assessment is undertaken post-event. However, first responders (emergency services), estate and site managers, and civil authorities require forecasts and real-time updates on both the spatial pattern of surface water flooding and also the impact of this (DEFRA, 2014). Moreover, certain impacts, such as transport disruption, power failure, damage to key infrastructure, are themselves inherently dynamic and can rapidly cascade and diffuse spatially through a city, amplifying the initial magnitude of the impact (DfT, 2014). In order to develop real-time and ahead-of-event risk mitigation to surface water flooding it is critical to be able to monitor and understand the spatiotemporal interactions between the dynamics of the surface water flood hazard and its impacts.
Modern multi-scale multi-source sensing and monitoring approaches potentially allow cities to move from being reactive to proactive with regards to surface water flood risk and mitigation; providing a mechanism by which better informed decisions can be made to minimise disruption and damage, and protect citizens. However, to achieve this, new analytics and models are required to improve the prediction and response to urban surface flooding from intense rainfall events. Moreover, these need to be combined in such a manner that monitoring and prediction of the urban system can be achieved in real-time and at scale (i.e., across the entire urban system at an appropriate spatial and temporal level of granularity).

Flood-PREPARED (Predicting Rainfall Events by Physical
Analytics of REaltime Data) is an example of such a systemsbased approach to the real-time prediction and mitigation of surface water flooding from intense rainfall events. It aims to provide a step change in the management of surface water flooding; allowing earlier and more geographically precise warning, operational decisions in real-time, and continuous situation awareness and intelligence.

FLOOD-PREPARED REQUIREMENTS
A 'systems' approach to surface water flooding in cities requires a start-to-end analytical and computational framework that rigorously evaluates and integrates real-time data and information from across the flooding systemincluding the sources of flooding (e.g. rainfall radar), pathways of flooding (e.g. sewer flow data, CCTV images of flooded locations), and flooding impacts (e.g. traffic disruption). Moreover, advanced analytics and methods for uncertainty handling are required to process and assimilate this data into state of the art hydrodynamic surface water flood modelling and flood risk models to provide spatial-temporal impacts forecasting and estimation. With regards to surface water flooding a systems approach requires the following features: 1. The ability to combine local high frequency weather radar observations, with regional/national monitoring and forecasts, along with geospatial social messaging tracking (e.g., tweets of events occurring) to provide improved early warning of potential intense rainfall trajectories and its impact; 2. The use of real-time data feeds from local weather stations, rainfall gauges and sewer gauging to undertake real-time data assimilation within hydrodynamic surface water flood models for improved prediction; 3. The ability to employ CCTV data feeds of surface water locations and social media feeds to validate in real-time the emergent patterns of hydrodynamic surface water flood models; 4. Integration of spatially heterogeneous sensor data feeds on flows and movements (e.g., traffic) with CCTV data for improved understanding of the temporal dynamics of impacts; 5. The ability to couple CCTV monitoring with social media data feeds, such as twitter and bespoke citizen science apps, to understand better citizen reaction and response to emerging impacts for improved future hazard mitigation; and, 6. The ability, via statistical or machine learning data assimilation approaches, to use knowledge from previous events, including modelling result-sets of both flooding and impacts, to improve 'ahead of event' response from the site to the city-scale for future 'events'.
Within Flood-PREPARED these specific requirements are being addressed by the development of a real-time data monitoring and management framework that seamlessly integrates heterogeneous quantitative sensor measurements (e.g., weather radar, weather stations, sewer gauging) with realtime monitoring (e.g., CCTV, traffic flows, pedestrian movements) and qualitative social media feeds. These sensor data flows will be used to parameterise (i) highly granular spatiotemporal hydrodynamic surface water flood models for cities, that use physical analytic principles (integrated physical models with statistical analytics) to incorporate real-time data for improved surface water flood prediction, and (ii) new analytical tools for the enhanced spatiotemporal recognition, assessment and understanding of surface water flood impacts for improved real-time decision making and risk mitigation. These new analytics and modelling approaches will be embedded within 'big data' computational workflows that integrate the data monitoring and management, modelling and analytical components to enable a full 'systems' modelling capability for real-time surface water flood risk in cities.

Rainfall Model
In Flood-PREPARED a spatiotemporal rainfall model is used to generate a spatial field of rainfall intensity ahead of the actual event reaching the city. The model is parametrised and calibrated using weather radar and a high density sensor network of ground based rain gauges. Rain gauges are also used to evaluate the performance and re-calibrate the inferred rainfall field in real-time. The model works on a 36Km 2 area centred on the weather radar location and can generate a regular lattice grid of rainfall intensity at a spatial resolution of 500m. Spatiotempoarl rainfall dynamics are derived using a numerical (forward-centred finite difference) solution to a collection of Stochastic (Partial) Differential Equations, where advection and diffusion are the main driving forces behind the evolution of the system (Stroud et al. 2010). Observations, both weather radar and the rain gauge, are assumed to be noisy, and potentially biased observations of the latent rain field. Within the model parameters are introduced to define this relationship and their values inferred using an ensemble Kalman smoother (Evensen and Van Leeuwen, 2000). The accuracy of forecasts are evaluated using the Energy Score (Gneiting and Raftery, 2007) to tune algorithm performance in real-time.

Surface Water Flood Model
HiPIMS (High-Performance Integrated hydrodynamic Modelling System) is a two-dimensional (2D) physically-based flood model. In a flood event, water depth is generally much smaller than horizontal inundation extent and the flow hydrodynamics can be mathematically described using the 2D depth-averaged shallow water equations (SWEs) (Liang and Marche, 2009). In order to predict the transient and complex flow hydrodynamics across different flow regimes that may occur during a flood event, HiPIMS solves the governing equations using a Godunov-type finite volume numerical scheme (Liang, 2010;Xia et al., 2017;Xia and Liang, 2018). The numerical scheme has been applied in accurate and stable simulations of rainfall-induced overland flows of urban areas (Xia and Liang, 2018;Xing et al., 2019). In order to substantially improve the computational efficiency for largescale simulations and real-time forecasting, HiPIMS is implemented on multiple GPUs. Parameters considered in the model include friction and infiltration rates, which are estimated from a land cover map of the model domain. Figure 1 shows the output generated by HiPIMS for the city of Newcastle upon Tyne UK. The resulting surface water flood inundation and depth map has a spatial resolution of 2m. The map shows the depth of surface water flooding for a 53mm rainfall event that occurred over 90 minutes. The resulting spatial locations of flood were verified against a similar magnitude event that occurred in Newcastle upon Tyne in 2012.

Road Transport Impact Analytics
Currently two forms of real-time monitoring of road transport impacts are assessed within Flood-PREPARED. The first is the use of Deep Learning (DL) methods applied to CCTV data feeds to capture and predict current and future disruption at fixed locations across the urban domain. The second is the use of spatial network modelling to simulate how the impacts of surface water flooding propagates across the entire urban road network.

CCTV Analysis of Vehicle Disruption
Traffic flow analysis and prediction consists training and finetuning a convolution neural network (R-CNN) to recognize vehicles under different weather conditions from existing CCTV images (Peppa, et al. 2017; Figure 2). The trained R-CNN is then used to estimate vehicle counts. In Flood-PREPARED multi-scale temporal training data has been employed where 24-hour and 7-day vehicle counts are used to estimate vehicle numbers for (i) weekend or weekday; (ii) day of the week; (iii) period of the day before; and (iv) whether before or after midnight (Figure 3). The R-CNN prediction also incorporates spatial dependencies from neighboring CCTV sensors into traffic predictions by using an OD matrix between CCTV locations. The OD matrix is constructed by calculating the shortest routes along the road network between all CCTV locations. This is then imported into the data preparation and training step to select traffic data from the four closest CCTVs that are as evenly distributed as possible to the north, east, south and west directions of a target camera.

Spatially Modelling Flood Impacts
The CNN approach to vehicle count estimation allows Flood-PREPARED to evaluate surface water flooding impacts and traffic disruption at discrete locations across the urban road network. However, it does not allow an estimation of how the impacts of surface water flooding will propagate over the road network. To evaluate this, Flood-PREPARED uses a 'nowcasting' network modelling approach to estimate the potential impacts across the entire road network in real-time. Surface water depths from the surface water flood model are used to recognise sections of the road network which are impassable due to flooding, and parts of the road network which are still passable but where surface water flooding impacts freeflow traffic speeds.
A threshold of 300mm is applied to recognise roads within the network which will be impassable by cars (Pregnolato, et al. 2017b). This is achieved by intersecting the surface water flood depths at a particular time interval, extracting footprints of surface water above 300mm and then spatially assessing if their footprint is sufficiently large to prohibit any vehicle movement along that part of the road (Figure 4). However, surface water depths below 300mm can have a significant impact on the road network due to reductions in speed and potential build-up of congestion. To account for this within the 'nowcasting' framework, surface water flood depths are derived for each road segment across the entire road network. These depths are then used to statistically derive the passable safe free-flow speed for any road segment using the approach developed in Pregnolato et al. (2017a) and refined in Pregnolato et al. (2017b).
The final output of this processing is a road status map which shows which roads will be impassable and the reduced speed (compared to free flow speeds) of other parts of the road network due to the presence of surface water (Figure 4). The real-time road speed impacts resulting from the surface water flood model predictions can be further utilised within the modelling framework. For example, Figure 5 shows the implementation of a dynamic real-time generalised least cost path analysis between the current location of a hypothetical vehicle and the destination it wishes to reach (Ford et al. 2015).
As the surface water flood model produces time-stamped predictions of surface water, this can be used to update dynamically the road status map; updating closed roads, free flow speeds and the least cost path to the destination. Figure 5. Result of applying a generalised least cost analysis that accounts for disruption to traffic free-flow speeds and flooded roads for part of the city of Newcastle upon Tyne UK.

WORKFLOW DEVELOPMENT
The individual models and analytics of Flood-PREPARED have been combined into a workflow deployment. Figure 6 shows the architecture of the workflow system which consists of two main components: a Dataflow manager and a Job manager. The system orchestrates various models across different computation resources including GPU and CPU clusters through the Job manager and the dependencies among the models are built via the Dataflow manager. The Job deployment module that feeds into the Job Manager is used to decide where to deploy and how to execute the models over the available computing resources. A set of APIs have been developed to allow users to deploy their models over specified resources, including whether to deploy on CPU or GPU, geolocation of the deployment and required computation capacity, the number of cpu cores required, and the required memory size of each microservice (containerised model). If users do not specify their required resources, the system randomly selects suitable resources for the models, i.e., a model will be deployed on any resources that matches its execution requirements. The Flow control module determines when and how models are executed within a workflow taking into account workflow dependencies. During workflow execution models may dynamically change in terms of size and arrival rate. Therefore, the Flow monitoring module is used to adaptively execute the models to meet current performance requirements.
The Resource provision module ensures that sufficient computing resources are available to meet the deployment requirements. The system utilises Kubernetes APIs to provision the resources. Kubernetes is utilised as it has excellent resource optimisation mechanisms for computer cluster deployment (Burns et al., 2016). As noted, the Flow control module chains different models as a workflow. Kafka (Kreps et al., 2011) is used as the message hub to orchestrate the inputs and outputs of each model. Kafka is a high performance and reliable streamprocessing software platform that follows a publish and subscribe messaging pattern. Thus, each model has one or more producers and consumers to exchange messages via Kafka.

A PROTOTYPE IMPLAMENTATION
On the 28 th June 2012 Newcastle upon Tyne UK experienced a significant convective rainfall event resulting in 53mm of rain over a period of less than 90 minutes (the equivalent to a months' worth). Local surface water flooding caused over £8m worth of damage and flooded over 500 homes in the city. The rainfall event also resulted in significant disruption to Newcastle's transport systems with major roads being inundated and the closure of regional metro and rail stations.
The rainfall, flood and impact models (Section 3) and the Big Data workflow (Section 4) were used to build a start-to-end workflow that looked to recreate the 2012 flood event within a real-time nowcasting system. Data from Newcastle Universities Urban Observatory (https://urbanobservatory.ac.uk/) was used to parametrise and calibrate the models. The Urban Observatory comprises of over 300+ environmental sensors distributed across Newcastle upon Tyne and its surrounding area, including a high density network of weather stations, rain gauges, traffic monitoring CCTV and a short range weather radar.
As weather radar data was not available for the 2012 rainfall event, more recent Newcastle Urban Observatory weather radar data was synthetically modified using rain gauge data to create synthetic input weather radar data-sets that effectively replicated the spatiotemporal dynamics of the 2012 event. Figure 7 shows the final workflow developed, including the capability to replace the synthetic rainfall data stream with real-time weather data directly from the Newcastle Urban Observatory.
In the Newcastle pilot implementation individual models were containerised using Docker (https://www.docker.com/) and a linear start-to-end workflow orchestration developed using a combination of Kubernets (https://kubernetes.io/) and Argo (https://blog.argoproj.io/). Rainfall data is streamed/presented to the system via Kafka (https://kafka.apache.org/), which filters the incoming data and monitors it for the trigger event in order to invoke the workflow. The trigger event can either be the recognition of an emergent convective rainfall event in the weather radar data or rain gauge readings at locations outside Newcastle which indicate a storm event is moving towards the city.
Once an event has been triggered the workflow is initiated and the weather radar data passed to the rainfall model to be used for initiation of the model. Upon completion, the rainfall model provides the surface flood model with spatially distributed rainfall values at the 500m resolution across the city, which are used to run the surface water flood model. In the current implementation, the resultant surface water flood predictions are used to target which CCTV monitors should be used for vehicle counting. The workflow does not currently integrate the spatial modelling of how flood impacts will propagate across the road network (Section 3.3.2); although this will be implemented in the next round of development. In the prototype, when the surface water flood model is executed a websocket is opened to a client dashboard mapping application developed in Leaflet. At each time-stamp of the surface water flood model real-time updates (in terms of the model timestamp interval) of the spatial pattern of flooding on the road network is passed to the

CONCLUSION
New environmental sensor networks offer significant potential to allow the improved monitoring of the impacts of extreme events such as convective rainfall induced surface water flooding. However, if we are to improve our response and realtime adaption to such events these need to be coupled with advanced models that can be deployed in real-time and provide nowcasting predictions of locational magnitude of impacts and of the events themselves.
The Flood-PREPARED workflow developed in this paper shows the potential of coupling sensor network data feeds with advanced analytics and models using Big Data workflow orchestration tools such as Docker, Kubernets and Argo. In the case of this work, this has allowed a prototype linear workflow and demonstrator to be developed and deployed. Figure 7. The prototype Flood-PREPARED workflow developed for a simulated extreme rainfall event of 53mm for the city of Newcastle upon Tyne UK.
However, while the development of this porotype is promising, significant work remains before an operational widely deployable system exists. In particular, the development of workflows that are non-linear which allow forking and farming out of new model instances as the systems dynamics are changing in real-time is a major challenge. However, such tools are required as the both the hazard system itself (in this case the rainfall event), the impact (surface water flooding) and the urban system (in this case the road transport system) are themselves evolving simultaneous to the predictions that are being generated by a running workflow instance. Next stages of the Flood-PREPARED project will investigate how advanced data assimilation and the ability to develop iterative Big Data workflows with feedbacks can be employed to allow incorporation of knowledge of how the event, impact and impacted system are changing.