--- title: "Understanding the Night-Sky? Developing AI-Enabled System for Exploring Night-Light Usage Patterns" authors: - "Jakob Hederich" - "Shreya Ghosh" - "Zeyu He" - "Prasenjit Mitra" doi: "10.24963/ijcai.2023/833" url: "https://ijcai.org/proceedings/2023/833" pdf: "https://ijcai.org/proceedings/2023/0833.pdf" venue: "Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence" year: 2023 pages: "7131-7134" track: "Demo Track" keywords: - "Data Mining: DM: Data visualization" - "Data Mining: DM: Applications" - "Data Mining: DM: Mining spatial and/or temporal data" --- # Understanding the Night-Sky? Developing AI-Enabled System for Exploring Night-Light Usage Patterns Jakob Hederich, Shreya Ghosh, Zeyu He, Prasenjit Mitra *Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence*, pages 7131-7134. DOI: [10.24963/ijcai.2023/833](https://doi.org/10.24963/ijcai.2023/833) ## Abstract We present a demonstration of nighttime light pattern (NTL) analysis system. Our tool named NightVIEW is powered by an efficient system architecture to easily export and analyse a huge volume of spatial data (NTL), image segmentation and clustering algorithms to find unusual NTL patterns and identify hotspots of excess night light usage as well as finding semantics of cities. ## Full text ## 1 Introduction While attempting to analyze night-time light data obtained from the Earthdata dataset published by NASA [Suomi-NPP, 2011] we observed several technical challenges that we seek to address. First, downloading and managing the data 1 needed significant effort. Second, analyzing the data automatically was time-consuming. Third, end-users often asked for visualizations of the patterns we were observing. No existing system addresses these challenges in a real-time setting to answer the questions we wanted to pose (outlined below). ## 1.1 Example Application Questions - Q1: What demographic factors influence NTL data and to what extent? For example, NTL may depend upon economic activity, population, income and wealth, environmental consciousness, etc. - Q2: Can we detect the boundaries of a city, urban area, or metropolitan area? How does it differ from the political boundaries and why? - Q3: Where are the differences with demographic data unusual? For example, discrepancies between government claimed economic activity and growth of NTL. This could be due to propaganda by governments that provide false data or due to environmental consciousness of cities. Given these questions and the limitations discussed above, we decided that a semi-automatic visual analytic solution would be the best to address these goals under these challenges. Hence, we have built NightVIEW, a visual analytics toolkit that can be used to display and analyze nighttime light data. Our system is primarily based on efficient NTL data search, access, and segmenting the regions on a map based on NTL intensity values. While NightVIEW presently includes generic functions and can be used in applications ranging from finding hotspots of unusual NTL usages, computing temporal trends, our system can be used to answer important spatio-temporal contrast questions such as the impact of COVID-19 on our cities and rural areas, how NTL patterns changes during natural calamities and man-made disaster like Russia-Ukraine war (Fig. 1), the Turkish-Syrian earthquake, etc. with a click of a few buttons. Our attempts at automating these questions resulted in sub-optimal answers and took significant amount of computation and time to run; hence we decided on a visual analytic solution (Fig. 2). 1 Example. NTL monthly composite data (2015), GeoTiff image datasize: ≈140GB Figure 1: NTL during Russia invasion where it is shown that nation gone dark (significant reduction of lights in March, 2022 after Russia invasion) ![Image](https://ijcai.org/proceedings/2023/fulltext/833/image_000000_a43347b3fc66b9ba7f662e7e32e99614c53717f6f8c72a2f7e78c67644aaf380.png) ## 1.2 Contributions To the best of our knowledge, ours is the first system 2 for visualization and computing services provided for analyzing nighttime light (NTL) patterns across the world. The NTL data is primarily provided by NASA/NOAA Suomi National Polar-orbiting Partnership (Suomi NPP) project [Suomi-NPP, 2011] where dataset is available as Gzipped GeoTIFFs and users have to pre-process a huge volume of data by segmenting the images. The only visualization that is closest to our tool is an initiative by International Dark Sky association [Sky, 2023] where they presented a map 3 of only dark sky places, however, their system does not provide the visualization and computing features of NightVIEW and is not robust as ours. Our sophisticated pipeline enables: 2 There are strands of research on using NTL for poverty estimation and NTL impact analysis [Jean et al., 2016; Levin et al., 2020; Falchi et al., 2016; Ni et al., 2020; McCallum et al., 2022; Chen et al., 2022; Falchi et al., 2023; Morelli et al., 2023], however, none presented an end-to-end NTL visualization and computing platform. We omitted the discussion of these papers due to pagelimit. Figure 2: NightVIEW consists of three major visual and computing components: (A) NTL data visualization (temporal scale: 2014-present) and OpenStreet Map view. Our gradient based coloring enables a user to create different ranges of NTL intensity and visualize the map. (B) Reverse geocoding (Fig. 2.(b)) is a mapping feature of NightVIEW that facilitates to selection of the present region based on the zoom level (country, state/ county, city). Users can export the NTL data in GeoTIFF or png format by selecting the region of their choice. (C) Analytics features support finding the temporal trend of NTL usages at different parts of the world, displaying the distribution of data (using boxplot) and clustering NTL values to get hotspots of unusual NTL usages (Fig. 2.(c)). Fig. 2(d-e) show how NTL can differentiate different regions based on night data semantics, e.g., North Korea and South Korea have a significant amount of difference in NTL distributions as expected. ![Image](https://ijcai.org/proceedings/2023/fulltext/833/image_000001_568ba33a574b9b28946223d41f5ce1b4ddf371ee2b17deac45deb9230fb16b70.png) - Usability: Users can easily search (both spatial and temporal (see Fig. 2)), access and export images (GeoTiff and png formats). Our tool mitigates the additional effort of pre-processing and segmenting images. Additionally, our reverse geocoding feature provides an intelligent search and selection of regions. - Unique interactive design: NightVIEW provides an interactive visualization dashboard where users can search and select any region, define threshold values for selecting the areas with NTL intensity more than user-defined threshold. Additionally, they can segment regions by [3 https://www.darksky.org/our-work/conservation/idsp/finder/](https://www.darksky.org/our-work/conservation/idsp/finder/) providing NTL intensity ranges (or breakpoints) and identify them using different colors. - Computing: NightVIEW enables finding spatial clusters in real-time, and temporal statistics of NTL patterns using boxplot, clusters and statistics tabs at the left panel. - System pipeline: NightVIEW's efficient system architecture provides real-time visualization and computation of NTL data, which is a compute and time-intensive tasks due to huge volume of imagery datasets. We believe that the system is generic and adaptable and can be easily used on any similar spatio-temporal data for different applications. ## 2 NightVIEW: A Brief Overview We are using the cloud-free monthly composite NTL data [Elvidge et al., 2017] of VIIRS (Visible Infrared Imaging Radiometer Suite Day/Night Band (DNB)). NightVIEW's frontend is developed using React application and backend is supported by NodeJS Typescript application and multithreading for resolving parallel requests. ## 2.1 User-interactive Visualization Users can search any place by name or by providing latitude, longitude at the top left panel and indicate the year at the bottom (See Fig. 2), and NightVIEW recenters the map based on the place and shows NTL map on that specific time. Next, user can download the data by using export feature of our tool. By using the reverse geocoding feature, user can doubleclick at any place on the map and NightVIEW will select the closest boundary of the place based on zoom level (Fig. 2(b)). Next, a user can define any threshold and segment the image to identify region-of-interests such as, business settlements with high NTL values, residential area, or forest areas. Furthermore, she can define any range of values (coloring tab at left panel) and NightVIEW will segment the NTL map based on the value-range and corresponding colors defined by the user. The box-plot, clustering and statistics feature help her analyse NTL usages, cluster hot-spots and summarize the temporal trend of NTL usage patterns and look at NTLs corresponding to external events (COVID-19 [Stokes and Rom´an, 2022], disaster etc.). Fig. 3 depicts that NTL usage is the minimum (compared to NTL intensity in 20162019) in 2020 in Seattle, which correlates with lockdown period and people going to their home place. Figure 3: Computing temporal trend of NTL usages at any search region. The figure shows the impact of COVID-19 on NTL ![Image](https://ijcai.org/proceedings/2023/fulltext/833/image_000002_bdf1bfa758b6d3334e507e1812a4d1427343eda7c8016426591296ccb40c6842.png) ## 2.2 Threshold-based Approach The user can adjust a light threshold to observe areas with NTL above and below that threshold within a bounding box as follows. Note that if we want to contrast a city or a metropolitan area with its suburb or rural areas, we may need to use different threshold. For example, differentiating New York City and its suburbs will need a different threshold than when we differentiate Hannover with its neighborhood (See Fig. 4). We can choose multiple ranges and assign different colors to those ranges to differentiate a core of a city from its less lit areas. ## 2.3 Marching Squares Algorithm We have deployed the marching squares algorithm [Maple, 2003] to generate the contours from NTL map (See Fig. 4(b)). There are two types of contours, namely, Isobands: multipolygons where the NTL intensity value is greater than a threshold and Isolines: boundary region of isobands. The process computes the NTL value of each cell in the grid independently followed by calculating a cell index using comparisons of the contour level(s) with the NTL intensity values at the cell corners. Next, we use the pre-built lookup table to describe the output geometry for the cell. Figure 4: (a) Selecting a threshold value helps in identifying specific features (such as, cities) and (b) clusters returned by our system. ![Image](https://ijcai.org/proceedings/2023/fulltext/833/image_000003_7220d6f5748d3e2e5b204f2d48d93684a0b4da40f02ea5cc4d5fc0d1c5f8d584.png) ## 2.4 Spatial Scan Statistics Spatial scan statistics [Kulldorff, 1997] has been deployed to detect spatial and space-time NTL clusters (based on the intensity value), and to see if they are statistically significant. Using this, we find out the biggest contagious regions within which NTL data exhibits anomalous behaviour. ## 2.5 Data-driven Interpretation Capacity NightVIEW provides a platform for integrating heterogeneous data sources (population density, income level that can be uploaded and analyzed using our tool). We will demonstrate the following: Q1: City Boundary Detection. We will show how the threshold feature and the countour map view can be used to detect city boundaries and areas within a city that are more "green" (could be physically or literally). Q2: How to identify specific map features, e.g., road, rail networks, POIs (business settlements, residential area etc.) from NTL data? A2: Highways and other busy roads show up in NTL data connecting cities. Different POIs have different NTL characteristics. We use these clues. Q3: How relevant data sources such as, population density, average income level of different counties/ states are related to NTL usages pattern at different time-scales? A3: Using our GeoJSON upload and overlay features, we can overlay more than one layer on the NTL map. End-users can find out the temporal correlations on how such contextual data effects overall NTL usages at different parts of the world. ## 2.6 Demonstration Plan and Conclusion We plan to showcase varied functionalities of our system using the frontend and API calls using a Jupyter Notebook environment and highlight a number of interesting findings of different urban and non-urban regions. The system is not only limited to perform only NTL data analytics, it can also be used for day satellite imagery analytics and combining NTL and daylight images in achieving complex goals and more generally, any data having spatial and temporal attributes. ## Acknowledgements This research was funded by the Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILabor with grant No. 01DD20003. ## References 1. [Chen et al., 2022] Jiandong Chen, Jialu Liu, Jie Qi, Ming Gao, Shulei Cheng, Ke Li, and Chong Xu. City-and county-level spatio-temporal energy consumption and ef- ficiency datasets for china from 1997 to 2017. Scientific Data, 9(1):101, 2022. 2. [Elvidge et al., 2017] Christopher D Elvidge, Kimberly Baugh, Mikhail Zhizhin, Feng Chi Hsu, and Tilottama Ghosh. Viirs night-time lights. International journal of remote sensing, 38(21):5860-5879, 2017. 3. [Falchi et al., 2016] Fabio Falchi, Pierantonio Cinzano, Dan Duriscoe, Christopher CM Kyba, Christopher D Elvidge, Kimberly Baugh, Boris A Portnov, Nataliya A Rybnikova, and Riccardo Furgoni. The new world atlas of artificial night sky brightness. Science advances, 2(6):e1600377, 2016. 4. [Falchi et al., 2023] Fabio Falchi, Salvador Bará, Pieranto- nio Cinzano, Raul C Lima, and Martin Pawley. A call for scientists to halt the spoiling of the night sky with artifi- cial light and satellites. Nature Astronomy, 7(3):237-239, 2023. 5. [Jean et al., 2016] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to pre- dict poverty. Science, 353(6301):790-794, 2016. 6. [Kulldorff, 1997] Martin Kulldorff. A spatial scan statis- tic. Communications in Statistics-Theory and methods, 26(6):1481-1496, 1997. 7. [Levin et al., 2020] Noam Levin, Christopher CM Kyba, Qingling Zhang, Alejandro Sánchez de Miguel, Miguel O Román, Xi Li, Boris A Portnov, Andrew L Molthan, An- dreas Jechow, Steven D Miller, et al. Remote sensing of night lights: A review and an outlook for the future. Re- mote Sensing of Environment, 237:111443, 2020. 8. [Maple, 2003] Carsten Maple. Geometric design and space planning using the marching squares and marching cube algorithms. In 2003 international conference on geometric modeling and graphics, 2003. Proceedings, pages 90-95. IEEE, 2003. 9. [McCallum et al., 2022] Ian McCallum, Christopher Con- rad Maximillian Kyba, Juan Carlos Laso Bayas, Elena Moltchanova, Matt Cooper, Jesus Crespo Cuaresma, Shonali Pachauri, Linda See, Olga Danylo, Inian Moorthy, et al. Estimating global economic well-being with unlit settlements. Nature Communications, 13(1):2459, 2022. 10. [Morelli et al., 2023] Federico Morelli, Piotr Tryjanowski, Juan Diego Ibáñez-Álamo, Mario Díaz, Jukka Suhonen, Anders Pape Møller, Jiri Prosek, David Moravec, Raphaël Bussière, Marko Mägi, et al. Effects of light and noise pol- lution on avian communities of european cities are corre- lated with the species' diet. Scientific Reports, 13(1):4361, 2023. 11. [Ni et al., 2020] Ye Ni, Xutao Li, Yunming Ye, Yan Li, Chunshan Li, and Dianhui Chu. An investigation on deep learning approaches to combining nighttime and daytime satellite imagery for poverty prediction. IEEE Geoscience and Remote Sensing Letters, 18(9):1545-1549, 2020. 12. [Sky, 2023] International Dark Sky. International Dark Sky. https://www.darksky.org/our-work/conservation/ idsp/, 2023. [Online; accessed 12-Jan-2023]. 13. [Stokes and Román, 2022] Eleanor C Stokes and Miguel O Román. Tracking covid-19 urban activity changes in the middle east from nighttime lights. Scientific reports, 12(1):8096, 2022. 14. [Suomi-NPP, 2011] Suomi-NPP. Visible Infrared Imaging Radiometer Suite (VIIRS). https://www.earthdata.nasa. gov/eosdis/daacs/laads, 2011. [Online; accessed 05-Nov- 2022].