publications

2024

1. Effects of occupant thermostat preferences and override behavior on residential demand response in CityLearn

   Kathryn Kaspar, Kingsley Nweye, Giacomo Buscemi, Alfonso Capozzoli, Zoltan Nagy, Giuseppe Pinto, Ursula Eicker, and Mohamed M. Ouf

   Energy and Buildings Sep 2024

   Abs URL

   As space heating accounts for 54% of annual residential electricity consumption in Quebec, demand response programs specifically target load shifting through the automated control of thermostat setpoints during peak hours. On a district scale, varied thermostat preferences and setpoint override behaviors can have an impact on the success of the demand response program. This study examines two unique occupant types (Average, Tolerant) in terms of thermostat setpoint preferences as well as three different occupant Levels-of-Detail (LoDs) and analyzes their effects on the energy flexibility provided during demand response periods. For our baseline scenario, LoD 1, a static setpoint schedule is used and there is no control of the heat pump, while LoD 2 and LoD 3 incorporate thermostat setbacks during demand response events. LoD 2 assumes the occupant is comfortable within 2°C from the setpoint while LoD 3 allows the occupant to override the DR setbacks. We estimate the flexibility services provided by a ten-house residential community through the automated control of heat pumps during a three-month winter period, and we implement and simulate our study in CityLearn using reinforcement learning-based control for district-level energy management. When comparing LoD 3 to LoD 1, electricity cost was reduced by approximately 12% and net electricity consumption was reduced by approximately 17% during demand response periods. Likewise, we find that LoD 2 could overestimate savings in net electricity consumption, cost, and peak demand by 5% compared to LoD 3. The number of hours where the indoor temperature deviated more than Image 1 from the setpoint occurred for less than 5% of the timesteps on average for the 10 buildings for LoD 3 while still achieving significant net electricity consumption reductions, thus highlighting that we can provide energy flexibility services to the grid while balancing occupant thermal comfort. Finally, the agents learned optimal decision-making that reduced the number of overrides across the training episodes for both Average and Tolerant occupants. We thus present a multi-agent framework as a means for addressing various occupant setpoint preferences and override behaviors for the control of heat pump systems at the neighborhood level.

2. Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts

   Kingsley Nweye, and Zoltan Nagy

   In Sep 2024

   Abs URL

   It is challenging to coordinate multiple distributed energy resources in a single or multiple buildings to ensure efficient and flexible operation. Advanced control algorithms such as model predictive control and reinforcement learning control provide solutions to this problem by effectively managing a distribution of distributed energy resource control tasks while adapting to unique building characteristics, and cooperating towards improving multi-objective key performance indicator. Yet, a research gap for advanced control adoption is the ability to benchmark algorithm performance. CityLearn addresses this gap an open-source Gym environment for the easy implementation and benchmarking of simple rule-based control and advanced algorithms that has an advantage of modeling simplicity, multi-agent control, district-level objectives, and control resiliency assessment. Here we demonstrate the functionalities of CityLearn using 17 different building control problems that have varying complexity with respect to the number of controllable distributed energy resources in buildings, the simplicity of the control algorithm, the control objective, and district size.

3. EVLearn: Extending the CityLearn Framework with Electric Vehicle Simulation

   Tiago Fonseca, Luis Ferreira, Bernardo Cabral, Ricardo Severino, Kingsley Nweye, Dipanjan Ghose, and Zoltan Nagy

   In Apr 2024

   Abs URL

   Intelligent energy management strategies, such as Vehicle-to-Grid (V2G) and Grid-to-Vehicle (G2V) emerge as a potential solution to the Electric Vehicles’ (EVs) integration into the energy grid. These strategies promise enhanced grid resilience and economic benefits for both vehicle owners and grid operators. Despite the announced prospective, the adoption of these strategies is still hindered by an array of operational problems. Key among these is the lack of a simulation platform that allows to validate and refine V2G and G2V strategies. Including the development, training, and testing in the context of Energy Communities (ECs) incorporating multiple flexible energy assets. Addressing this gap, first we introduce the EVLearn, a simulation module for researching in both V2G and G2V energy management strategies, that models EVs, their charging infrastructure and associated energy flexibility dynamics; second, this paper integrates EVLearn with the existing CityLearn framework, providing V2G and G2V simulation capabilities into the study of broader energy management strategies. Results validated EVLearn and its integration into CityLearn, where the impact of these strategies is highlighted through a comparative simulation scenario.

4. The CityLearn Challenge 2023 Dataset

   Kingsley Nweye, Max Langtry, Ruchi Choudhary, and Gyorgy Zoltan Nagy

   Apr 2024

   URL

2023

1. CityLearn v2: An OpenAI Gym environment for demand response control benchmarking in grid-interactive communities

   Kingsley Nweye, Kathryn Kaspar, Giacomo Buscemi, Giuseppe Pinto, Han Li, Tianzhen Hong, Mohamed Ouf, Alfonso Capozzoli, and Zoltan Nagy

   In Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2023

   Abs URL

   As more distributed energy resources become part of the demand-side infrastructure, it is important to quantify the energy flexibility they provide, as well as identify the best control strategies to accelerate their real-world adoption. CityLearn provides an environment for benchmarking of simple and advanced control algorithms in virtual grid-interactive communities. The updated CityLearn v2 environment introduced here extends the v1 environment to provide load shedding flexibility through heating ventilation and air conditioning power control coupled with a data-driven temperature dynamics model. The updated environment also includes the functionality to assess the resiliency of control algorithms during power outage events.

2. SCALEX: SCALability EXploration of Multi-Agent Reinforcement Learning Agents in Grid-Interactive Efficient Buildings

   Yara Almilaify, Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2023

   Abs URL

   Renewable energy transition and decarbonization pose significant challenges for grid-interactive efficient building communities. The optimization of intermittent renewable energy can be achieved using advanced control architecture and energy storage, enhancing energy flexibility. Reinforcement learning (RL) offers potential solutions, but its scalability and computational demands in large-scale settings remain unclear. This paper examines the scalability of Soft-Actor Critic (SAC) in multi-agent systems, comparing decentralized-independent SACs and centralized SACs using CityLearn, an OpenAI Gym environment. We consider neighborhoods consisting of 2 to 64 single-family residential buildings, each equipped with cooling and heating storage devices, domestic hot water storage devices, electrical storage devices, and solar PV systems. Our findings suggest that independent controllers outperform the centralized controller with increasing number of buildings. We also show that the performance on the building level can differ from the aggregated performance.

3. Heterogeneous Multi-Agent Reinforcement Learning for Grid-Interactive Communities

   Allen Wu, Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2023

   Abs URL

   Homogeneous Multi-Agent Reinforcement Learning (MARL) is well studied in games, robots, and simulations. What has not been fully explored is the effectiveness of Heterogeneous MARL in the building space. Heterogeneous MARL has been proven to be more effective than Homogeneous MARL in terms of performance in games. Heterogeneous MARL also has the added benefit of being a more realistic simulation because no two buildings can be expected to react in the same way. Here, we implement the MARLlib library with the CityLearn environment to analyze the benefits of Heterogeneous MARL and compare them to homogeneous agents in a small scale proof of concept.

4. A framework for the design of representative neighborhoods for energy flexibility assessment in CityLearn

   Kingsley Nweye, Kathryn Kaspar, Giacomo Buscemi, Giuseppe Pinto, Han Li, Tianzhen Hong, Mohamed Ouf, Alfonso Capozzoli, and Zoltan Nagy

   In Proceedings of building simulation 2023: 18th conference of IBPSA Sep 2023

   Abs URL

   The electricity grid in the U.S. continues to undergo system-wide changes due to electrification of end-uses, as well as the adoption of intermittent on-site renewable energy systems. When effectively managed, distributed energy resource in buildings can provide energy flexibility to alleviate grid loads at critical periods. However, it is important to understand the role of geographic, climatic and occupant behavioral differences on the effectiveness of their flexibility. Thus, we provide a framework that uses an open-source U.S. building-stock database with clustering techniques to design representative neighborhoods for distributed energy resource control algorithm benchmarking. We demonstrate an application in three neighborhoods that use reinforcement learning control for energy storage system management and simulation results show up to 42% reduction in peak demand, amongst other key performance indicators.

5. Unsupervised estimation of residential electricity outages from aggregated meter data

   Praveen Radhakrishnan, Kingsley Nweye, Ting-Yu Dai, and Zoltan Nagy

   In Proceedings of building simulation 2023: 18th conference of IBPSA Sep 2023

   Abs URL

   Winter Storm Uri left a devastating impact on Austin’s power grid and highlighted the need for preparation in the face of extreme weather events.To better understand the load lost during Uri and identify vulnerable areas, an ensemble approach was used, combining K-means clustering, Local outlierfactor algorithm for possible outage prediction. Linear regression and random forest models were used for load lost percentage estimation. Current researchdoes not focus on residential outage analysis during Winter Storm Uri and most of the studies are supervised learning which fails in the absence of truelabels. Our work fills this gap by using unsupervised learning and ensemble methodology to obtain robust estimates. Our results indicate that from Feb15th- 17th, 25 zip codes showed indications of possible power outages with an average load lost of around 60%. We also found 11 vulnerable zip codes with anaverage load lost of over 50%. Our work contributes to estimating demand losses during extreme weather events, which can help understand electricity supplydemand for potential future events. It also helps identifying vulnerable areas (by zip codes) which can help prioritize resources. Our methods use aggregated residential meter data to avoid privacy concerns and can be applied to other locations as well.

6. IEA EBC Annex 81 – Data-driven smart buildings: Building-to-grid applications

   Hicham Johra, Han Li, Flavia Andrade Pereira, Kingsley Nweye, Lasitha Chamari, and Zoltan Nagy

   In Proceedings of Building Simulation 2023: 18th Conference of IBPSA Sep 2023

   Abs URL

   Within the IEA EBC Annex 81 – “data-driven smart buildings”, the sub-task C3 – “Building-to-Grid (B2G) applications” focuses on developing an online platform to gather, evaluate, compare, present and promote concrete cases in which a building or cluster of buildings provide service to the energy grid such as demand response (DR). To that end, the sub-task addresses some important research gaps such as context, definitions and key performance indicators (KPIs) for building energy flexibility (EF) assessment and the heterogeneity of data representations in datasets of B2G services. It provides an overview of the data-driven KPIs for EF assessment, discusses open datasets of buildings performing DR, and proposes a framework to standardize KPIs’ definition and dataset collection. This framework uses semantic principles and a Python package to harmonize required and available data from KPIs and B2G datasets. It thus streamlines the computation of EF assessments, which are implemented in an online platform. This paper presents the results, conclusions and perspectives on B2G applications that can be drawn from sub-task C3.

7. MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities

   Kingsley Nweye, Siva Sankaranarayanan, and Zoltan Nagy

   Applied Energy Sep 2023

   Abs URL

   Building and power generation decarbonization present new challenges in electric grid reliability as a result of renewable energy source intermittency and increase in grid load caused by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide grid flexibility services through demand response. Reinforcement learning is well-suited for energy management in grid-interactive efficient buildings as it is able to adapt to unique building characteristics compared to rule-based control and model predictive control. Yet, factors hindering the adoption of reinforcement learning in real-world applications include its sample inefficiency during training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework for the training, evaluation, deployment and transfer of control policies for distributed energy resources in grid-interactive communities for different levels of data availability. We utilize a real-world community smart meter dataset to show that while independently trained battery control policies can learn unique occupant behavior and provide up to 60% performance improvement at the district level, transfer learning provides comparable building and district level performance while reducing training costs.

8. Ten questions concerning reinforcement learning for building energy management

   Zoltan Nagy, Gregor Henze, Sourav Dey, Javier Arroyo, Lieve Helsen, Xiangyu Zhang, Bingqing Chen, Kadir Amasyali, Kuldeep Kurte, Ahmed Zamzam, Helia Zandi, Ján Drgoňa, Matias Quintana, Steven McCullogh, June Young Park, Han Li, Tianzhen Hong, Silvio Brandi, Giuseppe Pinto, Alfonso Capozzoli, Draguna Vrabie, Mario Bergés, Kingsley Nweye, Thibault Marzullo, and Andrey Bernstein

   Building and Environment Aug 2023

   Abs URL

   As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

9. Data-driven key performance indicators and datasets for building energy flexibility: A review and perspectives

   Han Li, Hicham Johra, Flavia Andrade Pereira, Tianzhen Hong, Jérôme Le Dréau, Anthony Maturo, Mingjun Wei, Yapan Liu, Ali Saberi-Derakhtenjani, Zoltan Nagy, Anna Marszal-Pomianowska, Donal Finn, Shohei Miyata, Kathryn Kaspar, Kingsley Nweye, Zheng O’Neill, Fabiano Pallonetto, and Bing Dong

   Applied Energy Aug 2023

   Abs URL

   Energy flexibility, through short-term demand-side management (DSM) and energy storage technologies, is now seen as a major key to balancing the fluctuating supply in different energy grids with the energy demand of buildings. This is especially important when considering the intermittent nature of ever-growing renewable energy production, as well as the increasing dynamics of electricity demand in buildings. This paper provides a holistic review of (1) data-driven energy flexibility key performance indicators (KPIs) for buildings in the operational phase and (2) open datasets that can be used for testing energy flexibility KPIs. The review identifies a total of 48 data-driven energy flexibility KPIs from 87 recent and relevant publications. These KPIs were categorized and analyzed according to their type, complexity, scope, key stakeholders, data requirement, baseline requirement, resolution, and popularity. Moreover, 330 building datasets were collected and evaluated. Of those, 16 were deemed adequate to feature building performing demand response or building-to-grid (B2G) services. The DSM strategy, building scope, grid type, control strategy, needed data features, and usability of these selected 16 datasets were analyzed. This review reveals future opportunities to address limitations in the existing literature: (1) developing new data-driven methodologies to specifically evaluate different energy flexibility strategies and B2G services of existing buildings; (2) developing baseline-free KPIs that could be calculated from easily accessible building sensors and meter data; (3) devoting non-engineering efforts to promote building energy flexibility, standardizing data-driven energy flexibility quantification and verification processes; and (4) curating and analyzing datasets with proper description for energy flexibility assessm.

10. Ten questions concerning occupant-centric control and operations

    Zoltan Nagy, Burak Gunay, Clayton Miller, Jakob Hahn, Mohamed Ouf, Seungjae Lee, Brodie W. Hobson, Tareq Abuimara, Karol Bandurski, Maíra André, Clara-Larissa Lorenz, Sarah Crosby, Bing Dong, Zixin Jiang, Yuzhen Peng, Matteo Favero, June Young Park, Kingsley Nweye, Pedram Nojedehi, Helen Stopps, Lucile Sarran, Connor Brackley, Katherine Bassett, Krissy Govertsen, Nicole Koczorek, Oliver Abele, Emily Casavant, Michael Kane, Zheng O’Neill, Tao Yang, Julia Day, Brent Huchuk, Runa T. Hellwig, and Marika Vellei

    Building and Environment Jun 2023

    Abs URL

    Occupant-Centric Control and Operation (OCC) represents a transformative approach to building management, integrating sensing of indoor environmental quality, occupant presence, and occupant-building interactions. These data are then utilized to optimize both operational efficiency and occupant comfort. This paper summarizes the findings from the IEA-EBC Annex 79 research program’s subtask on real world implementations of OCC during the past 5 years. First, in Q1 and Q2, we provide a definition and categorization of OCC. Q3 addresses the role of building operators for OCC, while Q4 describes the implications for designers. Then, Q5 and Q6 discuss the role and possibilities of OCC for load flexibility, and for pandemic induced paradigm shifts in the built environment, respectively. In Q7, we provide a taxonomy and selection process of OCC, while Q8 details real world implementation case studies. Finally, Q9 explains the limits of OCC, and Q10 provides a vision for future research opportunities. Our findings offer valuable insights for researchers, practitioners, and policy makers, contributing to the ongoing discourse on the future of building operations management.

11. How spatio-temporal resolution impacts urban energy calibration

    Aysegul Demir Dilsiz, Kingsley E. Nweye, Allen J. Wu, Jérôme H. Kämpf, Filip Biljecki, and Zoltan Nagy

    Energy and Buildings May 2023

    Abs URL

    Building Energy Modeling tools help forecast the energy performance of buildings. Urban energy models (UBEMs) emerged as important instruments to analyze the energy performance of buildings aggregated at different spatial resolutions, from the building level to the district level. They heavily rely on available data on geometries and measurements to create accurately calibrated energy models. However, limited research has been conducted to understand the impact of spatial and temporal resolution on the simulation results because of the difficulty of comparing results and not having a standardized procedure to report simulation errors. We review the literature on UBEM validation compared to measured energy data and show the discrepancies in the reporting accuracy. We articulate the need for consistent reporting on model accuracy and introduce a multi-dimensional Level of Detail (LoD) specification for UBEM, including geometry, thermal zoning, and spatio-temporal resolution of the measured data used to calibrate the models. Using a university campus with 70 buildings as an extensive case study, we demonstrate the performance of Bayesian calibration from the building level to the aggregated level. Our results suggest that the accuracy of urban energy prediction with annual temporal resolution can be significantly increased if calibration is performed by using building-level data. However, whenever privacy is a concern, then the data should be provided by aggregating them based on primary use type. Additionally, using monthly data to calibrate uncertain input parameters is not improving the accuracy of the models because the obtained posterior distributions for the selected parameters are not informative for monthly data. To improve this shortcoming, we suggest seasonal calibration, which is computationally costly.

12. Analyzing the impact of COVID-19 on the electricity demand in Austin, TX using an ensemble-model based counterfactual and 400,000 smart meters

    Ting-Yu Dai, Praveen Radhakrishnan, Kingsley Nweye, Robert Estrada, Dev Niyogi, and Zoltan Nagy

    Computational Urban Science May 2023

    Abs URL

    The COVID-19 pandemic caused lifestyle changes and has led to the new electricity demand patterns in the presence of non-pharmaceutical interventions such as work-from-home policy and lockdown. Quantifying the effect on electricity demand is critical for future electricity market planning yet challenging in the context of limited smart metered buildings, which leads to limited understanding of the temporal and spatial variations in building energy use. This study uses a large scale private smart meter electricity demand data from the City of Austin, combined with publicly available environmental data, and develops an ensemble regression model for long term daily electricity demand prediction. Using 15-min resolution data from over 400,000 smart meters from 2018 to 2020 aggregated by building type and zip code, our proposed model precisely formalizes the counterfactual universe in the without COVID-19 scenario. The model is used to understand building electricity demand changes during the pandemic and to identify relationships between such changes and socioeconomic patterns. Results indicate the increase in residential usage , demonstrating the spatial redistribution of energy consumption during the work-from-home period. Our experiments demonstrate the effectiveness of our proposed framework by assessing multiple socioeconomic impacts with the comparison between the counterfactual universe and observations.

13. CityLearn: A tutorial on reinforcement learning control for grid-interactive efficient buildings and communities

    Kingsley E Nweye, Allen Wu, Hyun Park, Yara Almilaify, and Zoltan Nagy

    In ICLR 2023 workshop on tackling climate change with machine learning May 2023

    Abs URL

    Buildings are responsible for up to 75% of electricity consumption in the United States. Grid-Interactive Efficient Buildings can provide flexibility to solve the issue of power supply-demand mismatch, particularly brought about by renewables. Their high energy efficiency and self-generating capabilities can reduce demand without affecting the building function. Additionally, load shedding and shifting through smart control of storage systems can further flatten the load curve and reduce grid ramping cost in response to rapid decrease in renewable power supply. The model-free nature of reinforcement learning control makes it a promising approach for smart control in grid-interactive efficient buildings, as it can adapt to unique building needs and functions. However, a major challenge for the adoption of reinforcement learning in buildings is the ability to benchmark different control algorithms to accelerate their deployment on live systems. CityLearn is an open source OpenAI Gym environment for the implementation and benchmarking of simple and advanced control algorithms, e.g., rule-based control, model predictive control or deep reinforcement learning control thus, provides solutions to this challenge. This tutorial leverages CityLearn to demonstrate different control strategies in grid-interactive efficient buildings. Participants will learn how to design three controllers of varying complexity for battery management using a real-world residential neighborhood dataset to provide load shifting flexibility. The algorithms will be evaluated using six energy flexibility, environmental and economic key performance indicators, and their benefits and shortcomings will be identified. By the end of the tutorial, participants will acquire enough familiarity with the CityLearn environment for extended use in new datasets or personal projects.

14. The CityLearn Challenge 2021 Benchmark Results

    Kingsley Nweye, and Gyorgy Zoltan Nagy

    Feb 2023

    Abs URL

    The dataset comes from an application of The CityLearn Challenge 2021 dataset in CityLearn to generate benchmark results. To reproduce this dataset, refer to the README. This dataset is a collection of weather and carbon intensity time series, building and systems metadata as well as the resulting building level time series from an hourly 4-year-long simulation. The novel MARLISA agents for multi-agent reinforcement learning cooperative control are used to manage the buildings’ electrical and active thermal storage systems to provide energy flexibility in a demand response setting. Supplementary data from an hourly 4-year-long CityLearn environment simulation using a reference RBC are also included. There are a total of 9 buildings in the environment and all buildings are DOE stock commercial buildings (1 medium office, 1 fast-food restaurant, 1 standalone retail, 1 strip mall retail, and 5 medium multifamily buildings). The dataset contains the following files: 1. weather.csv - AMY Weather time series for New Orleans, LA (.csv) 2. carbon_intensity.csv - ERCOT carbon intensity time series (.csv) 3. building_id_agent_name.csv - Buildings time series for MARLISA and RBC control (.csv) 4. variable_metadata.csv - Variable metadata (.csv) 5. building_metadata.json - Building and systems metadata (.json)

15. The CityLearn Challenge 2022

    Kingsley Nweye, Sankaranarayanan Siva, and Gyorgy Zoltan Nagy

    Jan 2023

    Abs URL

    This is the dataset used for The CityLearn Challenge 2022. It contains the schema, buildings, weather, carbon intensity and electricity pricing datasets for Phases 1, 2 and 3 in separate folders. Note that the weather, carbon intensity and pricing data are the same in all folders. The CityLearn version that was used for The CityLearn Challenge 2022 can be installed through pip: pip install CityLearn==1.3.6 Earlier and later CityLearn versions may also be compatible with this dataset but could produce different simulation results. Please, refer to the example Jupyter notebook to see how to use this dataset.

2022

1. Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings

   Kingsley Nweye, Bo Liu, Peter Stone, and Zoltan Nagy

   Energy and AI Nov 2022

   Abs URL

   Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning (RL) control, here we propose a non-exhaustive set of nine real world challenges for RL control in grid-interactive buildings (GIBs). We argue that research in this area should be expressed in this framework in addition to providing a standardized environment for repeatability. Advanced controllers such as model predictive control (MPC) and RL control have both advantages and disadvantages that prevent them from being implemented in real world problems. Comparisons between the two are rare, and often biased. By focusing on the challenges, we can investigate the performance of the controllers under a variety of situations and generate a fair comparison. As a demonstration, we implement the offline learning challenge in CityLearn, an OpenAI Gym environment for the easy implementation of RL agents in a demand response setting to reshape the aggregated curve of electricity demand by controlling the energy storage of a diverse set of buildings in a district. We use CityLearn to study the impact of different levels of domain knowledge and complexity of RL algorithms and show that the sequence of operations (SOOs) utilized in a rule based controller (RBC) that provides fixed logs to RL agents during offline training affect the performance of the agents when evaluated on a set of four energy flexibility metrics. Longer offline training from an optimized RBC leads to improved performance in the long run. RL agents that train on the logs from a simplified RBC risk poorer performance as the offline training period increases. We also observe no impact on performance from information sharing amongst agents. We call for a more interdisciplinary effort of the research community to address the real world challenges, and unlock the potential of GIB controllers.

2. The CityLearn Challenge 2022: Overview, Results, and Lessons Learned

   Kingsley Nweye, Zoltan Nagy, Sharada Mohanty, Dipam Chakraborty, Siva Sankaranarayanan, Tianzhen Hong, Sourav Dey, Gregor Henze, Jan Drgona, Fangquan Lin, Wei Jiang, Hanwei Zhang, Zhongkai Yi, Jihai Zhang, Cheng Yang, Matthew Motoki, Sorapong Khongnawang, Michael Ibrahim, Abilmansur Zhumabekov, Daniel May, Zhihu Yang, Xiaozhuang Song, Han Zhang, Xiaoning Dong, Shun Zheng, and Jiang Bian

   In Proceedings of the NeurIPS 2022 Competitions Track 28 nov–09 dec 2022

   Abs URL

   The shift to renewable power sources and building electrification to decarbonize existing and emerging building stock present unique challenges for the power grid. Building loads and flexible resources e.g. batteries must be adequately managed simultaneously to unlock the full flexibility potential and reduce costs for all stakeholders. Simple control algorithms based on expert knowledge e.g. RBC, as well as, advanced control algorithms e.g. MPC and RLC can be utilized to intelligently manage flexible resources. The CityLearn Challenge is an opportunity to compete in investigating the potential of AI and distributed control systems to tackle multiple problems within the built-environment. The CityLearn Challenge 2022 is the third of its kind with the overall objective of crowd-sourcing generalizable control policies that improve energy, cost and environmental objectives by taking advantage of batteries for load shifting in a CityLearn digital twin of a real-world grid-interactive neighborhood. Highlighted here are the uniqueness of this third edition, baseline and top solutions, and lessons learned for future editions.

3. Design, fabrication, and calibration of the Building EnVironment and Occupancy (BEVO) Beacon: A rapidly-deployable and affordable indoor environmental quality monitor

   Hagen Fritz, Sepehr Bastami, Calvin Lin, Kingsley Nweye, Tung To, Lauren Chen, Dung Le, Angelina Ibarra, Wendy Zhang, June Young Park, William Waites, Mengjia Tang, Pawel Misztal, Atila Novoselac, Edison Thomaz, Kerry Kinney, and Zoltan Nagy

   Building and Environment Aug 2022

   Abs URL

   Indoor Air Quality (IAQ) monitoring is essential to assess occupant exposure to the wide range of pollutants present in indoor environments. Accurate research-grade monitors are often used to monitor IAQ but the expense and logistics associated with these devices often limits the temporal and spatial scale of monitoring efforts. More affordable consumer-grade sensors – frequently referred to as low-cost sensors – can provide insight into IAQ conditions across greater scales but their accuracy and calibration requirements need further evaluation. In this paper, we present the Building EnVironment and Occupancy (BEVO) Beacon. The BEVO Beacon is entirely open-source, including the software, hardware, and design schematics which are all provided on GitHub. We created 20 of these standalone, stationary devices which measure up to 24 parameters at a one-minute resolution of which we focus on carbon dioxide, carbon monoxide, total volatile organic compounds, temperature, and size-resolved particulate matter. We investigated the efficacy of two different calibration approaches – device-specific and environment-averaged – for these sensors as well as also provide an extensive discussion considerations for each of the sensors. Calibrated sensors performed well when compared to reference monitors or calibrated gas standards. The CO sensors yielded the best agreement (r2=0.98-0.99), followed by temperature (r2=0.89-0.99), CO2 (r2=0.62-0.99), and PM2.5 (r2=-0.13-0.91). In all cases, the device-specific calibration approach yielded the most accurate results. We evaluated our devices through a successful 11-week field study where we monitored the IAQ in participants’ bedrooms. The work we present on consumer-grade sensors adds to the existing literature by considering sensor-specific calibration techniques and analysis. The BEVO Beacon adds to the successful line of similarly developed devices by providing an open-source framework that researchers can readily adapt and modify to their own applications.

4. Water Demand and Human Behavior during Compounding Disasters: The Case of Winter Storm Uri and the COVID-19 Pandemic

   Lauryn A. Spearing, Kingsley Nweye, Helena R. Tiedmann, Zoltan Nagy, Lina Sela, and Kasey M. Faust

   Jun 2022

   Abs URL

   Engineered systems are designed for a specific operating context based on assumptions about the population served. In turn, management of these systems can be stressed during population shifts (and corresponding demand shifts), such as those seen during both discrete (e.g., hurricanes) and protracted (e.g., pandemics) events. For instance, the COVID-19 pandemic caused drastic changes in society, consequentially changing spatial and temporal water use as people worked from home. In another example, Winter Storm Uri led to utility service disruptions throughout Texas, causing people without power and water to seek shelter, leading to spatial changes in water use in conjunction with physical damage. This sheltering occurred during the COVID-19 pandemic, leading to increased uncertainty in demand and challenges to shelter while ensuring social distancing. Researchers have studied disaster scenarios independently, but there is a gap surrounding compounding disasters as human-infrastructure interactions are likely altered. Here, we assess water demand changes during Winter Storm Uri (which occurred during the COVID-19 pandemic) at the building level. We performed k-means clustering on demand data from four buildings at the University of Texas, Austin. Three buildings showed different daily demand profiles during the storm compared to the spring semester. Interestingly, there were demand increases in buildings not being used as warming centers, perhaps indicating increased occupancy. This trend reveals that people do not necessarily choose to shelter in places that are formally organized. In a museum, water use decreased compared to the already reduced demand during the pandemic, possibly leading to water stagnation and quality concerns.

5. Offline Training of Multi-Agent Reinforcement Agents for Grid-Interactive Buildings Control

   Kingsley Nweye, Zoltan Nagy, Bo Liu, and Peter Stone

   In Proceedings of the Thirteenth ACM International Conference on Future Energy Systems Jun 2022

   Abs URL

   Advanced controllers such as model predictive control and reinforcement learning (RL) control have both advantages and disadvantages that prevent them from being implemented in real world problems. Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning control, we demonstrate in CityLearn, the effect of training off-line from the fixed logs of an external behavior policy. We show that the design of the rule based controller (RBC) used for offline training affects the performance of the RL agents when evaluated on the average daily peak and net electric consumption. While longer offline training from an optimized RBC leads to improved performance in the long run, RL agents that learn from a simplified RBC risk poorer performance as the offline learning period increases.

6. MARTINI: Smart meter driven estimation of HVAC schedules and energy savings based on Wi-Fi sensing and clustering

   Kingsley Nweye, and Zoltan Nagy

   Applied Energy Jun 2022

   Abs URL

   HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an Energy Conservation Measure (ECM) where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building’s real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from ECMs that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived Heating, Ventilation and Air Conditioning (HVAC) schedules and energy savings that leverages the ubiquity of energy smart meters and Wi-Fi infrastructure in commercial buildings. We estimate the schedules by clustering Wi-Fi derived occupancy profiles and, estimate energy savings by shifting ramp-up and setback times observed in typical operational/static load profiles that are obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%–10.8% (summer) and 0.2%–5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from Building Energy Performance Simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%–2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%–5%.

7. CROOD: Estimating crude building occupancy from mobile device connections without ground-truth calibration

   June Young Park, Kingsley Nweye, Edward Mbata, and Zoltan Nagy

   Building and Environment May 2022

   Abs URL

   The occupancy information in buildings is fundamental for smart buildings (e.g., occupant-centric controls). Opportunistic occupancy detection (OOD) uses connection data of mobile devices. While OOD has been developed and applied, one critical drawback is that it requires the ground truth of occupants to calibrate, which is limited to gather. Here, we introduce CROOD: a capture and recapture (CRc) inspired OOD. In ecology, CRc has been established for the estimation of animal populations, when the manual count is impossible. We adopt this unique approach to estimate the number of mobile devices in a building. Then, using a simple estimate on the total population, CROOD determines the relationship between the numbers of occupants and mobile devices. We evaluate CROOD numerically on the synthetic building populations and demonstrate its application in a university library using WiFi connection data. We find that CROOD can estimate the number of mobile devices and subsequently the number of occupants with 1–2 weeks to converge a reasonable accuracy. A long term experiment shows that CROOD can adapt to varying population characteristics (e.g., occupants bring more mobile devices), outperforming the reference sample mean estimator. The real building implementation demonstrates that while in the first 1–2 weeks, the sample mean estimator is superior, eventually CROOD adapts and provides better estimates without ground-truth calibration. Although CROOD has a limitation of building types and systems, our results envision that CROOD could be a viable addition to other OOD methods to better utilize existing mobile device connection data to estimate occupancy in buildings.

2020

1. Impact of COVID-19 on Academic Campus Energy Use

   Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2020

   Abs URL

   This poster compares energy use of the campus of the University of Texas at Austin during the COVID-19 pandemic of 2020 to that of previous years. Our results show change in weekly aggregate chilled water and electricity consumption as a result of changes in building operation schedules and the response of utility demand to changing occupancy.

2. HVAC Scheduling Based on Wi-Fi Derived Occupancy

   Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2020

   Abs URL

   This poster showcases the integration of building management system data with Wi-Fi derived occupancy data to calibrate a building energy model for energy performance prediction and analysis. Our results show an overall improved building energy savings and unmet hours by the Wi-Fi derived schedules compared to the baseline and at 4% occupied threshold, similar energy consumption as baseline is achieved but with reduced unmet hours.