publications

2023

1. MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities

   Kingsley Nweye, Siva Sankaranarayanan, and Zoltan Nagy

   Applied Energy Sep 2023

   Abs URL

   Building and power generation decarbonization present new challenges in electric grid reliability as a result of renewable energy source intermittency and increase in grid load caused by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide grid flexibility services through demand response. Reinforcement learning is well-suited for energy management in grid-interactive efficient buildings as it is able to adapt to unique building characteristics compared to rule-based control and model predictive control. Yet, factors hindering the adoption of reinforcement learning in real-world applications include its sample inefficiency during training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework for the training, evaluation, deployment and transfer of control policies for distributed energy resources in grid-interactive communities for different levels of data availability. We utilize a real-world community smart meter dataset to show that while independently trained battery control policies can learn unique occupant behavior and provide up to 60% performance improvement at the district level, transfer learning provides comparable building and district level performance while reducing training costs.

2. Ten questions concerning reinforcement learning for building energy management

   Zoltan Nagy, Gregor Henze, Sourav Dey, Javier Arroyo, Lieve Helsen, Xiangyu Zhang, Bingqing Chen, Kadir Amasyali, Kuldeep Kurte, Ahmed Zamzam, Helia Zandi, Ján Drgoňa, Matias Quintana, Steven McCullogh, June Young Park, Han Li, Tianzhen Hong, Silvio Brandi, Giuseppe Pinto, Alfonso Capozzoli, Draguna Vrabie, Mario Bergés, Kingsley Nweye, Thibault Marzullo, and Andrey Bernstein

   Building and Environment Aug 2023

   Abs URL

   As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

3. Data-driven key performance indicators and datasets for building energy flexibility: A review and perspectives

   Han Li, Hicham Johra, Flavia Andrade Pereira, Tianzhen Hong, Jérôme Le Dréau, Anthony Maturo, Mingjun Wei, Yapan Liu, Ali Saberi-Derakhtenjani, Zoltan Nagy, Anna Marszal-Pomianowska, Donal Finn, Shohei Miyata, Kathryn Kaspar, Kingsley Nweye, Zheng O’Neill, Fabiano Pallonetto, and Bing Dong

   Applied Energy Aug 2023

   Abs URL

   Energy flexibility, through short-term demand-side management (DSM) and energy storage technologies, is now seen as a major key to balancing the fluctuating supply in different energy grids with the energy demand of buildings. This is especially important when considering the intermittent nature of ever-growing renewable energy production, as well as the increasing dynamics of electricity demand in buildings. This paper provides a holistic review of (1) data-driven energy flexibility key performance indicators (KPIs) for buildings in the operational phase and (2) open datasets that can be used for testing energy flexibility KPIs. The review identifies a total of 48 data-driven energy flexibility KPIs from 87 recent and relevant publications. These KPIs were categorized and analyzed according to their type, complexity, scope, key stakeholders, data requirement, baseline requirement, resolution, and popularity. Moreover, 330 building datasets were collected and evaluated. Of those, 16 were deemed adequate to feature building performing demand response or building-to-grid (B2G) services. The DSM strategy, building scope, grid type, control strategy, needed data features, and usability of these selected 16 datasets were analyzed. This review reveals future opportunities to address limitations in the existing literature: (1) developing new data-driven methodologies to specifically evaluate different energy flexibility strategies and B2G services of existing buildings; (2) developing baseline-free KPIs that could be calculated from easily accessible building sensors and meter data; (3) devoting non-engineering efforts to promote building energy flexibility, standardizing data-driven energy flexibility quantification and verification processes; and (4) curating and analyzing datasets with proper description for energy flexibility assessm.

4. How spatio-temporal resolution impacts urban energy calibration

   Aysegul Demir Dilsiz, Kingsley E. Nweye, Allen J. Wu, Jérôme H. Kämpf, Filip Biljecki, and Zoltan Nagy

   Energy and Buildings May 2023

   Abs URL

   Building Energy Modeling tools help forecast the energy performance of buildings. Urban energy models (UBEMs) emerged as important instruments to analyze the energy performance of buildings aggregated at different spatial resolutions, from the building level to the district level. They heavily rely on available data on geometries and measurements to create accurately calibrated energy models. However, limited research has been conducted to understand the impact of spatial and temporal resolution on the simulation results because of the difficulty of comparing results and not having a standardized procedure to report simulation errors. We review the literature on UBEM validation compared to measured energy data and show the discrepancies in the reporting accuracy. We articulate the need for consistent reporting on model accuracy and introduce a multi-dimensional Level of Detail (LoD) specification for UBEM, including geometry, thermal zoning, and spatio-temporal resolution of the measured data used to calibrate the models. Using a university campus with 70 buildings as an extensive case study, we demonstrate the performance of Bayesian calibration from the building level to the aggregated level. Our results suggest that the accuracy of urban energy prediction with annual temporal resolution can be significantly increased if calibration is performed by using building-level data. However, whenever privacy is a concern, then the data should be provided by aggregating them based on primary use type. Additionally, using monthly data to calibrate uncertain input parameters is not improving the accuracy of the models because the obtained posterior distributions for the selected parameters are not informative for monthly data. To improve this shortcoming, we suggest seasonal calibration, which is computationally costly.

5. Analyzing the impact of COVID-19 on the electricity demand in Austin, TX using an ensemble-model based counterfactual and 400,000 smart meters

   Ting-Yu Dai, Praveen Radhakrishnan, Kingsley Nweye, Robert Estrada, Dev Niyogi, and Zoltan Nagy

   Computational Urban Science May 2023

   Abs URL

   The COVID-19 pandemic caused lifestyle changes and has led to the new electricity demand patterns in the presence of non-pharmaceutical interventions such as work-from-home policy and lockdown. Quantifying the effect on electricity demand is critical for future electricity market planning yet challenging in the context of limited smart metered buildings, which leads to limited understanding of the temporal and spatial variations in building energy use. This study uses a large scale private smart meter electricity demand data from the City of Austin, combined with publicly available environmental data, and develops an ensemble regression model for long term daily electricity demand prediction. Using 15-min resolution data from over 400,000 smart meters from 2018 to 2020 aggregated by building type and zip code, our proposed model precisely formalizes the counterfactual universe in the without COVID-19 scenario. The model is used to understand building electricity demand changes during the pandemic and to identify relationships between such changes and socioeconomic patterns. Results indicate the increase in residential usage , demonstrating the spatial redistribution of energy consumption during the work-from-home period. Our experiments demonstrate the effectiveness of our proposed framework by assessing multiple socioeconomic impacts with the comparison between the counterfactual universe and observations.

6. CityLearn: A tutorial on reinforcement learning control for grid-interactive efficient buildings and communities

   Kingsley E Nweye, Allen Wu, Hyun Park, Yara Almilaify, and Zoltan Nagy

   In ICLR 2023 workshop on tackling climate change with machine learning May 2023

   Abs URL

   Buildings are responsible for up to 75% of electricity consumption in the United States. Grid-Interactive Efficient Buildings can provide flexibility to solve the issue of power supply-demand mismatch, particularly brought about by renewables. Their high energy efficiency and self-generating capabilities can reduce demand without affecting the building function. Additionally, load shedding and shifting through smart control of storage systems can further flatten the load curve and reduce grid ramping cost in response to rapid decrease in renewable power supply. The model-free nature of reinforcement learning control makes it a promising approach for smart control in grid-interactive efficient buildings, as it can adapt to unique building needs and functions. However, a major challenge for the adoption of reinforcement learning in buildings is the ability to benchmark different control algorithms to accelerate their deployment on live systems. CityLearn is an open source OpenAI Gym environment for the implementation and benchmarking of simple and advanced control algorithms, e.g., rule-based control, model predictive control or deep reinforcement learning control thus, provides solutions to this challenge. This tutorial leverages CityLearn to demonstrate different control strategies in grid-interactive efficient buildings. Participants will learn how to design three controllers of varying complexity for battery management using a real-world residential neighborhood dataset to provide load shifting flexibility. The algorithms will be evaluated using six energy flexibility, environmental and economic key performance indicators, and their benefits and shortcomings will be identified. By the end of the tutorial, participants will acquire enough familiarity with the CityLearn environment for extended use in new datasets or personal projects.

7. The CityLearn Challenge 2021 Benchmark Results

   Kingsley Nweye, and Gyorgy Zoltan Nagy

   Feb 2023

   Abs URL

   The dataset comes from an application of The CityLearn Challenge 2021 dataset in CityLearn to generate benchmark results. To reproduce this dataset, refer to the README. This dataset is a collection of weather and carbon intensity time series, building and systems metadata as well as the resulting building level time series from an hourly 4-year-long simulation. The novel MARLISA agents for multi-agent reinforcement learning cooperative control are used to manage the buildings’ electrical and active thermal storage systems to provide energy flexibility in a demand response setting. Supplementary data from an hourly 4-year-long CityLearn environment simulation using a reference RBC are also included. There are a total of 9 buildings in the environment and all buildings are DOE stock commercial buildings (1 medium office, 1 fast-food restaurant, 1 standalone retail, 1 strip mall retail, and 5 medium multifamily buildings). The dataset contains the following files: 1. weather.csv - AMY Weather time series for New Orleans, LA (.csv) 2. carbon_intensity.csv - ERCOT carbon intensity time series (.csv) 3. building_id_agent_name.csv - Buildings time series for MARLISA and RBC control (.csv) 4. variable_metadata.csv - Variable metadata (.csv) 5. building_metadata.json - Building and systems metadata (.json)

8. The CityLearn Challenge 2022

   Kingsley Nweye, Sankaranarayanan Siva, and Gyorgy Zoltan Nagy

   Jan 2023

   Abs URL

   This is the dataset used for The CityLearn Challenge 2022. It contains the schema, buildings, weather, carbon intensity and electricity pricing datasets for Phases 1, 2 and 3 in separate folders. Note that the weather, carbon intensity and pricing data are the same in all folders. The CityLearn version that was used for The CityLearn Challenge 2022 can be installed through pip: pip install CityLearn==1.3.6 Earlier and later CityLearn versions may also be compatible with this dataset but could produce different simulation results. Please, refer to the example Jupyter notebook to see how to use this dataset.

2022

1. Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings

   Kingsley Nweye, Bo Liu, Peter Stone, and Zoltan Nagy

   Energy and AI Nov 2022

   Abs URL

   Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning (RL) control, here we propose a non-exhaustive set of nine real world challenges for RL control in grid-interactive buildings (GIBs). We argue that research in this area should be expressed in this framework in addition to providing a standardized environment for repeatability. Advanced controllers such as model predictive control (MPC) and RL control have both advantages and disadvantages that prevent them from being implemented in real world problems. Comparisons between the two are rare, and often biased. By focusing on the challenges, we can investigate the performance of the controllers under a variety of situations and generate a fair comparison. As a demonstration, we implement the offline learning challenge in CityLearn, an OpenAI Gym environment for the easy implementation of RL agents in a demand response setting to reshape the aggregated curve of electricity demand by controlling the energy storage of a diverse set of buildings in a district. We use CityLearn to study the impact of different levels of domain knowledge and complexity of RL algorithms and show that the sequence of operations (SOOs) utilized in a rule based controller (RBC) that provides fixed logs to RL agents during offline training affect the performance of the agents when evaluated on a set of four energy flexibility metrics. Longer offline training from an optimized RBC leads to improved performance in the long run. RL agents that train on the logs from a simplified RBC risk poorer performance as the offline training period increases. We also observe no impact on performance from information sharing amongst agents. We call for a more interdisciplinary effort of the research community to address the real world challenges, and unlock the potential of GIB controllers.

2. Design, fabrication, and calibration of the Building EnVironment and Occupancy (BEVO) Beacon: A rapidly-deployable and affordable indoor environmental quality monitor

   Hagen Fritz, Sepehr Bastami, Calvin Lin, Kingsley Nweye, Tung To, Lauren Chen, Dung Le, Angelina Ibarra, Wendy Zhang, June Young Park, William Waites, Mengjia Tang, Pawel Misztal, Atila Novoselac, Edison Thomaz, Kerry Kinney, and Zoltan Nagy

   Building and Environment Aug 2022

   Abs URL

   Indoor Air Quality (IAQ) monitoring is essential to assess occupant exposure to the wide range of pollutants present in indoor environments. Accurate research-grade monitors are often used to monitor IAQ but the expense and logistics associated with these devices often limits the temporal and spatial scale of monitoring efforts. More affordable consumer-grade sensors – frequently referred to as low-cost sensors – can provide insight into IAQ conditions across greater scales but their accuracy and calibration requirements need further evaluation. In this paper, we present the Building EnVironment and Occupancy (BEVO) Beacon. The BEVO Beacon is entirely open-source, including the software, hardware, and design schematics which are all provided on GitHub. We created 20 of these standalone, stationary devices which measure up to 24 parameters at a one-minute resolution of which we focus on carbon dioxide, carbon monoxide, total volatile organic compounds, temperature, and size-resolved particulate matter. We investigated the efficacy of two different calibration approaches – device-specific and environment-averaged – for these sensors as well as also provide an extensive discussion considerations for each of the sensors. Calibrated sensors performed well when compared to reference monitors or calibrated gas standards. The CO sensors yielded the best agreement (r2=0.98-0.99), followed by temperature (r2=0.89-0.99), CO2 (r2=0.62-0.99), and PM2.5 (r2=-0.13-0.91). In all cases, the device-specific calibration approach yielded the most accurate results. We evaluated our devices through a successful 11-week field study where we monitored the IAQ in participants’ bedrooms. The work we present on consumer-grade sensors adds to the existing literature by considering sensor-specific calibration techniques and analysis. The BEVO Beacon adds to the successful line of similarly developed devices by providing an open-source framework that researchers can readily adapt and modify to their own applications.

3. Water Demand and Human Behavior during Compounding Disasters: The Case of Winter Storm Uri and the COVID-19 Pandemic

   Lauryn A. Spearing, Kingsley Nweye, Helena R. Tiedmann, Zoltan Nagy, Lina Sela, and Kasey M. Faust

   Jun 2022

   Abs URL

   Engineered systems are designed for a specific operating context based on assumptions about the population served. In turn, management of these systems can be stressed during population shifts (and corresponding demand shifts), such as those seen during both discrete (e.g., hurricanes) and protracted (e.g., pandemics) events. For instance, the COVID-19 pandemic caused drastic changes in society, consequentially changing spatial and temporal water use as people worked from home. In another example, Winter Storm Uri led to utility service disruptions throughout Texas, causing people without power and water to seek shelter, leading to spatial changes in water use in conjunction with physical damage. This sheltering occurred during the COVID-19 pandemic, leading to increased uncertainty in demand and challenges to shelter while ensuring social distancing. Researchers have studied disaster scenarios independently, but there is a gap surrounding compounding disasters as human-infrastructure interactions are likely altered. Here, we assess water demand changes during Winter Storm Uri (which occurred during the COVID-19 pandemic) at the building level. We performed k-means clustering on demand data from four buildings at the University of Texas, Austin. Three buildings showed different daily demand profiles during the storm compared to the spring semester. Interestingly, there were demand increases in buildings not being used as warming centers, perhaps indicating increased occupancy. This trend reveals that people do not necessarily choose to shelter in places that are formally organized. In a museum, water use decreased compared to the already reduced demand during the pandemic, possibly leading to water stagnation and quality concerns.

4. Offline Training of Multi-Agent Reinforcement Agents for Grid-Interactive Buildings Control

   Kingsley Nweye, Zoltan Nagy, Bo Liu, and Peter Stone

   In Proceedings of the Thirteenth ACM International Conference on Future Energy Systems Jun 2022

   Abs URL

   Advanced controllers such as model predictive control and reinforcement learning (RL) control have both advantages and disadvantages that prevent them from being implemented in real world problems. Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning control, we demonstrate in CityLearn, the effect of training off-line from the fixed logs of an external behavior policy. We show that the design of the rule based controller (RBC) used for offline training affects the performance of the RL agents when evaluated on the average daily peak and net electric consumption. While longer offline training from an optimized RBC leads to improved performance in the long run, RL agents that learn from a simplified RBC risk poorer performance as the offline learning period increases.

5. MARTINI: Smart meter driven estimation of HVAC schedules and energy savings based on Wi-Fi sensing and clustering

   Kingsley Nweye, and Zoltan Nagy

   Applied Energy Jun 2022

   Abs URL

   HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an Energy Conservation Measure (ECM) where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building’s real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from ECMs that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived Heating, Ventilation and Air Conditioning (HVAC) schedules and energy savings that leverages the ubiquity of energy smart meters and Wi-Fi infrastructure in commercial buildings. We estimate the schedules by clustering Wi-Fi derived occupancy profiles and, estimate energy savings by shifting ramp-up and setback times observed in typical operational/static load profiles that are obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%–10.8% (summer) and 0.2%–5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from Building Energy Performance Simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%–2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%–5%.

6. CROOD: Estimating crude building occupancy from mobile device connections without ground-truth calibration

   June Young Park, Kingsley Nweye, Edward Mbata, and Zoltan Nagy

   Building and Environment May 2022

   Abs URL

   The occupancy information in buildings is fundamental for smart buildings (e.g., occupant-centric controls). Opportunistic occupancy detection (OOD) uses connection data of mobile devices. While OOD has been developed and applied, one critical drawback is that it requires the ground truth of occupants to calibrate, which is limited to gather. Here, we introduce CROOD: a capture and recapture (CRc) inspired OOD. In ecology, CRc has been established for the estimation of animal populations, when the manual count is impossible. We adopt this unique approach to estimate the number of mobile devices in a building. Then, using a simple estimate on the total population, CROOD determines the relationship between the numbers of occupants and mobile devices. We evaluate CROOD numerically on the synthetic building populations and demonstrate its application in a university library using WiFi connection data. We find that CROOD can estimate the number of mobile devices and subsequently the number of occupants with 1–2 weeks to converge a reasonable accuracy. A long term experiment shows that CROOD can adapt to varying population characteristics (e.g., occupants bring more mobile devices), outperforming the reference sample mean estimator. The real building implementation demonstrates that while in the first 1–2 weeks, the sample mean estimator is superior, eventually CROOD adapts and provides better estimates without ground-truth calibration. Although CROOD has a limitation of building types and systems, our results envision that CROOD could be a viable addition to other OOD methods to better utilize existing mobile device connection data to estimate occupancy in buildings.

2020

1. Impact of COVID-19 on Academic Campus Energy Use

   Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2020

   Abs URL

   This poster compares energy use of the campus of the University of Texas at Austin during the COVID-19 pandemic of 2020 to that of previous years. Our results show change in weekly aggregate chilled water and electricity consumption as a result of changes in building operation schedules and the response of utility demand to changing occupancy.

2. HVAC Scheduling Based on Wi-Fi Derived Occupancy

   Kingsley Nweye, and Zoltan Nagy

   In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation Nov 2020

   Abs URL

   This poster showcases the integration of building management system data with Wi-Fi derived occupancy data to calibrate a building energy model for energy performance prediction and analysis. Our results show an overall improved building energy savings and unmet hours by the Wi-Fi derived schedules compared to the baseline and at 4% occupied threshold, similar energy consumption as baseline is achieved but with reduced unmet hours.