Reproducibility: Best practise examples

In the blog article "Why it is important to share your code and make your paper accessible", we advocated for sharing source code together with papers that involve a simulation-based evaluation. One argument is that such papers will be of higher utility to the reader and therefore, will create higher impact.

To help in this effort, the paper

Wilfried Elmenreich, Philipp Moll, Sebastian Theuermann, and Mathias Lux. Making simulation results reproducible - Survey, guidelines, and examples based on Gradle and Docker. PeerJ Computer Science, 5(e240):1–27, Dezember 2019. (doi:10.7717/peerj-cs.240) 

gives a good overview of which tools are suitable to provide your code in a useful way. It is also a paper that shares its code.

If you want to prepare datasets well, have a look at this paper, which provides a useful set of checks to be considered for electricity consumption datasets.

Christoph Klemenjak, Andreas Reinhardt, Lucas Pereira, Mario Berges, Stephen Makonin, and Wilfried Elmenreich. Electricity consumption data sets: Pitfalls and opportunities. In BuildSys ’19: The 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pages 1–4. ACM New York, November 2019. (doi:10.1145/3360322.3360867)

Last but not least I would like to draw your attention to an excellent list of "Papers with Code", compiled by researcher Christoph Klemenjak:

 

On this page a list of papers targeting the field of load disaggregation can be accessed. And the name of the page does not lie - all of them come with code that can be freely accessed.

Happy researching!

Privacy vs. NILM: Obfuscating your Power Consumption with Load Hiding

Load-based load hiding approach

With the development and introduction of smart metering, the energy information from costumers changes from infrequent manual meter readings to fine-grained energy consumption data. On the one hand, these measurements will lead to an improvement in costumers’ energy habits, but on the other hand, the fine-grained data produces information about a household and households’ inhabitants, which give rise to privacy issues because these monitoring results disclose user behavior which could be extracted by smart algorithms and techniques. The loss of privacy by load disaggregation and data mining is a huge upcoming smart grid and social issue which enforces the need for privacy-preserving techniques, which can be divided into the following three possibilities:

1. Anonymization of metering data: The metering data and customer identity are separated by a third-party id
2. Privacy-preserving metering data aggregation: Metering data is geographically encapsulated by aggregating the metering data of co-located consumers 
3. Masking and obfuscation of metering data: Masking the power demand by adding or withdrawing the to the meter visible energy demand with the help of rechargeable batteries or controllable loads.

In the paper

D. Egarter, C. Prokop, and W. Elmenreich. Load hiding of household's power demand. In Proc. IEEE International Conference on Smart Grid Communications (SmartGridComm'14), Venice, Italy, 2014.

a state-of-the-art battery-based load hiding (BLH) technique, which uses a controllable battery to disguise the power consumption and a novel load hiding technique called load-based load hiding (LLH) are presented and compared. A load-based load hiding system controls appliances in a specific way to obfuscate a household’s power demand. For example, an electric water boiler could be instrumented to consume energy in a way that masks the power consumption of smaller household devices like coffee machines or a TV. There is no comfort loss expected for the customer: Overall, the boiler will consume a typical amount of energy and produce the expected amount of hot water.
Using this approach, however, reduces the predictability of your energy consumption, which is good for privacy, but a disadvantage for grid operators.

Position Paper on Energy Datasets @ ACM BuildSys

Dear all,

last week, Christoph Klemenjak presented our research paper on Energy Consumption Datasets at ACM BuildSys. The paper is the outcome of a collaboration with NILM experts and discusses pitfalls and opportunities with regard to future measurement campaigns.

Abstract

Real-world data sets are crucial to develop and test signal processing and machine learning algorithms to solve energy-related problems. 
Their scope and data resolution is, however, often limited to the means required to fulfill the experimenters' objectives and moreover governed by personal experience, budgetary and time constraints, and the availability of equipment.
As a result, numerous differences between data sets can be observed, e.g., regarding their sampling rates, the number of sensors deployed, their amplitude resolutions, storage formats, or the availability and extent of ground-truth annotations. 
This heterogeneity poses a significant problem for researchers intending to comparatively use data sets because of the required data conversion, re-sampling, and adaptation steps.
In short, there is a lack of widely agreed best practices for designing, deploying, and operating electrical data collection systems.
We address this limitation by dissecting the collection methodologies used in existing data sets.
By offering recommendations for data collection, data storage, and data provision, we intend to foster the creation of data sets with increased usability and comparability, and thus a greater benefit to the community.

Find the paper here.

Please direct feedback to klemenjak@ieee.org

Have a great day,

Christoph

New NILM paper on Comparability!

Dear all,

we proudly announce our latest NILM paper on comparability in NILM scholarship. In this paper, we discuss data noise, appliance events as well as performance evaluation in general. The paper is to be presented at 2020 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT) in Washington DC.

Abstract

Non-Intrusive Load Monitoring (NILM) comprises of a set of techniques that provide insights into the energy consumption of households and industrial facilities. Latest contributions show significant improvements in terms of accuracy and generalisation abilities. Despite all progress made concerning disaggregation techniques, performance evaluation and comparability remains an open research question. The lack of standardisation and consensus on evaluation procedures makes reproducibility and comparability extremely difficult.
In this paper, we draw attention to comparability in NILM with a focus on highlighting the considerable differences amongst common energy datasets used to test the performance of algorithms. We divide discussion on comparability into data aspects, performance metrics, and give a close view on evaluation processes. Detailed information on pre-processing as well as data cleaning methods, the importance of unified performance reporting, and the need for complexity measures in load disaggregation are found to be the most urgent issues in NILM-related research.  In addition, our evaluation suggests that datasets should be chosen carefully. We conclude by formulating suggestions for future work to enhance comparability.


Get the paper here.   -   Explore supplemental material here.

Please direct feedback and further comments to klemenjak@ieee.org

Have a great day,

Christoph

Open thesis topics

Analysis of dataset for energy forecasting

Keywords: RES, energy forecasting, training dataset, testing dataset,  prediction error

Description and objectives:

Integration of Renewable energy resources (RES) in today's power grid depends highly on the quality of energy forecasting outcome. However, the variability of RES poses several challenges to this integration. The idea of the proposed thesis is to assess the influence of different dataset composition which includes varying training, validation and testing dataset on the prediction error and to come up with the most effective training method. The performance of the training method will be validated over publicly available dataset using available performance evaluation metrics.

Main tasks:

• The first task is to look for dataset having different characteristics in order to achieve better comparison and assessment of most effective approach when applied to different datasets.
• The next task is to train neural network using machine learning algorithms for different dataset composition. 

Qualifications: Programming skills in Python

Contact Details:  Ekanki SHARMA : Ekanki.Sharma@aau.at 

Impact of data pre-processing techniques on forecasting accuracy

Keywords: RES, data pre-processing, energy forecasting, feature selection, outlier rejection, forecasting accuracy

Description and objectives:

To integrate renewable energy sources (RES) in power grid, forecasting the photovoltaic (PV) yield is very important. Several techniques have been implemented in the literature which includes naive (time-series, statistical) methods to soft computing techniques (Artificial neural network, support vector machine, grey prediction) to improve the accuracy of the forecasting model. The idea of the thesis is to evaluate the impact of applying feature selection and outlier rejection techniques on forecasting accuracy. 

Main tasks:

• The first task is to search for available pre-processing techniques.
• The next step is to compare the forecast accuracy with and without applying data pre-processing techniques.

Qualifications: Programming skills in python

Contact Details:  Ekanki SHARMA : Ekanki.Sharma@aau.at 

Meetup on Future Research Challenges in Energy Informatics

18 - 19 April 2018 @ University of Klagenfurt, Austria

Objective

This two-day event aims to bring together researchers that are working on the topic of energy informatics in academia. 

The focus of this meetup will be on Non-Intrusive Load Monitoring (NILM). Other relevant subtopics of energy informatics such as data analytics, energy management systems, or artificial intelligence in Smart Microgrids are warmly welcome.

Within a small group, participants will present their latest findings, share experiences, discuss current issues, and discover possible ways of future cooperation and collaboration. 

Researchers interested in joining are asked to apply with a talk title and abstract. Application deadline is March 19th. A committee will decide upon acceptance till March 22nd.

Apply now

Remarks

The organisers would like to highlight that the University of Klagenfurt cannot provide any funding for expenses such as travelling or accommodation. Students of the University of Klagenfurt are eligible to attend the Keynote and Session 1.

Find further information here.

European Workshop on Non-Intrusive Load Monitoring (NILM)

From 6th to 7th of November, the 4th European Workshop on Non-Intrusive Load Monitoring (NILM) was held in London, United Kingdom. This event brought together researchers and professionals to present and discuss latest developments in the area of NILM as well as its applications. 

The sessions comprised topics such as commercial & industrial NILM, innovative algorithms, deep learning approaches, and evaluation. Also, several vendors such as Voltaware,  Qualisteo or Verv introduced their latest products.

In the poster & demo session, Christoph presented a poster on "Appliance Detection in Power Meter Readings". The poster illustrates how correlation can be utilised to detect electrical appliances in power readings. Especially for hardware with limited computational resources this approach shows promising results. For more information about the presented work refer to our paper on correlation filters for appliance detection.

Picture: Christoph Klemenjak (on the left) presenting his poster on appliance detection

A selection of the workshop's talks is available in form of a Youtube playlist.

The next major NILM event will be the International Workshop on Non-Intrusive Load Monitoring, which will take place in late February / early March 2018 in Austin, Texas.