Category Archives: Analytics

Empowering the Many

Posted on by

Hello MEAN stack

A few years ago I had boat loads of temperature envelop data of my house and outside temperature. When I was looking for quotes to re-insulate my old house, an insulation vendor expressed interest in purchasing my before and after analysis and results. I did not proceed with a full re-insulation of my house but did end up loosing my data which was 100% my fault. I did not back up to a NAS and experiences a hard drive failure.

Fast forward today. There is lots of talk of IoT, Analytics, and cloud services. Many, I feel are putting lipstick on their outdated products so buyer beware.  That said, the various IoT ecosystems provided through services such as Microsoft Azure, etc. are making it easier to mashup, collect, aggregate, and analyze data. Alarm Management, historians may become moot at some point unless vendors provide added value services such as predictive analytics and performance management solutions.

My interests these days revolve around machine learning and visual analytics but I do like to keep on top of some technology that can be used to marry IoT with the enterprise. With the handful of XBee devices lying around, I’ve set my eyes to ramp up on the MEAN (MongoDB, ExpressJS, Node.js, AngularJS stack and see what I can come up with for my own use at home. I chose a Typescript/Javascript environment as I can get by with basic open source tools and decent editors without having to get something like visual studio.

Key System Architecture Components

 

1-configure XBee end devices to sleep and send to coordinator AI/DI data. (I’ve tested this a few years ago so I know it works) (Temperature, ambient light, etc) Mesh network using API mode.

2. 0 or more routers to relay the messages from the end devices to the coordinator

3. 1 coordinator that feeds into the system via serial port

4. Node.JS+ Express to handle the configuration of the I/O wired to the XBees. e.g. scaling, tag name, etc. MongoDB to persist the data, and angularJS to render the UI.

5. There are three IoT platforms ( GE Predix , Xively, ThingSpeak, and Azure IoT )  that I have accounts with that I would like to push data to to test it out. I have two SCADA and one HMI system that I am also going to test out the IIoT readiness.

6. My home power monitoring has been running for 8 years on arduino and XBee. The next step is to push data rather than poll from the host to see what that SCADA system can do.

Further down the horizon the inclusion of some  MQTT flavour and and node.js integration.

Munging my “Smart” Home Data

Posted on by

Context

For some of you who live in older homes, the feeling of too cold in the winter and too hot during the summer comes with the package. Last year, I thought of getting spray foam installed and figured now is the time to start analyzing the some of the data I’ve been collecting over the past three years. I told the installer that I had three years of data and would do a before and after test to see what the spray foam did in terms of performance. He stated he would purchase the data analysis. Needless to say one could just look at the heating bill to see if there is difference.  With variance in unit  costs of fuel, admin, etc., I did not want to bother normalizing that info. The geek in me wants to explore data mining and inference. So off I go to explore  Linear discriminant analysis and random forests.

For this exercise, I had three in-home temperature points, one outside, and several power related measurements. So far I had close to 3 million data points. I searched the web for an open source toolset that could help me with data analytics and decent plotting capabilities. Given I used the R programming language a few months back and liked the graphing capabilities in the ggplot2 package, it became my tool of choice. Note that Python is making in-roads in the data analysis space and for now, I want to remain focused on data analysis so R it is.