Announcing project “Datazzle” – and connecting with CurrentCost

Standard

Today I have been working on the project I mentioned a few weeks back, to collect, store and graph data sent from my current cost unit. If you have been following this blog, you will know that I plan to make the service available to anyone, for free, so that they can do the same.

As I intend to release this service to the community in the coming months, I realised I couldn’t keep referring to the project as ‘the project’, so from here on in it has been lovingly code-named ‘Datazzle’ (say: dah-taz-all).

Ok, so what’s the deal? What is it? How will it work?

Datazzle will comprise initially of an XML web service (for receiving your sensor data)  and a companion web site, or web front-end, that will allow you to view all your sensor data from one place, graph it, and also let you define various options for sharing your data with others.

To get started, all one will have to do is sign-up for a free account, define at least one ‘environment’ (eg. “My House”) with one location inside it (eg. “Living Room”), and setup at least one sensor (eg. “Temperature sensor”). That’s it. It’s all point-and-click.

Then, you’ve got to get your hands dirty because it’s up to you to send compatible XML data to the API on a regular basis. To help you along the way, I will be releasing an open-source version of my test code (cleaned-up a little, of course!) that will initially connect to the current cost, take your API key, User Id and the environment configuration variables, and publish the data for you. This will basically convert the current cost XML data into XML that conforms to the “Datazzle Data Specification”, which is basically just a tweaked-version of the EEML standard.

You can use Datazzle to store data from ANY compliant device, not just the Current Cost…

Datazzle will be device-agnostic, meaning that you can use it to connect any physical or virtual sensor data (eg output from something like Second Life), and it will store it and allow you to navigate your data in a variety of different ways. From one Datazzle account, you will be able to setup multiple environments, and locations within those environments, to store data from as many sensors as you wish. You’ll get a ‘sensor dashboard’ to maintain all your sensors, from multiple locations.

Introducing the Datazzle Data Specification… feedback please 🙂

Although still only in the design phase, I have decided to base my XML format loosely around the EEML specification, already well established for sharing sensor data. The web service includes a web front-end that lets users setup and configure their devices and set a bunch of parameters (such as sensor types and locations) from the GUI, so the EEML that is sent to the server need only specify the corresponding ID values, instead of the actual string values every time. Take a look at the snippet to see what I mean:

<datazzle>
  <userId>123456</userId>
  <apiKey>00000000-0000-0000-0000-000000000000</apiKey>
   <eeml>
    <environment updated="2008-10-12T16:35:42" creator="" id="3456789">
      <location id="123" />
      <data sensorId="101"> <!-- Current temperature, in degrees celsius -->
       <value>28.7</value>
      </data>
      <data sensorId="102"> <!-- Current energy consumption, in watts -->
       <value>2692</value>
      </data>
    </environment>
  </eeml>
</datazzle>

You will note that there is less EEML here than in the specification and that some of the attributes have changed. I have also added “userId” and “apiKey” elements. A lof of the data included in the EEML specification has been ommitted from the EEML element you are required to send to the server, simply because you have already specified it once – when you were setting your sensor up online (i.e. the status of your feed, a description, a title, any links/copyright data etc, latitude and longitude, tags for each <data/> element and the units of measure).

Of course, when the service is available, there will be a nice UI to help you get up and running with the XML standard as quickly as possible (and a lot more documentation!). For now though, that’s all the info I have for you. If anyone has any thoughts or ideas, please feel free to comment – I’m open to adapting the specification to suit the needs of the community.

Here’s a quick preview of the tray app monitor that lets you see what data the current cost is sending to your computer, and the converted output being sent to Datazzle:

I will try to post more information and screenshots on the application and the service soon.

Thanks for reading!

Second CurrentCost display has arrived

Standard

Happy to report that my second CurrentCost display has arrived, just a day after ordering it off eBay from CurrentCost themselves. A bargain at just £15 plus postage, it enables me to have this unit on our kitchen worktop, while the other is connected to my server ready to connect to the web service I am developing.

A quick update on that: having had so much fun so far creating an infrastructure capable of handling the current cost data, I have decided to extend it so that other users from around the world can publish and share their sensor data, not just from the current cost. I know, I know… there’s also the very good Pachube out there for those who are inclined to use it, but I figured what I’d like to do is offer something a bit more consumer friendly, with graphing and social features as well.

If anyone is interested in helping me test this out (or even in lending a hand with the design and build), do get in touch. More on the current project progress soon…

Rude awakenings… Hooray!

Standard

I don’t think I have ever been so pleased to be woken from my saturday-morning slumber at 7.30am before. Still trying to open my eyes I made it to the front door, where the chipper postman handed me my package. Could it be? Was it my long-awaited CurrentCost? Yes!

After 22 days, my CurrentCost had arrived. At 7.30am, on Saturday morning!

Needless to say, it was fitted within 30 minutes. 🙂

As I write this I am just hooking it up to my PC to check all is working OK, but now the fun starts – I can begin developing my software to log the data. Watch this space!

Getting CurrentCost to talk to the web: planning

Standard
Getting it talking to the webI decided that now would be a good time to start planning for the arrival of CurrentCost. Who to invite, what food to serve … gosh, the list goes on! 🙂

Joking aside, I thought it might be beneficial to start thinking about what I want to do with the data I will collect, and, crucially, how I’m going to collect it.

Locating the CurrentCost on my network

I plan to connect my CurrentCost to my Windows Home Server. To do this, I have purchased the CurrentCost data cable direct from CurrentCost. This came with a serial connection, which the Home Server does not have. I also purchased a Serial-USB adapter from eBay for about £3.00 (let’s hope it works).

Hopefully that takes care of the connection to the server, so now I just need to start thinking about how to extract that data from the unit and store it.

I have decided to write a small, low-footprint Windows Service that will receive the data from the CurrentCost and post it via a web service to my web server. Following this, I will need a simple web service on my web server to a) authenticate the  Windows Service and b) insert the received data into a database.

I will then write a very simple web site, initially, to provide access to the data collected from the CurrentCost and offer to graph it in some way or another.

That should take care of the ground work, then it’s just a case of using my imagination to do some other cool stuff. Not sure how that will go 🙂

Hopefully the unit will arrive next week sometime and I can make a start on all this.

Watch this space for more updates.

Measuring electricity consumption, the geek’s way!

Standard

CurrentCost – The Beginning

Having just purchased our first home, and realising for the first time just how expensive everything actually is, I started to go on a ‘cost cutting’ excercise. Fuelled largely by a desire to reduce the overall amount of waste we produce and increase the amount of recycling we do, it wasn’t long before we were able to adapt our lifestyle to meet those goals. What really interests me though is our energy consumption… How much electricity and gas do we use, what are the worst offending appliances in our home over time?

This was back around winter of 2006 and what I really wanted to find was a consumer-targeted product that I could connect to my conventional electricity meter and that also connected to my PC, somehow. The goal was to start logging my energy usage into a database where I could more easily query all the data.

Unfortunately, either because one didn’t exist or because I was hopeless at searching, I couldn’t find one. It wasn’t until Christmas 2007 that I heard about a domestic product called the Owl wireless electricity meter that I had rekindled my interest in this topic once again. I placed my order for the meter in the early part of the year and it has been happily running since then, gently reminding us to turn off our lights when we’re not using them, to be more conscious about using our electric oven more efficiently and just how much it costs to run our dishwasher.

In the mean time, I had spent a lot of time diligently installing X10 home control units into various light fittings, appliances and even our alarm system, all controlled via our HP MediaSmart Windows Home Server.

This is all very well and good, but I still really wanted to start getting some useful data on our energy usage into the computer and I had been reading lots about extremely clever people with very large brains creating all manner of gizmos in their garages that could do just that. Realising that I probably couldn’t do the same, I started to search again for a consumer product that has these capabilities and finally, my long journey looked as though it had come to a successful conclusion: the CurrentCost meter. Not only did it look much better than the Owl meter I was using, but it also had a datalogging facility.

A few minutes on the CurrentCost web site and the CurrentCost wiki convinced me that if I didn’t order it soon I may possibly explode with excitement; and I promptly parted with a very reasonable £28. Later that same hour, more reading on the wiki turned up very interesting links (mostly to site’s owned by very clever people who work for IBM but also share a passion for, uh, gathering data, particularly from this very cool device) and it wasn’t long before I started reading about people who were very curious about the RJ45 socket on the bottom of the meter.

Like early pioneers, the electronics gurus set out with their multimeters and soldering irons and very quickly it seems, somebody had worked out that the CurrentCost unit diligently spits out an XML packet every six or seven seconds. Amazing. There we go: I had struck gold. Now if only I knew how on earth to make me one of these cables (this was my thought process anyway, although right now you have to remember I don’t actually own a CurrentCost yet – I had only just ordered it!).

After reading about breadboards, 9-pin D-SUB sockets and what not, I started to convince myself that actually, with all the clever guides out there, I could probably hash one together of my very own and could be ‘living the dream’, too.

Suddenly though, another article on that fantastic CurrentCost wiki told me that ‘those in the know’ could purchase a ready-made cable direct from CurrentCost themselves, all for the princely sum of £11.12, including VAT and delivery. Kerching! Approximately seven seconds later, my cheque book was flung open and the cheque was in the postbox on it’s merry way to CurrentCost in Surrey.

Ahhhh. So now all I have to do is sit and wait. Which, I’m not very good at.

You see, all this happened yesterday – which as all geeks know, is an eternity – especially when you’ve just ordered some very sexy gadgetry that you cannot wait to get your hands on. So, not wanting to leave the world of CurrentCost, I started to read around about things other people were doing. And then, I came across Arduino and Freeduino.

Both these boards appeal to me because they are specifically aimed at electronics novices, but they seem to be expandable and robust enough to be the gadgets that experienced pro’s love to use as well. So, what is it? Well, in a nutshell, the Arduino is a little electronics prototyping board that has a small processor and a series of digital inputs and outputs, and a connection to your computer. Bingo. Now we have a way of wiring up my whole home to my computer.

And I’m good with computers! As a programmer, I love writing software to do cool things. And what on this earth could be cooler than somehow getting all of this data from the ‘physical world’ of my home into a digital format I can pump into some software?

If you answered ‘lots of things’, then I am simply stunned that you have kept on reading through this geek-oriented babble! But, since you’re here, you might as well read on.

The day before ordering my CurrentCost and data cable, I had also splashed out on a USB 4 PORT CCTV capture card. It had to be USB, because I planned to be using my HP Media Smart Windows Home Server to capture all the imagery.

And that’s where I’m going to leave this post. Thanks to all the wonderful people blogging about the CurrentCost unit (as I have said, they all mainly seem to be IBM or ex-IBM employees!), and the super work they have done so far, they have rekindled my interest in this topic and I plan to write some software of my own, to sit on the MediaSmart Windows Home Server and do all sorts of wonderful things (of course, not quite sure what they’ll be yet).

From the next pay cheque, too, I think I’ll go out and buy all the cool Arduino bits because I really want to find a way to build my own gas meter reader! Watch this space…