Category Archives: Automation

Intellectual Property and Control System Integrators

TwinCAT 3 Tutorial Complete

Over the past year I’ve been working on a TwinCAT 3 Tutorial, and with the posting of the 13th chapter about the TwinCAT 3 Scope View, I’m considering it “complete.” I’ve covered all the major topics that I wanted to cover.

Writing a tutorial like this was actually a learning experience for myself. Before I made any statement about the software I tried to validate my assumptions first, and many times I learned that I had preconceived notions about how it worked which may not have been completely accurate. If you’re a TwinCAT 3 programmer, then you need to know your tool set inside and out. If you’re not sure how something works, it’s best to setup a quick test and try it out. That’s one of the advantages of PC-based software: if you want to run a test, your office PC is a laboratory environment. Write some code and run it.

Thank you to everyone who offered feedback on the existing sections, and to those of you who kept pushing me to continue writing more. I sincerely hope this can be that missing link to take people from the world of traditional PLCs into the more interesting and ultimately more powerful world of PC-based control.

It looks like TwinCAT 3 has a viable future ahead of it. Interest seems to be growing over time (according to Google Trends, anyway):

That makes TwinCAT 3 a valuable skill to learn if you’re an integrator, and an interesting technology to investigate for existing plants.

How to Write a Big PLC Program

5 Replies

Staring down the barrel of a big automation programming project is intimidating. It’s hard to even know where to start. Even when you’ve done a few before, you’re only marginally more confident the next time.

I have quite a few big automation programming projects under my belt, so I think I can generalize the process a bit. Here goes:

1. Get the Prints

There’s almost no point in starting to program unless you have an almost final set of electrical drawings. If you don’t have them yet, push for them, and go do something else productive until you get them.

2. Create a Functional Specification

You don’t always have to write out a functional specification, but it at least needs to exist very clearly in your head. If at any point, you don’t know exactly how the machine is supposed to work in excruciating detail, stop writing code and go figure it out. Ask stakeholders, talk to operators, whatever it takes. Functional specifications are best written as a list of “user stories”. If you’re not sure what a functional spec should look like, check out Painless Functional Specifications by Joel Spolsky.

3. Shamelessly Copy

Identify what other projects you can find with logic that you can steal. Any code that works in another machine has the advantage of already being debugged. Don’t re-invent the wheel. (At the same time, never blindly copy logic without understanding it. Copying the code by re-typing it one rung at a time is still faster than writing it from scratch, and it’s a form of software review.)

4. Structure Your Project

Now you break open the ladder logic programming software and start creating your project. Pick your CPU type, setup the I/O cards based on the electrical drawings. Map your inputs. Plan out your program by creating programs or routines for each functional unit of the machine. Setup your fault summary rungs and your alarm logic.

5. Write the Manual Mode Logic

PLC logic is typically written “bottom up.” Manual mode logic is the lowest level of logic because it deals directly with individual functions in the machine. Advance cylinder. Retract cylinder. Home axis. Jog axis. While you’re writing the manual mode, this is when you take extreme care making sure that actions are interlocked so the machine can’t crash. If you’re using the Five Rung Pattern, this means paying attention to what goes in the Safe rung. Does cylinder A always have to advance before cylinder B can advance? The Safe rungs should reflect that. Make sure that even in manual mode, the operator (or you) can’t break the machine. Make sure to hook your faults and alarms into the applicable fault summary rungs and alarm logic.

6. Write Part Tracking Logic

Now that manual mode is complete, write the logic that tracks parts (and their state) through the machine. Remember, you should be able to run the machine in manual mode, and the part tracking should (ideally) work correctly. I know this isn’t always the case but surprisingly part tracking in manual mode can work 95% of the time. That means part tracking works based on the state of the machine. Closing the gripper with the robot in the pick position and the part present in fixture sensor on should latch a bit “remembering” that the gripper has a part in it.

Once you’ve written your part tracking logic, go back and use the part tracking and state bits to condition your Safe rungs. Don’t let the operator (or you) mistakenly open the gripper if the gripper has a part and isn’t in a safe position to let go of the part. Of course, you may need to add a way to manually override this (that’s what output forcing was created for), but in most cases you want to prevent improper operation.

Part of writing the part tracking logic is adding “ghost buster” screens. Operators often need to remove parts from a cell, and if the machine can’t detect their removal, then you have to provide the operator with a way to clear these “ghosts.”

At this point you’re actually ready to dump the program in and start testing out the machine electrically and mechanically. While it’s ideal to have a fairly complete program when you go onsite, we all know that’s not always entirely possible. At the very least you want to get to this point before startup begins.

7. Write the Auto Mode Logic

The complexity of your auto mode logic depends on what type of machine you’re programming. You’ll always need a cycle start and a cycle stop feature. Even if you’re in auto mode, you usually don’t want the machine to start until the operator specifically tells it to start. Once it’s running, we call this “in auto cycle.”

In simple machines, you can write the auto logic by filling in the Trigger rungs in your Five Rung logic. Start by putting the In Cycle contact at the beginning of the rung, and then writing logic after that which expresses when the action should take place. For instance, an advance reject part cylinder’s Trigger rung could be as simple as In Cycle, Part Present, and Part Rejected. As long as the Part Present tracking bit gets cleared once the cylinder is in the advanced position, then this is all the auto mode logic you need for this motion. Have the retract Trigger rung be In Cycle, No Part Present and Not Retracted.

More complicated machines need more complicated auto mode logic. If your machine has to perform a series of steps (even if some of them are in parallel) then consider using the Step Pattern. If your machine needs to choose between several possible courses of action (commonly seen in a storage and retrieval system) then consider using the Mission Pattern.

8. Review

It’s hard to write correct logic. Review your functional specification, point by point, and make sure your logic meets all of the requirements. Check your logic for errors. A fresh look often uncovers incorrect assumptions, typos, and outright mistakes. The earlier you find and fix problems, the easier they are to fix.

Make a list of everything you have to do during startup. Starting up a machine is time consuming and therefore expensive. Anything you can do to prepare saves you time and money.

Good luck, and keep your fingers out of the pinch points!

Price of Oil Affecting Alberta and Ontario

Start your own Automation Blog!

9 Replies

One thing I’ve discovered about automation blogging is that it’s a pretty lonely place. Don’t get me wrong, there are a couple gems out there, but I don’t find many people writing about what it’s like in the trenches wading through rungs of ladder logic. In the .NET world there are tons of programming blogs with posts about every issue you could ever come across. Why such a dearth of information in the automation space?

It occurs to me that blogging seems difficult to a lot of people, as if you need to be a web programmer. That’s totally untrue (I’m absolutely not a web programmer). In fact there are a ton of inexpensive and simple options out there.

Maybe people wonder what to write about. That’s easy. At first I worried about writing something people didn’t already know, but it turns out there’s no shortage of shiny new graduates looking for any toe-hold they can get in this industry. Try to think back to when you didn’t know anything. How did you figure out how to get online with a PLC the first time? Did you have to call the office to ask someone, and feel foolish because you didn’t know what a Null Modem cable was? We were used to asking our peers for help, but the new generation grew up looking up how to do things on the web.

Maybe you’re worried about the cost. Shared hosting plans are very inexpensive. I use DreamHost, specifically because they’re inexpensive (starts at $9 per month including domain name, unlimited storage and bandwidth), the hosting is rock solid, they offer one-click installs of blogging software (such as WordPress), and their technical support is excellent.

It would really be great if I didn’t feel like the only voice in this cloud. Grab a blogging account and chime in. Write about a hard problem and how you solved it. Disagree with me! Help me learn something new!

Offline Changes to a PLC Program

11 Replies

As a PLC programmer, you’ll often be asked to do a change to an existing system. If there’s a significant amount of functionality to be added, you generally get your changes ready “offline” and then do all the changes during a short window of time to minimize disruption to the production schedule.

If you’re using an Allen-Bradley PLC, the procedure is typically this:

Get a copy of the latest program from the PLC (a.k.a. an “upload”)
Make your changes to the offline copy, and write down every change you had to make
Go online with the PLC and apply your changes as online changes

Step 3 is much safer than just taking your modified program and doing a “download”. That’s mainly because when you download, you’re not just downloading the program, but the memory state of the PLC as well. The PLC typically has to track things in memory (like recipe data, part tracking, data collection, sequence numbers, machine counters, etc.). If you do a download, you’re going to overwrite all those values with previous values, and that can cause a lot of problems. The other thing step 3 saves you from is simultaneous changes that were done online while you were busy making offline changes.

The only other option you have is upload-change-download, but you really have to shut the machine down for the duration to make sure that the internal state doesn’t change.

When I did a lot of Allen-Bradley programming, I didn’t question that. It’s just how it was. I remember visiting a plant one time for a service call, and the local maintenance person was a bit suspicious of what I was going to do (after all, I was a young kid who had never seen this machine before). He decided to quiz me a bit, and one of the things he asked was, “when you go online, do you download or upload?” I said “it depends,” but his answer was, “you never download.” I agreed that someone in a maintenance role should never need to do a download unless they’re replacing a CPU, or recovering from a corrupted PLC program.

Now that I mostly do Beckhoff TwinCAT 3 programming, I realized one of the benefits are that offline changes are a breeze. It’s due to the fact that TwinCAT 3 completely separates the program from the memory data. The program is stored in local files on your hard drive and compiled into a TMC file. The persistent data is stored in a different place on your hard drive.

When I want to do offline changes to a TwinCAT 3 project, here’s the procedure:

Get a copy of the latest program
Make your changes to the offline copy
Copy changes back to the machine (keeping a backup, of course), rebuild, and activate configuration

This makes offline changes go a lot more smoothly, of course. I don’t have to copy and paste my changes in while online, so it takes less time and eliminates the possibility of a copy/paste error.

Since we also use Mercurial for version control, getting a copy of the latest program is a matter of pulling the latest from the source control, and copying it to the machine is a matter of pulling the offline changes to the machine. Any changes that were done in parallel can be merged with Mercurial’s built-in diff and merge utilities. (Note: I/O changes can’t be merged nicely, so if someone changed the I/O while you were doing your offline changes, you have to copy those changes in manually, but that’s rare and at least it tells you that it can’t merge them.)

This got me thinking that Allen-Bradley probably has a better way of doing offline changes that most of us just don’t know about. I know that you can do an upload without uploading the memory. However, it seems like it requires you to download both the program and data at the same time. I wonder if anyone out there knows how to do better offline changes to a ControlLogix. If so, I would be interested to know that.

Announcing: the TwinCAT 3 Tutorial

Sending a Fanuc Robot’s Position to the PLC

13 Replies

This information is for a Fanuc R30-iA, RJ3-iA or RJ3-iB controller but might work with other ones.

If you’re looking for a way to send the robot world TCP position (X, Y, Z, W, P, R) over to the PLC, it’s not actually that difficult. The robot can make the current joint and world position available in variables, and you can copy them to a group output in a background logic task. There is one caveat though: the values only update when you’re running a program. They don’t update while jogging. However, there is a work-around for this too.

First you should make sure that the feature to copy the position to variables is enabled. To get to the variables screen, use MENU, 0 (Next), 6 (System), F1 (Type), Variables.

Find this variable and set it to 1 (or True): $SCR_GRP[1].$m_pos_enb

The name of that variable is Current position from machine pulse

Now create a new robot program, and in it write the following:

GO[1:X POS]=($SCR_GRP[1].$MCH_POS_X*10) GO[2:Y POS]=($SCR_GRP[1].$MCH_POS_Y*10) GO[3:Z POS]=($SCR_GRP[1].$MCH_POS_Z*10) GO[4:W ANG]=($SCR_GRP[1].$MCH_POS_W*100) GO[5:P ANG]=($SCR_GRP[1].$MCH_POS_P*100) GO[6:R ANG]=($SCR_GRP[1].$MCH_POS_R*100)

Note that I’ve multiplied the X, Y, and Z positions by 10, so you will have to divide by 10 in your PLC. Likewise I multiplied the W, P, and R angles by 100, so divide by 100 in the PLC.

To run this program in the background, use MENU, 6 (Setup), F1 (Type), 0 (Next), BG Logic. Configure it to run your new program as a background task.

Obviously you need to send these group outputs to the PLC. Ethernet/IP is great for this, but you can use hardwired interlocks too. You need to make sure that you have enough bits to handle the full range of motion. A 16-bit integer should work pretty well for all of these. Note that the robot will happily send negative numbers to a group output as two’s complement, so make sure you map the input to the PLC as a signed 16-bit integer (a.k.a. INT in most PLCs). For the X, Y, and Z positions, a 16-bit integer will give you from +3276.7 mm to -3276.8 mm of total range. For the W, P, and R angles you’ll get +327.67 deg to -327.68 deg. For most applications this is good (remember this is TCP, not joint angles). Please check that these are suitable though.

As I said, these numbers don’t update while you’re jogging, and won’t update until the robot starts a move in a program. One little trick is to do a move to the current position at the start of your program:

PR[100:SCRATCH]=LPOS J PR[100:SCRATCH] 10% FINE

This starts sending the position without moving the robot. In my programs I typically enter a loop waiting for an input from the PLC, and inside this loop I turn a DO bit on and off. The PLC detects this as a “ready for command” heartbeat, and as long as the PLC sees this pulsing, then it knows the program is running and the position data is valid.

Another trick you can use is to detect when the robot has been jogged:

DO[n]=$MOR_GRP[1].$jogged

The name of this variable is Robot jogged. The description from the manual is: “When set to TRUE,the robot has been jogged since the last program motion. Execution of any user program will reset the flag.”

That’s how you get the world position of the TCP into the PLC. If you just want joint angles, you can use $SCR_GRP[1].$MCH_ANG[n] as the variable, where “n” is the joint number.

Important note: The I/O will probably change asynchronously to the program scan, so what you want to do is make a copy of the X, Y, Z, W, P, R values coming into the PLC and compare the current values to the values from the last scan. If they haven’t changed, then update your actual values, otherwise throw them away because they might not be valid. If you have a fast scanning PLC and I/O then you should still be able to keep up with the robot even during a fast move. If you have a slow scan time on your PLC, then you might only get valid stable values when the robot is stopped.

Now what if you want to know what the TCP position is relative to one of your user frames? The robot controller doesn’t seem to give you access to this, but the PLC can at least calculate the X, Y, and Z positions of the TCP in your user frame itself, given the world position and the user frame parameters.

First you need to find the accurate user frame parameters. Under the normal frames screen you can only get one decimal point of accuracy, but you need the full 3 decimal points to have your numbers in the PLC match the user frame position given in the robot. You can find these accurate positions in a variable: use MENU – 0,6 (SYSTEM) – F1 (TYPE) – Variables – $MNUFRAME[1,9] – F2 (DETAIL). The second index in square bracket is the frame number, so $MNUFRAME[1,1] is frame one and $MNUFRAME[1,2] is frame 2. Copy these numbers down exactly.

Here’s the math for calculating the TCP relative to your user frame. All variables are LREAL (which is a 64-bit floating point variable). I don’t know if you can use a regular 32-bit float or not. Result is your TCP in user frame. Point is your point in world frame (from the robot) and Frame is the accurate user frame data you copied from the $MNUFRAME[] variable.

Result.X_mm := Point.X_mm - Frame.X_mm; Result.Y_mm := Point.Y_mm - Frame.Y_mm; Result.Z_mm := Point.Z_mm - Frame.Z_mm;


RadiansW := DegreesToRadians(-Frame.W_deg);

CosOfAngleW := COS(RadiansW);

SinOfAngleW := SIN(RadiansW);
RadiansP := DegreesToRadians(-Frame.P_deg);

CosOfAngleP := COS(RadiansP);

SinOfAngleP := SIN(RadiansP);
RadiansR := DegreesToRadians(-Frame.R_deg);

CosOfAngleR := COS(RadiansR);

SinOfAngleR := SIN(RadiansR);

// Fanuc applies rotations WPR as W (around Z), P (around Y), R (around X) // AROUND Z temp := Result.X_mm; Result.X_mm := Result.X_mm * CosOfAngleR - Result.Y_mm * SinOfAngleR; Result.Y_mm := Result.Y_mm * CosOfAngleR + temp * SinOfAngleR; // AROUND Y temp := Result.Z_mm; Result.Z_mm := Result.Z_mm * CosOfAngleP - Result.X_mm * SinOfAngleP; Result.X_mm := Result.X_mm * CosOfAngleP + temp * SinOfAngleP; // AROUND X temp := Result.Y_mm; Result.Y_mm := Result.Y_mm * CosOfAngleW - Result.Z_mm * SinOfAngleW; Result.Z_mm := Result.Z_mm * CosOfAngleW + temp * SinOfAngleW;

Note that DegreesToRadians() is just PI*deg/180.

Run that on your PLC and check that the values in your Result variable match the user frame TCP position reported on the teach pendant.

I haven’t gotten around to calculating the W, P, and R angles of the TCP in user frame yet. Currently I just look at W, P, and R in world frame if I need to know if I’m “pointed at” something. If you get the math to work for W, P, and R, I’d really appreciate if you could share it.

Automation and the Guaranteed Minimum Income

6 Replies

In recent years I’ve been interested in the effects of automation on our economy and our society. Throughout history every advance in technology has brought more wealth, health, and opportunity to pretty much everyone. With every revolution people changed jobs but their lives got significantly better. When farms mechanized, workers moved into the city and got factory jobs, and Henry Ford’s assembly lines made use of this labor to great effect.

Early factories needed labor in great quantities and as industrial processes became more efficient at utilizing labor, the value of human labor rose and the demand kept increasing. So did pay. Factory workers up through the 70’s could afford a nice house to raise a family, a big car, and even a boat or nice vacations. Since the 70’s however, the purchasing power of a factory worker or even a bank teller has been pretty flat. These are two professions that have seen the most advances in automation in the last 30 years, due to industrial robots and automated tellers. If automation makes workers more productive, why aren’t we seeing that translate into purchasing power?

There are two types of technological improvements at work here. A farmer with a tractor is very productive compared to one with a horse and plow. The displaced farm workers who went to the city were given the tools of the industrial revolution: steam engines, motors, pumps, hydraulics, and so forth. These technologies amplified the value of human labor. That’s the first kind of technological improvement. The second kind is the automated teller or the welding robot. The older technology adds value even to the lowest skilled employees, but the new technology is reducing their value and the new jobs require significantly higher skill levels. There’s something about this new revolution that’s just… different. The demand for low skill labor is drying up.

The increasing divide between the “haves” and the “have-nots” has been documented extensively. Some divide is good and promotes the economy and productivity. Too much separation is a recipe for significant problems.

I’m not the only one worrying about this issue, and as I’ve followed it over the last few years I’ve been surprised by the amount of interest in a Guaranteed Minimum Income or some such plan. Basically it involves getting rid of every low-income assistance plan such as social security, welfare, minimum wage laws, etc., and creating a single universal monthly benefit that everyone is entitled to. Some people are talking about a number as high as $24,000 per year per adult. Considering that the 2015 federal poverty level in the US is just below $12,000 for a single adult, you can see that $24,000 per adult isn’t just a trifling amount.

For comparison, a little Googling tells me that the US GDP per capita is around $55,000. Think about that for a second. You’re talking about guaranteeing almost 45% of the productivity output of the country to be distributed evenly across all adults. One presumes you would also provide some extra money per child in a household, but to be fair the “per capita” figure includes kids too. It’s possible. Sure seems a bit crazy though.

Is it practical? Won’t some people choose not to work? Will productivity go down? It turns out that we’ve done some experimenting with this type of program in Canada called MINCOME. The results were generally positive. There was a small drop in hours worked by certain people, mostly new mothers and teenagers. These costs were offset in other areas: “in the period that Mincome was administered, hospital visits dropped 8.5 percent, with fewer incidents of work-related injuries, and fewer emergency room visits from car accidents and domestic abuse.” More teenagers graduated. There was less mental illness.

I’m fiscally conservative, but I’m mostly pragmatic. It’s only my years of exposure to automation, technology and working in factories that makes me ask these questions. Not only do I believe that people should contribute, I believe that people need to contribute for their own happiness and well-being. That’s why I don’t think paying people to sit at home is the ultimate solution.

The elephant in the room is this: as technology improves, a greater proportion of the population will simply be unemployable. There, I said it. I know it’s a disturbing thought. Our society is structured around the opposite of that idea. Men are particularly under pressure to work. The majority of the status afforded to men in our society comes from their earning potential. The social pressure would still be there to work, even as a supplement to a guaranteed minimum income, so we still need to find something for those people to do. Perhaps if we expand the accepted role of men in society then we can fill that need with volunteer work. Maybe.

What’s the right answer? I don’t know. For lack of a better term, the “American Dream” was accessible to anyone if you were willing to work hard and reinvest that effort into yourself. Not everyone did that, but many people created significant fortunes for themselves after starting in the stockroom and working their way up. That security gave people a willingness to take risks and be entrepreneurial. Proponents of the idea say that a minimum income would bring back that innovative edge. Entrepreneurs could try new ideas repeatedly until they found one that worked, and not worry about their family starving. With your basic necessities met, you could start to realize your potential..

I do know that as we continue down this road of increasing automation, we can’t be leaving a greater and greater proportion of the populace without the basic resources they need to survive. Do we expect them to grow their own food? On what land? Do we expect them to do a job that I could program a robot to do, if the robot’s average cost is only $10,000/year? Do you have some valuable job we can retrain them to do? One that pays enough to support a family?

Look, I don’t like the alternatives either, but it’s better than an armed revolt.

Whole Home Energy Monitoring with the Brultech ECM-1240

2 Replies

This Christmas I asked Santa for a Brultech ECM-1240 whole home energy monitoring system, specifically the DUO-100 package. After reviewing various products, I really liked that this one was priced quite reasonably for the hardware, and that they published the communication protocol.

You (or a licensed electrian) install the ECM-1240 at your main electrical panel. Each ECM-1240 has 7 channels. In my case, the first channel on the first unit measures the incoming main line current. For the other 13 channels you are free to choose various circuits from your panel that you want to monitor. You can gang various circuits together into a single channel if you like (all lighting loads, for example). The device uses current transformers on each circuit for the monitoring. These are installed inside the panel. The hot wire coming out of each breaker has to go through a current transformer, so this isn’t a simple plug-in installation; there is wiring to be done.

The company distributes software for the device for free, or you can choose from various 3rd party software. Alternatively you can configure the device to send data to websites.

I’m not a fan of sending my home energy data to someone else’s server. Apart from being a huge privacy concern (it’s pretty easy to see when you’re home, and when you went to bed), I don’t want to pay a monthly fee, and I don’t want to worry about how to get my data from their server if they go out of business or decide to discontinue their product. For those reasons, I installed their software. It’s basically a flash website that I hosted on our Windows Home Server, which is conveniently the computer hooked up to the ECM-1240’s as well.

At first their software worked quite well, but over time it started to show problems. After just less than 2 months of logging, the flash program was so sluggish to load a page that it took over 2 minutes to refresh a screen. Admittedly I was running it on an older PC. Additionally, in that amount of time it had already logged about 680 MB of data. That seemed excessive. It also logged to a SQL Lite database (a single-user file-based database), and unfortunately kept the database file locked all the time. The website would end up locking the database and packets started getting buffered in the logging software until you closed down the website and released the lock.

I decided I’d just write my own software:

I’m a C# developer, so that was my language of choice. If you’re a .NET developer too, I’ll include a link to my source code at the end of this post. Note that this software isn’t commercial quality. It’s semi-configurable (channel names, and so on) but it assumes you have a 14 channel system where the first channel is the main panel input. If your system is different, you’ll have to make some modifications.

These devices I purchased use a serial port for communication, but they give you an RS232 splitter cable so you only have to use up one serial port on your PC. Their software relies on putting the device into an automatic send mode… the device itself chooses when to send packets. If the load changes significantly on one of the circuits, it triggers an immediate packet send. Unfortunately with 2 devices on the same RS232 port, you can sometimes get a collision. Their software deals with this by detecting it and ignoring the data, but I had a sneaking suspicion that once in a rare while, it ended up getting a corrupt packet, and there was at least one point where their software logged an extremely high energy reading, and it didn’t make any sense. Therefore I wrote my software to use a polling mode. It requests data from the first device, waits about 5 seconds, requests it from the other device, waits 5 seconds, and repeats. On average we get one reading about every 10 seconds from each device. Testing seemed to indicate this was about as fast as I could go because there was a reset period after you talked to one device where you had to wait for it to time out before you could address the other one. In retrospect, if you just hooked them up to 2 separate serial ports you could probably poll the data faster.

Next I had to choose how to log the data. I really wanted the data format to be compact, but I still wanted decently small time periods for each “time slice”. I also didn’t want to be locking the file constantly, so I just wanted to be able to write the data for each time slice, and be done with it. I settled on a binary format with fixed length records. Here’s how it works: each day’s data is stored in a separate file, so the data for April 1st is stored in a file called 2015-04-01.dat. The device generates values in watt-seconds. I calculated that at a maximum current of 200A (100A panel x 2 legs) x 120V, then in a 10 second time slice I should log a maximum of 240,000 watt-seconds. A 20-bit number (2.5 bytes) can store a maximum value of 1,048,575. I didn’t want to go smaller than half-byte increments. So, 14 channels at 2.5 bytes per channel gave me 35 bytes for each 10 second time slice, and since the devices generate voltage I figured I’d log the voltage as a 36’s byte just to make it an even number. Using those numbers, running 24 hours a day for 365 days a year, this will use up under 110 MB/year. Not bad. A flat binary file like this is also fast for data access. The data is small, and seeking a time slice can be done in O(1) time. After you find the first time slice you just keep reading bytes sequentially until you get to the end of the file or your last time slice. Hopefully no more 2 minute load times.

I broke up my application into multiple programs. The first is a utility program called Uploader.exe. This is a command line program that reads the data from a device and just spits it out to the screen. Useful for testing if it works.

The second is called Logger.exe and it does just that. It uses Uploader.exe to read values from the two ECM-1240 units at 5 second offsets and writes the data to XML files into a PacketLog folder. Each file has the timestamp of when it was read, and the device number to indicate which device it came from.

The third program is called PacketProcessor.exe. This program monitors the PacketLog folder and waits until it has at least one packet from each device after the end of the current time slice it’s trying to build. If it does, it calculates the values for the current time slice and appends it to the end of the current day’s data file. It then deletes any unnecessary packets from the PacketLog folder (it keeps at most one packet from each device before the end of the most recently written time slice).

To host Logger.exe and PacketProcessor.exe, I used nssm a.k.a. “the Non-Sucking Service Manager”. It’s a small command line program you can use to run another command line program as a service. Very handy.

The fourth program is called Viewer.exe. It’s the graphical interface for viewing the power data. Here’s a screenshot of data from one day (just the main panel power, click on it to make it bigger):

That’s a rather typical day. The big spike around 5 pm is the oven, and you can see the typical TV-watching time is after 8 pm once the kids are in bed. On the right you can see it breaks out the power usage in kWh for each channel, and in our area we have time-of-use billing, and this actually breaks the power usage out into off-peak, mid-peak, and on-peak amounts.

The viewer program uses a line graph for all time periods under 24 hours, but switches to an hourly bar graph for periods longer than that. Here is a 48-hour time period, and I’ve changed from just the Main Panel to all the branch circuit channels instead:

If you ask for a time period of more than a week it switches into daily bars:

Pulling up two weeks of data like that was actually very fast, just a few seconds.

One final feature I added was email alerts. One of the channels monitors just my backup sump pump circuit. If it ever turns on, that’s because my main sump pump has failed, and I want to know. Through a configuration file, I configured an alarm to email me if that circuit ever records more than 5W of power in a 10 second time slice. I did a little test and it works (it does require that you have a Gmail account though).

Here’s the source code, if you’re interested:

PowerCruncher 2015.04.19

As always, feel free to email me if you have questions or comments.

Contact and Coil

Nearly In Control