Author Archives: Scott Whitlock

About Scott Whitlock

I'm Scott Whitlock, an "automation enthusiast". By day I'm a PLC and .NET programmer at ETBO Tool & Die Inc., a manufacturer.

Doing it “Right”

2 Replies

As an engineer I notice I’m in a minority of people who are obsessed with discovering the “right” way to do something. At the same time, I know that there’s more than one way to do it. Not only that, but the “right” way isn’t the right way until someone discovers it, and the right way becomes the wrong way once someone discovers a better way (and then does the hard work of convincing everyone else that it’s better).

When we say there’s a “right” way, we’re implying that there’s also one or more “wrong” ways, but we’re also implying some more subtle nuances:

  • The “right” way is rarely the one that’s obvious to the novice
  • The “right” way takes more time, effort, and resources up front
  • It’s the “right” way because the additional investment pays off in the long term

I think these are interesting and insightful observations. All of them are strictly tied to experience. Nobody starts their first task on their first day of work and says, “wow, I can do this two different ways… I have no other way to weight the value of these strategies, so I’ll choose the one that takes more effort, time, and money.” By default, we choose the easiest, fastest, and cheapest route available to us. We change our strategy (if ever) only after seeing the outcome of the first trial. We only change if we see that the extra investment now will pay off for us down the road.

That’s why a homebuilder only builds your house “to code”. Have you ever seen how they build their own home? They put extra reinforcing material in the foundation, they use better materials, use longer lasting shingles, and they take care to get the best people to work on it. That’s because to them, the “right” way is different if they’re building your house vs. their home. Your house is just short-term profit, but they want their home to pay them back after the long haul. This is normal, logical, and selfish behaviour.

Yet I think Engineers, Architects, and Designers have some malfunction in their DNA that makes them obsessed with doing what’s in their client’s best long term interest even if there’s no benefit for them personally. There’s some kind of obsessive-compulsive aversion to a sub-optimal design. I would argue that it’s a prerequisite for those professions.

This often leads to frustrating conversations with clients, because the Engineer (not usually that good with social interactions to begin with) is trying to convince the client that they’re going to have to spend more money than they’d planned, it’s going to take longer than they thought, and it’s going to be more difficult than they’d imagined. That is, in fact, what the client is paying for: experience. The Engineer (who is a crazy deviant, always in search of some mythical “right” way of doing things) doesn’t understand why the client is upset, since they’ll be saving a boatload of anguish in the long term.

The most frequent complaint about Engineers has to be that they make things “too complicated”, and to be certain, you can take it too far, but what we’re really doing is inhabiting the losing end of every argument. As Engineers, we’re asking people to endure pain now for the promise of a better life later. If that were easy, the dessert industry would have perished long ago.

Reading the ControlLogix System Time in Ladder Logic

12 Replies

Since I’m in the tutorial mind-set right now, I thought I’d mention this little gem. Here’s how you can read the ControlLogix (or CompactLogix) PLC system time into a UDT so you can use the current time value in your ladder logic program.

First I created a UDT called “TIME”:

RSLogix 5000 - Read System Time - UDT

Then you just need to use the GSV (Get System Value) instruction with the WallClockTime class, and LocalSystemTime attribute to read the controller’s time into an instance of your UDT (here I created a new tag called LocalDateTime of type TIME). Note that I used the Year element of the LocalDateTime tag as the parameter, because that’s the first address of the tag. It starts writing there but fills in the entire UDT with the time values:

RSLogix 5000 - Read System Time in Ladder Logic

Now you can program your sprinklers to turn on and off in the middle of the night! 🙂


The Five Rung Logic Block

5 Replies

I’m still working on my RSLogix 5000 Tutorial, and I decided to include a brief introduction to a Five Rung Logic Block. If you’re like me, the first time you saw or heard about a five rung, you thought it was was a ludicrous amount of logic just to move a cylinder from one position to another. However, I realized most of my non-five-rung motion control logic always ended up being as complicated, or more complicated, than if I’d used a Five Rung from the beginning, so now I’m a convert.

A five rung block is named after the five coils that make up each block (these names differ between programmers and organizations, but they follow this pattern):

  • Safety
  • Precondition (aka Trigger)
  • Command
  • Complete (aka “In Position”)
  • Fault

Briefly, your safety rung contains all the conditions that must be present during the entire motion. You put conditions like “no critical faults” here. You also put conditions such as interfering axes clear here. For instance, if axis A and axis B should never advance at the same time, then put A Retracted in the safety rung for the five rung that moves axis B to the Advanced position, and vice-versa. This is your “sanity check” logic.

The “trigger”, or “precondition” rung is the signal that will initiate automatic motion. You can put auto conditions directly in this rung, or you can tie in auto mode sequence logic from elsewhere. The difference between the Safety rung and the Trigger rung is that the Safety rung has to stay on during the entire motion, but the Trigger run only has to be true to initiate the motion.

The Command rung is usually a sealed in coil that turns on while the machine should be trying to move this axis to this particular position. Sometimes this is synonymous with an output itself, but in many cases the Command coil will then be used to drive one or more outputs or motion instructions.

The In Position (sometimes called “complete”) rung condition is the logic that interprets the sensors to tell us that we’re actually in the given position. It’s handy to use this coil throughout your program to indicate the state of the machine, rather than looking at the inputs directly. That allows you to invert input logic or add debounce logic later in just one place, rather than everywhere that depends on this machine state.

Finally, the Fault condition is normally a timeout fault. If the command coil is on too long without seeing the In Position signal, then we determine that the motion timed out, and we stop trying. We use this fault in our alarm logic as well.

Once the command condition becomes true, it’s normally sealed in until one of three things happens: the safety condition changes to false, the motion completes and the In Position signal changes to true (normal), or the motion times out and the Fault condition changes to true.

Here’s what a generic five rung logic block looks like. I left an AFI in the trigger rung because you normally come back and fill in that logic later. Note that this is only to move one axis to one position:

RSLogix 5000 Tutorial - Generic Five Rung Logic Block

The “PB” bit in the command rung is the HMI pushbutton that initiates motion to this position while you’re in manual mode. Note that both manual mode motion and auto mode motion have to follow the same safety conditions, but just have different trigger conditions.

You have to customize it for each situation. The contents of the safety and trigger rungs are obviously unique, but the command rung sometimes changes depending on the scenario. You may have more than just manual and automatic modes. Some automatic actions may actually happen when you’re not in automatic mode. Sometimes you may want an axis to “jog” (for instance a hydraulic axis moves slow, and you may want it only to move in manual mode if you hold your finger on the button) and in that case, you couldn’t seal in the Command bit around the manual mode pushbutton (you’d just seal it in around the trigger contact).

There are other variations. I hope you find this useful. If nothing else, when you come across this logic in someone else’s program, you’ll have a better idea of how it works.

Industrial Automation Knowledge Sharing, Revisited

1 Reply

About 6 months ago I blogged about ControlsOverload, a knowledge-sharing Q&A site for the Industrial Automation community: Make More Money with ControlsOverload.

I was pleased to read Bill Lydon’s article in InTech called Putting knowledge to work. Bill shares a story similar to what many of us have experienced:

Early in my career, I ran up against an automation systems software problem I could not figure out. Being the “new guy” in an open office with five other engineers who had experience with these systems, I decided to get their input and explained the problem as best I could. They had suggestions, but no one offered a solution. After working on the problem for a few days, I discovered some software code that only executed under certain system circumstances, which was creating the problem. I changed the code to get the system in the field working properly and wrote an engineering change request.

I proudly shared the solution with the group. One of the most knowledgeable and experienced engineers exclaimed, “I solved that problem months ago!” Pulling out a folder from his desk file drawer, he announced his notes on the solution were, “right here.” I asked why he did not tell me this a few days before, and he responded, “You didn’t ask the right question about this specific code.” I learned this was normal operating procedures with him because he believed withholding knowledge created job security.

That’s exactly the reason why ControlsOverload was created. I’ve seen the impact of withholding knowledge, and it’s painful. Engineers are entering the industrial automation career path all the time and all of them have to re-learn the same lessons over and over because older generations aren’t taking the time to share their knowledge and mentor. It’s the opposite of the behavior that built our civilization. We need to pass these expensive lessons on to the next generation so they can spend more time building better things than we ever could.

One-on-one interaction and teaching is good, but social networking adds an economy of scale that allows knowledge sharing to explode. But somebody has to take the time to seed the field before we get our first harvest.

Beckhoff TwinCAT: a First Impression

25 Replies

Those of you who know me remember I’ve been itching to get my hands on a Beckhoff TwinCAT PLC industrial control system for a few years now, but I never found a project or customer willing to take on the risk of trying something new. Like a bunch of penguins on a cliff of ice, nobody wanted to be the first in the water.

I’ve recently had the opportunity to apply a TwinCAT PLC and HMI system to a relatively low-risk application, so I finally have something to share. Those of you who know me also know I don’t pull my punches when I talk about products, so here’s an honest review of the TwinCAT system: the good, the bad, and the ugly.

Benefits

Cost

What draws you to TwinCAT is its combination of low cost with a ton of features. I can’t publish prices, but I encourage you to compare prices from Beckhoff, Allen-Bradley, Omron, and others. Beckhoff offers a similar or better feature-set, with as many or more communication options than it’s competitors, for significantly less money (it’s real-time PC control instead of dedicated hardware). On specs it can do everything, and doesn’t bend you over the table when it comes to the price.

Scalability

First, Beckhoff TwinCAT PLC takes advantage of a PC CPU and memory, which means you can scale up the performance of your system for a fraction of the cost of scaling up to large PLC and PAC systems.

Secondly, if you use EtherCAT, you get to scale up your I/O sizes without as much impact on performance or price. The individual I/O slices are significantly cheaper than for other bus technologies, and the servo drives are less expensive because the bus speed is fast enough to close the loop from the PLC rather than in the device itself.

Where TwinCAT really shines is when you need to integrate it to the MES layer in your organization. Most other vendors require you to buy expensive OPC or other connection software, but Beckhoff provides a free DLL for Windows applications to communicate with the PLC over a communication protocol called ADS. (They also provide an OPC server as an add-on if you need it.)

What’s the Catch

Switching software vendors is never easy. If you’re used to your existing PLC software like RSLogix, you’ve got a bit of a learning curve ahead of you. Here are the things that down-right annoyed me about TwinCAT when I was coming from the RSLogix/PanelView realm:

  • When you do an online edit, it resets all of your forces. You have to reload your watch list to re-apply them. (Fortunately I always write perfect code the first time, so I never had to do an online change…)
  • The ladder logic editor doesn’t seem to auto-wrap (unless I’m missing a setting somewhere)
  • The HMI editor crashed a couple of times when I was modifying the HMI, and I had to restart the TwinCAT system (not the PC, just the software)
  • The English documention is lacking
  • The TwinCAT System Manager DeviceNET master configuration had a bug where it wouldn’t recognize EDS files where the Class value was stored as a 16 bit, rather than an 8 bit value. The EDS file was valid, according to EZ-EDS, but it wouldn’t read it. Changing the value to 8-bit made it recognize the parameter. This made it hard, but not impossible, to integrate an Allen-Bradley Point I/O DeviceNET node.
  • Even though the HMI will log an alarm history, there’s no alarm history control for you to view it in the HMI itself (it’s saved to a CSV file)

Here are some things that are just really different, but I’ve had no trouble adapting to:

  • You have to define all variables (tags) in a text syntax, instead of in a spreadsheet-like tag editor
  • I/O isn’t mapped directly. You define your hardware I/O and your program I/O and then you map them.
  • You don’t do online changes while you’re online with the PLC. You “log off”, then make your change, and then “log on” and tell it to upload your change. (You can do this while the machine is running.)

What’s Cool

Using a PC based controller opened up some interesting features, particularly in the HMI:

  • You can launch another application from a button click on the HMI (Excel, etc.)
  • You can configure some interesting alarm actions, like launching a program, or sending an email
  • There’s an (optional) add-in to let the PLC communicate in a limited fashion with a SQL Server
  • It runs on commodity hardware. PC hardware is cheap and you can get much larger touch screens for a lot less than the cost of an equivalent PanelView. The “fieldbus” card (EtherCAT) is under $100. Even a Beckhoff EtherCAT to DeviceNET master solution is cheaper than a straight-up DeviceNET card from another manufacturer.
  • TwinSAFE – a Safety Controller sitting right on your fieldbus for less than a stand-alone safety controller from another vendor.

Support

I had nothing but superb support from the local Beckhoff technical support guys during this experience. They respond to emails personally usually within a couple of hours, and they have no problem taking the two hour drive to see us if we have any issues that can’t be resolved over the phone or with email. We’re not a “big” customer in any way.

Summary

I’ve just spent about 2 weeks coming up to speed on TwinCAT, having come from an AB/Omron background, with a smattering of other PC based controllers like Phoenix Contact’s Visual Logic Controller (VLC) and MultiProg (formerly PCWorx).

The ease-of-use of TwinCAT doesn’t measure up to RSLogix, but it’s comparable to Omron, and it blows products like PCWorx/MultiProg out of the water (I think I was using version 3 of PCWorx, and I know there’s a version 5 now). Most of my issues with TwinCAT are that its ladder editor just isn’t polished enough compared with RSLogix, but that’s not surprising given its roots in Germany. North America tends to favor ladder logic, where Europe seems to favor instruction list, structured text, and function block diagram. It feels like the TwinCAT ladder logic editor was neglected a bit, but it is certainly workable. Comparing the TwinCAT sequential function chart editor with the RSLogix sequential function chart editor, they were actually reasonably close. I liked that I didn’t have to position individual steps in TwinCAT – it just auto-arranged everything for me.

Price-wise, TwinCAT is significantly less expensive than any other serious player in the industrial automation equipment space, particularly when you start ramping up the amount of I/O and drives you need, and it has similar (and in some cases better) features to offer. It’s even significantly less expensive than the Phoenix Contact solution (from my memory), and given that, there’s no reason I’d ever recommend a PCWorx/MultiProg solution from Phoenix Contact. The Phoenix offering is worse and more expensive.

If you’re in a position where your facility is in the market for a new automation platform, I definitely recommend including the Beckhoff TwinCAT platform in your deliberations. It hits a really unique sweet-spot on price, flexibility, and features that nobody else seems to be able to offer right now. Now that we’re a few weeks in, we have a running machine, it looks like the project will be successful and I have no reservations about saying: if you don’t mind learning something new, give them a try.

Format Shift

2 Replies

In case anyone out there is doubting my absolute nerdiness, I currently have a VCR hooked up to my computer through an NTSC to USB converter and I’m converting my old VHS copy of Hackers to an mpeg so I can store it on my Windows Home Server and stream it to my TiVo. Any other pseudo-nerd would just use BitTorrent to download a pirated copy of their favorite movies from the internet, but I’m one of those crazy people who think that copyright serves a purpose. At the same time, I absolutely stand behind my right to format shift the digital content I’ve already purchased to new formats.

“Of all the things I’ve lost, I miss my mind the most.”

That’s my rant on format shifting. Interested in robot shifting? Check out Robot Shift – make the move to industrial robots.

Let’s do it again for the last time

3 Replies

This week I did something I’ve probably done a hundred times before, but this time it felt absolutely absurd. It’ll probably be the last time I ever do it.

What was this crazy event? I purchased shrink-wrapped software. It was an upgrade copy of Visual Studio 2010 (upgrade from 2008 Standard Edition).
Visual Studio 2010 Upgrade

I had to add a cute little piggybank because otherwise it just looked pathetic.

This little box of bits says it was “made in Puerto Rico”. I guess that means the RTM build was FTP’d to some server in Puerto Rico where it was burned onto a dual layer DVD. In fact it’s likely the DVD’s were burned somewhere else and shipped to Puerto Rico, but anyway, this DVD was then stuffed into plastic case along with a 5×7 piece of heavy stock paper with a beat up yellow sticker on it: the license key.

This plastic case was then stuffed into a cardboard sleeve and shrink-wrapped. It probably made its way to a distribution center before being sold to an online distributor CDW, where it sat on a shelf for a few weeks.

It was at this time that I started hunting around for a Visual Studio 2010 license. I had a copy of VS 2008 and I knew there was a discount for upgrades. You can actually download fully licensed copies from the Microsoft store for $399 (CDN), but I found this boxed copy for about $30 cheaper. Of course I still had to pay S&H, but that was about $13 (UPS Ground).

That was on Friday. Today it’s Monday and I received a call from the local UPS distributor. They’d put it on the wrong delivery truck today, so they wouldn’t be able to deliver it until tomorrow. I wasn’t going to be home anyway, so I told them I’d pick it up tonight after work.

I drove a good 25 km out of my way to go pick it up. The truck wasn’t back from its run yet when I got there, so I stood in line another 20 minutes waiting. I’m not knocking UPS here: their whole system boggles my mind. They knew exactly where this package was 100% of the time, even when it ended up being loaded on the wrong truck, and it was in my hand about 20 seconds after the truck pulled up to the building.

Never underestimate the bandwidth of a UPS truck full of dual layer DVDs.

After installing the software, the first thing it asks you to do is check for updates online. That’s because the version that came on the disk was probably out-of-date before it made it out of Puerto Rico. The only thing I really bought was a yellow sticker with my 16 digit license key on it. I could have downloaded a fully working copy last Friday night, and had it in about 2 hours. It would have worked for 60 days, and you could extend it for another 60 for no cost at all. The only thing of value was the legal right to use this software past the evaluation date. The 16 digit license key is only a proof of purchase.

The absurdity of shipping useless plastic and paper all of the continent, driving out of my way and even standing in line to pick it up, just to “prove” that I paid for a legal license to use the software — it’s really striking isn’t it? What’s crazier is how normal this seemed ten or even five years ago!

It’s not like I wouldn’t have purchased the $399 copy from Microsoft if that’s the only one I could find. I went through this because, well, given the chance to save about $15, I’m just cheap. I guess they figure some people won’t bother to pay money for something they can’t hold in their hand, but aren’t we past that now? Apple figured it out. Look at the iPhone App Store, and iTunes.

If I relate this story to my daughter ten years from now, she’ll think I’m nuts. You bought software how? Why?!?

Well, good riddance, shrink-wrapped software. Rest in peace.

Automation and Software in the Next Decade

2 Replies

The “low intellectual property” Fashion Industry as a model for innovation?

I have to agree. Since I’ve started getting involved in open source software, I can tell you that intellectual property laws are the biggest obstacle to innovation that I face. I can say that because I took the time to consult with intellectual property lawyers on everything from trademark to copyright to open source license to software patent. Here’s what I found out: you can’t write a single line of code without violating someone’s patent, somewhere. However, the only time that person will enforce their patent claim is if you’re really successful. The only defense is to pay tens or hundreds of thousands of dollars to build up a patent portfolio of your own so you can partake in a ludicrous arms-race deterrent. On top of that, your patent portfolio offers no defense against so-called “patent trolls”: companies that only own patents and don’t write software of their own. They’re the equivalent of a terrorist cell with nuclear weapons… your nukes are ineffective because you don’t have a city to target.

Copyright is under control, mostly because it’s so much weaker than patents, and it’s free. It stops whole-sale copying of your software, but allows someone to look at your source code, learn from it, and then write it themselves without violating copyright (if they’re careful). Plus, the wide adoption of open source licenses give us lots of material that we can freely copy or integrate.

The irony is that patents were supposed to speed up innovation, but they’ve done exactly the opposite in the software industry.

Do you want to know why none of this matters though? I’m going to make a bold prognostication here:

If the last decade (have we agreed on what to call it yet? The Oh’s?) was about more crap, then the next decade will be about customized crap. Seriously, we gorged ourselves on the mountain of crappage that is sold at Wal-mart and people are starting to wake up with a hangover. We’ve already seen the desires start to change from more to better but I think we’re going to see it change to mine. People are going to want unique stuff. Things that they had a hand in customizing. Something they can use to express themselves.

This has major implications for manufacturing, so it affects the automation industry, and I think it also affects software. Just like the fashion industry, where people use clothes to express themselves, we’re about to enter an age when all the crap you sell to consumers has to have a story, a personality, and be unique. Mass production will remain for your basic staples, and China is going to continue to provide us with that, but North America and Europe are going to have to start producing bespoke crap.

What does this mean for automation? We’re going to see a rise in the build-a-bear style build-your-own-product-on-a-website and have it manufactured and sent to your door the next day. Imagine the logistical changes that need to take place to make that happen. Internet merchandising companies like cafepress already have a leg up in this department. Their merchandise is manufactured after you order it. Now imagine a product with 10,000 or a million different variations, and the automation that has to support it. Imagine the automation required to manufacture products whose specifications change as fast as fashion trends.

On the software side, we’re going to see every major application support custom 3rd party add-ins so everyone can customize their software so it works for them. I think we’re also going to see a rise in “pseudo-programming languages” that let people who aren’t programmers actually customize their applications in ways that we’ve never let them do before. Platforms that give people the power to build solutions to their own needs are going to flourish, and one-size-fits-all solutions like word processors will fade into obscurity.

So I think we’ll have two options: the vast majority of us are going to spend our time building custom things for individuals or businesses. The successful ones will produce products that people can customize themselves.

A Bigger Picture of Automation

2 Replies

When you’re immersed in the world of industrial automation for a long time, you may start to think that PLCs, relays and valves are automation. That’s really not the case. Automation applies to doing anything automatically. Those little timers you plug your lights into when you go on vacation are automation. Automatically billing your customer as soon as product is scanned when it goes on the truck is automation.

To me, we’ve solved a lot of the excruciatingly difficult technical challenges, like motion control, because they can be boiled down to academic pursuits and packaged into a black box. The pain is in the integration. The tyranny of dry contact I/O and 4-20 mA analog signals is because they’re universal standards that nobody’s figured out how to patent yet (or the patents have lapsed). These universal standards make integration of components from various manufacturers possible, and it’s the backbone that supports the industrial automation industry.

The leap from the mechanical and electrical to the digital world should have made integration easier and more flexible. To some extent it has, but we now have these proprietary walls built between our solutions. I once wouldn’t hesitate to connect a valve from company A to a relay from company B, and those companies knew it, and the competition kept prices down and drove innovation forward every year. Now I have to make design decisions about which vendor’s hardware plays better with others before I can buy the first item on my bill of materials. What kind of valve bank do I buy? It depends on whether or not I need a ProfiBus or a DeviceNET communication module on it.

There’s no doubt that the digital alternatives are cheaper than hard wired solutions, and we have made progress on standard bus systems thanks to organizations like the ODVA, but what we really lack is a common open file format for storing our automation programs. As long as we’re stuck with the legacy traditional PLCs, then automation itself is stuck in the mud. But there are open automation systems right around the corner. Systems that aren’t going to be focused on industrial automation, and they’re going to tackle the integration problem in a way that proprietary systems just can’t do for us.

I’m building one of these next-generation systems right now. It’s an automation platform. It’s open. It’s free. It’s not a PLC.