Systems Integration is a tough topic. I've been planning to write this series for a long time.
This series is going to take you from zero to systems integration hero.
In this series you will learn the process I created when I first started systems integration way back in 2008. Since 2008 I've refined this process to align more closely with enterprise architecture models but at the end of the day it's still my same process, just a little bit cleaner :-D
So, what will this series consist of?
Well, over the next 8 weeks I will cover the following topics.
- Level Setting on what Systems Integration is vs System Interfacing
- How to Create a Systems Integration Use Case
- Identifying and Diagramming the Existing Systems
- How to Use a Gap Analysis to Detail out the New Systems
- Detailing out the Physical and Logical Integration Points to include:
- How to Create an Integration Map
- Detailing out the Data Model and System Requirements
- Developing a Sequence of Operations
Each week I will create an individual post for each topic, except topic #1 which is included in this post.
As I create these topics I will continue to add to this series and will re-release this series when it is fully complete.
I hope you all are ready to take a deep dive into the topic of Systems Integration!
Summary of the Lessons
Lesson # 1: Level setting on what Systems Interfacing is vs System Integration
In this lesson I explain the differences between systems interfacing and systems integration. You will learn the differences and you will also learn the importance of a properly defined use case.
Lesson # 2: How to Create a Systems Integration Use Case
In lesson 1 I discussed how important a use case is. In lesson 2 I walk you through how to create a use case.
Lesson # 3: How to Identify and Diagram the Existing Systems
In lesson 2 I discussed how to create a use case. Once this use case is done you will have an idea of what systems you need to integrate. In order to do this you need to understand what systems you have in place. I will talk through this in this lesson.
Lesson # 4: How to Use a Gap Analysis to Detail out the New Systems
In lesson 3 I discussed how to identify existing systems. Once this process is done you may realize you need additional systems. I will talk you through how to perform a gap analysis and how to identify the new systems you may need.
Lesson # 5: Detailing out the physical and logical integration points
Lesson 4 described how to identify all of the systems you need to enact your use case. In this lesson I will detail out how to handle the physical and logical connections that are needed to make your use case work.
Lesson # 6: How to create an Integration map
Lesson 5 went through detailing out the physical and local integration points. In lesson 6 I walk through creating your final integration map.
Lesson # 7: Detailing out the data model and system requirements
Lesson 6 gave us a solid picture of our integrated system. Now it's time to make sure our data types match up and that our system requirements are covered.
Lesson # 8: Developing a Sequence of Operations
Lesson 7 took us prepared us with a solid data model and robust system requirements. In this final step we detail out our sequence of operations and get ready to hand the package over to our operations group to execute.
Lesson # 1: Level setting on what Systems Interfacing is vs System Integration
Systems integration and systems interface two often confused terms and the perfect place to start the conversation around systems integration. To often folks approach a systems interfacing problem with a systems integration solution. This results in confusion, scope difficulties and cost overruns that didn't need to exist in the first place.
Therefore, we are going to start this series by defining each of these terms and then detailing out how they are different.
What is Systems Interfacing?
Systems interfacing is the simple bridging of two separate solutions using a common format. The fact is most of the "systems integration" folks do is actually systems interfacing.
Do you want to bring your lighting systems into your BAS so you can see their status? That is interfacing.
At the end of the day, folks will spend thousands of dollars to get "systems integration" done that is really simple systems interfacing that can be accomplished for quite a low-cost.
How low a cost you ask? Let me give you an example.
Say I have a lighting system with a BACnet/IP interface and a BAS that supports BACnet/IP as well. How much should I budget to integrate these two?
You should be saying it depends on the use case. If you said that, good job it does depend on the use case. And the use case is what determines if you need an integration or an interface.
For the sake of simplicity let's say we simply want to see the lights on the graphic for each room. That is a simple interface, we are simply mapping data from a separate system.
So then how much would that cost? Would you be surprised to know that I can map 1000 light fixtures into a BAS in under an hour? Before you think that I am some integration genius, let me tell you that hundreds of other folks have done the same thing.
How do I know that? I've worked on their systems and asked them how long it took!
A proper protocol will make a system interface super quick and easy.
So the question then becomes, what the heck is systems integration?
What is Systems Integration?
Systems integration is the process of bringing two separate solutions together so that they act as a single solution. Throughout this series I am going to walk you through a scenario most of you face everyday, integrating two separate BAS systems together. In order to keep this vendor agnostic we will simply refer to these systems as system A and system B.
Now here's the deal, when is tying two separate BAS's together considered systems integration?
If we can connect two BAS's via BACnet/IP isn't that simply systems interfacing? Good question!
Remember what I said, it's all about the use case.
Well in this particular scenario we have a customer who really likes system A. Unfortunately due to procurement rules, budget, their manager (you pick!) they have been told they need to go with system B.
How can this customer get system A to work seamlessly with system B?
This is the challenge folks face everyday, with different naming conventions, data structures and all the other nuances systems integration can be a nightmare!
This is the scenario that systems integration handles. When trying to get a BAS to act as a single front-end for multiple other BAS's most folks just map in points and call it a day (systems interfacing).
This leads to finger-pointing, late night job site visits, and high service bills. At the end of the day you want it to just work!
Lesson #2: How to Create a Systems Integration Use Case
Da, da, da....
Well, that was weird, making a trumpet noise announcing the start of the article doesn't work quite as well in text..
How to create a systems integration use case.
We in the BAS world tend to have a bad habit. We like to do things are way and often do not look outside our industry for new methods and approaches. The use case is a perfect example of this.
How many times have you seen use cases formatted according to the CSI MasterFormat. Sure all the parts and pieces are there, but at the end of the day, most of these glimmering pieces of structural perfection are... junk.
They are about as useful as me at an art show, sorry art lovers, I don't get art, never will.
So, why don't we suspend disbelief for a moment and assume that we in the BAS world can do use cases a different way, and by different way I mean the way the rest of the world "does" them.
What are the pieces of a use case?
If you remember our scenario in our first article we are going to tie two BAS together into a single user interface. This use case scenario is surprisingly common in today's BAS world.
Our first step to creating this use case is to identify a few key things.
- What systems or people are involved?
- What does success look like?
- What is the goal of the use case?
What systems are involved?
If you remember earlier in the article I mentioned some unfamiliar terms. One of those terms was the term actor. I want you to think of a use case a play. The use case is going to tell the actors what do at certain points in the process.
So at this point you may be thinking an actor is a person, you're partially correct. An actor, according to most use case documentation, defines a role played by a user or system that interacts with the subject.
Logically that brings us to our next question. What is a subject?
A subject is the system or systems that should perform a task based on the use case scenario.
Ok, so time for our first question. Who are our actors and who are our subjects? I want you to take a second and think this through, a lot of people mess up here and its important to identify the correct actors and subjects up front.
So, pause for a second look a way from the screen and think through who your actors and subjects are. Remember we are thinking through the use case of two BAS tying into a single user interface.
Ok you done?
Great, so your list probably looks something like this.
Actor(s): BAS 1, BAS 2, Technician, Supervisor, Common User
System(s): New BAS Graphical User Interface (GUI)
At this point we have our actors and systems identified now we need to determine how to put these pieces together.
How do you put these pieces together?
This part is often called your narrative. It is the narrative that will form your use case and will tell how the different actors communicate with the system.
We know that we want a user to be able to use the new GUI to be able to access both BAS. Do you know what we call that? We call that our main success scenario (MSS). Our MSS is our success criteria.
A use case should have one MSS.
If you write a use case and you have more than one MSS you are most likely combining use cases. It is critically important that you separate your use cases to focus on one MSS.
You see, systems integration is complicated enough, the idea is that your use cases will be built for a single purpose. That way when you create the integrations you can segment your use cases. This will allow you to troubleshoot, code, and support your use cases individually.
So then, how many use cases do we have with the single GUI scenario. Off the top of my head I can think of several.
- User Logs into the single GUI
- User logs out of the single GUI
- User modifies data in the single GUI
- User runs reports in the single GUI
Now you may be thinking, oh come on Phil, it's a simple integrated BAS GUI. I'd caution you to pause anytime you start to think this way. You need to think through each scenario.
For example, if the new GUI needs a login, which login does it use?
Does it use BAS 1's credentials or BAS 2?
Does the new GUI log user actions?
If so where?
How does it tie to IT systems?
And those are just the set of questions for the first scenario.
So, lets build out a use case.
Building the Use Case
Building out a use case isn't very hard. In this scenario we are going to build out two use cases:
- User logs into the single GUI
- User modifies data in the single GUI
User logs into the single GUI UC#1
Actors: BAS Technician, BAS 1 and BAS 2
Systems: Single GUI, BAS User Databases
Main Success Scenario: User Logs Into Single GUI
Description: The BAS Technician will be prompted for a login at the single GUI. The user can use his/her login credentials from BAS 1 or BAS 2. Upon the GUI authenticating with BAS 1 or BAS 2 the user will be granted access to the application.
User modifies data in the single GUI UC#2
Actors: BAS Technician, BAS 1 and BAS 2
Systems: Single GUI
Main Success Scenario: User modifies data in the single GUI
Description: Once the BAS technician is logged into the single GUI (see use case 1 user logs into the single gui), the user will be able to access BAS point data and modify that data. The single GUI will confirm and log any data modifications.
Ok, this use case has something glaringly wrong with it. Can you tell me what that is? I'll give you a hint, it occurs in the last sentence.
Give up? Well, here is the problem. Use case #2 is actually two use cases. The modification use case is good, but at the end of the use case I added confirm and log any data modifications.
Now at first glance this may not seem like a big deal. It actually may seem like I am splitting hairs and being anal retentive. However, do you know how the single GUI will log data modifications?
Will it log them in BAS 1's database?
How will the confirmation happen?
Do you see how this could lead to finger-pointing at a job site?
The BAS technician for BAS #1 says "It wasn't my job to log any data modifications, your use case clearly says the new GUI will log data modifications."
In the BAS techs defense the use case does say that, but is that what you want?
I will leave you to ponder that question.
Finalizing the use case
With our use cases mapped out we should now be ready to move onto our next step.
We aren't ready yet! Do you know why? Because we haven't gotten our stakeholders buy-in.
Stakeholder, there's a hundred-dollar word for you! A stakeholder is a person who can be impacted by the use case. To often I've seen, cause I would never do this right :-D, people go and create use cases that are absolutely beautiful.
And these use cases float their beautiful selves right to the garbage can! This happens because the use case creator did not take the time to meet with stakeholders in order to get "buy-in".
In order to avoid a ton of headache after the fact I encourage you to meet with your stakeholders in order to ensure that your use cases have their "buy-in".
Lesson #3: How to Identify and Diagram Existing Systems
In the world of enterprise architecture system mapping is a common event and in the enterprise IT world system integration is a daily event. I argue, that in the BAS world, systems integration is going to become an increasingly important task.
Yet, there is a problem. No formal process exists for systems integration in the BAS world!
But, why should we reinvent this process when a perfectly good process exists in the IT world.
In, the IT space this process is called mapping the AS-IS architecture. AS-IS architecture means the current architecture as it currently is in the business. There are four main domains, or focus areas, that an IT mapping session will focus on.
Those are the Business, Data, Application, and Technology domains. These domains are often abbreviated to the acronym BDAT. I believe, based on my experience, that any systems integration effort should focus on all 4 domains.
But, what does that look like in the BAS world?
BDAT in BAS
Believe it or not, there are business, data, application and technology systems in the BAS space. What might this look like in our simple 2 BAS into a GUI scenario. Let's move through each of the systems.
Believe it or not BAS can have business value. For example, does a certain business group rely on reports from the BAS?
There's nothing quite as fun as going through an integration effort only to find out that you missed a key report that the facilities or energy management groups require.
- In this category you want to find out:
- What business groups use the systems?
- What do they use the systems for?
- Is there a unique capability that needs to remain or be replicated?
It's all about that data!
Seriously, the whole reason we are pursuing a single GUI is to make data easier to access and manipulate! So then, what do we need to know in this section?
We need to understand:
- How is data stored?
- What format is the data in?
- How is the data collected?
- How is the data accessed?
At the end of the day the two BAS are applications. However, each one of the BAS have their own applications as well. Some BAS allow you to configure their databases and programming directly from the BAS and some require external software.
In this category we want to identify:
- What are Applications being used?
- What Dependencies do these applications have?
- How these applications can share data (API's and protocols)?
Technology makes the world go round! Well, it at least lets the BAS run!
So when we say technology what are we talking about?
We are talking about hardware and connections.
In this category we identify the following information:
- What are the hardware requirements?
- What does the physical and logical topology (device layout) look like?
- How are the systems connected to the network?
- Are the systems scalable?
Putting it All Together
At the end of this exercise we will have clearly defined the BDAT systems that our two BAS are dependent on.
By answering these questions we will have an accurate picture of our AS-IS environment.
With this information we are able to understand what our constraints are within our current environment. Constraints are hard limitations that we must adhere to because of our technologies.
For example, since both of our BAS use SQL databases we are constrained to using a SQL database or using some sort of database translator. Knowing these constraints will become invaluable as we move through the next several steps of the systems integration process.
Lesson #4: How to Use a Gap Analysis to Detail out the New Systems
What is a gap analysis
A gap analysis is a key part of the integration process but what is it?
In a gap analysis you take the functional specifications you created in your use case and you compare your existing system(s) to the functional specifications. In laments terms you look at the capabilities required for the use case and the capabilities you currently have.
The missing capabilities become the gap.
Why is it critical to perform a gap analysis
Think about it, imagine you go and decide to buy some really cool video game… Ok, I am admitting to one of my really bad hobbies here.
But, you go and buy this game and you don’t check if your computer can run it. You come home and all you have is a $60 dollar brick because the game won’t run on your computer.
Now, for most $60 dollars isn’t the end of the world. Imagine though, if you had went and purchased a $300,000 analytics package.
You didn’t go and check if your BAS could provide the required information for the analytics software to run. Now, here you are with this $300,000 software package and you don’t have the points in your system that the analytics software needs! So, now you have two choices, cancel the contract and absorb any termination fees OR go and spend the time, money, and effort to add all the necessary points to your BAS.
Chose your poison..
How do you perform a gap analysis
Ok Phil, you’ve sold me on the need to perform a gap analysis but how do you do one?
Quite simply you take the requirements of your use case and you put them next to your current capabilities.
Let’s see this in action, by continuing our scenario, laid out at the beginning of this series, where we are integrating two BAS into a single graphical user interface (GUI).
Step 1: Identify your use case requirements
In the beginning of this series we created a use case for our single GUI integration. While we laid out several use cases, we did not identify the technical requirements of our single GUI. Let’s do that now.
Our ultimate goal is to have a single GUI that the two existing BAS can tie into to provide full access and control of real-time systems and historical data. This GUI shall use existing logins and not require new logins.
Let’s break this into bite size chunks.
We need a way to capture and control the existing data points from the two BAS. The new GUI supports BACnet/IP
We need a single database or a way to access the existing two databases and pull them into the new GUI. The new GUI supports SQL
We need to allow the users to continue to use their current logins and passwords. If the user changes their login or the user is removed then the GUI needs to reflect this. The new GUI stores its user name and passwords in a SQL database but does not support external user names and passwords.
Step 2: Identify your current capabilities
Currently our two BAS systems support BACnet/IP, have SQL databases, and they have SQL databases for user accounts.
The two BAS’s however, do not have a common data model.
Step 3: Map out the present and missing requirements
Because of these requirements we know that we have:
- Database connectivity (SQL)
- Protocol connectivity (BACnet/IP)
- User Interfaces in SQL in the two BAS and new GUI
The big gap here is, how do we get and manage the new user logins into the new GUI?
Step 4: Identify a solution to close the gap
Ok, so we have a gap. We need to find a way to transfer the user name and logins to the new GUI.
Problem is, we need to maintain the state of the logins.
What does maintain the state mean?
In any database you have what is called the state, the state is the current condition, or snapshot if you will of the database. In order to achieve our scope of using the user accounts and passwords that exist in the BAS we need to make sure that the user database for the GUI reflects, is consistent with the accounts that exist in the new BAS.
Minor problem, what if two users accounts with the same name exist?
This is quite easy to solve, we can simply import both user names and using some SQL magic we can make them unique. When someone tries to login the GUI will check both passwords to see if that is correct. Naturally this assumes that Bob on system A is the same Bob on system B.
Because of this, it’s often a good practice to pull all user names look for identical names and request the user change their name. This is done in the beginning of the integration process.
So far we have identified 2 issues:
- Maintaining the database state
- Dealing with identical user names
Now we need to document our resolution for these two issues. Let’s say that in order to maintain the database state it will take 8 hours a week of a Database Administrator at $60/hr.
The facility manager doesn’t want to carry this cost.
What can he/she do?
This is where alternate solutions come in play. The facility manager will ask the integrator or rather the integrator will present alternate options to the facility manager.
The facility manager can now go and chose between accepting the cost of managing database state OR going and looking at a second option, like making folks create new user names and passwords on the single GUI…
Lesson #5: Detailing out the physical and logical integration points
Don’t misunderstand. Without a use case and gap analysis you can get your butt into a whole heaping mess of trouble.
Oh, those were supposed to be BACnet/IP cards on the rooftop…
I’ve lived in that world, I’ve got the battle scars.
So, what is our next step? How can you go from concept to working systems integration?
In order to do this you need to detail out your physical and logical integration points.
But what are physical and logical integration points? And what exactly does detailing them out mean?
Well, my friend, I’m going to answer those exact questions below.
What are Physical Integration Points
When folks think of systems integration, their minds usually don’t go straight to the physical integration points and that’s a shame! I would argue that without physical integration, nothing happens.
You can write the best code in the world. You can take two completely disparate data models and make them one. But none of that “stuff” matters if the two systems can’t talk to one another.
And that is why I start with physical integration points.
What are physical integration points you ask?
Physical integration points are the physical connections between the systems you identified earlier in your gap analysis and use case. These physical connections can be direct cabling, networking, wireless connections, the list goes on and on.
The important point is that physical connections, physically connect the systems…
What are Logical Integration Points
Logical integration points on the other hand, don’t physically exist. You can’t go to Home Depot and pick up 20ft of logical wiring, well if you can let me know and we can make a killing on Amazon…
Logical integration points are the connections between the systems that utilize the physical connections.
For example, you may physically connect a lighting system to a building automation system. But, without a logical connection to tell the two systems how to communicate and to resolve data and all sorts of other “programming” problems, your integration may be a dud.
How do you “detail” them out…
So then, how do you make all this integration “stuff” work and what the heck is detailing something out…
Well, here’s the deal, every systems integration job is unique, can we all agree on that?
Ok, I’ll take that virtual head nod as agreement 😀
So, if every job is unique it makes sense that we would need to plan out each project and “detail” out the plans for each step.
Well, in this section I am going to teach you my 3 step process for “detailing” out physical and logical integrations. You might want to go grab a pen and pencil for this, don’t worry I’ll wait for you…
Ok, you ready? Here are the steps:
Step 1: Identify Your Physical Connections
Ok, well that was obvious…
But was it?
Using our use case from the previous posts, we can map out our connections.
If you don’t remember our original use case, or you haven’t read my other posts (naughty, naughty…) then here it is.
We want to integrate two building automation systems into a single graphical user interface (GUI) on a new system.
Which systems connect to what?
Do you connect the two building automation systems directly to the new GUI?
You need to look at your use case and follow how the systems are supposed to interact. Then, you need to draw your physical connections on a piece of paper. Specifically you should cover:
- Which systems are there?
- What system connects to another system?
- What physical connection is used?
Step 2: Identify your communication flow
Before we get to the logical connections and interfaces we need to identify how the data flows.
Using the use case from step 1 maybe we are routing some information from the first building automation system into the second building automation system.
Why would we do that?
Maybe the first building automation system controls the air handlers which then send data to the second building automation system that controls the central plant. The important part of this step is that we show that connection.
Could you imagine the chaos if we broke that connection and just simply directly connected the building automation systems to the single GUI?
You will want to know:
- Which way should data flow?
- Is a process in a system dependent on another system?
Step 3: Logically map your logical connections
Well shucks, I could only fit the word logical into that heading twice… Oh, well.
It’s important that we figure out how our data is going to be connected to the other system(s). In the case of our single GUI, one of our design constraints is to have a new single database.
Well, what if the data is completely different in both BAS?
We might have to go and create an intermediary database to normalize that data.
You know what that means right?
We’ve got to map out how and what data flows. In order to that we need to identify what data we are looking to communicate and diagram out where that data goes to.
Questions you want to be asking yourself here are:
What priority is the data at?
Are there any Quality-of-Service (QoS) issues?
Where does the data come from and where does it go to?
Lesson #6: How to create an Integration map
What is integration map?
So what exactly is an integration map?
In lesson number five I discussed the concept of physical and logical integration connections, man is that ever a mouthful.
I described that in any systems integration project you will have either physical, logical or both connections.
I also discussed how you need to go and map out the connections between your integrations. However, I did not show you the exact process for doing this.
That is what I’m covering here.
An integration map, is a method, for mapping out the connections, dataflow, and process flow. Usually this is done by use case.
What do I mean this is done by use case?
In lesson two I discussed, how you can create a use case for your integrations. I also showed you how to list out the systems for this use case.
Since each use case is specific systems you’re going to want to create an integration map for each use case. In any systems integration project you want each integration to be as standalone as possible.
This will help you and troubleshooting as well as supporting and upgrading the use case because it will eliminate any interdependencies between use case
Now if you send yourself what is on interdependency?
An interdependency is where one use case depends on logical and/or physical systems from another use case.
Now it’s not possible to eliminate all interdependencies because some use cases require a previous use case before they can take place.
For example, if you were to go and say that you wanted to have use case on creating a schedule. That scheduling use case may be dependent on a login logout use case.
Ok, so back to the question “what is an integration map?”.
As I mentioned an integration map is a method or process for mapping out your connections, data flow, and use case flow. I also mentioned that an integration map usually created for each individual use case.
What is an integration map used for?
Okay, now that you understand what integration map is.
What exactly is it used for?
I mean, saying it’s a process for mapping out your connections, data flow, and use case flow, isn’t exactly the clearest way of defining what an integration map is used.
When you go to execute an integration project, you’re going to have to perform some specific tasks. These tasks commonly are:
- Connecting the systems
- Normalizing the data
- Prioritizing the data flows
- Creating any sequence or trigger logic (basically if this happens then that happens)
Now if you only had two systems to integrate these tasks would most likely be quite simple.
However, with the data center example I provided, I had to perform the following tasks:
- Data normalization on each individual system
- Quality of service (basically making sure the messages get prioritized over other network traffic)
- Physically connecting the systems on the network
- Creating an integration gateway to make all the systems communicate via OPC
- Triggering use case is based on specific points, alarms and commands
Could you imagine trying to do this on the fly?
I had over a dozen different use cases, and several dozen data centers, could you imagine the chaos of trying to map out all of those systems in my head?
I’m hope that you see, why I would want to create an integration map.
So quite simply, an integration map is a way of structuring your physical connections, logical connections, and data flow in a way that allows you to execute and document your projects step-by-step.
When and when not to use an integration map?
At this point you may be wondering when you should be using an integration map?
I’m not going to tell you that you should use an integration map on every project because it’s simply not true.
If you are connecting two systems, then as long as you identify the physical and logical connections and any data normalization that has to take place you should be fine.
So the general rule of thumb, is whenever you have to connect more than two systems or your project involves multiple use cases, with interdependencies, you should look at creating an integration map.
How do you create an integration map?
Okay, we’ve come to the point of the post that most of you been waiting for!
So you been following along, and you now agree that you should create an integration map, but how? If you go and scour the Internet, you can find resources on systems integration but it’s focused on IT systems.
How do you create an integration map for building automation systems?
Over my I’ve come up a step-by-step process for mapping out your integrations by use case. There are 6 steps creating an effective integration map. The steps are:
- Step 1: Identify the use case that mapping out the integrations for (I described this in lesson 2)
- Step 2: Identify your physical connections, logical connections, systems, and data flow (by the way I went through how to do this in lessons 3, 4 and 5)
- Step 3: Create an integration drawing to list out your systems
- Step 4: Draw the physical and logic connections between the systems and number those connections
- Step 5: Draw the direction, priority, and type of data that flows across the physical and logical connections, number your data flows.
- Step 6: Take your numbered connections and data flows and list them in a spreadsheet
Step 1: Identify the use case that you are mapping out the integrations for (I described this in lesson 2)
As I mentioned in lesson 2, during the use case creation process you identify the systems that will be used. It is important now that you identify the use case that you will be mapping out integrations for.
Otherwise you may not identify the proper systems and connections that are required.
Step 2: Identify your physical connections, logical connections, and data flow between your systems (by the way I went through how to do this in lessons 3, 4 and 5)
Speaking of systems and connections, it is important now that you identify the physical connections, logical connections and data flow between your systems.
You will use this information to create your integration.
You will need to identify:
- The physical connection type
- What systems the physical connection is between
- The logical connection type
- What systems the logical connection is between
- The protocol being used for the logical connection
- The direction of the data flow
- The priority of the data flow
- The size of the data flow
Once you have this information you’ll be able to perform the next step
Step 3: Create an integration map to list out your systems
Now, I’m not gonna walk you step-by-step how to create a system drawing. However, for those of you that are interested I will be creating a systems integration course in the future that will go through this process.
That being said, the important thing is to pick the format and software that you’re most comfortable with using.
I personally use Microsoft Visio and would recommend that to others.
Once you’ve picked the software that you’re going to create your system drawing in, you will need to place each individual system on the integration map.
Step 4: Draw the physical and logic connections between the systems and number those connections
Next, using the information you collected in step two, you will want to add the physical and logical connections to your system integration map.
I personally recommend that, physical connections be solid lines and logical connections be dashed lines ( - - -).
In most cases I recommend using a color code for the connections to show what protocol, physical medium (Ethernet, RS-485, RS-232, etc.) is being used.
You would typically list this in the upper right-hand corner as a legend.
Step 5: Draw the direction, priority, and type of data that flows across the physical and logical connections, number your data flows
Now that you have the physical and logical connections added to the integration map, I recommend that you go and add the direction in type of your data flows.
For data flows I recommend using dots ( ● ● ●), instead of dashes or solid lines.
You will want to list out the direction that your data flows in, the priority level of the data (four commands and quality of service), the size of the data being sent, and the type of data that is being sent.
You will also want to number your data flows.
I will be talking more about data flow, data models, and system requirements in lesson 7.
Step 6: Take your numbered connections and data flows and list them in a spreadsheet
Now that you have your connections in data flows numbered, you will want to list them out in a spreadsheet. In this spreadsheet you will want to show, what physical connections, logical connections, and data flows connect between what systems.
You will also want to list out the following information.
For your physical connections you will want to list out:
- What systems are connected by the physical connection
- The media type of the physical connection
With the logical connections you will want to list out:
- What systems are connected by the logical connection
- The protocol type of the logical connection
- Any software and/or services that the logical connections originate from or and at (often times these are called endpoints)
Finally, with your data flows will want to list out:
- The system(s) that the data flows from
- The system(s) that the data flows to
- The priority level at which the data flows
- The size of the data flow (typically defined in bytes per second)
with all of this information defined you’re now ready to move on to laying out your data model and system requirements. Some of you may be asking why I waited to have the data model after the integration.
I have found that if you don’t map out should your data flows and logical connections prior to creating the data model you can often miss or add data that you do not need.
Lesson #7: Detailing out the data model and system requirements
A brief segue on data
Data is everywhere. Some have said, “Data is the new oil”. Having the right “data” can provide you insight into how your building operates. With this information, you are able to make operational decisions that will impact the efficiency and effectiveness of your building automation system.
What is a data model and how do you create one?
A data model organizes data and details out how that data is formatted. When you are exchanging data between systems it is important to understand the format that the data is in. This is represented visually below:
As you can see I created a simple data model for a VAV box. This data model describes the primary key (which is how we will find the data) and two properties, zone temperature (ZN-T) and damper output (DPR-O).
You will notice that the primary key, controller name, is defined as the data type string and the two other properties are defined as the data type float.
In the programming world, a string, is a string of text and a float, is typically a number, with decimal precision.
Now, if I had, another system, that I was going to be integrating the VAV box data into and that system had a different data type then there would be a conflict. This is illustrated below:
As you can see, the VAV box data is not of the same data type as the third-party BAS data. This will cause a data conflict.
What is happening is the third-party BAS, has a primary key of device ID, which is of type int. An INT, stands for an integer. Unlike a float, an integer, does not have decimal precision. This means that the zone temperature and damper positions would lose their “precision” or accuracy.
But the bigger issue, is that the primary keys conflict both in type and in name. This will cause a mismatch. In order to resolve this the data needs to be normalized. You can see how data is normalized in the image below:
What I just did in the above image, is I created a higher level data set that bridges the two data sets utilizing what is called foreign keys. This allows the third-party BAS and VAV box data sets to access the data from the normalized data set.
When you purchase a device like a field server gateway, this is what is happening. Inside that gateway there is software that is performing this data normalization and then allowing the two data sets to connect.
When you look at an individual system you will want to go and create the data sets for each system. Now you may not know the data types that each system has. This is where you will have to turn to the system vendor to provide you that information.
Now I take a quick second the make sure I’m clear. If you’re performing a protocol integration using similar protocols, you most likely will not have to go this deep. The examples I am providing are geared more towards dissimilar protocols, for example integrating a LON system into a BACnet/IP system.
You can ask the vendor to provide you the data model for their specific system. Then you need to understand the points that you want to share between the systems. Once you have defined the points you will need to map up the data types.
Once you have done this you will need to determine if a normalization schema is needed. For some of you the word schema may be unfamiliar. When you hear the word schema, think format.
Schema is another way of saying the format of the data.
What are system requirements and how do you define them?
System requirements list out the physical and logical requirements of the systems that are part of the integration project. You will typically want to understand:
- Network Throughput
In an earlier post we discussed how to identify your systems. Now that you know what your systems are its time to formally document the system requirements. This step is not very complex but it is time-consuming.
If you do not have an IT person on staff than is important that you retain the services of someone with an information technology background.
Here is a five step process for defining your system requirements:
Step 1: Define the network, storage, and compute requirements for each system
Using the list of systems that you created in (earlier post), you will need to go and work with your IT staff and vendors to determine the network (also known as bandwidth), storage, and compute (also known as processing) requirements of each individual system.
Step 2: Identify where the systems will be hosted
Based on the information from step one, you will want to work with your IT representative to determine where the systems will be hosted and what part of the network they will be connected to.
Step 3: Create an overall architecture diagram
Now that you have listed out where the systems will be hosted and the requirements you will need to create an overall architecture diagram. This architecture diagram will visually represent where the systems reside on the network and how they are connected to one another.
Step 4: List out any system, software, requirements
With all of this information defined, your final step is to work with your IT representative and your vendor to determine any software that is required to be installed on the systems in order for them to run.
Lesson #8: Developing a sequence of operations
Developing a sequence of operations will help you or your integrator to effectively execute a systems integration project. Without this document it is very likely that your project won't work. I know, I know.
It seems like I've said that about every post in this series. Well folks, I only say that, because it is true!
Look, I'm walking you through the mine field but it's up to you if you want to stay by my side or run ahead and play with your projects life.
What a sequence of operations is
So what is a sequence of operations. Chances are, if you are working on a systems integration project you are familiar with a sequence of operations. But, I'm going to assume for a second that you aren't familiar with a sequence of operations.
A sequence of operations is like a use case on steroids. Whereas a use case walks you through the how the sequence of operations is the how, the what, the when and even more.
For example, I'm going to describe a use case where the main success scenario is the user being able to use credentials from another BAS login into a system, hint, hint this may look familiar to a use case I described in an earlier post.
Use Case Text
The user will be able to use login credentials from an existing BAS to log into the new graphical user interface.
Sequence of Operations Text
A user will enter their credentials from the existing BAS to log into the new graphical user interface. Upon entering their credentials the graphical user interface will perform an MD5 hash algorithm against the users password. The graphical user interface will then communicate to the back-end database and will perform a select query where the username and hashed password match. If a matching data set can be located then the user will be granted access to the graphical user interface.
Do you see the difference? The sequence of operations is the end result of information from the physical and logical integration map, the data model and the use case.
How to create a sequence of operations
Creating a sequence of operations is actually quite easy once you get the hang of it. I used to suck at writing sequence of operations and then one day it all just clicked for me.
This happened because I came up with a step-by-step process that empowered me to take what was in my head and put it on paper.
In this section, I am going to teach you that exact 7 step process.
I am going to walk you through the steps using the example of tying in a lighting system.
Step 1: Gather the main actions from the use case
For this step what you want to do, is to gather the main actions from the use case that you created. You will then take these main actions and expand upon them in order to create a sequence of operations.
A use case may say, the lighting system will be integrated to the building automation system. The building automation system will turn the lights on when the building is occupied.
The main actions for this use case are:
- The lighting system turning on
- The building automation system going occupied
- The building automation system commanding the lighting system
Step 2: Identify the systems from your integration map
Next you want to go and take the systems that you identified as part of your integration map. These systems will then expand upon the sequence of operations you started in step one.
From the integration map I will be able to identify that the building automation system and the lighting system are connected via BACnet/IP. I will also know that the building automation system will be commanding the lighting system have priority level 10.
I will also be able to identify, that the building automation system syncs its timeclock with the building's local time-server.
Step 3: Clarify any settings and/or timings
Here I will decide that I want the lighting system to turn on within 30 seconds of the building automation system switching to occupied mode.
Step 4: Chunk out your actions and build out a flow map
I will then take each one of these actions and build a flow map. This flow map will visually represent how and in what order the different actions will take place.
Step 5: Capture the wording for the actions
Once I'm sure that the actions I want the integration to execute are in the right order. I will break them out and begin to add details around the wording.
At first this will look like:
- The buildings time-server sends a time synchronization message to the building automation system
- Building automation system is occupied
- Lighting system is commanded to occupied
- Lights are commanded on by the lighting system
Step 6: Add your settings and timings
Now I will take those four bullets that I created in step five and I will add details around each bullet.
- The buildings time-server will send a time synchronization message to the building automation system every 15 minutes (adj). The time synchronization message will be sent using the NTP protocol to the building automation server located in the data closet.
- The building automation system will have a set schedule that will command the building to occupied. This schedule will be from 7AM CST to 5pm CST (adj).
- The building automation system will command the lighting enable object to on (adj) within 30 seconds (adj) of the building automation system occupancy signal switching to the occupied state. This command will be sent via the BACnet/IP protocol from the BAS server to the lighting server at priority level 10.
- Upon receiving the on command from the BAS, the lighting system will command all of the buildings interior lights to on.
Step 7: Consolidate into a sequence
Now for the easy part. All I need to do is simply combine the bullets together and I will have my sequence of operations.
The buildings time-server will send a time synchronization message to the building automation system every 15 minutes (adj). The time synchronization message will be sent using the NTP protocol to the building automation server located in the data closet.The building automation system will have a set schedule that will command the building to occupied. This schedule will be from 7AM CST to 5pm CST (adj). The building automation system will command the lighting enable object to on (adj) within 30 seconds (adj) of the building automation system occupancy signal switching to the occupied state. This command will be sent via the BACnet/IP protocol from the BAS server to the lighting server at priority level 10. Upon receiving the on command from the BAS, the lighting system will command all of the buildings interior lights to on.
How to use a sequence of operations to execute your systems integration project
Once you have the sequence written what do you do? How do you take your sequence and execute a project.
Well, everyone has a different way of executing their systems integration projects. If you have a way that you like and you have followed the steps up to this point then I encourage you to use your method.
However, if you want a proven process that has helped me to execute some crazy complex projects, then check out my 5 step process below:
Step 1: Build out the physical architecture
In this first step you will want to install the systems that you identified as part of the to be architecture.
Once you have the systems installed you will then want to make sure that any software associated with the systems set up including, drivers, databases, and dependencies.
Using our lighting example, you would want to install the lighting and building automation servers, controllers, and end devices.
Step 2: Build out the logical architecture
Next you will want to go and set up logical architecture for the integration project. As part of this process you will want to configure any network, database, API, and other logical pieces of the system.
Continuing our lighting example, you would want to set up any IP addresses, device instances, VLANs, databases, and system names.
Step 3: Add any programming
Next you will want to go and develop any programming that is required in order to execute the systems integration sequence. This would include data adapters, interlocks, dependency logic, and any control logic.
For our lighting solution, you would want to build out, the interlock logic along with the prioritization scheme for the integration points.
Step 4: Insert settings and timers
Once all of the code is written you will want to put in the initial settings and timers for the sequence. This would include any schedules, set points, on or off delays, etc.
You will want to set the timers and schedules for the lighting integration.
Step 5: Test the integration
Finally, you will want to test the integration to verify that it works.
In order to test the lighting integration you would command the building automation system from occupied to unoccupied and verify that the lights changed state.
You would also want to change the master time-server and verify that the building automation system adjusts accordingly.
Finally, you would want to make sure that the lighting system cannot override the building automation system.
So there you have it folks. That is my step-by-step process for approaching systems integration projects.
If you started this series with limited information ... if after reading this information you realized that there is much more to building automation systems and you'd like a simple resource to learn more then I have a solution for you.
Click on the image below to find out more about our solution