A Network Operations Center, or NOC for short, is the heart of your system. It forms the core of any medium or large-scale network monitoring effort. In it, your staff will monitor for network problems and react to them.
Your NOC forms the vital link between the detection of a network issue and the implementation of a solution (normally it means that a technician will be dispatched to a remote site).
We've helped hundreds of clients to design and deploy custom monitoring systems, so we know that if you don't have the right equipment, you can't even begin to do your job well. That's why you need the great monitoring gear to provide you with detailed alarm notifications.
But how do you set up a quality command center that avoids common (and expensive) mistakes? If you already have a NOC center in place, what can you do improve its performance?
Let's take a look at some NOC centers best practices.
For maximum visibility of your network, your NOC personnel must have access to all of the following capabilities.
Power failure monitoring
Root cause analysis
Problem alerts for your key personnel
Escalating notifications that are forwarded to supervisors after user-defined time limits
The only way to obtain these capabilities in your NOC center is to deploy an advanced alarm monitoring system. That system will provide you with important alarm alerts, while also tracking alarm histories to help you tackle the source of frequently occurring problems.
While many NOC centers are open 7x24x365, this isn't always the case. Some companies are in a transitional phase of growth. Their network is large enough to have the investment on a NOC center construction approved, but they can't yet justify the expense of staffing it outside of regular or possibly extended business hours. In cases like this, an alarm monitoring system can provide after-hours alarm notifications (to email or phones) to alert on-call technicians of alarms in the network.
Without alarm monitoring in your NOC, you won't be able to know about problems in your network until it's too late. By that time, your customers or end-users might have already lost service.
A good alarm monitoring system provides staff in your NOC with all of the important info they need about problems throughout your network. NOC techs will be able to look at this alarm into in a variety of ways, enabling them to dispatch the right staff to your various remote sites.
To deploy the most efficient alarm monitoring system for your NOC, you need advanced remote telemetry units (RTUs), as well as a modern and effective alarm master. The most advanced gear will provide you with the best alarm collection and reporting.
The RTU's job is to handle the alarm collection portion of your network monitoring. These remotes should monitor a number of devices at a remote site. After collecting an alarm, the RTUs will then report the data to the alarm master in your NOC - providing your personnel with important information about the problem. This alarm should be displayed within your NOC, as well as in emails or text messages to your appropriate network technicians.
The core of any NOC is an alarm master. It accepts inputs from the handful, or hundreds, or even thousands of remote devices in your network. In another words, it aggregates alarms from RTUs and gear across the network, notifying NOC staff and field technicians of all problems that arise. Your NOC operators can specify how the alarm master displays this information, customizing your system to make the NOC as efficient as possible.
When designing your NOC data center, it's important to follow some monitoring best practices. Keeping the following best practices in mind, you'll make sure you're investing your budget in an effective system.
In a lot of networks, remote monitoring is covered by several different, incompatible systems each one sending alarms to a separate monitoring alarm master.
This can be a big issue. Multiple systems increase the chances that network alarms will be unnoticed, forgotten, or incorrectly prioritized. You can't be sure every problem in your network will be caught, and you can't trust that all those screens full of green lights really mean that your network is problem-free.
An important best practice to follow is the integration strategy. This is the secure way to monitor your network from a single platform that incorporates alarms from all your equipment, with redundancy capabilities and alternate reporting paths to prevent single points of failure. Incidentally, integration is also a cheaper way to monitor all your remote sites, since it saves equipment and training costs.
Your NOC won't be able to secure your network reliability, with an inadequate, aging monitoring system.
You need to upgrade, but perhaps you're reluctant to change your system. Not because you're blind to its faults, but because you think upgrades are more trouble than they're worth.
After all, your existing system is deeply embedded in your network. It's going to cost a lot of money and lot of man-hours to replace it, and replacement is going to disrupt your network for months.
The major pitfall of upgrading is making the upgrade harder than it should be. It's not an either/or proposition. An upgrade doesn't have to mean tearing your network out by the roots. You can immediately achieve a better level of visibility and reliability by just replacing your master, the central alarm collection, and processing point, and leaving your existing RTUs where they are.
The master station is the weakest point of an older system. If it dies, the entire system is gone. That's the component that most urgently needs to be replaced.
However, the master is also potentially the strongest point of your reliability management system. That's where intelligent alarm processing can be applied to your network alarms, extracting the most useful, actionable information from what your remotes are telling you. This is the information your staff needs to know in order to determine the root cause of network faults quickly, and to effectively take corrective actions, keeping your network up and running.
A new master should be able to immediately take your existing monitoring and your existing RTUs to a new level of effectiveness.
And, to achieve both different systems integration and legacy compatibility, a best practice is to invest in a multiprotocol alarm master. This way no matter which protocol or legacy device you have in your network, you'll still have visibility over them in one unified screen at your NOC center.
The function of a network alarm management system is to inform your NOC of events that threaten your network, so that your staff can take corrective actions. Ideally, your system would stay quiet until an event that requires human intervention occurs.
However, what happens when your system is constantly bombarding your staff with status events or non-alarms that require no action other than an acknowledgment? Your staff might start ignoring alarm reports, because they might believe that all alarms are nonessential alarms. The staff might miss and not respond to critical alarms, defeating the function of the network management system.
These self-defeating, nonproductive alarms are called nuisance alarms, and they can have destructive effects. So, you also need to make sure that whatever alarm master you deploy in your NOC can filter nuisance alarms.
When you deploy an advanced alarm master, that can hide unimportant messages from staff, you can reduce or even eliminate nuisance alarms.
How to determine which alarms you need to know about?
The first thing to determine is whether the alarm is needed at all. If the alarm is simply a status indication and no action is ever required, you may want to turn the point off by setting it to "No Log". You can still have the alarm report go to your history file for future data analysis.
You can also have an alarm that when occurs by itself is not relevant, but it is important when happening in combination with other alarms. In this case, you can use a derived alarm feature to report that alarm only in that circumstance. Keep in mind, though, that one of the circumstances could be that the condition is only an alarm on a given day of the week or time of the day.
If problems are self-correcting, there might not be a need to know something happened if it has already been fixed. If that's what you want, you can filter these alarms by using an alarm qualification time that says the condition must be in existence for a given number of minutes before an alarm is declared. You may want to use alarm qualifications for things such as power failures, fades, etc. There is also a more powerful alarm qualification feature that will alert you when an alarm point fails more than a given number of times in a specified time period.
Now, let's assume that you have a point that must be monitored, but unfortunately it enters an oscillating condition that creates a lot of alarm activity. In this case, you would want to use an "alarm silencing" feature that turns off that alarm point for a specified amount of time. This saves you time because if you already have taken action to fix the problem or scheduled a correction time for the problem, there is no need to be continuously reminded of it.
Implementing some or all of these strategies will make your monitoring system to run quieter and become easier to manage. So, make sure you're selecting a system that does feature all these nuisance alarm capabilities.
Another monitoring best practice to follow when designing your NOC is choosing an alarm master that has a convenient and intuitive interface.
There are many companies out there that have alarm masters that are difficult to understand. The screens look like a wall of numbers, and no potential network problems are obvious or easy to understand. This kind of alarm master leads to poor remote alarm management, as more and more training is required before any new team member can use it. Even after comprehensive training, you still won't have optimal situational awareness.
You don't want your staff to waste time trying to figure it out what an alarm means when they could be reacting to it. Every minute wasted in your NOC means more expenses for you and greater potential for a missed problem leading to extended network downtime.
Fortunately, finding an alarm master with an intuitive interface is very easy. Within 2 seconds of looking at the device's display, you'll know if it's going to be easy or hard to use.
That's why it's so important to demand that your vendors show your screenshots or a live demonstration of their alarm master interface. You don't have to accept anything that isn't clear. An alarm master is a significant investment you'll have to deal with for at least the next several years.
Map-based alarm displays tend to be one of the most intuitive interfaces. Almost everyone can understand a map of their local region, and it's easy to see network threats evolve across multiple sites when you're viewing real geography. Viewing your alarms on maps ("war room" display), instead of just on a text-based list, adds instant usability to your alarm manager.
Most of the times, especially for sizable networks, a wide range of people access aggregated alarm data both within your NOC center and out in the field.
That's why it's so important to choose a master station that supports many simultaneous users. Ideally, your operators will access the master station remotely via LAN from their individual workstations.
While map-based interfaces are typically installed software applications, it's also handy to have a web interface (especially if it includes Web 3.0 functionality). Your technicians can have access to the network from anywhere - including from their laptops when out in a remote site.
Your network infrastructure was expensive to build, and you spend more money each day to maintain it. Your customers or end-users all depend on your ability to provide reliable service.
Considering all this, one very important best practice is to avoid running your alarm master network management software on a consumer-grade PC workstation.
Instead, look for a telco-grade hardware platform in a metal chassis containing redundant hard drives. For larger networks, the ability to run a redundant pair of master stations in different locations is also valuable.
Your NOC center is a vital part of your operation. It's where that important decisions are made regarding the status and performance of your network. These decisions help protect your network and provide reliable service to your customers or end-users.
But, can you afford the alarm monitoring you need? Especially you have a small network, budget considerations can put centralized alarm monitoring out of your reach.
As an experienced provider of monitoring solutions, we've had many people reach out to us after expensive (and preventable) damages caused by small-site failures.
To help you avoid problems like these, our team of experts put together the How to Build a NOC without Destroying Your Budget white paper.
With this white paper you learn how it's possible for even a small network to enjoy the benefits of centralized, NOC-style alarm monitoring - without blowing your budget to pieces.
Download your free copy of How to Build a NOC without Destroying Your Budget and find out how.