« I'm a genius, you're an idiot | Main | How did they get here? »
March 15, 2003
Network Management Observations
A major part of my work life has been spent managing networks. I am still overwhelmed by how complicated it is to manage a big network well. In spite of all the tools that are designed to simplify and automate network management, my experience has been that those tools only make one more aware of other problems that need to be resolved.
Some recent experiences and observations:
- our last two major security incidents were discovered by network people who were actively using the network at the time they occurred, and this was the primary reason we were able to protect the network long before most other networks were aware there was a problem (through their automated tools)
- collaboration is one of the best defenses we have against security incidents right now. The nsp-sec mailing list has been one of our most valuable resources in identifying security issues. Collaboration has also been the most effective way to resolve security issues within the network. I think that this is a vision into the future of security methods, that the most effective ones will be collaborative.
- sharing information on current network hot-spots. We set up a system to notify us when any configuration changes are made to core routers. This was primarily to make us aware of any security issues with those routers, but I've found it far more valuable in another way. Any time a change is made, I know there is an issue being worked on. If I get a couple of notifications, I will usually contact the person working the issue, just to find out what is going on and if I can help in any way. It's a great way to stay in touch with problems affecting the most critical areas of the network.
Information management is one of the two primary responsibilities of a network manager (the other is incident remediation). Collecting, analyzing, distributing and acting on the information is a huge, complicated task. Organizations are frequently distracted by thinking that the right tool will make this a simple task, and they go through one tool after another trying to find the Holy Grail (a substantial part of the tools industry is built on selling software that never gets used). I've been through three organizations that have tried this approach, and I'm convinced that which tool doesn't matter so much as how the tool is used, or more importantly, that the tool is used. It's the realization that a tool itself isn't the solution, it is a way to enable those who use the tool to create a solution.
Posted by pete at March 15, 2003 7:57 AM