« February 2003 | Main | April 2003 »

March 31, 2003

Networks and Government

This article from Slashdot got me thinking this weekend.

Many of the rights we enjoy as citizens (and residents) of the United States are based on the assumption that access is unrestricted. The rights to freedom of assembly, voting, justice, participation in government and more are based on the assumption that people can get to the places where those rights are exercised.

This assumption can be justified because the access to public buildings is facilitated by publicly-owned and regulated streets and roads. Citizens can be assured that they can get access, with minimal or no cost, to participate in their government and with each other.

It's been talked about for decades that the network would eventually replace some or all of the in-person interaction we have with each other and with our government. E-government is all the rage now; broadband networks proclaim the future of telepresence.

The majority of network are privately owned. What obligation or incentive do those companies have to ensure that the Constitutional rights of their users are upheld? Are they obligated to ensure that their users can vote or participate electronically in a town meeting? What's to prevent the network owner from indirectly impeding citizen participation through subtle means, such as not addressing a performance issue that affects voting, or not responding quickly to a network failure during a critical public meeting?

To some extent some of these indirect actions are happening now. Carriers are not aggressively deploying broadband residential access, often for largely political reasons. Broadband providers restrict what users can and can't do on their networks. National networks won't peer on a regional level to enable high-speed network applications within a community.

Community networks may be part of the answer, since they are citizen-owned they can be assumed to be aligned with the needs of the citizens in that community. Many are built with e-government as an expected use.

Undoubtably, regulation will be another part of the answer. I am not aware of any regulations of this type yet, but e-government and broadband are both quite new. As they both become more pervasive, it's possible that broadband providers will be regulated to some extent to ensure that citizens have unbiased and unrestricted access to their government through the network.

Posted by pete at 10:20 AM

March 27, 2003

Jumbo Frames and Jumbo Bandwidth

I've been researching lately some of the limitations to fully utilizing a Gigabit (or faster) network.

The congestion-avoidance algorithms of TCP create one very significant limitation. The equation for the maximum performance on a given TCP session is:

TCP Throughput <= ~0.7 * MSS / (rtt * sqrt(packet_loss))

MSS = Maximum segment size, usually 1460 on an Ethernet network
RTT = round-trip (end-to-end) time, in milli-seconds

On a typical TCP transfer with RTT of 40ms (very typical), the maximum throughput is 6 Mb/sec. Upgrading to GigE or 10GbE, upgrading the server and workstation--the upper limit is still 6Mb/sec. And performance issues on the workstation, server or within the network result in typical peformance much lower.

Throughtput is dramatically increased by reducing RTT (sometimes, but not often, possible through peering or local caching), or more likely, by increasing the MSS. A Jumbo Frame MSS of 9000 bytes would increase maximum throughput to 40 Mb/sec.

UDP and other protocols don't necessarily have these limitations, but any protocol or application with error-control and congestion-avoidance mechanisms (which any widely-deployed protocol has to have on a statistically-multiplexed network like the Internet) will have similar performance limitations.

Another critical benefit of Jumbo Frames is lower processing requirements, on the workstation, server and network infrastructure (switches and routers). With 1500-byte packets, a full 1000 Mb/s would be 83,330 packets/second (or 833,330/sec at 10GbE). With 9000-byte packets, it's only 13,890 packets/sec. Jumbo frames use 1/6th the processing power of 1500-byte frames, which is absolutely necessary to ensure that the workstations, servers, routers and switches can actually achieve Gigabit speeds and higher.

Since GigE is still usually deployed just within the core of a network, these issues haven't received a lot of attention yet. However, with most workstations now shipping with 10/100/1000 network cards, and networks like Abilene encouraging wide-scale deployment of jumbo frames (the median bulk TCP flow throughput on Abilene, a high-speed research network, is 1.9Mb/s), and users pushing higher-speed applications, and the wide-scale availability of broadband and high-speed networks--all of these point to the inevitability that jumbo frames will be an absolute necessity within the next couple of years, particularly within networks that are aggressively deploying GigE.

We are testing jumbo frames over the next couple of weeks, in preparation for deploying jumbo frames within the University of Utah campus and hopefully in some parts of the UEN network (where the equipment supports jumbo frames). As we deploy 100Mb and 1000Mb Ethernet over the next few months, jumbo frames will be an important capability of the new network, to ensure that users will be able to utilize this additional bandwidth.

Posted by pete at 10:13 PM

March 25, 2003

Thinking about my visit to The Library

I had my first chance today to stop by the new Salt Lake City Library. This is an exciting development for Salt Lake, and especially for the downtown area.

I used to live downtown. I spent a fair amount of time at the old library. The new one is so different, it doesn't feel much like a library at all. It is more like a mall with books and lots of other things. I really like it, and I look forward to spending a lot of time there.

As I walked around, I saw a poster that listed some of the comments from a citizen survey done before the new library project was started. Some of the comments were things like "there should be more light" and "it should have a coffee shop and cafe, like a book store" and "it should be an icon in our community" and "there should be more computers." I didn't write them down, and there were some other very good comments.

Standing there, looking around at the new library, I could see how well the design accomodated these suggestions. Just walking around, I could feel that the new library was much more of what I want in a place to meet, read, study, and think. I suspect that many of the suggestions were things that hadn't been considered in the plans for the library, and may have been counter-intuitive to people who "know" what libraries should look like.

I got to thinking about all of the things I work on, and how often those projects aren't built to meet the needs of the people who will ultimately use them. How often am I designing a library without windows, because that's how a library is, when light is one of the most important things library patrons want (there are few places in the new library without natural light and an incredible view, and that's a big part of making it so spectacular); and, how often do I decide not to include a cafe, because I'm designing a library, not a book store.

I'm glad that the library was designed with the input of the people who would use it, and with consideration for how they would use it.

Posted by pete at 9:59 PM

March 24, 2003

Vacation Area Network

Mike and I have spent years developing the idea of the Vacation Area Network. Each year when we take a family trip together (his family and mine, we even include our wives and kids in some activities), we try to "improve" the experience with the use of technology (usually somehow involving computers).

Over the years, we have made several attempts at a near-real-time reporting on our trip, with stories, pictures and video. We haven't been successful just yet. But this year, things will be much different, I think.

We now know about Weblogs. This seems like a good possible fit.

We've figured out how to get video from a camera to the Web. In theory it's not tough, but try making a decent-looking video, and then make it stream-able to a typical Internet connection. An educational process.

The most recent piece is something we've worked on the last couple of days as Mike's been visiting me. I tried out Gallery a couple of months ago, and we decided it was the perfect solution for our family Web site. We've been integrating it into our site, and figuring out the logistics of making it multi-user and useable across a broad range of computer platforms and computer skills. I expect this will also be a part of our next VAN.

Network technology has some amazing capabilities to keep people together. Families that are spread out (Mike lives in Boston) can keep together now better than ever (we use email, Instant Messenger, video chat, a family web site, a web forum, a family event reminder, an email forwarder service, weblogs, on-line pictures, on-line video and probably a few other things--all to keep in touch, not to mention phone and in-person). These network technologies make distance matter much less than it used to, though it's still a long way from Salt Lake to Boston.

We are planning to vacation in Acadia National Park in Maine this summer. The VAN will be an incredible one this year. I'm looking forward to it.

Posted by pete at 10:56 PM

March 22, 2003

Open-source network measurement

My recent focus/interest has been network measurement. This is partly motivated by a requirement for an IP QoS solution to allow us to converge our video, voice and data networks at UEN. But it started shortly after I joined UEN, when Jim and I started talking about what kinds of metrics a network operator can be and should be measured by, and that would drive the right kind of technical, operational and organizational changes to make the network better.

At NGN last fall, Jim and I attended a great tutorial on application performance measurement. This started a focus to find ways to better measure users' experience on our network. We've found some metrics that help identify obvious problems, such as saturated circuits and overloaded routers, but there are so many performance issues that don't show up in the metrics we can currently measure.

Vendors like Brix, NetScout, NetQoS have systems to passively or actively measure performance of certain applications across the network. These look like great solutions and are successfully used by several (maybe many) networks. The latest BCR magazine has reviews of these and other similar (commercial) tools. They are all quite expensive, and we would only be able to deploy them in a limited fashion on our network. So I've spent the last few weeks looking for other options, especially open-source ones. I'm impressed.

iperf is a measurement tool developed by NLANR. It's been used very extensively in the research community for projects like Web100 and the Abilene network. iperf can measure node-to-node throughput, packet loss, jitter and latency. It has a wide variety of options to measure all sorts of network characteristics. It runs on Linux, Solaris, Windows, MacOSX and any other platform that can compile the source-code. iperf (coupled with MRTG or Cricket for graphing) looks like a great possibility to actively measure the performance of different kinds of traffic across the network, and because it is portable and inexpensive (free), it could easily be deployed pervasively throughout the network. At least functionally, this would do the same thing as the commercial tools offer, but not as beautifully.

PastTmon is an Application Response Time monitor. It passively watches TCP traffic flows and records the performance of the application using PostgreSQL. It has a nice interface using PHP and the R graphing tool. Though PasTmon is early in development, it already looks very useful, and be able to provide the information we need to monitor TCP application performance. As an early user of the product, we may be able to make it more successful through our feedback and direct contributions to the project.

I think it would be interesting to try a Web Services approach to integrating these tools with the rest of our network management tools. These tools could also be extended to support more direct monitoring of applications, where a Web Services approach would also be very powerful.

These options probably pale in comparison to the commercial solutions we've investigated. But I have been impressed with the capabilities of an free/open-source solution, and I think it can meet the majority of our needs for measuring user-relevant metrics. Given the costs of the commercial tools, the capabilities of the free/open-source tools, and the budget constraints most organizations are dealing with, the free/open-source tools present a pretty compelling option financially and functionally.

Posted by pete at 1:24 PM

March 19, 2003

Metrics, the network operator and the network user

Jim has been writing about another aspect of network measurement.

Since I first started working with networks in the pre-Mosaic days, I've been intrigued by network operations and particularly network metrics.

Network users think about the network much differently than network operators. As a network user, even a technical user, my perception of the network is based on how my applications perform over the network. I get accustomed to applications responding in a certain way, and when they perform differently--even slightly--I notice it.

A network operator sees the network much differently. Usually the network operator is connected to the network much better than the users are. The operator also uses a different set of applications--the ones that manage the network--than users do, and the management applications rarely run over the network in the same way user applications do. Network operators interact differently with the network management applications, and network problems rarely impact the way the management applications work.

The best measurement of a network is how it performs for the people that use it. Few network management tools can measure more than a few factors that related to user experience. The metrics they measure are sometimes relevant (especially when something breaks), but just because the metrics are good doesn't mean that the user experience is good. Unfortunately, this is not often recognized by network operators, who will usually point to all the green lights on their management tool and tell the network user it can't be a network problem, must be "something else". But if the user can prove it's a network problem, the operator will be "happy" to look into it further.

The irony is that when that operator goes home and becomes a network user (especially if it's on someone else's network), he will immediately switch metrics and measure the network like any other user. Other metrics (like those he uses at work) are only used when he senses theres a problem because applications don't work normally.

The more I've thought about it, the more I believe that the reason why network operators have a different perspective than their users is simply because they don't have the metrics to measure their users' experience. I think that if operators could measure metrics relevant to their users' experience, those would quickly become the most important metrics and the other metrics--the metrics they rely on solely now--would supplement and fill in the detail of the user experience metrics.

Posted by pete at 10:48 PM | Comments (1)

March 18, 2003

Blogeneering

I've been Hot-Teaming this week, it's been a fulfilling two days already to get some significant progress on a project that has languished for years.

I took a brief segway with Jim to do some blogeneering. Jim has been experimenting with activeRenderer, a nifty tool for Radio that does some cool DHTML things.

Radio is pretty powerful, and there are some cool add-in tools that I run into every so often. I wish there was a "Radio Userland Unleashed" book that would cover all of the intricacies of Radio's powerful capabilities. Most of this information is scattered around in various Weblogs and is pretty hard to find.

I run MovableType on some other blogs, and I have looked into some of the plug-in and back-end capabilities it has. I am pretty comfortable with Perl, the language MT is written in and uses for plug-ins, and that is making me start thinking the unthinkable: moving to MovableType for my own Weblog.

As cool as Radio is, I am not sure I am willing to commit the time to learn how to fully utilize it. Especially since the resources aren't very good to do that. I have to pick carefully where I want to invest my learning/experimenting time, and I don't take a new technology learning experience lightly. There are lots of new things I want to learn, and I'm not sure that Radio intricacies are high on my list.

Posted by pete at 1:59 PM

March 17, 2003

QoS pioneering

Last week we met with Brix, a company that makes QoS- and SLA-measurement products.

One of the major concerns I have about implementing QoS in our network is whether it solves the problem, or contributes to it. How do we baseline the current quality of the network, how do we know if it improves when we implement QoS in the routers? When there's a problem, how do we know if it's being caused by the network QoS feature or something else?

I have long suspected, and now feel sure, that we are going into territory that few people have been through yet. While there is no shortage of "experts" on IP QoS, all of these speak from a (theoretical) design and maybe implementation perspective. Few, if any, have any QoS experience outside of a LAN or small WAN environment. And only a handful can talk about operational QoS issues. I wish that "I've never seen QoS cause problems" could be reassuring; somehow it's not.

Finding Randy Weathermon and MOREnet was a welcome relief, and we learned a lot of invaluable information from both of them. We are meeting this week with some local people who have now failed three times in their QoS implementation, I am looking forward to what they have to say.

Posted by pete at 9:46 PM

March 16, 2003

Getting to the experience

Solving problems is a practical art, like swimming, or skiing, or playing the piano: you learn it only by imitation and practice. ... if you wish to learn swimming you have to go into the water, and if you wish to become a problem solver you have to solve problems. --George Polya (from Rich's blog)

One of my mentors taught me the value of being "in the space of the problem," and finding people who can speak from that perspective. Something about our human nature makes it hard to distinguish between thinking about doing something and actually doing it. We can so clearly see ourselves doing it, we often think that's the same as actually doing it.

About two years after starting my own company, after two of us had put all of our (very many) waking hours into getting the business off the ground, and had become very successful as a result, we had a falling-out with the third founder, who had stayed at his secure, high-paying job. In a last effort, we met once more to reconcile differences. In that meeting, the third founder told us that he could easily have done what we did, and probably better than we had. He could not possibly have comprehended how little he understood about what it took to accomplish what we had done. Needless to say, that only steeled our resolve. That was my first painful realization of the difference between thinking about doing something, and actually doing it.

Last week we talked to MOREnet about their experience with H.323 video and IP QoS. It was so refreshing to talk to someone who had actual experience with these technologies. Many of the things I've heard people theorize about H.323 and QoS turned out to be useless, impractical or just too complex in their actual experience. MOREnet implemented a simple design that has proven to be as effective as they need, without adversely increasing the complexity of their network. That one hour will probably save us months or more in time we would have wasted following the advice of people who could only tell us how they think it should be done.

With any project or new technology, I try to find people who have actual experience in that area. I try to move as quickly as possible to get experience myself. I try to engage the people who will use the final product early in the process, so they can experience it. I find that is the quickest, least-painful way to succeed. This often isn't intuitive, but I have learned it ... from my own experience.

Posted by pete at 10:59 AM

March 15, 2003

How did they get here?

I was poking around my Web log file (not to be confused with my Weblog) today. I needed some distraction from working on taxes (now I know why they say to keep good organized records). I noticed that a lot of people get to my Web site through searches, and some of them are rather interesting:

"comprehending big numbers": Power of many
"essay on network computing": What is the Best Network?
"XML Juniper": XML-based Network Management
"network computing in the 90's": Why I Work at UEN
"ipv6 weblog": IPv6
"troy jessup": Dufus
"threat to the music industry history": Power of many
"dufus award": Dufus
"groove windows peer-to-peer": P2P collaboration
"alin internetworking": Thoughts about application routing

Posted by pete at 4:44 PM

Network Management Observations

A major part of my work life has been spent managing networks. I am still overwhelmed by how complicated it is to manage a big network well. In spite of all the tools that are designed to simplify and automate network management, my experience has been that those tools only make one more aware of other problems that need to be resolved.

Some recent experiences and observations:

Information management is one of the two primary responsibilities of a network manager (the other is incident remediation). Collecting, analyzing, distributing and acting on the information is a huge, complicated task. Organizations are frequently distracted by thinking that the right tool will make this a simple task, and they go through one tool after another trying to find the Holy Grail (a substantial part of the tools industry is built on selling software that never gets used). I've been through three organizations that have tried this approach, and I'm convinced that which tool doesn't matter so much as how the tool is used, or more importantly, that the tool is used. It's the realization that a tool itself isn't the solution, it is a way to enable those who use the tool to create a solution.

Posted by pete at 7:57 AM

March 14, 2003

I'm a genius, you're an idiot

Like many who grow up in the technology industry, I was educated early on that technologists are the people who really "get it", and the sooner we can convince the rest of the world how smart we are, the sooner they'll get out of our way and let us do our jobs, and then everything will be better. Technologists usually identify sales, marketing and executives (and sometimes "users" and "customers") as the targets who stand in the way of the Right Way.

I don't think technology is the only field that promotes this kind of perspective. I'm sure doctors develop a similar perspective about patients and nurses, and sales people about engineers, finance managers about the people who spend money, executives about technical people.

I've worked with strong technical teams. I've also worked with executives, sales people, managers, marketing and end users. I found the technical perspective had some validity (technical people can come up with solutions, for better or worse), but it was too black-and-white, too either-or for me. I enjoy understanding the "other" perspectives, and advocating for the people who use technology (we are all technology users at some level, so I found the pure technologist perspective is often self-defeating).

Politics is probably the most confusing and frustrating aspect of relationships, especially to technologists (though I hear it from all across the spectrum). Political science was one of my favorite topics in high school and college, and continues to interest me.

I have a fear of politics. It's not that I don't understand politics, or that I am afraid to engage in the political aspects of a project. In fact, a lot of my interest in understanding other perspectives can be considered political. I'm more concerned about how politics will change me, how it can affect my ability to think critically, take risks, change what I do and what the organization does.

Politics has a slippery slope, that it can easily become the end instead of the means, and it can easily create self-serving individuals and organizations. Better understanding of political risks can paralize individuals and organizations, and make them unwilling to make changes, take risks or engage in important dialogues, because of political risks they perceive (or just fear of political impact). People can get caught up in seeking more power through political means, instead of using that political power to help the organization be more successful. People can also get caught up in spending their time reinforcing their current position (and as a result, the organization comes to a stand-still as well), instead of taking risks that can propel them and their organization to higher levels of success.

I will often deliberately ignore the "political realities" of a situation, because I think they inhibit my ability to see the situation for what it is, instead of what everyone thinks it is. This lets me take risks that I would otherwise be afraid to. And it sometimes gets me into trouble. Through experience, I hope I can get better at minimizing the downside without also minimizing the upside.

It would be nice if there was a right and wrong, or someone really smart that could tell us all what the right choice is. There are plenty of people who claim to be able to do that, but I have yet to see it happen.

Technology is the means, not the end, so inherently the success of technology is determined by those who decide to use it and become more successful by using it. Technology is at the intersection of users, technologists, managers, buyers, sellers, business motivations, technical advantage, politics, finance and all of the other factors that make up individuals and organizations. A successful technologist has to be at that same intersection.

Posted by pete at 9:21 PM

March 13, 2003

Quilting in Boston

I attended a meeting of The Quilt in Boston Tuesday and Wednesday. The Quilt is a consortium of gigapops, that facilitates collaboration on issues that are important to gigapops throughout the US. Recent Quilt projects include group negotiations for better Internet pricing, regional peering and fiber infrastructure development.

This was my first Quilt meeting. I have participated in the Peering Project since January, so I know some of the people but hadn't met anyone yet. One of the reasons I went was to participate in the roll-out of the first major milestones of the Peering Project. This included a Peering Toolkit to help Quilt members learn how to start peering regionally, something I'm very proud to have been part of developing. I got to present on the CommIX project UEN has worked on for many years, and had several other gigapops express interest in doing the same.

It's really exciting to see what The Quilt has accomplished in a short period of time. The results of the Commodity Internet Services project are phenominal; UEN will be able to save about 30% on our Internet budget once we can take advantage of Quilt prices. I expect we will find similar benefits from the peering project, that instead of trying to attract peers on our own, we can negotiate for peering on behalf of a huge group of users and large amounts of bandwidth.

The Quilt is a collaborative consortium. There are no membership fees, as an Internet2 gigapop we qualify by having four connectors. The consortium succeeds because gigapops freely contribute time to the group, and buy into the success of the consortium. It's really exciting to be part of a group that has produced great results and has such great potential.

Posted by pete at 7:06 PM

March 10, 2003

H.323 deficiencies and SIP opportunities

When I met with Cisco last week, we talked at length about H.323 architecture. I had heard about H.323 being a telco-mindset-inspired protocol, but had not delved deep enough into it to see how that might be the case.

Frankly, I was quite surprised at the immaturity of H.323 architecture. Obvious components (at least in the data networking workld) like distributed architecture and redundancy are not available. Traffic is managed through local gatekeepers with statically-defined sessions to other gatekeepers at other sites. Sure, H.323 can be used point-to-point, but that creates a nightmare for managing an H.323 network. It was hard not to see the telco psychology showing through the design.

I haven't read much about SIP yet. But if it's like any other IETF response to a ITU standard, it will replace the telco psychology with an network user perspective. Redundancy, interoperability, scalability, flexibile distributed management and functionality are some areas that look like a competing protocol could beat out H.323. I need to get a book to see if the SIP standard tries to address H.323 deficiencies like the ones I've seen.

Posted by pete at 7:59 PM

March 7, 2003

Network QoS

I met earlier this week with Randy Weathermon from Cisco. Randy is one of the most knowledgeable people on QoS technology that I've meet from Cisco.

I don't think the meeting inspired confidence in our QoS project. I asked Randy about QoS design and management tools. There are plenty of good design tools and tutorials. But the only tools for monitoring, manageing and diagnosing QoS-enabled networks are a hodge-podge of kludges.

One problem I expect would be very common would be diagnosing a network performance problem that could be caused by a QoS node. Randy said that the only two solutions they have now are to use Internet Performance Monitor to externally measure per-hop QoS behavior. Once you identify the individual node (if that's possible), use the QoS Policy Manager to remove each policy and see if that fixes the problem. Time consuming and not exactly inspiring.

Looks like we have a lot of learning to do once we implement QoS in our network.

Posted by pete at 10:57 PM

March 6, 2003

DNS Hot Team Week 1

I have written recently about the concept of Hot Teams.

We started our first Hot Team experiment this week, tackling a project that has languished for years, a DNS system clean-up and refresh. The current system does not support the latest DNS standards, and has a cumbersome interface that requires a substantial amount of manual babysitting.

The focus of the Hot Team is to quickly and effeciently fix both the interface (by updating or replacing the existing tool) and then clean up all of the DNS irregularitles. DNS is one of the critical services to create a functional network (as important as routers and circuits), so this project is one of our most important.

Our first task was to evaluate some potential replacement DNS management tools. Some work had been done previous to Monday, so we had a bit of a head-start. By lunch on Monday, we had completed evaluation of about a dozen potential tools, and had narrowed them down to 2 or 3 possibilities. We completed the evaluation Tuesday, 3 days early, and decided to fix our current tools. In spite of the speed of the decisions, I felt that because of the focus of the group, these were well-researched decisions that would not have gotten any better by waiting longer.

By the 3rd day of our first week (we only met mornings the first week), we were able to aggressively move into the Week2 of our project, and pursue Week3 tasks in parallel. While one team member focused on revising the softeware, the others started work on the tedious process of cleaning up the DNS system. By the end of the week, we are well into both of these tasks.

We start the Hot Team again March 17, but this time we will be working full days on the project. I am excited for the rapid progress we will be able to make that week.

Some things I've observed about this initial Hot Team project:

This Hot Team is successful already, and at the rate we go, I'm confident we will reach our goals. I expect that this success will ripple into other projects, and I hope it will help us get more done better with the few people we have.

Posted by pete at 2:33 PM

March 3, 2003

Developing belief

To develop belief, you must do things that require you to believe.

I heard this statement recently, or something really close to it. I think it applies to many intangible aspects of the things we do.

Many of the things I do require belief in something that is unclear, undefined, or may be perceived as undoable. I find often that whether I believe in the ultimate result or not has a huge impact on whether I achieve that result. It is often difficult to work towards a result that can only be imagined vaguely, but those difficulties are practice that makes the next project easier to do, though it may still not have a tangible result at the start.

Being motivated by belief causes your mind to focus on opportunities that will help you achieve what to others might appear unlikely. I am constantly surprised at the subtle opportunities that arise because I was tuned to them as a result of believing in the project I was working on. Those opportunities would have gone un-noticed if I had believed otherwise. Finding them made the belief become reality.

Posted by pete at 10:47 PM

March 2, 2003

Quotes

That which we persist in doing becomes easier for use to do; not that the nature of the thing itself is changed, but that our power to do is increased.

- Ralph Waldo Emerson

(who didn't like to be quoted)

A much more elegant way to say "practice makes perfect." Rarely is someone as good today as they were when they started; we often forget that they once struggled to do something different, just like we must.

It was a high counsel that I once heard given to a young person, ‘Always do what you are afraid to do.’ -- yes, it's the same guy

Jim's writing quotes, taking a break from being a Knowledge Manager. Never dress better than the boss, I've been told. And I adhere to that advise rigorously.

Posted by pete at 3:04 PM

March 1, 2003

Bandwidth liquidity

I got my first exposure to liquidity when I became an individual investor in June, 1999. That was the month I opened my first on-line trading account, and vigorously read everything I could about investing as an individual.

Market liquidity became very noticeable as the stock market crashed in 2001. I heard more and more people talking about how amazing it was that even when companies were failing, there were people buying up the stock as it was sold by ever-more-humble individual investors. It's a failing of the human mind, that we just can't imagine how many people are buying and selling stocks, so we have a hard time comprehending all of the motivations from all of the people and institutions who are always buying and selling stocks, no matter what the price. I may not get the price I want for a stock I want to buy or sell, but I can always count on being able to buy it or sell it. Amazing.

eBay is another example of liquidity. About a year ago, I tried an experiment to see how liquid eBay is. I cleaned out my supply of various electronic parts that I was sure to use someday, ten years ago. This included relics such as old 56k routers, old ISDN terminals and a beautiful fully-functional 5.25" floppy drive. It cost me $0.50 to list each one on eBay, plus some time putting together a nice posting. I sold everything. I didn't make much money, but somewhere in the millions of eBay users, there was at least one (and sometimes more) person who wanted my junk. Contrast that to the much-less liquid classifieds where I would never have had a chance.

The job market is another example of liquidity. I thought about this as I sat at a new very good Mexican restaurant at Jordan Landing (Enrique's). Some no-name start-up company can put a sign in the window to hire a couple of employees, and no matter how tight the market is, people will be available. There are enough people entering the work-force or floating between jobs to fill most of the positions that open up. Maybe not as good an example, but still interesting to think about.

So, my point. Bandwidth is much different. As an Internet user, and Internet company, or an Internet developer, I always have to think about bandwidth. And probably latency, too. But mostly bandwidth right now. As a dial-up user, I have to find applications that work on my 56k (or 28k) dial-up, and I often run into sites and pages and files that are just too slow to use, and I get frustrated. As an Internet company, I have to spend lots of time focused on my customers' experience at various levels of bandwidth (this would be somewhat similar to retailers having to track the driving experience of each customer from home to their store--which they probably do to an extent, but not as core to their business). As an Internet developer, a major part of my time (unless I plan to be fired) is spent considering how my application works on dial-up, broadband and high-speed connections.

Imagine how different the stock market would be if most people spent most of their time figuring out if they could sell or buy a stock, and trying to figure out how to sell or buy it, instead of just selling or buying to meet their investment or amusement goals. What if businesses had to worry about how to convince people to even consider applying for a job and if there were people who would want to work, instead of just worrying about interviewing, selecting and paying the people who apply for the job.

What if bandwidth was liquid? What if a users experience was reasonably consistent no matter what site they went to? What if Internet companies could focus on improving their customer experience through improved functionality, rather than figuring out how to squeeze more into a tiny chunk of bandwidth? What if developers could develop like they do on LANs, where only a small amount of effort is focused on bandwidth issues, and focus more on the usability and functionality of the Internet application?

High-speed networks are enabling bandwidth liquidity. Internet2 is one environment where most users, organizations and developers spend very little of their time worrying about bandwidth (though the network planners spend quite a bit of time). There is even an annual event to see if the available bandwidth can be filled. I think UEN is getting to that point, at least on our backbone, where concerns about wide-spread deployment of video and voice and bandwidth-intensive applications will soon be a thing of the past, at least within the network. Through peering and regional exchanges and lower Internet transit, we hope to improve liquidity outside the network as well.

As long as most of our efforts are focused on whether the network can support what we want to do, instead of focusing on doing what we want to do, the network isn't going to be very valuable. The sooner we can have enough ubitiquous, inexpensive bandwidth that we stop thinking about it, the sooner we will have a network-enabled society.

Posted by pete at 8:51 AM

We got what we want, now what?

I have worked in strategic technical positions most of my career (sometimes I actually had titles that matched what I did, too). I was able to take direction from industry luminaries, futurist pundits, some of my favorite magazines (Boardwatch Magazine, Wired, Red Herring, Business 2.0), synthesize it, expand and extend it through my own thinking and experience and discussions with a lot of other people. Very often my role, as strategic as it was, was mostly tactical maneuvering to get to where it was obvious that we should be.

UEN wasn't much different of a situation. Two years ago I began formulating our three-year strategic plan (since I'd previously spent most of my time at start-up companies, this was a real treat to do a plan longer than six months). In fall 2001, we began aggressively pursuing what we thought would at best be a 3-year effort to convince carriers and network vendors to start building to our requirements for a GigE network. We were literally laughed out of some of our first meetings, with many people telling us that we would never get GigE services and even if we did, what would we possibly need 1000 Mb/sec of bandwidth for, anywhere on our network (our network is mostly DS-3 on the backbone with T1 to the schools).

In spring 2002, we signed our first contract with two carriers to build GigE between three core sites in the metro area. That was quickly followed later in the year with contracts and plans with a few cities in Utah County who were building municipal fiber networks that passed a few of our schools, and later, a contract with UBTA to build GigE to our schools in Vernal and Roosevelt. We hoped that this would be the start of something, but it looked more like low-hanging fruit and we were fully expecting another several years until other areas would get GigE service.

Within a few months, the momentum created by those first few successes resulted in every telco in the state finding a way to get GigE to some initial core sites and schools, with plans to build to every school in the state over the next several years. Many sites will get a 10x bandwidth increase for the same cost. Suddenly, our 7-10-year plan turned into a 3-5-year plan, with many of the most critical locations addressed in the first wave. Now, "all" that needs to be done is get the funding and build the network. Simple execution.

So, we're 3 or 4 years ahead of where we expected to be, and probably 5-10 years ahead of the rest of the industry, which is only recently began deploying GigE in a limited fashion (certainly not to 100's of sites in mostly rural and residential areas), and with disruptive pricing only in the largest cities. The industry pundits are still talking about how we should be deploying GigE at disruptive prices. If we're already doing that, what do we do next? Who do we look to for direction? It's an unusual and rather uncomfortable position to be in. This is the position every strategic thinker always wants to be in, but few have been able to reach. This may be a once-in-a-lifetime experience.

We are already finding that being at the forefront with GigE means a shift in how we engineer and operate the network. Our first implementations have turned up lots of minor but frustrating problems as vendors, telcos and UEN work out kinks with new products, new designs and new operations processes. There are a lot of unknowns, and a lot of things to be afraid of. We will have a lot more of these experiences getting new technology implemented, and learning from experience what works well in designing and operating this new network.

QoS will be another interesting area. For the decade people have talked about converging all networks to a single IP backbone with QoS to make them work together, there's very little experience in the industry with making this work. We will learn a lot as we figure out what works and what doesn't, and find out what it means to manage a converged network.

For the future, we have to look to different, maybe less-obvious and less-clear direction. We will have to look into a much less clear future and figure out ourselves what needs to be next. We may very well become the pundits who start writing about what's next, at least what's next for UEN.

Posted by pete at 8:51 AM

golf tips