Around this time last year I was visiting Rome on vacation, and, while it was packed with incredible sights and history, my visit today was nearly as awe inspiring, albeit on a different, more geeky level. As you may already know, I cover cell phone news for LAPTOP, so when I was offered a press tour of AT&T's Global Headquarters—the heart of AT&T—to see the Global Network Operations Center (GNOC), pronounced "knock," you can imagine my excitement. It's the location where all of AT&T's network, Internet, data, voice, and wireless information are monitored. As I walked into the stone building, I couldn't help but notice a statue garden with the words, "Rivers of the World Reflect The Art Of Networking," engraved on one of the art pieces that sat just near the front entrance.
I had never thought of large network providers being like rivers until then; our phones, computers, and televisions all sit like ports along a river that sends and receives data to different destinations. AT&T, in many ways, acts like the ship. I was fortunate enough to sit down with Chuck Kerschner, network operations director, to get a tour and have a few questions answered. First Look at the Control Center When I first walked into the command center it looked like a cross between a NASA control center and a stock trading floor. There are a few dozen desks with multiple monitors and about thirty or so workers watching the displays. When there's a problem with the connection between Mexico and England, for example, the engineers can see a color-coded line connecting the two places to highlight the problem. It's either red, yellow, or green depending on the status of the connection.
I sat down with Kerschner, who has a helluva handshake and speaks quite calmly for working in an environment that I can imagine gets stressful. He told me he oversees everything in the 198,000-square-foot command center, from the network and shift managers that work there to the support regions in destinations like Atlanta, Chicago, Dallas, and other cities. Data Volume "GNOC oversees 127 billion phone calls per year and 13.4 petabytes of data each day here," Kerschner said, including 80 billion text messages across the mobile network. Of course, the first thing I pictured was my text messages floating across their screens, but he quickly informed me that this wasn't the case: "Everything is on the aggregate network levels; we never view a customer's data."
Emergency Preparedness Now that I had about a dozen World and U.S. maps in front of me, I couldn't help but ask what shows up on the maps when disaster strikes, say, for example, when Hurricane Katrina hit New Orleans. "We focus on getting all calls out of the disaster zone during those situations," he explained. "We will reroute calls if necessary to make sure that people in an emergency can call out instead of letting calls in." If an emergency strikes the command center itself, Kerschner has two hours to get his team to an off-site location and then back up and running, and they practice this five times a year, one time unannounced. Speaking of disasters, you may remember the recent transatlantic cables that were cut by a ship's anchor. Kerschner told us that the first two actually gave them minimal, albeit noticeable, IP network issues, but that the second two did not impact his control center at all. I wondered if the command center ever turned into a loud and chaotic place during an emergency. Kerschner told us it didn't, and that it was usually as quiet as it was today. The noise level was slightly above that of a library. What Causes Spikes in Activity Next to the large maps, there were big charts of data and lists of switch locations. The charts showed the current volume of the networks, as well as yesterday's volumes. Surprisingly, it's a phenomenon that they look nearly identical. Changes in world news actually affect the charts dramatically. For example, when O.J. Simpson was all over the headlines, the network data spiked because of phone calls being made. When American Idol voting occurs, there are spikes from the immense increase of traffic. "Half the volume of American Idol traffic is in the first 30 minutes," Kerschner told me, laughing at the predictability of the network. Shifting Phone Habits I also wanted to know whether AT&T was able to see a difference in their network as people moved away from standard phone lines and more toward cell phones. "There are 350 to 400 million calls a day. IP is growing by leaps and bounds, but the shift towards cell phones increases in volume a tic each year," Kirschner said. Next Gen Networks As we finished up our discussion, I asked one final question: What will happen at the control center as we move toward fourth generation cell phone networks? "I haven't seen anything here," he remarked, "but it will be the GNOC of the future, and we're investigating the next generation and the future applications."
