Are there real benefits to using structured cabling in your data centre, or is this just a bandwagon effect?
In our experience, a structured cabling approach can offer real benefits. But like any engineering solution, the use of structured cabling in a data centre is a matter of optimization and trade-off, not a one-way answer. It is suitable for some businesses, and prioritizes some types of operations over others.
Rather than preaching a one-size-fits-all solution, we think it is better to explore and understand how this approach works, and what trade-offs it favours, so that you can make your own decision. And that is what we are going to do today.
Structured Cabling in the Data Centre: A Quick Introduction
The concept behind structured cabling is mundanely simple: instead of designing a data network strictly in terms of connections between equipment items, we design it in terms of smaller, standardized building blocks called subsystems.
This approach breaks down and organizes the structure of traditional point-to-point wiring approaches. It imposes a standard structure to the network, making it easier to manage, extend and maintain.
How Is this Idea Applied in Data Centres?
Today, the most prevalent point-to-point wiring approach for small- and medium-sized data centres is the Top-of-Rack (ToR) approach. In this scheme, all equipment in a rack is connected to a rack switch (which is usually placed at the top of the rack — hence the name) via copper links, and the rack switch is connected to the core network via one or more separate fiber links.
In a structured cabling approach, all equipment, regardless of type, is connected to a set of patch panels in a subsystem called the control patching area. Instead of routing connections throughout the data centre, the connections are routed from the patch panel, using copper or fibre jumpers.
In larger networks, especially those that involve not only more network equipment but a more diverse set of equipment (such as separate, per-service sets of servers and SAN arrays), it may become necessary to introduce a separate layer of hierarchy. In this case, equipment from a service area is connected to patch panels in a zone patching area, and connections between zone patching areas are routed upstream, in a central patching area.
Note that the change is only in terms of where and how the wiring is done. The topology of the network is not affected. The network can retain its three-tier or spine-leaf structure if desired, but the location of the equipment is no longer constrained by point-to-point connections.
This is the essence of structured cabling in the data centre. Implementation details vary from one network to another. For example, there are users who prefer to segregate equipment into zones based not only on equipment type, but also on cabling type (copper or fibre), length and speed (e.g. separate zones for 1 Gbps copper, 40 Gbps copper, and 100 Gbps fibre respectively).
The exact installation options are based less on the regulatory framework (which is intentionally flexible) and more on the logistics, maintenance and scaling requirements.
Structured Cabling in the Data Centre: Tradeoffs and Benefits
One of the things that makes ToR point-to-point deployment so attractive is that it naturally lends itself to deployment of rack-sized modules. Most of the deployment operation is constrained to a single rack cabinet; there is a great deal of flexibility at that level, and some tolerance for “local patchwork” — such as “borrowing” a console port or a power supply outlet from the neighbouring rack.
Structured cabling trades local flexibility for greater uniformity and global/intra-zone flexibility, especially in terms of moving, adding and changing (MAC) equipment.
When new equipment is deployed, most of the connections are done on the patch panels. The ability to connect equipment arbitrarily is retained (although it tends to be equally undesirable — even though working with patch panel labeling is a lot easier than working directly with equipment ports).
However, routing connections like this may require a higher volume of cable — not necessarily in terms of numbers, but quite possibly in terms of length — and leaves little room for local workarounds.
What you lose in local flexibility, though, you gain terms of network-level flexibility. Routing between any two endpoints in the network requires only a patch jumper.
Introducing additional SAN arrays or redundancy layers, for example, becomes trivial in terms of routing effort. Long-distance cable routing is confined to fixed cabling, running only between the cabinet racks and the patching areas — uniform cabling that can be deployed once, in bulk.
Cabling volume vs. port volume is another trade-off that structured cabling touches. Structured cabling does rely on a high volume of fixed cabling. Upfront, this tends to result in higher cabling costs.
However, since equipment function and connections are no longer constrained by physical location, port oversubscription can be dramatically reduced by consolidating multiple rack-to-core switches. You no longer need to dedicate network ports and redundant power supply outlets to racks that will never use them.
If your network design can accommodate this solution, and depending on equipment it is possible to reduce oversubscription rates by up to 50%.
Cable lifetime and maintenance cost vs. upgrade cycles and upfront cost is another aspect that you should consider when it comes to using structured cabling in your London data centre.
The bulk of the cabling deployed in a structured cabling scenario is fixed cabling. This makes for easy installation, but high cable volumes that are straightforward, but costly to upgrade frequently.
However, if you can invest in high-speed, future-proof cabling, structured cabling can offer the best ROI. Its straightforward maintenance, reduced potential for human error (and mostly constrained to the patching areas) and — as we will discuss below — the excellent cooling and routing potential allow you to significantly extend the lifetime of your cabling infrastructure.
Challenges and Benefits of Structured Cabling in the Data Centre
Besides the trade-offs we mentioned above, there are several important benefits — and several major challenges — that a structured cabling approach introduces. These need to be considered in terms of absolute impact.
There are three major improvements that structured cabling brings: improved cooling, reduced administration effort, and smoother infrastructure integration.
Improved cooling and power consumption is currently touted as one of the most important — if not the most important — benefit of structured cabling. Since most of the local connections within a rack are moved to the patching area, there are very few cables that need to be routed within the confines of a single rack.
The cables that do remain are long and lead either to local power outlets, or to the patching zones. Either way, they are far more straightforward to route.
This can result in significantly improved airflow and, consequently, in drastically reduced power and cooling costs. More importantly, better cooling also results in longer equipment lifetime and fewer failures: the Communications Cable & Connectivity Association (CCCA), quoting the Uptime Institute, notes that failure rates for equipment placed at the top of the rack in ToR systems are almost three times higher than those of equipment placed at the bottom of the rack.
Another often-advertised benefit of structured cabling in the data centre is a reduced maintenance and administration effort. Patch panels connections tend to be easier to diagnose and adjust, assuming that they are adequately labeled and documented.
However, an unlabeled (or incorrectly-labeled) patch panel is far more difficult to navigate than even the messiest rack, since the connections are harder to trace.
Infrastructure integration is a detail that we often lose sight of, because data cabling network equipment power cabling obviously dominate the data centre environment.
However, a great deal of additional (and critical!) equipment is deployed in data centres: fire detection equipment, lighting outlets, cooling infrastructures — all of which are critical for the safety of your business’ data.
Running new overhead cable pathways — a typical feature of network expansion in a point-to-point deployment — under a ceiling which hosts all this additional infrastructure gets more and more difficult as your network grows.
The typical alternatives — running cable paths through floors — is not a panacea. When additional floor space is needed, entire rows of racks have to be moved. Structured cabling avoids this sort of problem by centralizing connections and easing cable pathway provisioning.
Structured cabling in the data centre is no free lunch, though. Structured cabling brings its own set of challenges to the table.
First of all, the price of the cabling infrastructure itself is almost always going to be higher for a structured cabling installation than for a ToR installation. We can debate choices based on ROI figures, but upfront costs are important, too.
Second, structured cabling is a radically different approach to infrastructure. Migration paths for existing ToR designs do exist, but they can be costly and downtime is difficult to avoid.
Many of the benefits of structured cabling are primarily relevant for networks with fast growth, where equipment is frequently moved, added or changed. The investment in a structured cabling deployment can be difficult to justify for networks where little growth is planned; more often than not, for largely static networks, careful maintenance and keeping cables tidy is all you need.
Finally, the rack-oriented structure of ToR deployments is not without its administration advantages. ToR deployments, for example, make it very easy to physically segregate network services.
In terms of logistics, we also tend to plan service expansion and deployment in terms of racks and units, too. The advent of technologies like Software-Defined Networking (SDN) may change that for some aspects of network development planning, but only slowly, and mostly for large deployments.
In other words, there are plenty of scenarios where traditional ToR deployments are a compelling option. Plenty of new ToR deployments are being made, and existing deployments are not going away anytime soon.
Conclusions and Recommendations
Structured cabling offers a credible alternative to traditional ToR designs. However, structured cabling in the data centre is not a silver bullet. It can improve operating costs, infrastructure and equipment lifetime, and offer greater and cheaper scalability. However, it does so at the cost of reduced rack-level flexibility and higher upfront costs.
At ACCL, we have been helping London businesses choose the right type of cabling for their data centre for more than 25 years. We can do the same for you! Following a FREE, no-obligations site survey, our team of engineers can recommend the best approach for you in terms of both costs and performance.
Get in touch with us to learn more about our process and how it can help you boost the performance of your network, keep costs under control and guarantee that your investment is future-proof.