Issue link: http://cp.revolio.com/i/734838
THE DATA CENTER JOURNAL | 27 www.datacenterjournal.com n etwork experts who grew up with the Internet can ensure two core assumptions: • e rapid increase in network utilization will continue, making scalability crucial. • Flow is data, and lots of flow is big data. Understanding network visibility as a big data problem, part of the solution lies in architecting a big data solution: a purpose-built platform, optimized for net- work data (NetFlow, SNMP and BGP), that is unified, open and comprehensive. How does this new approach address the speed, scale and usability requirements of network operators, now and in the future? the growing challenge Nearly two decades ago, rich media on the web began driving up per-user bandwidth and attracting masses of new users worldwide. Network traffic and complexity have been growing robustly ever since. Accelerating demand drove the build-out of broadband infrastructure, creating a faster and more powerful Inter- net. And that dynamic laid the foundation for the online services, such as streaming, shopping and banking, that are integral to today's network-centric lifestyle. is cycle—innovation pushing bandwidth and enabling more innova- tion—has seen parallels in intranets and other private networks, and it continues unabated to this day. e result is ever increasing network utilization. As networks have grown, they've become a much bigger target for threats to both security and availability. Network operators across the spectrum—retail- ers, financial institutions, health insurers, media companies and so on—live in the sights of skilled practitioners with mali- cious intent. reats to security and availabil- ity add to the challenges already facing network operators as they cope with rapid growth: • More hosts, and more bandwidth demand per host, puts networks under continual pressure to scale. • Increased use of streaming and online transactions brings increased sensitivity to performance issues. In some sectors (ad auctions, retail and so on) slight delays can translate directly into loss of revenue. • Traffic management hasn't seen any sig- nificantly upgrades since the introduc- tion of Multiprotocol Label Switching (MPLS) in the 1990s, and soware- defined networking (SDN) has yet to roll out to the mainstream infrastructure. • ere's no single-pane solution that provides fast, comprehensive insight into network status and traffic. Growth typically flattens as markets and technologies mature, but current trends in the network space argue against that dynamic happening anytime soon: • e range of connected activities on existing devices (mobile, tablet, desktop and so on) continues to expand. • e extension of new capabilities to mobile lets more people use more data in more places at more times. • e up-and-coming Internet of ings (IoT) promises to extend connectiv- ity to broad new classes of everyday objects. e net effect is that no respite is on the horizon for network operators. e Cisco VNI Global IP Traffic Forecast proj- ects an increase in global IP traffic from 59 exabytes per month in 2013 to 167 exabytes per month by 2019. As networks become more integral to our business and personal lives, it's increasingly critical to ensure performance, availability and security. All of these priorities rest on the shoulders of network architects, network engineering, network operations and infrastructure- security engineering. What they need are tools that can keep pace with the job. the visiBility Mandate As network challenges intensify in coming years, visibility that's broad, deep and fast will become ever more vital to net- work management. Visibility has a direct impact across important areas of network operations, providing answers to the most critical questions: • Traffic analytics: Which users, applica- tions, and services are driving network utilization? • DDoS detection: Are attacks affecting our services? What's their signature? Are they happening right now? • Traffic engineering: Are our links over- loaded? Is re-engineering needed? Are more links required? • Peering analytics: What changes can we make in our peering to increase perfor- mance and decrease costs? Every network operator has visibility to some degree, but the quality of visibility makes the difference between seeing and knowing. It's one thing to be aware that a problem exists or did exist. It's quite another to understand—in real time and/ or forensically—where that problem is (internal, external, CDN, servers, stack and so on), what's causing it, and how to fix it. In an ideal world, the same innovation that now makes possible nearly instant execu- tion of online transactions would enable nearly instant insight into network-perfor- mance issues, or would expose precursors to brewing attacks. But so far that develop- ment hasn't happened. e problem isn't a lack of raw data but rather the abundance of it. e basic information needed to glean insights from traffic—NetFlow, sFlow, IPFIX and so on—has long been available for harvest from routers and hosts. But even at mod- est sample rates, flow data adds up very quickly. e key for network visibility and analytics is to collect and store that data in fine detail. It's already a big challenge, and as growth accelerates in streaming, mobile and IoT, it's going to get bigger. at makes scalability a core necessity for any architec- ture deployed to enable visibility. a scalaBle foUndation Scalability is what makes a flow-based data system capable of meeting important requirements: • Elastic capacity: Keep full detail (e.g., unsummarized flow) for a long enough time frame to analyze patterns and find causes. When capacity doesn't scale, you're le with less detail, shorter look- back and compromised analytics. • Fast ingest: Capture in full detail from thousands of devices simultane- ously. e faster data can be ingested, the sooner that data is available for querying. • Fast queries: e faster queries are handled, the sooner flow-derived in- sights can inform operational decisions (human or automated). • Multitenancy: Serving unlimited clients without bottlenecks maximizes system utility and value. Scalability must be built in from the outset. A system with inherent architec- tural obstacles to scaling is a system that