cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

Data != Information or Got Bias?

1020
Views
0
Helpful
1
Comments
anhines
Beginner

I recently had the less-than-entertaining experience of a series of emails exchanged with a customer (manager level) on how to measure the “perceived user experience of throughput” through one of our nodes in their network. Since this node is one of many – very many – in the end customer’s data path, I responded with a link to the DSL Reports web site and its series of speed tests (http://www.dslreports.com/speedtest?flash=1 – I find them useful for comparison purposes). That, after all, is the throughput the user will perceive.

He replied, most unhappily.

It seemed, after much email talk back-and-forth, that what he really wanted was a means to measure the per-user throughput via a MIB on this particular node. Since the node has up to 40 distributed processors, each of which is capable of supporting 65,000 (old hardware)/136,000 (new hardware) user sessions simultaneously, and creating/deleting them at a rate of 100/200 per second, we don’t track the per-session throughput for each session.

What I was able to provide him was a set of MIBs – which they were already collecting – which measured the instantaneous throughput in each direction and which counted the number of active sessions at any given time. From this, they can derive averages, but only averages. Those averages may or may not be representative of their customers’ actual throughput experience … their customers, for instance, could be quite bursty in their traffic, but on average not use very much at all. In addition, the “user experience” involves latency at a large number of other nodes in the path of the data flows.

What I’m afraid this manager is going to get is some specific data which conveys to him a picture that doesn’t necessarily match reality.

Data is not the same thing as information.

Information has contextual meaning. As an example, consider this blog post, which I found quite interesting: http://pajamasmedia.com/richardfernandez/2009/07/20/hockey-stick/ I, too, thought the graph displayed was something else, though the particular thing I expected it to be reflected my biases, and wasn’t the thing the author suggested most people would expect.

That’s an important thought – I had biases in what I thought the data was telling me. And – like anyone else – I have biases in how I ask a question.

What I look for in KPIs from my nodes reflects my biases as to what’s important. What I look for reflects my education, my experience, and my exposure to other peoples’  experiences, usually in the form of horror stories. As the saying goes, Your Mileage May Vary.

1 Comment
knagesh
Beginner

This is actually one of my favorite topics. Often, I imagine the web as spaghetti (data) and the challenge is to extract the strand you want to consume (information).

I wholeheartedly agree that information has contextual meaning. Therefore, I believe there are lots of Web 2.0 businesses possible with the notion of delivering right information to the right person at the right time in the right format. Computing still has some ways to go in synthesizing data to derive information.  Which is where traditional performance indicators might be misleading.

You brought the example of MIB data. NMS solutions historically have been the culprits provding reams of data to the poor network administrator but very little information. For example, application level performance problems are difficult to debug looking at traditional MIB data.

In the world of personalized experiences how can we get the information to validate if the network is delivering what the user has paid for or expects?  Although I pay for "16 Mbps Service" from my local cable provider, I hardly get 5.5 Mbps.  But I suspect there is a fine print I am missing!!

Any ideas on how we can capture the characteristics, say for Mobility Quality of Experience when so many aspects are subjecdtive? Video performance, personalized policies (best rate, best performance, turbo mode...), nomadicity and roaming, profiles and sub-profiles (say for children of the subscriber), application performance for Unified Communications, latency and QoS SLAs, ... are all possible interpretations. Industry has to set some benchmarks to eliminate some of the subjectivity.

Nagesh

Content for Community-Ad

This widget could not be displayed.