Why the Tasmanian filtering trial is a failure
July 29, 2008 – 8:42 pmSenator Conroy’s office could spin a thread of gold from a lump of crap, I’ll give them that. But if you’ve read the Tasmanian filtering report, it probably didn’t take long before you noticed red flags. I decided to compare the report to what Mr Conroy says in his press release and show why the trial is nothing but a miserable failure.
What Senator Conroy says: The performance or ‘network degradation’ for one of the tested products was less than 2%, whilst three products were less than 30% and two products were in excess of 75%.
What the report really says: The filter that showed less than 2% network degradation was also one of the least accurate filters at identifying illegal and inappropriate sites. The more accurate filters showed a larger drop in network performance.
What Senator Conroy says: Successful blocking (the proportion of illegal and inappropriate content that should have been blocked that was successfully blocked) was between 88% and 97% with most achieving over 92%.
What the report really says: It probably won’t take any more than 12 or 13 clicks before a filtered user can access a site containing adult or inappropriate content.
What Senator Conroy says: Overblocking (the proportion of content that was blocked that should not have been blocked) was between 1% and 6%, with most falling under 3%.
What the report really says: Even if you choose the best result (1%), out of every one million websites, 10,000 will be blocked when they shouldn’t be.
What Senator Conroy says: All filter products tested were able to block traffic entirely across a wide range of non-web protocols such as instant messaging and peer-to-peer protocols. However, most filters are not presently able to identify illegal content and content that may be regarded as inappropriate that is carried via the majority of non-web protocols.
What the report really says: The only way the filters could block traffic on non web protocols was to ban access to them completely. That means if you want to chat to Gran about her garden or drinking habit on Messenger, you wouldn’t be able to. No matter how innocent.
Those are just the starters. Here’s the main course:
1) The number of simulated users was too low. Large ISPs have hundreds of thousands of customers and even small ISPs have thousands.
2) During the trial, only 3930 URLs were filtered. When you consider Mr Conroy wants to block ‘inappropriate content’ to children, 3930 URLs is simply too low and doesn’t show the potential real impact on network performance or filtering effectiveness. The Internet contains hundreds of thousands of websites not appropriate for children by our classification standards.
3) The report claims all but one of the six filters was able to ‘filter’ HTTPS traffic. That means the Government could intercept your private and secure banking details (or PayPal transactions) and leave the door open to fraud and identity theft. Nobody wants their online purchases monitored by the Government let alone their banking put at risk.
4) There is no analysis of circumvention methods and that’s crucial to understanding why filters – ISP and software based – are ineffective. Filtering can be bypassed in minutes by a savvy net user and in hour by anyone following instructions.
5) There is no analysis of the costs of deploying and implementing a filter at ISP level, nor is there any analysis of the associated costs that will be passed onto customers.
That to me looks like a failure dressed up by a Senator with an obsession on controlling what you can and can’t see.
Visit NoCleanFeed.com to find out how you can take action. I for one wouldn’t want my blog or businesses website blocked for no reason.
Oh! If you spotted other doozies in the report, please post them in the comments. No doubt I’ve missed many.
Update: More flaws surface.
49 Responses to “Why the Tasmanian filtering trial is a failure”
Get this into tomorrow’s Crikey mate! That’s read by all the policy wonks and people who can make hay with it.
By Simon Rumble on Jul 29, 2008
Didn’t think she was quite up to Crikey standard. Maybe an extended reader comment
By Mike on Jul 29, 2008
Michael,
I agree with Simon, at the very least a letter to the editor, and to the Opposition who would love to taste Conroy’s blood.
By Terrence Valter on Jul 30, 2008
Hey, thanks for the summary. That’s one heck of a long report, and the further I get into it, the more exasperated I get.
This reads more like a promotional brochure than an actual report. It also delves into throttling P2P & non-web content toward the end, which I find a little bit trashy and presumptuous. Overall, it looks like this is going to hit the general web user more than it will hinder the curious thirteen year old.
By Ash on Jul 30, 2008
Ash, your right. It does read like a promotional brochure. It’s hard to find places where the report is critical of any of its findings.
By Mike on Jul 30, 2008
I am a network engineer and I’m reading the report. It is riddled with technical errors.
Page 9 claims that ISP filters can’t be circumvented, which is nonsense — all it takes is a tunnel to outside the filter. Exactly the same technical people use to avoid the Great Firewall of China.
Page 24 claims that subscriber bandwidth is no more than a “few megabits per second”. Which is bogus, as it is the peak customer bandwidth, not the average, which will stress the filter. That is significantly more bandwidth as 1Gbps ethernet is the cheapest way to light dark fiber.
Page 25 talks about congestion in the “prime” network. This is jargon I’ve never encountered — and I’ve been working on data networks since 1985. I’d say it has been invented for this report. Anyway, the thought that “congestion” in the network core is less worse than congestion in the access layer is totally arse-about.
Page 25 says that networks are rated on both “bandwidth” and “throughput”. Which is an odd thought since these are essentially measuring the same thing, as T=1/B.
Page 26 is totally bogus. It uses as an example a gigabit ethernet link. It then presupposes retransmissions (presumably collisions) when GbE is only available on ethernet switches, which are collision-free. Errors compound as the author then states that routers change packet sizes in response to congestion. No they don’t — routers queue packets in response to congestion and discard packets once the queue length is too long.
Figure 5 is totally bogus, as its it’s explanation: “The performance of Ethernet traffic degrades as traffic becomes increasingly ‘bursty’; that is, where packet size becomes random.”. This simply isn’t so: packet sizes remain the same as the transmitter sent them.
What’s worst about all this analysis is that it is looking at the wrong thing. It is the performance of the TCP protocol, not of the Ethernet protocol, which will dominate the user’s experience when encountering a bottleneck like a filter.
TCP performance is determined by latency, jitter and loss. Nowhere does the report measure the effect of the insertion of a filter middlebox on those essential variables. In short, these amateurs have measured the things that don’t matter and not measured the things which do.
The very worst error is in the testbed design. 80-90% of Australia’s Internet traffic goes to the USA, a considerable distance and a large amount of latency. This is not simulated in the network design — rather the testbed assumes that all the content is local to Australia and thus of low latency. Effects like jitter and loss become exponentially worse as latency increases, so the testbed understates by orders of magnitude the performance effects on traffic. This single error is so huge that its effect swamps the results, rendering the results of no value at all.
Unbelievable that my taxes have paid for such an amateurish effort.
By Glen Turner on Jul 30, 2008
And Figures 13 and 14 are a classic. Inserting some filters into the network actually improves response time. The text doesn’t note that, nor ask why.
If it is experimental error then that seems to be about 5%, and so a lot of claims about the results are invalidated by the noise of this error.
The baseline itself is worrying, it is 425Mbps. The network is a GbE switch — so where is the bottleneck? It must be the web server. That is, the tests are measuring the performance of the web server. That’s a problem because the Internet may well have web servers which can serve at 1Gbps
And there’s another glaring error in the testbed. ISP networks are designed to be resilient — to continue on even if the ISP loses a POP. The testbed shows no resiliency techniques — such as redundant filters — at all. This would considerably alter the results.
By Glen Turner on Jul 30, 2008
I would dearly love to see this on Crikey and other media sites. It’s very well written and concise, it got the point across without sending me crosseyed with jargon
By Ms Naughty on Jul 30, 2008
The reason I suggest you submit this to Crikey is it’s clearer than Stilgherrian’s pithy comment on the same report. Your length is about right too.
By Simon Rumble on Jul 31, 2008
I spoke with Crikey yesterday and they requested a 150 word version of this… no doubt for comments. Hopefully it is included in today’s issue.
@Glen Turner: Thank you very much for the more in depth tech summary! I’ll try and publish a post tonight based around your comments.
By Mike on Jul 31, 2008
Update: Stil has a China focused filtering article in today’s Crikey. My comment re Aus filtering is in the comments section.
By Mike on Jul 31, 2008
Thanks for such a detailed analysis, Glen!
Please don’t be surprised by the amateurish nature of the study. I’ve worked in govt IT before and seen first-hand what kind of moronic things go on. The general level of wastage and under-the-table deals to mates would boil most professional IT people’s blood
By est on Aug 1, 2008
Would it be possible to DDoS the living shit out of these internet filtering boxes? This is completly bullshit. Fucking nazi fucks.
By J on Oct 2, 2008
This is ridiculous…. we are becoming the next China. Maybe its time to leave the country…?
By Troy on Oct 3, 2008
On network efficiency due to “retransmits”. Just checked our core switches, between the few gig interfaces I checked (of the few hundred) I couldn’t find a single dropped frame since last time the counters were dropped.
That’s only over the last 255734561704 bytes or so per interface.
Dreadfully painful to read. It’s like the network version of reading an Intelligent Design proponent critique evolution.
By Al on Oct 14, 2008
The frustrating thing is Conroy is pushing ahead with this no matter what, and Kevin-747 is too busy buying votes to care. How can this thing be stopped? Will the Liberals even stop it?
By Billythechimp on Oct 24, 2008
I’m Strongly against the possible ISP-level filtering. I know my views aren’t important to you but seriously when did it become necessary to restrict what people can see. Have we not experienced enough that we could all see this is a bad idea. I feel freedom is what Australia is about … If we introduce this and make it compulsory it is like falling backwards… Twice!.
If in the future this happens it could lead to the government controlling what we see. I’m not just talking about porn I’m talking about site’s that have opinions like this one will most probably be extinct. Not to mention sites that are detrimental to the government that show the flaws in own society.
We need to keep our opinions and our ability to think intact. We don’t want to conform and just follow the norm… because our ability to have our own personality is what separates us from most animals and I think its not how we should live our lives.
By Ronald on Oct 30, 2008
Yeah I agree with Ronald, trying to block porn or anything people believe is evil/illegal is just stupid,not to mention very hard to block the bad websites and allow the good.It makes the governments small efforts for faster and australia wide access to the internet a waste of time. I wouldnt like to go online just to look at the playschool website lol.It is a rubbish idea and if it gets through i wouldnt be surprised if someone creates a DOS attack software and tells everyone to mass protest.The lack of faith the government has that we can choose our own websites to look at and that we can raise our own children is pathetic.
By David on Nov 12, 2008
I must really say when I read this and information concerning this I was literally shocked this sounds like a second china. I live in europe so I do not have this problem directly but I have friends that probably won’t find a way to contact me easily which is rubbish. The “curious” children that look at content which isn’t suited for them are children who’s parents don’t care or more likely don’t know how to control this. In my opinion they will try to get this to work there will be many protests but after all it will crash either way. The system will fail due to lack of common sense and techiqual abillaties or because of the “real” or more direct protest once the system is running, which will be an unbeleavable number of peoples which will ddos this rubbish so long until everyone is also doing it and at the very latest then will it be forced to stop. My point is that no matter what happens people should not be “forced” to able or not to view what the hell they want.
By BeEgEeES on Dec 5, 2008
I’ve found absolutely no evidence in any of the proposal that this plan will ever work from a technical standpoint, ISP’s are shooting it down in flames left right and centre and the majority of the public detest the idea. It’s patronising and offensive that Conroy (and by extension for doing nothing to stop it, Kevin F*&king Rudd) has even considered the idea.
By Will on Dec 10, 2008
OMFG conroy u dumbass it will not work you should be spending money on giving us faster internet speeds australia has one of the slowest internet connections in the world we are even behind countries that are considereed third world i could bypass ur filter in minutes all it would do is slow down my speed argh if u wont to stop anything stop torrents thats where all the child porn and adult content is being transfered there will be riots if this comes to pass u cannot affectivly filter the internet do u want us to become like china where the govemement can block anything they want i along with 98 percent of australia are against this and the other 2 percent dont reliesehow this will affect our connections
By Colin Kirkness on Dec 23, 2008
Most people want to protect children but that’s not what this is about so it makes a good premise to hide behind. Chairman Kevin seems too close to China. Though I & most children could get around this filter, with a slower & dearer internet I & probably many others just won’t bother with it, which won’t be good for the economy or the ISPs. I’ve heard a lot of ‘they’ve got Filtering in Europe’, when a more honest comparison would be China. I would urge everyone to write to lots of pollies & ISPs.
By John on Dec 26, 2008
Does Chairman Rudd intend to break election promises of a faster & cheaper internet for Australians?
By John on Dec 26, 2008
ISP filtering doubleplus ungood
By smith on Mar 22, 2009
I’m glad I’m not the only one going insane from this stupid idea by our government. If your like me and curious as to why the government is so damn adamant with going ahead with ISP-level filtering even after it’s own feasibility report stated it was not economically or technically viable then I guess you just have to blame Steven Fielding. At the moment the ALP requires Steven Fielding’s support to pass bills through the senate. Without this balance of power the coalition and greens senator would block majority of their proposals. For the ALP this decision is a matter of political safety, which makes me think they’re going to go for it even if it means fudging reports, bullying ISP’s and arguing that we’re “saving the children” even though we all know there’s no way it can be done!
By Antonia on Apr 26, 2009