High-level report which found filtering flawed kept secret
December 22, 2008 – 11:53 pm
Update 23/12/08: Thanks to front page pressure on The Sydney Morning Herald, the DBCDE has been forced to release this report. Download the PDF. I’ve published Conroy’s response here.
Asher Moses at Fairfax reports that a ‘high-level report’ on ISP filtering given to the Federal Government in February was kept secret. The report, which was prepared by the Internet Industry Association for the Howard government, found ISP filtering technology does not work, will significantly slow internet speeds and will block access to legitimate sites. It also found schemes for blocking ‘inappropriate’ content such as child pornography were fundamentally flawed.
Moses writes:
The report, based on comprehensive interviews with many parties with a stake in the internet, was written by several independent technical experts, including the University of Sydney’s Associate Professor Bjorn Landfeldt. It was handed to the Government in February but has been kept secret.
“I definitely think that what the Government is showing publicly … is such a small part of what they need to do in order to get this right,” Professor Landfeldt said. He said he believed the Government had not released his report because its conclusions were too damaging. “It’s definitely not going to be workable to get a very significant reduction in access to this (unwanted) content that is available out there, it’s fundamentally just not viable.”
[...]
Senator Conroy refused to comment directly on why the report has not been released or why the trials are going ahead given its findings.
23 Responses to “High-level report which found filtering flawed kept secret”
How long will Conroy carry on with this. I assume he will take some political damage from it. Maybe that’s an aditional avenue of attack, ridicule Conroy, satire with rubber puppets is needed damn it
By Sean the Blogonaut on Dec 23, 2008
LOL this is scandalous but hilarious. All the stars are aligning against this thing. Maybe we should start placing bets on when it will get dropped. I reckon about a month after the ‘live trials’ finish.
By Danu Poyner on Dec 23, 2008
I notice that the report was commissioned by the IIA who have many members that are opposed to Internet filtering and very few who have even installed an Internet filter and know anything about them. You would expect that a report on Internet filtering would have researched those who have successfully installed filters in the region and investigated other installations around the world. But no, if you include those you might find out that properly installed filters actually do work and don’t slow the Internet down!
By Skeptic on Dec 23, 2008
Hey Skeptic:
The report was prepared by the Internet Industry Association for the Howard government. According to Moses it was based on comprehensive interviews with many parties with a stake in the internet and was written by several independent technical experts, including the University of Sydney’s Associate Professor Bjorn Landfeldt.
By Mike on Dec 23, 2008
Mike
I’m not denying that these people are experts on the Internet but are they experts in internet filtering? Did they talk to an ISP in Australia who has been doing this for years and is using technology that has no effect on user performance? Ditto for an ISP in New Zealand. Did they talk to ISPs in the UK and Canada who have been using ISP filters for years across their whole networks without affecting user speeds (with more Internet users that Australia has, I might add)? Did they talk to ISPs in New Zealand including Telstra’s own subsidiary over there who have been involved in an ISP filtering trial without affecting user performance? I think not. Then what value is the report?
By Skeptic on Dec 23, 2008
@Skeptic
Can you point us to the source of this claim: ‘an ISP in Australia who has been doing this for years and is using technology that has no effect on user performance’?
The systems in the UK are different to what the Government has proposed, both from a technical and regulatory perspective. See http://libertus.net/censor/ispfiltering-au-govplan.html#s_6 for information.
As I have not seen the report, I can’t verify its worth or accuracy.
By Mike on Dec 23, 2008
The ISP in Australia is Webshield in SA.
Although I agree that the regulatory framework is different in the UK, Canada and New Zealand from Australia, that does not effect the technical perspective. It is exactly the same, so what works on an ISP in the UK for blocking at ISP level for a blacklist will work in Australia. I have not seen the report either, so can only comment on the contents of the report that was reported by the media.
By Skeptic on Dec 23, 2008
@Skeptic
If Webshield claim that performance is not reduced (can you point to where they make such a claim?), it does not mean filtering an entire country or a large ISP like Internode would not see a reduction in performance.
As we saw from the report on filtering products tested in Tasmania, performance was always reduced and that was in a closed environment.
The more sites those filters blocked – the less a performance drop – but with that came a higher rate of innocent sites being blocked. The filter that showed less than 2% network loss did not accurately identify content.
Saying the systems used in the UK ‘work’ could be seen as overstating their effectiveness.
As we saw last week with the UK wiki incident, a lot can go wrong from a technical standpoint. The lists have also been used to block non child porn sites in other countries and with any country that uses DNS blocking, legit sites can always be restricted.
BT’s system is designed solely for BT, deals with just over 1000 URLs, is designed to prevent accidental access, and is not dynamic ( which is part of the proposed Australian system ). The Australian system requires considerably more sites be blocked given our restrictive Broadcasting Services Act. Blocking child pornography websites is not the Australian Government’s only goal.
By Mike on Dec 23, 2008
@Mike
Webshield do not make that claim themselves. I make that claim because I know that they are using pass-by technology which does not slow traffic down due to it’s inherent design as it does not interfere with the data stream. It will also scale to a large ISP. Optional filtering (as apart from mandatory filtering) is only done on a subset of the ISPs traffic anyway (as only a percentage of customers will opt for it) so you do not need to provision the optional filter for all of the ISPs traffic.
The Tasmanian trial did not trial the pass-by technology used by Webshield nor did it test the hybrid BGP and URL filtering done in the BT Cleanfeed system and as far as I can see only tested filters designed for a corporate environment which do have performance limitations as we can see.
As for the problem in the UK with Wikipedia – that was not a technical issue but a URL list issue which has been sorted by the IWF removing the URL from their list.
I cannot comment on what the government will use on their mandatory list as this has yet to be clearly defined. Whatever, the fact remains that the hybrid BGP and URL technnology used by BT (and other ISPs) does work for a blacklist and would work in Australia.
I’m off on holiday.. Have a great Christmas.
By Skeptic on Dec 23, 2008
how long before the blacklist is leaked? if introduced, i say no more than a month…
By the master on Dec 23, 2008
Skeptic:
> very few who have even installed an Internet filter and know anything about them.
I’m quite certain many of the companies and organisations involved use filtering on their internal networks. It’d be insane for most of them not to.
And even if I were wrong and office buildings around Aust were filled with employees with one hand under the desk, the only factor there’s any doubt over here is the implications on speed.
Speaking of which, I’d like to make a genuine request for further reading on that very issue.
Mike:
> The more sites those filters blocked – the less a performance drop
Isn’t this backwards? As I understood the Tas tests, each http request resulted in the desired URL being compared to the blacklist. The longer the blacklist, the more intensive and longer the process of comparison.
Skeptic:
> As for the problem in the UK with Wikipedia – that was not a technical issue but a URL list issue which has been sorted by the IWF removing the URL from their list.
How was that not a technical issue (the technical being a pretty minor implication, considering what happened)? Not being able to block a single piece of material without breaking a site like Wikipedia seems like a pretty big technical limitation.
By Icaria on Dec 23, 2008
&the master
There are many ISPs in the UK and Europe who receive the IWF list of blocked illegal sites and this has been happening for years without a major issue. If ISPs in Australia are less trustworthy than their counterparts overseas and are prepared to risk leaking a list when they sign an agreement not to well there’s not a lot anyone can do about that. The other alternative is the system used by ISPs in Canada where the list is protected by Cybertip.ca and they do not have any visibility of it.
@Icaria
Internal network filters as used by corporate networks are not suitable for ISP-level filtering as the majority of these use proxying or work in conjunction with an existing server. These technologies have performance and scalability limitations. Those tested in the Tasmanian trial showed that.
OK – I’ll concede that the Wikipedia problem was a technical issue exacerbated by the URL list issue. The Cleanfeed system in the UK uses proxies which are what caused the problem due to multiple Wikipedia edit requests being routed through the proxy server and therefore coming from the same IP address. That is one reason that I do not recommend proxying for ISP filters and that NetClean have come up with a much better system than Cleanfeed called Whitebox that uses BGP routing but does not proxy the traffic but routes it through a filter unaltered.
By Skeptic on Dec 24, 2008
the way Skeptic is carrying on , i would be be spurised if it is actually Senator Steven Conroy himself posting this information under an assumed name
By Dez on Dec 24, 2008
personaly, the blacklist will be leaked sometime, but I bet a whole lot of sites will be revealed early on. I can think of a number of sites that are legal that will probably be on the blacklist due to being “unwanted”.
By alphamone on Dec 24, 2008
Mike
Here’s the report – I’ll read it over the holidays.
http://www.dbcde.gov.au/communications_for_consumers/funding_programs__and__support/cyber-safety_plan/internet_service_provider_isp_filtering/
By Skeptic on Dec 24, 2008
@Dez
I’m offended that you think I’m a politician! No, I’m not Sen Conroy – just a guy who knows a bit about Internet filtering who likes to take an opposing side to the mainstream!
By Skeptic on Dec 24, 2008
> the way Skeptic is carrying on , i would be be spurised if it is actually Senator Steven Conroy himself posting this information under an assumed name
You’re giving Conroy way too much credit (since when does he answer questions or know what he’s talking about?) and Skeptic far too little.
By Icaria on Dec 24, 2008
Alright so I’ve finally found some details on the UK system:
http://en.wikipedia.org/wiki/Cleanfeed_(content_blocking_system)#Technical_implementation
It’s basically a two-pass process, where all traffic only passes through a preliminary ‘filter’ with a more basic list of suspect IPs. So you wouldn’t likely see a network performance drop in the range of 80%, however, the total URL list in the UK is supposedly in the upper range of 1200 URLS (number of suspect IPs is unknown), which wouldn’t even come close to satisfying the Aust gov’t ambitions.
By Icaria on Dec 24, 2008
@Icaria
Isn’t the (Australian) list like 1400 now? I don’t know that they have said it definitely would grow, I thought the point was they were told any more the 10K would kill the system and that’s why they are testing it.
By Phillip Molly Malone on Dec 24, 2008
I’ve not read anything which indicates that 10k is anything other than an arbitrary number for testing purposes, nor would it be a magic threshold number at which point the system ‘breaks’.
By Icaria on Dec 24, 2008
(sigh) Of course, any filtering scheme that may be introduced will be trivially bypassed. How? By purchasing an encrypted VPN service from a US ISP. Costs AUS$60 per year (software is free and open-source, Mac+Windows+Linux). Adds around a 20% overhead to your monthly data quota. Overall, it slows down access to Asia-Pac sites, but actually improves speed for some US + Euro sites.
Can this be blocked? Yes it can, but only if Conroy violates his own stated “rules” and adds the IP addresses of legitimate, inoffensive ISPs to his “blacklist” (thereby inviting legal action). Furthermore, once every citizen has their own encrypted SSL VPNs, the job of detecting real criminal activity will become that much harder. Oh, and of course any 10+ year old is capable of setting up such a VPN.
I’ve pointed the above out to the terminally-intransigent Conroy and Rudd, but the baptismal water seems to be blinding their eyes.
By Mickzie on Dec 28, 2008