Live filter pilot will not involve real customers
December 5, 2008 – 6:11 pmIt has been revealed that one of the most important elements of the live ISP filtering pilot, testing the impact filtering a blacklist of 10,000 URLs has on network performance, will be a closed network test and will not involve actual customers.
Here’s an extract from a letter sent by Senator Conroy to an Australian Whirlpool member (the full letter is here – PDF):
In consultations with ISPs, concerns have been raised that filtering a blacklist beyond 10 000 URLs may raise network performance issues, depending on the configuration of the filter. The pilot will therefore seek to also test network performance against a test list of 10 000 URLs.
This will be a closed network test and will not involve actual customers. The list of 10 000 sites will be developed by the technical organisation assisting the Department on the pilot, which has access to lists of this size. As this test is only being performed to test the impact on network performance against a list of this size, and actual customers are not involved, the make-up of the list is not an issue.
EFA board member, Geordie Guy, sought confirmation from DBCDE and asked whether the pilot would use “simulated users” and was told a clear “yes”.
He writes on his personal blog:
The live filtering pilot is not a pilot at all, it’s a prototype. It’s not the same thing as the real thing, it’s something different intended to illustrate similar concepts. You could also argue (and I do), that it’s not live – instead of what-you-see-is-what-you-get like live Television, it’ll be pre-performed and the results supplied after editing.
What will happen according to my determination (which DBCDE have confirmed), is that the prototype with ISPs will involve simulated users and the aim is to determine much the same sort of thing as the Tasmanian trials but on the ISP networks. Instead of thirty users like the Tasmanian trials it would be assumed that more would be simulated. Will this give a clearer picture (despite prototype vs pilot) than Tasmania did?
[...]
It’s certainly worth the cynical note that simulated users also do not publically complain that their Internet performance is degraded under the system.
And here I was thinking that the reason Senator Conroy didn’t answer Senator Cory Bernardi’s question about how many customers an ISP would be required to enlist for a trial to be credible was because he didn’t know.
Update 6/12/2008: Quote corrected – “It’s certainly worth the cynical note that simulated users also do not publically complain that their Internet performance is degraded under the system” attributed to Geordie Guy. Originally I attributed it to WP member in first quote. Sorry, my mistake – Mike.
32 Responses to “Live filter pilot will not involve real customers”
This doesn’t make sense against the tender document (or related document) that I have read that talks about surveying users. From my experience, bot users don’t tend to survey well!
I think you will find that the test to see how many URLs can be blocked without killing the products will be tested separately, in a lab with bots. Benchmarking is nearly always done with “bots” as they are more reproducible then real people.
The document that has this info is this one:
http://www.dbcde.gov.au/__data/assets/pdf_file/0006/89160/technical-testing-framework.pdf
It definitely points to real users using the system!
I am finding that the anti-filter campaigners are being really lacks with the truth or deliberately not digging very deeply into the context/meaning of things. Is there a reason for this?
Like there all very quick to report the 30% slower figure in the government tests, but not many point out that one of the products only had a 2% degradation? Again, is there a reason for that?
JMTC
Molly
By Phillip Molly Malone on Dec 5, 2008
I honestly can say that I did not see this coming.
Though it doesnt realy surprise me though.
By alphamone on Dec 5, 2008
I noticed the same thing as Phillip after I received presumably the same form letter from Senator Conroy and read the testing framework. I replied to Senator Conroy’s letter with a letter of my own asking him to answer my specific questions, including this one. Lets see if I get the same form letter in response again!
By Jennifer on Dec 5, 2008
Good work on your blog, worth reading every day.
Meanwhile of course, Conroy continues taking advice on how to govern from the wrong sources.
“The state must declare the child to be the most precious treasure of the people. As long as the government is perceived as working for the benefit of the children, the people will happily endure almost any curtailment of liberty and almost any deprivation” – Mein Kampf
By mld on Dec 5, 2008
@Molly
The pilot itself may have live users, but the 10,000 URL test is not using live users, only simulated users. This means that it won’t produce reliable numbers for a 10,000 URL test.
“Like there all very quick to report the 30% slower figure in the government tests, but not many point out that one of the products only had a 2% degradation?”
There is a product that has 2% degradation, but it also had the worse effectiveness rating. The Government uses the best effectiveness and best performance figures from two different filters and uses these to suggest that filtering can achieve both simultaneously which is just totally misleading. Assuming the Government wants an effective filter (and this is perhaps a big assumption), it is reasonable to quote the performance degradation that would result from using that filter.
Of course, if this is just about social control of the masses (and not the extremists), they may not actually care about effectiveness in the technical sense.
By Jon Seymour on Dec 6, 2008
JOn: Normally you choose a balance of Performance and accuracy. My understanding is that there will be rules put in place for the ISPs rather then a product dictated to them. Therefore smart ISPs will go for the quickest product that meets the effectiveness requirements.
Also when you say some where as accurate as each other, you are talking about the difference between blocking 88% to 94% of the sites it was meant to block and 3-8% blocking wrong sites. Not a huge difference.
Plus my understanding is that a lot of the slow down is caused by having to process a list of URLs/rules. Due to Moores law, this will get twice as quick and half as expensive each year!
JMTC
By Phillip Molly Malone on Dec 6, 2008
So Jon, are you going to fix the title of this post to be that some parts of the live filter pilot will not involve real people? I would think you interested in an intelligent debate on the subject, so getting the facts correct is all import, right?
By Phillip Molly Malone on Dec 6, 2008
I disagree Phillip. You cite a DBCDE document, Geordie Guy says he contacted DBCDE and they told him they were going to use simulated users. Why would the titled be changed? Seems to me, the document is wrong.
“talking about the difference between blocking 88% to 94% of the sites it was meant to block and 3-8% blocking wrong sites. Not a huge difference.”
Lets take a magic number, say, 1 million. Lets imagine that there are 1 million pages viewed daily by Australians. You are saying that 30,000-80,000 incorrect blocked pages DAILY is an ACCEPTABLE thing?
“Plus my understanding is that a lot of the slow down is caused by having to process a list of URLs/rules.”
Incorrect, some of the filter products that were tested in the trial showed a reduction of more than 30% in PASSIVE mode, where they weren’t even filtering content, just hooked up to the network.
“Due to Moores law, this will get twice as quick and half as expensive each year!”
And i imagine once it’s implemented, the list will double each year in number of URLs.
If you are interested where the numbers are comming from, go to the original source: http://www.acma.gov.au/webwr/_assets/main/lib310554/isp-level_internet_content_filtering_trial-report.pdf
Keep in mind that this trial was done with 30(yes that’s 30) users. ISP’s often have hundreds of thousands of users.
By Spikeles on Dec 6, 2008
Molly,
I can’t fix the title of the post, because it isn’t my post and it isn’t my blog.
jon.
By Jon Seymour on Dec 6, 2008
Another point is that the measured effectiveness figures are somewhat meaningless since they don’t take into account illegal content flowing across a Tor-overlay network. You could make the filter 99.9999% effective with 0.0001% performance degradation by the measures used there and it still wouldn’t block illegal content flowing over a Tor-overlay network through those very same filters.
By Jon Seymour on Dec 6, 2008
Well actually the effectiveness was only measuring what of the banned list was being blocked successfully and what was being blocked wrongly. Yes there are ways and means to get around this. My mother always said that locks only keep the honest people out. Same here, but just because there is ways around the filter is the poorest of arguments against it. I mean if we used that argument, we have to stop trying to stop drugs into the country, stop trying to catch drink drivers and speeding drivers, etc, etc, etc, to most laws of the country.
Sorry about the mistake about the post title, I saw on your blog you mentioned “your post” on here and missed that it was a guest post.
By Phillip Molly Malone on Dec 7, 2008
I would argue that it is a non-sequitur that because something is illegal it must be blocked by any and all available technical means, especially if there are costs in doing so (both social and fiscal).
Child porn is illegal. If there is hard evidence that vast numbers of people are accessing it via port 80, that hard evidence should be used to lock those people up – it is trivial to track an IP address back to the subscriber. If there isn’t hard evidence, why are we proposing to spend vast sums of money on an easily subverted mechanism. So that we can feel righteous? That is hardly responsible or effective social policy.
We should not create an all pervasive censorship mechanism that can be repurposed at will to block anything that the Government of the day deems politcally acceptable to block just because some people can feel a warm inner glow that we are doing something. Especially when that something is so utterly, completely ineffective.
This brings me back to the church choir analogy that I mentioned in a previous post. Church choirs (uncensored Internet) provide paedophilic priests with access to young boys (child porn). We don’t deal with this problem by eliminating all church choirs (uncensored Internet) – we target the problem – paedophilic priests (users of child porn). Attacking the medium doesn’t work when the medium is inherently resilient to censorship. You really have to attack the problem at the source instead of pretending that you can fix it with band aid technical solutions at the receiver. You won’t and every dollar you spend this way would be better spent on more effective law enforcement.
This policy is completely irrational unless one understands that the true objective is to moderate the porn consumption habits of middle Australia – not the perverts – middle Australia. It has absolultely nothing to do with eliminating access to “Refused Classification” material – that is just moral panic being used to corral people to accept the imposition of an otherwise politically unacceptable social control.
By Jon Seymour on Dec 7, 2008
On the Church Choir analogy, I would argue they have put a “filter” like mechanism in place. Its called the Working with Children law. Does the working with Children law/qualification/pass/check (whatever you want to call) stop all pedophiles getting access to kids? No. Is it a pain and does it slow down some good people (who have done absolutely nothing wrong) from working with kids? Sure does. But it is considered a worth while protection.
On the porn debate, its an interesting one but although some sites might be on the illegal list, I hardly think it is a big number considering the number of sites out there. Plus if they aren’t illegal, you can apply to have them removed from the illegal filter. If they are on the Opt-out filtering, just opt-out of the filter.
Who will ever know what is really in the heart of pollies but from my view of it, the “illegal stuff” part of the filter is the secondary part. The main part is the opt-out “Clean part of the filter”. I could be wrong, but thats the feeling that I get.
Anyway, I thought the government was doing it to block unfriendly comments about them and their policies? (my tongue is firmly in my cheek)
By Phillip Molly Malone on Dec 7, 2008
What’s legal in print becomes illegal online. Content that would otherwise be MA is about the most we’re ‘allowed’ to see online. I think the government is targeting the internet because it is way too hard to control. The ISPs are __NOT__ the distribution points of illegal content, it sits on independent servers all around the world, and the ISPs are being unfairly punished and forced to do a job they __CANNOT__ feasibly do. Enforcement is a much better option in this case, as a filter just covers these perverts up and shoves them underground, rather than putting them in prison and punishing them and actually getting them the hell away from potential victims.
By Stephen on Dec 7, 2008
I think if the government could catch them, they would! I don’t think this punishes the ISPs. I mean if they all have too do it, then I don’t see how its a punishment any more then the Movie industry is punished by having rating system.
By Phillip Molly Malone on Dec 7, 2008
“On the Church Choir analogy, I would argue they have put a “filter” like mechanism in place. Its called the Working with Children law.”
Which, if reflected back onto the other side of the analogy would be equivalent to denying access to the Internet to anyone who has not been certified as capable of using the Internet responsibly.
The Working With Children law is a sensible way to address a real problem. It even has some chance of being effective.
Onto the question of the mandatory nature of the filter. It still isn’t clear what material would be in the mandatory part of the filter. Conroy has talked of it being limited to material on the ACMA blacklist and to “prohibited” material and that the ACMA blacklist would be expanded to include other “unwanted and inappropriate” material.
The ACMA blacklist currently contains material that is defined as “prohibited, or potentially prohibited, online content”
Let’s see what is currently prohibited online content, as defined in Australian Law
http://www.acma.gov.au/WEB/STANDARD/1001/pc=PC_90102
Aside from the perverse stuff, there is X-rated depictions of consensual adults having actual sex, any R18+ material not behind an age verification test, all RC games, any paid or content not subject to an age verification test which is or would be classified MA15+.
That is a very large range of material that could potentially be blocked by the mandatory part of the filter. How does one effectively challenge a listing on the ACMA blacklist since the list is secret and any detailed knowledge of what’s on the list may be proof that a felony has been committed.
The proponents of the filter haven’t sufficiently justified why an opt-out rather than opt-in solution is required. The protecting the children arguments only imply that households who have children should have the filter, not the entire population. But restricting the filter like that would incur the wrath of working families so they have to extend it to everyone, even though not one additional child will be protected by doing that.
The Government clearly wants to assert the right to meddle with people’s Internet connections at will. I don’t claim that the Government knows how it wants to use this power yet, but once it has the technical power it will find ways to do so. It will never be total censorship, because that won’t work. But it will be about creating the sense that people are being watched, if they ask too many questions about the next atrocity in Palestine or Afghanistan, the next Haneef scandal or the next SIEV-X incident. The occasional HTTP 403 thrown up there will be a very handy way to quell political dissent, particularly amongst minority groups.
By Jon Seymour on Dec 7, 2008
Yay, you finally came to the Government are going to throw 403 to stop political debate. The baby is finally over board!!!
On your analogy, its all over the place!!!!!!
At one point your saying all priests are Child Porn! The you are saying that paedophilic priests are users of child porn! So yes, in your analogy that might be right, but I think I now realise that you analogy is totally broken!
Forgetting your analogy, I think the WWCL is like the filter in that you check the person before you let them work with kids.
Now, I don’t understand why making the filter opt-out is any great problem compared to opt-in as long as the result of Opting out is gets you to the same state as opting in!!! I will be one of the first people to opt-out. I find in the reading that I am doing of people against it have ago of the opt-out but normally they end up having ago that they can’t opt-out of the legal section.
On it, the government have already flagged that the number of sites to be blocked on the legal part of the filter will slow things down the more there are so i find it hard to believe that they are going to put things on the list just for the hell of it, because if they stuff with it enough, even the people that support the scheme will turn on them.
By Phillip Molly Malone on Dec 7, 2008
The analogy is consistent. In my analogy, pedophilic priests are like users of the uncensored internet who choose to use it to commit a crime. It was you who claimed at paedophilic priests would be subject to the filter, in effect making them equivalent to child porn. That was the leap you made, it was not part of my analogy. So really all you have done is shown that your critique of the analogy is all over the place.
By Jon Seymour on Dec 7, 2008
Opting-out leaves an individual user in the same state as opting-in. It does not leave the society as a whole in the same state.
Suppose 20% opt out. That leaves 80% subject to the filter. It is precisely these 80% that the filter will be most effective against if the Government ever decides that it wants to use its powers to quell dissent by preventing them straying to far away from mainstream sources of fact and opinion. I am not claiming that the mechanism will be used that way anywhere in the near term, but the existence of an opaque censorship mechanism enables those scenarios in the future. I happen to believe we should not be facilitating those future scenarios in the name of the present moral panic.
By Jon Seymour on Dec 7, 2008
To quote you from an earlier comment:
“This brings me back to the church choir analogy that I mentioned in a previous post. Church choirs (uncensored Internet) provide paedophilic priests with access to young boys (child porn).”
Correct me if I am wrong but you are saying that all priests (in your analogy) are Child Porn (in your analogy).
My point was if Priests are taken as people that access the choir (for some reason) and so are other people, then if you want access to the children on the Choir, you would have to go through the Working with children Check. Yes its not a perfect analogy but if you accept that your first Priests analogy to Child Porn and change that to being users of the internet, then I think your analogy holds up better and shows that we now have a filter for that analogy as well.
By Phillip Molly Malone on Dec 7, 2008
Ok, I see why you misinterpreted what I had written. I omitted one of the terms of the analogy. I should have written:
“Church choirs (uncensored Internet) provide paedophilic priests (deviant Internet users) with access to young boys (access to child porn)”.
The point is that church choirs provide a medium by which evildoers can commit unspeakable crimes. Similarly the Internet does the same. The solution for church choirs is to apply social controls to the actors, not technical controls to the medium.
By Jon Seymour on Dec 7, 2008
Geordie addresses the argument that Moore’s Law will rescue a filter from technological oblivion.
https://www.howtobeasystemsengineer.com/blog/?p=42
The quickest way to improve the performance of a filter-crippled ISP feed will be to opt-out of the filter.
By Jon Seymour on Dec 7, 2008
Isn’t the Working with Children Check a “technical controls to the medium”? I don’t think they are doing this in lieu of trying to catch perverts and I don’t think they are going to say that you no longer have to monitor your kids internet usage either. So the filter is the technical control and trying to catch them is social control.
I agree with your quickest way to improve the performance and I will opt-out day 1 (and have said this all along)!
The Moores law argument is interesting. I am not sure its accurate, but its interesting.
By Phillip Molly Malone on Dec 7, 2008
Molly,
You say “I don’t think they are doing this in lieu of trying to catch perverts and I don’t think they are going to say that you no longer have to monitor your kids internet usage either.”
The OCSET team that actually catch perverts for the AFP has had expansion put off for another year and their budget slashed by about 3 million bucks. Parents who are told that their children are safe online with a filter that doesn’t make them safe will assume it is. Sorry champ, you’re 0/2.
By Geordie on Dec 7, 2008
Who’s telling them that they are “Safe online with a filter”? Sure there not saying safer? The Government seems to be saying that its a multi-facetited approach: http://www.dbcde.gov.au/communications_for_consumers/funding_programs__and__support/cyber-safety_plan
By Phillip Molly Malone on Dec 7, 2008
Clive Hamilton has argued that parents don’t want to be looking over their children’s shoulder because this will “reduce their (children’s) sense of responsibility and freedom”. In stating this he is effectively advocating the abdication of the exercise of parental responsibility in favour a Government mandated ISP filter. This is irresponsible and will only place children at greater risk.
Either the filter relieves parents of the responsibility to monitor their childrens internet access or it doesn’t. If it doesn’t want is the real advantage of having it?
By Jon Seymour on Dec 7, 2008
Not that this is exactly central to the topic at hand, but in the interests of accuracy… most of the Hitler quote cited by mld is bogus.
The author appears to be Rabbi Daniel Lapin, who wrote an imaginative article called ‘Hitler writes from the Grave’ in 2004. URL below. If anyone can prove otherwise please do.
http://www.worldnetdaily.com/news/article.asp?ARTICLE_ID=36544
By Syd Walker on Dec 7, 2008
Syd
*sigh* Don’t buy anything Derspatz says…The quote is from a book called Mein Kampf. A Book that Hitler wrote that book before Daniel Lapin was born.
“The state must declare the child to be the most precious treasure of the people. As long as the government is perceived as working for the benefit of the children, the people will happily endure almost any curtailment of liberty and almost any deprivation” – Mein Kampf
By stwa on Dec 8, 2008
Syd, are you aware that the article you link to quotes and give page references to Mein Kampf, Effectively contradicting the very argument you propose.
By Stevian on Dec 8, 2008
Hmmm. This is all very OT.
I raised the issue of the bogus Hitler quote as an incidental correction. Anyhow, I’ve written an article about it, which IMO is an interesting case of a contemporary ‘internet legend’. It’s HERE for those who care.
On a lighter note, Conroy Junior’s End of Term School Report has arrived.
It’s a shocker!
By Syd Walker on Dec 8, 2008
Conroy’s office still insisting today there will be a live trial involving ISPs filtering their customers’ Internet use.
Are there in fact two trials running simulataneously?
Is he ordering a live trial involving the current ACMA blacklist and a closed trial of the hypothetical 10,000 ‘bad’ sites?
I’m totally confused.
By clarencegirl on Dec 17, 2008