What the hell is metadata and mandatory data retention, anyway?

With all of the talk about ‘metadata’ data retention schemes for internet history, we need a straightforward and easy way to understand what metadata is.


Data’s pretty straightforward, let’s look at it in the scope of a web request. Say you’re logging into facebook and you look at a picture that a friend has updated. The content of that picture is the data. The digital contents of the file that mean the file can be displayed on any computer which supports decoding that file is the data itself.

Metadata is the data that’s generated when you view data, or when you access data.


So when you visited that facebook page you probably hit a URL (the addressing system for system on the internet, for the most part). The metadata created was that you hit that URL, at that time, who the person was that queried that URL (your IP address, which your ISP can easily correlate with who you physically are), and any other information, such as the size of the request (which could be used to intuit the kind of information transferred). So in terms of a basic facebook request we’ve now got:

  • The time of day the request was made
  • basically who made the request
  • How big the request was
  • Any other resources that were related to that request (such as files that tell your browser [the thing you’re using right now] how to display the page you’re looking at).

This is a basic example of the kinds of metadata that can be generated.

What can just the collected metadata let us discover

Let’s jump straight to a more ribald example. Let’s say you’re unwinding and relaxing and want to watch some pornography. You browse to your favourite website, do your thing and then afterwards decide to watch some funny cat videos. With the metadata, any individual with access to that info would basically be able to figure out how long you choked the chicken or flicked the bean and what to. So if you’re into stuff that is your own kind of kink, just the data about the data gives whoever is reading that metadata log (be they an ASIO officer, RSPCA officer, someone from the local council or a russian or chinese hacker) information about your go to woe time and the kind of shit you’re into.

Okay so how can the fact that I look at porn be used to fight terrorism

First off, it’s a furphy to say that data retention can fight terrorism. Data retention would provide a mechanism for security services or police to access your browsing history and use it to prove a case against you for a particular crime you are accused of. Without data retention there are still, surprisingly, people who are accused of and convicted of crimes and who are dealt with accordingly by the justice system. People who have been planning (or attempting to plan) terrorist attacks on Australian soil have been detected and caught through old fashioned policework. This data retention information provides another source of evidence for police to build a tighter case. Data retention in and of itself does nothing to ‘fight’ terrorism.

But wait a minute, if they have all that data, won’t they be able to see what terrorists do and get them before they commit a crime

That’s the rolled-gold claim of the software vendors and the security hawks who peddle these pieces of software. Through a few complex algorithms and some number crunching we can detect criminal intent before it actually happens. The thing is the technology to correlate between actions and intent has been around for a while, and if you’ve been weirded out when google ads track you around the internet and show you ads for things you’ve searched for, you’ve seen this technology in action. When you’ve then been doubly confused that google is showing you an ad for something completely irrelevant to you or you don’t need, you’ve found something even more interesting, a misfiring of the algorithm or an ‘overfit’. This is where stuff gets very interesting and also very, very scary. The government (or, let’s be real, Palantir or whatever other contractor manages and performs dredging on this vast dataset) could set certain red flags or websites that when visited, trigger something to happen. That trigger could be to log more information about the request, it could be to look for more requests like it, it could be to add flags to a person’s file.

So what, shouldn’t we be keeping an eye on terrorist websites?

It depends on what you’re trying to do, if you’re trying to catch people that are either researching terrorism or documenting terrorist websites then sure. If you’re trying to combat terror, this mechanism is only going to work once. Terrorists are engaged in what is known as ‘asymmetric warfare’, they don’t play by the existing rules which is why it’s particularly difficult to combat them and sniff them out. The war on terror has also shown they’re incredibly capable of adapting to whatever we throw at them. If data retention was purely about combating terror, then we wouldn’t have had the broader slip by Tony Abbott today (6/8/14) saying that retained data would also be used for other ‘law enforcement purposes’. These ‘law enforcement purposes’ are already incredibly broad, and not constrained. As this article in the telegraph shows, more than half of UK councils are using extraordinary powers under anti-terror laws to spy on people who… don’t use their bins correctly.

Okay but that’s an extreme and ridiculous example, with oversight this stuff can be used correctly

Systems aren’t perfect and people certainly aren’t perfect, as someone who has worked with software systems in secure settings for years, stuff slips through the gaps. The good thing is we don’t have to deal with the case that it’s an impenetrable system with limited access to a few people, Abbott himself said it would be used for other forms of criminal investigations, so there are going to be multiple points of access (or multiple tiers of access) to the system. With those multiple tiers come multiple points of failure that could be abused, leading to massive privacy breaches.

But what about measures needed to keep our country safe?

As of the 6th of August, 2014, there’s only ever been one fatal attack classified as a ‘terrorist attack’ on Australian soil. That was the hilton hotel bombings. That attack also happened in 1978. As addressed earlier, data retention will actually do nothing to keep Australians safe and in many instances the false positives created by systems scanning for ‘behaviour patterns’ will waste the time and energy of the security services and potentially allow people to slip through the cracks. There are obviously two broader concerns here, the first is do we want to live in a society where such mass warrantless surveillance of our citizenry is a mundane fact of everyday life? Do we trust the government (or really, whomever they contract in to handle this) to securely manage our browsing histories [keep in mind it’s not just browsing, it’s everything that uses the internet, but that’s a point for another day] and make sure they’re only ever used ethically and when absolutely justified? The thing is, we already have a system with stringent checks and balances that works to protect privacy and make sure data is being used legitimately, they’re called warrants and the police, ASIO, ASIS and others use them every single day. Again, data retention is not about fighting terror, data retention is about something much bigger. Data retention is step one towards substantial internet control by the government. With legislation mandating that ISPs put in place infrastructure to snoop on and retain internet traffic, it’s trivial to say, hand over the data of those engaging in alleged piracy to copyright agencies to institute a three strikes system. With a system in place to monitor and record internet traffic, it’s trivial to institute a blacklist or a whitelist carte-blanche internet filter. The data retention play isn’t about data retention, it’s about the government wresting a greater degree of control over how we use the internet and putting into place a system of mass surveillance that will almost certainly be misused to the detriment of many Australian citizens. But there’s one final thing that makes the entire data retention play a complete furphy

How easy is it to bypass metadata collection under a data retention regime?

It’ll take you less than five minutes. Go to a website like easyvpn or strongvpn, sign up for a VPN service and follow their super simple instructions to route all of your traffic across an encrypted channel that can’t be snooped on. The metadata the government will see if you put all of your traffic down an encrypted pipe? The size of what you’re transferring (maybe, it depends on what kind of inspection they do) and that you’re connecting to a VPN. So unless they want to make using a VPN service a crime (which again, is probably feasible), the entire data retention regime is easily defeated by johnny jihad who can get back to plotting his war against the great satan. The best thing about those VPN services? They don’t keep logs, so when the cops come calling after a month long process to compel access to the information, there’s nothing to hand over. This point in and of itself completely explains why data retention is an absolute farce and is in no way a deterrent to terrorism.

Related posts
FeelpinionsHome & LivingLifestyleNOM NOM NOMParty

16th Birthday Present and Party Ideas for Him and Her/Boys and Girls

Your son, daughter, family member or friend’s 16th birthday is certainly one worth celebrating…
Read more
BusinessFeelpinionsHome & LivingLifestyleTech

How does a push-button work? Where are push buttons used? Is a push button and switch the same?

Have you ever wondered how a push-button actually works? You press a button and something happens…
Read more
FeelpinionsLifestyleNOM NOM NOMTech

How much does office cleaning cost on average in Australia? How much do cleaners charge per hour? What can a cleaner do in 2 hours?

Keeping the work environment clean and clutter-free is one of the most underrated aspects of running…
Read more
Become a Trendsetter
Sign up for Davenport’s Daily Digest and get the best of Davenport, tailored for you.