In recent days there have been claims that there has been both more and less spying by New Zealand intelligence agencies. Proponents and opponents of the intelligence community have seized on one or the other claim to argue in favour or against NZ’s involvement in the 5 Eyes signals intelligence network and the expansion of powers awarded the NZ intelligence community under amendments to various security Acts during the past few years. Given that there is a forthcoming parliamentary review of the NZ intelligence community, it is worth cutting to the gist of the issue of “balance” between civil liberties and intelligence operations.

Monitoring and intercept technologies available to signals and technical intelligence agencies today are superior to those of ten years ago, especially in the field of telecommunications. This allows signals and technical intelligence agencies to do much more than was possible before, something that legal frameworks governing signals and technical intelligence collection have had difficulty keeping pace with. It would therefore seemingly defy credulity to claim that that spy agencies are doing less spying now than in the past, especially given what is known about the 5 Eyes network from the Snowden documents currently being introduced into the public domain.

But perhaps there is a way to reconcile the opposing claims. Can spy agencies actually be doing less with more?

The assertion that there is less spying by NZ intelligence agencies now than seven years ago can be reconciled with the recently released GCSB annual report stating otherwise by understanding that under the intelligence community’s interpretation, “mass collection” is not equivalent to “mass surveillance.” Although the 5 Eyes and other national signals intelligence agencies use systems like PRISM to grab as much meta-data as possible as it passes through nodal points, that data has to be mined using systems like XKEYSCORE to obtain collectable information. Bulk “hovering” of all telecommunications in specific geographic or subject areas by agencies like the GCSB still has to be searched and analysed for it to become actionable intelligence. That is where the use of key words and phrases comes in, and these are not just of the usual “jihad” or “al-Qaeda” variety (since the bulk of intelligence collection is not focused on terrorism).

Although the GCSB may be doing more bulk collection of electronic data, it claims to be analysing proportionately less of what is collected than during the last year of the Fifth Labour government. So it is doing less with more. But a fundamental problem remains when it comes to intercepting telecommunications in democracies.

That problem is that whether it is analysed or not, mass collection of so-called meta-data of everyone’s personal and professional telecommunications presumably violates the democratic right to privacy as well as the presumption of innocence because it is obtained without there being a particular suspicion or specific reason for its collection (much less a warrant for its collection). Bulk intercepts can then be data-mined after the fact using classified search vehicles in order to build a case against individuals or groups.

That runs against basic tenets of democratic jurisprudence. Moreover, indefinite storing of meta-data that has not been analysed but which could be in the future in the event target (and key word) priorities change is something that is the subject of legal argument at this very moment.

There are therefore fundamental principles of democratic governance at stake in the very collection of meta-data, and these cannot be easily set aside just because the threat of terrorism is used as a justification. The issue is constitutional and needs to be resolved before the issue of “balance” can effectively be addressed.

However, for the sake of argument let’s accept that bulk collection is not mass surveillance and that the former is legal. How does one balance civil liberties and security under such circumstances?

The implementation of balance under such conditions starts at the point where data mining begins. What are the key phrases and words that identify targets for closer scrutiny? What are legitimate targets and what are not? Some search terms may be easy to understand and broadly accepted as necessary filters for the acquisition of more precise information about threats. Others might be more controversial and not widely accepted (say, “opposition leader sex life” or “anti-TPPA protest leaders”).

That is where the issue of effective intelligence oversight comes into play and on that score NZ is sorely wanting. There have been some cosmetic changes in the workings of and a slight extension of the powers of the Inspector General of Intelligence and Security, and the process of issuing domestic security warrants made more robust with the participation of the Commissioner of Security Warrants. Yet any honest assessment of the oversight mechanisms of the NZ intelligence community will show that they are inadequate when it comes to providing effective and transparent proactive as well as retroactive oversight and review of our intelligence community’s activities given the range and scope of the latter.

These mechanisms are fewer and less effective than those of most liberal democracies (including our 5 Eyes partners), which means that NZ’s intelligence partners may well ask it to do things that they cannot do themselves due to the restrictions imposed by their own oversight mechanisms. That possibility should be of concern and needs to be addressed. Relying on the good faith of NZ intelligence agencies involved is not enough, especially given their history of playing loose with the rules when it suits them.

Therein lies the core problem with regard to balancing civil liberties and intelligence operations. If there is effective intelligence oversight before the fact (“proactive” in the sense that oversight mechanisms dictate was is permissible data-mining before it occurs) as well as after the fact (“retroactive” in the sense that oversight mechanisms hold intelligence officials to account for their use of bulk collection and data-mining), then balance can be achieved. However, if such effective oversight is lacking–again, both proactive and retroactive in nature–then the “balance” will be skewed heavily in favour of unaccountable intelligence collection and usage. That is not acceptable in a democracy but is in fact the situation at present in New Zealand.

Then there are the issues of how national security is defined and what role intelligence agencies play in its defense, on whose behalf NZ intelligence agencies engage in espionage, and with who the intelligence obtained by human, signals and technical means is shared. This matters because trying to achieve balance between civil liberties and intelligence operations without addressing the larger context in which the latter occur is much like putting the cart before the horse.