19 Jul 2016

Big Data and open source intelligence – A game-changer for counter-terrorism

Andrew Staniforth

The threat from terrorism continues to present a clear and present danger to many communitiesacross the world. The suicide bombingsin Turkey on 28th June which killed 41 people at the Ataturk international airport in Istanbul, and the series of deadly bombings in Baghdad and Iraq during July, the string of attacks in Saudi Arabia recently, provides further evidence that more must be done to identify the early signs and weak signals of attack planning to prevent further terrorist atrocities.

Bridging the gap

Since before the advent of advanced technological means of gathering information, Law Enforcement Agencies (LEAs)have planned, prepared, collected, and produced intelligence from publicly available information and open sources to gain knowledge and understanding in support of preventing terrorism and pursuing terrorists.In today’s interconnected and interdependent world, contemporary terrorism increasingly involves cross-border and trans-national dimensions, being amplified by the Internet, online social networks and smarter mobile communications. Social and technical innovations are now occurring at an ever-increasing speed, causing fast and drastic changes to society. These changes, driven by the possibilities offered by new and emerging technologies, have been embraced by terrorist groups to support their recruitment, propaganda and attack planning. To counter the terrorist use of new technologies, LEAshave been forced to keep up and to do so, have tapped into an increasingly rich source of information that is gathered from publicly available information known as Open Source Intelligence (OSINT).

LEAs have come to learn that if they are to have any kind of impact upon combatting terrorism, it is essential for their efforts to be increasingly intelligence-led. At its simplest, ‘intelligence-led’ conveys the relatively obvious notion that LEA activity – whether focused upon local community aspects of policing, the investigation of crime or public order – should be informed and directed rather than undertaken randomly.[i] An intelligence-led approach to LEA activity also serves to support the evidence-gathering of police investigators and detectives to achieve successful prosecutions.[ii]For the LEA practitioner, and especially those engaged in specialist intelligence-gathering and intelligence development roles, intelligence is not so much a way of working as a way of thinking. Such is the importance of intelligence to LEAs success is that it is has become a holistic discipline to which the whole of an LEA contributes and in which all officers and staff have a role to play directly or indirectly.[iii]  An intelligence-led approach to policing and security is therefore not just something that the intelligence unit or the proactive investigation unit within an LEA undertakes – intelligence-led policing demands that the whole organisation undertakes or supports this vital function.

Command and control

OSINT is being extensively used by local and national LEAs, intelligence agencies and the military.  Given the scale, accessibility and high yield of intelligence return for minimum resource, OSINT compliments and increasingly corroborates and confirms other traditional LEA intelligence functions, all of which are relevant operational reasons as to why OSINT has quickly become a rich source of information to disrupt and detect the modern terrorist. OSINT is the intelligence discipline that relates to intelligence produced from ‘publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence and information requirement’. OSINT is also intelligence developed from the overt collection and analysis of publicly available and open-source information which is not under the direct command or control of government authorities and is derived from the systematic collection, processing and analysis of publicly availableinformation in response to intelligence requirements.

Unlike other intelligence collection disciplines such as Human Intelligence (HUMINT) or Signals Intelligence (SIGNIT), OSINT is not the sole responsibility of any one single government agency but instead is collected by the entire LEA community. One advantage of OSINT is its accessibility, although the sheer amount of available information can make it difficult to know what is of value. Determining the data’s source and its reliability can also be complicated and problematic. OSINT data therefore still requires review and analysis to be of use to LEA decision-makers and must be appropriately processed alongside intelligence gathered from other collection methods.

Big Data Analytics

An important aspect of LEAs use of OSINT is social media, which represents an increasing and fundamental part of the online environment in which the users are authors of the content who do not passively receive information, but they create, reshape and share it.  In some cases, the interaction among users based on social media creates communities and virtual worlds providing an excellent source of information for LEAs. Although there are significant differences in the nature of these outputs, two aspects are always present and are relevant to the work of LEAs: large amounts of information and user-generated content. The social media platforms aggregate huge amounts of data generated by users which are in many cases identified or identifiable.  When combined with other online and stand-alone datasets, this contributes to create a peculiar technological landscape in which the predictive ability that is Big Data Analytics (BDA) has relevant impact for the implementation of social surveillance systems.

Big Data is an evolving term that describes any voluminous amount of structured, semi-structured and unstructured data that has the potential to be mined for information.Big Data can be characterized by 3Vs: the extreme volume of data, the wide variety of types of data and the velocity at which the data must be processed. Big Data is nothing new but it is currently at the final stage of a long evolution of the capability to analyse data using computer resources which forLEAs is an excellent opportunity to tackle terror and keep communities safe.  Big Data represents the convergence of different existing technologies that permits enormous data-centres to be built, creating high-speed electronic highways and having ubiquitous and on-demand network access to computing resources, which are more commonly referred to as ‘cloud computing’.[iv] These technologies offer substantially unlimited storage, allow the transfer of huge amounts of data from one place to another, and allow the same data to be spread in different places and re-aggregated in a matter of seconds. All of these resources permit a large amount of information from different sources to be collected and the petabytes of data generated by social media represents the ideal context in which BDA can be used.  The whole dataset can be continuously monitored by analytics in order to identify emerging trends in the flows of data and obtaining real-time or nearly real time results in a way that is revolutionary. Within the context of counter-terrorism, the availability of these new technologies and large datasets provides a competitive advantage, representing the greatest opportunity to increase the effective delivery of counter-terrorism.

Big Data can help the identification of terrorist networks and their associations using OSINT and provide valuable corroboration of other intelligence sources to support the holistic development of intelligence. It can also rapidly support the identification of radical roots within online communities providing significantly increased capabilities and opportunities not just to prevent terrorist attacks, but to identify attack planning activity and most importantly, spot the early signs and signals of radicalization and recruitment to stop violent and extremist development at source – this is a game-changer for counter-terrorism.

Future challenges

The world is being reinvented by open sources. Publicly available information can be used by a variety of individuals and organisations to expand a broad spectrum of objectives and LEAs are increasingly making effective use of this free and accessible source of information.While the internet and online social networks have positively enriched societal communications and economic opportunities, these technological advancements have changed – and continue to change – the very nature of terrorism, serving to breed a new sophisticated and technically capable terrorist.

Significant changes in counter-terrorism practices have also been made. Counter-terrorism is no longer the hidden dimension of statecraft. It has over recent years moved out of the shadows due in part to the understanding by LEAs and intelligence agencies that not all counter-terrorism measures need to be cloaked in secrecy in order for them to be effective.Harnessing the power of OSINT via BDA capabilities presents a unique opportunity for governments to address the increasing threats from international terrorism at relatively low cost. But the handling of such large data-sets raises acute concerns for existing storage capacity, together with the ability to share and analyse large volumes of data.  LEAs must also ensure that their access and use of publicly availableinformation is within national and international legal frameworks which provides legitimacy and confidence that open source information is being used appropriately. The accessibility of OSINT and the introduction of BDA capabilities will also require the rigorous review and potential overhaul of existing intelligence models and associated processes to ensure all in authority are ready to exploit BDAof OSINT.

LEAs and intelligence agencies will continue to prevent further terrorist atrocities, but there is a very strong likelihood that they will not stop them all.  In the light of that conclusion, all in authority must dedicate themselves to increasing counter-terrorism capabilities and developing new approaches to better protect the public.  To ignore or dismiss the positive benefits of OSINT would be both misplaced and unwise as all citizens expect the LEAs and intelligence agencies of their governments to take the necessary steps to keep them safe. Harnessing the power of OSINT via BDA continues to be a game-changer for counter-terrorism policy-makers, professionals and practitioners. Given the scale and complexity of the threats from international terrorism, intelligence agencies must continue to advance counter-terrorism measures to keep us all safe; and most importantly, seek new ways in which to embed progressive developments to ensure that the primary driver for change in counter-terrorism practice is not simply the next terrorist atrocity.

[i] Harfield C & Harfield K, Intelligence: Investigation, Community and Partnership (2008) Oxford: Oxford University Press

[ii] Ratcliffe, J Intelligence-Led Policing, (2008) Devon: Willan Publishing

[iii] Harfield C & Harfield K, Intelligence: Investigation, Community and Partnership (2008) Oxford: Oxford University Press

[iv] Akhgar, B  Bayerl, S Saathoff, G  Application of Big Data for National Security, A Practitioner’s Guide to Emerging Technologies London:Elsevier 2015.

How useful was this post?

Click on a star to rate it!

Average rating 2 / 5. Vote count: 1

No votes so far! Be the first to rate this post.


Your email address will not be published.