Tracking Russian Online Influence Operations and Possible Points of Disruption with Big Data Cross-Lingual Text Analysis

Transcription

1 Tracking Russian Online Influence Operations and Possible Points of Disruption with Big Data Cross-Lingual Text Analysis Kenneth Hacker, Peter Chew, Ahmed Abdelali Intelligence in the Non-English Speaking World Annual National Security Colloquium University of Texas, El Paso, National Security Studies Institute June, 20, 2018 In this analysis, we seek to expose some empirically identifiable patterns of Russian political warfare conducted online. This effort is done with big data analysis and automated text analysis. The analysis is presented in the context of published United States government (USG) and European ally concerns with Russian online information warfare. There is a great deal of news and political controversy today about Russian influence in American political communication, election campaign venues, and cyber-spatial aspects of national security operations. In totality, various indicators leave little room to doubt that that Russia is attacking the United States in what is called the information space. Clearly, the American military is aware of this and is using its own Information Operations (IO) machinery to protect American assets from Russian cyber invasion and interference. However, the Russian information warfare (IW) invading American news, social media, election campaigns, and general political communication appears to be much less recognized and less countered. The prevalence of Russian information operations and online information warfare has increased in the past 4 years ( ). The timing of this is no coincidence, as Russian general Valery Gerasimov wrote in 2013 about the necessity for Russia to learn the lessons of the Arab Spring (in which social media was used to mobilize protests), and engage in this kind of activity (Gerasimov, 2013), in what is widely referred to as the Gerasimov Doctrine. Russia, for many years, has sought promote its political narratives domestically and externally. Underpinning its efforts in cyber warfare and information warfare is its faith in information dominance (Jaitner & Mattsson, 2015). Information superiority is achieved with kinetic, information-based, and cyber weapons. While it is tempting to separate war from communication, the two are intertwined throughout history. Information and communication can be used in ways that make adversaries easier to defeat kinetically (Jaitner & Mattsson, 2015). As Sun Tzu, said, All warfare is based on deception. Current military doctrines address the need to gain footholds in cognitive space as well as physical space. 1

2 Russian national security strategy documents state that global information struggles are 1 increasing (Gerasimov 2013, Jaitner & Mattsson, 2015). Recent news indicates that Russia has exploited Facebook and other social media sites for spreading political disinformation and causing division among American citizens (Sheth, 2017). Phony Facebook accounts were created such as a supposed California organization known as United Muslims of America which seeded anti-american feelings among American Muslims (Sheth, 2017). Russian trolls used Facebook to push out false stories, for example that Hillary Clinton admitted that the United States created and trained Al Qaeda, or that John McCain was the founder of Daesh (Sheth, 2017). Twitter and Instagram can also be used to distribute dividing messages. The report released by the Office of the Director of National Intelligence in January 2017 stated that one of the goals that the Russians had was helping the Trump election in 2016, a conclusion which is borne out by what leading Russians such as Dmitry Kiselev and Aleksandr Dugin have 2 openly said. Like the Chinese, the Russians are focused on information and communication spaces as the new battlefield. Ironically, Trump was helped in this battle space to get elected, but now is under attack by the Russians who helped get him elected. While the interests of the Trump campaign and Russia may have aligned temporarily (due to Russian disdain for Hillary Clinton dating back to her questioning the legitimacy of the 2012 Russian presidential election when she was Secretary of State), much evidence points to longer-term Russian interests being simply to destabilize the American political system rather than to support any particular candidate, opening up the possibility that Trump was played in the short term by Russia. This would be consistent with the paradigm of the useful idiot, a phrase attributed to Lenin: a phrase which denoted an individual in the West who could be unwittingly used to advance the Soviet agenda. Since the 2016 US general election, Russia has used online influence operations to oppose U.S. ally, Syria and its leader, Bashar Assad (Sheth, 2017). Russia uses online messages that are targeted at specific communities. Those communities are linked with certain issues (Sheth, 2017). It is justifiable to say that Russia is doing these things, because Russian trolls doing all of this disinformation distribution are not lone wolves, but rather workers paid by the Kremlin (Sheth, 2017).The dezinformatsiya they create and spread is based on tactics rooted in Russian Cold War propaganda (Sheth, 2017). One easy tactic is planting political messages in non-political communication contexts. 1 Among the Russian population, there may be a perception that IW is defensive while other nations like the United States, NATO, and EU use IW aggressively. Similarly, some Russians may view Russia s efforts not as propaganda but as counter-propaganda and truth telling, and Russia itself as a target of Western IW (Jaitner & Mattsson, 2015). However, the view among leading Russians like Valery Gerasimov (ibid), Aleksandr Dugin, and others is quite clear: IW is to be used aggressively (Gerasimov, 2013, Dugin 2016), and IW consists of manipulation of the truth to serve Russian national interests (Dugin, 2016), in a way which would be consistent with classic Soviet dezinformatsiya (disinformation)

3 Russia s Full Spectrum Information Warfare Information operations (IO) is closely related to the concept of information warfare. Political and information warfare can occur at any level of political communication while IO is usually done only by military and intelligence agencies. IO is both informational and kinetic even if done in different domains. The National Security Strategy 2020 of Russia states that global information struggles will continue. Russia argues that it is being defensive. A simple narrative guides its IW (Jaitner & Mattsson, 2015). A key Russian narrative is that nations like the United States consistently conduct IO on other nations (Jaitner & Mattsson, 2015). This allegedly includes anti-messages and the instigation of the Euromaidan (anti-russian) uprising in Ukraine. The thrust of this narrative is essentially why shouldn t we do the same. Ironically, together with Russian narratives about US aggression, this serves to undermine the argument that Russia s actions are purely defensive. Russian military thinking and doctrine have long included information warfare. Some of this thinking divides historical Russia from the West as adversaries like the United States. This rift includes both ideological and cultural. The West is accused of being the antagonist of the historical Russian world. (Jaitner & Mattsson, 2015, p 41). The full-spectrum nature of Russian IW includes old media, new media, and various forms of cyber propaganda and IO. Overt and covert communication are all part of unified strategic communication. RIA Novosti is one media channel to other nations. Russia Today (RT), which is more well known, also does propaganda outreach. RT is more targeted to external audiences and is integrated with social media (Jaitner & Mattsson, 2015). Another major propaganda outlet is Sputnik. Some Russians claim that Sputnik was designed to disseminate counter-propaganda (Jaitner & Mattsson, 2015). With their social media channels, Russians use the term Runet to describe their overall online presence or interconnectedness (Jaitner & Mattsson, 2015). Runet is a worldwide network of nodes that are dedicated to Russian IW. Some content is produced within the nation and some by Russian speakers living abroad (Jaitner & Mattsson, 2015). Many in the Russian leadership fear the part of the information and communication space that works against them and which they cannot dominate. They know that since the early part of the 21st century, blogs, forums and other spaces online were used in demonstrations against Russian leader Vladimir Putin. Pro-Putin forces then used online communication to discredit the opposition (Jaitner & Mattsson, 2015). This started the emergence of Russian government trolls. Advertising was run by a Russian company seeking to hire Internet operators. (Jaitner & Mattsson, 2015). 3

4 Empirical Analysis of Russian IW/IO What is the currently identifiable online Russian information warfare reach and apparatus? The study and data analysis here reported examine online Twitter data. We employ big data and automated text analysis to identify matches between Russian political discourse and discourse that is allegedly not Russian, as American online messages. Discursive anchors and resonance terms are located in both Russian and English. The data analysis conducted here is multilingual and goes beyond looking for messages, themes, or frames in Russian discourse. The Twitter messages that are analyzed from from government sources (websites). The Tweets are general messages. The cross-lingual computational linguistics analysis shows both the Russian and English semantic spaces. Content similarities can then be examined. For example, the term democracy is likely to have differing semantic contexts and spaces in the two languages. The goals of the data analysis reported here are: 1. Show signals in online discourse are traceable by identifying differences between Russian and English discourse in Twitter messages. 2. Indicate some ways in which big data analysis can be useful in neutralizing some aspects of online Russian influence operations. 3. Identify what Twitters posts in our data correspond with Russian narratives emanating from the Kremlin. 4. Show what clusters of word co-occurrences indicate key similarities in online posts. 5. Demonstrate how data analytics can expose successful Russian online propaganda/influence by showing what messages or posts appear to be rapidly disseminated or going viral. 6. Show how unbiased data analytics can aid counter information warfare. 7. Display what statistical prevalence and significance exists in certain data patterns related to influencing posting in the Twitter posts. 8. Explain the importance of human coding and bias as an additional component of a big data analytics approach to studying influence operations online. 9. Argue how big data analysis can have a specific role in clarifying how Russian information warfares uses multiple strategies to affect political discourse. Our analysis also includes these focal points: 1. How propaganda spreads in social media like Twitter. 2. The ways in which Russian online influence operations appear to match the principles of information warfare articulated by Russian military leader, Valery Gerasimov (Gerasimov, 2013). 3. How propaganda can be observed in emerging propaganda vectors. 4. What topics in Twitter Tweets in our sample are skewed toward Russian or English language discussions. 4

5 5. How differences in Russian and English information spaces can indicate the movement of of propaganda from the former to the latter. 6. Which topics take the most hold in either Russian or English information spaces. Results of the Study In the study reported here, we explored how unsupervised analytics can be used in two ways to gain a top-down understanding of a previously unseen corpus of social media data, yielding useful insights to an analyst. Motivating this study is the premise that unsupervised techniques, such as those that underlie Google s search engine, automated translation, or Amazon s recommender systems (recommending products a purchaser like based on inter-account similarities of purchasing patterns), have attained a high level of commercial success and usefulness with big data. With Google, for example, the world wide web is constantly changing, yet Google s algorithms continue to work regardless of the language data is in or the overall quantity of data on the web. The question is raised therefore as to whether national security problems can be framed in such a way as to make them amenable to similar kinds of data-driven, unsupervised techniques that require a minimum of subject-specific tuning and recalibration. We believe that the strength of unsupervised techniques boils down to something very simple: they are good at finding groups of things that are similar (clustering), or individual things that stand out (anomaly detection). In Chew (2017), researchers set out a method by which unsupervised techniques can be used to look in a top-down fashion at how discourse in one language as a whole differs from that in another language. Such differences could result simply from differing parochial interests (say, there is more discussion of a Moscow metro shutdown in Russian than in English), or they could result from differing sentiment (e.g. negative sentiment towards NATO among Russian speakers versus positive sentiment among English speakers), or even differing narratives about the same entities. This is, in a sense, an attempt to reformulate other attempts to making sense of social media, which have to rely on supervised learning or labor-intensive tuning, in a way which plays to the strengths of unsupervised analysis. The approach, essentially, works as follows. A cross-language topic analysis is run on the corpus as a whole, using multilingual Latent Semantic Analysis (LSA). How this works is outlined in Chew (2017). Each topic s overall strength in one or another language can then be measured in aggregate (see figure below). 5

6 The topics do not need to be known in advance; they simply emerge with LSA, in the same way that Facebook can currently show a list of trending topics at any point in time. Topics cohere across languages, so if topic on say, NATO, is discussed in both Russian and English, albeit more in one language than in the other, this topic would show up in a single row in the table on the right of the above figure. This list of topics can then quickly be inspected to determine which topics are most skewed towards one rather than another language; these are the cases where the narratives, sentiment, or interests most differ. Note that because all topics emerge from the same integrated framework (in mathematical terms, the same signal processing decomposition of the original corpus), the skewedness of topics is comparable from one topic to the next. We should note that this method was inspired, in fact, by Valery Gerasimov s work (Gerasimov 2013), and other work which discusses the Gerasimov doctrine, e.g. Darczewska 2014). Gerasimov talks of the need for Russia to operate in the information space, and LSA gives us a natural way to measure, without bias, what is going on in an information space, and even to characterize mathematically what an information space is, when bounded by one or another language, e.g. Russian. Following 2014, the Gerasimov doctrine has, as we have already noted, been fully in evidence in the way in which Russia has sought to saturate, first and foremost, its own information space - that is, the space in which Russian is used, for example on Russian-language TV - with information favorable to the Kremlin s narratives. In this sense, we hypothesize that this kind of influence should lead to objectively measurable results in terms of 6

7 differences that will emerge between Russian-language media, and those media in other languages which are less subject to Russian influence. In a dataset of Twitter data from early 2016, for example, this kind of analysis revealed an interesting topic on the NATO Global Strike capability. Global Strike is envisioned by NATO as a defensive capability with the goal of being able to strike back at an adversary threatening a NATO country anywhere in the globe within a certain amount of time. In our dataset and the associated analysis, the analysis of information space differences revealed that this was one of the topics most skewed towards the Russian information space. A top-down analysis does not necessarily in itself provide answers ; what it does is quickly focus an analyst s attention on key aspects of the data that may warrant attention. It can thus be seen as a triaging mechanism that allows the analyst faced with big data to utilize the scarce resource of his time as efficiently as possible. Playing the role of the analyst, we therefore looked more closely at the tweets most representative of this topic, and it turned out some key posts in Russian (translated) were along the lines of NATO is preparing a global strike on Russia (see Figure below). This statement was in fact made not just by bloggers (though it was heavily amplified by them), but originated in the Russian Federation Council. It emerged therefore that what NATO envisages as defensive, a speaker in the Russian Federation Council had depicted as an offensive initiative specifically against Russia, and possibly even as a specific attack being planned. Clearly, this is a significant difference between 7

8 how global strike is discussed in the Russian-language space versus the English-language space, and the implications go to sentiment about NATO, and also to narratives about NATO. A key narrative this statement seems to invoke, for example, is that of Fortress Russia (Bouveng 2010), the narrative that Russia is surrounded by enemies just waiting for the chance to bring Russia down. The key here, however, is that we were able to pick up on this difference using the unsupervised data analytics techniques that have made Google, Amazon and others successful, without having to know to look for this particular narrative, or having to know anything about this particular dataset in setting up the computational analysis. To confirm that the phenomenon we noted was a reality, we ran a simple query to determine in what quantity of English-language versus Russian-language posts the words global and strike were used together. The results of this query (shown above) showed that indeed, the global strike topic was heavily amplified in Russian-language Twitter, whereas it sparked comparatively little interest in English Twitter. The effect is, moreover, disproportionate in that in the particular dataset in question, there were more English-language tweets to start with than Russian-language ones. In the second part of the first study, we turned our attention to another very topical area: that of bots or automated accounts. It has been known for some time (e.g. Nimmo 2017) that Russia s Internet Research Agency has not only been prolific in the use of bots on Twitter to lend artificial amplification to certain topics, but has set up bots in such a way as to make it appear that the 8

9 posts in these accounts are from real people, and, moreover, people in America (whereas the posts may be fully automated and controlled by someone in Russia). Using this as a starting point, we realized that Nimmo s (2017) twelve heuristics in essence again boil down to a more general point: it is an inescapable part of botness that bots behave similarly to other bots. This stems from the very reason that bots are set up in the first place: to automate the spreading of messages, thereby reducing the costs and increasing the return on investment of bot operators. When multiple bots are set up to repost certain approved content (say, news articles by RT or Sputnik), then what is observed en masse is that multiple different accounts have very similar patterns of posting, likes, retweets, etc., and that different accounts move in formation at similar times. What would fly under the radar when looked at in a bottom-up fashion becomes clear when looked at top-down. Bot identification, then, also is a task that we believe lends itself to unsupervised learning. The overview of our approach, which is similar to the survey of information space differences described above, is shown below. A Twitter (or other social media) dataset is the starting-point. This is cast as a term-by-account matrix, which, as above, is run through multilingual and language-agnostic Latent Semantic Analysis topic analysis. Note that terms are understood broadly and can include features such as the date and time of posts, elements of the account name, and so on. From this, one obtains a vector for each Twitter account. It is then possible to cluster the accounts by computing inter-account similarities. Based on the idea outlined above, we expect bots to be tightly 9

10 clustered together with other, similar bots (those that share the same patterns of posts, likes, retweets and so on), while humans, who naturally show more variation, will tend to be outliers. It is possible to establish a threshold of inter-account similarity, where pairs of accounts which exceed that threshold are judged likely to be bots. The full approach is described in Chew (2018). Running this algorithm on a dataset of tweets from around the US general election of 2016 (approximately three weeks prior to three weeks after), and using a threshold of 0.95 (where the maximum similarity is 1), we compiled a list of just over 1,000 bots. A review of a sample of around 40 of these showed that Twitter had already suspended a significant proportion of the sample (at least a quarter) for abusive posting patterns. Even if account suspensions after the fact tend to confirm the findings of an automated procedure, however, it is one thing to come up with a black box which generates lists of suspected bots, and it is another to provide an audit trail of evidence which is convincing to a human. We firmly believe that the place of the human in the loop must not be neglected in developing automated techniques, and that computer-generated analysis must be usable (in this case, provide convincing evidence of bot activity). For this, we use a technique related to LSA called PARAFAC. While Singular Value Decomposition, on which LSA is based, takes as input a two-dimensional matrix (rows and columns), PARAFAC is a generalization of SVD which takes as input a tensor with an arbitrary number of modes (or dimensions). Here, we cast the data for groups of suspected bots within the same botnet (i.e., that all tend to repost similar content) as a three-mode tensor: terms by accounts by time. With PARAFAC as with SVD, the tensor is decomposed, and as with SVD, the outputs include a set of topics. One of the issues that is notable with adversarial bots, and other adversarial activity on Twitter, is deliberate obfuscation. Obfuscation is, in effect, artificial noise which can be introduced into an already noisy dataset (many tweets include typos, say the same thing in slightly different ways, include different hashtags, etc.) to hide the fact that different accounts otherwise act in identical fashion. It is this noise, both deliberately introduced and that which is part of the background, that makes it a highly non-trivial problem to highlight groups of bots in botnets. PARAFAC, like SVD, abstracts away from the noise, cutting through to the signal. Once the PARAFAC topics are identified, it is then a relatively simple matter to pivot the output data as shown in the following figure, where it then becomes crystal clear that four accounts are basically posting the same tweets within seconds of one another. Note, however, that in the original tweets (prior to PARAFAC eliminating the noise), the account _deti_zhdut_ includes an extraneous character at the end of each tweet, while molodost bz prefixes each tweet with the hashtag #news. 10

11 As with the information space difference technique, it is important to note that none of the artificial or background noise need be stripped away by rules-based techniques; these patterns of botnet activity simply emerge from the data. This is key, because rules-based or heuristic-based techniques are easy for an adversary to defeat: once it becomes evident that bots with certain characteristics are being shut down or becoming ineffective, all that is necessary is to introduce new methods of obfuscation. Or, as Ben Nimmo pointed out to us in personal communication, when the heuristic was published, 72 tweets a day or more was considered suspicious, Nimmo noted that a number of accounts were then observed to drop down in output to just below 72. The key value of an unsupervised technique is that no heuristics are ever stated or programmed: all that is used is the accounts and their features. Features - such as words - cannot avoid being used, because these are what influence humans. And if bot creators are to capitalize on automation and attempt to propagate only certain types of content, it becomes hard to impossible to avoid bots being similar to other bots. 11

12 In this sense, an unsupervised approach holds the promise of always being able to stay ahead of the adversary, increasing his costs while decreasing the costs of those who are simply attempting to expose the patterns of abuse. Discussion and Conclusions The main findings from the two studies described above are: a. The techniques help an analyst very quickly gain a top-down down understanding of tens or hundreds of thousands of social media posts. What are the main topics being discussed? b. ATA systems provide insights about the how different communities perceive the same concept. c. Emergence of new natural language techniques can support ATA systems and fill the gap for language expertise. As some of the techniques work cross-lingually. d. These techniques work as a triaging system. They do not eliminate a human analyst from the loop, but they enable the analyst to spend his or her time much more effectively, and not miss the forest for the trees. e. The techniques bring to light high-level patterns that might escape notice if the source data were studied tweet by tweet. f. Specific findings from the data studied included: i. Key differences between narratives in the English-language and Russian-language information spaces - which in turn may indicate effects of propaganda. ii. Specific bots and botnets were revealed. I n this research, we shown the following: a. We have techniques that can be used on any social media dataset, because they re unsupervised. (Regardless of what the posts are about, what language they re in.) b. We demonstrate these techniques with a particular Twitter dataset. The demonstration shows what specific insights these techniques yield from this dataset. Other, but similar, insights will result from other datasets: c. Differences between the way Russian-speaking and English-speaking users talk about NATO Global Strike. These differences seem consistent with differences in national master narratives. d. Specific suspected bots/botnets, with examples of what topics they coalesce around. e. We can use the techniques to come up with these results very quickly and without prior knowledge of the dataset, because they re unsupervised. 12

13 Three possible areas of application for the research we are conducting are: 1. Identifying and tracking discourse that moves from one political discourse space to another. 2. Intelligence collection and analysis for social media. 3. Determining points of vulnerability in Russian IW for counter-propaganda IO. While it might be tempting to argue the truth-telling can undo lie-telling in cases of political propaganda, the history of political propaganda and political warfare shows that public often do not sort out truth from lies and often the lies looks like truths to them. Our age of speed-of-light fake news and phony social media site accounts makes all of this more of a challenge that ever before. The implications of the empirical analysis of Russian IW/IO are very important to the preservation of rational political discourse, intelligent decision making, and faith in democracy. It is apparent that Russia is dedicated to propaganda techniques that both continue and supersede the techniques used by the USSR during the Cold War. Before their annexation of Crimea, the Russians had succeeded in shutting down most of the free press in their nation. Full surveillance and censorship was tightened. After the annexation, however, Putin and other Kremlin leaders decided to take things to the next level, which was generating tidal waves of propaganda in social media (Soldatov & Borogan, 2015). As with other forms of communication in the past, Russia views social media as an information battlefield. This battlefield included Twitter, Facebook, and Youtube. The Russian weapons were words and people (trolls) seeding various narratives. For example, some trolls looked at forums online for negative comments about Putin and then posted comments praising Putin (Soldatov & Borogan, 2015). Comments like these commonly appeared in comment sections of both traditional and new media (Soldatov & Borogan, 2015).Russian IW online has included attacks on American policies, leaders, and its political system. It has also included abusive descriptions of European and American news sources. Now these nations are attempting to develop means of reistanting the online warfare. 13

14 References: Bartles, C. (2016). Getting Gerasimov right. Military Review, Jan/Feb, Bouveng, K. (2010). The Role of Messianism in Contemporary Russian Identity and Statecraft. Durham Theses, Durham University. Accessed at Chew, P. (2017). Understanding Russian information operations using unsupervised multilingual topic modeling. Lee D., Lin YR., Osgood N., Thomson R. (eds) Social, Cultural, and Behavioral Modeling. SBP-BRiMS Lecture Notes in Computer Science, vol Springer, Cham. Chew, P., & Turnley, J. (2018) Using big-data analytics to counter Russian information warfare. Galisteo Consulting Group. Albuquerque, NM. Darczewska, J. (2014) The Anatomy of Russian Information Warfare. Center for Eastern Studies. Dugin, A. (2016). Interview with BBC Newsnight. Accessed on June 15, 2018 at Gerasimov, V. (2013). Ценность науки в предвидении (The value of science in foresight). Военно-промышленный курьер. VPK News, 27 February 2013, available at Giles, K. (2009). Russia s National Security Strategy. NATO Defense College. Halpern, S. (2018). Cambridge Analytica, Facebook, and the revelations of open secrets. The New Yorker, Jaitner, M., & Matsson, P. (2015). Russian information warfare of th International Conference on Cyber Conflict. NATO. Matz, S., Kosinski, M., Nave, G., & Stillwell, D. (2017). Psychological targeting as an effective approach to digital mass persuasion. PNAS, 1-6. McDermott, R. (2016). Does Russia have a Gerasimov doctrine? Parameters, 46, Nimmo, B. (2017). #BotSpot: Twelve Ways to Spot a Bot. Atlantic Council Digital Forensic Research Lab. Accessed on June 3, 2018 at ODNI. (2017). Assessing Russian activities and intentions in recent US elections. Available at 14

15 Onuch, O. (2015). Brothers Grimm or Brothers Karamazov: The myth and the reality of how Russians and Ukrainians view the other. In A. Pikulicka-Wilczewska & R. Sakwa (Eds), Ukraine and Russia: People, Politics, Propaganda and Perspectives. Sanovich, S. (2017). Computational Propaganda in Russia The Origins of Digital Misinformation. University of Oxford Working Paper No Sheth, S. (2017). New evidence emerges that Russia infiltrated Facebook to sow political chaos in the U.S. Business Insider, Sept. 28, Available: on Soldatov, A., & Borogan, I. (2015). The Red Web. New York: Public Affairs. Tatham, S. (2015). The Solution to Russian Propaganda is Not EU or NATO Propaganda but Advanced Social Science to Understand and Mitigate Its Effect in Targeted Populations. National Defence Academica of Latvia Center for Security and Strategic Research. Kenneth Hacker is Professor and Department Head of Communication Studies at New Mexico State University. Peter Chew is President of Galisteo Consulting in Albuquerque, New Mexico. Ahmed Abdelali, PhD. Senior Software Engineer at Qatar Computing Research Institute. 15