In threat intelligence, by definition, an analyst will most of the times have to perform assessments in an environment of incomplete information, and/or with information that is being produced with the purpose of misleading the analyst.

One of the well-known methodologies is the Analysis of Competing Hypotheses (ACH) [1], developed by Richards J. Heuer, Jr., a former CIA veteran. ACH is an analytic process that identifies a set of alternative hypotheses, and assesses whether data available are either consistent or inconsistent with each hypothesis. The hypotheses with most inconsistent data will be rejected. To quote Heuers words

ACH is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions. It helps an analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult to achieve.

It is very important to note that the goal of ACH is to reject as many hypotheses as possible, not to confirm any.

One of the advantages of using ACH is that it reduces analysts confirmation bias. In fact, a common pitfall during analysis usually is to form a hypothesis on our head and to look for evidences that support it, confirming indeed our initial line of thought following the most likely hypothesis. However, such approach does not take into account possible alternate hypotheses, as well as what other data is missing that should be there if that given hypothesis would be true. This is achieved by imposing the analyst to identify, and then refuse, as many competing hypotheses as possible using all data available.

How it works

ACH requires the analyst to collect all the related information and organize them in a matrix: all the hypotheses on the top (first row), and all the relevant information on the left side (the first column). In this way, every piece of information can be evaluated against each of the hypotheses, by assessing if its consistent or inconsistent. Heuer describes the full process in eight steps, which could be summarized as follow:

  1. Identify all hypotheses. Ideally, all hypotheses should be mutually exclusive, meaning that if one is true all the others must be false.
  2. Lists evidences and arguments relevant for and against each hypothesis. This has to include also assumptions and logical deductions.
  3. Create a matrix as described above and analyze each evidence against every hypotheses by defining whether is Consistent, Inconsistent or Not applicable / Not relevant, in an attempt to disprove as many hypotheses as possible. In Heuer words, analyze the diagnosticity width:500px" />

    Conclusions

    ACH is just one of the possible structured analytic techniques available. I personally like it very much and find it quite useful. But remember that while the matrix helps in creating a model for the analysis of problems with conflicting information and it generates a definitive mathematical total for each hypothesis, at the end it is still up to the analyst to use his/her judgment to make the final conclusion.

    This was a very brief introduction of ACH, and in my next diary I will apply ACH to a practical recent case. But I definitely encourage those interested to go through Heuers book for a deeper explanation of such model.

    Happy Hunting,
    Pasquale

    References:

    [1] Psychology of Intelligence Analysis, Center for the Study of Intelligence, Central Intelligence Agency. https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis
    [2] Scott J. Roberts, ACH Analysis of a Trump Campaign Compromise, https://sroberts.github.io/2016/12/12/rnc-hack/

    Pasquale Stirparo, Ph.D.
    @pstirparo

    (c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.
 
Internet Storm Center Infocon Status