Hash Verification

Once you're authenticated you can use your JWT to access the Content Hash List

The Hash Verification API provides platforms with an interface to check if any content uploaded by their users has already been identified as terrorist content by the TCAP.

Platforms query the API with the hash strings of their media files from their own systems. In turn, they receive a response indicating if any of those files are, or are likely to be, terrorist content.

What is the Hash Verification API

The API provides an endpoint that users can query with their own hash strings. These hash strings are then checked for exact or partial matches against the TCAP repository of hashed terrorist content and a response is returned indicating the results of these checks.

What hash algorithms are provided

We currently provide exact cryptographic hash-matching for the MD5, SHA256 and SHA512 algorithms as well as partial, or perceptual, hash matching for the PDQ (image) and TMK (video) algorithms.

How to use the Hash verification API v2

In order to use the API, you need to hash any content you wish to check against our archive. In the case of MD5, SHA256 and SHA512 this is fairly straightforward, however for TMK and PDQ you will likely have to fork Threat Exchange's code.

Your requests

Once you're authenticated you can use your short lived JWT to access the Content Hash List

You can send a json list of items, where an item is of the following form for cryptographic hashes:

Python
TypeScript
cURL

import requests

url = "https://beta.terrorismanalytics.org/hash-verification/api/v2"

headers = {
    "Authorization": f'Bearer {token}"
    "Content-Type": "application/json"
}

body = [
    {
        "hash_value": "YOUR_HASH_STRING",
        "hash_type": <"MD5" | "SHA256" | "SHA512">
    },
    ...
  ]

response = requests.post(url, json=body, headers=headers)
print(response.json())

type HashType = 'MD5' | 'SHA256' | 'SHA512';

interface HashRequest {
  hash_value: string;
  hash_type: HashType;
}

const url = 'https://beta.terrorismanalytics.org/hash-verification/api/v2';

const body: HashRequest[] = [
  {
    hash_value: 'YOUR_HASH_STRING',
    hash_type: 'SHA256'
  },
];

const verifyHashes = async () => {
  try {
    const response = await fetch(url, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${token}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(body)
    });
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error('Error:', error);
  }
}

curl -X POST 'https://beta.terrorismanalytics.org/hash-verification/api/v2' \
  -H 'Authorization: Bearer YOUR_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '[
    {
      "hash_value": "YOUR_HASH_STRING",
      "hash_type": "SHA256"
    }
  ]'

and the following for perceptual_hashes:

Python
TypeScript
cURL

import requests

url = "https://beta.terrorismanalytics.org/hash-verification/api/v2"

headers = {
    "Authorization": f'Bearer {token}"
    "Content-Type": "application/json"
}

body = [
    {
        "hash_value": "YOUR_HASH_STRING",
        "hash_type": <"TMK" | "PDQ">
        "confidence": 0.7 # number between 0 and 1
    },
    ...
  ]

response = requests.post(url, json=body, headers=headers)
print(response.json())

type HashType = 'TMK' | 'PDQ';

interface HashRequest {
  hash_value: string;
  hash_type: HashType;
  confidence: number;
}

const url = 'https://beta.terrorismanalytics.org/hash-verification/api/v2';

const body: HashRequest[] = [
  {
    hash_value: 'YOUR_HASH_STRING',
    hash_type: 'TMK',
    confidence: 0.7 // number between 0 and 1
  },
];

const verifyHashes = async () => {
  try {
    const response = await fetch(url, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${token}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(body)
    });
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error('Error:', error);
  }
}

curl -X POST 'https://beta.terrorismanalytics.org/hash-verification/api/v2' \
  -H 'Authorization: Bearer YOUR_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '[
    {
      "hash_value": "YOUR_HASH_STRING",
      "hash_type": "TMK",
      "confidence": 0.7
    }
  ]'

Query Parameters

Parameter	Type	Description
`ideology`	`'islamist' \| 'far-right' \| 'all'`	Specifies the ideology to filter the results.

tip

If you do not include the query parameter, all content will be searched regardless of ideology.

How many items can I send in a request?

For non-TMK hash verification requests, you can send up to 20 items in a list.

For TMK hash-verification requests, one item in a list. We may increase the number of TMKs per request in due course and, as such, have built the service to accept a list in the request.

Our responses

After receiving your list of items, we will match each one against our system and return a list with the same number of items, where an item is the following for cryptographic hashes (where True represents a match against content in our archive):

Parameter	Type	Description
`hash_value`	`String`	The hash digest
`hash_type`	`'MD5' \| 'SHA256' \| 'SHA512'`	The type of hash algorithm used
`result`	`Boolean`	Result of the hash check operation
`error`	`String \| null`	An error message or null if successful

and the following for perceptual_hashes:

Parameter	Type	Description
`hash_value`	`String`	The hash digest
`hash_type`	`'TMK' \| 'PDQ'`	The type of hash algorithm used
`result`	`Boolean`	Result of the hash match operation
`confidence`	`Float`	Match confidence score between 0-1, or range [0-1, 0-1]
`error`	`String \| null`	An error message or null if successful

What are confidence scores?

While cryptographic hash verification simply involves checking your hash string for an exact match against our system, perceptual hashes check for similarity between two hashes and their respective pieces of content.

With this in mind, when you request a check on a cryptographic hash, you must include a confidence parameter between 0 and 1 where 1 is complete confidence, ie. an exact match.

We will return True in the result field if we have a piece of content whose hash matches or exceeds the confidence threshold you provided when compared with your hash_value. If we do not have any confidence that matches or exceeds this threshold, we will return False.

In addition, if the result is True, we will return a confidence score that represents just how similar the piece of content on our system was to your piece of content. For PDQ hashes, this score will a decimal between 0 and 1, where 1 is an exact match. For TMK hashes, this score will be a pair of decimals between 0 and 1, where [1,1] is an exact match.

What is the Hash Verification API​

What hash algorithms are provided​

How to use the Hash verification API v2​

Your requests​

Query Parameters​

How many items can I send in a request?​

Our responses​

What are confidence scores?​