Fleshing Out the Bones: Studying the Human Remains Trade with Tensorflow and Inception

Over the last decade, the trade in human remains has been facilitated by social media platforms. Research is still in its infancy, but indications are that its impact is much larger than what is known. While laws vary from the state to federal level, there are ethical and legal concerns, including damage to archaeological sites, vandalism of both historic and modern cemeteries, theft from collections, loss of Indigenous heritage and the violation of the rights of descendent communities.

The research here takes a look at whether machine learning can detect visual signals in photographs, indicating whether human remains are for sale. Huffer and Graham used Tensorflow and Google Inception-v3 to identify clusters of visually similar images, at scale, relatively quickly on Instagram. These clusters were then referenced against the data mining of the text in the accompanying posts. By finding clusters of similar images according to their content, they are able to elucidate patterns in the visual rhetoric of these images, which can become the basis for filtering and tracking the trade in human remains online.

Meaningful clusters of similar images were found containing human remains. For example, the position of the skull relative to the camera, the arrangement of materials on shelving and mimicking a museum display case all seem to be relevant signals. The results are only the beginning of what Huffer and Graham expect to be able to do using a deep learning approach. Their research could also be useful in identifying other kinds of illegal materials bought and sold online, such as drugs or wildlife.