An assessment of submissions made to the Predictive Toxicology Evaluation Challenge

Abstract

Constructing "good" models for chemical carcinogenesis was identified in IJCAI-97 as providing a substantial challenge to "knowledge discovery" programs. Attention was drawn to a comparative exercise which called for predictions on the outcome of 30 rodent carcinogenicity bioassays. This - the Pr edictive Toxicology Evaluation (or PTE) Challenge - was seen to provide AI programs with an opportunity to participate in an enterprise of scientific merit, and a yardstick for comparison against strong competition. Here we provide an assessment of the machine learning (ML) submissions made. Models submitted are assessed on: (1) their accuracy, in comparison to models deve loped with expe rt collaboration; and (2) their ex planatory value for toxicology. Th e principal findings were: (a) using structural information available from a standard modelling package, layman-devised features, and outcomes of established biological tests, resu lt s from MLderived models were at least as good as those with expert-derived techniques. This was surprising; (b) the combined use of structural and biological features by ML-derived models was unusual, and suggested n ew avenues for toxicology modelling. This was also unexpected; and (c) significant effort was required to interpret the output of even the most "symbolic" of ML-derived models. Much of this could have been alleviated with measures for converting the results into a more "toxicology-friendly" form. As it stands, their absence is sufficient to prevent a whole-hearted acceptance of these promising methods by toxicologists. This suggests that ML techniques have been able to respond- not fully, but nevertheless substantiall y - to the PTE Challenge.