Proceedings

Raising the Standards of AI Products

Download BibTeX Cite

Proceedings of the Ninth International Joint Conference on Artificial Intelligence. General. Pages 1289-1293.

© 1985 International Joint Conferences on Artificial Intelligence

Abstract

We propose a mechanism for the promotion of high-standards in commercial Artificial Intelligence product s , namely an association of companies which would regulate their own membership using a code of practice and the pre c edents set by previous cases . Membership would provide some assurance of quality . We argue the benefits of s uch a mechan i sm, and discuss some of the deta ils including the proposal of a code of practice . This paper i s intended as a vehicle for discus s ion rather than as the pr·esentation of a definitive solution.

Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.

1. The Need for High Standards in AI Products

Credibility has always been a precious asset for AI, but never m6re so than now. The current commercial interest in AI is giving us the chance to prove ourselves. If the range of AJ products now coming onto the market are shown to provide genuine solutions to hard problems then we have a rosy future. A few such useful products have been produced, but our future could still be jeopardized by a few, weil publicised , failw·es.

Genuine failures where tnere was determined, but ultimately unsuccessful, effort to solve a problern - are regrettable, but not fatal. Every technology has its limitations. What we have to worry about are charlatans and incompetents taking advantage of the current fashion and selling products which are overrated or useless. AI might then be stigmatised as a giant con-trick, and the current tide of enthusiasm would ebb as fast as it flowed. (Remember Machine Translation it could still happen.) Both companies selling AJ products and ac.:ademic.: AI research groups would suffer in the r·esulting cr·ash.

AJ companies are very dependent on the good-will of their customers. The current life-span of typi­ cal AJ products is ab out 1-5 years. The customers

RAISING THE STANDARDS OF AI PRODUCTS

Alan Bundy Department of Artificial Intelligence Univer·sity of Edinburgh Richard Clut terbuck School of Social Seiences University of Sussex of AJ products are likely to stay in the market for several times this period ; typi cally they are them­ selves companies or ac ademic groups engaged in AI research or interested in the long term appli cation of AJ techniques. To stay in bus iness the AJ company must sell successive upgrades of i t s products to the same group of customer s and , therefore, must build up and maintain a good reputation. If AJ business is to expand then new customers must be brought into thi s exist ing group. This will only happen if the overall range of AJ products is of high-quality and the reputation of this particular company is good. Thus it is in the interests of each company to raise both the generar and its particular standard. It must al so convince customers that its products are of high standard . When the market was small a company with high­ quality products could win new customers by word of mouth. Now the market is growing they must use advert isements, and i t becomes harder for a company to convince potential customers that its products are of high quality.

Apart from improving the publ ic image of AI and increasing the market for AJ products, producing more high-quality products would rai se morale and standards in AJ itself, leading to a virtuous c ir c le of standards being raised , better work being done , good people being attracted to the f ield , and even more high quality products emerging. Poor-qual ity products will produce a vicious circle going in the opposite dir·ection.

But these internal reasons for wanting high stan­ dards, while important to insiders, are perhaps less important than external reasons. AJ products Iook destined to play a major role in soc.:iety. That society deserves, and has the right to expec t, protection from exploitation by AI companies and fr·om being harmed by Al pr·oducts.

An extreme, potential example of such harm is described in [Thompson et al 64], which argues that it is not possible to build an automatic or semi ­ automatic launch-on-warning system for nuclear weapons with anything like an ac.:ceptable failure rate. Anybody who cla imed to have done so , or who claimed to be able to do so, would be guilty of misleading the public in a way that could have disastraus consequences. If such a c laimant were an AI company then the whistle might be blown on i t by the mechanism described below.

However, the main purpose of this proposal is t o catch less apocalyptic , but more common-plac e, misleading claims, whether or not they might give rise to a legal remedy. Examples might be an expert systems shell whose advertised range far exceeds the problems it is really suitable for, or a natural language front end which is presented as being able to deal with a much wider input than it, in fact, c an.

2. A Professional Association

The academic field guards itself against charlatans and incompetents by the peer review of research papers, grants, PhDs, etc. There is no equivalent safeguard in the commercial AI field. Faced with this problern other fields set up professional as­ sociations and codes of practice. 1 AI needs a similar set-up. We propese that the responsible AI companies should get tagether now to found such an association. Continued membership should depend on a constant high-standard of AI products and in-house expertise. Members would be able to ad­ vertise their membership, and customers would have some assurance of quality. Charlatans and incom­ petents would be excluded or ejected, so that the failure of their products would not be seen to reflect on the field as a whole nor on the com­

panies in the Association.

Since the trade in AI products is international, with multi-national companies involved both as ven­ dors and customers, the Association would also need to be international. Otherwise, there would be difficulty over membership of multi-national com­ panies and about dealing with complaints resulting from international sales. It is particularly important to make membership attractive to multi-nationals because, with their existing reputation, they have less to gain from the cachet of membership and are more able to avoid the full impact of national r·egistr·ation.

The Association would be self-regulating. If its decisions were seen to be too arbitrary then the value of Association member·ship would be devalued in the eyes of the public, the customers and the vendors, and the importance of its decisions would decrease in proportion; customers would take no account of Association membership when deciding to buy, so vendors would not bother to join. For such self-regulation to work it is necessary for the Assocrat10n to be publically visible. Both vendors and potential customers must be aware of the As­ sociation and must see its decisions as fair and its sanctions as effective. Customers will then use Association membership as a major determinant when decrdrng whether and what product to buy. Ven­ dors wrll regard Association membership as a valuable asset to their company, and will aim for high qualrty rn therr products in order to retain mem­ bershrp. They will want to use Association member­ shrp in their advertising, and this will, in turn rmprove the visibility of the Association. It will b~ necessary for the Association to maintain a high profile of both its existence and its actions, to be open about its decisions, and to employ effective sanctions.

1 E.g. the Vehicle Builders and Aepairers Aosociation and the National Association of Eotate Agents. Note that, unless otherwise stated , all examples are of UK institutions or laws .

The Association would need a panel to consider applications for membership and to hear complaints against members. Its main sanction would be tefus­ ing membership or expelling existing members, backed up by lesser sanctions like a public admoni­ tion, payment of compensation, etc. The rules of the Association might include a contractually binding committment by members to fulfil any compensation order made by the panel. The Association might also insist that contracts issued by members con­ tained various standard clauses, e.g. giving customers the right of reimbursement if returning products within a certain period, guaranteeing compensation under certain circumstances, insisting that precise, testable statements be made about the product, etc. 2 The panel need only take a passive role in determing disputes; it would publicise its address and its willingness to hear complaints. The burden of making a case complaint would fall on the com­ plainant. The panel might . then need to employ a small team of experts to investigate discrepancies between the evidence brought by the complainant and that by the company complained of. This investigative team could be recruited on an ad hoc basis, e.g. academic researchers as consultants. The complainant would usually be the dissatisfied purchaser of one of the companies products, but could be any member of the public with a legitimate interest in the product. There might be a multi-stage process, so that cases were only heard by the full panel when a prima-facie case had been established. This is to filter out malicious complaints and those that are outwith the remit of the panel. It would not be necessary actively to investigate AI products or companies before any complaint had been received, and would probably be pr·ohibitively expensive to do so.

The panel needs a code of practice to which members would agree to adhere and which would serve as a basis for applying sanctions. What form should such a code take, i.e. what counts as malpractice in AI? We suspect malpractice may be a Iot harder to define in AI than in insurance, ar·chitectur·e or travel agency.

  • Due to the state of the art, AI products cannot be perfect. No-one expects 1 OOl accurate diagnosis of all known diseases. On the other hand a program which only works for slight variations of the stan­ dard demo is clearly a con. Where is the threshold to be drawn and how can it be defined?
  • It is unlikely that any current AI product could fulfil a claim to: understand any natural language input, or to make pro­ gramming redundant, or to allow the user to volunteer an~· infor·mation what-so-eveL However, the claimant could defend the

2 c.f. the Vehicle Buildno and Aepairera Association which has standard forms for eetimatea and recornmends claus·s in repair contracta.

claim by debating the meaning of 'understand', 'programming', or 'information', and this would muddy the water. What constitutes an exaggerated claim?

  • Given the ambiguity of such terms it would be difficult to decide whether an exaggerated claim was intended to deceive or was due to a difference in terms or was just a genuine oversight on the part of the vendor. How is a vendor's claim to be assessed? Should one try to assess the vendor's intention, or should one ignore thi.s and only assess how a reasonable cus­ tomer· might inter·pr·ete the claim?
  • Because of the ambiguity of such terms the full description of an AI product must describe its limitations, e.g. what sentences it cannot understand, as weil as its abilities. It is not enough to refrain fr·om false claims.
  • A vendor may claim that it cannot ac­ curately describe the limitations of its product without revealing confidential in­ formation about the technique and/or software it is based on. The problern is particularly acute for software products because of the Iack of protection af­ forded by patent and copyright law. Where do we draw the line between a complete and accurate description of the capabilities of the product and the pr·otection of tr·ade secr·ets?

The difficulty is to give a precise dcfinition of what constitutes reasonable behaviour on the part of a vendor. It seems impossible to cover all the possible situations, in advance, with a Iist of precise standards to be attained, but it is often possible retrospectively to detect unreasonable practice in particular cases. The usual legal solution to such problems is to use a high-level code in combination with judgements about individual cases in order to build up gradually a picture of the reasonable ven­ dor, i.e. to establish case 'taw'.J and to use this to evaluate complaints rather than to pre-vet products. 4 Note, however, that it takes a long time to build up an extensive range of cases most of the early judgements must be made

solely on the basis of the high-level code.

The panel would evaluate a complaint against this code. They would be able to take account of the state of the art and compare the product with the claims made for it. A high-level code and the injunction to judge 'reasonableness' would enable the panel to assess whether the spirit of the code had been broken, rather than the Ietter of some spuriously precise, low-level code. The ac.:cumulation of case 'law' would ensure some uniformity of treatment, and prevent favouritism or victimisation. Openness about the grounds for decisions would also help ensure uniformity. There is a current trend in other areas towards the giving of reasons a procedural safeguard which promotes the the quality of decision making. 5 Diffic.:ult decisions, like those outlined above, would be decided in particular cases, rather than in general. The general answers would emerge over time with the accumulation of judge­ ments. Previous ajudic.:ations would guide future ones without pre-empting them. The case 'law' would provide a guide to vendors as to how to prac.:tic.:e r·easonably.

3 Law is in scare ~xtra-legal mechaniam. quotea becauae are defining an

4 Compar@, for instanc@, the Unfajr Contract Trrms Af! ~. which uses case law to define reasonableness in relation to exclusion clauses.

3. A Proposed Code of Practice

As a basis for discussion, we propese below such a code of practice for AJ vendors. Before we give this we must define our terms. In what follows below, the term:

  • AI product, means any piece of software or hardware or any service or any com­ bination of these which is based on AJ techniques and which is offer·ed for sale;
  • vendor, means a company selling an AI product, either to a customer direc.:t or to a middleman, whether that company made the product or not;
  • customer, means a person or group who buys or at tempts to buy an AJ product fr·om a vendor·;
  • user, means the person who uses the AJ product, in particular the person who in­ teracts with the product if it is inter­ active.

The proposed code is:

The vendor of an AI product should describe to the customer, and where ap­ propriate the general public, the abilities and limitations of the product as ac.:curately as possible, taking ac.:count of the likely expec.:tations of the intended customer. In particular, the vendor should ac.:curately describe, in so far as the state of the art and its own knowledge enables this to be

done:

  1. what the product does, including an account of its scope, limitations and r·eliability;
  2. known bugs in the product;
  3. the consequences of failure of the pr·oduct;
  1. the amount and type of user inter­ action required and how and at what cost it is to be obtained;
  2. the skill and knowledge required of the user·;
  3. the computational requirements of the product, e.g. hardware and software environment, space and time requirements in different envir·onments;
  4. the amount and type of maintenance required and the cost of this;
  5. any social, economic or legal impl ica­ tions of the use of the product, where these can be assessed.

It is the responsibility of the vendor to see that this code is observed by any agent ac ting an its behalf, e.g. a salesman. The vendor is also responsible for ensuring that any middlemen which sell its products are fully acquainted with the necessary in­ formation to enable them to comply with the c:ode.

An informed customer will, in any case, ask about 1-7 above, and a reasonable vendor should supply the information unasked. Thus this part of the code merely makes good practice explicit, as it was intended to da. In conjunction with the case 'law', it should help protect the uninformed customer and define the Standards to be met to become a r·easonable vendor·.

Point 8 is rather different from the others. It

was inserted to try to prcitect the wider interests of society as weil as those of the customers. It might be cr-iticised as being impr·ac:tical to realise or· as not appropriate in this context. However, we feel that something like it is required somewhere, and we would welcome suggestions as to how best to meet this requirement. Maybe it needs to be dealt with by separate machinery.

It is not our intention that vendors be required to state political or ethical opinions, nor that the Association be asked to judge such opinions; it would be beyond their competence to da so. In point 8 wc wanted only to encourage vendors to make Statements which were within their technical, legal, etc. competence so as to enable others to form accurate political and ethical opinions about the impact of the product. For instance, all the photocopiers at Sussex University have a prominent notice above them detailing the law relating to copyright. Vendors of AI products should, similarly, draw the attention of users to illegal uses of their pr·oduc: t .

It is not intended that complainants actually in­ tend to be customers of the vendors they complain of, but they should have a legitimate interest in the product over and above commercial competition. For· instance, suppose an exper·t in the field believes that a vendor is misleading customers about a product or that the vendor is producing a product that will be harmful to society; we would like that expert to be able to bring a case to the Associa­ tion. On the other hand the Association would have to be alive to attempts by vendors to undermine their rivals by bringing malicious complaints and should filter· out such complaints at an ear·ly stage.

4. Related Codes and Laws

The above pr·oposal is complementary to 'the exist­ ing system of codes and laws applying to AI.

For instance, in the UK, the British Computer Society and, in the USA, the Association for Com­ puting Machinery both provide codes of conduct for their members, [BCS 81, ACM 82]. We imagine that most other national computing societies have similar codes. Neither of these codes apply to AI products or vendors, as such. There is a small area of overlap in that a salesman who is a member of the BCS or ACM is required to behave honestly and competently in promoting · a product. However, any sanctions for breaking the BCS or ACM codes would fall an the individual rather than the company. Our code is intended to apply to vendors, which would usually be companies rather than individuals.

In the UK, the Trade Descriptions Ac: t uses criminal sanctions to protect customers from false claims, and the Sale of Goods Acts and the Unfair Contract Terms ßtl 1977 permit civil remedies for defective goods. Other countries have similar laws. However, as illustrated above, there is a large grey area between illegal behaviour and the behaviour of a reasonable vendor. It is the purpese of the above proposal to deal with this grey area, where the vendor· has clearly behaved unr·easonably, . but not in such a way as to constitute a criminal offence or grounds for a civil action. For this reason it would not be appropriate for the proposed Associa­ tion to impose the kind sanctions that would be imposed by a court of law. An unreasonable vendor should still be allowed to trade, but should not be allowed to use the cachet of membership of an association of reasonable vendors, with whatever as­ surance of high-quality that that was generally feit to imply.

The British Standards Institute defines Standards for many products. Vendors whose products meet these standards are able to advertise that fact with a 'kite mark'. In some cases the standards set have bet>n adopted by the law. 6 There is currently an at tempt to define a BSI standard for Prolog. Unfortunately, few AI products I end them­ selves to such definitions of standards. It is not worthwhile to try to define one unless very similar products are being produced by a number of ven­ dors and there is a wide agreement an a de facto standard. Major and stabilised programming lan­ guages seem possible candidates, but customised ex­ pert systems, and even expert systems shells, da not .

6 E.g . motorcycle crash helmets must be worn and must comply

with a minimum BSI standard.

5. A Discussion of Problems

There is a danger of a few companies annexing the Association to themselves and excluding worthy competition. But this is not a major danger. Firstly, in the current state of the AI market, AI companies have a Iot to gain by encouraging high­ quality in other AI companies. Every success in­ creases the market for everyone, whereas failure decreases it. Until the size of the market has been established and the capacity of the companies has risen to meet it, AI companies have more to gain than to lose by mutual support. Secondly, excluded companies can always set up a rival as­ sociation. There is room for more than one as­ sociation, and they could compete by trying to set the highest standard for membership. However, too many associations would be confusing to consumers and would diminish their influence and effectiveness.

There is also a danger of the Association develop­ ing into a trade protection society, i.e. of main­ taining low standards by protecting its members from disgruntled customer by affering weak excuses for faulty products and by not employing effective sanctions. If this haooened then customers would lose confidence in the Association and membership would cease to carry any assurance of high-quality. Membership would sti ll be attractive to vendors who wanted to use the Association"s excuses to fob off disgruntled customers. However, provided the As­ sociation did not have a monopoly, there would be nothing to stop a more principled group of vendors for·ming a rival association as above.

The Association would always be at risk of legal action against it from disgruntled vendors who might sue for libel . It would need to take care that its pronouncements were "fair comment " and to take legal advice on them. It should also try to create the conditions under which vendors would have more to lose from the bad publicity accruing from the court case than :they would gain from any damages awarded. But to guard against such actions, the Association would have to maintain a legal defenr:e fund contributed by the member·s.

6. Passive vs Active RoJe

We have proposed that the Association take a passive role, reacting to complaints, rather than an active role, pre-vetting products and/or instigating investigations of products with its own team of investigators. Our reasons are practical rather than principled. An active role would require money and people. Most AJ companies are small and newly set-up; they might be loath to provide the large sums of money required to set up the investigative machinery, but might be prepared to fund a, much eheaper, passive association. AJ experts are currently rare and expensive. Most such experts want either to conduct their own research or set-up their own companies. It would be hard to recruit good full-time investigators. One might find academics pr~pared to werk part-time, but most candidates would have some existing consultancy ar­ rangements that might disqualify them. We have also argued that, while the field is still immature, it is more difficult to set standards that a pre­ vetted product must meet than to evaluate the complaints of a customer against the code of prac­ tice and pr·evious cases.

For the same reasons we have not proposed the direct registration of AJ products. To issue a certificate of good quality to an individual product w.ould require a prior investigation of that product; rt would no Ionger be possible to employ the default assumption that the product of a member of the Association was assumed good unless proved otherwrse. That is, direct registration of products would require an active Association with all the associated expense, employment of rare expertise and set ting of prior standards. The passive mechanism proposed above indirectly ensures good quality products by encouraging the vendors to produce products that will not at tract complaints. The burden of criticising the product and proving that criticism falls an the complainant. The panel need only investigate differences in the evidence presented to them by the complainant and vendor about the product complained of; this investigation would be relatively cheap compared with that re­ quir·ed to r·egister every pr·oduct .

Unfortunately, this means that the burden of proof falls on the customer. But this is not as bad as it seems; currently, most AJ customers are themselves AI practioners, to some degree. For· instance, the customers for expert systems shells and knowledge representation systems are often researchers from the AI Iaberateries of other com­ panies. Hence, they are in a position to investigate the product and bring a complaint. They only Iack a body to bring it to.

All this might change: AI companies might get richer and more able to fund an active association, AI experts might get thicker on the ground, stan­ dards might get better defined as the field ma­ tures, the average AI expertise of customers might decline as the customer base expands. In this case, there is nothing to stop the Association moving to

a mor·e ar:tive r·ole.

7. Relationship to the Law

We have proposed a extra-legal regulatory mechanism, i.e. one without legally enforceable sanctions. One reason is that we are aiming to regulate in the grey area of legal but unreasonable practice. Another reason is that wc would like to see an international association covering countries with different legal systems, so no one legal framewerk can be assumed. A third reason is that we want to make it as easy as possible for the Association to get started; an extra-legal mechanism

involves the m'inimum of bur·eaucr·atic hassle.

This proposal, however, creates problems. The boundary between legal and illegal practice is fuzzy; infringement of some aspects of the code of prac­ ti ce (e.g. points 1-3) may sometimes give rise to legal remedies. Even when legal remedies were pos ­ sible there may still be situations in which both complainant and vendor would prefer to refer the matter to the Association because it provided a eheaper and quicker mechanism for settling the dispute. However, if cases were referred for legal remedv then the workin<as of the Association miqht be undermined; such cases would become sub-judice and the Association would no Ionger be able to comment on them until they were decided (which might take a lang time).

However, as it develops the Association might seek statutory authority in the countries in which it operated . 7 This authority might include, for in­ stanr:e:

  • the right to grant licences enabling com­ panies to become vendors of AT products;
  • protection from libel action in its judge­ ments;
  • that infringement of the code of prac­ tice would raise a presumption of fault against an infringing vendor in any legal action. 8

Note that such legal powers increase the dangers of the Association becoming a trade protection society. In particular, the right to grant licences gives a monopoly that might be used to exclude competi­ tion. Therefore, it would need to be offset with rights to abuse, e.g action if Association underwrite the customer and the vendor to prevent the vendor might have the right of legal unfairly excluded from membership, the might be required to pre-vet and then the products of its members, so that dissatisfied customers could sue the Association.

8. Conclusion

In this paper we have proposed a mechanism for peer-policing of standards in AI products. The mechanism consists of an association of vendors of AI products who would use a code of practice to sanction vendors who are guilty of unreasonable practice. It is similar to associations used by other groups of companies and professionals affering ser­ vices or selling products. It is necessary if AJ is not to harm itself and society by the products it produces. Same such mechanism is vital to the existing, responsible AI companies if they are to protect their investment in AI; they must try to prevent a few irresponsible companies from exploit­ ing their customers with overrated or useless products and putting those customers off of AI pr·oducts in ganer·al.

Most of the discussion above is relevant to com­ puting generally. We have limited ourself to AI because of the nature of this conference and in order to help focus our ideas and to identify an area small enough to be tackled. AI is growing fast, but there are not yet many major AJ companies, and it seems more likely that they · would be prepared to get tagether than that IBM, ICL, Honeywell, etc. would be. However, since many AJ products will also contain non-AI techniques, it may eventually be necessary to widen the remit of any AI association.

7 See, ". g. th· statutory diacipl inary pow·ra of the Law Society over solicitors.

8 See , e.g. that an infringement of th· Highway Code raises a presumption, albeit rebuttable, that the driver of · vehicle involved in an accident is at fault.

The above mechanism is only able to regulate commercial companies. We have argued above that academics already have a self-regulatory mechanism of peer review. However, there is major gap in that neither mechanism covers government organisations , e.g. the military. That is beyond the scope of this paper·, but not beyond our· desir·es.

This paper is intended as a vehicle for discussion of the problems of maintaining high standards in AJ products. It does this by presenting a proposal, but this proposal is not intended as definitive. We would welcome feedback - especially on the code of practice itself. We hope that this paper will inspire those companies that care about high stan­ dards to get together, and we hope that they will see that it is in their dir·ect inter·est to do so.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards