Mining Definitions from RDF Annotations Using Formal Concept Analysis

1 Introduction

World Wide Web has tried to overcome the barrier of data sharing by converging data publication into Linked Open Data (LOD) [Bizer et al., 2009]. The LOD cloud stores data in the form of subject-predicate-object triples based on the RDF language 1 , a standard formalism for information description of web resources. In this context, DBpedia is the largest reservoir of linked data in the world currently containing more than 4 million triples. All of the information stored in DBpedia is obtained by parsing Wikipedia, the largest open Encyclopedia created by the collaborative effort of thousands of people with different levels of knowledge in several and diverse domains.

More specifically, DBpedia content is obtained from semistructured sources of information in Wikipedia, namely infoboxes and categories. Infoboxes are used to standardize entries of a given type in Wikipedia. For example, the infobox for "automobile" has entries for an image depicting the car, the name of the car, the manufacturer, the engine, etc. These attributes are mapped by the DBpedia parser to a set of "properties" defined in an emerging ontology 2 [Benz et al., 2010] (infobox dataset) or mapped through a handcrafted lookup table to what is called the DBPedia Ontology (mapped-based ontology). Categories are another important tool in Wikipedia used to organize information. Users can freely assign a category name to an article relating it to other articles in the same category. Example of categories for cars are "Category:2010s automobiles", "Category:Sports cars" or "Category:Flagship vehicles". While we can see categories inWikipedia as an emerging "folksonomy", the fact that they are curated and "edited" make them closer to a controlled vocabulary. DBpedia exploits the Wikipedia category system to "annotate" 3 objects using a taxonomy-like notation. Thus, it is possible to query DBpedia by using annotations (e.g. all cars annotated as "Sport cars"). While categorical information in DBpedia is very valuable, it is not possible to use a category as one could expect, i.e. as a definition of a class of elements that are instances of the class or, alternatively, that are "described" by the category. In this sense, such a category violates the actual spirit of semantic Web.

1 Resource Description Framework - http://www.w3.org/RDF/

2 Emerging in the sense of "dynamic" or "in progress".

Let us explain this with an example. The Web site of DBpedia in its section of "Online access" contains some query examples using the SPARQL query language. The first query has the description "People who were born in Berlin before 1900" which actually translates into a graph-based search of entities of the type "Person", which have the property "birthPlace" pointing to the entity representing the "city of Berlin" and another property named "birthDate" with a value less than 1900. We can see here linked data working at "its purest", i.e. the form of the query provides the right-hand side of a definition for "People who were born in Berlin before 1900". Nevertheless, the fourth query named "French films" does not work in the same way. While we could expect also a graph-based search of objects of the type "Film" with maybe a property called "hasCountry" pointing to the entity representing "France", we have a much rougher approach. The actual SPARQL query asks for objects (of any type) annotated as "French films".

In general, categorization systems express "information needs" allowing human entities to quickly access data. French films are annotated as such because there is a need to find them by these keywords. However, for a machine agent this information need is better expressed through a definition, like that provided for the first query (i.e. "People who were born in Berlin before 1900"). Currently, DBPedia mixes these two paradigms of data access in an effort to profit from the structured nature of categories, nevertheless further steps have to be developed to ensure coherence and completeness in data.

3 Notice that in DBPedia the property used to link entities and categories is called "subject". We use "annotation" instead of "subject" to avoid confusions with the "subject" in an RDF triple.

Accordingly, in this work we describe an approach to bridge the gap between the current syntactic nature of categorical annotations with their semantic correspondent in the form of a concept definition. We achieve this by mining patterns derived from entities annotated by a given category, e.g. All entities annotated as "Lamborghini cars" are of "type automobile" and "manufactured by Lamborghini", or all entities annotated as "French films" are of "type film" and of "French nationality". We describe how these category-pattern equivalences can be described as "definitions" according to implication rules among attributes which can be mined using Formal Concept Analysis (FCA [Ganter and Wille, 1999]). The method considers the analysis of heterogeneous complex data (not necessarily binary data) through the use of "pattern structures" [Ganter and Kuznetsov, 2001], which is an extension of FCA able to process complex data descriptions. A concept lattice can be built from the data and then used for discovering implication rules (i.e. association rules whose confidence is 100%) which provide a basis for "subject definition" in terms of necessary and sufficient conditions.

The remainder of this article is structured as follows: Section 2 gives a brief introduction to the theoretical background necessary to sustain the rest of the paper. Section 3 describes the approach used for data completion in the DBpedia knowledge base. Section 4 provides experimental results on four datasets created from DBpedia and a brief discussion over our findings. Finally, Section 5 concludes the paper offering some perspectives over our approach.

2 Preliminaries

Linked Open Data (LOD) [Bizer et al., 2009] is a formalism for publishing structured data on-line using the resource description framework (RDF). RDF stores data in the form of RDF triples represented as xsubject, predicate, objecty. The profile of an RDF triple xs, p, oy is given by pU Y Bq � pU Y Bq � pU Y B Y Lq where a set of RDF triples is an RDF graph, denoted by G. Here, U denotes a set of URI references, B refers to the blank node and L to literals. For the sake of simplicity, in the current study we do no take into account blank nodes pBq. An RDF triple is represented as U � U � pU Y Lq. For convenience, in the following we denote the set of predicate names as P and the set of object names as O. LOD can then be queried and accessed through SPARQL 4 , which is a standard query language for RDF data. SPARQL is based on matching graph patterns (present in the WHERE clause of a query) against RDF graphs. For example, let us consider the SPARQL query given in Listing 1, for all the entities of type Automobile manufactured by Lamborghini, annotated as "Sport_cars" and as "Lamborgh-ini_vehicles", Formal Concept Analysis (FCA) is a mathematical framework used for classification, data analysis, information retrieval and knowledge discovery among other tasks [Carpineto and Romano, 2005]. The basics of FCA can be found in [Ganter and Wille, 1999], but in the following we recall some important definitions. Let G be a set of entities, M a set of attributes, and I � G �M an incidence relation. Actually, in FCA, elements of G are called "objects". In this article we call them "entities" to avoid confusions with "objects" as defined for RDF triples. Then, the relation gIm means that entity g P G has attribute m P M. The triple K � pG,M, Iq is called a "formal context". Two derivation operators, both denoted by 1 , formalize the attribute sharing among entities, and (dually) the entity sharing among attributes:

4 http://www.w3.org/TR/rdf-sparql-query/

A1 denotes the set of attributes shared by all the entities in A and B1 denotes the set of entities having all the attributes in B. The pair pA,Bq is a formal concept of K iff A1 � B and B1 � A, where A is called the "extent" and B is called the "intent" of pA,Bq. Given two formal concepts pA1,B1q and pA2,B2q, a partial-ordering is defined between them as pA1, B1q /K pA2, B2q �æ A1 � A2porB2 � B1q. Then, pA1,B1q is called a subconcept of pA2,B2q (pA2,B2q a superconcept pA1,B1q). The set of all formal concepts in K (denoted by CK) together with the order /K forms the concept lattice denoted by LK.

For example, consider the formal context in Figure 1 where G � U, M � pP � Oq and pu, pp, oqq P I �æ xu, p, oy P G, i.e. xu, p, oy is a triple built from different triples manually extracted from DBpedia about nine different Lamborghini cars (35 RDF triples in total). Given a subject-predicateobject triple, the formal context contains subjects in rows, the pairs predicate-object in columns and a cross in the cell where the triple subject in row and predicate-object in column exists.

Figure 1 depicts the concept lattice in reduced notation calculated for this formal context and contains 12 formal concepts. Consider the first five cars (subjects) in the table for which the maximal set of attributes they share is given by the first four predicate-object pairs. Actually, they form a formal concept depicted by the gray cells in Figure 1 and labelled as "Islero, 400GT" in Figure 1 (actually, the extent of this concept is "Islero, 400GT, 350GT, Reventon").

Given a concept lattice, rules can be extracted from the intents of concepts which are comparable. For two sets X, Y � M, a rule has the formX øæ Y where X and Y are subsets of attributes. A rule X øæ Y has a support given by the proportion of entities having the set of attributes in X and Y (i.e. |X1 XY 1 |5 ) w.r.t. the whole set of entities, and a confidence which is the proportion of entities having the (same) set of attributes in X and Y , w.r.t. X (i.e. |X1 X Y 1 |{|X1 |). For instance, consider the association rule e øæ a, b, c, d. Its support is given by |ta, b, c, du1 X teu1 | which is the same as |tReventon,Countachu| � 2. Since |teu1 | � 3, we have that the confidence of this association rule is 2{3 � 0.67. A rule X øæ Y of confidence 1 (when |X1 X Y 1 | � |X1 | or X1 � Y 1 ) is called an implication. Otherwise, the confidence is less than 1 and the rule is called an association rule. When X øæ Y and Y øæ X are implications, we say that X �æ Y is an equivalence or a definition. This can happen when two attributes have the same "attribute concept", e.g. type-Automobile and manufacturer-Lamborghini in the concept lattice of Figure 1.

3 Improving DBpedia with FCA

3.1 Problem context

Consider the following fictional scenario. You are a bookkeeper in a library of books written in a language you do not understand. A customer arrives and asks you for a book about "Cars". Since you do not know what the books are about (because you cannot read them), you ask the customer to browse the collection on his own. After he finds a book he is interested to read, you will mark the symbol � on that book for future references. Then, in an empty page you will write (� - Cars). After several cases like this, you will probably end up with a page full of symbols representing different topics or categories of your books, among them (a - Sports), (� - Football) and (� - History). Now you can even combine symbols when customers ask you for "Sport Cars" which you translate into �a. Actually, the demand for books about "Sport Cars" is so high that you create a new symbol : just for it. So doing, you have created your own categorization system of a collection of books you do not understand.

5 | � | denotes set cardinality.

In general, given a topic, you are able to retrieve books without many troubles, however since you do not understand the books, you are restricted to the set of symbols you have for doing this. Furthermore, if you are not careful some problems start to arise, such as books marked with � and without a. Finally, people do not get books marked with : when they look for "Cars", since they only search for the symbol a.

It is easy to stablish an analogy on how DBpedia profits from Wikipedia's categorization system and the above scenario. DBpedia is able to retrieve entities when queried with an annotation (as the example of "French films"), however any information need not initially provided as a category is unavailable for retrieval (such as "French films about the Art Nouveau era"). Incoherences in categorical annotations are quite frequent in DBpedia, for example there are over 200 entities annotated as "French films" which are not typed as "Films". Finally, DBpedia is not able to provide inferencing. For example, in Figure 1, the entities Veneno and Aventador, even though they are annotated as "Lamborghini vehicles", cannot be retrieved when queried simply by "vehicles". In such a way, it is exactly as if they were marked with a symbol such as :.

3.2 The completion of DBpedia data

Our main concern in this case lies in two aspects. Firstly, are we able to complete data using logical inferences? For example, can we complete the information in the dataset by indicating that the entities "Estoque" and "Gallardo" should be categorized as "Lamborghini vehicles" and "Sport cars"?

Secondly, are we able to complete the descriptions of a given type? For example, DBpedia does not specify that an "Automobile" should have a "manufacturer". In the following, we try to answer these two questions using implications and association rules.

Consider rules provided in Table 2. Of course, the first three implications are only true in our dataset. This is due to the fact that we use the "closed world" assumption, meaning that our rules only apply in "our world of data" where all cars are of "Lamborghini" brand, i.e. all other information about cars that we do not know can be assumed as false [Fürber and Hepp, 2011]. While these implications are trivial, they provide a good insight of the capabilities of our model. For instance, including a larger number of triples in our dataset would allow discovering that, while not all automobiles are manufactured by Lamborghini, they are manufactured by either a Company, an Organization or an Agent. These three classes 6 are types of the entity Lamborghini in DBpedia. Such a rule would allow providing a domain characterization to the otherwise empty description of the predicate "dbo:manufacturer" in the DBpedia schema.

The association rule given in the fourth row in Table 2 shows the fact that 29% of the subjects of type "Automobile" and manufactured by "Lamborghini" should be categorized by "Sports cars" and "Lamborghini vehicles" to complete the data. This actually corresponds to the entities "Estoque" and "Gallardo" in Figure 1. Based on this fact, we can use association rules also to create new triples that allow the completion of the information included in DBpedia.

3.3 Pattern structures for the completion process

The aforementioned models to support linked data using FCA are adequate for small datasets as the example provided. Actually, LOD do not always consists of triples of resources (identified by their URIs) but contains a diversity of data types and structures including dates, numbers, collections, strings and others making the process of data processing much more complex. This calls for a formalism able to deal with this diversity of complex and heterogeneous data.

Accordingly, pattern structures are an extension of FCA which enables the analysis of complex data, such as numerical values, graphs, partitions, etc. In a nutshell, pattern structures provide the necessary definitions to apply FCA to entities with complex descriptions. The basics of pattern structures are introduced in [Ganter and Kuznetsov, 2001]. Below, we provide a brief introduction using interval pattern structures [Kaytoue et al., 2011].

6 In the OWL language sense.

Let us consider Table 3 showing the predicate dbo:productionStartYear for the subjects in Figure 1. In such a case we would like to extract a pattern in the year of production of a subset of cars. Contrasting a formal context as introduced in Section 2, instead of having a set M of attributes, interval pattern structures use a semilattice of interval descriptions ordered by a subsumption relation and denoted by pD,�q7 . Furthermore, instead of having an incidence relation set I, pattern structures use a mapping function δ : G � D which assigns to any g P G the corresponding interval description δpgq P D. For example, the entity "350GT" in Table 3 has the description δp350GT q � xr1963, 1963sy.

Let us consider two descriptions δpg1q � xrl1 i , r1 i sy and δpg2q � xrl2 i , r2 i sy, with i P r1..ns where n is the number of intervals used for the description of entities. The similarity operation [ and the associated subsumption relation � between descriptions are defined as the convex hull of two descriptions as follows:

Finally, a pattern structure is denoted as pG, pD,�q, δq where operators p�ql between ℘pGq and pD, �q are given below:

An interval pattern concept pA, dq is such as A � G, d P D, A � dl, d � Al. Using interval pattern concepts, we can extract and classify the actual pattern (and pattern concepts) representing the years of production of the cars. Some of them are presented in the lower part of Table 3. We can appreciate that cars can be divided in three main periods of time of production given by the intent of the interval pattern concepts.

7 It can be noticed that standard FCA uses a semi-lattice of set descriptions ordered by inclusion, i.e. (M,�).

3.4 Heterogeneous pattern structures

Different instances of the pattern structure framework have been proposed to deal with different kinds of data, e.g. graph, sequences, interval, partitions, etc. For linked data we propose to use the approach called "heterogeneous pattern structure" framework introduced in [Codocedo and Napoli, 2014] as a way to describe objects in a heterogeneous space, i.e. where there are relational, multi-valued and binary attributes. It is easy to observe that this is actually the case for linked data where the set of literals L greatly varies in nature depending on the predicate. For the sake of simplicity we provide only the most important details of the model used for working with linked data.

When the range of a predicate (hereafter referred to as "relation") p P P is such that rangeppq � U, we call p an "object relation". Analogously, when the range is such that rangeppq � L, p is a "literal relation". For any given relation (object or literal), we define the pattern structure Kp � pG, pDp,[q, δpq, where pDp,�q is an ordered set of descriptions defined for the elements in rangeppq, and δp maps entities g P G to their descriptions in Dp. Based on that, the triple pG,H,∆q is called a "heterogeneous pattern structure", where H � � Dppp P Pq is the Cartesian product of all the descriptions sets Dp, and ∆ maps an entity g P G to a tuple where each component corresponds to a description in a set Dp.

For an "object relation", the order in pDp,�q is given by standard set inclusion and thus, the pattern structure Kp is just a formal context. Regarding "literal relations", such as numerical properties, the pattern structure may vary according to what is more appropriate to deal with that specific kind of data. For example, considering the predicate dbo:productionStartYear discussed in the previous section, Kdbo:productionStartYear should be modelled as an interval pattern structure. For the running example, the heterogeneous pattern structure is presented in Table 4. Cells in grey mark a heterogeneous pattern concept the extent of which contains cars "350GT, 400GT, Islero". The intent of this heterogeneous pattern concept is given by the tuple pta, bu, tcu, tdu, xr1963, 1967syq, i.e. "Automobiles manufactured by Lamborghini between 1963 and 1967". The model of heterogeneous pattern structures is the basis of the experiments which are presented in the next section.

Table 5: Summary table of experimental procedures. Upper table shows the predicates used to construct each datasets. Properties without a prefix have the default namespace "dbo:". Underlined properties have numerical ranges. Middle table show each dataset characteristics. Lower table shows experimental results. P@20 stands for "Precision at the first 20 implication".

4 Experimentation

To evaluate our model, four datasets were created from DBpedia, namely "Cars", "Videogames", "Smartphones" and "Countries" (see characteristics of the datasets in Table 5). Each dataset was created using a single SPARQL query with a unique restriction (either a fixed subject or a fixed type). A dataset consists of a set of triples whose predicate is given by the properties in Table 5. The heterogeneous aspect of data is illustrated by the fact that in two of the four datasets there are properties with numerical ranges.

For each dataset we calculated the set of all implications derived from the heterogeneous pattern concept lattice. Each rule of the formX øæ Y was ranked according to the confidence of the rule Y øæ X (the latter is referred as the "inverted rule" of the former). Thus, given that implications have always a confidence of 100%, the confidence of the inverted rule tells us how close we are from a definition, i.e. X �æ Y or both rules are implications. Having an implication or not leads to the decision whether a set of RDF triples should be completed or not. For example, the following implication from the Cars dataset has an inverted rule of 92% of confidence:

rdf:type-dbo:MaseratiVehicles øæ dbo:manufacturer-dbp:Maserati

Accordingly, we can make of this implication a definition stating that the remainder 8% of the entities manufactured by Maserati should also be "typed" as MaseratiVehicles (recall in here that we have constructed our "world of data" by taking all "Sport Cars" from DBpedia, thus things built by Maserati which are not vehicles do not belong in our data). Of course, there are cases in which this will not be true. For example with a 90% of confidence in the opposite direction we have the implication:

dbo:layout-dbp:Quattro øæ dbo:manufacturer-dbp:Audi

The creation of a definition from this rule (i.e. making the remainder 10% of the cars manufactured by Audi have a layout 4x4) would be wrong. While we expect that the high confidence of the opposite association rule distinguish the case when a definition should be made, a human ruling to include background information will always be needed.

Considering that there is no ground truth for any of the datasets, the reported results are given for assessing the feasibility of our approach. For each of the ranked basis of implications in the experimentation we performed a human evaluation. With the help of DBpedia, it was evaluated if an implication was likely to become a definition. The answer provided for each of the rule was binary (yes or no). For instance, in the previous examples the first implication would render a "yes" answer, while the second, a "no". Afterwards, we measured the precision for the first 20 ranked implications (P@20) as the proportion of the rules that were likely to become a definition (those evaluated as yes) over 20 (the total number of rules taken). Actually, the precision value works as an indicator of how likely implications are useful for RDF data completion (see Table 5).

By contrast, since we do not have a ground truth of all the triples that should be added to each dataset, we are not able to report on recall. Nevertheless, to complement this evaluation, we provide the values of the precision at 11 points (P@11p) [Manning et al., 2008]. We consider each human evaluation as the ground truth for its respective dataset and thus, each list of implications has a 100% recall. Precision at 11 points provides a notion on how well distributed are the answers in the ranking. Figure 2 contains the curves for each of the datasets. Values for precision at 20 points are high for all datasets and particularly for the dataset "Countries". This may be due to the fact that Countries was the only dataset built for resources with a fixed type.

A precision of 0.9 indicates that 9 out of 10 implications can be transformed into definitions by creating RDF triples that would complete the entities descriptions. Precision at 11-points shows that confidence is a good indicator on the usefulness of implications for data completion. For example, regarding the worst result i.e. the Videogames dataset, when the evaluator provides the last "yes" answer for an implication, he/she has also given a "yes" to 6 out 10 (from a total of 46). For our best result (Countries dataset) it is over 8 out of 10. Results show that confidence is a good indicator for the selection of implications in terms of data completion.

Further experimentation should be performed to assess if the triples being created are "correct" or not. As already mentioned, we assume that resources being completed are correctly linked to the implication. While this may not always be true, our approach is still useful under those circumstances given that it would allow discovering such "incorrectly" annotated entities. Finally, regarding execution times, Table 5 shows that even for the larger dataset, the execution time is less than a minute, and this time is perfectly acceptable for

5 Related work, discussion and conclusion

In [Paulheim and Bizer, 2013] authors use an inference mechanism which considers the links between instances to obtain their class types, assuming that some relations occur only with certain classes. Moreover, there are some studies which focus on the correction of numerical data present in DBpedia [Wienand and Paulheim, 2014] using outlier detection method, which identify those facts which deviate from other members of the sample. By contrast, our approach focuses on completing RDF data with the help of association rule mining. In [Zaveri et al., 2013], authors propose a manual and semi-automatic methodology for evaluating the quality of LOD resources w.r.t. a taxonomy of "quality problems". Quality assessment is based on user inputs (crowd-sourcing) and measures the correctness of schema axioms in DBpedia. In [Yu and Heflin, 2011a; 2011b], authors try to detect triples which are regarded as erroneous w.r.t. similar triples. The detection is based on probabilistic rule learning and on the discovery of generalized functional dependencies that are used to characterize the abnormality of the considered triples.

Different interesting perspectives are opened following this work. As we have discussed, categories represent some pre-loaded information needs in Wikipedia, i.e. a pre-answered questions whose answer is relevant for a group of people. Thus, an interesting application would be to translate these information needs into description logics definitions, instead of attributes. It is possible to think that, instead of annotating each French film with a "FrenchFilm" tag, we could define the category as FrenchFilm � Film [ hasCountry.tFRANCEu, or LamborghiniCars � Automobile[manufacturedBy .tLAMBORGHINIu. Given that these definitions are more restrictive than typing (rdf:type), our work should be adapted to deal with "near-definitions" in which both directions (X øæ Y and Y øæ X) are association rules with high confidence. While we have presented our approach applied to DBpedia, its applicability is more general than this specific knowledge resource. Actually, using category taxonomies to annotate resources in LOD is a common practice for several knowledge bases, to the extent that a meta-model for vocabularies (Simple Knowledge Organization System - SKOS) has been proposed by the World Wide Web Consortium 8 . SKOS organizes "Concepts" in a hierarchical taxonomy, while resources are said to be "subjects" of these concepts.

To conclude, in the current study we introduce a mechanism based on association rule mining for the completion of the RDF dataset. Moreover, we use heterogeneous pattern structures to deal with heterogeneity in LOD. Several experiments have been conducted over four datasets and an evaluation was conducted for each of the experiments. This study shows the capabilities of FCA for completing complex RDF structures.

8 http://www.w3.org/2004/02/skos/