Web Intelligence (WI): What Makes Wisdom Web?

1 Been There, Done That

The Web has irrevocably revolutionized the world we live in. This impact is inevitable due to the facts that the Web con­ nectivity rapidly increases and that the online information as­ tronomically explodes. In order not only to live with such a change but also to benefit from the information infrastructure that the Web has empowered, we have witnessed the fast de­ velopment as well as applications of many WI techniques and technologies, which cover:

1. Internet-level communication, infrastructure, and security protocols. The Web is regarded as a computer­ network system. WI techniques for this level include, for instance, Web data prefetching systems built upon Web surfing patterns to resolve the issue of Web latency. The intelligence of the Web prefetching comes from adaptive learning based on observations of user surfing behavior.
2. Interface-level multimedia presentation standards. The Web is regarded as an interface for human-Internet interaction. WI techniques for this level are used to develop intelligent Web interfaces in which the capa­ bilities of adaptive cross-language processing, person-

alized multimedia representation, and multimodal data processing are required.

1. Knowledge-level information processing and man­ agement tools. The Web is regarded as a distributed data/knowledge base. We need to develop semantic markup languages to represent the semantic contents of the Web available in machine-understandable formats for agent based computing, such as searching, aggre­ gation, classification, filtering, managing, mining, and discovery on the Web [3].
2. Application-level ubiquitous computing and social intelligence environments. The Web is regarded as a basis for establishing social networks that contain com­ munities of people (or organizations or other social en­ tities) connected by social relationships, such as friend­ ship, coworking or information exchange with common interests. They are Web-supported social networks or virtual communities. The study of WI concerns the im­ portant issues central to social network intelligence (so­ cial intelligence for short) [7; 15]. Furthermore, the mul­ timedia contents on the Web are not only accessible from stationary platforms, but increasingly from mobile plat­ forms [16]. Ubiquitous Web access and computing from various wireless devices need adaptive personalization for which WI techniques are used to construct models of user interests by inferring from user behavior and ac­ tions [1; 4].

In spite of on-going WI technological advances, it is still rather unclear to many what will be the next paradigm shift in WI. In what follows, I will present new perspectives on this question and at the same time point out some WI research challenges.

2 What's Next?

In the movie Star Wars: Episode II, there is an interesting scene: When Obi Wan Kenobi failed to locate any relevant information about a mysterious planet (where later he discov­ ered the clone manufacturing ground), he turned to his friend for advice. His friend, who apparently knew more than the Jedi's academy knowledge banks combined, had the follow­ ing in reply: Other people seek knowledge, but you my friend know wisdom.

The reply in the above scene also provides an answer to our earlier question. The next paradigm shift in WI lies in the notion of wisdom. The goal of the new generation WI is to enable users to gain new wisdom of living, working, playing, and learning, in addition to information search and knowledge queries. Here, the word of wisdom, according to the Webster Dictionary (Page: 1658) [14], implies the following mean­ ings:

1. The quality of being wise; knowledge, and the capacity to make due use of it; knowledge of the best ends and the best means; discernment and judgment; discretion; sagacity; skill; dexterity.
2. The results of wise judgments; scientific or practical truth; acquired knowledge; erudition.

In the Web context, the manifestation of wisdom can best be illustrated with a minimalist Wisdom Web example.

3 When the Web Offers Practical Wisdom ...

Imagine that this is your first trip to the city of Montreal. You would like to find a really nice place to spend your evening. So, you walk into a Cyber Cafe on Sherbrook Street (the only street that you can recognize), and decide to get some prac­ tical wisdom from a public Wisdom Web outlet. You log in with a user name, "Spiderman", and ask:

"What is the best night life in Montreal during this season of the year?"

The Wisdom Web thinks for about a second or two and then responds:

"Spiderman, the hockey games are on during this season of the year. Would you like to go?''

You reply:

"Yes."

Then the Wisdom Web suggests:

"As far as I know, there are still some tickets left and you may purchase some at the Montreal Forum. It is easy to get there by taking Metro to the Atwater station."

Now you decide that this could be an interesting evening for you ...

One hour later, you arrive at the ticket office by Metro, but surprisingly find that the tickets left are all for the day after tomorrow when you will be traveling in Quebec City.

As you are a bit disappointed, you notice that there is a free Wisdom Web Kiosk right beside the ticket office. Well, that's convenient. So, without too much hesitation, you log on to the Wisdom Web, again as "Spiderman". The Wisdom Web still remembers your conversations an hour ago. As soon as it recognizes that you are "Spiderman", it says to you:

"Hello Spiderman, you were in such a hurry last time that I couldn't have a chance to tell you that all tickets available here are only for the day after tomorrow. They are quite expensive too ... "

4 10 Capabilities of the Wisdom Web

In order to make the above Wisdom Web scenario a reality, the following 10 fundamental capabi lities have to be incorpo­ rated and standardized:

1. Self-organizing servers: The Wisdom Web will au­ tomatically regulate the functions and cooperations of related Websites and application services available. A Wisdom Web server automatically self-nominates to other services its functional roles as well as correspond­ ing spatial or temporal constraints and operational set­ tings.
2. Specialization: A Wisdom Web server is an agent by itself, which is specialized in performing some roles in a certain service. The association of its roles with any service will be measured and updated dynamically, for instance, the association may be forgotten if it is not used for some time.
3. Growth: The population of Wisdom Agents will dy­ namically change, as new agents are self-reproduced by their parent agents in order to become more specialized, or aged agents are deactivated.
4. Autocatalysis: As various roles of wisdom agents are created through specialization and activated by the Wis­ dom Search requests, their associations with some ser­ vices and among themselves must be autocatalytically aggregated. In this respect, the autocatalysis of asso­ ciations is similar to the pheromone laying for positive feedback in an ant colony.
5. Problem Solver Markup Language (PSML): PSML is necessary for wisdom agents to specify their roles and settings as well as relationships with any other services.
6. Semantics: The Wisdom Web needs to understand what are meant by "Montreal", "season", "year", and "night life", and what is the right judgment of "best", by under­ standing the granularities of their corresponding subjects and the whereabouts of their ontology definitions.
7. Metaknowledge: Besides semantic knowledge extracted and manipulated in the Wisdom Search, it is also essential for wisdom agents to incorporate a dynamically created source of metaknowledge that deals with there­ lationships between concepts and the spatial or tempo­ ral constraint knowledge in planning and executing ser­ vices. It allows agents to self-resolve their conflict of interests.
8. Planning: In the above example, the goal is to find a function or an event that may sound attractive to a visi­ tor. The constraint is that they must be happening during this season. There are involved two associated subgoals: In order to have an access to the recommended function or event, one needs a ticket. Further, in order to go to get the ticket, one can travel by Metro. In the Wisdom Web, ontology alone will not be sufficient.
9. Personalization: The Wisdom Web remembers there­ cent encounters and relates different episodes together, according to (I) "Spiderman", (2) time, and (3) attain­ ability of (sub)goals. It may further identify other goals

as well as courses of actions for this user as their conver­ sation goes on.

10. A sense of humor: Although the Wisdom Web does not explicitly tell a funny story, it adds some punch lines to the situation or anxiety that "Spiderman" is presently in when he/she logs on for the second time, which will make "Spiderman" feel absurd.

5 Research Challenges

In the preceding sections, I have stated the goal and vision of the new generation WI, and what constitutes a Wisdom Web. In this section, I will point out some WI research challenges.

5.1 Problem Solver Markup Language (PSML)

One practical Wisdom Web issue that has to be addressed is how to integrate distributed and centralized information sources/structures. Information/knowledge on the Web can be either globally distributed throughout the Web or locally hosted, centralized on an intelligent portal providing Web ser­ vices (i.e., an intelligent service provider) that is integrated to its own cluster of specialized intelligent applications. How­ ever, each approach has a serious flaw. As pointed out by Alesso and Smith [1], the intelligent portal approach limits uniformity and access, while the global semantic Web ap­ proach faces combinatory complexity limitations.

A way to solve the above issue is to develop and use the Problem Solver Markup Language (PSML) for collect­ ing globally distributed contents and knowledge from Web­ supported, semantic social networks and incorporating them with locally operational knowledge/databases in an enterprise or community for locally centralized, adaptable Web intelli­ gent services.

The core of PSML is based on distributed inference en­ gines that can perform automatic reasoning on the Web by in­ corporating contents and metaknowledge, autonomically col­ lected and transformed from the semantic Web, with locally operational knowledge/databases. A feasible way to work with PSML is to use existing (Prolog-like) logic languages, together with dynamic contents and metaknowledge collec­ tion and transformation agents.

By using this information transformation approach, the dy­ namic, global information sources on the Web can be com­ bined with the local information sources in an enterprise por­ tal for decision making and business intelligence.

Ohsuga et al. previously used KAUS for representing local information sources and for performing inferences and rea­ soning. KAUS is a knowledge based system that involves Multi-Layer Logic and databases based on the relational data model [13]. KAUS enables the representations of knowledge and data in the first order logic form with multi-layer data structures. It can readily be used for inferences and reason­ ing as well as for transforming and managing both knowledge and data.

5.2 Automatic Ontology Construction

Ontology is the basis for designing and implementing the Se­ mantic Web for Web intelligence. It plays a crucial role in enabling Web based knowledge processing and management, such as Web community communication, semantics based agent communication, knowledge based Web retrieval , Web content understanding, and Web community discovery.

Although ontology engineering has been studied over the last decade, there is still a demand for more automatic methods in comprehensive ontology construction and learn­ ing [20]. The tedious, cumbersome task of existing manual ontology construction can easily become a bottleneck in the development of semantics based Web intelligence.

5.3 Wisdom Web Based Computing Paradigm

The paradigm of Wisdom Web based computing is aimed at providing not only a medium for seamless information ex­ change and knowledge sharing but also a type of man-made resources for sustainable knowledge creation, and scien­ tific and social evolution. The Wisdom Web will reply on grid-like service agencies that self-organize, learn, and evolve their courses of actions in order to perform service tasks as well as their identities and interrelationships in communities. They will also cooperate and compete among themselves in order to optimize their as well as others resources and utili­ ties.

Self-organizing, learning agents are computational entities that are capable of self-improving their performance in dy­ namically changing and unpredictable task environments. In [8], Liu has provided a comprehensive overview of several studies in the field of autonomy oriented computing, with in-depth discussions on self-organizing and adaptive tech­ niques for developing various embodiments of agent based systems, such as autonomous robots, collective vision and motion, autonomous animation, and search and segmentation agents. The core of those techniques is the notion of synthetic or emergent autonomy based on behavioral self-organization.

5.4 A New Turing Test

In order to effectively develop the new generation WI sys­ tems, we need to define benchmark applications, i.e., a new Turing Test, that will capture and demonstrate the Wisdom Web capabilities.

Take the Wisdom Web based computing benchmark as an example. We can use a service task of compiling and generat­ ing a market report on an existing product or a potential mar­ ket report on a new product. In order to get such service jobs done, an information agent on the Wisdom Web will mine and integrate available Web information, which will in turn be passed onto a market analysis agent. Market analysis will involve the quantitative simulations of customer behavior in a marketplace, instantaneously handled by other serviced agen­ cies, involving a large number of grid agents [2]. Since the number of variables concerned may be in the order of hun­ dreds or thousands, it can easily cost a single system years to generate one predication.

6 Recent Initiatives

In this section, I will describe two recent WI research ini­ tiatives: (1) information foraging agents and (2) ubiquitous agent communities.

6.1 Information Foraging Agents

One of the important roles of wisdom agents is concerned with searching and filtering information from distributed Web sources. In this respect, understanding and developing the right information foraging behaviors for wisdom agents will become a real challenge. In order to tackle this challenge, the first step is to have a clear picture of how humans search in­ formation, what kind of human behavior dominates, and what is the origin of Web usage regularities. There have been few efforts on describing various aspects of human Web surfing regularities [5; 6]. However, the origin and interrelated vari­ ables of the regularities still remain unknown.

Liu and Zhang have developed a model for information for­ aging agents on the Web [11]. The foraging agent model takes into account Web topology, information distribution, and agent interest profile. They have validated this model using empirical Web log datasets, and used it to explain the origin of regularities in Web surfing. In particular, they have discovered that it is the unique distribution of agent interest that leads to the regularities in agent surfing behavior, i.e., a power law distribution of agent surfing depth. The Web topology can only influence the shape parameters of the dis­ tribution without changing the nature of the distribution. Also discovered is that the power law of link click frequency is largely due to agent purposeful surfing behavior. This work shows that Web regularities are interrelated.

Also in their studies, they have studied three categories of foraging agents, according to their interests and familiarities with the Web: Random agents who have no obvious inten­ tion in Web surfing, rational agents who have certain goals to achieve but are not familiar with the Web structure, and recurrent agents who have certain specific intents and are very familiar with the Web structure. The ability to predict the contents at the next level nodes becomes stronger when moving from random to recurrent agents. Experimental re­ sults with respect to the three agent categories have unveiled that the regularities of agent surfing depth on pages and do­ mains still remain the same, while a power law of link click frequency distribution will disappear as we move from recur­ rent agents to random agents. This result shows that the order existing in link click frequency comes from agent's content prediction ability, that is whether or not an agent can deter­ mine the next step according to its own interest and current information.

6.2 Ubiquitous Agent Communities

One of the key features in wisdom agents is ubiquity. Ubiq­ uitous agents are autonomous entities that automatically ex­ plore and exploit WI-based Web services. There have been some earlier work performed in the domains of e-business and e-commerce. Related issues and applications in which various tasks and objectives were achieved can be found in [9].

In this area of research, a reasonable challenge to begin with would be to develop and demonstrate a Ubiquitous Agent Community (UAC), i.e., an intelligent infrastructure that enables agents to look ahead, plan, and deliver what a user wants [ 12]. It works like a personal agency. For in­ stance, it can help a user to effectively manage tedious daily routine activities, such as processing emails, placing orders, making meeting arrangements, downloading news, etc.

A UAC can interact with its users in various ways. For instance, as a person uses a smart handheld device to enter a subway station on his/her way to work, a UAC can seamlessly upload a list of things to be done to the person's handheld device. Along with each of the things listed, the informa­ tion provided by the UAC can also include its corresponding tasks necessary to be carried out. Thus, inside the moving subway train, he/she will be able to go through each of the recommended task items and to further verify and delegate certain jobs to a community agent (c-agent) in the UAC, such as forwarding a report or making a meeting appointment. In addition to planning and executing the items on a calendar, the user can also receive other personalized services based on his/her profile, which may include sports news, urgent emails, and specific documents prepared for the day's meetings.

Figure 1 gives an illustrative example of personal services that can be provided by a community of competing and coop­ erating agents, i.e., a Ubiquitous Agent Community (UAC).

A UAC contains two types of agents: community agents (c-agents) and service agents (s-agents). The former makes task plans as well as contracting decisions on what and where some tasks will be carried out, whereas the latter performs the delegated tasks. In the illustrative example of Figure I , suppose a user writes a few words on his/her PDA regarding what he/she plans to do. A c-agent in the distributed UAC will recognize the intention of the user and find similar ser­ vice cases from its case base. Thereafter, it will perform task planning based on the similarity between the task at hand and the previous tasks. Once the c-agent completes its task plan­ ning, a cluster of s-agents will be called upon to distribute and carry out the planned subtasks.

In the UAC, s-agents will not act alone, but work coordi­ nately, in order to accomplish their subtasks. Coordination is one of the most important characteristics of UAC agents. The coordination of UAC agents is essentially the coordination of their ubiquitous service actions.

Take a buyer-seller market community as an example. The goal of each agent is to maximize its own interest; agents are antagonistic in a certain sense. At the same time, the market exists on the basis of buyer-seller competitions. In this case, the coordination of the market agents is reflected in their com­ petitions for the best prices.

Generally speaking, the coordination of UAC agents in­ volves both competition and cooperation. UAC agents han­ dle their services in two main stages: task allocation and task execution. As mentioned above, a service task often consists of several subtasks. Each of the subtasks will be allocated to a specific s-agent. Thus, as far as the task allocation is con­ cerned, various s-agents will compete among themselves in order to get a subtask assignment. In this case, coordination manifests itself as competition. On the other hand, during the course of task execution, an s-agent will require assistance from other s-agents in order to effectively fulfill the task at hand. Thus, coordination also manifests itself as cooperation, as s-agents exchange information and services among each other. In many ways, the technical issues of implementing UAC can be reduced to those of implementing coordination among ubiquitous agents.

7 Conclusions

I have started this paper by giving perspectives on the new generation WI. The Web, as an information infrastructure, will remain to create a strong impact in the future by offering world-wide presence and connectivity. The major challenge of the Web research and development in the next decade will be WI-centric, focusing on how we can intelligently make the best use of the widely available Web connectivity. The new WI technologies to be developed will be precisely determined by human needs in a post-industrial era; namely:

1. information empowerment;
2. knowledge sharing;
3. virtual social clustering;
4. service enrichment;
5. practical wisdom development.

The next paradigm shift in WI will be towards the notion of wisdom. Developing the Wisdom Web will become a tangi­ ble goal for WI research. The new generation WI will enable humans to gain wisdom of living, working, playing, in addi­ tion to information search and knowledge queries.

I believe that our current efforts will pave the way toward the next biggest technological invention of this century, the Wisdom Web, whose significance and impact will be equiva­ lent to, or even far beyond, those of Alexander Graham Bell's invention of the telephone in 1876.

The Web Intelligence Consortium (WIC)

To meet the strong demands for participation and the growing interests in WI, the Web Intelligence Consortium (WI C) was formed in spring 2002. The WIC (http://wi-consortium.org/) is an international non-profit organization dedicated to pro­ moting world-wide scientific research and industrial devel­ opment in the era of Web and agent intelligence. Its activi­ ties include collaborations with WI research centers through­ out the world and organization/individual members, technol­ ogy showcases at IEEEIWIC conferences and workshops, and WIC publications.

In addition to various special issues on WI published by several international journals, including IEEE Computer, a WI-focused scientific journal, Web Intelligence and Agent Systems: An International Journal (http://wi­ consortium.org/journal.html), has been successfully launched as the official journal of the WI C.

Acknowledgments

I would like to thank Ning Zhong and Yiyu Yao for their con­ tinuing collaborations in the development of Web Intelligence (WI). The contents of this paper include their contributions. I am very grateful to people who have joined or supported the WI community, in particular, the WIC Advisory Board members: Edward A. Feigenbaum, Setsuo Ohsuga, Benjamin Wah, Philip Yu, and Lotti A. Zadeh, all the WIC Technical Committee Members, as well as the IEEE TCCI Chair, Xin­ dong Wu. I thank them for their strong support.