Swarm Systems in the Visualization of Consumption Patterns

1 Introduction

With the advance of technology, more information is being generated and stored. This burst of information has enabled the access to unprecedented amounts of data in new domains, with increased speed and variability [Zhang et al., 2013]. Data Visualization is emerging as a tool to comprehend and explore all of this new data. Through visualization, we can interpret, synthesise, and present complex and vast amounts of information in various ways [Tory and Moller, 2004]. Although Data Visualization has its roots in scientific reasoning and is traditionally seen as an analytical tool, the creation of programming languages directed to the visual design communities, and the democratisation of data, expanded its conceptual boundary to a more aesthetic practice. Information Aesthetics is an emerging field that analyses data and creates new media works based on the beautification of information processing [Manovich, 2000]. In other words, Information Aesthetics works are situated in the intersection of functional Data Visualization and the fine arts [Moere, 2007], which are more subjective by nature.

Creating appropriate new visual metaphors and new ways to manipulate big datasets of information are fundamental challenges for Data Visualization and Information Aesthetics. Besides, the development of visualization models that are able to continuously readjust to the changes of dynamic data and to the intentions of the user [Ishizaki, 1996] are also a key issue. To address these challenges, Data Visualization gets inspiration from various fields and adopts some of their techniques. The appropriation of Multi-Agent Systems (MAS) by Data Visualization is one of those cases. MAS is a sub-field of computer science and can be described as a computerised system composed of multiple interacting intelligent agents within an environment [Wooldridge, 2009]. This technique is mostly applied through the implementation of swarming agents to represent datasets changing over time.

In this paper, we give an overview of the applicability of Swarm Systems in Data Visualization. Then, we give an overview of what is Information Aesthetics and present an exploration of the application of a Swarm System in this field to represent data regarding the consumption in 729 Portuguese hypermarkets. The goals for this work are: (i) to visually explore the consumption evolution over time; (ii) to detect possible periodic behaviours; and (iii) to explore the boundaries between Data Visualization and Information Aesthetics.

2 Swarm Systems in Data Visualization

Data Visualization provides a powerful way to make sense of data by mapping its attributes into visual properties such as position, size, shape, and colour. The earlier Data Visualizations were static representations but, with the advance of technology, they started to be generated through computational processes and to be more dynamic and interactive. Data Visualization uses different techniques to present data. Some of these techniques are based on MAS and are commonly used in interactive visualizations. With MAS, it is possible to create a diverse series of visualizations, from simple graphs to more sophisticated agent-based geographic representations [Roard and Jones, 2006]. The simulation of swarming and flocking behaviours [Reynolds, 1987] stimulated the interest of scientists, designers, and artists for two main characteristics: selforganization and emergence [Jones, 2007].

The use of Swarm Systems in an artistic context was explored by artists and researchers such as Mauro Annunziato, Jon McCormack, Tim Barrass, Daniel Shiffman, Alice Eldridge, Gary Greenfield, Christian Jacob, Penousal Machado, Nicolas Monmarch´e, Paulo Urbano and Yann Semet [Greenfield and Machado, 2014], for the creation of a wide variety of static and interactive artworks. For instance, in Swarm by Daniel Shiffman [Shiffman, 2004] -presented at SIGGRAPH 2004- the organic paths created by the virtual boids where used to produce a non-photorealistic rendering of live video input. Swarming techniques were also used for visualization purposes. In In-Formation Flocking, Andrew Vande Moere and Andrea Lau used a Swarming System to group and visualize similar data entities without the need for supervision. In-Formation Flocking is able to create dynamic patterns and can represent volatile and chaotic time-varying datasets while sustaining a comprehensible representation in a general level as well as revealing more detailed patterns. Michael Ogawa and Kwan-Liu Ma [2009] created a swarm visualization application. With this visualization, the authors generated a series of visualization videos representing the history and evolution of software development. We Feel Fine [Kamvar and Harris, 2011] aggregates signifiers of emotion in social media, and visualizes emotional statements as animated coloured particles. We Feel Fine conveys the general mood through the colour of the particles.

3 Information Aesthetics

Since Data Visualization has its roots in scientific reasoning, it has traditionally been viewed as an analytical tool. It aims to represent data graphically in order to convey knowledge to the reader and, therefore, to enable him/her to detect patterns and formulate some conclusions over the visual form [Moere, 2007]. However, with the evolution of computer graphics and the democratisation of data sources, artists and designers expanded the conceptual horizon of Data Visualization to an artistic practice [Vi´egas and Wattenberg, 2007]. Information Aesthetics is an emerging field that analyses data and creates new media works that beautify information processing [Manovich, 2000]. Therefore, Information Aesthetics lays in the boundary between Data Visualization and Computational Art. As such, the aesthetics and artistic qualities of the artifacts often become more relevant and prominent than its functional properties [Moere, 2007].

A thorough overview of the field of Information Aesthetics is beyond the scope of this paper. Nevertheless, considering that the work presented herein embraces this field, we present a short survey of works in this area. In 2009, Jer Thorp created a visualization of the UKs National DNA Database (NDNAD). With the use of a Perlin Noise, and the representation of each DNA with a single coloured dot, Thorp generated a single continuous strand that filled up a certain area size, creating a tangle like visuals [Thorp, 2009]. The Project Out of Statistics: Beyond Legal, created by Rebecca Xu and presented at SIGGRAPH 2009, is a series of abstract digital prints based on the latest crime statistics in the United States. This project employs an aesthetic-oriented approach rather than a more conventional visualization technique. The final output are images with a dual purpose: poetic/aesthetic, but with underlying information encoded. The authors expect that the pleasing visuals catch the eye of the viewers, engaging them sufficiently so that they can decode the visualization with the aid of the legend [Xu and Zhai, 2009]. In 2011, Frederik de Wilde created a series of images, the Numer1cal Recipe Series [NRS], to emphasise the fact that we live in a time where large amounts of data are constantly presented to us in various ways. Wilde's intent is to study how the merging of art and data can provide a mirror of society by giving data "ears and eyes". Wilde refers to these series as digital landscapes, electronic shadows and subjective representations of our environment and our lives [de Wilde, 2011].

4 Visualizing Consumption Patterns

We apply a swarm based system as a method to create emergent visualizations of data that convey meaningful information and, at the same time, explore the boundaries between Data Visualization and Information Aesthetics. We visualize the consumptions in 729 supermarkets and hypermarkets of the SONAE chains, which are the largest in Portugal, during two years, from May 2012 to April 2014. We focused on the application of a swarm system in this particular area; however, our approach can be used in other time-dependent datasets.

We implement a Swarm System, constituted by several boids in an environment, that reacts to the evolution of consumption over time (Figure 1). In this system, each boid represents the consumptions in one of the seven Departments of the product hierarchy of the hypermarkets. Through the three rules proposed by Craig Reynolds, i.e. cohesion, separation, and alignment, the boids interact with each other. These rules have different forces applied to them depending whether two interacting boids represent the same Department or not. Besides, each boid has a radius property which affects how distant it must be from the other boids. To give a spacial order to the boids movement, we implement a target boid which should be followed by all the others. This boid describes a spiral in the canvas and therefore, the boids create also a spiral with their paths. To represent the evolution of the consumption values over time, we defined that each lap of the spiral represents one month. So, as the boids move the time is incremented and the radius of the boids vary according to the consumption at each time interval. We represent these changes over time through two methods: (i) with circles centred in the position of the boids and with the radius changing depending on the consumption; and (ii) by connecting the centre of every boid of the same Department with a line.

With this system we intend to show consumption patterns and detect periodical behaviours. We want to explore the qualitative representations of data, instead of a quantitative one, as we are not interested in exact numbers but in an overview of how data behaves through time. With the use of a Swarm System this project gives the machine some liberty in the creation of the visuals. Our main goals for this project are: (i) to visually explore the consumption evolution over time; and (ii) to represent possible periodic behaviours.

We are interested in stimulating the user to explore the data visualizations and detect the periodicity of the behaviour patterns from the visual inspection of the visualizations. In this context, we are not applying any algorithmic technique to automatically detect or highlight periodic behaviours.

We present the different aspects of the project: the data, the Swarm System, and the different renderings. The section pertaining the data details the characteristics of the dataset used to create the visualizations. Then, we describe the Swarm System, defining the boids' behaviour, their interactions, and their relation with the consumption data.

4.1 Data

The data consists of the consumptions in 729 Portuguese supermarkets and hypermarkets of the SONAE chains, who cover the entire country. When shopping in these chains costumers tend to use their client cards to accumulate discounts and other benefits. Currently, the number of active cards is above 6 million, which can be considered an impressive number, specially if we take into consideration that the Portuguese population is below 11 million, and that the cards are issued by "household", and shared by the entire family. We choose this dataset due to its richness, size, quality and nature. We believe that the dataset is a valuable asset of the work, offering us the opportunity to transform the consumption patterns of the Portuguese into aesthetic artifacts, while exploring, highlighting and visualizing their periodic nature.

We analysed all the transactions made on those supermarkets and hypermarkets from May 2012 to April 2014. Each transaction corresponds to one product bought and it has properties such as price, date, and time of its purchase. Each product is placed in the product hierarchy of the company, which has 6 levels. For this work, we aggregate all the purchases by the highest level in the hierarchy, i.e. the Department. There are a total of seven different Departments in the dataset: Grocery (biscuits, cereals, frozen foods, hygiene, and cleaning products), Fresh Food (fresh meat, fish, vegetables, and fruits), Food & Bakery (bread, cakes, and coffee), Home (household essentials), Leisure (books, office supplies, pet care, and bricolage), Textile (clothing), and Health (products from nutrition to beauty). Each transaction has the hour, minute, and second of the purchase. However, the representation with this degree of detail would be too subtle to the human eye. Therefore, we choose to aggregate the consumptions in intervals of two hours, resulting in 12 intervals per day. Once represented in this format, the size of the data corresponds to, roughly, 289 GB.

4.2 Swarm System

To visualize the consumption habits over time, we implement a Swarm System, which simulates the behaviour of multiple boids. Each boid is characterised by properties such as velocity, position, size, and colour. This last property identifies the type of the boid, i.e. one of the seven Departments.

Based on the work of Reynolds, each boid follows the three basic rules: (i) cohesion, they remain close to nearby boids; (ii) separation, that makes them avoid collisions with nearby boids and other objects; and (iii) alignment, they match the velocity of the nearby boids. Additionally, we define different force values for these three rules and, in each rule, different forces depending on the neighbour's characteristics. If the neighbour's type is different from the current boid's type, the repulsion force is bigger, and the alignment and cohesion smaller. If the neighbour's type is equal to the current boid's type, the separation is smaller, and the alignment and cohesion bigger. This makes the boids seek for others of the same type, and hence to cluster by type, and keep slightly away from boids of a different type. The separation forces between two boids are directly proportional to the sum of their separation radii, which prevents boids from overlapping in space.

Figure 3: Definition of the Departments' colour.

To prevent the boids from randomly moving on the canvas, i.e. the environment, we add a target boid for them to seek. Thus, in addition to the previously described forces, all boids are under the influence of an attraction force towards this moving target. This target boid starts from the centre of the canvas and swirls around, creating a spiral with equal distance between each lap. The steering boid is not represented on the final visualizations so that it is not confused with the boids representing consumptions. Since we want to represent a time-series data, time representation must be added to the visualization. To do so, we consider that each lap represents one month of data. We divide one lap (2π) by 31 days multiplied by 12, since, as previously stated, the data is aggregated in intervals of two hours. Note that we define all laps to have 31 days. By doing so, all months start at the same angle, and if they have less than 31 days the consumption values during the nonexistent days are null (see Figure 2).

To represent the consumption in each Department over time, we vary the size of the boid according to the consumption value in the current time mark. The radius can be mapped between the minimum and maximum sale of each individual Department or by the minimum and maximum sale of all Departments. With these two possibilities, we can compare the different behaviours of each Department and see which one has more consumptions over time.

4.3 Rendering

To visually represent the path made by each boid, we define a set of properties that can be altered to explore different visualizations with different levels of expressiveness. These properties change how each boid is represented and, consequently, how the consumption data is depicted. First, we define a different colour for each boid so the viewer can associate it with the different Departments (Figure 3). Then, although the system enables the fine tuning of several rendering and behavioural options, which allow the creation of numerous alternative compositions, we focus on two specific rendering approaches.

In the first approach, we represent each boid with a circle. The system allows this circle to be coloured in two different ways: one uses the colour of the corresponding Department to fill the circle area, and the other uses it to draw the stroke line. Furthermore, the circles have different radii according to the consumption in a specific time. Since the circles can be overlapped, we resort to transparency to enable the identification of the different shapes. We also implement a mechanism which sorts the circles in depth according to their radii, so that the smaller circles are drawn over the larger ones. Thus, the smaller circles are never hidden by the larger ones. The second approach consists in the connection of the boids which represent the same Department using lines. With this approach we are able to visualize the space between the boids.

In addition to these two approaches, the system allows the modification of the forces of separation, alignment, and cohesion depending on the radius of each boid, and, consequently, depending on the consumption value. Changing these values changes how the boids behave and represent the data. If we give a strong force of separation, they will move away. This movement is associated with a moment of high consumption and can be easily recognized by the viewer. If the boids have strong forces of separation and alignment, they will create a zigzagging path, considering that they will try to separate and, at the same time, be closer to each other. The modification of these forces enables us to experiment with different compositions, ones more concerned with the aesthetics of the final result, and others more concerned with the readability of the represented data.

Finally, to help the viewer to analyse the visualizations, we draw a metric system which divides the spiral into 31 days. This eases the visual division of each lap of the spiral in the different days, however it can be hidden if the user wants to.

5 Experimental Results

As previously mentioned, the system allows the modification of behavioural and visual properties. This leads to numerous different possibilities to represent the same data. In addition, the system is prepared to represent different periods of time and different number of Departments.

In this section we summarise the experimental process, wherein different parameter settings and approaches were designed, applied, and analysed. During these experiments, we studied the impact of the parameter settings, as well as of the rule forms, on the equilibrium between readability and aesthetics. These different experimental setups provided different levels of readability and aesthetics.

To illustrate this experimentation, we present some explorations of our system. To give some coherence between the explorations we defined a fixed period of time of 4 months, from October 2012 to January 2013. We chose this period of time because it contains moments of high consumption, such as in Christmas, and more calm periods of time, such as in January.

We begin our exploration with a simple example where there are no forces of separation, alignment or cohesion. The only rule for each boid is to follow the target boid and change the radius depending on the consumption values. This first exploration is intended to show the different possibilities of visual representation using circles (Figures 4 and 5) 1 . In Figure 4 the consumption values are normalised according to the global minimum and maximum consumption value, and in Figure 5, the consumption values are normalised according to each individual Department. We can easily perceive the differences between these two types of visualization. In the first, we see that Grocery and Fresh Food are the Departments with higher consumptions, since their colours are more predominant. This type of normalisation enables the reader to compare the consumptions in the different Departments, and to see which are the Departments with higher consumptions. The second type of normalisation gives the reader the possibility to understand how the consumptions behave through time. It is not possible to compare the consumptions between Departments, but we can see the moments in which certain Departments have more sales. Despite these main differences, as it would be expected, we can also observe some similarities, resulting from the fact that the data being explored is the same. For instance, we can see that the consumption behaviour throughout the days is similar: less sales in the first part of the day, and more consumptions during the evening. Then, in the beginning of November, we can see an atypical consumption in the Leisure Department. This atypical consumption is caused by a discount made every year in a product category of this Department-toys. Finally, as it was expected, we can also see higher consumptions in December, and discern the impact of weekends in the consumption patterns (Fridays, Saturdays, and Sundays have higher consumptions than the remaining days of the week).

In a second step, we analyze the impact of the flocking rules (separation, cohesion and alignment) in the visualization. We apply higher forces to the separation rule, define that the cohesion rule is just applied to the boids of the same Department, apply higher alignment forces among boids of the same Department than for the boids of different Departments. The results of this exploration are similar to the ones of the previous exploration. In this visualization the boids are slightly more separated, and we can see that when the consumption grows the boids tend to break away from the main path (Figures 6 and 7).

1 The visualization interface allows the user to adjust the zoom level. Large scale renderings of all the visualizations presented in this paper can be found at: http://cdv.dei.uc.pt/swarmviz/.

Then, to further explore this effect of separation, we maintain the same separation forces but we define different minimum distances between the boids. If the boids belong to the same Department, the distance between the centres of the boids can be equal to the radius of each boid. If the boids represent different Departments, the minimum distance between them is equal to the sum of the two radii. In Figures 8 and 9, it is possible to see that the boids are too separated, making it almost impossible to differentiate between the paths of different months. This exploration creates a cluttered graph almost impossible to read. From the different visual possibilities, the one with the sorted filled circles is the most readable. In this case, for example, we can distinguish the high consumption in the Leisure Department, represented in purple. With this setup, and since the boids are so apart from each other, we wanted to test the visualization of the connections between the boids of the same Department, as a way to increase the visibility of the patterns. The rightmost image of figures 8 and 9 illustrate the results obtained when drawing lines to connect the boids of the same Department. As can be observed, since some boids are far apart, possibly because they cannot reach the boids of the same Department, these types of visualizations create big areas corresponding to Departments which do not have big consumptions. Thus, in this case, the emphasis given to a Department tends to be inversely proportional to the number of sales. Admittedly, this type of visualization is difficult, perhaps impossible, to interpret, and violates the expectations of the viewer. Nevertheless, and to some extent because of that, we find them intriguing and aesthetically pleasing. Thus, we consider that although they would be hard to justify in a purely functional Data Visualization project, they have a place in the context of Information Aesthetics, where the artifacts should also be seen as a form of self-expression.

One way to get a less cluttered visualization is to have bigger alignment forces so that the boids do not disperse so much. For this exploration we apply higher forces to the alignment rule than for the separation rule. Additionally, we used different alignment forces depending on the neighbour's Department. If the neighbour boid is from the same Department, the alignment force is bigger than if the boid is from a different one. This setup results in a more comprehensible visualization. In Figure 10, where the values are normalised with the global values, the boids corresponding to Departments with higher consumptions, the Grocery and Fresh Food Departments, tend to go to the "outside" of the spiral. Thus, the rules induce an ordering of the boids, making those associated with Departments with fewer consumptions to approach the centre, while those associated with Departments with high sales are pushed to the periphery. This effect is particularly visible when lines are drawn among the bots of the same Department, as is the case in the rightmost image of Figure 10. It is also interesting to observe the disruption of this pattern during Christmas, which is associated with a change in the consumption habits.

When inspecting visualizations using the normalised values for each Department (Figure 11), we can see the same boid's behaviour, where the boids with higher consumption values tend to go to the "outside" of the spiral. In this case, we do not see one or two predominant Departments, but we can perceive the moment at which certain Department has the biggest consumption value.

As a final experimentation, we vary the separation forces along time making them proportional to the associated consumption volumes. Some examples are presented in Figures 12 and 13. We also altered the force that makes the boids go after the target boid (see Figure 14). We consider that the results are aesthetically appealing in both cases. However, these visualizations become hard to interpret and lack some functionality.

6 Conclusion

We presented a swarm-based system as a method to create emergent visualizations of data, exploring the boundaries between Data Visualization and Information Aesthetics. Our contribution extends the literature by applying swarm techniques to the field of Data Visualization and Information Aesthetics, namely to the visualization of consumption patterns. Although not entirely new, the use of swarming techniques in this context is relatively unexplored, particularly for the production of static artifacts. The use of Artificial Intelligence (AI) techniques in this field is uncommon, which contributes for the novelty of the presented work, and opens opportunities to explore the intersections between Data Visualization, Information Aesthetics and AI.

The approach was used to visually convey the consumption patterns in 729 Portuguese hypermarkets over the course of two years. We analysed the experimental results based on the ability of the emergent visualizations to communicate information while engaging the viewer with organic visuals. With this project, we generated multiple visual representations of data to get a qualitative view, which enabled us to have an overview of how consumptions behave through time. The validation of the results through user evaluation will be performed in future work, to better comprehend how users read the graphics. Although it does not replace a thorough user evaluation, it is important to notice that our partner, SONAE, finds the visualizations valuable and is already using them to analyse consumption patterns. Furthermore, as previously mentioned, the artifacts were also created for artistic and aesthetic purposes and constitute a form of self-expression.

It is our intent to further explore this type of representations giving more emphasis to their functionality, exploring new behavioural rules that enable the boids to convey the consumption values and patterns more efficiently. Additionally, we intent to explore our approach with other time-based data.

7 Acknowledgements

This project was developed within a partnership with SONAE: Sonae Viz - Big Data Visualization for retail. Special thanks to Tiago Carvalho, Frederico Santos and Jo˜ao Amaral.