{"type":"ScholarlyArticle","title":"Light Agents Searching for Hot Information","authors":[{"name":"Dariusz R. Kowalski","slug":"dariusz-r-kowalski","given":"Dariusz R.","family":"Kowalski","affiliation":"School of Computer and Cyber Sciences , Augusta University"},{"name":"Dominik Pajak","slug":"dominik-pajak","given":"Dominik","family":"Pajak","affiliation":"Wrocław University of Science and Technology"}],"abstract":"Agent-based crawlers are commonly used in network maintenance and information gathering. In order not to disturb the main functionality of the system, whether acting at nodes or being in transit, they need to operate online, perform a single operation fast and use small memory. They should also be preferably deterministic, as crawling agents have limited capabilities of generating a large number of truly random bits. We consider a system in which an agent receives an update, typically an insertion or deletion, of some information upon visiting a node. On request, the agent needs to output hot information, i.e., with the net occurrence above certain frequency threshold. A desired time and memory complexity of such agent should be poly-logarithmic in the number of visited nodes and inversely proportional to the frequency threshold. Ours is the first such agent with rigorous analysis and a complementary almost-matching lower bound.","doi":"10.24963/ijcai.2022/52","url":"https://ijcai.org/proceedings/2022/52","pdf":"https://ijcai.org/proceedings/2022/0052.pdf","markdown":"https://ijcai.org/proceedings/2022/52.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/52","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":52,"pages":{"first":363,"last":369},"keywords":["Data Mining: Mining Data Streams","Agent-based and Multi-agent Systems: Agent Theories and Models"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986922","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985065","length_sec":342,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aho et al., 1983] Alfred V. Aho, John E. Hopcroft, and Jeffrey D. Ullman. Data Structures and Algorithms. Com- puter Science and Information Processing. Addison-Wesley, Reading, Massachusetts, 1st edition, January 1983.","title":"Data Structures and Algorithms","authors":"Alfred V Aho, John E Hopcroft, Jeffrey D Ullman","year":"1983","venue":"Computer Science and Information Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alon et al., 1996] Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In STOC 1996, pages 20-29. ACM, 1996.","title":"The space complexity of approximating the frequency moments","authors":"Yossi Noga Alon, Mario Matias, Szegedy","year":"1996","venue":"STOC 1996","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cafaro et al., 2017] Massimo Cafaro, Italo Epicoco, Marco Pulimeno, and Giovanni Aloisio. On frequency estimation and detection of frequent items in time faded streams. IEEE Access, 5:24078-24093, 2017.","title":"On frequency estimation and detection of frequent items in time faded streams","authors":"Massimo Cafaro, Italo Epicoco, Marco Pulimeno, Giovanni Aloisio","year":"2017","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Charikar et al., 2002] Moses Charikar, Kevin C. Chen, and Martin Farach-Colton. Finding frequent items in data streams. In ICALP 2002, pages 693-703. Springer, 2002.","title":"Finding frequent items in data streams","authors":"Charikar","year":"2002","venue":"ICALP 2002","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cormode and Muthukrishnan, 2005a] Graham Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. J. Algorithms, 55(1):58-75, 2005.","title":"An improved data stream summary: the count-min sketch and its applications","authors":"Muthukrishnan Cormode, Graham Cormode, S Muthukrishnan","year":"2005","venue":"J. Algorithms","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cormode and Muthukrishnan, 2005b] Graham Cormode and Shan Muthukrishnan. What's hot and what's not: tracking most frequent items dynamically. ACM Transactions on Database Systems (TODS), 30(1):249-278, 2005.","title":"What's hot and what's not: tracking most frequent items dynamically","authors":"Muthukrishnan Cormode, Graham Cormode, Shan Muthukrishnan","year":"2005","venue":"ACM Transactions on Database Systems (TODS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Demaine et al., 2002] Erik D. Demaine, Alejandro López-Ortiz, and J. Ian Munro. Frequency estimation of internet packet streams with limited space. In Algorithms -ESA 2002, 10th Annual European Symposium, Rome, Italy, September 17-21, 2002, Proceedings, volume 2461 of Lecture Notes in Computer Science, pages 348-360. Springer, 2002.","title":"Frequency estimation of internet packet streams with limited space","authors":"Erik D Demaine, Alejandro López-Ortiz, J, Ian Munro","year":"2002","venue":"Algorithms -ESA 2002, 10th Annual European Symposium","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Epicoco et al., 2018] Italo Epicoco, Massimo Cafaro, and Marco Pulimeno. Fast and accurate mining of correlated heavy hitters. Data Mining and Knowledge Discovery, 32(1):162-186, 2018.","title":"Fast and accurate mining of correlated heavy hitters","authors":"Epicoco","year":"2018","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 1998] Min Fang, Narayanan Shivakumar, Hector Garcia-Molina, Rajeev Motwani, and Jeffrey D. Ullman. Computing iceberg queries efficiently. In VLDB 1998, pages 299-310.","title":"Computing iceberg queries efficiently","authors":"Fang","year":"1998","venue":"VLDB 1998","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Morgan Kaufmann, 1998.","title":null,"authors":"Morgan Kaufmann","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ganguly and Majumder, 2007] Sumit Ganguly and Anirban Majumder. Cr-precis: A deterministic summary structure for update data streams. In Combinatorics, Algorithms, Proba- bilistic and Experimental Methodologies, First International Symposium, ESCAPE 2007, Hangzhou, China, April 7-9, 2007, Revised Selected Papers, volume 4614 of Lecture Notes in Computer Science, pages 48-59. Springer, 2007.","title":"Cr-precis: A deterministic summary structure for update data streams","authors":"Sumit Ganguly, Anirban Majumder","year":"2007","venue":"Combinatorics, Algorithms, Probabilistic and Experimental Methodologies, First International Symposium, ESCAPE 2007","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gibbons and Matias, 1998] Phillip B. Gibbons and Yossi Matias. New sampling-based summary statistics for improving approximate query answers. In SIGMOD 1998, Proceedings ACM SIGMOD International Conference on Management of Data, June 2-4, 1998, Seattle, Washington, USA, pages 331-342. ACM Press, 1998.","title":"New sampling-based summary statistics for improving approximate query answers","authors":"Phillip B Gibbons, Yossi Matias","year":"1998","venue":"Proceedings ACM SIGMOD International Conference on Management of Data","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gibbons and Matias, 1999] Phillip B. Gibbons and Yossi Matias. Synopsis data structures for massive data sets. In SODA, pages 909-910. ACM/SIAM, 1999.","title":"Synopsis data structures for massive data sets","authors":"Phillip B Gibbons, Yossi Matias","year":"1999","venue":"SODA","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gibbons et al., 1997] Phillip B. Gibbons, Yossi Matias, and Viswanath Poosala. Fast incremental maintenance of approximate histograms. In VLDB 1997, pages 466-475.","title":"Fast incremental maintenance of approximate histograms","authors":"B Phillip, Yossi Gibbons, Viswanath Matias, Poosala","year":null,"venue":"VLDB 1997","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Morgan Kaufmann, 1997.","title":null,"authors":"Morgan Kaufmann","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gilbert et al., 2002a] Anna C. Gilbert, Sudipto Guha, Piotr Indyk, Yannis Kotidis, S. Muthukrishnan, and Martin Strauss. Fast, small-space algorithms for approximate histogram maintenance. In STOC 2002, pages 389-398. ACM, 2002.","title":"Fast, small-space algorithms for approximate histogram maintenance","authors":"Gilbert","year":"2002","venue":"STOC 2002","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gilbert et al., 2002b] Anna C. Gilbert, Yannis Kotidis, S. Muthukrishnan, and Martin Strauss. How to summarize the universe: Dynamic maintenance of quantiles. In VLDB 2002, pages 454-465.","title":"How to summarize the universe: Dynamic maintenance of quantiles","authors":"Gilbert","year":"2002","venue":"VLDB 2002","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Morgan Kaufmann, 2002.","title":null,"authors":"Morgan Kaufmann","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ioannidis and Christodoulakis, 1993] Yannis E. Ioannidis and Stavros Christodoulakis. Optimal histograms for limiting worst-case error propagation in the size of join results. ACM Trans. Database Syst., 18(4):709-748, 1993.","title":"Optimal histograms for limiting worst-case error propagation in the size of join results","authors":"Yannis E Ioannidis, Stavros Christodoulakis","year":"1993","venue":"ACM Trans. Database Syst","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ioannidis and Poosala, 1995] Yannis E. Ioannidis and Viswanath Poosala. Balancing histogram optimality and practicality for query result size estimation. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, USA, May 22-25, 1995, pages 233-244. ACM Press, 1995.","title":"Balancing histogram optimality and practicality for query result size estimation","authors":"Yannis E Ioannidis, Viswanath Poosala","year":"1995","venue":"Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Karp et al., 2003] Richard M. Karp, Scott Shenker, and Christos H. Papadimitriou. A simple algorithm for finding frequent elements in streams and bags. ACM Trans. Database Syst., 28:51-55, 2003.","title":"A simple algorithm for finding frequent elements in streams and bags","authors":"Richard M Karp, Scott Shenker, Christos H Papadimitriou","year":"2003","venue":"ACM Trans. Database Syst","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kushilevitz and Nisan, 1997] Eyal Kushilevitz and Noam Nisan. Communication complexity. Cambridge University Press, 1997.","title":"Communication complexity","authors":"Eyal Kushilevitz, Noam Nisan","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lahiri et al., 2016] Bibudh Lahiri, Arko Provo Mukherjee, and Srikanta Tirthapura. Identifying correlated heavy-hitters in a two-dimensional data stream. Data mining and knowledge discovery, 30(4):797-818, 2016.","title":"Identifying correlated heavy-hitters in a two-dimensional data stream","authors":"Lahiri","year":"2016","venue":"Data mining and knowledge discovery","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lin et al., 2005] Chih-Hsiang Lin, Ding-Ying Chiu, Yi-Hung Wu, and Arbee LP Chen. Mining frequent itemsets from data streams with a time-sensitive sliding window. In Proceedings of the 2005 SIAM International Conference on Data Mining, pages 68-79. SIAM, 2005.","title":"Mining frequent itemsets from data streams with a time-sensitive sliding window","authors":"Chih-Hsiang Lin, Ding-Ying Chiu, Yi-Hung Wu, Arbee Lp Chen","year":"2005","venue":"Proceedings of the 2005 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Manku and Motwani, 2002] Gurmeet Singh Manku and Rajeev Motwani. Approximate frequency counts over data streams. In VLDB 2002, pages 346-357.","title":"Approximate frequency counts over data streams","authors":"Gurmeet Singh, Manku, Rajeev Motwani","year":null,"venue":"VLDB 2002","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Morgan Kaufmann, 2002.","title":null,"authors":"Morgan Kaufmann","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Misra and Gries, 1982] Jayadev Misra and David Gries. Find- ing repeated elements. Sci. Comput. Program., 2(2):143-152, 1982.","title":"Finding repeated elements","authors":"Jayadev Misra, David Gries","year":"1982","venue":"Sci. Comput. Program","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ta-Shma et al., 2001] Amnon Ta-Shma, Christopher Umans, and David Zuckerman. Loss-less condensers, unbalanced expanders, and extractors. In STOC 2001, pages 143-152. ACM, 2001.","title":"Loss-less condensers, unbalanced expanders, and extractors","authors":"Ta-Shma","year":"2001","venue":"STOC 2001","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu et al., 2017] Shanshan Wu, Huaizhong Lin, Yunjun Gao, Dongming Lu, et al. Novel structures for counting frequent items in time decayed streams. World Wide Web, 20(5):1111-1133, 2017.","title":"Novel structures for counting frequent items in time decayed streams","authors":"Shanshan Wu, Huaizhong Lin, Yunjun Gao, Dongming Lu","year":"2017","venue":"World Wide Web","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}