{"type":"ScholarlyArticle","title":"Real-time Multi-modal Object Detection and Tracking on Edge for Regulatory Compliance Monitoring","authors":[{"name":"Jia Syuen Lim","slug":"jia-syuen-lim","given":"Jia Syuen","family":"Lim","affiliation":"The University of Queensland , Australia; CSIRO , Australia"},{"name":"Ziwei Wang","slug":"ziwei-wang","given":"Ziwei","family":"Wang","affiliation":"CSIRO , Australia"},{"name":"Jiajun Liu","slug":"jiajun-liu","given":"Jiajun","family":"Liu","affiliation":"CSIRO , Australia; The University of Queensland , Australia"},{"name":"Abdelwahed Khamis","slug":"abdelwahed-khamis","given":"Abdelwahed","family":"Khamis","affiliation":"CSIRO , Australia"},{"name":"Reza Arablouei","slug":"reza-arablouei","given":"Reza","family":"Arablouei","affiliation":"CSIRO , Australia"},{"name":"Robert Barlow","slug":"robert-barlow","given":"Robert","family":"Barlow","affiliation":"CSIRO , Australia"},{"name":"Ryan McAllister","slug":"ryan-mcallister","given":"Ryan","family":"McAllister","affiliation":"CSIRO , Australia"}],"abstract":"Regulatory compliance auditing in agrifood processing facilities is crucial for upholding the highest standards of quality assurance and traceability. However, the current manual and intermittent approaches to auditing present significant challenges and risks, potentially leading to gaps or loopholes in the system. To address these shortcomings, we introduce a real-time, multi-modal sensing system that utilizes 3D time-of-flight and RGB cameras and leverages unsupervised learning techniques on edge AI devices. The proposed system enables continuous object tracking, leading to improved efficiency in record-keeping and reduced manual labor. We demonstrate the effectiveness of the system in a knife sanitization monitoring scenario, showcasing its capability to overcome occlusion and low-light performance limitations commonly encountered with conventional RGB cameras.","doi":"10.24963/ijcai.2024/1018","url":"https://ijcai.org/proceedings/2024/1018","pdf":"https://ijcai.org/proceedings/2024/1018.pdf","markdown":"https://ijcai.org/proceedings/2024/1018.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/1018","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Demo Track","track":"Demo Track","number":1018,"pages":{"first":8725,"last":8728},"keywords":["Computer Vision: CV: Video analysis and understanding","Computer Vision: CV: Action and behavior recognition","Computer Vision: CV: Applications","Machine Learning: ML: Applications","Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Bewley et al., 2016] Alex Bewley, ZongYuan Ge, Lionel Ott, Fabio Tozeto Ramos, and Ben Upcroft. Simple online and realtime tracking. In 2016 IEEE International Confer- ence on Image Processing, ICIP 2016, Phoenix, AZ, USA, September 25-28, 2016, pages 3464-3468. IEEE, 2016.","title":"Simple online and realtime tracking","authors":"Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Tozeto Ramos, Ben Upcroft","year":"2016","venue":"2016 IEEE International Conference on Image Processing, ICIP 2016","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Khamis et al., 2020a] Abdelwahed Khamis, Branislav Kusy, Chun Tung Chou, Mary-Louise McLaws, and Wen Hu. Rfwash: a weakly supervised tracking of hand hygiene technique. In Jin Nakazawa and Polly Huang, editors, SenSys '20: The 18th ACM Conference on Embedded Networked Sensor Systems, Virtual Event, Japan, November 16-19, 2020, pages 572-584. ACM, 2020.","title":"Rfwash: a weakly supervised tracking of hand hygiene technique","authors":"Khamis","year":"2020","venue":"SenSys '20: The 18th ACM Conference on Embedded Networked Sensor Systems, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Khamis et al., 2020b] Abdelwahed Khamis, Branislav Kusy, Chun Tung Chou, Marylouise McLaws, and Wen Hu. Poster abstract: A weakly supervised tracking of hand hygiene technique. In 19th ACM/IEEE International Con- ference on Information Processing in Sensor Networks, IPSN 2020, Sydney, Australia, April 21-24, 2020, pages 331-332. IEEE, 2020.","title":"Poster abstract: A weakly supervised tracking of hand hygiene technique","authors":"Khamis","year":"2020","venue":"19th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2020","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[McInnes et al., 2017] Leland McInnes, John Healy, and Steve Astels. hdbscan: Hierarchical density based clus- tering. J. Open Source Softw., 2(11):205, 2017.","title":"hdbscan: Hierarchical density based clustering","authors":"Leland Mcinnes, John Healy, Steve Astels","year":"2017","venue":"J. Open Source Softw","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Stäcker et al., 2021] Lukas Stäcker, Juncong Fei, Philipp Heidenreich, Frank Bonarens, Jason R. Rambach, Didier Stricker, and Christoph Stiller. Deployment of deep neu- ral networks for object detection on edge AI devices with runtime optimization. In IEEE/CVF International Confer- ence on Computer Vision Workshops, ICCVW 2021, Mon- treal, BC, Canada, October 11-17, 2021, pages 1015- 1022. IEEE, 2021.","title":"Deployment of deep neural networks for object detection on edge AI devices with runtime optimization","authors":"Lukas Stäcker, Juncong Fei, Philipp Heidenreich, Frank Bonarens, Jason R Rambach, Didier Stricker, Christoph Stiller","year":"2021","venue":"IEEE/CVF International Conference on Computer Vision Workshops","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Wang et al., 2022] Ziwei Wang, Jiajun Liu, Reza Arablouei, Greg Bishop-Hurley, Melissa Matthews, and Paulo Borges. Multi-modal sensing for behaviour recognition. In ACM MobiCom '22: The 28th Annual International Conference on Mobile Computing and Networking, Syd- ney, NSW, Australia, October 17 -21, 2022, pages 900- 902. ACM, 2022.","title":"Multi-modal sensing for behaviour recognition","authors":"Ziwei Wang, Jiajun Liu, Reza Arablouei, Greg Bishop-Hurley, Melissa Matthews, Paulo Borges","year":"2022","venue":"ACM MobiCom '22: The 28th Annual International Conference on Mobile Computing and Networking","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Zhang et al., 2020] Fan Zhang, Valentin Bazarevsky, An- drey Vakunov, Andrei Tkachenka, George Sung, Chuo- Ling Chang, and Matthias Grundmann. Mediapipe hands: On-device real-time hand tracking. CoRR, abs/2006.10214, 2020.","title":"Mediapipe hands: On-device real-time hand tracking","authors":"Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, Matthias Grundmann","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}