Overview

How can we meaningfully evaluate ubiquitous computing (UbiComp) systems and ensure their performance, trustworthiness, and societal relevance? As the field increasingly incorporates complex AI models, real-time sensing, and human-centered applications, responsible evaluation has become more critical than ever. Yet, evaluation in UbiComp remains fragmented—often system-specific, ad hoc, or limited to narrow metrics and lacking standardized benchmarks. This workshop contends that we cannot improve what we cannot measure, and seeks to establish a community-wide dialogue around evaluation practices. From a technical perspective, we will explore new methods, metrics, and benchmarks that account for uncertainty, generalization, and adaptation in dynamic real-world environments. From a human-centered perspective, we will discuss how to evaluate usability, trust, and interpretability in context-aware systems. From a social and ethical perspective, we will question what we should evaluate, and for whom—beyond traditional performance metrics. EvalComp aims to lay the groundwork for a long-term, interdisciplinary agenda focused on building robust, reflective, and reproducible evaluation frameworks for UbiComp. By launching this workshop, we seek to create a dedicated space for shaping the future of evaluation in the community. EvalComp 2025 will be hosted at UbiComp 2025 and will bring together researchers and practitioners from both academia and industry to explore versatile topics related to fairness and robustness in ubiquitous computing.

The Call for Papers is out now! You can find more information here. Accepted papers will be published in UbiComp/ISWC 2025 Adjunct Proceedings.