Mark Sanderson

A Location-Query-Browse Graph for Contextual Recommendation

Publisher: Institute of Electrical and Electronics Engineers (IEEE)

Date: 02-2018

DOI: 10.1109/TKDE.2017.2766059

Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study

Publisher: ACM

Date: 18-07-2023

DOI: 10.1145/3539618.3591960

User Perception of Information Credibility of News on Twitter

Publisher: Springer International Publishing

Date: 2014

DOI: 10.1007/978-3-319-06028-6_50

Popularity Bias in False-positive Metrics for Recommender Systems Evaluation

Publisher: Association for Computing Machinery (ACM)

Date: 25-05-2021

DOI: 10.1145/3452740

Abstract: We investigate the impact of popularity bias in false-positive metrics in the offline evaluation of recommender systems. Unlike their true-positive complements, false-positive metrics reward systems that minimize recommendations disliked by users. Our analysis is, to the best of our knowledge, the first to show that false-positive metrics tend to penalise popular items, the opposite behavior of true-positive metrics—causing a disagreement trend between both types of metrics in the presence of popularity biases. We present a theoretical analysis of the metrics that identifies the reason that the metrics disagree and determines rare situations where the metrics might agree—the key to the situation lies in the relationship between popularity and relevance distributions, in terms of their agreement and steepness —two fundamental concepts we formalize. We then examine three well-known datasets using multiple popular true- and false-positive metrics on 16 recommendation algorithms. Specific datasets are chosen to allow us to estimate both biased and unbiased metric values. The results of the empirical study confirm and illustrate our analytical findings. With the conditions of the disagreement of the two types of metrics established, we then determine under which circumstances true-positive or false-positive metrics should be used by researchers of offline evaluation in recommender systems. 1

Principles for robust evaluation infrastructure

Publisher: ACM

Date: 28-10-2011

DOI: 10.1145/2064227.2064247

Designing and Evaluating Presentation Strategies for Fact-Checked Content

Publisher: ACM

Date: 21-10-2023

DOI: 10.1145/3583780.3614841

Differences in effectiveness across sub-collections

Publisher: ACM

Date: 29-10-2012

DOI: 10.1145/2396761.2398553

How People Use the Web in Large Indoor Spaces

Publisher: ACM

Date: 03-11-2014

DOI: 10.1145/2661829.2661929

Quantifying test collection quality based on the consistency of relevance judgements

Publisher: ACM

Date: 24-07-2011

DOI: 10.1145/2009916.2010057

Improving patient record search: A meta-data based approach

Publisher: Elsevier BV

Date: 03-2016

DOI: 10.1016/J.IPM.2015.07.005

Overview of the INEX 2011 Snippet Retrieval Track

Publisher: Springer Berlin Heidelberg

Date: 2011

DOI: 10.1007/978-3-642-35734-3_27

Size and Source Matter: Understanding Inconsistencies in Test Collection-Based Evaluation

Publisher: ACM

Date: 03-11-2014

DOI: 10.1145/2661829.2661945

Features of Disagreement Between Retrieval Effectiveness Measures

Publisher: ACM

Date: 09-08-2015

DOI: 10.1145/2766462.2767824

Correlation Analysis of Reader’s Demographics and Tweet Credibility Perception

Publisher: Springer International Publishing

Date: 2016

DOI: 10.1007/978-3-319-30671-1_33

Towards understanding the impact of length in web search result summaries over a speech-only communication channel

Publisher: ACM

Date: 09-08-2015

DOI: 10.1145/2766462.2767826

Document Summarization for Answering Non-Factoid Queries

Publisher: Institute of Electrical and Electronics Engineers (IEEE)

Date: 2018

DOI: 10.1109/TKDE.2017.2754373

A new approach for indoor customer tracking based on a single Wi-Fi connection

Publisher: IEEE

Date: 10-2014

DOI: 10.1109/IPIN.2014.7275489

Improving test collection pools with machine learning

Publisher: ACM

Date: 26-11-2014

DOI: 10.1145/2682862.2682864

TREC: topic engineering exercise

Publisher: ACM

Date: 03-07-2014

DOI: 10.1145/2600428.2609531

Imagining future digital assistants at work: A study of task management needs

Publisher: Elsevier BV

Date: 12-2022

DOI: 10.1016/J.IJHCS.2022.102905

Using score differences for search result diversification

Publisher: ACM

Date: 03-07-2014

DOI: 10.1145/2600428.2609530

SpeakerLDA: Discovering topics in transcribed multi-speaker audio contents

Publisher: ACM Press

Date: 2015

DOI: 10.1145/2802558.2814649

Identifying Re-finding Difficulty from User Query Logs

Publisher: ACM

Date: 26-11-2014

DOI: 10.1145/2682862.2682867

Extending test collection pools without manual runs

Publisher: ACM

Date: 03-07-2014

DOI: 10.1145/2600428.2609473

Discovery Projects - Grant ID: DP0666154

Start Date: 2006

End Date: 12-2008

Amount: $227,000.00

Funder: Australian Research Council

Discovery Projects - Grant ID: DP170102726

Start Date: 2017

End Date: 12-2019

Amount: $337,000.00

Funder: Australian Research Council

Discovery Projects - Grant ID: DP180102687

Start Date: 12-2018

End Date: 06-2023

Amount: $367,775.00

Funder: Australian Research Council

Linkage Projects - Grant ID: LP120200413

Start Date: 06-2013

End Date: 05-2016

Amount: $225,000.00

Funder: Australian Research Council

Discovery Projects - Grant ID: DP130104007

Start Date: 2013

End Date: 12-2016

Amount: $315,000.00

Funder: Australian Research Council

Linkage Projects - Grant ID: LP150100252

Start Date: 06-2016

End Date: 06-2019

Amount: $394,000.00

Funder: Australian Research Council

Linkage Projects - Grant ID: LP130100563

Start Date: 07-2013

End Date: 12-2017

Amount: $215,000.00

Funder: Australian Research Council

Discovery Projects - Grant ID: DP140102655

Start Date: 2014

End Date: 12-2017

Amount: $372,110.00

Funder: Australian Research Council

ARC Centres Of Excellence - Grant ID: CE200100005

Start Date: 08-2020

End Date: 08-2027

Amount: $31,783,576.00

Funder: Australian Research Council

Linkage Infrastructure, Equipment And Facilities - Grant ID: LE180100094

Start Date: 2018

End Date: 08-2020

Amount: $1,361,651.00

Funder: Australian Research Council