ORCID Profile
0000-0001-9094-0810
Current Organisation
RMIT University
Does something not look right? The information on this page has been harvested from data sources that may not be up to date. We continue to work with information providers to improve coverage and quality. To report an issue, use the Feedback Form.
In Research Link Australia (RLA), "Research Topics" refer to ANZSRC FOR and SEO codes. These topics are either sourced from ANZSRC FOR and SEO codes listed in researchers' related grants or generated by a large language model (LLM) based on their publications.
Library and Information Studies | Information Retrieval and Web Search | Computer-Human Interaction | Data Storage Representations | Information Storage, Retrieval And Management | Data Structures
Electronic Information Storage and Retrieval Services | Information processing services | Information Services not elsewhere classified | Application Tools and System Utilities |
Publisher: ACM
Date: 27-06-2018
Publisher: ACM
Date: 26-10-2010
Publisher: ACM
Date: 23-11-2009
Publisher: Elsevier BV
Date: 2012
Publisher: Springer International Publishing
Date: 2016
Publisher: Springer International Publishing
Date: 2015
Publisher: Wiley
Date: 18-07-2015
DOI: 10.1002/ASI.23222
Publisher: Wiley
Date: 28-03-2012
DOI: 10.1002/ASI.22639
Publisher: ACM
Date: 11-08-2002
Publisher: ACM
Date: 06-11-2009
Publisher: ACM
Date: 26-08-2014
Publisher: ACM
Date: 28-11-2017
DOI: 10.1145/3152771
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 02-2008
Publisher: Springer Berlin Heidelberg
Date: 2009
Publisher: ACM
Date: 06-08-2006
Publisher: Springer Berlin Heidelberg
Date: 2011
Publisher: Springer Berlin Heidelberg
Date: 2008
Publisher: Springer Berlin Heidelberg
Date: 2011
Publisher: ACM Press
Date: 2013
Publisher: ACM
Date: 09-08-2015
DOI: 10.1145/2766462
Publisher: ACM
Date: 03-11-2003
Publisher: ACM
Date: 08-10-2023
Publisher: Association for Computing Machinery (ACM)
Date: 20-05-2012
Abstract: INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2011 evaluation c aign, which consisted of a five active tracks: Books and Social Search, Data Centric, Question Answering, Relevance Feedback, and Snippet Retrieval. INEX 2011 saw a range of new tasks and tracks, such as Social Book Search, Faceted Search, Snippet Retrieval, and Tweet Contextualization.
Publisher: Association for Computing Machinery (ACM)
Date: 05-06-2017
DOI: 10.1145/2975590
Abstract: Re-finding is the process of searching for information that a user has previously encountered and is a common activity carried out with information retrieval systems. In this work, we investigate re-finding in the context of vertical search, differentiating and modeling user re-finding behavior within different media and topic domains, including images, news, reference material, and movies. We distinguish the re-finding behavior in vertical domains from re-finding in a general search context and engineer features that are effective in differentiating re-finding across the domains. The features are then used to build machine-learned models, achieving an accuracy of re-finding detection in verticals of 85.7% on average. Our results demonstrate that detecting re-finding in specific verticals is more difficult than examining re-finding for general search tasks. We then investigate the effectiveness of differentiating re-finding behavior in two restricted contexts: We consider the case where the history of a searcher’s interactions with the search system is not available. In this scenario, our features and models achieve an average accuracy of 77.5% across the domains. We then examine the detection of re-finding during the early part of a search session. Both of these restrictions represent potential real-world search scenarios, where a system is attempting to learn about a user but may have limited information available. Finally, we investigate in which types of domains re-finding is most difficult. Here, it would appear that re-finding images is particularly challenging for users. This research has implications for search engine design, in terms of adapting search results by predicting the type of user tasks and potentially enabling the presentation of vertical-specific results when re-finding is identified. To the best of our knowledge, this is the first work to investigate the issue of vertical re-finding.
Publisher: Springer Berlin Heidelberg
Date: 2008
Publisher: Springer International Publishing
Date: 2015
Publisher: Association for Computing Machinery (ACM)
Date: 04-2011
Abstract: Machine transliteration is the process of automatically transforming the script of a word from a source language to a target language, while preserving pronunciation. The development of algorithms specifically for machine transliteration began over a decade ago based on the phonetics of source and target languages, followed by approaches using statistical and language-specific methods. In this survey, we review the key methodologies introduced in the transliteration literature. The approaches are categorized based on the resources and algorithms used, and the effectiveness is compared.
Publisher: Springer Berlin Heidelberg
Date: 2008
Publisher: Association for Computing Machinery (ACM)
Date: 21-01-2013
Abstract: INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2013 evaluation c aign, which consisted of four activities addressing three themes: searching professional and user generated data (Social Book Search track) searching structured or semantic data (Linked Data track) and focused retrieval (Snippet Retrieval and Tweet Contextualization tracks). INEX 2013 was an exciting year for INEX in which we consolidated the collaboration with (other activities in) CLEF and for the second time ran our workshop as part of the CLEF labs in order to facilitate knowledge transfer between the evaluation forums. This paper gives an overview of all the INEX 2013 tracks, their aims and task, the built test-collections, and gives an initial analysis of the results.
Publisher: ACM
Date: 12-08-2012
DOI: 10.1145/2348283
Publisher: Springer Berlin Heidelberg
Date: 2005
DOI: 10.1007/11575832_23
Publisher: Springer Berlin Heidelberg
Date: 2006
DOI: 10.1007/11610113_7
Publisher: ACM
Date: 27-06-2018
Publisher: ACM
Date: 09-08-2015
Publisher: ACM
Date: 07-07-2016
Publisher: ACM
Date: 24-10-2016
DOI: 10.1145/2983323
Publisher: ACM
Date: 07-12-2017
DOI: 10.1145/3166072
Publisher: ACM
Date: 24-09-2007
Publisher: ACM
Date: 05-12-2012
Publisher: ACM
Date: 05-12-2012
Publisher: Wiley
Date: 11-10-2014
DOI: 10.1002/ASI.22951
Publisher: ACM
Date: 06-08-2006
DOI: 10.1145/1148170
Publisher: Association for Computing Machinery (ACM)
Date: 05-06-2017
DOI: 10.1145/3052768
Abstract: Information retrieval systems aim to help users satisfy information needs. We argue that the goal of the person using the system, and the pattern of behavior that they exhibit as they proceed to attain that goal, should be incorporated into the methods and techniques used to evaluate the effectiveness of IR systems, so that the resulting effectiveness scores have a useful interpretation that corresponds to the users’ search experience. In particular, we investigate the role of search task complexity, and show that it has a direct bearing on the number of relevant answer documents sought by users in response to an information need, suggesting that useful effectiveness metrics must be goal sensitive . We further suggest that user behavior while scanning results listings is affected by the rate at which their goal is being realized, and hence that appropriate effectiveness metrics must be adaptive to the presence (or not) of relevant documents in the ranking. In response to these two observations, we present a new effectiveness metric, INST, that has both of the desired properties: INST employs a parameter T , a direct measure of the user’s search goal that adjusts the top-weightedness of the evaluation score moreover, as progress towards the target T is made, the modeled user behavior is adapted, to reflect the remaining expectations. INST is experimentally compared to previous effectiveness metrics, including Average Precision (AP), Normalized Discounted Cumulative Gain (NDCG), and Rank-Biased Precision (RBP), demonstrating our claims as to INST’s usefulness. Like RBP, INST is a weighted-precision metric, meaning that each score can be accompanied by a residual that quantifies the extent of the score uncertainty caused by unjudged documents. As part of our experimentation, we use crowd-sourced data and score residuals to demonstrate that a wide range of queries arise for even quite specific information needs, and that these variant queries introduce significant levels of residual uncertainty into typical experimental evaluations. These causes of variability have wide-reaching implications for experiment design, and for the construction of test collections.
Publisher: Wiley
Date: 25-02-2004
DOI: 10.1002/ASI.20011
Publisher: Association for Computing Machinery (ACM)
Date: 22-11-2022
DOI: 10.1145/3483237
Abstract: In many search scenarios, such as exploratory, comparative, or survey-oriented search, users interact with dynamic search systems to satisfy multi-aspect information needs. These systems utilize different dynamic approaches that exploit various user feedback granularity types. Although studies have provided insights about the role of many components of these systems, they used black-box and isolated experimental setups. Therefore, the effects of these components or their interactions are still not well understood. We address this by following a methodology based on Analysis of Variance (ANOVA). We built a Grid Of Points that consists of systems based on different ways to instantiate three components: initial rankers, dynamic rerankers, and user feedback granularity. Using evaluation scores based on the TREC Dynamic Domain collections, we built several ANOVA models to estimate the effects. We found that (i) although all components significantly affect search effectiveness, the initial ranker has the largest effective size, (ii) the effect sizes of these components vary based on the length of the search session and the used effectiveness metric, and (iii) initial rankers and dynamic rerankers have more prominent effects than user feedback granularity. To improve effectiveness, we recommend improving the quality of initial rankers and dynamic rerankers. This does not require eliciting detailed user feedback, which might be expensive or invasive.
Publisher: ACM
Date: 22-10-2015
Publisher: Springer Berlin Heidelberg
Date: 2013
Publisher: ACM
Date: 07-08-2017
Publisher: Springer Berlin Heidelberg
Date: 2013
Publisher: ACM
Date: 24-07-2011
Publisher: ACM
Date: 19-07-2009
Publisher: Springer International Publishing
Date: 2018
Publisher: ACM
Date: 05-12-2016
DOI: 10.1145/3015022
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2018
Publisher: ACM
Date: 03-07-2014
Publisher: ACM
Date: 03-07-2014
Publisher: Springer Science and Business Media LLC
Date: 06-2016
Publisher: IEEE
Date: 04-2014
Publisher: ACM
Date: 21-10-2023
Publisher: Springer International Publishing
Date: 2015
Publisher: ACM
Date: 29-10-2012
Publisher: Springer International Publishing
Date: 2016
Publisher: ACM
Date: 12-08-2012
Publisher: ACM
Date: 27-06-2018
Publisher: Springer International Publishing
Date: 2021
Publisher: ACM
Date: 24-07-2011
Publisher: Springer International Publishing
Date: 2017
Publisher: ACM
Date: 09-08-2015
Publisher: ACM
Date: 07-07-2016
DOI: 10.1145/2911451
Publisher: ACM
Date: 18-08-2010
Publisher: ACM
Date: 20-08-2017
DOI: 10.1145/3106668
Publisher: Association for Computing Machinery (ACM)
Date: 04-01-2017
DOI: 10.1145/3002172
Abstract: Magnitude estimation is a psychophysical scaling technique for the measurement of sensation, where observers assign numbers to stimuli in response to their perceived intensity. We investigate the use of magnitude estimation for judging the relevance of documents for information retrieval evaluation, carrying out a large-scale user study across 18 TREC topics and collecting over 50,000 magnitude estimation judgments using crowdsourcing. Our analysis shows that magnitude estimation judgments can be reliably collected using crowdsourcing, are competitive in terms of assessor cost, and are, on average, rank-aligned with ordinal judgments made by expert relevance assessors. We explore the application of magnitude estimation for IR evaluation, calibrating two gain-based effectiveness metrics, nDCG and ERR, directly from user-reported perceptions of relevance. A comparison of TREC system effectiveness rankings based on binary, ordinal, and magnitude estimation relevance shows substantial variation in particular, the top systems ranked using magnitude estimation and ordinal judgments differ substantially. Analysis of the magnitude estimation scores shows that this effect is due in part to varying perceptions of relevance: different users have different perceptions of the impact of relative differences in document relevance. These results have direct implications for IR evaluation, suggesting that current assumptions about a single view of relevance being sufficient to represent a population of users are unlikely to hold.
Publisher: ACM
Date: 20-08-2017
Publisher: Association for Computing Machinery (ACM)
Date: 05-2011
Abstract: Searchers on the Web often aim to find key resources about a topic. Finding such results is called topic distillation. Previous research has shown that the use of sources of evidence such as page indegree and URL structure can significantly improve search performance on interconnected collections such as the Web, beyond the use of simple term distribution statistics. This article presents a new approach to improve topic distillation by exploring the use of external sources of evidence: link structure, including query dependent indegree and outdegree and web page characteristics, such as the density of anchor links. Our experiments with the TREC .GOV collection, an 18GB crawl of the US .gov domain from 2002, show that using such evidence can significantly improve search effectiveness, with combinations of evidence leading to significant performance gains over both full-text and anchor-text baselines. Moreover, we demonstrate that, at a different scope level, both local query-dependent outdegree and query-dependent indegree out-performed their global query-independent counterparts and at the same scope level, outdegree out-performed indegree. Adding query-dependent indegree or page characteristics to query-dependent outdegree could have a small, but not significant, improvement.
Publisher: Springer Berlin Heidelberg
Date: 2006
DOI: 10.1007/11880561_21
Publisher: ACM
Date: 18-07-2023
Publisher: ACM
Date: 07-08-2017
Publisher: ACM
Date: 08-12-2015
Publisher: ACM
Date: 08-12-2015
Publisher: ACM
Date: 06-11-2017
Publisher: ACM Press
Date: 2018
Publisher: IEEE
Date: 2007
DOI: 10.1109/ICIS.2007.61
Publisher: ACM
Date: 19-07-2009
Publisher: Springer Science and Business Media LLC
Date: 12-10-2010
Publisher: Association for Computing Machinery (ACM)
Date: 09-06-2016
DOI: 10.1145/2882782
Abstract: We present a study of which baseline to use when testing a new retrieval technique. In contrast to past work, we show that measuring a statistically significant improvement over a weak baseline is not a good predictor of whether a similar improvement will be measured on a strong baseline. Sometimes strong baselines are made worse when a new technique is applied. We investigate whether conducting comparisons against a range of weaker baselines can increase confidence that an observed effect will also show improvements on a stronger baseline. Our results indicate that this is not the case -- at best, testing against a range of baselines means that an experimenter can be more confident that the new technique is unlikely to significantly harm a strong baseline. Examining recent past work, we present evidence that the information retrieval (IR) community continues to test against weak baselines. This is unfortunate as, in light of our experiments, we conclude that the only way to be confident that a new technique is a contribution is to compare it against nothing less than the state of the art.
Publisher: ACM
Date: 28-07-2013
Publisher: Association for Computing Machinery (ACM)
Date: 29-09-2018
DOI: 10.1145/3239572
Abstract: One typical way of building test collections for offline measurement of information retrieval systems is to pool the ranked outputs of different systems down to some chosen depth d and then form relevance judgments for those documents only. Non-pooled documents—ones that did not appear in the top- d sets of any of the contributing systems—are then deemed to be non-relevant for the purposes of evaluating the relative behavior of the systems. In this article, we use RBP-derived residuals to re-examine the reliability of that process. By fitting the RBP parameter ϕ to maximize similarity between AP- and NDCG-induced system rankings, on the one hand, and RBP-induced rankings, on the other, an estimate can be made as to the potential score uncertainty associated with those two recall-based metrics. We then consider the effect that residual size—as an indicator of possible measurement uncertainty in utility-based metrics—has in connection with recall-based metrics by computing the effect of increasing pool sizes and examining the trends that arise in terms of both metric score and system separability using standard statistical tests. The experimental results show that the confidence levels expressed via the p -values generated by statistical tests are only weakly connected to the size of the residual and to the degree of measurement uncertainty caused by the presence of unjudged documents. Statistical confidence estimates are, however, largely consistent as pooling depths are altered. We therefore recommend that all such experimental results should report, in addition to the outcomes of statistical significance tests, the residual measurements generated by a suitably matched weighted-precision metric, to give a clear indication of measurement uncertainty that arises due to the presence of unjudged documents in test collections with finite pooled judgments.
Publisher: ACM
Date: 06-08-2006
Publisher: ACM
Date: 07-08-2017
Publisher: ACM
Date: 26-11-2014
Publisher: ACM Press
Date: 2013
Publisher: Elsevier BV
Date: 07-2010
Publisher: ACM
Date: 20-07-2008
Publisher: Springer Berlin Heidelberg
Date: 2012
Publisher: Springer Berlin Heidelberg
Date: 2009
Publisher: Springer International Publishing
Date: 2022
Publisher: ACM
Date: 09-08-2015
Publisher: Elsevier BV
Date: 11-2021
Publisher: Springer Berlin Heidelberg
Date: 2009
Publisher: ACM
Date: 26-11-2014
Publisher: Springer International Publishing
Date: 2017
Publisher: ACM
Date: 17-10-2015
Publisher: ACM
Date: 05-12-2013
Publisher: ACM
Date: 05-12-2013
Publisher: ACM
Date: 18-07-2023
Publisher: ACM
Date: 04-11-2002
Publisher: Springer Berlin Heidelberg
Date: 2008
Publisher: ACM
Date: 07-12-2017
Publisher: ACM
Date: 20-07-2008
Publisher: Association for Computing Machinery (ACM)
Date: 21-12-2012
Abstract: INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX'12 evaluation c aign, which consisted of a five tracks: Linked Data, Relevance Feedback, Snippet Retrieval, Social Book Search, and Tweet Contextualization. INEX'12 was an exciting year for INEX in which we joined forces with CLEF and for the first time ran our workshop as part of the CLEF labs in order to facilitate knowledge transfer between the evaluation forums.
Publisher: ACM
Date: 12-09-2016
Publisher: ACM
Date: 03-2018
DOI: 10.1145/3176349
Publisher: Springer International Publishing
Date: 2015
Publisher: Springer International Publishing
Date: 2015
Publisher: Springer International Publishing
Date: 2015
Publisher: ACM
Date: 03-11-2014
Publisher: ACM
Date: 24-10-2016
Publisher: ACM
Date: 06-11-2009
DOI: 10.1145/1651318
Publisher: ACM
Date: 26-11-2014
Publisher: Elsevier BV
Date: 2007
Publisher: ACM
Date: 07-12-2017
Start Date: 12-2018
End Date: 06-2023
Amount: $367,775.00
Funder: Australian Research Council
View Funded ActivityStart Date: 2013
End Date: 12-2016
Amount: $315,000.00
Funder: Australian Research Council
View Funded ActivityStart Date: 08-2006
End Date: 08-2009
Amount: $210,000.00
Funder: Australian Research Council
View Funded ActivityStart Date: 06-2019
End Date: 12-2024
Amount: $380,000.00
Funder: Australian Research Council
View Funded ActivityStart Date: 06-2016
End Date: 06-2019
Amount: $394,000.00
Funder: Australian Research Council
View Funded ActivityStart Date: 2014
End Date: 12-2017
Amount: $372,110.00
Funder: Australian Research Council
View Funded Activity