ORCID Profile
0000-0002-4093-7557
Current Organisations
University of Technology Sydney
,
Zhejiang University
Does something not look right? The information on this page has been harvested from data sources that may not be up to date. We continue to work with information providers to improve coverage and quality. To report an issue, use the Feedback Form.
In Research Link Australia (RLA), "Research Topics" refer to ANZSRC FOR and SEO codes. These topics are either sourced from ANZSRC FOR and SEO codes listed in researchers' related grants or generated by a large language model (LLM) based on their publications.
Neural, Evolutionary and Fuzzy Computation | Pattern Recognition and Data Mining | Computer Vision | Artificial Intelligence and Image Processing
Film and Video Services (excl. Animation and Computer Generated Imagery) | Electronic Information Storage and Retrieval Services |
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2021
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2021
Publisher: Springer Science and Business Media LLC
Date: 13-07-2017
Publisher: ACM
Date: 15-10-2018
Publisher: Association for Computing Machinery (ACM)
Date: 05-07-2020
DOI: 10.1145/3390891
Abstract: In Visual Dialog, an agent has to parse temporal context in the dialog history and spatial context in the image to hold a meaningful dialog with humans. For ex le, to answer “what is the man on her left wearing?” the agent needs to (1) analyze the temporal context in the dialog history to infer who is being referred to as “her,” (2) parse the image to attend “her,” and (3) uncover the spatial context to shift the attention to “her left” and check the apparel of the man. In this article, we use a dialog network to memorize the temporal context and an attention processor to parse the spatial context. Since the question and the image are usually very complex, which makes it difficult for the question to be grounded with a single glimpse, the attention processor attends to the image multiple times to better collect visual information. In the Visual Dialog task, the generative decoder (G) is trained under the word-by-word paradigm, which suffers from the lack of sentence-level training. We propose to reinforce G at the sentence level using the discriminative model (D), which aims to select the right answer from a few candidates, to ameliorate the problem. Experimental results on the VisDial dataset demonstrate the effectiveness of our approach.
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 10-2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: ACM
Date: 15-10-2018
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: IEEE
Date: 06-2019
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 06-2023
Publisher: IEEE
Date: 10-2019
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Association for Computing Machinery (ACM)
Date: 25-02-2023
DOI: 10.1145/3569584
Abstract: It is crucial to s le a small portion of relevant frames for efficient video classification. The existing methods mainly develop hand-designed s ling strategies or learn sequential selection policies. However, there are two challenges to be solved. First, hand-designed s ling strategies are intrinsically non-adaptive to different video backbones. Second, sequential frame selection policies ignore temporal relations among all video frames. The sequential selection process also hinders the application of these video s lers in speed-critical systems. In this article, we propose a differentiable parallel video s ling network (PSN) to tackle the aforementioned challenges, First, we optimize the video s ler with a differentiable surrogate loss, allowing to dynamically learn the s ler with the cooperation from the video classification model. Our s ler considers the feedback from all frames jointly, eliminating the learning difficulties of sequential decision making. The learning process is fully gradient-based, making the s ler be learned efficiently. Our video s ler can assess a set of frames swiftly and determine the importance of each frame in parallel. Second, we propose to model the inter-relation among contextual frames, which encourages the s ler to select frames based on a comprehensive inspection of the entire video. We observe that a simple context relation mining instantiation would significantly improve the classification performance. The experimental results on three standard video recognition benchmarks demonstrate the efficacy and efficiency of our framework.
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Association for the Advancement of Artificial Intelligence (AAAI)
Date: 17-07-2019
DOI: 10.1609/AAAI.V33I01.33018263
Abstract: Predicting future frames in videos has become a promising direction of research for both computer vision and robot learning communities. The core of this problem involves moving object capture and future motion prediction. While object capture specifies which objects are moving in videos, motion prediction describes their future dynamics. Motivated by this analysis, we propose a Cubic Long Short-Term Memory (CubicLSTM) unit for video prediction. CubicLSTM consists of three branches, i.e., a spatial branch for capturing moving objects, a temporal branch for processing motions, and an output branch for combining the first two branches to generate predicted frames. Stacking multiple CubicLSTM units along the spatial branch and output branch, and then evolving along the temporal branch can form a cubic recurrent neural network (CubicRNN). Experiment shows that CubicRNN produces more accurate video predictions than prior methods on both synthetic and real-world datasets.
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: IEEE
Date: 07-2017
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2023
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2021
Publisher: Springer Science and Business Media LLC
Date: 02-03-2016
Publisher: IEEE
Date: 10-2019
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2022
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 10-2023
Publisher: IEEE
Date: 07-2017
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 09-2023
Publisher: Springer International Publishing
Date: 2018
Publisher: IEEE
Date: 10-2019
Publisher: International Joint Conferences on Artificial Intelligence Organization
Date: 07-2018
Abstract: We aim to significantly reduce the computational cost for classification of temporally untrimmed videos while retaining similar accuracy. Existing video classification methods s le frames with a predefined frequency over entire video. Differently, we propose an end-to-end deep reinforcement approach which enables an agent to classify videos by watching a very small portion of frames like what we do. We make two main contributions. First, information is not equally distributed in video frames along time. An agent needs to watch more carefully when a clip is informative and skip the frames if they are redundant or irrelevant. The proposed approach enables the agent to adapt s ling rate to video content and skip most of the frames without the loss of information. Second, in order to have a confident decision, the number of frames that should be watched by an agent varies greatly from one video to another. We incorporate an adaptive stop network to measure confidence score and generate timely trigger to stop the agent watching videos, which improves efficiency without loss of accuracy. Our approach reduces the computational cost significantly for the large-scale YouTube-8M dataset, while the accuracy remains the same.
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Date: 2021
Start Date: 05-2020
End Date: 03-2025
Amount: $486,000.00
Funder: Australian Research Council
View Funded Activity