Understanding self-paced learning under concave conjugacy theory

DOI10.4310/CIS.2018.V18.N1.A1zbMATH Open1390.93884arXiv1805.08096OpenAlexW2963395590WikidataQ129711264 ScholiaQ129711264MaRDI QIDQ1644337FDOQ1644337

Authors: Shi-Qi Liu, Zi-Lu Ma, Deyu Meng, Kai-Dong Wang, Yong Zhang 

Publication date: 21 June 2018

Published in: Communications in Information and Systems (Search for Journal in Brave)

Abstract: By simulating the easy-to-hard learning manners of humans/animals, the learning regimes called curriculum learning~(CL) and self-paced learning~(SPL) have been recently investigated and invoked broad interests. However, the intrinsic mechanism for analyzing why such learning regimes can work has not been comprehensively investigated. To this issue, this paper proposes a concave conjugacy theory for looking into the insight of CL/SPL. Specifically, by using this theory, we prove the equivalence of the SPL regime and a latent concave objective, which is closely related to the known non-convex regularized penalty widely used in statistics and machine learning. Beyond the previous theory for explaining CL/SPL insights, this new theoretical framework on one hand facilitates two direct approaches for designing new SPL models for certain tasks, and on the other hand can help conduct the latent objective of self-paced curriculum learning, which is the advanced version of both CL/SPL and possess advantages of both learning regimes to a certain extent. This further facilitates a theoretical understanding for SPCL, instead of only CL/SPL as conventional. Under this theory, we attempt to attain intrinsic latent objectives of two curriculum forms, the partial order and group curriculums, which easily follow the theoretical understanding of the corresponding SPCL regimes.

Full work available at URL: https://arxiv.org/abs/1805.08096

Recommendations

• A theoretical understanding of self-paced learning
• scientific article; zbMATH DE number 7415125
• Learning and the pace of lessons: A theoretical model
• On convergence properties of implicit self-paced objective
• Heterogeneous Learning Dynamics and Speed of Convergence

zbMATH Keywords

concave conjugacy theorycurriculum learning (CL) and self-paced learning (SPL)easy-to-hard learning manners of humans/animalsnonconvex regularized penalty

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Stochastic learning and adaptive control (93E35)

Cited In (3)

• Title not available (Why is that?)
• On convergence properties of implicit self-paced objective
• A theoretical understanding of self-paced learning

This page was built for publication: Understanding self-paced learning under concave conjugacy theory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1644337)