A Sarsa() algorithm based on double-layer fuzzy reasoning

DOI10.1155/2013/561026MaRDI QIDQ473823zbMATH OpenOpenAlexWikidataFDO

Authors Quan Liu, Xiang Mu, Wei Huang, Qi-Ming Fu, Yonggang Zhang

Publication date 24 November 2014

Published in Mathematical Problems in Engineering (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1155/2013/561026

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Reasoning under uncertainty in the context of artificial intelligence (68T37)

Review text

Summary: Solving reinforcement learning problems in continuous space with function approximation is currently a research hotspot of machine learning. When dealing with the continuous space problems, the classic \(Q\)-iteration algorithms based on lookup table or function approximation converge slowly and are difficult to derive a continuous policy. To overcome the above weaknesses, we propose an algorithm named DFR-\(Sarsa(\lambda)\) based on double-layer fuzzy reasoning and prove its convergence. In this algorithm, the first reasoning layer uses fuzzy sets of state to compute continuous actions; the second reasoning layer uses fuzzy sets of action to compute the components of \(Q\)-value. Then, these two fuzzy layers are combined to compute the \(Q\)-value function of continuous action space. Besides, this algorithm utilizes the membership degrees of activation rules in the two fuzzy reasoning layers to update the eligibility traces. Applying DFR-\(Sarsa(\lambda)\) to the Mountain Car and Cart-pole Balancing problems, experimental results show that the algorithm not only can be used to get a continuous action policy, but also has a better convergence performance.

Recommendations

• Double fuzzy implications-based restriction inference algorithm
• SIS algorithms of Lukasiewicz-type intuitionistic fuzzy reasoning and its continuity
• An algebraic fuzzy structure for approximate reasoning
• scientific article; zbMATH DE number 2161055
• A fuzzy soft set based approximate reasoning method
• Fuzzy logic programming via multilattices
• On L-double fuzzy rough sets
• Double-quantitative rough fuzzy set based decisions: a logical operations method

Cites work

• An analysis of temporal-difference learning with function approximation
• Reinforcement distribution in fuzzy Q-learning
• Simulation-based algorithms for Markov decision processes.
• The convergence of \(TD(\lambda)\) for general \(\lambda\)
• Type-2 fuzzy logic: Theory and applications (Cover title: Contributions to fuzzy and rough sets theories and their applications)

Cited in

(1)

• Reinforcement Distribution in Continuous State Action Space Fuzzy Q–Learning: A Novel Approach

This page was built for publication: A \(Sarsa(\lambda)\) algorithm based on double-layer fuzzy reasoning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q473823)