Compositions and averages of two resolvents: relative geometry of fixed points sets and a partial answer to a question by C.\,Byrne (Q548635)

Given two positive numbers \(\lambda_{1}, \lambda_{2}\) with \(\lambda _{1}+\lambda_{2}=1\) and two maximal monotone operators \(A_{1}, A_{2}\) in a Hilbert space, the resolvent average of \(A_{1},A_{2}\) with weights \(\lambda_{1},\lambda_{2}\) is the operator \(A=\left( \lambda_{1}J_{A_{1} }+\lambda_{2}J_{A_{2}}\right) ^{-1}-\text{*Id}\); it is recalled that the resolvent of an operator \(B\) is, by definition, the operator \(J_{B}=\left( B+\text{*Id}\right) ^{-1}\). The terminology ``resolvent average'' \ is due to the identity \(J_{A}=\lambda_{1}J_{A_{1}}+\lambda_{2}J_{A_{2}}\). In this paper, it is established that the set of fixed points of \(J_{A}\) is homeomorphic to the set of fixed points of \(J_{A_{1}/\lambda_{2}}J_{A_{2}/\lambda_{1}}\). The special case where \(A_{1}\) and \(A_{2}\) are the subdifferentials of two proper, lower semicontinuous convex functions is also treated.