Bauer's maximum principle and hulls of sets (Q1592846)

Bauer's maximum principle states that any convex function defined on a compact convex subset of \({\mathbf R}^n\) attains its maximum at some extreme point of this set. In the paper a version of Bauer's maximum principle is derived for polyconvex, quasiconvex and rank-one convex functions defined on compact subsets of \({\mathbf R}^n\). It is also shown that polyconvex, quasiconvex and rank-one convex hulls of compact sets are generated by polyconvex, quasiconvex and rank-one convex extreme points of the sets. The results are applied to the study of cones of polyconvex, quasiconvex and rank-one convex functions. Some properties of quasiconvex and rank-one convex extreme points are also discussed. In particular, it is shown that they are generally different, and that quasiconvex extreme points are not invariant under the composition with affine mappings which map rank-one matrices into rank-one matrices.