Fish-hook shaped global bifurcation branch of a spatially heterogeneous cooperative system with cross-diffusion (Q635440)

The authors consider the strongly coupled cooperative system \[ \begin{aligned} & \Delta u+u(\lambda -u+b(x)v)=0, \\ & \Delta [ (1+k\rho (x)u)v]+v(\mu -v+d(x)u)=0, \end{aligned} \] in a spatially heterogeneous environment \(\Omega \), with homogeneous Neumann boundary conditions for \(u\) and \(v.\) In this system \(b\) and \(d\) are nonnegative (and not identically zero) continuous functions in \(\overline{ \Omega },\) \(\rho \) is a smooth positive function in \(\overline{\Omega },\) whose normal derivative to \(\partial \Omega \) vanishes, \(k\) is a positive constant and \(\lambda \) and \(\mu \) are real constants. If \(\mu >0\) the authors show that if \(\left| \mu \right| \) is small and \(k\) is large, a spatial segregation of \(\rho (x)\) and \(b(x)\) can produce the positive solution curve to form an unbounded fish-hook shaped curve with parameter \(\lambda .\) When \(\mu <0,\) if \(\left| \mu \right| \) is small and \(k\) is large, and the cooperative effect is strong for the species \(u\) and very weak for the species \(v,\) then the positive solution set forms an unbounded fish-hook shaped continuum. These results are different from those of predator-prey systems and the cooperative system with Dirichlet boundary conditions, which reveal the formation of a bounded continuum.