Extreme value analysis without the largest values: what can be done?

DOI10.1017/S0269964818000542zbMATH Open1440.62174arXiv1712.07381OpenAlexW2964044417WikidataQ128496619 ScholiaQ128496619MaRDI QIDQ5111481FDOQ5111481

Authors: Jingjing Zou, Richard A. Davis, Gennady Samorodnitsky 

Publication date: 27 May 2020

Published in: Probability in the Engineering and Informational Sciences (Search for Journal in Brave)

Abstract: In this paper we are concerned with the analysis of heavy-tailed data when a portion of the extreme values is unavailable. This research was motivated by an analysis of the degree distributions in a large social network. The degree distributions of such networks tend to have power law behavior in the tails. We focus on the Hill estimator, which plays a starring role in heavy-tailed modeling. The Hill estimator for this data exhibited a smooth and increasing "sample path" as a function of the number of upper order statistics used in constructing the estimator. This behavior became more apparent as we artificially removed more of the upper order statistics. Building on this observation we introduce a new version of the Hill estimator. It is a function of the number of the upper order statistics used in the estimation, but also depends on the number of unavailable extreme values. We establish functional convergence of the normalized Hill estimator to a Gaussian process. An estimation procedure is developed based on the limit theory to estimate the number of missing extremes and extreme value parameters including the tail index and the bias of Hill's estimator. We illustrate how this approach works in both simulations and real data examples.

Full work available at URL: https://arxiv.org/abs/1712.07381

Recommendations

• Handling missing extremes in tail estimation
• On tail index estimation using a sample with missing observations
• Estimation of the maximal moment exponent with censored data
• New estimators of the extreme value index under random right censoring, for heavy-tailed distributions
• Gaussian approximation to the extreme value index estimator of a heavy-tailed distribution under random censoring

zbMATH Keywords

Hill estimatorheavy-tailed distributionsfunctional convergencemissing extremes

Mathematics Subject Classification ID

Gaussian processes (60G15) Extreme value theory; extremal stochastic processes (60G70) Statistics of extreme values; tail inference (62G32) Applications of statistics to environmental and related topics (62P12) Missing data (62D10) Applications of statistics to physics (62P35) Functional limit theorems; invariance principles (60F17) Seismology (including tsunami modeling), earthquakes (86A15)

Cites Work

• Extreme value theory. An introduction.
• An approximation of partial sums of independent RV's, and the sample DF. II
• An approximation of partial sums of independent RV'-s, and the sample DF. I
• Title not available (Why is that?)
• A simple general approach to inference about the tail of a distribution
• How to make a Hill plot.
• Networks. An introduction.
• On Smooth Statistical Tail Functionals
• Approximate distributions of order statistics. With applications to nonparametric statistics
• Tail product-limit process for truncated data with application to extreme value index estimation
• Tail estimates motivated by extreme value theory
• Parameter Estimation for the Truncated Pareto Distribution
• Tail fitting for truncated and non-truncated Pareto-type distributions
• Fitting tails affected by truncation
• Approximation of the Hill estimator process

Cited In (4)

• An enhanced method for tail index estimation under missingness
• Data-adaptive trimming of the Hill estimator and detection of outliers in the extremes of heavy-tailed data
• Handling missing extremes in tail estimation
• Estimation of tail parameters with missing largest observations

This page was built for publication: Extreme value analysis without the largest values: what can be done?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5111481)