stats

Power-law tail fits with likelihood-ratio comparisons against lognormal and exponential tails.

Heavy-tail tests for degree and strength distributions.

A straight line on a log-log plot is weak evidence of a power law. This module follows Clauset, Shalizi and Newman (2009), "Power-law distributions in empirical data", SIAM Review 51(4):

1. Estimate the exponent by maximum likelihood for every candidate lower bound xmin, and keep the xmin whose fit has the smallest Kolmogorov-Smirnov distance to the data.
2. Fit lognormal and exponential distributions to the same tail.
3. Compare each alternative with the power law using Vuong's normalized log-likelihood ratio test.

Degrees are integers and use the discrete likelihoods (Hurwitz zeta for the power law). Strengths are real-valued and use the continuous likelihoods.

The implementation uses SciPy directly instead of the powerlaw package, which would add matplotlib and mpmath as dependencies. The test suite checks the continuous estimates against powerlaw.

FitQuantity module-attribute

FitQuantity: TypeAlias = Literal[
    "degree",
    "in_degree",
    "out_degree",
    "strength",
    "in_strength",
    "out_strength",
]

Which per-node quantity to fit. Degrees are discrete; strengths continuous.

Distribution module-attribute

Distribution: TypeAlias = Literal[
    "power_law", "lognormal", "exponential"
]

A fitted distribution family: the power law, or one of the two alternatives it is compared against (lognormal and exponential).

DistributionComparison dataclass

DistributionComparison(
    alternative: str,
    loglikelihood_ratio: float,
    p_value: float,
)

Result of a likelihood-ratio test of the power law against one alternative.

Attributes:

Name	Type	Description
alternative	str	Name of the alternative distribution.
loglikelihood_ratio	float	Vuong's R: the summed log-likelihood of the power law minus that of the alternative, over the tail. Positive values favour the power law.
p_value	float	Two-sided p-value for R != 0. A large p-value means the data cannot tell the two distributions apart.

favoured

favoured(significance: float = 0.1) -> str

Return the better-supported distribution, or "inconclusive".

Parameters:

Name	Type	Description	Default
significance	float	Threshold for the p-value. Clauset et al. use 0.1.	0.1

Returns:

Type	Description
str	"power_law", the alternative's name, or "inconclusive".

PowerLawFit dataclass

PowerLawFit(
    quantity: str,
    discrete: bool,
    alpha: float,
    xmin: float,
    n: int,
    n_tail: int,
    ks_distance: float,
    vs_lognormal: DistributionComparison,
    vs_exponential: DistributionComparison,
    lognormal_mu: float,
    lognormal_sigma: float,
    exponential_rate: float,
    values: tuple[float, ...],
)

A power-law fit to the tail of a distribution, with model comparisons.

Attributes:

Name	Type	Description
quantity	str	What was fitted, for example "degree".
discrete	bool	Whether discrete likelihoods were used.
alpha	float	Estimated exponent of p(x) ~ x**-alpha for x >= xmin.
xmin	float	Lower bound of the power-law tail.
n	int	Number of positive observations.
n_tail	int	Number of observations at or above xmin.
ks_distance	float	Kolmogorov-Smirnov distance between the tail and the fitted power law.
vs_lognormal, vs_exponential		Likelihood-ratio tests against the two alternatives.
lognormal_mu, lognormal_sigma		Parameters of the lognormal fitted to the tail.
exponential_rate	float	Rate of the exponential fitted to the tail.
values	tuple[float, ...]	The observations, sorted ascending. Excluded from repr and equality.

ccdf

ccdf(
    x: ArrayLike, distribution: Distribution = "power_law"
) -> FloatArray

Evaluate a fitted model's complementary CDF, conditional on x >= xmin.

Parameters:

Name	Type	Description	Default
x	ArrayLike	Points at or above xmin.	required
distribution	Distribution	"power_law", "lognormal" or "exponential".	'power_law'

Returns:

Type	Description
ndarray	P(X >= x | X >= xmin).

summary

summary() -> str

Return a one-paragraph plain-text summary of the fit.

fit_power_law

fit_power_law(
    values: ArrayLike,
    *,
    discrete: bool,
    xmin: float | None = None,
    min_tail: int = 10,
    quantity: str = "values",
) -> PowerLawFit

Fit a power law to the tail of a sample and compare it with alternatives.

Parameters:

Name	Type	Description	Default
values	ArrayLike	Observations. Non-positive values are ignored.	required
discrete	bool	Use discrete likelihoods. Values are then expected to be integers.	required
xmin	float | None	Fix the lower bound instead of choosing it by KS distance.	None
min_tail	int	Smallest tail size considered when choosing xmin.	10
quantity	str	Label stored on the result.	'values'

Returns:

Type	Description
PowerLawFit	The fit and its comparisons with lognormal and exponential models.

Raises:

Type	Description
InsufficientDataError	If fewer than min_tail positive observations are available, or all tail values are equal.

degree_distribution_fit

degree_distribution_fit(
    g: Graph[Any],
    quantity: FitQuantity = "degree",
    *,
    xmin: float | None = None,
    min_tail: int = 10,
) -> PowerLawFit

Test whether a graph's degree or strength distribution has a power-law tail.

Parameters:

Name	Type	Description	Default
g	Graph[Any]	A graph; strengths use the edge attribute "weight".	required
quantity	FitQuantity	Per-node quantity to fit; see FitQuantity.	'degree'
xmin	float | None	Fix the lower bound of the tail instead of estimating it.	None
min_tail	int	Smallest tail size considered when estimating xmin.	10

Returns:

Type	Description
PowerLawFit	Exponent, xmin, goodness of fit, and likelihood-ratio tests against lognormal and exponential tails.

Raises:

Type	Description
UnknownMetricError	If quantity is not supported.
InsufficientDataError	If the graph is too small for a tail fit.

Examples:

>>> import netviz_tools as nv
>>> g = nv.build_graph(nv.datasets.faostat.load_sample(items="Wheat", years=2021))
>>> fit = nv.stats.degree_distribution_fit(g, "out_strength")
>>> fit.vs_lognormal.alternative
'lognormal'