pyspark.pandas.window.Expanding.var#

Expanding.var()[source]#

Calculate unbiased expanding variance.

Note

the current implementation of this API uses Spark’s Window without specifying partition specification. This leads to move all data into single partition in single machine and could cause serious performance degradation. Avoid this method against very large dataset.

Returns:
Series or DataFrame

Returns the same object type as the caller of the expanding calculation.

See also

pyspark.pandas.Series.expanding

Calling object with Series data.

pyspark.pandas.DataFrame.expanding

Calling object with DataFrames.

pyspark.pandas.Series.var

Equivalent method for Series.

pyspark.pandas.DataFrame.var

Equivalent method for DataFrame.

numpy.var

Equivalent method for Numpy array.

Examples

>>> s = ps.Series([5, 5, 6, 7, 5, 5, 5])
>>> s.expanding(3).var()
0         NaN
1         NaN
2    0.333333
3    0.916667
4    0.800000
5    0.700000
6    0.619048
dtype: float64

For DataFrame, each unbiased expanding variance is computed column-wise.

>>> df = ps.DataFrame({"A": s.to_numpy(), "B": s.to_numpy() ** 2})
>>> df.expanding(2).var()
          A           B
0       NaN         NaN
1  0.000000    0.000000
2  0.333333   40.333333
3  0.916667  130.250000
4  0.800000  113.000000
5  0.700000   98.566667
6  0.619048   87.000000