Skip to contents

Categorical Cost Calculations

Source: vignettes/categorical.Rmd
categorical.Rmd

The purpose of this vignette is to present the calculations of the costs for the categorical distribution over NN classes.

Each time step tt belongs to group kk whose time stamps are the set TkT_{k}. A group has either an a priori known probability of being in each class p=(p1,…,pN)p = \left(p_{1},\ldots,p_{N}\right) or an unknown probability of being in each class λk=(λk,1,…,λk,N)\lambda_{k} = \left(\lambda_{k,1},\ldots,\lambda_{k,N}\right)

No Anomaly (Baseline)

The data generating distribution gives for t∈Tkt \in T_{k}

P(yt|p)=∏i=1Npiyt,i P\left(y_t \left| p \right.\right) = \prod\limits_{i=1}^{N} p_{i}^{y_{t,i}}

where yt,i=1y_{t,i}=1 if the ttth sample is in the iith class and zero otherwise. For convience let there be nkn_{k} samples in TkT_{k} of which nk,in_{k,i} are of class ii.

The cost is computed as twice the negative log likelhiood and there is no penalty term giving

CB(yt∈Tk|p)=−2∑t∈Tk∑i=1Nyt,ilog(pi)=−2∑i=1Nnk,ilog(pi) C_{B}\left(y_{t \in T_{k}} \left| p \right.\right) = -2 \sum\limits_{t \in T_{k}} \sum\limits_{i=1}^{N} y_{t,i} \log\left( p_{i} \right) =-2 \sum\limits_{i=1}^{N} n_{k,i} \log\left( p_{i} \right)

Anomaly

In the case of the anomaly the cost is computed by

CA(yt∈Tk|λk)=β−2∑t∈Tk∑i=1Nyt,ilog(λ̂k,i)=β−2∑i=1Nnk,ilog(λ̂k,i) C_{A}\left(y_{t \in T_{k}} \left| \lambda_{k} \right.\right) = \beta - 2 \sum\limits_{t \in T_{k}} \sum\limits_{i=1}^{N} y_{t,i} \log\left( \hat{\lambda}_{k,i} \right) = \beta - 2 \sum\limits_{i=1}^{N} n_{k,i} \log\left( \hat{\lambda}_{k,i} \right)

where

λ̂k,j=∑t∈Tkyt,j∑i=1N∑t∈Tkyt,i=nk,jnk \hat{\lambda}_{k,j} = \frac{ \sum\limits_{t \in T_{k}} y_{t,j} } { \sum\limits_{i=1}^{N} \sum\limits_{t \in T_{k}} y_{t,i} } = \frac{n_{k,j}}{n_{k}}

An anomalous region is created when

CA(yt∈Tk|λk)−CB(yt∈Tk|p)=β−2∑i=1Nnk,i(log(nk,i)−log(nkpi))<0 C_{A}\left(y_{t \in T_{k}} \left| \lambda_{k} \right.\right) - C_{B}\left(y_{t \in T_{k}} \left| p \right.\right) = \beta - 2 \sum\limits_{i=1}^{N} n_{k,i} \left( \log\left( n_{k,i} \right) - \log\left( n_{k} p_{i} \right) \right) <0

In the case of a poitn anomaly nk=nk,j=1n_{k}=n_{k,j}=1 giving

CA(yt∈Tk|λk)−CB(yt∈Tk|p)=β+2log(pj) C_{A}\left(y_{t \in T_{k}} \left| \lambda_{k} \right.\right) - C_{B}\left(y_{t \in T_{k}} \left| p \right.\right) = \beta + 2 \log\left( p_{j} \right)