How are we Measuring Knowledge Graph Changes?

We have a system that tracks the confidence score returned by the Knowledge Graph API on a dataset of several thousand known Entities. We have isolated the three aspects of these changes and measure them daily. We measure:

1. Magnitude: How many entities saw an increase or decrease (reach/breadth).
2. Amplitude: How significant that change, in confidence scores of the affected entities, was on a micro-level (scope/height).
3. Shift: How significant the change was to the overall score (scale/depth).

The graph below shows Magnitude (Volatility) in purple, and Shift in red. Amplitude is a measurement we are using internally and won’t be adding that in the foreseeable future.