We have a system that tracks the confidence score returned by the Knowledge Graph API on a dataset of several thousand known Entities. We have isolated the three aspects of these changes and measure them daily. We measure:
Magnitude: How many entities saw an increase or decrease (reach/breadth).
Amplitude: How significant that change, in confidence scores of the affected entities, was on a micro-level (scope/height).
Shift: How significant the change was to the overall score (scale/depth).
The graph below shows Magnitude (Volatility) in purple, and Shift in red. Amplitude is a measurement we are using internally and won’t be adding that in the foreseeable future.