Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derived Telemetry Prototype #7815

Open
wants to merge 106 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
4316003
skeleton for comps
scottbell Jul 25, 2024
6c74e84
more scaffolding
scottbell Jul 29, 2024
f5d7a33
more drafts
scottbell Jul 30, 2024
0dd0442
add comps manager
scottbell Aug 1, 2024
ff81454
worker ready
scottbell Aug 2, 2024
74c3a95
telemetry is null for some reason
scottbell Aug 2, 2024
ce5e435
can add two sin waves
scottbell Aug 2, 2024
69db534
adding works
scottbell Aug 2, 2024
90a24b3
exact values enabled
scottbell Aug 5, 2024
37d222f
subscriptions work
scottbell Aug 6, 2024
55c8609
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Aug 8, 2024
a94c752
slowing adding gui
scottbell Aug 8, 2024
0113ec0
fix plots
scottbell Aug 12, 2024
a0a2ead
rudimentary expression editor
scottbell Aug 12, 2024
e5719fc
correctly persist
scottbell Aug 13, 2024
0326e38
parameters
scottbell Aug 13, 2024
413338d
can take arbitrary expressions
scottbell Aug 13, 2024
f174515
styles and timing for loading worker
scottbell Aug 14, 2024
9e3e739
ui works kinda
scottbell Aug 14, 2024
dcc8938
need to figure out ranger
scottbell Aug 14, 2024
87dc272
works with time
scottbell Aug 15, 2024
df3ca84
timesystem works
scottbell Aug 15, 2024
484a81b
get rid of batch for now
scottbell Aug 15, 2024
446c811
gui still leaking data
scottbell Aug 15, 2024
aa8750e
memory conserved
scottbell Aug 15, 2024
4d546fb
styles
scottbell Aug 16, 2024
d6c8bee
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Aug 19, 2024
d7d7913
fix initial load
scottbell Aug 19, 2024
d4b2986
expression validator
scottbell Aug 19, 2024
a9d63b9
fix test data
scottbell Aug 20, 2024
e4dcda8
do not persist
scottbell Aug 20, 2024
58568b8
persist after adding
scottbell Aug 20, 2024
d96c3fc
fix editing issues
scottbell Aug 21, 2024
28ec13a
mutate when parameters or expression changes
scottbell Aug 22, 2024
386c3b4
implement output format
scottbell Sep 3, 2024
5af6413
implement output format
scottbell Sep 3, 2024
1c68c7e
be consistent with output key
scottbell Sep 3, 2024
5894363
add icons and fix errant telemetry
scottbell Sep 4, 2024
40a95cf
lint
scottbell Sep 4, 2024
ccc12de
lint
scottbell Sep 4, 2024
3ad21ee
use key instead of name
scottbell Sep 4, 2024
4260cc3
Style enhancements for Derived Telemetry object
charlesh88 Sep 4, 2024
cce3008
Style enhancements for Derived Telemetry object
charlesh88 Sep 4, 2024
2793da4
Enhancements for Derived Telemetry object
charlesh88 Sep 5, 2024
1d686c2
Style enhancements for Derived Telemetry object
charlesh88 Sep 5, 2024
f8ceaa5
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 Sep 5, 2024
e1f50fe
Enhancements for Derived Telemetry object
charlesh88 Sep 5, 2024
eac4676
do not apply test data when edit mode changes to false
scottbell Sep 5, 2024
02cf701
Revert "do not apply test data when edit mode changes to false"
scottbell Sep 5, 2024
9aeb454
trying this again without blowing away the very pretty styling
scottbell Sep 5, 2024
fb8730c
clean up composition effects
scottbell Sep 5, 2024
06e916e
allow deleting of parameters
scottbell Sep 5, 2024
1996e66
can chain and plot derived data
scottbell Sep 5, 2024
dde0d1a
can delete properly
scottbell Sep 5, 2024
cde34e3
Enhancements for Derived Telemetry object
charlesh88 Sep 5, 2024
fdcece8
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 Sep 5, 2024
80d8bab
handle arrays and add skeleton tests
scottbell Sep 5, 2024
1615c36
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 Sep 5, 2024
59461d6
Enhancements for Derived Telemetry object
charlesh88 Sep 6, 2024
1b186d7
add tests
scottbell Sep 6, 2024
b251fde
ensure output format propagates
scottbell Sep 6, 2024
89e18d4
add output format test
scottbell Sep 6, 2024
dd57d78
add some basic object path label tests
scottbell Sep 7, 2024
8b4eed9
good job code scanner. changing to or condition
scottbell Sep 9, 2024
60e07e6
refactor aria labeling
scottbell Sep 10, 2024
e02217a
Merge branch 'master' into telemetry-comps
scottbell Sep 10, 2024
c1a361d
Merge branch 'master' into telemetry-comps
scottbell Sep 11, 2024
ee612a6
resolve conflicts
scottbell Sep 17, 2024
dfcfa47
pass options on request to underlying collections
scottbell Sep 24, 2024
4a301a1
ensure derived DERIVED telemetry loads in the proper order. also have…
scottbell Sep 25, 2024
eb5d32c
fix domains
scottbell Oct 1, 2024
5e013b6
ensure we only ask for latest for comp editor
scottbell Oct 1, 2024
1a94010
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Oct 2, 2024
8e5ac68
improvement in loading
scottbell Oct 3, 2024
154e8c6
fix slow loading errors
scottbell Oct 3, 2024
4bbbd17
more telemetry collection changes to acommodate bound changes
scottbell Oct 3, 2024
38316bd
more debug just in case
scottbell Oct 4, 2024
f718ccd
stacked plots are overriding telemetry object configurations
scottbell Oct 4, 2024
58d6cdb
some changes
scottbell Oct 4, 2024
d859322
add new flag to deal with plots asking for zoomed out data that shoul…
scottbell Oct 4, 2024
395436a
works
scottbell Oct 9, 2024
3f92deb
add sample size
scottbell Oct 9, 2024
c680694
check if sample size greater than zero
scottbell Oct 9, 2024
8e7bfd0
ensure number for sample size
scottbell Oct 9, 2024
2de628a
ensure we have a reference value before we slice
scottbell Oct 9, 2024
ba7f291
can do derived derived accumulated comps now
scottbell Oct 9, 2024
7c01a5e
ensure we are checking for not equal
scottbell Oct 9, 2024
bf9d5ef
add a few more awaits
scottbell Oct 9, 2024
d6f5056
allow blank test values for arrays
scottbell Oct 9, 2024
85a7769
check for blank test values
scottbell Oct 9, 2024
3be414e
Merge remote-tracking branch 'origin/master' into telemetry-comps-wit…
scottbell Oct 9, 2024
3b06c77
change to just value
scottbell Oct 9, 2024
4de0354
impute requested data properly
scottbell Oct 9, 2024
3b78d4d
fix output and add accumulation label
scottbell Oct 10, 2024
d5ee430
update output when comp changes
scottbell Oct 10, 2024
6710ad0
remove unused function
scottbell Oct 10, 2024
7d46afb
add placeholder
scottbell Oct 10, 2024
00c8c0e
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Oct 11, 2024
51f73bb
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Oct 14, 2024
1516524
support lambda values
scottbell Oct 14, 2024
96b1ef0
reduce size
Oct 22, 2024
4a4dabf
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Oct 28, 2024
9ca489c
widen size to accomodate sample size
scottbell Oct 28, 2024
55063a0
removing debug statements for PR review
scottbell Oct 28, 2024
64ae96e
Closes #7823
charlesh88 Nov 14, 2024
5683320
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix slow loading errors
  • Loading branch information
scottbell committed Oct 3, 2024
commit 154e8c695d021c474300f3d18396d7442ffabf07
31 changes: 21 additions & 10 deletions src/api/telemetry/TelemetryCollection.js
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ export default class TelemetryCollection extends EventEmitter {
this.isStrategyLatest = this.options.strategy === 'latest';
this.dataOutsideTimeBounds = false;
this.modeChanged = false;

console.debug(
`🫙 Created telemetry for ${this.domainObject.name} with bounds ${options.start} and ${options.end}`
);
}

/**
Expand All @@ -86,6 +90,16 @@ export default class TelemetryCollection extends EventEmitter {

this._setTimeSystem(this.options.timeContext.getTimeSystem());
this.lastBounds = this.options.timeContext.getBounds();
// prioritize passed options over time bounds
if (this.options.start) {
this.lastBounds.start = this.options.start;
}
if (this.options.end) {
this.lastBounds.end = this.options.end;
}
console.debug(
`🫙 Bounds for collection are start ${this.lastBounds.start} and end ${this.lastBounds.end}`
);
this._watchBounds();
this._watchTimeSystem();
this._watchTimeModeChange();
Expand Down Expand Up @@ -215,20 +229,13 @@ export default class TelemetryCollection extends EventEmitter {
let hasDataBeforeStartBound = false;
let size = this.options.size;
let enforceSize = size !== undefined && this.options.enforceSize;
// prioritize request bounds over time context bounds
let boundsToUse = this.lastBounds;
if (this.options.start) {
boundsToUse.start = this.options.start;
}
if (this.options.end) {
boundsToUse.end = this.options.end;
}
console.debug(`🫙 Bounds are telemetry are currently`, this.lastBounds);

// loop through, sort and dedupe
for (let datum of data) {
parsedValue = this.parseTime(datum);
beforeStartOfBounds = parsedValue < boundsToUse.start;
afterEndOfBounds = parsedValue > boundsToUse.end;
beforeStartOfBounds = parsedValue < this.lastBounds.start;
afterEndOfBounds = parsedValue > this.lastBounds.end;

if (
!afterEndOfBounds &&
Expand Down Expand Up @@ -335,6 +342,10 @@ export default class TelemetryCollection extends EventEmitter {
return;
}

console.debug(
`🫙 Bounds CHANGED for ${this.domainObject.name} are start ${bounds.start} and end ${bounds.end}`
);

let startChanged = this.lastBounds.start !== bounds.start;
let endChanged = this.lastBounds.end !== bounds.end;

Expand Down
12 changes: 6 additions & 6 deletions src/plugins/comps/CompsManager.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,11 @@
#composition;
#telemetryObjects = {};
#telemetryCollections = {};
#dataFrame = {};
#telemetryLoadedPromises = [];
#telemetryOptions = {};
#loaded = false;
#compositionLoaded = false;
#telemetryProcessors = {};
// make id random 4 digit number
#id = Math.floor(Math.random() * 9000) + 1000;

constructor(openmct, domainObject) {
super();
Expand All @@ -37,7 +34,7 @@
}
}
// Increment suffix after exhausting the alphabet
suffix = (parseInt(suffix, 10) || 0) + 1;

Check warning on line 37 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L37

Added line #L37 was not covered by tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why wasn't this covered by tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'd need 26 different parameters before slapping numbers on them. e.g., a,b,..z,a1,b2,c2,etc.

}
}

Expand All @@ -55,7 +52,7 @@
let valueToUse = rangeItems[0]?.key;
if (!valueToUse) {
// if no numeric data type, just use the first one
valueToUse = metaData.valueMetadatas[0]?.key;

Check warning on line 55 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L55

Added line #L55 was not covered by tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SWGs have numeric data types, so we never fall through here.

}
this.#domainObject.configuration.comps.parameters.push({
keyString,
Expand Down Expand Up @@ -89,16 +86,16 @@
}

deleteParameter(keyString) {
this.#domainObject.configuration.comps.parameters =

Check warning on line 89 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L89

Added line #L89 was not covered by tests
this.#domainObject.configuration.comps.parameters.filter(
(parameter) => parameter.keyString !== keyString

Check warning on line 91 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L91

Added line #L91 was not covered by tests
);
// if there are no parameters referencing this parameter keyString, remove the telemetry object too
const parameterExists = this.#domainObject.configuration.comps.parameters.some(
(parameter) => parameter.keyString === keyString

Check warning on line 95 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L94-L95

Added lines #L94 - L95 were not covered by tests
);
if (!parameterExists) {
this.emit('parameterRemoved', this.#domainObject);

Check warning on line 98 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L97-L98

Added lines #L97 - L98 were not covered by tests
}
}

Expand All @@ -111,9 +108,12 @@
}

async load(telemetryOptions) {
// if our options change, we need to reload the composition
if (!_.isEqual(telemetryOptions, this.#telemetryOptions) && this.#loaded) {
console.debug(

Check warning on line 112 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L112

Added line #L112 was not covered by tests
`😩 Reloading comps manager ${this.#domainObject.name} due to telemetry options change`,
telemetryOptions
);
this.#destroy();

Check warning on line 116 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L116

Added line #L116 was not covered by tests
}
this.#telemetryOptions = telemetryOptions;
if (!this.#compositionLoaded) {
Expand All @@ -121,13 +121,13 @@
this.#compositionLoaded = true;
}
if (!this.#loaded) {
await this.startListeningToUnderlyingTelemetry();
await this.#startListeningToUnderlyingTelemetry();
this.#telemetryLoadedPromises = [];
this.#loaded = true;
}
}

async startListeningToUnderlyingTelemetry() {
async #startListeningToUnderlyingTelemetry() {
Object.keys(this.#telemetryCollections).forEach((keyString) => {
if (!this.#telemetryCollections[keyString].loaded) {
this.#telemetryCollections[keyString].on('add', this.#getTelemetryProcessor(keyString));
Expand All @@ -141,12 +141,12 @@
}

#destroy() {
this.stopListeningToUnderlyingTelemetry();
this.#composition = null;
this.#telemetryCollections = {};
this.#compositionLoaded = false;
this.#loaded = false;
this.#telemetryObjects = {};

Check warning on line 149 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L144-L149

Added lines #L144 - L149 were not covered by tests
}

stopListeningToUnderlyingTelemetry() {
Expand All @@ -161,7 +161,7 @@
}

getTelemetryObjects() {
return this.#telemetryObjects;

Check warning on line 164 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L164

Added line #L164 was not covered by tests
}

async #loadComposition() {
Expand All @@ -185,7 +185,7 @@
dataFrame[newTelemetryKey] = newTelemetryData;
// initialize the other telemetry data
otherTelemetryKeys.forEach((keyString) => {
dataFrame[keyString] = [];

Check warning on line 188 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L188

Added line #L188 was not covered by tests
});

// march through the new telemetry data and add data to the frame from the other telemetry objects
Expand All @@ -193,16 +193,16 @@

newTelemetryData.forEach((newDatum) => {
otherTelemetryKeys.forEach((otherKeyString) => {
const otherCollection = this.#telemetryCollections[otherKeyString];
let insertionPointForNewData = otherCollection._sortedIndex(newDatum);
const otherCollectionData = otherCollection.getAll();
if (insertionPointForNewData && insertionPointForNewData >= otherCollectionData.length) {
insertionPointForNewData = otherCollectionData.length - 1;

Check warning on line 200 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L196-L200

Added lines #L196 - L200 were not covered by tests
}
// get the closest datum to the new datum
const closestDatum = otherCollectionData[insertionPointForNewData];
if (closestDatum) {
dataFrame[otherKeyString].push(closestDatum);

Check warning on line 205 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L203-L205

Added lines #L203 - L205 were not covered by tests
}
});
});
Expand All @@ -210,12 +210,12 @@
}

#removeTelemetryObject = (telemetryObjectIdentifier) => {
const keyString = this.#openmct.objects.makeKeyString(telemetryObjectIdentifier);
delete this.#telemetryObjects[keyString];
this.#telemetryCollections[keyString]?.destroy();
delete this.#telemetryCollections[keyString];

Check warning on line 216 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L213-L216

Added lines #L213 - L216 were not covered by tests
// remove all parameters that reference this telemetry object
this.deleteParameter(keyString);

Check warning on line 218 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L218

Added line #L218 was not covered by tests
};

requestUnderlyingTelemetry() {
Expand All @@ -229,7 +229,7 @@

#getTelemetryProcessor(keyString) {
if (this.#telemetryProcessors[keyString]) {
return this.#telemetryProcessors[keyString];

Check warning on line 232 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L232

Added line #L232 was not covered by tests
}

const telemetryProcessor = (newTelemetry) => {
Expand All @@ -240,7 +240,7 @@
}

#telemetryProcessor = (newTelemetry, keyString) => {
this.emit('underlyingTelemetryUpdated', { [keyString]: newTelemetry });

Check warning on line 243 in src/plugins/comps/CompsManager.js

View check run for this annotation

Codecov / codecov/patch

src/plugins/comps/CompsManager.js#L243

Added line #L243 was not covered by tests
};

clearData(telemetryLoadedPromise) {
Expand Down
Loading