-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Derived Telemetry Prototype #7815
Open
scottbell
wants to merge
106
commits into
master
Choose a base branch
from
telemetry-comps
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
106 commits
Select commit
Hold shift + click to select a range
4316003
skeleton for comps
scottbell 6c74e84
more scaffolding
scottbell f5d7a33
more drafts
scottbell 0dd0442
add comps manager
scottbell ff81454
worker ready
scottbell 74c3a95
telemetry is null for some reason
scottbell ce5e435
can add two sin waves
scottbell 69db534
adding works
scottbell 90a24b3
exact values enabled
scottbell 37d222f
subscriptions work
scottbell 55c8609
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell a94c752
slowing adding gui
scottbell 0113ec0
fix plots
scottbell a0a2ead
rudimentary expression editor
scottbell e5719fc
correctly persist
scottbell 0326e38
parameters
scottbell 413338d
can take arbitrary expressions
scottbell f174515
styles and timing for loading worker
scottbell 9e3e739
ui works kinda
scottbell dcc8938
need to figure out ranger
scottbell 87dc272
works with time
scottbell df3ca84
timesystem works
scottbell 484a81b
get rid of batch for now
scottbell 446c811
gui still leaking data
scottbell aa8750e
memory conserved
scottbell 4d546fb
styles
scottbell d6c8bee
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell d7d7913
fix initial load
scottbell d4b2986
expression validator
scottbell a9d63b9
fix test data
scottbell e4dcda8
do not persist
scottbell 58568b8
persist after adding
scottbell d96c3fc
fix editing issues
scottbell 28ec13a
mutate when parameters or expression changes
scottbell 386c3b4
implement output format
scottbell 5af6413
implement output format
scottbell 1c68c7e
be consistent with output key
scottbell 5894363
add icons and fix errant telemetry
scottbell 40a95cf
lint
scottbell ccc12de
lint
scottbell 3ad21ee
use key instead of name
scottbell 4260cc3
Style enhancements for Derived Telemetry object
charlesh88 cce3008
Style enhancements for Derived Telemetry object
charlesh88 2793da4
Enhancements for Derived Telemetry object
charlesh88 1d686c2
Style enhancements for Derived Telemetry object
charlesh88 f8ceaa5
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 e1f50fe
Enhancements for Derived Telemetry object
charlesh88 eac4676
do not apply test data when edit mode changes to false
scottbell 02cf701
Revert "do not apply test data when edit mode changes to false"
scottbell 9aeb454
trying this again without blowing away the very pretty styling
scottbell fb8730c
clean up composition effects
scottbell 06e916e
allow deleting of parameters
scottbell 1996e66
can chain and plot derived data
scottbell dde0d1a
can delete properly
scottbell cde34e3
Enhancements for Derived Telemetry object
charlesh88 fdcece8
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 80d8bab
handle arrays and add skeleton tests
scottbell 1615c36
Merge branch 'telemetry-comps' of github.com:nasa/openmct into teleme…
charlesh88 59461d6
Enhancements for Derived Telemetry object
charlesh88 1b186d7
add tests
scottbell b251fde
ensure output format propagates
scottbell 89e18d4
add output format test
scottbell dd57d78
add some basic object path label tests
scottbell 8b4eed9
good job code scanner. changing to or condition
scottbell 60e07e6
refactor aria labeling
scottbell e02217a
Merge branch 'master' into telemetry-comps
scottbell c1a361d
Merge branch 'master' into telemetry-comps
scottbell ee612a6
resolve conflicts
scottbell dfcfa47
pass options on request to underlying collections
scottbell 4a301a1
ensure derived DERIVED telemetry loads in the proper order. also have…
scottbell eb5d32c
fix domains
scottbell 5e013b6
ensure we only ask for latest for comp editor
scottbell 1a94010
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell 8e5ac68
improvement in loading
scottbell 154e8c6
fix slow loading errors
scottbell 4bbbd17
more telemetry collection changes to acommodate bound changes
scottbell 38316bd
more debug just in case
scottbell f718ccd
stacked plots are overriding telemetry object configurations
scottbell 58d6cdb
some changes
scottbell d859322
add new flag to deal with plots asking for zoomed out data that shoul…
scottbell 395436a
works
scottbell 3f92deb
add sample size
scottbell c680694
check if sample size greater than zero
scottbell 8e7bfd0
ensure number for sample size
scottbell 2de628a
ensure we have a reference value before we slice
scottbell ba7f291
can do derived derived accumulated comps now
scottbell 7c01a5e
ensure we are checking for not equal
scottbell bf9d5ef
add a few more awaits
scottbell d6f5056
allow blank test values for arrays
scottbell 85a7769
check for blank test values
scottbell 3be414e
Merge remote-tracking branch 'origin/master' into telemetry-comps-wit…
scottbell 3b06c77
change to just value
scottbell 4de0354
impute requested data properly
scottbell 3b78d4d
fix output and add accumulation label
scottbell d5ee430
update output when comp changes
scottbell 6710ad0
remove unused function
scottbell 7d46afb
add placeholder
scottbell 00c8c0e
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell 51f73bb
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell 1516524
support lambda values
scottbell 96b1ef0
reduce size
4a4dabf
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell 9ca489c
widen size to accomodate sample size
scottbell 55063a0
removing debug statements for PR review
scottbell 64ae96e
Closes #7823
charlesh88 5683320
Merge remote-tracking branch 'origin/master' into telemetry-comps
scottbell File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
more debug just in case
- Loading branch information
commit 38316bd2f54cae9d420fb8ca3abcbb0c766e002a
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,287 +5,287 @@ | |
#openmct; | ||
#domainObject; | ||
#composition; | ||
#telemetryObjects = {}; | ||
#telemetryCollections = {}; | ||
#telemetryLoadedPromises = []; | ||
#telemetryOptions = {}; | ||
#loaded = false; | ||
#compositionLoaded = false; | ||
#telemetryProcessors = {}; | ||
|
||
constructor(openmct, domainObject) { | ||
super(); | ||
this.#openmct = openmct; | ||
this.#domainObject = domainObject; | ||
this.clearData = this.clearData.bind(this); | ||
} | ||
|
||
#getNextAlphabeticalParameterName() { | ||
const parameters = this.#domainObject.configuration.comps.parameters; | ||
const existingNames = new Set(parameters.map((p) => p.name)); | ||
const alphabet = 'abcdefghijklmnopqrstuvwxyz'; | ||
let suffix = ''; | ||
// eslint-disable-next-line no-constant-condition | ||
while (true) { | ||
for (let letter of alphabet) { | ||
const proposedName = letter + suffix; | ||
if (!existingNames.has(proposedName)) { | ||
return proposedName; | ||
} | ||
} | ||
// Increment suffix after exhausting the alphabet | ||
suffix = (parseInt(suffix, 10) || 0) + 1; | ||
} | ||
} | ||
|
||
addParameter(telemetryObject) { | ||
const keyString = this.#openmct.objects.makeKeyString(telemetryObject.identifier); | ||
const metaData = this.#openmct.telemetry.getMetadata(telemetryObject); | ||
const timeSystem = this.#openmct.time.getTimeSystem(); | ||
const domains = metaData?.valuesForHints(['domain']); | ||
const timeMetaData = domains.find((d) => d.key === timeSystem.key); | ||
// in the valuesMetadata, find the first numeric data type | ||
const rangeItems = metaData.valueMetadatas.filter( | ||
(metaDatum) => metaDatum.hints && metaDatum.hints.range | ||
); | ||
rangeItems.sort((a, b) => a.hints.range - b.hints.range); | ||
let valueToUse = rangeItems[0]?.key; | ||
if (!valueToUse) { | ||
// if no numeric data type, just use the first one | ||
valueToUse = metaData.valueMetadatas[0]?.key; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same here There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. SWGs have numeric data types, so we never fall through here. |
||
} | ||
this.#domainObject.configuration.comps.parameters.push({ | ||
keyString, | ||
name: `${this.#getNextAlphabeticalParameterName()}`, | ||
valueToUse, | ||
testValue: 0, | ||
timeMetaData | ||
}); | ||
this.emit('parameterAdded', this.#domainObject); | ||
} | ||
|
||
getParameters() { | ||
const parameters = this.#domainObject.configuration.comps.parameters; | ||
const parametersWithTimeKey = parameters.map((parameter) => { | ||
return { | ||
...parameter, | ||
timeKey: this.#telemetryCollections[parameter.keyString]?.timeKey | ||
}; | ||
}); | ||
return parametersWithTimeKey; | ||
} | ||
|
||
getTelemetryObjectForParameter(keyString) { | ||
return this.#telemetryObjects[keyString]; | ||
} | ||
|
||
getMetaDataValuesForParameter(keyString) { | ||
const telemetryObject = this.getTelemetryObjectForParameter(keyString); | ||
const metaData = this.#openmct.telemetry.getMetadata(telemetryObject); | ||
return metaData.valueMetadatas; | ||
} | ||
|
||
deleteParameter(keyString) { | ||
this.#domainObject.configuration.comps.parameters = | ||
this.#domainObject.configuration.comps.parameters.filter( | ||
(parameter) => parameter.keyString !== keyString | ||
); | ||
// if there are no parameters referencing this parameter keyString, remove the telemetry object too | ||
const parameterExists = this.#domainObject.configuration.comps.parameters.some( | ||
(parameter) => parameter.keyString === keyString | ||
); | ||
if (!parameterExists) { | ||
this.emit('parameterRemoved', this.#domainObject); | ||
} | ||
} | ||
|
||
setDomainObject(passedDomainObject) { | ||
this.#domainObject = passedDomainObject; | ||
} | ||
|
||
isReady() { | ||
return this.#loaded; | ||
} | ||
|
||
async load(telemetryOptions) { | ||
if (!_.isEqual(telemetryOptions, this.#telemetryOptions) && this.#loaded) { | ||
console.debug( | ||
`😩 Reloading comps manager ${this.#domainObject.name} due to telemetry options change`, | ||
`😩 Reloading comps manager ${this.#domainObject.name} due to telemetry options change. New bounds are: ${new Date(telemetryOptions.start).toISOString()} to ${new Date(telemetryOptions.end).toISOString()}`, | ||
telemetryOptions | ||
); | ||
this.#destroy(); | ||
} | ||
this.#telemetryOptions = telemetryOptions; | ||
if (!this.#compositionLoaded) { | ||
await this.#loadComposition(); | ||
this.#compositionLoaded = true; | ||
} | ||
if (!this.#loaded) { | ||
await this.#startListeningToUnderlyingTelemetry(); | ||
this.#telemetryLoadedPromises = []; | ||
this.#loaded = true; | ||
} | ||
} | ||
|
||
async #startListeningToUnderlyingTelemetry() { | ||
Object.keys(this.#telemetryCollections).forEach((keyString) => { | ||
if (!this.#telemetryCollections[keyString].loaded) { | ||
this.#telemetryCollections[keyString].on('add', this.#getTelemetryProcessor(keyString)); | ||
this.#telemetryCollections[keyString].on('clear', this.clearData); | ||
const telemetryLoadedPromise = this.#telemetryCollections[keyString].load(); | ||
this.#telemetryLoadedPromises.push(telemetryLoadedPromise); | ||
} | ||
}); | ||
await Promise.all(this.#telemetryLoadedPromises); | ||
this.#telemetryLoadedPromises = []; | ||
} | ||
|
||
#destroy() { | ||
this.stopListeningToUnderlyingTelemetry(); | ||
this.#composition = null; | ||
this.#telemetryCollections = {}; | ||
this.#compositionLoaded = false; | ||
this.#loaded = false; | ||
this.#telemetryObjects = {}; | ||
} | ||
|
||
stopListeningToUnderlyingTelemetry() { | ||
this.#loaded = false; | ||
Object.keys(this.#telemetryCollections).forEach((keyString) => { | ||
const specificTelemetryProcessor = this.#telemetryProcessors[keyString]; | ||
delete this.#telemetryProcessors[keyString]; | ||
this.#telemetryCollections[keyString].off('add', specificTelemetryProcessor); | ||
this.#telemetryCollections[keyString].off('clear', this.clearData); | ||
this.#telemetryCollections[keyString].destroy(); | ||
}); | ||
} | ||
|
||
getTelemetryObjects() { | ||
return this.#telemetryObjects; | ||
} | ||
|
||
async #loadComposition() { | ||
this.#composition = this.#openmct.composition.get(this.#domainObject); | ||
if (this.#composition) { | ||
this.#composition.on('add', this.#addTelemetryObject); | ||
this.#composition.on('remove', this.#removeTelemetryObject); | ||
await this.#composition.load(); | ||
} | ||
} | ||
|
||
getFullDataFrame(newTelemetry) { | ||
const dataFrame = {}; | ||
// can assume on data item | ||
const newTelemetryKey = Object.keys(newTelemetry)[0]; | ||
const newTelemetryData = newTelemetry[newTelemetryKey]; | ||
const otherTelemetryKeys = Object.keys(this.#telemetryCollections).filter( | ||
(keyString) => keyString !== newTelemetryKey | ||
); | ||
// initialize the data frame with the new telemetry data | ||
dataFrame[newTelemetryKey] = newTelemetryData; | ||
// initialize the other telemetry data | ||
otherTelemetryKeys.forEach((keyString) => { | ||
dataFrame[keyString] = []; | ||
}); | ||
|
||
// march through the new telemetry data and add data to the frame from the other telemetry objects | ||
// using LOCF | ||
|
||
newTelemetryData.forEach((newDatum) => { | ||
otherTelemetryKeys.forEach((otherKeyString) => { | ||
const otherCollection = this.#telemetryCollections[otherKeyString]; | ||
let insertionPointForNewData = otherCollection._sortedIndex(newDatum); | ||
const otherCollectionData = otherCollection.getAll(); | ||
if (insertionPointForNewData && insertionPointForNewData >= otherCollectionData.length) { | ||
insertionPointForNewData = otherCollectionData.length - 1; | ||
} | ||
// get the closest datum to the new datum | ||
const closestDatum = otherCollectionData[insertionPointForNewData]; | ||
if (closestDatum) { | ||
dataFrame[otherKeyString].push(closestDatum); | ||
} | ||
}); | ||
}); | ||
return dataFrame; | ||
} | ||
|
||
#removeTelemetryObject = (telemetryObjectIdentifier) => { | ||
const keyString = this.#openmct.objects.makeKeyString(telemetryObjectIdentifier); | ||
delete this.#telemetryObjects[keyString]; | ||
this.#telemetryCollections[keyString]?.destroy(); | ||
delete this.#telemetryCollections[keyString]; | ||
// remove all parameters that reference this telemetry object | ||
this.deleteParameter(keyString); | ||
}; | ||
|
||
requestUnderlyingTelemetry() { | ||
const underlyingTelemetry = {}; | ||
Object.keys(this.#telemetryCollections).forEach((collectionKey) => { | ||
const collection = this.#telemetryCollections[collectionKey]; | ||
underlyingTelemetry[collectionKey] = collection.getAll(); | ||
}); | ||
return underlyingTelemetry; | ||
} | ||
|
||
#getTelemetryProcessor(keyString) { | ||
if (this.#telemetryProcessors[keyString]) { | ||
return this.#telemetryProcessors[keyString]; | ||
} | ||
|
||
const telemetryProcessor = (newTelemetry) => { | ||
this.emit('underlyingTelemetryUpdated', { [keyString]: newTelemetry }); | ||
}; | ||
this.#telemetryProcessors[keyString] = telemetryProcessor; | ||
return telemetryProcessor; | ||
} | ||
|
||
#telemetryProcessor = (newTelemetry, keyString) => { | ||
this.emit('underlyingTelemetryUpdated', { [keyString]: newTelemetry }); | ||
}; | ||
|
||
clearData(telemetryLoadedPromise) { | ||
this.#loaded = false; | ||
this.#telemetryLoadedPromises.push(telemetryLoadedPromise); | ||
} | ||
|
||
setOutputFormat(outputFormat) { | ||
this.#domainObject.configuration.comps.outputFormat = outputFormat; | ||
this.emit('outputFormatChanged', outputFormat); | ||
} | ||
|
||
getOutputFormat() { | ||
return this.#domainObject.configuration.comps.outputFormat; | ||
} | ||
|
||
getExpression() { | ||
return this.#domainObject.configuration.comps.expression; | ||
} | ||
|
||
#addTelemetryObject = (telemetryObject) => { | ||
const keyString = this.#openmct.objects.makeKeyString(telemetryObject.identifier); | ||
this.#telemetryObjects[keyString] = telemetryObject; | ||
this.#telemetryCollections[keyString] = this.#openmct.telemetry.requestCollection( | ||
telemetryObject, | ||
this.#telemetryOptions | ||
); | ||
|
||
// check to see if we have a corresponding parameter | ||
// if not, add one | ||
const parameterExists = this.#domainObject.configuration.comps.parameters.some( | ||
(parameter) => parameter.keyString === keyString | ||
); | ||
if (!parameterExists) { | ||
this.addParameter(telemetryObject); | ||
} | ||
}; | ||
|
||
static getCompsManager(domainObject, openmct, compsManagerPool) { | ||
const id = openmct.objects.makeKeyString(domainObject.identifier); | ||
|
||
if (!compsManagerPool[id]) { | ||
compsManagerPool[id] = new CompsManager(openmct, domainObject); | ||
} | ||
|
||
return compsManagerPool[id]; | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why wasn't this covered by tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we'd need 26 different parameters before slapping numbers on them. e.g., a,b,..z,a1,b2,c2,etc.