Skip to content

Move StructureCrawler request to clientside widget #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
prestoncraw wants to merge 3 commits into
base: main
Choose a base branch
from
+173 −22
Open

Move StructureCrawler request to clientside widget #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b09a5ec
Move StructureCrawler request to clientside widget
prestoncraw Aug 28, 2026
57a9761
Add proxy route and add fallback logic
prestoncraw Aug 28, 2026
ab974e7
cleanup
prestoncraw Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions API/Visualizations/ESRIMapController.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ public ESRIMapController(IAPICredentialRetriever retriever) : base(retriever) {
public async ServerResponse GetLightningInfo(int eventID, int timeWindow, CancellationToken token) =>
await ForwardRequest(token).ConfigureAwait(false);

[Route("SubstationLocation/{eventID:int}"), HttpGet]
public async ServerResponse GetSubstationLocation(int eventID, CancellationToken token) =>
await ForwardRequest(token).ConfigureAwait(false);

[Route("NearestStructure/{station}/{line}"), HttpGet]
public async ServerResponse GetNearestStructure(string station, string line, CancellationToken token) =>
await ForwardRequest(token).ConfigureAwait(false);
Expand Down
191 changes: 169 additions & 22 deletions TSX/Widget/ESRIMap.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,13 @@ interface ILayerSetting {
}

interface IFaultInfo {
StationName: string,
Inception: number,
Key: string,
Value: string | number | null
}

interface IStructureLocation {
Latitude: number,
Longitude: number,
Distance: number,
AssetName: string
Longitude: number
}

interface ISettings {
Expand All @@ -73,6 +74,7 @@ interface ISettings {
ClientID: string,
PortalURL: string,
TransmissionLineQuery: string,
StructureCrawlerURL: string,
UserAuthentication: boolean
}

Expand Down Expand Up @@ -113,6 +115,7 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {

TransmissionLineLayer: `http://pq/arcgisproxynew/proxy.ashx?https://gis.tva.gov/arcgis/rest/services/EGIS_Transmission/Transmission_Grid_Restricted_2/MapServer/6`,
TransmissionLineQuery: `UPPER(LINENAME) like '%{line}%'`,
StructureCrawlerURL: `http://opsptpsnet.cha.tva.gov:8025/TLI/StructureCrawler/FaultFinder.asp?Station={StationID}&Line={LineAssetKey}&Mileage={FaultDistance}`,
UserAuthentication: false

},
Expand Down Expand Up @@ -161,6 +164,18 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {
/>
</div>
</div>
<div className="row">
<div className="col">
<Input<ISettings>
Record={props.Settings}
Field={'StructureCrawlerURL'}
Help={'The full structure crawler URL, including query parameters. Populate a parameter from fault information using its field name in braces, for example Station={StationID} or Mileage={FaultDistance}.'}
Setter={(record) => props.SetSettings(record)}
Valid={() => true}
Label={'Structure Crawler URL'}
/>
</div>
</div>
<div className="row">
<div className="col">
<Input<ISettings>
Expand Down Expand Up @@ -252,6 +267,13 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {
const [window, setWindow] = React.useState<number>(2);
const [layerErrors, setLayerErrors] = React.useState<string[]>([]);
const [authToken, setAuthToken] = React.useState<string>("");
const [structureLocation, setStructureLocation] = React.useState<IStructureLocation | null>(null);
const [structureStatus, setStructureStatus] = React.useState<Application.Types.Status>('uninitiated');
const [stationStatus, setStationStatus] = React.useState<Application.Types.Status>('uninitiated');
const [crawlerReturnedNoCoordinates, setCrawlerReturnedNoCoordinates] = React.useState<boolean>(false);
const locationWarning = getLocationWarning(structureStatus, stationStatus, crawlerReturnedNoCoordinates, structureLocation != null);
const mapWarning = [layerErrors.length > 0 ? `Unable to load ${layerErrors.length} map ${layerErrors.length === 1 ? 'layer' : 'layers'}.` : '', locationWarning]
.filter(message => message.length > 0).join(' ');

/* Get Lightning Info */
React.useEffect(() => {
Expand Down Expand Up @@ -328,6 +350,74 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {

}, [props.EventID])

/* Get the nearest structure location, falling back to the meter's substation location. */
React.useEffect(() => {
setStructureLocation(null);
setStructureStatus('uninitiated');
setStationStatus('uninitiated');
setCrawlerReturnedNoCoordinates(false);
Comment on lines 355 to +358

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would sepperate this. see comment below


let stationHandle: JQuery.jqXHR<IStructureLocation[]> | undefined;

/** Loads the event meter's substation location as a fallback. */
const loadStationLocation = () => {
setStationStatus('loading');
stationHandle = $.ajax({
type: 'GET',
url: `${props.HomePath}api/EventWidgets/ESRIMap/SubstationLocation/${props.EventID}`,
dataType: 'json',
cache: true
}) as JQuery.jqXHR<IStructureLocation[]>;

stationHandle.done((data) => {
setStructureLocation(data[0] ?? null);
setStationStatus('idle');
}).fail((response) => {
setStationStatus('error');
console.error('Unable to fetch the substation location: ' + JSON.stringify(response));

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is truely an error - in the sense that something is wrong with your data and most likel requires GPA intervention. I would throw an exception in this case

});
};

const station = getFaultInfoValue(faultInfo, 'StationID');
const line = getFaultInfoValue(faultInfo, 'LineAssetKey');
const distance = getFaultInfoValue(faultInfo, 'FaultDistance');

if (station.length === 0 || line.length === 0 || distance.length === 0 || props.Settings.StructureCrawlerURL.trim().length === 0)
return;

setStructureStatus('loading');
const handle = $.ajax({
type: 'GET',
url: resolveVars(props.Settings.StructureCrawlerURL, faultInfo),
dataType: 'text',
cache: true,
xhrFields: { withCredentials: true }
}).done((response) => {
const location = parseStructureLocation(response);
setCrawlerReturnedNoCoordinates(location == null);

if (location == null) {
setStructureStatus('idle');
loadStationLocation();
return;
}

setStructureLocation(location);
setStructureStatus('idle');
}).fail((response) => {
setStructureStatus('error');
console.error('Unable to fetch structure crawler data: ' + JSON.stringify(response));
loadStationLocation();
});

return () => {
if (handle?.abort != null)
handle.abort();
if (stationHandle?.abort != null)
stationHandle.abort();
};
}, [faultInfo, props.EventID, props.HomePath, props.Settings.StructureCrawlerURL]);

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the general logic is off somewhere.
(1) props.EventID changes fires off logic to get EventInfo
(1a) props.EventID changes fires off logic to get Meter Location
(2) EventInfo changes fires off call to structure crawler
(3) Map should show location from structure Crawler. Fall back to Meter location
(1) and (1a) are useEffects that fire on props.EventID
(2) is a useEffect that depends on FaultInfo

(3) is not really an effect (could be one but really doesn't necessarily fires of an calls, if anything it's just because of the need to drop the layer

I know error handling is important but there is too much cross dependency going on here I think


React.useEffect(() => {
map.current = leaflet.map(div.current, { center: [props.Settings.CenterLat, props.Settings.CenterLong], zoom: props.Settings.Zoom });
basemapLayer('Gray').addTo(map.current);
Expand Down Expand Up @@ -396,15 +486,17 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {

/* Adds fault marker */
React.useEffect(() => {
if (faultInfo.length === 0 || map.current == null) return;
if (structureLocation == null || map.current == null) return;

const fault_marker = leaflet.marker([faultInfo[0]?.Latitude, faultInfo[0]?.Longitude]).addTo(map.current);
const coordinates: [number, number] = [structureLocation.Latitude, structureLocation.Longitude];
const fault_marker = leaflet.marker(coordinates).addTo(map.current);
map.current.setView(coordinates, map.current.getZoom());

return () => {
map.current?.removeLayer(fault_marker);
}

}, [faultInfo]);
}, [structureLocation]);

/* Adds lightning markers */
React.useEffect(() => {
Expand Down Expand Up @@ -492,13 +584,13 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {
</div>
</div>
<link rel="stylesheet" href="node_modules/leaflet/dist/leaflet.css" />
{layerErrors.length > 0 ?
{mapWarning.length > 0 ?
<div className="row">
<div className="col">
<Alert Class='alert-warning'>Unable to load the {layerErrors.length} map layers.</Alert>
<Alert Class='alert-warning'>{mapWarning}</Alert>
</div>
</div> :
null
</div>
: null
}
<div className="row">
<div className="col">
Expand Down Expand Up @@ -581,30 +673,85 @@ const ESRIMap: EventWidget.IWidget<ISettings> = {
}
}

/** Replaces map-setting placeholders with values from fault information. */
function resolveVars(str: string, faultInfo: IFaultInfo[]): string {
let result = str;

for (const info of faultInfo) {
if (info.Value != null)
result = result.split(`{${info.Key}}`).join(info.Value.toString());
}

const vars = {
const aliases = {
'time': '',
'station': '',
'line': '',
'distance': ''
};

if (faultInfo.length > 0) {
const t = moment(faultInfo[0]?.Inception);
vars["time"] = t.utc().format('YYYY-MM-DDTHH') + ':' + (t.minutes() - t.minutes() % 5).toString();
vars["station"] = faultInfo[0]?.StationName.toUpperCase();
vars["line"] = faultInfo[0]?.AssetName.toUpperCase();
}
const t = moment(getFaultInfoValue(faultInfo, 'FaultTime'));
if (t.isValid())
aliases["time"] = t.utc().format('YYYY-MM-DDTHH') + ':' + (t.minutes() - t.minutes() % 5).toString();

let result = str;
for (const key in vars) {
if (str.includes(`\{${key}\}`))
result = result.replace(`\{${key}\}`, vars[key]);
aliases["station"] = getFaultInfoValue(faultInfo, 'StationID').toUpperCase();
aliases["line"] = getFaultInfoValue(faultInfo, 'LineAssetKey').toUpperCase();
aliases["distance"] = getFaultInfoValue(faultInfo, 'FaultDistance');
}

for (const key in aliases)
result = result.split(`{${key}}`).join(aliases[key]);

return result;
}

/** Returns a fault-information value as text. */
function getFaultInfoValue(faultInfo: IFaultInfo[], key: string): string {
const value = faultInfo.find(info => info.Key === key)?.Value;
return value == null ? '' : value.toString();
}

/** Parses the first valid latitude and longitude from a structure crawler HTML/CSV response. */
function parseStructureLocation(response: string): IStructureLocation | null {
const document = new DOMParser().parseFromString(response, 'text/html');
const csv = document.body.textContent?.trim() ?? '';
const lines = csv.split(/[\r\n]+/).map(line => line.trim()).filter(line => line.length > 0);

if (lines.length < 2)
return null;

const fields = lines[0].split(',').map(field => field.trim());
const latitudeIndex = fields.findIndex(field => field.toLowerCase() === 'latitude');
const longitudeIndex = fields.findIndex(field => field.toLowerCase() === 'longitude');

if (latitudeIndex === -1 || longitudeIndex === -1)
return null;

for (const line of lines.slice(1)) {
const values = line.split(',').map(value => value.trim());
const latitude = parseFloat(values[latitudeIndex]);
const longitude = parseFloat(values[longitudeIndex]);

if (Number.isFinite(latitude) && Number.isFinite(longitude))
return { Latitude: latitude, Longitude: longitude };
}

return null;
}

/** Returns the warning for the structure-to-substation fallback. */
function getLocationWarning(structureStatus: Application.Types.Status, stationStatus: Application.Types.Status, crawlerReturnedNoCoordinates: boolean, hasLocation: boolean): string {
if (stationStatus === 'uninitiated') return '';

const crawlerResult = structureStatus === 'error' ? 'failed' : crawlerReturnedNoCoordinates ? 'returned no coordinates' : '';
if (crawlerResult.length === 0) return '';
if (stationStatus === 'loading')
return `The structure crawler ${crawlerResult}. The map is attempting to use the meter's substation location instead.`;

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to rephrase the message.

if (hasLocation)
return `The structure crawler ${crawlerResult}. The map is using the meter's substation location instead.`;

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not even be a warning. if no a Line or not a Fault this could be perfectly normal. May need to revisit error messages

return `The structure crawler ${crawlerResult}, and no valid substation location was available.`;

@clackner-gpa clackner-gpa Aug 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is impossible. All Meters have a station. So this would be a true error not a warning

}

const LayerSettings = (props: { Layer: ILayerSetting, SetLayer: (layer: ILayerSetting | undefined) => void, Index: number }) => {
return <>
<div className="col-8">
Expand Down