Ao usar a localização privada do monitoramento sintético com os alertas do New Relic, você pode ser notificado se um local estiver subprovisionado, mal configurado ou se comportar mal de maneira geral.
Este guia irá ajudá-lo a responder às seguintes perguntas básicas sobre saúde de localização privada usando o painel New Relic e alertas NRQL:
- Meus gerenciadores de jobs privados estão online?
- Meu local privado precisa de mais gerenciadores de jobs?
- Posso verificar o status de um gerenciador de jobs específico diretamente?
Pré-requisitos
Antes de seguir as instruções deste guia, certifique-se de ter:
- A Sintético localização privada
- Pelo menos um SJM instalado nesse local
- Verificações programadas para execução nesse local
- Uma política de alertas para o local privado, com um canal de notificação configurado para notificar sua equipe quando ocorrer um evento de alerta
O seguinte exemplo de JSON de dashboard do Private Job Manager pode ser importado para sua conta usando:
{ "name": "Synthetics Job Manager", "description": "Details on events from SyntheticsPrivateLocationStatus, SyntheticsPrivateMinion, SyntheticCheck, SyntheticRequest, NrAuditEvent, plus Monitor Analysis and Performance Analysis.", "permissions": "PUBLIC_READ_WRITE", "pages": [ { "name": "SyntheticsPrivateLocationStatus", "description": null, "widgets": [ { "title": "queue size by location", "layout": { "column": 1, "row": 1, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(checksPending) FACET name" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "private location queues", "layout": { "column": 4, "row": 1, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs) FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "queue size by location and type", "layout": { "column": 1, "row": 6, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs) FACET name " } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "rate of queue growth", "layout": { "column": 4, "row": 6, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT clamp_min(derivative(pingJobs,1 minute),0) AS 'ping',clamp_min(derivative(jobManagerHeavyweightJobs,1 minute),0) AS 'job manager heavy' FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "audit events for private locations", "layout": { "column": 1, "row": 11, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'PRIVATE_LOCATION' LIMIT MAX" } ] } } ] }, { "name": "SyntheticsPrivateMinion", "description": null, "widgets": [ { "title": "private locations", "layout": { "column": 1, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionLocation" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "details", "layout": { "column": 3, "row": 1, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(timestamp) AS 'latest timestamp',earliest(timestamp) AS 'earliest timestamp',(latest(timestamp) - earliest(timestamp))/864e5 AS 'max age (d)',latest(minionIpv4),max(minionJobsQueued),max(minionJobsFinished),100*max(minionJobsFailed)/max(minionJobsQueued) AS 'job failure rate',latest(minionJobsRunning),max(minionJobsTimedout),max(minionJobsSkipped),max(minionJobsInternalEngineError),latest(minionWorkers),latest(minionProcessors),latest(minionSwapMemoryUsedBytes/pow(1024,3)) AS 'used swap (GiB)',latest(minionSwapMemoryTotalBytes/pow(1024,3)) AS 'total swap (GiB)',latest(minionPhysicalMemoryUsedBytes/pow(1024,3)) AS 'used memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3)) AS 'total memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3))/latest(minionProcessors) AS 'memory (GiB) per CPU' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber,minionLocation,minionId,minionOsVersion,minionContainerSystemEnv,minionContainerSystemVersion,minionHostname LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "instances", "layout": { "column": 1, "row": 4, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "restarts", "layout": { "column": 3, "row": 4, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname,minionId TIMESERIES LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "builds", "layout": { "column": 1, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'version' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "OSes", "layout": { "column": 4, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'os versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionOsVersion LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Docker versions", "layout": { "column": 7, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'docker versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d{1,}[.\\d{1,}]{1,}).*') LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Kubernetes versions", "layout": { "column": 10, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'K8s versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d.\\d{2}).*') LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "CPU utilization (%)", "layout": { "column": 1, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionProcessorsUsagePercentage) AS 'avg',max(minionProcessorsUsagePercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "memory utilization (%)", "layout": { "column": 4, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionPhysicalMemoryUsedPercentage) AS 'average',max(minionPhysicalMemoryUsedPercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "used vs free memory (GiB)", "layout": { "column": 7, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionPhysicalMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionPhysicalMemoryTotalBytes)-max(minionPhysicalMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "used vs free swap (GiB)", "layout": { "column": 10, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionSwapMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionSwapMemoryTotalBytes)-max(minionSwapMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs running", "layout": { "column": 1, "row": 13, "width": 4, "height": 9 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsRunning) AS 'concurrent',clamp_min(derivative(minionJobsFinished,1 minute),0) AS 'finished per minute' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs finished", "layout": { "column": 5, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsFinished) - min(minionJobsFinished) AS 'total jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job failure rates", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT (max(minionJobsFailed/minionJobsReceived))*100 AS 'failed rate',(max(minionJobsInternalEngineError/minionJobsReceived))*100 AS 'IEE rate' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "jobs skipped", "layout": { "column": 5, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsSkipped) - min(minionJobsSkipped) AS 'skipped jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": false } } }, { "title": "count of Internal Engine Errors (IEE) vs failed jobs", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError) AS 'IEE',max(minionJobsFailed) AS 'failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": false } } }, { "title": "jobs timed out", "layout": { "column": 5, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsTimedout) - min(minionJobsTimedout) AS 'timed out jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE as a percentage of failed jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError/minionJobsFailed)*100 AS 'IEE as a percentage of failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 1", "description": null, "widgets": [ { "title": "Internal Engine Errors", "layout": { "column": 1, "row": 1, "width": 4, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'comparing IEE' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Internal Engine Errors", "layout": { "column": 5, "row": 1, "width": 8, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'IEE count' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE", "layout": { "column": 1, "row": 5, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,minionId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 5, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by IEE", "layout": { "column": 7, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 9, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by IEE", "layout": { "column": 11, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(id),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'jobs' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by code: 41", "layout": { "column": 1, "row": 8, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [ { "name": "count", "precision": 6, "type": "decimal" } ], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT filter(count(*),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error count', percentage(count(*), WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error rate' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 41", "layout": { "column": 5, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 31", "layout": { "column": 7, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 31') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 3", "layout": { "column": 9, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 3') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 39", "layout": { "column": 11, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 39') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping-runtime not enough memory", "layout": { "column": 1, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) WHERE type = 'SIMPLE' AND error LIKE '%/usr/bin%' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND runtimeType IS NOT NULL FACET error,location LIMIT 20 TIMESERIES" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "runaway monitors (frequency < minionDuration)", "layout": { "column": 4, "row": 11, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT minionDuration-frequency as 'magnitude',minionDuration,frequency,monitorId,type FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId,type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX) WHERE frequency < minionDuration AND frequency < 10 LIMIT MAX ORDER BY minionDuration DESC" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "runaway monitors", "layout": { "column": 9, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT percentage(uniqueCount(monitorId),WHERE frequency < minionDuration AND frequency < 10) as 'of monitors' FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX)" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 12, "row": 11, "width": 1, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Runaway monitors are defined as monitors that add jobs to the queue faster than they can be processed. \nIf any appear on the charts to the left, the job managers serving this private location will need to be scaled up in order to have enough heavy workers to accommodate these runaway monitors. An equilibrium will eventually be reached, but it will be an inefficient use of resources since many heavy workers will be occupied with jobs from the same monitor in parallel." } }, { "title": "IEE by monitor", "layout": { "column": 1, "row": 14, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,monitorId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "abnormal job duration by monitorId", "layout": { "column": 8, "row": 14, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg monitor frequency (m)", "layout": { "column": 1, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by location (m)", "layout": { "column": 3, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by type (m)", "layout": { "column": 5, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by status (m)", "layout": { "column": 7, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration", "layout": { "column": 9, "row": 17, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET minion WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 1, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 4, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration", "layout": { "column": 6, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue)", "layout": { "column": 8, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration", "layout": { "column": 10, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 1, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 4, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by location", "layout": { "column": 6, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by location", "layout": { "column": 8, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by location", "layout": { "column": 10, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET location WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 1, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 4, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration by runtime type", "layout": { "column": 6, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by runtime type", "layout": { "column": 8, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by runtime type", "layout": { "column": 10, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET runtimeType WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 1, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 4, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by type", "layout": { "column": 6, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by type", "layout": { "column": 8, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by type", "layout": { "column": 10, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 1, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 4, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by status", "layout": { "column": 6, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by status", "layout": { "column": 8, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by status", "layout": { "column": 10, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET result WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 1, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 4, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for successes by type", "layout": { "column": 6, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for successes by type", "layout": { "column": 8, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for successes by type", "layout": { "column": 10, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 1, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 4, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for failures by type", "layout": { "column": 6, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for failures by type", "layout": { "column": 8, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for failures by type", "layout": { "column": 10, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 2", "description": null, "widgets": [ { "title": "ping monitors by location", "layout": { "column": 1, "row": 47, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 ping errors", "layout": { "column": 5, "row": 47, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type = 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping duration with monitor count", "layout": { "column": 5, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping timeout vs other error durations", "layout": { "column": 7, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping jobs per minute", "layout": { "column": 9, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "ping timeout rate", "layout": { "column": 11, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping monitors by location", "layout": { "column": 1, "row": 53, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 non-ping errors", "layout": { "column": 5, "row": 53, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(id) as 'latest job id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type != 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping duration with monitor count", "layout": { "column": 5, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout vs other error durations", "layout": { "column": 7, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping jobs per minute", "layout": { "column": 9, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout rate", "layout": { "column": 11, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute", "layout": { "column": 1, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minion TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "errors per minute by type", "layout": { "column": 7, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error != '' AND error IS NOT NULL FACET type,result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by type", "layout": { "column": 1, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "error counts by class", "layout": { "column": 7, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT count(*) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result != 'SUCCESS' FACET capture(error, r'(?P<firstWord>\\S+)\\s.+') AS 'error class' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "monitor details", "layout": { "column": 1, "row": 67, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) AS 'results',uniqueCount(location)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET monitorId,type,nr.apiVersion,runtimeType LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticRequest", "description": null, "widgets": [ { "title": "locations", "layout": { "column": 1, "row": 1, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET locationLabel LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 error response codes", "layout": { "column": 6, "row": 1, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(responseCode) AS 'responses' WHERE responseCode NOT IN (200,201,202,203,204,300,301,302,303,304) AND responseCode IS NOT NULL AND location NOT LIKE 'AWS_%' FACET responseCode,responseStatus TIMESERIES AUTO LIMIT 5" } ], "yAxisLeft": { "zero": true } } }, { "title": "Average duration (ms) by location", "layout": { "column": 1, "row": 4, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max duration (ms) by URL", "layout": { "column": 6, "row": 4, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max duration (ms) by location", "layout": { "column": 1, "row": 7, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Average duration (ms) by URL", "layout": { "column": 6, "row": 7, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ] } }, { "title": "request rate", "layout": { "column": 1, "row": 10, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "request rate by job", "layout": { "column": 6, "row": 10, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' FACET id TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs", "layout": { "column": 1, "row": 13, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'jobs' WHERE location NOT LIKE 'AWS_%' FACET locationLabel,minion,minionId" } ] } }, { "title": "response status", "layout": { "column": 6, "row": 13, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseStatus LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content type", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentType LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "incomplete jobs", "layout": { "column": 1, "row": 16, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(uniqueCount(id),WHERE domComplete IS NULL) AS 'percent incomplete',uniqueCount(id) AS 'jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET monitorName,monitorId,runtimeType LIMIT 20" } ] } }, { "title": "response codes", "layout": { "column": 6, "row": 16, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseCode LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content category", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentCategory LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "percent of incomplete requests", "layout": { "column": 1, "row": 19, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(count(*),WHERE domComplete IS NULL) AS 'percent incomplete',count(*) AS 'requests',latest(responseCode),latest(responseStatus) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain" } ] } }, { "title": "complete vs incomplete jobs", "layout": { "column": 6, "row": 19, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'number of jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete')" } ] } }, { "title": "top 5 domains for incomplete jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) AS 'requests',latest(URL) WHERE domComplete IS NULL AND isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain LIMIT 5" } ] } }, { "title": "page load duration for complete vs incomplete jobs", "layout": { "column": 1, "row": 22, "width": 3, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete') TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "page load duration for complete vs incomplete jobs by monitor", "layout": { "column": 4, "row": 22, "width": 9, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete'),monitorName TIMESERIES MAX LIMIT MAX" } ], "yAxisLeft": { "zero": true } } }, { "title": "domains", "layout": { "column": 1, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "request type", "layout": { "column": 5, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET verb LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by domain", "layout": { "column": 9, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "hosts", "layout": { "column": 1, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET host LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "requests by URL", "layout": { "column": 5, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by URL", "layout": { "column": 9, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by URL", "layout": { "column": 1, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by domain", "layout": { "column": 7, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max request and response size by URL", "layout": { "column": 1, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max Request and Response size by Domain", "layout": { "column": 7, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ] } }, { "title": "audit events for monitors", "layout": { "column": 1, "row": 38, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'MONITOR' LIMIT MAX" } ] } } ] }, { "name": "Monitor Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 3, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Copy a Monitor Id from the table below and paste it into the box above to filter the charts on this page. It is represented by the {{monitor_id}} variable in NRQL queries so that `targetId` from `NrAuditEvent` can work alongside `monitorId` from `SyntheticCheck`." } }, { "title": "", "layout": { "column": 4, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET type WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 1, "width": 7, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT latest(nr.apiVersion),latest(runtimeType),latest(runtimeTypeVersion),latest(browserVersion),latest(scriptLanguage),latest(minionDeploymentMode) WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 2, "width": 3, "height": 7 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' WHERE location IN ({{private_location}}) FACET monitorId LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 2, "width": 2, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET result WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 2, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET result WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 4, "width": 4, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results',latest(minionId) FACET location WHERE monitorId = {{monitor_id}} LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 4, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET location WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 6, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 6, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error TIMESERIES MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 9, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM NrAuditEvent SELECT * WHERE targetId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Lightweight Job Performance Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "To estimate many CPU cores (heavyweight workers) will be needed to meet your private location's job demand, see [Sizing Considerations for Docker and Podman](https://docs.newrelic.com/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration/#vm-sizing)." } }, { "title": "count of lightweight worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors)*25 AS 'Lightweight Workers',latest(timestamp) FACET minionId,minionBuildVersion,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of job managers", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'ping monitors' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (SJM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "To estimate many CPU cores (heavyweight workers) will be needed to meet your private location's job demand, see [Sizing Considerations for Docker and Podman](https://docs.newrelic.com/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration/#vm-sizing)." } }, { "title": "minionIds and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors),latest(timestamp) FACET minionId,minionLocation,minionBuildVersion WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of job managers", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of job managers' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs per minute", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg success duration", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg failure duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "job failure rate", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of monitors", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg cron job duration", "layout": { "column": 9, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] } ], "variables": [ { "name": "private_location", "items": null, "defaultValues": [], "nrqlQuery": { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniques(minionLocation,10000) WHERE minionLocation NOT LIKE 'AWS_%' SINCE 1 week ago" }, "title": "Private Location", "type": "NRQL", "isMultiSelection": true, "replacementStrategy": "STRING" }, { "name": "monitor_id", "items": null, "defaultValues": [], "nrqlQuery": null, "title": "Monitor ID", "type": "STRING", "isMultiSelection": null, "replacementStrategy": "STRING" } ]}Etapas para importar:
- Copie o JSON dashboard e cole em um editor de texto.
- Substitua
"accountId": 0,e"accountIds": [ 0 ]pelo ID da sua conta New Relic ou lista de IDs para cada ocorrência no código JSON. - Copie o JSON dashboard do seu editor de texto e importe usando um dos métodos descritos acima.
- Edite todos os gráficos com os quais você gostaria de usar a filtragem de facetas .
Dica
Caso sua localização privada exista em uma conta pai e o monitor Sintético em uma subconta, insira o ID da conta pai para consulta NRQL que usa SyntheticPrivateLocationStatus e SyntheticsPrivateMinion, e o ID da subconta para consulta que usa SyntheticCheck e SyntheticRequest.
Meus gerenciadores de jobs privados estão online?
Para responder a esta pergunta, você pode se basear em atributos do evento SyntheticsPrivateMinion. Os gerenciadores de jobs do Synthetics privado enviam este evento para a New Relic a cada 30 segundos. Uma maneira simples de verificar se seus gerenciadores de jobs estão online é comparar a contagem única de IDs de minion com o número de gerenciadores de jobs que você espera que estejam online.
Para entender quantos gerenciadores de jobs estão reportando, execute este exemplo de consulta NRQL:
SELECT uniqueCount(minionId)FROM SyntheticsPrivateMinionWHERE minionLocation = '1-acme_okc_dc-309'Usando esta consulta, você pode criar uma condição de alerta para notificar sua equipe quando menos gerenciadores de jobs estiverem reportando do que o esperado. Esta condição está configurada com um limite estático de 2 units, o que significa que você receberá um alerta se algum dos seus gerenciadores de tarefas estiver offline.
Você pode verificar se a política de alerta funciona conforme o esperado parando manualmente um dos seus gerenciadores de tarefas. Então, quando o incidente de alerta ocorrer, você será notificado por quaisquer canais de notificação que tenham sido configurados. Assim que o gerenciador de tarefas for reiniciado e voltar a ficar online, o alerta será recuperado.
Existem maneiras mais robustas de verificar se os gerenciadores de jobs estão funcionando corretamente, mas esta consulta e condição tratam de forma simples e bem-sucedida o caso em que uma máquina falha, é descomissionada acidentalmente ou o processo do gerenciador de jobs falha. Isso também garante que o gerenciador de jobs possa se comunicar com a New Relic.
Meu local privado precisa de mais gerenciadores de jobs?
Para responder a essa pergunta, você pode usar o atributo checksPending do evento SyntheticsPrivateLocationStatus. O atributo checksPending reflete o número de verificações de monitor agendadas (ou "enfileiradas"), mas que ainda não foram aceitas por um gerenciador de tarefas de sintéticos no local designado. Para um local com verificações agendadas e sem gerenciadores de jobs, este gráfico cresceria linearmente para cima e para a direita.
Atributos adicionais estão disponíveis para identificar quais tipos de trabalho estão fazendo com que o atributo checksPending cresça e onde concentrar seus esforços de resolução de problemas:
pingJobssão executados pelo gerenciador de tarefas do SyntheticsjobManagerHeavyweightJobssão jobs não-ping esperando para serem executados em tempos de execução suportados pelo gerenciador de jobs Sintético
Essa métrica é mais complicada de monitor do que uniqueCount(minionId) porque um valor alto não significa necessariamente que o local esteja em mau estado. Contanto que a métrica não esteja crescendo linearmente para cima e para a direita (e as verificações estejam sendo executadas dentro do cronograma), o local estará em bom estado.
Este caso de uso é perfeito para anomalia NRQL condição do alerta, que permite monitor o Desvio de uma métrica em vez de seu valor estático. Por exemplo:
SELECT average(checksPending)FROM SyntheticsPrivateLocationStatusWHERE name = '1-acme_tokyo_dc-512'Para testar esta condição do alerta, agende um monitor baseado em browser de um minuto para ser executado em seu local. Os trabalhos baseados em browser consomem mais recursos do que os trabalhos de ping, e é por isso que são mais adequados para simulação de carga. A New Relic irá notificá-lo rapidamente sobre um número crescente de checks pendentes.
Após duplicar o número de gerenciadores de jobs para lidar com a carga, o alerta se recupera. Por exemplo, usando o exemplo de dashboard Synthetics private location, observe o crescimento e o declínio das verificações pendentes ao longo do incidente e da recuperação. Ao usar a condição NRQL, o New Relic notificará você se e quando o local precisar de mais capacidade do gerenciador de tarefas.
Posso verificar o status de um gerenciador de jobs específico diretamente?
Você também pode verificar como um gerenciador de jobs está operando entrando em contato diretamente com ele. Você pode usar um conjunto de endpoints HTTP expostos pelo gerenciador de jobs para determinar o que a aplicação está fazendo. Para acessar esses endpoints, vincule as portas 8080 e 8180 a portas no host para o gerenciador de jobs do synthetics. Por exemplo, para o Docker, use docker run -p 8080:8080 -p 8082:8082 ...):
:8080/status/check: Detalhes sobre verificações de integridade internas que o gerenciador de jobs executa; HTTP 200 significa "saudável".:8080/status: Detalhes sobre o status de um gerenciador de tarefas; os mesmos dados são então relatados como um eventoSyntheticsPrivateMinion.:8082/: endpoint de administração do aplicativo JVM ; uma visão avançada do estado interno de um gerente de trabalho.
Esta abordagem não é tão automatizada ou flexível quanto o exemplochecksPending . No entanto, se houver uma falha total de conectividade de rede, essa abordagem manual pode ajudar a solucionar a situação.