de-cellprofiler cloudwatch dashboard

Author: ErinWeisbart

documentation/DS-documentation/customizing_DS.md

@@ -90,4 +90,6 @@ More configuration information is available in [Step 1: Configuration](step_1_co
 
 #### run.py
 
+You need to customize the Dashboard creation function by changing 'start run' to whatever your run command is.
 If you have changed anything in config.py, you will need to edit the section on Task Definitions to match.
+

documentation/DS-documentation/dashboard.md

@@ -64,16 +64,17 @@ The relative slope of your graph can also be informative.
 For the run pictured at top, we discovered that a fraction of our jobs were erroring because the machines were running out of memory. 
 Midway through 7/12 we upped the memory of the machines in our fleet and you can see from that point on a greater slope as more jobs were finishing in the same amount of time (as fewer were failing to complete because of memory errors.)
 
-## Distinct Logs with “cellprofiler -c”
+## Distinct Logs
 
-![Distinct Logs with “cellprofiler -c”](images/logs_comparison.png)
+![Logs comparison](images/logs_comparison.png)
 
-This widget shows you the number of different specific CellProfiler jobs that start within your given time window. 
+This widget shows you the number of different specific jobs that start within your given time window by plotting the number of Cloudwatch logs that have your run command in them. 
+In this example, our run command is "cellprofiler -c".
 It is not necessarily informative on its own, but very helpful when compared with the following widget.
 
-## “cellprofiler -c”
-This widget shows you the number of times that CellProfiler is started within your log group within the given time window. 
-Ideally, you want this number to match the number in the previous widget as it means that each job is starting in CellProfiler only once. 
+## All logs
+This widget shows you the number of total times that jobs are started within your log group within the given time window. 
+Ideally, you want this number to match the number in the previous widget as it means that each job is starting in your software only once. 
 
 If this number is consistently larger than the previous widget’s number, it could mean that some of your jobs are erroring and you’ll need to figure out why (see MemoryUtilization and Show Errors or look directly in your Cloudwatch Logs for insights).
 

run.py

@@ -342,6 +342,7 @@ def export_logs(logs, loggroupId, starttime, bucketId):
 
 def create_dashboard(requestInfo):
     cloudwatch = boto3.client("cloudwatch")
+    # Change 'start run' to whatever your run command is
     DashboardMessage = {
         "widgets": [
             {
@@ -391,7 +392,7 @@ def create_dashboard(requestInfo):
                 "type": "metric",
                 "properties": {
                     "metrics": [
-                        [ "AWS/SQS", "ApproximateNumberOfMessagesVisible", "QueueName", "2021_04_26_Production_AnalysisQueue" ],
+                        [ "AWS/SQS", "ApproximateNumberOfMessagesVisible", "QueueName", f"{APP_NAME}Queue" ],
                         [ ".", "ApproximateNumberOfMessagesNotVisible", ".", "."],
                     ],
                     "view": "timeSeries",
@@ -408,10 +409,10 @@ def create_dashboard(requestInfo):
                 "x": 12,
                 "type": "log",
                 "properties": {
-                    "query": f"SOURCE {APP_NAME} | fields @message| filter @message like 'cellprofiler -c'| stats count_distinct(@message)\n",
+                    "query": f"SOURCE {APP_NAME} | fields @message| filter @message like 'start run'| stats count_distinct(@message)",
                     "region": AWS_REGION,
                     "stacked": False,
-                    "title": "Distinct Logs with \"cellprofiler -c\"",
+                    "title": "Distinct Logs",
                     "view": "table"
                 }
             },
@@ -422,10 +423,10 @@ def create_dashboard(requestInfo):
                 "x": 0,
                 "type": "log",
                 "properties": {
-                    "query": f"SOURCE {APP_NAME} | fields @message| filter @message like 'cellprofiler -c'| stats count(@message)",
+                    "query": f"SOURCE {APP_NAME} | fields @message| filter @message like 'start run'| stats count(@message)",
                     "region": AWS_REGION,
                     "stacked": False,
-                    "title": "All Logs \"cellprofiler -c\"",
+                    "title": "All Logs",
                     "view": "table"
                 }
             },
@@ -436,7 +437,7 @@ def create_dashboard(requestInfo):
                 "x": 0,
                 "type": "log",
                 "properties": {
-                    "query": f"SOURCE {APP_NAME} | fields @message\n\n   | filter @message like \"Error\"\n\n   | display @message\n",
+                    "query": f"SOURCE {APP_NAME} | fields @message | filter @message like \"Error\"\n\n   | display @message",
                     "region": AWS_REGION,
                     "stacked": False,
                     "title": "Errors",