Edits to documentation (#601)

* Remove bash syntax highlighting for CLI

* Reorder guides and add integrations engine page to mkdocs.yml

* Fix links in concepts/overview and quick_start

* Edit guides/connections

* Edit guides/scheduling

* Edit guides/models

* Edit integrations/github

* minor edits to reference/ docs

Author: treysp

docs/concepts/overview.md

@@ -59,7 +59,7 @@ In contrast to tests, SQLMesh "audits" validate the results of model code applie
 
 You create audits by writing SQL queries that should return 0 rows. For example, an audit query to ensure `your_field` has no `NULL` values would include `WHERE your_field IS NULL`. If any NULLs are detected, the query will return at least one row and the audit will fail.
 
-Audits are flexible - they can be tied to a specific model's contents, or you can use [macros](concepts/macros.md) to create audits usable by multiple models. SQLMesh also includes pre-made audits for common use cases like detecting NULL or duplicated values.
+Audits are flexible - they can be tied to a specific model's contents, or you can use [macros](macros.md) to create audits usable by multiple models. SQLMesh also includes pre-made audits for common use cases like detecting NULL or duplicated values.
 
 You specify which audits should run for a model by including them in the model's metadata properties. 
 

docs/guides/connections.md

@@ -4,34 +4,34 @@
 
 **Note:** The following guide only applies when using the built-in scheduler. Connections are configured differently when using an external scheduler like Airflow. See the [Scheduling guide](scheduling.md) for more details.
 
-In order to deploy models and apply changes to them, a connection to the Data Warehouse must first be configured. This can be done through the `config.yaml` file in your project folder or in the `~/.sqlmesh` one.
+In order to deploy models and apply changes to them, you must configure a connection to the Data Warehouse. This can be done in either the `config.yaml` file in your project folder or the one in `~/.sqlmesh`.
 
-Each configured connection has a unique name associated with it, which can be used to select a specific connection when using CLI. For example:
+Each configured connection has a unique name associated with it, which can be used to select a specific connection when using the CLI. For example:
 ```yaml linenums="1"
 connections:
     local_db:
         type: duckdb
 ```
 
-Now the defined connection can be specified in CLI as follows:
+Now the defined connection can be specified in the `sqlmesh plan` CLI command as follows:
 ```bash
 sqlmesh --connection local_db plan
 ```
 
 ## Default connection
-If no connection name is provided, then the first connection in the order of how connections are defined in the configuration will be picked.
+If no connection name is provided, then the first connection in the `config.yaml` connections specification will be used.
 
-Additionally the default connection can be set in the configuration:
+Additionally, you can set a default connection by specifying the connection name in the `default_connection` key:
 ```yaml linenums="1"
 default_connection: local_db
 ```
 
 ## Test connection
-By default, when running [tests](../concepts/tests.md) SQLMesh uses an in-memory DuckDB database connection. This behavior can be overridden in the configuration:
+By default, when running [tests](../concepts/tests.md) SQLMesh uses an in-memory DuckDB database connection. You can override this behavior by specifying a connection name in the `test_connection` key:
 ```yaml linenums="1"
 test_connection: local_db
 ```
-Or in CLI:
+Or you can specify the test connection in the `sqlmesh plan` CLI command:
 ```bash
 sqlmesh --test-connection local_db plan
 ```
@@ -41,11 +41,11 @@ sqlmesh --test-connection local_db plan
 ### BigQuery
 TBD
 
-See the [configuration reference](../reference/configuration.md#bigquery) for more details.
+See the [engine configuration reference](../integrations/engines.md#bigquery---localbuilt-in-scheduler) for more details.
 
 ### Databricks
 
-The Databricks configuration should be configured as follows:
+A Databricks connection should be configured as follows:
 ```yaml linenums="1"
 connections:
     my_databricks_connection:
@@ -57,23 +57,23 @@ connections:
         concurrent_tasks: [optional, should be greater than 0]
 ```
 
-See the [configuration reference](../reference/configuration.md#databricks) for more details.
+See the [engine configuration reference](../integrations/engines.md#databricks---localbuilt-in-scheduler) for more details.
 
 ### DuckDB
 
-The DuckDB configuration should be configured as follows:
+A DuckDB connection should be configured as follows:
 ```yaml linenums="1"
 connections:
     my_duckdb_connection:
         type: duckdb
         database: [optional, path to the database file]
 ```
 
-See the [configuration reference](../reference/configuration.md#duckdb) for more details.
+See the [engine configuration reference](../reference/configuration.md#duckdb) for more details.
 
 ### Redshift
 
-The Redshift configuration should be configured as follows:
+A Redshift connection should be configured as follows:
 ```yaml linenums="1"
 connections:
     my_redshift_connection:
@@ -100,11 +100,11 @@ connections:
         concurrent_tasks: [optional, should be greater than 0]
 ```
 
-See the [configuration reference](../reference/configuration.md#redshift) for more details.
+See the [engine configuration reference](../integrations/engines.md#redshift---localbuilt-in-scheduler) for more details.
 
 ### Snowflake
 
-The Databricks configuration should be configured as follows:
+A Snowflake connection should be configured as follows:
 ```yaml linenums="1"
 connections:
     my_snowflake_connection:
@@ -118,4 +118,4 @@ connections:
         concurrent_tasks: [optional, should be greater than 0]
 ```
 
-See the [configuration reference](../reference/configuration.md#snowflake) for more details.
+See the [engine configuration reference](../integrations/engines.md#snowflake---localbuilt-in-scheduler) for more details.

docs/guides/models.md

@@ -12,7 +12,7 @@ Before adding a model, ensure that you have [already created your project](/guid
 
 To add a model:
 
-1. Within your `models` folder, create a new database file. For example, `example_new_model.sql`.
+1. Within your `models` folder, create a new file. For example, `example_new_model.sql`.
 2. Within the file, define your model as follows:
 
         MODEL (
@@ -33,12 +33,12 @@ To add a model:
 To edit an existing model:
 
 1. Open the model file you wish to edit in your preferred editor and make a change.
-2. To preview an example of what your change looks like without actually creating a table, use the `evaluate` command. Refer to [evaluating a model](#evaluating-a-model).
-3. To materialize this change, use the `plan` command. Refer to [previewing changes using the `plan` command](#previewing-changes-using-the-plan-command).
+2. To preview an example of what your change looks like without actually creating a table, use the `sqlmesh evaluate` command. Refer to [evaluating a model](#evaluating-a-model).
+3. To materialize this change, use the `sqlmesh plan` command. Refer to [previewing changes using the `plan` command](#previewing-changes-using-the-plan-command).
 
 ### Evaluating a model
 
-The `evaluate` command will run a query against your database or engine and return a DataFrame. It is used to test or iterate on models without side effects, and since SQLMesh isn't materializing any data, with minimal cost.
+The `evaluate` command will run a query against your database or engine and return a dataframe. It is used to test or iterate on models without database side effects and at minimal cost because SQLMesh isn't materializing any data.
 
 To evaluate a model:
 
@@ -49,7 +49,7 @@ To evaluate a model:
         id  item_id          ds
         0   7        1  2020-01-07
 
-2. Once the `evaluate` command detects the changes made to your model, observe that the output from your new model is shown.
+2. When you run the `evaluate` command, SQLMesh detects the changes made to the model, executes the model as a query using the options passed to `evaluate`, and shows the output returned by the model query.
 
 ### Previewing changes using the `plan` command
 
@@ -58,123 +58,126 @@ When SQLMesh runs the `plan` command on your environment, it will show you wheth
 To preview changes using `plan`:
 
 1. Enter the `sqlmesh plan <environment name>` command. 
-2. Enter `1` to classify the changes as `Breaking`, or enter `2` to classify the changes as `Non-Breaking`:
-
-        $ sqlmesh plan dev
-        ======================================================================
-        Successfully Ran 1 tests against duckdb
-        ----------------------------------------------------------------------
-        Summary of differences against `dev`:
-        ├── Directly Modified:
-        │   └── sqlmesh_example.example_incremental_model
-        └── Indirectly Modified:
-            └── sqlmesh_example.example_full_model
-        ---      
-        +++   
-        @@ -1,6 +1,7 @@         
-        SELECT                
-        id,              
-        item_id,              
-        +  1 AS new_column,         
-        ds              
-        FROM (VALUES
-        (1, 1, '2020-01-01'),
-        Directly Modified: sqlmesh_example.example_incremental_model
-        └── Indirectly Modified Children:
-            └── sqlmesh_example.example_full_model
-        [1] [Breaking] Backfill sqlmesh_example.example_incremental_model and indirectly modified children
-        [2] [Non-breaking] Backfill sqlmesh_example.example_incremental_model but not indirectly modified children: 2
-        Models needing backfill (missing dates):
-        └── sqlmesh_example.example_incremental_model: (2020-01-01, 2023-02-17)
-        Enter the backfill start date (eg. '1 year', '2020-01-01') or blank for the beginning of history: 
-        Enter the backfill end date (eg. '1 month ago', '2020-01-01') or blank to backfill up until now: 
-        Apply - Backfill Tables [y/n]: y
-
-        All model batches have been executed successfully
-
-        sqlmesh_example.example_incremental_model ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.0% • 1/1 • 0:00:00
+2. Enter `1` to classify the changes as `Breaking`, or enter `2` to classify the changes as `Non-Breaking`. In this example, the changes are classified as `Non-Breaking`:
+
+```hl_lines="23 24"
+$ sqlmesh plan dev
+======================================================================
+Successfully Ran 1 tests against duckdb
+----------------------------------------------------------------------
+Summary of differences against `dev`:
+├── Directly Modified:
+│   └── sqlmesh_example.example_incremental_model
+└── Indirectly Modified:
+    └── sqlmesh_example.example_full_model
+---      
++++   
+@@ -1,6 +1,7 @@         
+SELECT                
+id,              
+item_id,              
++  1 AS new_column,         
+ds              
+FROM (VALUES
+(1, 1, '2020-01-01'),
+Directly Modified: sqlmesh_example.example_incremental_model
+└── Indirectly Modified Children:
+    └── sqlmesh_example.example_full_model
+[1] [Breaking] Backfill sqlmesh_example.example_incremental_model and indirectly modified children
+[2] [Non-breaking] Backfill sqlmesh_example.example_incremental_model but not indirectly modified children: 2
+Models needing backfill (missing dates):
+└── sqlmesh_example.example_incremental_model: (2020-01-01, 2023-02-17)
+Enter the backfill start date (eg. '1 year', '2020-01-01') or blank for the beginning of history: 
+Enter the backfill end date (eg. '1 month ago', '2020-01-01') or blank to backfill up until now: 
+Apply - Backfill Tables [y/n]: y
+
+All model batches have been executed successfully
+
+sqlmesh_example.example_incremental_model ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.0% • 1/1 • 0:00:00
+```
 
 For more information, refer to [plans](../concepts/plans.md).
 
 ## Reverting a change to a model
 
 ---
 
-Before reverting a change, ensure that you have already made a change and that you have run the `sqlmesh plan` command.
+Before trying to revert a change, ensure that you have already made a change and that you have run the `sqlmesh plan` command.
 
 ---
 
 To revert your change:
 
-1. Open the model file you wish to edit in your preferred editor, and undo a change you made earlier. For this example, we'll remove the column we added from the [quickstart](/quick_start).
+1. Open the model file you wish to edit in your preferred editor, and undo a change you made earlier. For this example, we'll remove the column we added in the [quickstart](../quick_start.md) example.
 2. Run `sqlmesh plan` and apply your changes. Enter `y` to run a Virtual Update.
 
-        $ sqlmesh plan dev
-        ======================================================================
-        Successfully Ran 1 tests against duckdb
-        ----------------------------------------------------------------------
-        Summary of differences against `dev`:
-        ├── Directly Modified:
-        │   └── sqlmesh_example.example_incremental_model
-        └── Indirectly Modified:
-            └── sqlmesh_example.example_full_model
-        ---     
-        +++     
-        @@ -1,7 +1,6 @@ 
-
-        SELECT
-        id,          
-        item_id,              
-        -  1 AS new_column,
-        ds
-        FROM (VALUES
-            (1, 1, '2020-01-01'),
-        Directly Modified: sqlmesh_example.example_incremental_model (Non-breaking)
-        └── Indirectly Modified Children:
-            └── sqlmesh_example.example_full_model
-        Apply - Virtual Update [y/n]: y
-
-        Virtual Update executed successfully
-
+```hl_lines="24"
+$ sqlmesh plan dev
+======================================================================
+Successfully Ran 1 tests against duckdb
+----------------------------------------------------------------------
+Summary of differences against `dev`:
+├── Directly Modified:
+│   └── sqlmesh_example.example_incremental_model
+└── Indirectly Modified:
+    └── sqlmesh_example.example_full_model
+---     
++++     
+@@ -1,7 +1,6 @@ 
+
+SELECT
+id,          
+item_id,              
+-  1 AS new_column,
+ds
+FROM (VALUES
+    (1, 1, '2020-01-01'),
+Directly Modified: sqlmesh_example.example_incremental_model (Non-breaking)
+└── Indirectly Modified Children:
+    └── sqlmesh_example.example_full_model
+Apply - Virtual Update [y/n]: y
+
+Virtual Update executed successfully
+```
 ### Virtual Update
 
-Reverting to a previous version is a quick operation as no additional work is being done. For more information, refer to [plan application](../concepts/plans.md#plan-application) and [Virtual Update](../concepts/plans.md#virtual-update).
+Reverting to a previous model version is a quick operation since no additional work is being done. For more information, refer to [plan application](../concepts/plans.md#plan-application) and [Virtual Update](../concepts/plans.md#virtual-update).
 
-**Note:** The janitor runs periodically and automatically to clean up SQLMesh artifacts no longer being used, and determines the time-to-live (TTL) for tables (how much time can pass before reverting is no longer possible).
+**Note:** The SQLMesh janitor runs periodically and automatically to clean up SQLMesh artifacts no longer being used, and determines the time-to-live (TTL) for tables (how much time can pass before reverting is no longer possible).
 
 ## Validating changes to a model
 
 ### Automatic model validation
 
-SQLMesh automatically validates your models in order to ensure the quality and accuracy of your data. This is done by the following:
+SQLMesh automatically validates your models in order to ensure the quality and accuracy of your data. This is done via the following:
 
-* Running unit tests by default when you run the `plan` command. This ensures all changes to applied to any environment is logically validated. Refer to [testing](/concepts/tests) for more information.
-* Running audits whenever data is loaded to a table (both backfill and loading on a cadence). This way you know all data present in any table has passed defined audits. Refer to [auditing](/concepts/audits) for more information.
+* Running unit tests by default when you execute the `plan` command. This ensures all changes to applied to any environment are logically validated. Refer to [testing](../concepts/tests.md) for more information.
+* Running audits whenever data is loaded to a table (either for backfill or loading on a cadence). This way you know all data present in any table has passed all defined audits. Refer to [auditing](../concepts/audits.md) for more information.
 
-CI/CD is another way SQLMesh provides automatic validation, since the preview environment is automatically created.
+SQLMesh also provides automatic validation via CI/CD by automatically creating a preview environment.
 
 ### Manual model validation
 
 To manually validate your models, you can perform one or more of the following tasks:
 
 * [Evaluating a model](#evaluating-a-model)
-* [Testing a model using unit tests](/guides/testing#testing-changes-to-models)
-* [Auditing a model](/guides/testing#auditing-changes-to-models)
+* [Testing a model using unit tests](../guides/testing.md#auditing-changes-to-models)
+* [Auditing a model](../guides/testing.md#auditing-changes-to-models)
 * [Previewing changes using the `plan` command](#previewing-changes-using-the-plan-command)
 
 ## Viewing the DAG
 
 ---
 
-Before generating the DAG, ensure that you have already installed the graphviz package. 
+Before generating a DAG, ensure that you have already installed the graphviz package. 
 
-To install the graphviz package, enter the following command:
+To install the package with `pip`, enter the following command:
 
 ```bash
 pip install graphviz
 ```
 
-Alternatively, enter the following command:
+Alternatively, enter the following command to install graphviz with `apt-get`:
 
 ```bash
 sudo apt-get install graphviz
@@ -198,8 +201,8 @@ Before deleting a model, ensure that you have already run `sqlmesh plan`.
 
 To delete a model:
 
-1. Within your `models` folder, delete the file containing the model you wish to remove. For this example, we'll delete the `example_full_model.sql` file from our [quickstart](/quick_start) project.
-2. Run the `sqlmesh plan <environment>` command, and enter the environment to which you want to apply the change. In the following example, we apply the change to our development environment:
+1. Within your `models` folder, delete the file containing the model you wish to remove. For this example, we'll delete the `example_full_model.sql` file from our [quickstart](../quick_start.md) project.
+2. Run the `sqlmesh plan <environment>` command, specifying the environment to which you want to apply the change. In the following example, we apply the change to our development environment `dev`:
 
         ```
         $ sqlmesh plan dev
@@ -220,7 +223,7 @@ To delete a model:
         sqlmesh_example.example_incremental_model ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.0% • 1/1 • 0:00:00
         ```
 
-    **Note:** If you have other files that reference the model you wish to delete, an error message will note the file(s) containing the reference. You will need to also delete these files in order for the change to be applied.
+    **Note:** If you have other files that reference the model you wish to delete, an error message will note the file(s) containing the reference. You will need to also delete these files in order to apply the change.
 
 3. Plan and apply your changes to production, and enter `y` for the Virtual Update. By default, the `sqlmesh plan` command targets your production environment: