feat(pipelines): add nav links for pipeline docs (#292)

* feat(pipelines): add nav links for pipeline docs

* Add docs about indexing pipeline with initial setup and authorization

* Add docs about the add location stage in index pipeline

* Add remaining stages for indexing pipeline

* Add init content for external results pipeline

* Add more details about the knowledge graph pipeline

* Complete the how to for adding external search results pipeline

* Add a test command for the external results request

* Add the external search link in sidebar

* Update the headings of the external search result doc

* Add how to link for overview

* Add context in the overview page

* Add init content for a new how to guide

* Add fix for side effect link

* Fix link in the index page

* Add top level item in the sidebar

* Add explanation about the stages for the saved search guide

* Add rest of the how to guide for saved search

* Add support for highlighting code properly with prism

* Add support to highlight sh and curl as well

* Add init relevant search stages

* Add some basic details on the relevant search page

* Add support for a copy button

* Add init doc for vector search guide

* Add sidebars for pipelines

* Add init knn response doc

* Add complete pipeline with test example

* Add section explaining script field for knn input

* Work on init content of relevant search pipeline

* Add doc for the first few stages of relevant search query

* Add more stage definitions till the reactive search stage

* Add all stages after the es stage

* Add test example and way to dynamically use query rules

* Add an overview page for concepts

* Add concepts in the index page as a chapter

* Change the title of the concepts page

* Add basic content into overview page

* Add details for the sharing data between stages doc

* Add sidebar for passing data between stages

* Add doc for writing to global context

* Add a doc to explaing the needs propery

* Add a doc for the async field in stage definitions

* Add details about when the async field should be used

* Add doc for inputs

* Add doc for using envs in a stage

* Add doc for error handling concept

* Make some changes to make the sidebar show pipeline docs properly

* Remove list of guides from how-to index

* Add a create pipeline step to the external search example

* Add create step for some more pipeline

* Add create pipeline step to all how to guides

* Fix typo in external search how to

* Fix path to bind the pipeline examples with

* Replace the pipeline create example URL

* Fix the index paths for concepts and how to

* Fix the navbar link for pipelines

* Add content in the how to index page

* Add missing slash at the end of concepts URL

* Add a Read More at the end of paragraphs in how-to index

* Add concept map in concepts image with some explanations

* Add a how-to guide for effective authorization

* Add the authorization guide in how-to index

* Add authorization guide in sidebar

* format: update title for RS pipeline pages

Co-authored-by: Deepjyoti Barman <deep.barman30@gmail.com>

Author: siddharthlatest

content/docs/pipelines/concepts/envs-for-stage.md

@@ -0,0 +1,36 @@
+---
+title: 'Pass envs to stage'
+meta_title: 'Pass envs to stage | Introduction to Appbase.io'
+meta_description: 'Learn how to pass envs on a per stage basis and when to use inputs instead'
+keywords:
+    - concepts
+    - appbase.io
+    - envs
+    - pipelines
+    - reactivesearch
+    - stages
+sidebar: 'docs'
+---
+
+Environments can be passed through the global envs using the `envs` key in the pipeline. `envs` is an object that can contain key value pairs.
+
+## How to pass envs
+
+Envs can be passed using the `envs` key in the following way:
+
+```yml
+envs:
+  - testKey: test value
+```
+
+These envs will be exposed through the `context` to the user. These envs can be accessed in a custom script using `context.envs` field. `envs` will be an object that will contain the values passed when the pipeline was created.
+
+## Are inputs alternative to envs?
+
+Inputs are not an alternative to environments. Instead inputs can be thought of as parameters to a function whereas envs are just _environments_ which are set once and accessed multiple times.
+
+Inputs should be used when the data is directly accessed by the stage. Inputs are used in the case of pre-built stages that expect certain values from the user.
+
+On the other hand, custom stages that run JS scripts might need certain values to be passed from the user. In this case, `envs` can be used.
+
+Another important thing to note is, `envs` should be used when there is a value that might be sensitive to just pass or keep in code.

content/docs/pipelines/concepts/error-handling.md

@@ -0,0 +1,81 @@
+---
+title: 'Error handling within Pipelines'
+meta_title: 'Error handling within pipelines | Introduction to Appbase.io'
+meta_description: 'Learn how to handle errors in the pipeline or throw errors in custom stages'
+keywords:
+    - concepts
+    - appbase.io
+    - errors
+    - pipelines
+    - reactivesearch
+    - stages
+sidebar: 'docs'
+---
+
+Errors are always prone to happen. Be it a properly defined bit of code or a handcrafted book. Here, we will learn how to handle errors in an user defined pipeline when it occurs at a particular stage or even on a pre-built stage.
+
+First of all, let's understand the field `continueOnError` that is exposed for every stage by ReactiveSearch pipelines.
+
+## What is continueOnError
+
+`continueOnError` is a field accepted on a per stage basis. This field tells the pipeline whether or not to **continue** when there is an error at the stage.
+
+By default, it is set to `true` which means even if there is an error at a certain stage, it will be ignored and the execution will move on to the next stage.
+
+While defining the pipeline, this field can be set to `false` to make sure that the execution does not continue when there is an error and it is shown back to the user.
+
+### continueOnError example
+
+Let's take the example of the pre-built stage `reactivesearchQuery`. Let's say we are passing a body to reactive search query that is not valid. This is in the sense that the body does not contain some required fields.
+
+By default, this error will be ignored and the execution will go to the next stage (which in most cases is `elasticsearchQuery`). However, the thing to note here is that the `elasticsearchQuery` stage will fail.
+
+To get around this issue, we can use the `coninueOnError` flag in the following way:
+
+```yml
+- id: reactive search
+  use: reactivesearchQuery
+  # Make sure the execution stops if error occurs
+  # at this stage.
+  continueOnError: false
+- id: elastic search
+  use: elasticsearchQuery
+```
+
+## Custom Stage Errors
+
+Let's say there is a custom stage that is running a JavaScript script and we want to make sure that if the script fails then the execution should not continue. As explained above, by default the execution will **not stop** even if there is an error at any stage.
+
+Let's now say, we have the following script:
+
+```js
+function handleRequest() {
+    const a = 5;
+    // Zero division error
+    return a / a - 5;
+}
+```
+
+Above script will fail because it will raise a zero division error but we will have to explicitly tell the pipeline to stop execution in the following way:
+
+```yml
+- id: custom script
+  scriptRef: custom.js
+  continueOnError: false
+```
+
+When we define the stage in the above way, it will stop execution if there is an error in the script.
+
+### Throwing errors from scripts
+
+Now, let's say there is a script that utilizes an environment variable. This script will not work if the env variable is not present. So we can add a check in the script to determine if the variable is present or not and accordingly throw an error. This can be achieved in the following way:
+
+```js
+function handleRequest() {
+    if (context.envs.requiredKey == undefined) {
+        throw Error('requiredKey is a required environment key!');
+    } 
+}
+```
+
+With the above script, if the key is not passed and the `continueOnError` is set to `false`, the stage will fail with the error as thrown from the script.

content/docs/pipelines/concepts/index.md

@@ -0,0 +1,36 @@
+---
+title: 'ReactiveSearch Pipelines: Concepts'
+meta_title: 'Concepts for ReactiveSearch Pipelines'
+meta_description: 'Learn about the concepts for ReactiveSearch pipelines and understand how to use them'
+keywords:
+    - concepts
+    - appbase.io
+    - elasticsearch
+    - pipelines
+    - reactivesearch
+sidebar: 'docs'
+---
+
+Pipeline Concepts consist of basic concepts that would be useful to understand how pipelines work and how pipelines can be utilized to get the best out of it.
+
+Following is a visualization of how pipelines are executed and how they go from one stage to another.
+
+![Pipeline Concept](/images/concepts/pipeline_concept.png "Pipeline Execution Visualized")
+
+In the above image, the stages can access the global context and modify it. However, the stages with [async] can `get/add` the context which means they cannot modify the already existing values in the context.
+
+Moreover, the context contains `response`, `request` and `envs` during initialization and other fields can be added to them in other stages.
+
+Notice that the `QueryTranslate` step has a check to see if it should continue when an error occurs or not. This is default behaviour and is executed in all stages but is only shown in the case of `queryTranslate` to demonstrate how it works.
+
+This behaviour can be tweaked with the `continueOnError` field.
+
+Following pages explain the concepts in detail with examples:
+
+- [Pass data between Stages](pass-data-between-stages)
+- [Pass inputs to Stage](pass-inputs-to-stage)
+- [Run Stage Asychronously](run-stage-async)
+- [Wait for Other Stages](wait-for-other-stage)
+- [Write to Global Context](write-to-global-context)
+- [Pass Environments to Stage](envs-for-stage)
+- [Error Handling in Pipelines](error-handling)

content/docs/pipelines/concepts/pass-data-between-stages.md

@@ -0,0 +1,126 @@
+---
+title: 'Pass data between stages'
+meta_title: 'Pass data between stages | Introduction to Appbase.io'
+meta_description: 'Learn how to pass data between pipeline stages'
+keywords:
+    - concepts
+    - appbase.io
+    - elasticsearch
+    - pipelines
+    - reactivesearch
+    - stages
+sidebar: 'docs'
+---
+
+Passing data between stages can be considered one of the most essential parts of pipelines. Since we provide quite a lot of pre-built stages, it is essential that the data between them is shared in some way and is passable from one stage to another.
+
+This can be achieved by writing the data to the `context` and accessing them through the `context`.
+
+## What is Context
+
+Every stage in the pipeline has access to a `context` variable. `context` is a JSON object that contains details about the pipeline being executed. Some of the keys that are populated in the context are:
+
+- `request`
+- `response`
+- `envs`
+
+Here's an example context:
+
+```json
+{
+    "request": {
+        "body": "{\"query\": [{\"id\": \"some ID\", \"value\": \"search term\", \"dataField\": [\"text\"]}]}",
+        "headers": {
+            "Content-Type": "application/json"
+        }
+    },
+    "envs": {
+        "someEnvKey": "test"
+    },
+    "response": {
+        "body": ""
+    }
+}
+```
+
+Note that `envs` key will contain an object where key and values are populated from the pipeline config or per stage `envs`.
+
+> `request.body` and `response.body` are stringified JSON. This is because some ElasticSearch endpoints support non JSON data like nd-json.
+
+## How to write/access context data
+
+### Writing to context
+
+Anything can be written to the context. Since it is just a JSON object, it can be easily modified from a custom JavaScript stage.
+
+Let's say we want to add a field `customData` to the context with the value `{'test': 'nothing here'}`. We can do that with the following JavaScript script:
+
+```js
+function handleRequest() {
+    return {
+        customData: {
+            test: "nothing here"
+        }
+    }
+}
+```
+
+Once we have the script defined, we can just define the stage like this:
+
+```yml
+- id: add custom data
+  scriptRef: addCustom.js
+```
+
+> Assuming the above JS script is named as `addCustom.js`.
+
+### Accessing from context
+
+Field can be easily accessed from the context. Let's say we have a script that needs to access the `customData` field from above, that can be done in the following way:
+
+```js
+customData = context.customData;
+```
+
+> Every script that is running is populated with a `context` variable that contains the latest updated context in that stage of the pipeline.
+
+### Things to note
+
+All stages can modify context but stages that run asychronously come with a downside. Stages can be run asynchronously by setting the `async: true` flag while defining the pipeline.
+
+However, stages that run with the above flag are not allowed to update **already existing** fields in the context. This means that if a stage has a script that runs asynchronously, it cannot modify the `request.body` value or any value that exists in the context from before.
+
+It can, however, **add new fields** to the context that can be accessed in further stages.
+
+## Example Scenario: Dynamic input
+
+Let's take the example of the `replaceWords`. We have to pass the input to `replaceWords` through the `inputs.data` field.
+
+Following is how we replace the term `test` with `not test` in the search term:
+
+```yml
+- id: replace search term
+  use: replaceWords
+  inputs:
+    data:
+      test: not test
+```
+
+This is pretty straight forward, however, if we want to make the input dynamic, we can do that through context. Let's say before the above stage runs we have another stage that adds a `replaceWordDetails` field to the context:
+
+```yml
+- id: populate details
+  script: "function handleRequest() { return { replaceWordDetails: {'test': 'not test'} } }"
+```
+
+We can then directly pass this context field to the `replaceWords` stage as an input in the following way:
+
+```yml
+- id: replace search term
+  needs:
+    - populate details
+  inputs:
+    data: "{{replaceWordDetails}}"
+```
+
+> Note that we are using the `needs` field to make sure that the populate details stage runs before the replace search term stage in order to have the `replaceWordDetails` field populated.

content/docs/pipelines/concepts/pass-inputs-to-stage.md

@@ -0,0 +1,60 @@
+---
+title: 'Pass inputs to a stage'
+meta_title: 'Pass inputs to a stage | Introduction to Appbase.io'
+meta_description: 'Learn how to pass inputs to a stage and when not to do that'
+keywords:
+    - concepts
+    - appbase.io
+    - async
+    - pipelines
+    - reactivesearch
+    - stages
+sidebar: 'docs'
+---
+
+ReactiveSearch provides quite a lot of pre-built stages in order to make pipelines useful. Most of these stages accept some inputs from the user.
+
+## What are inputs
+
+Inputs are like parameters in a function. If we have a function that is doing something there are a few things that we absolutely need from the user who's calling this function. This is where inputs come in.
+
+### How to pass
+
+Inputs can be passed with the `inputs` field in the stage definition. `inputs` field will be accessed only if the pre-built stage uses the data else it will be ignored without any error.
+
+## Example: replace words
+
+Let's take the example of the pre-built stage `replaceWords` that provides an way to replace words in the search term. Let's say we want to replace the word `test` in the search term with `no test`.
+
+We can do that by passing this stage an input. This can be done in the following way:
+
+```yml
+- id: replace words
+  use: replaceWords
+  inputs:
+    data:
+      - test: no test
+```
+
+This particular stage expects the `inputs.data` field to contain the data for this stage to work properly. This field differs based on the pre-built stage being used.
+
+For example, for the stage `searchRelevancy` the `inputs.search` field can be passed or `inputs.suggestion` field can be passed.
+
+## Dynamic inputs
+
+ReactiveSearch Pipelines also support dynamic inputs to stages. Let's say the input for a stage is generated in a stage prior to this stage. We can then pass the input dynamically by using the `context.
+
+What we have to do is write a new field in the context and access that field in the `inputs` field.
+
+Let's say in the above example of `replaceWords`, we want to pass a dynamic value for the `inputs.data` field.
+
+Say, we populate a field called `wordDetails` in the context. We can then access this field in the following way:
+
+```yml
+- id: replace words
+  use: replaceWords
+  inputs:
+    data: "{{wordDetails}}"
+```
+
+That's it, the pipeline will take care of substituting the dynaimc input by accessing it from the context.