Skip to content

Commit 9f9bb92

Browse files
authored
Merge pull request #99 from data-catering/feature/clean-docs
Update links to use current repo instead of data-caterer-example repo
2 parents 6d2223b + 101428c commit 9f9bb92

35 files changed

Lines changed: 263 additions & 181 deletions

docs/docs/advanced.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,9 +65,9 @@ You can alter the `status` field in the account data to only generate `open` acc
6565
and define a foreign key between Postgres and parquet to ensure the same `account_id` is being used.
6666
Then in the parquet task, define 1 to 10 transactions per `account_id` to be generated.
6767

68-
[Postgres account generation example task](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/jdbc/postgres/postgres-account-task.yaml)
69-
[Parquet transaction generation example task](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/file/parquet/parquet-transaction-task.yaml)
70-
[Plan](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/plan/scenario-based.yaml)
68+
[Postgres account generation example task](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/jdbc/postgres/postgres-account-task.yaml)
69+
[Parquet transaction generation example task](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/file/parquet/parquet-transaction-task.yaml)
70+
[Plan](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/plan/scenario-based.yaml)
7171

7272
## Cloud storage
7373

docs/docs/configuration.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ metadata gets saved.
1111

1212
These configurations are defined from within your Java or Scala class via `configuration` or for YAML file setup,
1313
`application.conf` file as seen
14-
[**here**](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/application.conf).
14+
[**here**](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/application.conf).
1515

1616
## Flags
1717

@@ -109,7 +109,7 @@ Enable fast generation to maximize throughput. This automatically disables slowe
109109
sink metadata, unique checks, save reports, validations, alerts) and applies runtime optimizations (e.g. lower shuffle
110110
partitions, enable AQE, Kryo serializer) and increases `numRecordsPerBatch`.
111111

112-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/FastGenerationAndReferenceJavaPlanRun.java)
112+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/FastGenerationAndReferenceJavaPlanRun.java)
113113

114114
=== "Java"
115115

docs/docs/connection.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -534,7 +534,7 @@ found [**here**](https://spark.apache.org/docs/latest/structured-streaming-kafka
534534

535535
When defining your schema for pushing data to Kafka, it follows a specific top level schema.
536536
An example can be
537-
found [**here**](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/kafka/kafka-account-task.yaml)
537+
found [**here**](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/kafka/kafka-account-task.yaml)
538538
. You can define the key, value, headers, partition or topic by following the linked schema.
539539

540540
### JMS

docs/docs/deployment.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Run the OS native application from [downloading the specific OS application here
1919
## Docker
2020

2121
To package up your class along with the Data Caterer base image, you can follow
22-
the [Dockerfile that is created for you here](https://github.com/data-catering/data-caterer-example/blob/main/Dockerfile).
22+
the [Dockerfile that is created for you here](https://github.com/data-catering/data-caterer/blob/main/example/Dockerfile).
2323

2424
Then you can run the following:
2525

@@ -30,13 +30,14 @@ docker build -t <my_image_name>:<my_image_tag> .
3030

3131
## Helm
3232

33-
[Link to sample helm on GitHub here](https://github.com/data-catering/data-caterer-example/tree/main/helm/data-caterer)
33+
[Link to sample helm on GitHub here](https://github.com/data-catering/data-caterer/tree/main/example/helm/data-caterer)
3434

3535
Update
36-
the [configuration](https://github.com/data-catering/data-caterer-example/blob/main/helm/data-caterer/templates/configuration.yaml)
36+
the [configuration](https://github.com/data-catering/data-caterer/blob/main/example/helm/data-caterer/templates/configuration.yaml)
3737
to your own data connections and configuration or own image created from above.
3838

3939
```shell
40-
git clone git@github.com:data-catering/data-caterer-example.git
41-
helm install data-caterer ./data-caterer-example/helm/data-caterer
40+
git clone git@github.com:data-catering/data-caterer.git
41+
cd data-caterer/example
42+
helm install data-caterer ./data-caterer/example/helm/data-caterer
4243
```

docs/docs/generator/data-generator.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ Förlåt", "你好吗", "Nhà vệ sinh ở đâu", "こんにちは", "नम
6060

6161
#### Sample
6262

63-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/DocumentationPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/DocumentationJavaPlanRun.java)
63+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/DocumentationPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/DocumentationJavaPlanRun.java)
6464

6565
=== "Java"
6666

@@ -643,7 +643,7 @@ You can control behavior at the step level for generation.
643643

644644
Disable data generation for specific steps when you want to use them only for reference in foreign key relationships.
645645

646-
[:material-run-fast: Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
646+
[:material-run-fast: Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
647647

648648
=== "Java"
649649

@@ -688,7 +688,7 @@ Disable data generation for specific steps when you want to use them only for re
688688

689689
Control data partitioning and parallelism for improved performance with large datasets.
690690

691-
[:material-run-fast: Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
691+
[:material-run-fast: Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
692692

693693
=== "Java"
694694

@@ -727,7 +727,7 @@ Control data partitioning and parallelism for improved performance with large da
727727

728728
Use existing data as reference instead of generating new data for a step. This is useful when you want to reference real data in foreign key relationships.
729729

730-
[:material-run-fast: Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
730+
[:material-run-fast: Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
731731

732732
=== "Java"
733733

@@ -760,7 +760,7 @@ Use existing data as reference instead of generating new data for a step. This i
760760

761761
For databases that support primary key positions and clustering order (like Cassandra), you can specify the order of primary keys and clustering keys.
762762

763-
[:material-run-fast: Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
763+
[:material-run-fast: Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/FastGenerationAndReferencePlanRun.scala)
764764

765765
=== "Java"
766766

@@ -809,7 +809,7 @@ For databases that support primary key positions and clustering order (like Cass
809809

810810
Data Caterer supports complex SQL expressions for generating sophisticated data relationships and calculations. SQL expressions are evaluated after all non-SQL fields have been generated.
811811

812-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/DocumentationPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/DocumentationJavaPlanRun.java)
812+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/DocumentationPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/DocumentationJavaPlanRun.java)
813813

814814
### Array Operations and Aggregations
815815

docs/docs/guide/data-source/database/bigquery.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,24 +21,27 @@ for the tables you configure.
2121

2222
## Get Started
2323

24-
First, we will clone the data-caterer-example repo which will already have the base project setup required.
24+
First, we will clone the data-caterer repo which will already have the base project setup required.
2525

2626
=== "Java"
2727

2828
```shell
29-
git clone git@github.com:data-catering/data-caterer-example.git
29+
git clone git@github.com:data-catering/data-caterer.git
30+
cd data-caterer/example
3031
```
3132

3233
=== "Scala"
3334

3435
```shell
35-
git clone git@github.com:data-catering/data-caterer-example.git
36+
git clone git@github.com:data-catering/data-caterer.git
37+
cd data-caterer/example
3638
```
3739

3840
=== "YAML"
3941

4042
```shell
41-
git clone git@github.com:data-catering/data-caterer-example.git
43+
git clone git@github.com:data-catering/data-caterer.git
44+
cd data-caterer/example
4245
```
4346

4447
=== "UI"
@@ -348,7 +351,7 @@ To tell Data Caterer that we want to run with the configurations along with the
348351

349352
### Run
350353

351-
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer-example` to run the
354+
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer/example` folder to run the
352355
class we just
353356
created.
354357

docs/docs/guide/data-source/database/cassandra.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
99
Creating a data generator for Cassandra. You will build a Docker image that will be able to populate data in Cassandra
1010
for the tables you configure.
1111

12-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/CassandraPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/CassandraJavaPlanRun.java) | [:material-file-yaml-outline: Basic YAML](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/cassandra) | [:material-file-yaml-outline: Advanced YAML](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/cassandra/advanced-cassandra-task.yaml)
12+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/CassandraPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/CassandraJavaPlanRun.java) | [:material-file-yaml-outline: Basic YAML](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/cassandra) | [:material-file-yaml-outline: Advanced YAML](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/cassandra/advanced-cassandra-task.yaml)
1313

1414
## Requirements
1515

@@ -21,24 +21,27 @@ for the tables you configure.
2121

2222
## Get Started
2323

24-
First, we will clone the data-caterer-example repo which will already have the base project setup required.
24+
First, we will clone the data-caterer repo which will already have the base project setup required.
2525

2626
=== "Java"
2727

2828
```shell
29-
git clone git@github.com:data-catering/data-caterer-example.git
29+
git clone git@github.com:data-catering/data-caterer.git
30+
cd data-caterer/example
3031
```
3132

3233
=== "Scala"
3334

3435
```shell
35-
git clone git@github.com:data-catering/data-caterer-example.git
36+
git clone git@github.com:data-catering/data-caterer.git
37+
cd data-caterer/example
3638
```
3739

3840
=== "YAML"
3941

4042
```shell
41-
git clone git@github.com:data-catering/data-caterer-example.git
43+
git clone git@github.com:data-catering/data-caterer.git
44+
cd data-caterer/example
4245
```
4346

4447
=== "UI"
@@ -405,7 +408,7 @@ To tell Data Caterer that we want to run with the configurations along with the
405408

406409
### Run
407410

408-
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer-example` to run the class we just
411+
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer/example` folder to run the class we just
409412
created.
410413

411414
=== "Java"

docs/docs/guide/data-source/database/mysql.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
99
Creating a data generator for MySQL. You will build a Docker image that will be able to populate data in MySQL
1010
for the tables you configure.
1111

12-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/MySqlPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/MySqlJavaPlanRun.java) | [:material-file-yaml-outline: YAML Example](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/jdbc/mysql)
12+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/MySqlPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/MySqlJavaPlanRun.java) | [:material-file-yaml-outline: YAML Example](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/jdbc/mysql)
1313

1414
## Requirements
1515

@@ -20,24 +20,27 @@ for the tables you configure.
2020

2121
## Get Started
2222

23-
First, we will clone the data-caterer-example repo which will already have the base project setup required.
23+
First, we will clone the data-caterer repo which will already have the base project setup required.
2424

2525
=== "Java"
2626

2727
```shell
28-
git clone git@github.com:data-catering/data-caterer-example.git
28+
git clone git@github.com:data-catering/data-caterer.git
29+
cd data-caterer/example
2930
```
3031

3132
=== "Scala"
3233

3334
```shell
34-
git clone git@github.com:data-catering/data-caterer-example.git
35+
git clone git@github.com:data-catering/data-caterer.git
36+
cd data-caterer/example
3537
```
3638

3739
=== "YAML"
3840

3941
```shell
40-
git clone git@github.com:data-catering/data-caterer-example.git
42+
git clone git@github.com:data-catering/data-caterer.git
43+
cd data-caterer/example
4144
```
4245

4346
=== "UI"
@@ -374,7 +377,7 @@ To tell Data Caterer that we want to run with the configurations along with the
374377

375378
### Run
376379

377-
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer-example` to run the
380+
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer/example` folder to run the
378381
class we just
379382
created.
380383

docs/docs/guide/data-source/database/postgres.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
99
Creating a data generator for Postgres. You will build a Docker image that will be able to populate data in Postgres
1010
for the tables you configure.
1111

12-
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/scala/io/github/datacatering/plan/PostgresPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer-example/blob/main/src/main/java/io/github/datacatering/plan/PostgresJavaPlanRun.java) | [:material-file-yaml-outline: YAML Example](https://github.com/data-catering/data-caterer-example/blob/main/docker/data/custom/task/jdbc/postgres)
12+
[:material-run-fast: Scala Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/scala/io/github/datacatering/plan/PostgresPlanRun.scala) | [:material-coffee: Java Example](https://github.com/data-catering/data-caterer/blob/main/example/src/main/java/io/github/datacatering/plan/PostgresJavaPlanRun.java) | [:material-file-yaml-outline: YAML Example](https://github.com/data-catering/data-caterer/blob/main/example/docker/data/custom/task/jdbc/postgres)
1313

1414
## Requirements
1515

@@ -20,24 +20,27 @@ for the tables you configure.
2020

2121
## Get Started
2222

23-
First, we will clone the data-caterer-example repo which will already have the base project setup required.
23+
First, we will clone the data-caterer repo which will already have the base project setup required.
2424

2525
=== "Java"
2626

2727
```shell
28-
git clone git@github.com:data-catering/data-caterer-example.git
28+
git clone git@github.com:data-catering/data-caterer.git
29+
cd data-caterer/example
2930
```
3031

3132
=== "Scala"
3233

3334
```shell
34-
git clone git@github.com:data-catering/data-caterer-example.git
35+
git clone git@github.com:data-catering/data-caterer.git
36+
cd data-caterer/example
3537
```
3638

3739
=== "YAML"
3840

3941
```shell
40-
git clone git@github.com:data-catering/data-caterer-example.git
42+
git clone git@github.com:data-catering/data-caterer.git
43+
cd data-caterer/example
4144
```
4245

4346
=== "UI"
@@ -374,7 +377,7 @@ To tell Data Caterer that we want to run with the configurations along with the
374377

375378
### Run
376379

377-
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer-example` to run the
380+
Now we can run via the script `./run.sh` that is in the top level directory of the `data-caterer/example` folder to run the
378381
class we just
379382
created.
380383

0 commit comments

Comments
 (0)