Skip to content

Commit 599a20b

Browse files
ramya18101duedares-rvj
ramya18101
and
duedares-rvj
authored
Add new PIIConfig and filters flag on logStreams (#1280)
* Add new PIIConfig flag on logStreams * Fix testcases * Fix Docs * Update docs and help examples * Add new filters flag on logStreams and update docs * Update display logic and docs for logs streams * Updated to check pointpagination * Updated lint * Update checkpoint pagination logic * Fix unit tests * Fix checkpoint pagination logic * Fix lint * Fix panics in TestFetchTemplateData --------- Co-authored-by: duedares-rvj <duedares@gmail.com> Co-authored-by: Rajat Bajaj <rajat.bajaj@okta.com>
1 parent 28a7959 commit 599a20b

29 files changed

Lines changed: 1030 additions & 414 deletions

‎docs/auth0_logs_streams_create_datadog.md‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ auth0 logs streams create datadog [flags]
2222
auth0 logs streams create datadog
2323
auth0 logs streams create datadog --name <name>
2424
auth0 logs streams create datadog --name <name> --region <region>
25-
auth0 logs streams create datadog --name <name> --region <region> --api-key <api-key>
25+
auth0 logs streams create datadog --name <name> --region <region> --api-key <api-key> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
26+
auth0 logs streams create datadog --name <name> --region <region> --api-key <api-key> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "hash", "algorithm": "xxhash"}'
2627
auth0 logs streams create datadog -n <name> -r <region> -k <api-key>
2728
auth0 logs streams create datadog -n mylogstream -r eu -k 121233123455 --json
2829
auth0 logs streams create datadog -n mylogstream -r eu -k 121233123455 --json-compact
@@ -32,12 +33,18 @@ auth0 logs streams create datadog [flags]
3233
## Flags
3334

3435
```
35-
-k, --api-key string Datadog API Key. To obtain a key, see the Datadog Authentication documentation (https://docs.datadoghq.com/api/latest/authentication).
36-
--json Output in json format.
37-
--json-compact Output in compact json format.
38-
-n, --name string The name of the log stream.
39-
-r, --region string The region in which the datadog dashboard is created.
40-
If you are in the datadog EU site ('app.datadoghq.eu'), the Region should be EU otherwise it should be US.
36+
-k, --api-key string Datadog API Key. To obtain a key, see the Datadog Authentication documentation (https://docs.datadoghq.com/api/latest/authentication).
37+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
38+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
39+
--json Output in json format.
40+
--json-compact Output in compact json format.
41+
-n, --name string The name of the log stream.
42+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
43+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
44+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
45+
(default "{}")
46+
-r, --region string The region in which the datadog dashboard is created.
47+
If you are in the datadog EU site ('app.datadoghq.eu'), the Region should be EU otherwise it should be US.
4148
```
4249

4350

‎docs/auth0_logs_streams_create_eventbridge.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,9 @@ auth0 logs streams create eventbridge [flags]
2222
auth0 logs streams create eventbridge
2323
auth0 logs streams create eventbridge --name <name>
2424
auth0 logs streams create eventbridge --name <name> --aws-id <aws-id>
25+
auth0 logs streams create eventbridge --name <name> --aws-id <aws-id> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
2526
auth0 logs streams create eventbridge --name <name> --aws-id <aws-id> --aws-region <aws-region>
27+
auth0 logs streams create eventbridge --name <name> --aws-id <aws-id> --aws-region <aws-region> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}'
2628
auth0 logs streams create eventbridge -n <name> -i <aws-id> -r <aws-region>
2729
auth0 logs streams create eventbridge -n mylogstream -i 999999999999 -r "eu-west-1" --json
2830
auth0 logs streams create eventbridge -n mylogstream -i 999999999999 -r "eu-west-1" --json-compact
@@ -34,9 +36,15 @@ auth0 logs streams create eventbridge [flags]
3436
```
3537
-i, --aws-id string ID of the AWS account.
3638
-r, --aws-region string The AWS region in which eventbridge will be created, e.g. 'us-east-2'.
39+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
40+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
3741
--json Output in json format.
3842
--json-compact Output in compact json format.
3943
-n, --name string The name of the log stream.
44+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
45+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
46+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
47+
(default "{}")
4048
```
4149

4250

‎docs/auth0_logs_streams_create_eventgrid.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ auth0 logs streams create eventgrid [flags]
2424
auth0 logs streams create eventgrid --name <name> --azure-id <azure-id>
2525
auth0 logs streams create eventgrid --name <name> --azure-id <azure-id> --azure-region <azure-region>
2626
auth0 logs streams create eventgrid --name <name> --azure-id <azure-id> --azure-region <azure-region> --azure-group <azure-group>
27+
auth0 logs streams create eventgrid --name <name> --azure-id <azure-id> --azure-region <azure-region> --azure-group <azure-group> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
28+
auth0 logs streams create eventgrid --name <name> --azure-id <azure-id> --azure-region <azure-region> --azure-group <azure-group> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "hash", "algorithm": "xxhash"}'
2729
auth0 logs streams create eventgrid -n <name> -i <azure-id> -r <azure-region> -g <azure-group>
2830
auth0 logs streams create eventgrid -n mylogstream -i "b69a6835-57c7-4d53-b0d5-1c6ae580b6d5" -r northeurope -g "azure-logs-rg" --json
2931
auth0 logs streams create eventgrid -n mylogstream -i "b69a6835-57c7-4d53-b0d5-1c6ae580b6d5" -r northeurope -g "azure-logs-rg" --json-compact
@@ -36,9 +38,15 @@ auth0 logs streams create eventgrid [flags]
3638
-g, --azure-group string The name of the Azure resource group.
3739
-i, --azure-id string Id of the Azure subscription.
3840
-r, --azure-region string The region in which the Azure subscription is hosted.
41+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
42+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
3943
--json Output in json format.
4044
--json-compact Output in compact json format.
4145
-n, --name string The name of the log stream.
46+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
47+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
48+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
49+
(default "{}")
4250
```
4351

4452

‎docs/auth0_logs_streams_create_http.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ auth0 logs streams create http [flags]
2424
auth0 logs streams create http --name <name> --endpoint <endpoint>
2525
auth0 logs streams create http --name <name> --endpoint <endpoint> --type <type>
2626
auth0 logs streams create http --name <name> --endpoint <endpoint> --type <type> --format <format>
27+
auth0 logs streams create http --name <name> --endpoint <endpoint> --type <type> --format <format> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
28+
auth0 logs streams create http --name <name> --endpoint <endpoint> --type <type> --format <format> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "hash", "algorithm": "xxhash"}''
2729
auth0 logs streams create http --name <name> --endpoint <endpoint> --type <type> --format <format> --authorization <authorization>
2830
auth0 logs streams create http -n <name> -e <endpoint> -t <type> -f <format> -a <authorization>
2931
auth0 logs streams create http -n mylogstream -e "https://example.com/webhook/logs" -t "application/json" -f "JSONLINES" -a "AKIAXXXXXXXXXXXXXXXX" --json
@@ -36,10 +38,16 @@ auth0 logs streams create http [flags]
3638
```
3739
-a, --authorization string Sent in the HTTP "Authorization" header with each request.
3840
-e, --endpoint string The HTTP endpoint to send streaming logs to.
41+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
42+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
3943
-f, --format string The format of data sent over HTTP. Options are "JSONLINES", "JSONARRAY" or "JSONOBJECT"
4044
--json Output in json format.
4145
--json-compact Output in compact json format.
4246
-n, --name string The name of the log stream.
47+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
48+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
49+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
50+
(default "{}")
4351
-t, --type string The "Content-Type" header to send over HTTP. Common value is "application/json".
4452
```
4553

‎docs/auth0_logs_streams_create_splunk.md‎

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ auth0 logs streams create splunk [flags]
2424
auth0 log streams create splunk --name <name> --domain <domain>
2525
auth0 log streams create splunk --name <name> --domain <domain> --token <token>
2626
auth0 log streams create splunk --name <name> --domain <domain> --token <token> --port <port>
27+
auth0 log streams create splunk --name <name> --domain <domain> --token <token> --port <port> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
28+
auth0 log streams create splunk --name <name> --domain <domain> --token <token> --port <port> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "hash", "algorithm": "xxhash"}'
2729
auth0 log streams create splunk --name <name> --domain <domain> --token <token> --port <port> --secure=false
2830
auth0 log streams create splunk -n <name> -d <domain> -t <token> -p <port> -s
2931
auth0 log streams create splunk -n mylogstream -d "demo.splunk.com" -t "12a34ab5-c6d7-8901-23ef-456b7c89d0c1" -p "8088" -s false --json
@@ -34,13 +36,19 @@ auth0 logs streams create splunk [flags]
3436
## Flags
3537

3638
```
37-
-d, --domain string The domain name of the splunk instance.
38-
--json Output in json format.
39-
--json-compact Output in compact json format.
40-
-n, --name string The name of the log stream.
41-
-p, --port string The port of the HTTP event collector.
42-
-s, --secure This should be set to 'false' when using self-signed certificates.
43-
-t, --token string Splunk event collector token.
39+
-d, --domain string The domain name of the splunk instance.
40+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
41+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
42+
--json Output in json format.
43+
--json-compact Output in compact json format.
44+
-n, --name string The name of the log stream.
45+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
46+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
47+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
48+
(default "{}")
49+
-p, --port string The port of the HTTP event collector.
50+
-s, --secure This should be set to 'false' when using self-signed certificates.
51+
-t, --token string Splunk event collector token.
4452
```
4553

4654

‎docs/auth0_logs_streams_create_sumo.md‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ auth0 logs streams create sumo [flags]
2222
auth0 logs streams create sumo
2323
auth0 logs streams create sumo --name <name>
2424
auth0 logs streams create sumo --name <name> --source <source>
25+
auth0 logs streams create sumo --name <name> --source <source> --filters '[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]'
26+
auth0 logs streams create sumo --name <name> --source <source> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "hash", "algorithm": "xxhash"}'
2527
auth0 logs streams create sumo -n <name> -s <source>
2628
auth0 logs streams create sumo -n "mylogstream" -s "demo.sumo.com" --json
2729
auth0 logs streams create sumo -n "mylogstream" -s "demo.sumo.com" --json-compact
@@ -31,10 +33,16 @@ auth0 logs streams create sumo [flags]
3133
## Flags
3234

3335
```
34-
--json Output in json format.
35-
--json-compact Output in compact json format.
36-
-n, --name string The name of the log stream.
37-
-s, --source string Generated URL for your defined HTTP source in Sumo Logic.
36+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
37+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
38+
--json Output in json format.
39+
--json-compact Output in compact json format.
40+
-n, --name string The name of the log stream.
41+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
42+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
43+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
44+
(default "{}")
45+
-s, --source string Generated URL for your defined HTTP source in Sumo Logic.
3846
```
3947

4048

‎docs/auth0_logs_streams_update_datadog.md‎

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ auth0 logs streams update datadog [flags]
2222
auth0 logs streams update datadog
2323
auth0 logs streams update datadog <log-stream-id> --name <name>
2424
auth0 logs streams update datadog <log-stream-id> --name <name> --region <region>
25-
auth0 logs streams update datadog <log-stream-id> --name <name> --region <region> --api-key <api-key>
26-
auth0 logs streams update datadog <log-stream-id> -n <name> -r <region> -k <api-key>
25+
auth0 logs streams update datadog <log-stream-id> --name <name> --region <region> --api-key <api-key> --filters '[{"type":"category","name":"user.fail"},{"type":"category","name":"scim.event"}]'
26+
auth0 logs streams update datadog <log-stream-id> --name <name> --region <region> --api-key <api-key> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}'
27+
auth0 logs streams update datadog <log-stream-id> -n <name> -r <region> -k <api-key> -c null
2728
auth0 logs streams update datadog <log-stream-id> -n mylogstream -r eu -k 121233123455 --json
2829
auth0 logs streams update datadog <log-stream-id> -n mylogstream -r eu -k 121233123455 --json-compact
2930
```
@@ -32,12 +33,18 @@ auth0 logs streams update datadog [flags]
3233
## Flags
3334

3435
```
35-
-k, --api-key string Datadog API Key. To obtain a key, see the Datadog Authentication documentation (https://docs.datadoghq.com/api/latest/authentication).
36-
--json Output in json format.
37-
--json-compact Output in compact json format.
38-
-n, --name string The name of the log stream.
39-
-r, --region string The region in which the datadog dashboard is created.
40-
If you are in the datadog EU site ('app.datadoghq.eu'), the Region should be EU otherwise it should be US.
36+
-k, --api-key string Datadog API Key. To obtain a key, see the Datadog Authentication documentation (https://docs.datadoghq.com/api/latest/authentication).
37+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
38+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
39+
--json Output in json format.
40+
--json-compact Output in compact json format.
41+
-n, --name string The name of the log stream.
42+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
43+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
44+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
45+
(default "{}")
46+
-r, --region string The region in which the datadog dashboard is created.
47+
If you are in the datadog EU site ('app.datadoghq.eu'), the Region should be EU otherwise it should be US.
4148
```
4249

4350

‎docs/auth0_logs_streams_update_eventbridge.md‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,10 @@ auth0 logs streams update eventbridge [flags]
2121
```
2222
auth0 logs streams update eventbridge
2323
auth0 logs streams update eventbridge <log-stream-id> --name <name>
24-
auth0 logs streams update eventbridge <log-stream-id> -n <name>
24+
auth0 logs streams update eventbridge <log-stream-id> --name <name>
25+
auth0 logs streams update eventbridge <log-stream-id> --name <name> --filters '[{"type":"category","name":"user.fail"},{"type":"category","name":"scim.event"}]'
26+
auth0 logs streams update eventbridge <log-stream-id> --name <name> --pii-config '{"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}'
27+
auth0 logs streams update eventbridge <log-stream-id> -n <name> -p null
2528
auth0 logs streams update eventbridge <log-stream-id> -n mylogstream --json
2629
auth0 logs streams update eventbridge <log-stream-id> -n mylogstream --json-compact
2730
```
@@ -30,9 +33,15 @@ auth0 logs streams update eventbridge [flags]
3033
## Flags
3134

3235
```
33-
--json Output in json format.
34-
--json-compact Output in compact json format.
35-
-n, --name string The name of the log stream.
36+
-m, --filters string Events matching these filters will be delivered by the stream, Formatted as JSON.
37+
Example: "[{"type":"category","name":"auth.login.fail"},{"type":"category","name":"auth.signup.fail"}]" (default "[]")
38+
--json Output in json format.
39+
--json-compact Output in compact json format.
40+
-n, --name string The name of the log stream.
41+
-c, --pii-config string Specifies how PII fields are logged, Formatted as JSON.
42+
including which fields to log (first_name, last_name, username, email, phone, address),the protection method (mask or hash), and the hashing algorithm (xxhash).
43+
Example : {"log_fields": ["first_name", "last_name"], "method": "mask", "algorithm": "xxhash"}.
44+
(default "{}")
3645
```
3746

3847

0 commit comments

Comments
 (0)