Skip to content

Commit 450c373

Browse files
mboglesbymboglesby
committed
Pull request #4: Feature/flexcache k8s
Merge in SIE-BB/netapp-dataops-toolkit from feature/flexcache-k8s to release-v2.7.0 * commit '31050cac6316bdb898bf7d1c7469c284fafc6ae2': (76 commits) correction correction correction alignment in -h deleting k8s representations before deleting actual resources corrections to instructions and warnings update .md files for delete_flexcache_volume warning on deleting flexcache origin volume correction validating flexcache volume unmount corrections to volume unnmounting taking flexcache offline flexcache unmounting correction debugging Revert "raise error if delete_flexcache_volume is used to delete non-flexcache volume" raise error if delete_flexcache_volume is used to delete non-flexcache volume error message while deleting flexcache volume using delete_volume unmounting and taking flexcache volume offline before deleting correction ...
2 parents 8ae2835 + 31050ca commit 450c373

4 files changed

Lines changed: 868 additions & 22 deletions

File tree

‎netapp_dataops_k8s/docs/volume_management.md‎

Lines changed: 190 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,18 @@ The NetApp DataOps Toolkit for Kubernetes can be used to manage persistent volum
88

99
You can perform volume management operations using the toolkit's command line utility. The command line utility supports the following operations.
1010

11-
| Kubernetes persistent volume management operations | Supported by BeeGFS | Supported by Trident |
12-
| ------------------------------------------------------------------------------------ | ------------------- | -------------------- |
13-
| [Clone a persistent volume.](#cli-clone-volume) | No | Yes |
14-
| [Create a new persistent volume.](#cli-create-volume) | Yes | Yes |
15-
| [Delete an existing persistent volume.](#cli-delete-volume) | Yes | Yes |
16-
| [List all persistent volumes.](#cli-list-volumes) | Yes | Yes |
17-
| [Create a new snapshot for a persistent volume.](#cli-create-volume-snapshot) | No | Yes |
18-
| [Delete an existing snapshot.](#cli-delete-volume-snapshot) | No | Yes |
19-
| [List all snapshots.](#cli-list-volume-snapshots) | No | Yes |
20-
| [Restore a snapshot.](#cli-restore-volume-snapshot) | No | Yes |
11+
| Kubernetes persistent volume management operations | Supported by BeeGFS | Supported with Trident |
12+
| ------------------------------------------------------------------------------------ | ------------------- | ---------------------- |
13+
| [Clone a persistent volume.](#cli-clone-volume) | No | Yes |
14+
| [Create a new persistent volume.](#cli-create-volume) | Yes | Yes |
15+
| [Delete an existing persistent volume.](#cli-delete-volume) | Yes | Yes |
16+
| [List all persistent volumes.](#cli-list-volumes) | Yes | Yes |
17+
| [Create a new snapshot for a persistent volume.](#cli-create-volume-snapshot) | No | Yes |
18+
| [Delete an existing snapshot.](#cli-delete-volume-snapshot) | No | Yes |
19+
| [List all snapshots.](#cli-list-volume-snapshots) | No | Yes |
20+
| [Restore a snapshot.](#cli-restore-volume-snapshot) | No | Yes |
21+
| [Create a new FlexCache volume](#cli-create-flexcache) | No | Yes |
22+
| [Delete an existing FlexCache volume](#cli-delete-flexcache) | No | Yes |
2123

2224
### Kubernetes Persistent Volume Management Operations
2325

@@ -120,6 +122,9 @@ Volume successfully created and bound to PersistentVolumeClaim (PVC) 'test1' in
120122

121123
The NetApp DataOps Toolkit can be used to near-instantaneously delete an existing persistent volume within a Kubernetes cluster. The command for deleting an existing persistent volume is `netapp_dataops_k8s_cli.py delete volume`.
122124

125+
>[!NOTE]
126+
When deleting a volume that is the origin for a FlexCache volume using the NetApp DataOps Toolkit, the PVC will be deleted, but the volume will still exist on the ONTAP cluster. You must delete all associated FlexCache volumes first to avoid errors when deleting the origin volume directly from ONTAP.
127+
123128
The following options/arguments are required:
124129

125130
```
@@ -317,28 +322,116 @@ Restoring VolumeSnapshot 'snap1' for PersistentVolumeClaim 'project1' in namespa
317322
VolumeSnapshot successfully restored.
318323
```
319324
325+
<a name="cli-create-flexcache"></a>
326+
327+
#### Create a FlexCache Volume
328+
329+
The NetApp DataOps Toolkit can be used to create a FlexCache volume in ONTAP and then create a PV and PVC representing the FlexCache in Kubernetes. The command to create a FlexCache volume is `netapp_dataops_k8s_cli.py create flexcache`.
330+
331+
>[!NOTE]
332+
>The source and target Storage Virtual Machines (SVMs) need to have been peered as a prerequisite to running this command.
333+
334+
The following options/arguments are required:
335+
336+
```
337+
-f, --flexcache-vol= Name of the FlexCache volume to be created. This is the name that will be applied to the new Kubernetes PersistentVolumeClaim (PVC).
338+
-s, --source-svm= Name of the source Storage Virtual Machine (SVM) that contains the origin volume to be cached.
339+
-v, --source-vol= Name of the source volume in the source SVM that will be cached by the FlexCache volume.
340+
-z, --flexcache-size= Size of the FlexCache volume to be created. The size must be specified in a format such as '1024Mi', '100Gi', '10Ti', etc. Note: The size must be at least 50Gi.
341+
-b, --backend-name= Name of the tridentbackendconfig.
342+
343+
```
344+
345+
The following options/arguments are optional:
346+
347+
```
348+
-c, --junction The junction path for the FlexCache volume.
349+
-h, --help Print help text.
350+
-n, --namespace= Kubernetes namespace to create the new PersistentVolumeClaim (PVC) in. If not specified, the PVC will be created in the "default" namespace.
351+
-t, --trident-namespace= Kubernetes namespace where Trident is installed. Default is "trident".
352+
```
353+
354+
##### Example Usage
355+
356+
Create a FlexCache volume 'test-cache-vol1' of size '53Gi', and attach it to a Kubernetes PersistentVolumeClaim (PVC) named 'test-vol1' in namespace 'trident', using 'ontap' tridentbackendconfig.
357+
358+
```sh
359+
netapp_dataops_k8s_cli.py create flexcache --flexcache-vol=test-cache-vol1 --source-vol=test-vol1 --source-svm=svm0 --flexcache-size=53Gi --backend-name=ontap --namespace=trident --trident-namespace=netapp-trident
360+
Creating FlexCache: svm0:test-vol1 -> svm0:test-cache-vol1
361+
FlexCache created successfully.
362+
[K8s] Creating PV 'pv-test-cache-vol1' in namespace 'trident'...
363+
[K8s] PV 'pv-test-cache-vol1' created successfully.
364+
[K8s] Creating PVC 'test-cache-vol1' in namespace 'trident'...
365+
[K8s] PVC 'test-cache-vol1' created successfully.
366+
Waiting for Kubernetes to bind volume to PVC.
367+
[K8s] PVC 'test-cache-vol1' is bound to PV 'pv-test-cache-vol1'.
368+
Volume successfully created and bound to PersistentVolumeClaim (PVC) 'test-cache-vol1' in namespace 'trident'.
369+
```
370+
371+
<a name="cli-delete-flexcache"></a>
372+
373+
#### Delete an existing FlexCache volume
374+
375+
The NetApp DataOps Toolkit can be used to delete a FlexCache volume in ONTAP and remove its associated Kubernetes PVC and PV. The command to delete a FlexCache volume is `netapp_dataops_k8s_cli.py delete flexcache-volume`.
376+
377+
Required Options/Arguments:
378+
379+
```
380+
-p, --pvc-name= Name of Kubernetes PersistentVolumeClaim (PVC) to be deleted.
381+
-b, --backend-name= Name of Trident backend config.
382+
```
383+
384+
Optional Options/Arguments:
385+
386+
```
387+
-f, --force Do not prompt user to confirm operation.
388+
-h, --help Print help text.
389+
-n, --namespace= Kubernetes namespace that PersistentVolumeClaim (PVC) is located in. If not specified, namespace "default" will be used.
390+
-t, --trident-namespace= Kubernetes namespace where Trident is installed. If not specified, the namespace "trident" will be used.
391+
```
392+
393+
##### Example Usage
394+
395+
Delete FlexCache volume 'test-cach-vol1', using 'ontap' tridentbackendconfig.
396+
397+
```sh
398+
netapp_dataops_k8s_cli.py delete flexcache-volume -p test-cache-vol1 -b ontap
399+
Warning: This FlexCache volume will be permanently deleted.
400+
Are you sure that you want to proceed? (yes/no): yes
401+
Unmounting FlexCache volume 'test-cache-vol1' in SVM 'svm0'.
402+
Taking FlexCache volume 'test-cache-vol1' offline in SVM 'svm0'.
403+
Deleting FlexCache volume 'test-cache-vol1' in SVM 'svm0'.
404+
FlexCache volume 'test-cache-vol1' successfully deleted.
405+
Deleting PVC 'test-cache-vol1' in namespace 'default'.
406+
PVC 'test-cache-vol1' successfully deleted.
407+
Deleting PersistentVolume (PV) 'pv-test-cache-vol1' associated with FlexCache PVC 'test-cache-vol1'.
408+
PersistentVolume (PV) 'pv-test-cache-vol1' successfully deleted.
409+
```
410+
320411
<a name="library-of-functions"></a>
321412
322413
## Advanced: Set of Functions
323414
324415
The NetApp DataOps Toolkit for Kubernetes provides a set of functions that can be imported into any Python program or Jupyter Notebook. In this manner, data scientists and data engineers can easily incorporate Kubernetes-native data management tasks into their existing projects, programs, and workflows. This functionality is only recommended for advanced users who are proficient in Python.
325416
326417
```py
327-
from netapp_dataops.k8s import clone_volume, create_volume, delete_volume, list_volumes, create_volume_snapshot, delete_volume_snapshot, list_volume_snapshots, restore_volume_snapshot
418+
from netapp_dataops.k8s import clone_volume, create_volume, delete_volume, list_volumes, create_volume_snapshot, delete_volume_snapshot, list_volume_snapshots, restore_volume_snapshot, create_flexcache, delete_flexcache_volume
328419
```
329420
330421
The following volume management operations are available within the set of functions.
331422
332-
| Kubernetes persistent volume management operations | Supported by BeeGFS | Supported by Trident |
333-
| ------------------------------------------------------------------------------------ | ------------------- | -------------------- |
334-
| [Clone a persistent volume.](#lib-clone-volume) | No | Yes |
335-
| [Create a new persistent volume.](#lib-create-volume) | Yes | Yes |
336-
| [Delete an existing persistent volume.](#lib-delete-volume) | Yes | Yes |
337-
| [List all persistent volumes.](#lib-list-volumes) | Yes | Yes |
338-
| [Create a new snapshot for a persistent volume.](#lib-create-volume-snapshot) | No | Yes |
339-
| [Delete an existing snapshot.](#lib-delete-volume-snapshot) | No | Yes |
340-
| [List all snapshots.](#lib-list-volume-snapshots) | No | Yes |
341-
| [Restore a snapshot.](#lib-restore-volume-snapshot) | No | Yes |
423+
| Kubernetes persistent volume management operations | Supported by BeeGFS | Supported with Trident |
424+
| ------------------------------------------------------------------------------------ | ------------------- | ---------------------- |
425+
| [Clone a persistent volume.](#lib-clone-volume) | No | Yes |
426+
| [Create a new persistent volume.](#lib-create-volume) | Yes | Yes |
427+
| [Delete an existing persistent volume.](#lib-delete-volume) | Yes | Yes |
428+
| [List all persistent volumes.](#lib-list-volumes) | Yes | Yes |
429+
| [Create a new snapshot for a persistent volume.](#lib-create-volume-snapshot) | No | Yes |
430+
| [Delete an existing snapshot.](#lib-delete-volume-snapshot) | No | Yes |
431+
| [List all snapshots.](#lib-list-volume-snapshots) | No | Yes |
432+
| [Restore a snapshot.](#lib-restore-volume-snapshot) | No | Yes |
433+
| [Create a new FlexCache volume.](#lib-create-flexcache) | No | Yes |
434+
| [Delete an existing FlexCache volume](#lib-delete-flexcache) | No | Yes |
342435
343436
### Kubernetes Persistent Volume Management Operations
344437
@@ -415,6 +508,9 @@ APIConnectionError # The Kubernetes API returned an error.
415508
416509
The NetApp DataOps Toolkit can be used to near-instantaneously delete an existing persistent volume within a Kubernetes cluster as part of any Python program or workflow.
417510
511+
>[!NOTE]
512+
>When deleting a volume with associated FlexCache volumes using the NetApp DataOps Toolkit, the PVC will be deleted, but the volume will still exist on the ONTAP cluster. You must delete all associated FlexCache volumes first to avoid errors when deleting the origin volume directly from ONTAP.
513+
418514
##### Function Definition
419515
420516
```py
@@ -590,4 +686,77 @@ If an error is encountered, the function will raise an exception of one of the f
590686
```py
591687
InvalidConfigError # kubeconfig file is missing or is invalid.
592688
APIConnectionError # The Kubernetes API returned an error.
689+
```
690+
691+
<a name="lib-create-flexcache"></a>
692+
693+
#### Create a FlexCache Volume
694+
695+
The NetApp DataOps Toolkit can be used to create a FlexCache volume in ONTAP and then create a PV and PVC representing the FlexCache in Kubernetes.
696+
697+
>[!NOTE]
698+
>The source and target Storage Virtual Machines (SVMs) need to have been peered as a prerequisite to running this command.
699+
700+
##### Function Definition
701+
702+
```py
703+
def create_flexcache(
704+
source_vol: str, # Name of the source volume in the source SVM that will be cached by the FlexCache volume (required).
705+
source_svm: str, # Name of the source Storage Virtual Machine (SVM) that contains the origin volume to be cached (required).
706+
flexcache_vol: str, # Name of the FlexCache volume to be created (required).
707+
flexcache_size: str, # Size of the FlexCache volume to be created. Format: '1024Mi', '100Gi', '10Ti', etc. Note: The size must be at least 50Gi (required).
708+
backend_name: str, # Name of the tridentbackendconfig (required).
709+
junction: str = None, # The junction path for the FlexCache volume (optional).
710+
namespace: str = "default", # Kubernetes namespace to create the new PersistentVolumeClaim (PVC) in. If not specified, the PVC will be created in the "default" namespace (optional).
711+
trident_namespace: str = "trident", # Kubernetes namespace where Trident is installed (optional).
712+
print_output: bool = False # Denotes whether or not to print messages to the console during execution (optional).
713+
) :
714+
```
715+
716+
##### Return Value
717+
718+
The function returns a dictionary containing details about the created FlexCache volume and the associated Kubernetes PersistentVolumeClaim (PVC). The keys for the values in this dictionary are:
719+
720+
- "ontap_flexcache": The FlexCache volume in ONTAP in the format "{svm}: {flexcache_vol}".
721+
- "k8s_pvc": The name of the Kubernetes PersistentVolumeClaim (PVC) created.
722+
723+
##### Error Handling
724+
725+
If an error is encountered, the function will raise an exception of one of the following types. These exception types are defined in `netapp_dataops.k8s`.
726+
727+
```py
728+
InvalidConfigError # kubeconfig file is missing or is invalid.
729+
APIConnectionError # The Kubernetes API returned an error.
730+
InvalidVolumeParameterError # Invalid parameters specified for the FlexCache volume.
731+
ConnectionTypeError # Invalid connection type specified.
732+
NetAppRestError # Error with the NetApp REST API.
733+
```
734+
735+
<a name="lib-delete-flexcache"></a>
736+
737+
The NetApp DataOps Toolkit can be used to delete a FlexCache volume in ONTAP and remove its associated Kubernetes PVC and PV.
738+
739+
##### Function Definition
740+
741+
```py
742+
def delete_flexcache_volume(
743+
pvc_name: str, # Name of Kubernetes PersistentVolumeClaim (PVC) to be deleted (required).
744+
backend_name: str, # Name of Trident backend config (required).
745+
namespace: str = "default", # Kubernetes namespace that PersistentVolumeClaim (PVC) is located in. If not specified, namespace "default" will be used.
746+
trident_namespace: str = "trident", # Namespace where Trident is installed. If not specified, "trident" will be used.
747+
print_output: bool = False # Print progress messages to the console during execution.
748+
)
749+
```
750+
751+
##### Return Value
752+
753+
None
754+
755+
##### Error Handling
756+
757+
If an error is encountered, the function will raise an exception of one of the following types. These exception types are defined in `netapp_dataops.k8s`.
758+
759+
```py
760+
InvalidConfigError # kubeconfig file is missing or is invalid.
761+
APIConnectionError # The Kubernetes or ONTAP API returned an error.
593762
```

0 commit comments

Comments
 (0)