Skip to content

Commit a76a400

Browse files
committed
documentation update
1 parent d5b9c04 commit a76a400

1 file changed

Lines changed: 78 additions & 0 deletions

File tree

examples/tutorials/demo-vipsession.ipynb

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -499,6 +499,84 @@
499499
"! tree {new_session.output_dir}"
500500
]
501501
},
502+
{
503+
"cell_type": "markdown",
504+
"metadata": {},
505+
"source": [
506+
"## Re-use a same dataset multiples times\n",
507+
"You can re-use the same dataset multiples times. For that you will only need to keep the trace of where the inputs where uploaded on VIP. \n",
508+
"You will also need to not run `finish()` at the end of the session which has uploaded the dataset, it will prevent your dataset from being deleted. \n",
509+
"When you will reuse the data don't forget to adapt the paths in `inputs_settings` by using the path where they were initially uploaded."
510+
]
511+
},
512+
{
513+
"cell_type": "code",
514+
"execution_count": null,
515+
"metadata": {},
516+
"outputs": [],
517+
"source": [
518+
"session = VipSession(\"session-A\")\n",
519+
"session.upload_inputs(input_dir)\n",
520+
"\n",
521+
"inputs_settings = {\n",
522+
" \"file\": \"initial.file\",\n",
523+
" \"value\": 5\n",
524+
"}\n",
525+
"\n",
526+
"input_folder_on_vip = session._vip_input_dir\n",
527+
"print(f\"The dataset is located on VIP here: {input_folder_on_vip}\") # keep this information somewhere\n",
528+
"\n",
529+
"session.launch_pipeline(pipeline_id, input_settings)\n",
530+
"session.monitor_workflows()\n",
531+
"# no finish for the first session"
532+
]
533+
},
534+
{
535+
"cell_type": "code",
536+
"execution_count": null,
537+
"metadata": {},
538+
"outputs": [],
539+
"source": [
540+
"session = VipSession(\"session-B\")\n",
541+
"# do not forget to prepend your inputs !\n",
542+
"adapted_inputs_settings = {\n",
543+
" \"file\" : f\"{input_folder_on_vip}/initial.file\", # reuse the stored inforation !\n",
544+
" \"value\": 5\n",
545+
"}\n",
546+
"session.launch_pipeline(pipeline_id, adapted_inputs_settings)\n",
547+
"session.monitor_workflows()\n",
548+
"session.finish()"
549+
]
550+
},
551+
{
552+
"cell_type": "markdown",
553+
"metadata": {},
554+
"source": [
555+
"> [!NOTE]\n",
556+
"> At the very end when you won't need the dataset anymore don't forget to run `VipSession(session-A).finish()` for cleaning the data from VIP servers.\n",
557+
"> You must name your session like you named it for uploading your dataset."
558+
]
559+
},
560+
{
561+
"cell_type": "code",
562+
"execution_count": null,
563+
"metadata": {},
564+
"outputs": [],
565+
"source": [
566+
"upload_session = VipSession(\"upload-session\")\n",
567+
"upload_session.upload_inputs(input_dir)\n",
568+
"# * running the session * #\n",
569+
"\n",
570+
"reuse_session_a = VipSession(\"reuse-session_a\")\n",
571+
"# * reunning the session on the previous dataset * #\n",
572+
"\n",
573+
"reuse_session_b = VipSession(\"reuse-session_b\")\n",
574+
"# * reunning the session on the previous dataset * #\n",
575+
"\n",
576+
"# finally deleting the dataset\n",
577+
"VipSession(\"upload-session\").finish()"
578+
]
579+
},
502580
{
503581
"attachments": {},
504582
"cell_type": "markdown",

0 commit comments

Comments
 (0)