Skip to content

Commit 2e3686a

Browse files
dsajdakdsajdak
committed
Merge remote-tracking branch 'upstream/master'
2 parents aed967b + 5b0b6e7 commit 2e3686a

9 files changed

Lines changed: 179 additions & 6 deletions

File tree

‎database/xdmod.cnf‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[mysqld]
2+
sql_mode =

‎ondemand/Dockerfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,6 @@ COPY notebook_data /data/notebook_data
3535
COPY initializers/dashboard.rb /etc/ood/config/apps/dashboard/initializers/dashboard.rb
3636
COPY themes/hpc-coop /usr/share/ondemand-dex/web/themes/hpc-coop
3737

38+
RUN sed -i '/ passenger_load_shell_envvars off;/a passenger_preload_bundler on;' /opt/ood/nginx_stage/templates/pun.conf.erb
39+
3840
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

‎ondemand/README.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Live tutorial steps we took during PEARC. See the PEARC video recording to follo
66

77
- [Dashboard-developer-mode-tutorial](#dashboard-developer-mode-tutorial)
88
- [Jupyter App Tutorial](#jupyter-app-tutorial)
9+
- [Dynamic Batch Connect Fields](#dynamic-batch-connect-fields)
910
- [Passenger App Tutorial](#passenger-app-tutorial)
1011
- [XDMoD Integration Tutorial](#xdmod-integration-tutorial)
1112

@@ -58,6 +59,7 @@ mkdir -p ~/ondemand/dev
5859
cd ~/ondemand/dev
5960
ln -s ../../ondemand-src-full/apps/dashboard/ dashboard
6061
cd dashboard
62+
bin/bundle config --local path vendor/bundle
6163
bin/setup
6264
```
6365

@@ -913,6 +915,95 @@ Jupyter system app in the menu along with your sandbox development app.
913915

914916
![deploy to production](imgs/deploy_to_production.gif)
915917

918+
## Dynamic Batch Connect Fields
919+
920+
Since 2.0 sites can enable dynamic batch connect fields through setting the `OOD_BC_DYNAMIC_JS` environment
921+
variable. This has already been done within these containers.
922+
923+
```text
924+
# /etc/ood/config/apps/dashboard/env
925+
OOD_BC_DYNAMIC_JS=1
926+
```
927+
928+
With this feature - client side javascript can dynamically change the form fields based on user
929+
choices. Sites only have to add more YAML to a `form.yml` to enable this behaviour. Let's
930+
see some examples.
931+
932+
### Changing min & max values
933+
934+
Let's put some rules around the debug queue. We set a static `min` and `max` of 200 and 1000
935+
respectively. But in this example, we want different min and max values for the debug queue.
936+
937+
We can configure this behaviour with these `data-min-` and `data-max-` directives attached
938+
to a given option. When the `debug` queue is choosen we'll automatically set the min and
939+
maximum values of the `memory` field.
940+
941+
Note that we're also setting the `compute` min and maxes again. This is currently the only way
942+
to reset back to defaults if there are any.
943+
944+
```yaml
945+
# form.yml, only showing custom_queue for brevity.
946+
custom_queue:
947+
widget: "select"
948+
label: "Partition"
949+
options:
950+
- [
951+
"Compute", "compute",
952+
data-min-memory: 200, # set the compute queue back to static defaults
953+
data-max-memory: 1000,
954+
]
955+
- [
956+
"Debug", "debug",
957+
data-min-memory: 400, # change min & max for debug queue
958+
data-max-memory: 600,
959+
]
960+
```
961+
962+
![A gif of a user interacting with the form with the dynamic additons described above. The default queue compute is chosen. The user can range memory form from 200 to 1000. The user chooses the debug queue. The user can now range the memory form item from 400 to 600. Switching back to the compute queue and the user can again range the memory form item from 200 to 1000.](imgs/dynamic_min_max.gif)
963+
964+
### Changing values
965+
966+
Let's take this a little further. Now, when we choose `compute` or `debug` queue, let's automatically
967+
set the Slurm account we want to use. Note we'll need to add `bc_account` back, as it's what we'll be
968+
setting.
969+
970+
We can add the `data-set` directives on the same `custom_queue` form options. When users choose the
971+
`debug` queue we'll automatically set the account to `staff`. When we choose the `compute` queue we
972+
will set the `sfoster` account.
973+
974+
```yaml
975+
# form.yml, only showing custom_queue for brevity.
976+
attributes:
977+
custom_queue:
978+
widget: "select"
979+
label: "Partition"
980+
options:
981+
- [
982+
"Compute", "compute",
983+
data-min-memory: 200,
984+
data-max-memory: 1000,
985+
986+
data-set-bc-account: 'sfoster' # set the account to sfoster when using compute
987+
]
988+
- [
989+
"Debug", "debug",
990+
data-min-memory: 400,
991+
data-max-memory: 600,
992+
993+
data-set-bc-account: 'staff' # set the account to staff when using debug
994+
]
995+
form:
996+
- bc_account
997+
```
998+
999+
To use the `sfoster` account you need to run these commands to add the `hpcadmin` user to
1000+
that account.
1001+
1002+
```
1003+
sudo sacctmgr add user hpcadmin account=sfoster
1004+
sudo sacctmgr modify user where user=hpcadmin set defaultaccount=staff
1005+
```
1006+
9161007
## Passenger app tutorial
9171008
9181009
Access OnDemand dashboard https://localhost:3443

‎ondemand/imgs/dynamic_min_max.gif‎

434 KB
Loading

‎xdmod/entrypoint.sh‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,9 +75,15 @@ then
7575
export TERM=linux
7676
/srv/xdmod/scripts/supremm.py
7777

78+
echo "---> XDMoD Open OnDemand module setup"
79+
expect /srv/xdmod/scripts/xdmod-setup-ondemand.tcl | col -b
80+
7881
echo "---> Make sure we have a place to keep our backups"
7982
mkdir -p /srv/xdmod/backups
8083
fi
84+
echo "---> Staring php-fpm"
85+
mkdir -p /run/php-fpm
86+
php-fpm
8187

8288
echo "---> Starting HTTPD on xdmod..."
8389
# Sometimes on shutdown pid still exists, so delete it

‎xdmod/install.sh‎

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@ ARCHTYPE=`uname -m`
2121

2222
dnf install -y \
2323
expect \
24-
python2-pip
24+
python2-pip \
25+
php-pear \
26+
php-devel
2527

2628
pip2 install pexpect==4.4.0
2729

@@ -37,7 +39,10 @@ pip2 install pexpect==4.4.0
3739
# be installed in the same container. In a production deployment they may be installed
3840
# on separate hosts.
3941
#------------------------
40-
dnf install -y https://github.com/ubccr/xdmod/releases/download/v10.0.0/xdmod-10.0.0-1.0.beta1.el8.noarch.rpm
42+
dnf install -y https://github.com/ubccr/xdmod/releases/download/v10.0.0-beta4-el8/xdmod-10.0.0-1.4.beta4.el8.noarch.rpm \
43+
https://github.com/ubccr/xdmod-ondemand/releases/download/v10.0.0/xdmod-ondemand-10.0.0-1.0.beta1.el8.noarch.rpm \
44+
https://github.com/ubccr/xdmod/releases/download/v10.0.0-beta4-el8/xdmod-supremm-10.0.0-1.4.beta4.el8.noarch.rpm \
45+
https://github.com/ubccr/supremm/releases/download/2.0.0-beta3/supremm-2.0.0-1.0_beta3.el8.x86_64.rpm
4146

4247
#------------------------
4348
# The Job Performance software uses MongoDB to store the job-level performance
@@ -49,7 +54,15 @@ dnf install -y https://github.com/ubccr/xdmod/releases/download/v10.0.0/xdmod-10
4954
dnf install -y \
5055
https://repo.mongodb.org/yum/redhat/8/mongodb-org/5.0/${ARCHTYPE}/RPMS/mongodb-org-shell-5.0.9-1.el8.${ARCHTYPE}.rpm
5156

52-
pip3 install pymongo --upgrade
57+
58+
59+
#------------------------
60+
#
61+
#------------------------
62+
pecl install mongodb
63+
echo "extension=mongodb.so" >> /etc/php.d/40-mongodb.ini
64+
65+
pip3 install pymongo==3.7.0 --upgrade
5366
pip2 install pymongo --upgrade
5467

5568
#------------------------
@@ -74,6 +87,14 @@ sed -i 's/.*date.timezone[[:space:]]*=.*/date.timezone = UTC/' /etc/php.ini
7487
#------------------------
7588
rm -f /etc/httpd/conf.d/ssl.conf
7689

90+
#------------------------
91+
# We need to make sure that we have access to this file so that SSO works.
92+
#------------------------
93+
if [[ -f /etc/pki/tls/private/localhost.key ]]; then
94+
chown root:apache /etc/pki/tls/private/localhost.key
95+
chmod 750 /etc/pki/tls/private/localhost.key
96+
fi
97+
7798
#------------------------
7899
# These commands remove cached files to reduce the overall image size.
79100
#------------------------

‎xdmod/scripts/supremm.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
#!/usr/bin/env python
1+
#!/usr/bin/env python2
22
import pexpect
33
import sys
44

‎xdmod/scripts/xdmod-setup-ondemand.tcl‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/usr/bin/env expect
2+
# Expect script that run s xdmod-setup to configure a freshly installed
3+
# XDMoD instance. This script will fail if run against an already installed
4+
# XDMoD.
5+
6+
#-------------------------------------------------------------------------------
7+
# Configuration settings for the XDMoD resources
8+
9+
#-------------------------------------------------------------------------------
10+
11+
# Load helper functions from helper-functions.tcl
12+
source [file join [file dirname [info script]] helper-functions.tcl]
13+
14+
#-------------------------------------------------------------------------------
15+
# main body - note there are some hardcoded addresses, usernames and passwords here
16+
# they should typically not be changed as they need to match up with the
17+
# settings in the docker container
18+
19+
set timeout 240
20+
spawn "xdmod-setup"
21+
22+
# Add an OnDemand resource
23+
selectMenuOption 4
24+
25+
selectMenuOption 1
26+
provideInput {Resource Name:} ondemand
27+
provideInput {Formal Name:} {Open OnDemand Instance}
28+
provideInput {Resource Type*} Gateway
29+
provideInput {How many nodes does this resource have?} 0
30+
provideInput {How many total processors (cpu cores) does this resource have?} 0
31+
32+
selectMenuOption s
33+
confirmFileWrite yes
34+
enterToContinue
35+
confirmFileWrite yes
36+
enterToContinue
37+
38+
# Setup the OnDemand database
39+
selectMenuOption 10
40+
41+
selectMenuOption d
42+
43+
answerQuestion {DB Admin Username} root
44+
providePassword {DB Admin Password:} {}
45+
confirmDropDb yes
46+
provideInput {Do you want to see the output*} {no}
47+
48+
selectMenuOption q
49+
50+
selectMenuOption q
51+
52+
lassign [wait] pid spawnid os_error_flag value
53+
exit $value

‎xdmod/scripts/xdmod-setup-start.tcl‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,6 @@ spawn "xdmod-setup"
1717
selectMenuOption 1
1818
answerQuestion {Site Address} https://localhost:4443/
1919
provideInput {Email Address:} ccr-xdmod-help@buffalo.edu
20-
provideInput {Java Path:} /usr/bin/java
21-
provideInput {Javac Path:} /usr/bin/javac
2220
provideInput {Chromium Path:} /usr/lib64/chromium-browser/headless_shell
2321
provideInput {Center Logo Path:} /srv/xdmod/small-logo.png
2422
provideInput {Center Logo Width:} 354

0 commit comments

Comments
 (0)