Skip to content

Commit 034f224

Browse files
filiperochalopesfiliperochalopes
authored
Improve development environment defaults and storage setup (#849)
* chore(dev-env): add env template and archive override * Add .env.example documenting development defaults for services, storage, and optional integrations * Rename docker-compose.override.yml to docker-compose.override.yml.bak and retain expanded rustfs/minio setup * Ignore override file to prevent accidental inclusion in version control * fix(core): use external MinIO endpoint for presigned URLs * Add MINIO_PORT and MINIO_EXTERNAL_ENDPOINT defaults to expose stable host and port for browser access * Introduce MINIO_EXTERNAL_ENDPOINT and MINIO_REGION settings for external client configuration * Create dedicated external MinIO client for url_for to generate correct presigned URLs * Update docker-compose overrides to pass new MinIO environment variables * chore(s3-like-export): rename external_client to _presign_client because it was ambiguous
1 parent c0d920a commit 034f224

File tree

5 files changed

+148
-3
lines changed

5 files changed

+148
-3
lines changed

‎.env.example‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# =============================================================================
2+
# OCL API2 - Environment Variables
3+
# Copy this file to .env and adjust the values for your environment.
4+
# =============================================================================
5+
6+
# -----------------------------------------------------------------------------
7+
# General
8+
# -----------------------------------------------------------------------------
9+
ENVIRONMENT=development
10+
DEBUG=TRUE
11+
# Disables the lookup of language maps on startup (speeds up dev boot)
12+
NO_LM=TRUE
13+
SERVICE_NAME=oclapi2
14+
15+
# -----------------------------------------------------------------------------
16+
# API
17+
# -----------------------------------------------------------------------------
18+
API_BASE_URL=http://localhost:8000
19+
API_INTERNAL_BASE_URL=http://api:8000
20+
API_HOST=0.0.0.0
21+
API_PORT=8000
22+
API_SUPERUSER_PASSWORD=Root123
23+
API_SUPERUSER_TOKEN=891b4b17feab99f3ff7e5b5d04ccc5da7aa96da6
24+
SECRET_KEY=
25+
26+
# -----------------------------------------------------------------------------
27+
# Database (PostgreSQL)
28+
# -----------------------------------------------------------------------------
29+
DB=postgres
30+
DB_HOST=db
31+
DB_PORT=5432
32+
DB_USER=postgres
33+
DB_PASSWORD=Postgres123
34+
35+
# -----------------------------------------------------------------------------
36+
# Redis
37+
# -----------------------------------------------------------------------------
38+
REDIS_HOST=redis
39+
REDIS_PORT=6379
40+
41+
# -----------------------------------------------------------------------------
42+
# Elasticsearch
43+
# -----------------------------------------------------------------------------
44+
ES_HOSTS=es:9200
45+
ES_SCHEME=http
46+
47+
# -----------------------------------------------------------------------------
48+
# Export Storage
49+
# Supported backends:
50+
# core.services.storages.cloud.aws.S3 (AWS S3)
51+
# core.services.storages.cloud.minio.MinIO (MinIO or any S3-compatible, e.g. RustFS)
52+
# core.services.storages.cloud.azure.BlobStorage
53+
# -----------------------------------------------------------------------------
54+
EXPORT_SERVICE=core.services.storages.cloud.minio.MinIO
55+
56+
# MinIO / RustFS (S3-compatible) — used when EXPORT_SERVICE=MinIO
57+
MINIO_PORT=9090
58+
MINIO_ENDPOINT=rustfs:9000
59+
MINIO_EXTERNAL_ENDPOINT=localhost:${MINIO_PORT:-9090}
60+
MINIO_ACCESS_KEY=rustfsadmin
61+
MINIO_SECRET_KEY=rustfsadmin
62+
MINIO_BUCKET_NAME=oclapi2-dev
63+
MINIO_SECURE=false
64+
65+
# AWS S3 — used when EXPORT_SERVICE=S3
66+
AWS_ACCESS_KEY_ID=
67+
AWS_SECRET_ACCESS_KEY=
68+
AWS_STORAGE_BUCKET_NAME=oclapi2-dev
69+
AWS_REGION_NAME=us-east-2
70+
71+
# Azure Blob Storage — used when EXPORT_SERVICE=BlobStorage
72+
AZURE_STORAGE_ACCOUNT_NAME=
73+
AZURE_STORAGE_CONTAINER_NAME=
74+
AZURE_STORAGE_CONNECTION_STRING=
75+
76+
# -----------------------------------------------------------------------------
77+
# Flower (Celery monitoring UI)
78+
# -----------------------------------------------------------------------------
79+
FLOWER_USER=root
80+
FLOWER_PASSWORD=Root123
81+
82+
# -----------------------------------------------------------------------------
83+
# FHIR Validator
84+
# -----------------------------------------------------------------------------
85+
FHIR_VALIDATOR_URL=http://fhir_validator:3500
86+
87+
# -----------------------------------------------------------------------------
88+
# Error tracking (optional)
89+
# -----------------------------------------------------------------------------
90+
SENTRY_DSN_KEY=
91+
ERRBIT_URL=
92+
ERRBIT_KEY=
93+
94+
# -----------------------------------------------------------------------------
95+
# Email (optional)
96+
# -----------------------------------------------------------------------------
97+
EMAIL_NOREPLY_PASSWORD=
98+
99+
# -----------------------------------------------------------------------------
100+
# Throttling (optional)
101+
# -----------------------------------------------------------------------------
102+
ENABLE_THROTTLING=

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,7 @@ venv/
257257
ENV/
258258
env.bak/
259259
venv.bak/
260+
docker-compose.override.yml
260261

261262
# Spyder project settings
262263
.spyderproject

‎core/services/storages/cloud/minio.py‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,22 @@ class MinIO(CloudStorageServiceInterface):
1414
def __init__(self):
1515
super().__init__()
1616
self.endpoint = settings.MINIO_ENDPOINT
17+
self.external_endpoint = settings.MINIO_EXTERNAL_ENDPOINT or self.endpoint
1718
self.access_key = settings.MINIO_ACCESS_KEY
1819
self.secret_key = settings.MINIO_SECRET_KEY
1920
self.bucket_name = settings.MINIO_BUCKET_NAME
2021
self.secure = settings.MINIO_SECURE
2122
self.client = Minio(endpoint=self.endpoint, access_key=self.access_key, secret_key=self.secret_key,
2223
secure=self.secure)
24+
# Dedicated client for presigned URL generation.
25+
# Uses the external endpoint so the signature carries the publicly accessible host.
26+
# Region is set explicitly to avoid a network round-trip for region discovery
27+
# (the external endpoint is not reachable from inside the container).
28+
# All other operations use self.client (internal endpoint) since they run server-side.
29+
self._presign_client = Minio(
30+
endpoint=self.external_endpoint, access_key=self.access_key, secret_key=self.secret_key,
31+
secure=self.secure, region=settings.MINIO_REGION
32+
) if self.external_endpoint != self.endpoint else self.client
2333
# Ensure the bucket exists
2434
if not self.client.bucket_exists(self.bucket_name):
2535
self.client.make_bucket(self.bucket_name)
@@ -77,11 +87,14 @@ def upload_base64(self, doc_base64, file_name, append_extension=True, public_rea
7787

7888
def url_for(self, file_path):
7989
"""
80-
Generates a presigned URL for the given file.
90+
Generates a presigned URL for the given file using the external client so the
91+
signature is computed with the publicly accessible host (MINIO_EXTERNAL_ENDPOINT).
92+
Falls back to the internal client when no external endpoint is configured.
8193
"""
8294
try:
83-
return self.client.get_presigned_url(method='GET', bucket_name=self.bucket_name, object_name=file_path) \
84-
if file_path else None
95+
return self._presign_client.get_presigned_url(
96+
method='GET', bucket_name=self.bucket_name, object_name=file_path
97+
) if file_path else None
8598
except S3Error as e:
8699
raise Exception(f"Could not generate presigned URL for file {file_path}. Error: {e}") from e # pylint: disable=broad-exception-raised
87100

‎core/settings.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -625,6 +625,8 @@ def get_set_from_env(name):
625625

626626
# MINIO storage settings
627627
MINIO_ENDPOINT = os.environ.get('MINIO_ENDPOINT', '')
628+
MINIO_EXTERNAL_ENDPOINT = os.environ.get('MINIO_EXTERNAL_ENDPOINT', '')
629+
MINIO_REGION = os.environ.get('MINIO_REGION', 'us-east-1')
628630
MINIO_ACCESS_KEY = os.environ.get('MINIO_ACCESS_KEY', '')
629631
MINIO_SECRET_KEY = os.environ.get('MINIO_SECRET_KEY', '')
630632
MINIO_BUCKET_NAME = os.environ.get('MINIO_BUCKET_NAME', '')

docker-compose.override.yml renamed to docker-compose.override.yml.bak

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,25 @@
11
services:
2+
rustfs:
3+
image: rustfs/rustfs
4+
ports:
5+
- ${MINIO_PORT:-9000}:9000
6+
- 9001:9001
7+
environment:
8+
- RUSTFS_ROOT_USER=${MINIO_ACCESS_KEY-rustfsadmin}
9+
- RUSTFS_ROOT_PASSWORD=${MINIO_SECRET_KEY-rustfsadmin}
10+
volumes:
11+
- rustfs-data:/data
12+
restart: "no"
213
db:
314
ports:
415
- 5432:5432
516
restart: "no"
17+
command: postgres -c max_connections=300
618
redis:
719
ports:
820
- 6379:6379
921
restart: "no"
22+
command: /opt/bitnami/scripts/redis/run.sh --maxmemory 4096mb
1023
api:
1124
build: .
1225
ports:
@@ -20,9 +33,18 @@ services:
2033
- es
2134
- celery
2235
- flower
36+
- rustfs
2337
environment:
2438
- ENVIRONMENT=development
2539
- DEBUG=${DEBUG-TRUE}
40+
- NO_LM=TRUE
41+
- EXPORT_SERVICE=core.services.storages.cloud.minio.MinIO
42+
- MINIO_ENDPOINT=rustfs:9000
43+
- MINIO_EXTERNAL_ENDPOINT=${MINIO_EXTERNAL_ENDPOINT-localhost:${MINIO_PORT:-9000}}
44+
- MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY-rustfsadmin}
45+
- MINIO_SECRET_KEY=${MINIO_SECRET_KEY-rustfsadmin}
46+
- MINIO_BUCKET_NAME=${MINIO_BUCKET_NAME-oclapi2-dev}
47+
- MINIO_SECURE=${MINIO_SECURE-false}
2648
celery:
2749
build: .
2850
volumes:
@@ -93,3 +115,8 @@ services:
93115
ports:
94116
- 9200:9200
95117
restart: "no"
118+
environment:
119+
- m=4gb
120+
121+
volumes:
122+
rustfs-data:

0 commit comments

Comments
 (0)