Skip to content

Commit 7437d96

Browse files
authored
Added Python Library support
1 parent c648562 commit 7437d96

3 files changed

Lines changed: 190 additions & 6 deletions

File tree

README.md

Lines changed: 80 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ A powerful CLI and GUI tool to export Salesforce SOQL query results to local CSV
2121
- [Usage](#usage)
2222
- [CLI Usage](#cli-usage)
2323
- [GUI Usage](#gui-usage)
24+
- [Library Usage](#library-usage)
2425
- [Post-processing](#post-processing)
2526
- [Examples](#examples)
2627
- [Troubleshooting](#troubleshooting)
@@ -52,7 +53,8 @@ pip install -e "git+https://github.com/datsom1/soql2csv.git#egg=soql2csv[dev]"
5253
soql2csv requires Salesforce authentication credentials in a `.env` file.
5354

5455
Example `.env` file:
55-
```
56+
57+
```env
5658
SF_USERNAME=your_salesforce_username
5759
SF_PASSWORD=your_salesforce_password
5860
SF_SECURITY_TOKEN=your_salesforce_security_token
@@ -72,6 +74,7 @@ soql2csv [OPTIONS] SOQLFILE
7274
```
7375

7476
Get help:
77+
7578
```bash
7679
soql2csv --help
7780
```
@@ -99,7 +102,7 @@ soql2csv --help
99102
soql2csv path/to/query.soql --outfilename=Extract_Table__c.csv --outfilepath=./output
100103
```
101104

102-
**CLI Preview**
105+
### CLI Preview
103106

104107
![soql2csv v2 2 CLI helptext](https://github.com/user-attachments/assets/a630de3d-d0d3-4c12-a2f2-80129d39f157)
105108

@@ -113,6 +116,7 @@ soql2csv --gui
113116
```
114117

115118
The GUI provides fields for:
119+
116120
- SOQL File: Select your query file
117121
- Output Folder: Where exported CSVs will be saved
118122
- Env File: Path to your .env file with Salesforce credentials
@@ -121,13 +125,82 @@ The GUI provides fields for:
121125

122126
A console output area displays progress and results.
123127

124-
**Gui Preview**
128+
### GUI Preview
125129

126130
![soql2csv v2 2 GUI](https://github.com/user-attachments/assets/a0a18088-65ff-4f06-b0a7-cc5fc3a52d4e)
127131

132+
## Library Usage
133+
134+
You can also call `soql2csv` directly from Python code without invoking the CLI.
135+
136+
Primary helper:
137+
138+
```text
139+
export_soql_to_csv(soql_path, output_base_name, output_dir='.', env_path='.env', postprocess_path=None, timestamp=True) -> str
140+
```
141+
142+
Parameters:
143+
144+
| Parameter | Description |
145+
|-----------|-------------|
146+
| `soql_path` | Path to the `.soql` file containing your query. |
147+
| `output_base_name` | Base name for the exported CSV (with or without `.csv`). |
148+
| `output_dir` | Destination directory (created if it does not exist). Default: current directory. |
149+
| `env_path` | Path to the `.env` file containing Salesforce credentials. Default: `.env`. |
150+
| `postprocess_path` | Optional path to a Python script for post-processing. Receives temp_input and final_output CSV paths as argv[1], argv[2]. |
151+
| `timestamp` | If True (default) prefixes filename like the CLI with a timestamp. Set to False to suppress. |
152+
153+
Returns: Absolute path to the final CSV file.
154+
155+
Example:
156+
157+
```python
158+
from soql2csv import export_soql_to_csv
159+
160+
csv_path = export_soql_to_csv(
161+
soql_path="queries/Accounts.soql",
162+
output_base_name="Accounts.csv",
163+
output_dir="./exports",
164+
env_path="./salesforce.env", # or leave as default '.env'
165+
postprocess_path=None, # or a script like 'scripts/clean_accounts.py'
166+
timestamp=True # set False to disable timestamp prefix
167+
)
168+
169+
print("CSV exported to", csv_path)
170+
```
171+
172+
Minimal example (defaults to `.env` in cwd, current directory output, adds timestamp):
173+
174+
```python
175+
from soql2csv import export_soql_to_csv
176+
export_soql_to_csv("query.soql", "MyData")
177+
```
178+
179+
With post-processing (script signature identical to CLI expectations):
180+
181+
```python
182+
from soql2csv import export_soql_to_csv
183+
export_soql_to_csv(
184+
soql_path="query.soql",
185+
output_base_name="FilteredData",
186+
postprocess_path="scripts/filter_script.py"
187+
)
188+
```
189+
190+
Errors you might encounter:
191+
192+
| Exception | Cause |
193+
|-----------|-------|
194+
| `FileNotFoundError` | The `.soql`, `.env`, or postprocess script path does not exist. |
195+
| `EnvironmentError` | Missing required Salesforce environment variables. |
196+
| `SalesforceAPIError` | Any Salesforce API interaction failed. |
197+
198+
The function internally mirrors CLI logic: loads env vars, executes Bulk API v2 query, waits for completion, downloads paginated results, optionally runs a postprocess script (on a temporary copy), and returns the final CSV path.
199+
128200
## Post-processing
129201

130202
Post-processing scripts allow you to transform the exported data. The script automatically receives two arguments:
203+
131204
1. `sys.argv[1]`: Path to the input CSV file (exported from Salesforce)
132205
2. `sys.argv[2]`: Path to the output CSV file (to write processed data)
133206

@@ -185,23 +258,27 @@ with open(output_csv, 'w', newline='', encoding='utf-8') as outfile:
185258
```
186259

187260
Run with:
261+
188262
```bash
189263
soql2csv query.soql --outfilename=Extract.csv --outfilepath=./output --postprocess=process_script.py
190264
```
191265

192266
## Examples
193267

194268
Basic extraction:
269+
195270
```bash
196271
soql2csv queries/Accounts.soql --outfilename=Accounts.csv --outfilepath=./exports
197272
```
198273

199274
Using a custom environment file:
275+
200276
```bash
201277
soql2csv queries/Contacts.soql --outfilename=Contacts.csv --outfilepath=./exports --envpath=./salesforce.env
202278
```
203279

204280
With post-processing:
281+
205282
```bash
206283
soql2csv queries/Opportunities.soql --outfilename=Opportunities.csv --outfilepath=./exports --postprocess=scripts/clean_opps.py
207284
```

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ def readme():
77

88
setup(
99
name="soql2csv",
10-
version="2.2",
10+
version="2.3",
1111
description="A powerful CLI and GUI tool to export Salesforce SOQL query results to CSV using Bulk API v2.",
1212
long_description=readme(),
1313
long_description_content_type="text/markdown",

soql2csv/__init__.py

Lines changed: 109 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@
66
import requests
77
import sys
88
import logging
9-
from typing import Tuple
9+
from pathlib import Path
10+
from typing import Tuple, Optional
1011

1112
logger = logging.getLogger("soql2csv")
1213
logging.basicConfig(level=logging.INFO, format="[%(levelname)s] %(message)s")
@@ -190,4 +191,110 @@ def generate_csv_from_soql(soql_file_location: str, output_csv_location: str) ->
190191
raise SalesforceAPIError(f"Job did not complete successfully. Status: {status}")
191192
logger.info("Downloading results...")
192193
download_bulk_v2_results(instance_url, access_token, job_id, output_csv_location)
193-
logger.info(f"Results written to {output_csv_location}")
194+
logger.info(f"Results written to {output_csv_location}")
195+
196+
# ------------------------------ Public Library API ------------------------------
197+
def export_soql_to_csv(
198+
soql_path: str,
199+
output_base_name: str,
200+
output_dir: str = ".",
201+
env_path: str = ".env",
202+
postprocess_path: Optional[str] = None,
203+
timestamp: bool = True,
204+
) -> str:
205+
"""Programmatic helper to export a SOQL query to CSV (with optional post-processing).
206+
207+
This mirrors the CLI / GUI behavior in a single simple call.
208+
209+
Parameters
210+
----------
211+
soql_path : str
212+
Path to the .soql file containing the query.
213+
output_base_name : str
214+
Base name for the output CSV (with or without .csv extension).
215+
output_dir : str, default "."
216+
Directory where the CSV should be written (created if missing).
217+
env_path : str, default ".env"
218+
Path to a .env file containing Salesforce credentials.
219+
postprocess_path : str | None, default None
220+
Optional path to a Python script to postprocess the CSV. The script
221+
will be invoked as: python postprocess_path temp_input_csv final_output_csv
222+
timestamp : bool, default True
223+
Whether to prefix the file with a timestamp like the CLI (YYYY-MM-DD_HH-MM-SS_Export_...).
224+
225+
Returns
226+
-------
227+
str
228+
Absolute path to the final (possibly post-processed) CSV file.
229+
230+
Raises
231+
------
232+
FileNotFoundError
233+
If the provided soql_path (or postprocess script path) does not exist.
234+
EnvironmentError
235+
If required Salesforce environment variables are missing.
236+
SalesforceAPIError
237+
If any Salesforce interaction fails.
238+
"""
239+
from dotenv import load_dotenv # Local import so library users without dotenv still get a clear error earlier
240+
import subprocess
241+
import shutil
242+
243+
soql_file = Path(soql_path)
244+
if not soql_file.is_file():
245+
raise FileNotFoundError(f"SOQL file not found: {soql_path}")
246+
247+
output_dir_path = Path(output_dir)
248+
output_dir_path.mkdir(parents=True, exist_ok=True)
249+
250+
base_name = output_base_name
251+
if not base_name.lower().endswith(".csv"):
252+
base_name += ".csv"
253+
254+
if timestamp:
255+
ts = time.strftime("%Y-%m-%d_%H-%M-%S")
256+
filename = f"{ts}_Export_{base_name}"
257+
else:
258+
filename = base_name
259+
260+
final_csv_path = output_dir_path / filename
261+
262+
# Load environment variables
263+
if env_path:
264+
env_file = Path(env_path)
265+
if not env_file.is_file():
266+
raise FileNotFoundError(f".env file not found: {env_path}")
267+
load_dotenv(str(env_file), override=True)
268+
269+
start = time.time()
270+
generate_csv_from_soql(str(soql_file), str(final_csv_path))
271+
272+
if postprocess_path:
273+
post_script = Path(postprocess_path)
274+
if not post_script.is_file():
275+
raise FileNotFoundError(f"Postprocess script not found: {postprocess_path}")
276+
temp_csv = final_csv_path.with_name(final_csv_path.stem + "_temp.csv")
277+
shutil.copy2(final_csv_path, temp_csv)
278+
try:
279+
subprocess.run([sys.executable, str(post_script), str(temp_csv), str(final_csv_path)], check=True)
280+
finally:
281+
if temp_csv.exists():
282+
try:
283+
temp_csv.unlink()
284+
except OSError:
285+
pass
286+
287+
elapsed = time.time() - start
288+
logger.info(f"Completed export in {elapsed:.2f}s -> {final_csv_path}")
289+
return str(final_csv_path.absolute())
290+
291+
__all__ = [
292+
"SalesforceAPIError",
293+
"salesforce_login",
294+
"create_bulk_v2_query_job",
295+
"poll_bulk_v2_job",
296+
"download_bulk_v2_results",
297+
"strip_soql_comments",
298+
"generate_csv_from_soql",
299+
"export_soql_to_csv",
300+
]

0 commit comments

Comments
 (0)