Skip to content

Commit 5cffb7b

Browse files
author
Peng Ren
committed
Add sqlalchemy support
1 parent eebeba5 commit 5cffb7b

8 files changed

Lines changed: 1673 additions & 2 deletions

docs/sqlalchemy_integration.md

Lines changed: 314 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,314 @@
1+
# PyMongoSQL SQLAlchemy Integration
2+
3+
PyMongoSQL now includes a full SQLAlchemy dialect, enabling you to use MongoDB with SQLAlchemy's ORM and Core functionality through familiar SQL syntax.
4+
5+
## Version Compatibility
6+
7+
**Supported SQLAlchemy Versions:**
8+
- ✅ SQLAlchemy 1.4.x (LTS)
9+
- ✅ SQLAlchemy 2.0.x (Current)
10+
- ✅ SQLAlchemy 2.1.x+ (Future)
11+
12+
The dialect automatically detects your SQLAlchemy version and adapts accordingly. Both 1.x and 2.x APIs are supported seamlessly.
13+
14+
## Quick Start
15+
16+
### Installation
17+
18+
```bash
19+
# Install SQLAlchemy (1.4+ or 2.x)
20+
pip install "sqlalchemy>=1.4.0,<3.0.0"
21+
22+
# PyMongoSQL already includes the dialect
23+
```
24+
25+
### Version Detection
26+
27+
```python
28+
import pymongosql
29+
30+
# Check SQLAlchemy support
31+
print(f"SQLAlchemy installed: {pymongosql.__supports_sqlalchemy__}")
32+
print(f"SQLAlchemy version: {pymongosql.__sqlalchemy_version__}")
33+
print(f"SQLAlchemy 2.x: {pymongosql.__supports_sqlalchemy_2x__}")
34+
35+
# Get compatibility info
36+
from pymongosql.sqlalchemy_compat import check_sqlalchemy_compatibility
37+
info = check_sqlalchemy_compatibility()
38+
print(info['message'])
39+
```
40+
41+
### Basic Usage (Version-Compatible)
42+
43+
```python
44+
from sqlalchemy import create_engine, Column, String, Integer
45+
from sqlalchemy.orm import sessionmaker
46+
import pymongosql
47+
48+
# Method 1: Use compatibility helpers (recommended)
49+
from pymongosql.sqlalchemy_compat import get_base_class, create_pymongosql_engine
50+
51+
# Create engine with version-appropriate settings
52+
engine = create_pymongosql_engine("pymongosql://localhost:27017/mydb")
53+
54+
# Get version-compatible base class
55+
Base = get_base_class()
56+
57+
class User(Base):
58+
__tablename__ = 'users'
59+
60+
id = Column('_id', String, primary_key=True) # MongoDB's _id field
61+
username = Column(String, nullable=False)
62+
email = Column(String, nullable=False)
63+
age = Column(Integer)
64+
65+
# Create session with compatibility helper
66+
SessionMaker = pymongosql.get_session_maker(engine)
67+
session = SessionMaker()
68+
69+
# Use standard SQLAlchemy patterns (works with both 1.x and 2.x)
70+
user = User(id="user123", username="john", email="john@example.com", age=30)
71+
session.add(user)
72+
session.commit()
73+
74+
# Query with ORM (syntax identical across versions)
75+
users = session.query(User).filter(User.age >= 25).all()
76+
```
77+
78+
### Manual Version Handling
79+
80+
```python
81+
# Method 2: Manual version detection
82+
from sqlalchemy import create_engine, Column, String, Integer
83+
from sqlalchemy.orm import sessionmaker
84+
import pymongosql
85+
86+
# Check SQLAlchemy version
87+
if pymongosql.__supports_sqlalchemy_2x__:
88+
# SQLAlchemy 2.x approach
89+
from sqlalchemy.orm import DeclarativeBase
90+
91+
class Base(DeclarativeBase):
92+
pass
93+
94+
engine = create_engine("pymongosql://localhost:27017/mydb", future=True)
95+
else:
96+
# SQLAlchemy 1.x approach
97+
from sqlalchemy.ext.declarative import declarative_base
98+
Base = declarative_base()
99+
100+
engine = create_engine("pymongosql://localhost:27017/mydb")
101+
102+
# Model definition (identical for both versions)
103+
class User(Base):
104+
__tablename__ = 'users'
105+
id = Column('_id', String, primary_key=True)
106+
username = Column(String, nullable=False)
107+
108+
# Rest of the code is version-agnostic
109+
Session = sessionmaker(bind=engine)
110+
session = Session()
111+
```
112+
113+
## Features
114+
115+
### ✅ Supported SQLAlchemy Features
116+
117+
- **ORM Models**: Define models using `declarative_base()`
118+
- **Core Expressions**: Use SQLAlchemy Core for query building
119+
- **Sessions**: Full session management with commit/rollback
120+
- **Relationships**: Basic relationship mapping
121+
- **Query Building**: SQLAlchemy's query builder syntax
122+
- **Raw SQL**: Execute raw SQL through `text()` objects
123+
- **Connection Pooling**: Configurable connection pools
124+
- **Transactions**: Basic transaction support where MongoDB allows
125+
126+
### 🔧 MongoDB-Specific Adaptations
127+
128+
- **Primary Keys**: Automatically maps to MongoDB's `_id` field
129+
- **Collections**: SQL tables map to MongoDB collections
130+
- **Documents**: SQL rows map to MongoDB documents
131+
- **Schema-less**: Flexible schema handling for MongoDB's document nature
132+
- **JSON Support**: Native handling of nested documents and arrays
133+
- **Aggregation**: SQL GROUP BY translates to MongoDB aggregation pipelines
134+
135+
### ⚠️ Limitations
136+
137+
- **No Foreign Keys**: MongoDB doesn't enforce foreign key constraints
138+
- **No ALTER TABLE**: Schema changes must be handled at application level
139+
- **Limited Transactions**: Multi-document transactions have MongoDB limitations
140+
- **No Sequences**: Auto-incrementing IDs must be handled manually
141+
142+
## URL Format
143+
144+
The PyMongoSQL dialect uses the following URL format:
145+
146+
```
147+
pymongosql://[username:password@]host[:port]/database[?param1=value1&param2=value2]
148+
```
149+
150+
### Examples
151+
152+
```python
153+
# Basic connection
154+
"pymongosql://localhost:27017/mydb"
155+
156+
# With authentication
157+
"pymongosql://user:pass@localhost:27017/mydb"
158+
159+
# With MongoDB options
160+
"pymongosql://localhost:27017/mydb?ssl=true&replicaSet=rs0"
161+
162+
# Using helper function
163+
url = pymongosql.create_engine_url(
164+
host="mongo.example.com",
165+
port=27017,
166+
database="production",
167+
ssl=True,
168+
replicaSet="rs0"
169+
)
170+
```
171+
172+
## Advanced Usage
173+
174+
### Raw SQL Execution
175+
176+
```python
177+
from sqlalchemy import text
178+
179+
# Execute raw SQL
180+
with engine.connect() as conn:
181+
result = conn.execute(text("SELECT COUNT(*) FROM users WHERE age > 25"))
182+
count = result.scalar()
183+
```
184+
185+
### Aggregation Queries
186+
187+
```python
188+
# SQL aggregation translates to MongoDB aggregation pipeline
189+
from sqlalchemy import func
190+
191+
query = session.query(
192+
User.age,
193+
func.count(User.id).label('count')
194+
).group_by(User.age).order_by(User.age)
195+
196+
results = query.all()
197+
```
198+
199+
### JSON Document Operations
200+
201+
```python
202+
# Query nested document fields (if supported by your SQL parser)
203+
users_with_location = session.query(User).filter(
204+
text("profile->>'$.location' = 'New York'")
205+
).all()
206+
```
207+
208+
### Connection Configuration
209+
210+
```python
211+
from sqlalchemy import create_engine
212+
from sqlalchemy.pool import StaticPool
213+
214+
# Configure connection pool
215+
engine = create_engine(
216+
"pymongosql://localhost:27017/mydb",
217+
poolclass=StaticPool,
218+
pool_size=5,
219+
max_overflow=10,
220+
echo=True # Enable SQL logging
221+
)
222+
```
223+
224+
## Type Mapping
225+
226+
| SQL Type | MongoDB BSON Type | Notes |
227+
|----------|-------------------|-------|
228+
| VARCHAR, CHAR, TEXT | String | Text data |
229+
| INTEGER | Int32 | 32-bit integers |
230+
| BIGINT | Int64 | 64-bit integers |
231+
| FLOAT, REAL | Double | Floating point |
232+
| DECIMAL, NUMERIC | Decimal128 | High precision decimal |
233+
| BOOLEAN | Boolean | True/false values |
234+
| DATETIME, TIMESTAMP | Date | Date/time values |
235+
| JSON | Object/Array | Nested documents |
236+
| BINARY, BLOB | BinData | Binary data |
237+
238+
## Error Handling
239+
240+
```python
241+
from pymongosql.error import DatabaseError, OperationalError
242+
243+
try:
244+
session.query(User).all()
245+
except OperationalError as e:
246+
# Handle MongoDB connection errors
247+
print(f"Connection error: {e}")
248+
except DatabaseError as e:
249+
# Handle query/data errors
250+
print(f"Database error: {e}")
251+
```
252+
253+
## Migration from Raw PyMongoSQL
254+
255+
If you're already using PyMongoSQL directly, migrating to SQLAlchemy is straightforward:
256+
257+
### Before (Raw PyMongoSQL)
258+
```python
259+
import pymongosql
260+
261+
conn = pymongosql.connect("mongodb://localhost:27017/mydb")
262+
cursor = conn.cursor()
263+
cursor.execute("SELECT * FROM users WHERE age > 25")
264+
results = cursor.fetchall()
265+
```
266+
267+
### After (SQLAlchemy)
268+
```python
269+
from sqlalchemy import create_engine, text
270+
from sqlalchemy.orm import sessionmaker
271+
272+
engine = create_engine("pymongosql://localhost:27017/mydb")
273+
Session = sessionmaker(bind=engine)
274+
session = Session()
275+
276+
# Option 1: Raw SQL
277+
with engine.connect() as conn:
278+
result = conn.execute(text("SELECT * FROM users WHERE age > 25"))
279+
results = result.fetchall()
280+
281+
# Option 2: ORM
282+
results = session.query(User).filter(User.age > 25).all()
283+
```
284+
285+
## Best Practices
286+
287+
1. **Use _id for Primary Keys**: Always map your primary key to MongoDB's `_id` field
288+
2. **Schema Design**: Design your models considering MongoDB's document nature
289+
3. **Connection Pooling**: Configure appropriate pool sizes for your application
290+
4. **Error Handling**: Implement proper error handling for MongoDB-specific issues
291+
5. **Testing**: Use the provided test utilities for development
292+
293+
## Examples
294+
295+
See the `examples/sqlalchemy_integration.py` file for complete working examples and advanced usage patterns.
296+
297+
## Troubleshooting
298+
299+
### Common Issues
300+
301+
1. **"No dialect found"**: Ensure PyMongoSQL is properly installed and the dialect is registered
302+
2. **Connection errors**: Verify MongoDB is running and accessible
303+
3. **Schema issues**: Remember MongoDB is schema-less, some SQL patterns may not translate directly
304+
4. **Performance**: Use indexes appropriately in MongoDB for optimal query performance
305+
306+
### Debug Mode
307+
308+
Enable SQL logging to see generated queries:
309+
310+
```python
311+
engine = create_engine("pymongosql://localhost:27017/mydb", echo=True)
312+
```
313+
314+
This will print all SQL statements and their MongoDB translations to the console.

0 commit comments

Comments
 (0)