|
| 1 | +# SQL Plugin |
| 2 | + |
| 3 | +A SQL datasource plugin for Perses that supports multiple SQL databases with interactive querying capabilities. |
| 4 | + |
| 5 | +## Supported Databases |
| 6 | + |
| 7 | +- **PostgreSQL** (with TimescaleDB support) |
| 8 | +- **MySQL** |
| 9 | +- **MariaDB** |
| 10 | + |
| 11 | +## Features |
| 12 | + |
| 13 | +- **SQL Datasource**: Configure connections to SQL databases via Perses proxy |
| 14 | +- **Time Series Queries**: Query time-series data with automatic visualization |
| 15 | +- **Explore Mode**: Interactive SQL query builder with Table and Graph views (Prometheus-style) |
| 16 | +- **SQL Macros**: Use `$__timeFilter`, `$__timeFrom`, `$__timeTo`, `$__interval` in queries |
| 17 | +- **Multi-Database Support**: Connect to PostgreSQL, MySQL, and MariaDB |
| 18 | +- **TLS/SSL Support**: Secure connections with customizable SSL modes and certificate management |
| 19 | +- **Secret Management**: Store credentials securely using Perses secrets |
| 20 | +- **Connection Pooling**: Efficient connection management via backend proxy |
| 21 | + |
| 22 | +## Installation |
| 23 | + |
| 24 | +This plugin is part of the Perses plugins monorepo. Install dependencies: |
| 25 | + |
| 26 | +```bash |
| 27 | +cd perses/plugins/sql |
| 28 | +npm install |
| 29 | +``` |
| 30 | + |
| 31 | +## Usage |
| 32 | + |
| 33 | +### Creating a SQL Datasource |
| 34 | + |
| 35 | +1. Navigate to **Configuration → Datasources** in Perses UI |
| 36 | +2. Click **Add Datasource** |
| 37 | +3. Select **SQL Datasource** |
| 38 | +4. Configure: |
| 39 | + - **Driver**: Choose database type (PostgreSQL, MySQL, or MariaDB) |
| 40 | + - **Host**: Database host and port (e.g., `localhost:5432`) |
| 41 | + - **Database**: Database name |
| 42 | + - **Secret**: Select a secret containing credentials (BasicAuth) |
| 43 | + - **SSL Mode** (PostgreSQL): `disable`, `require`, `verify-ca`, or `verify-full` |
| 44 | + - **TLS Certificates** (optional): CA cert, client cert, client key |
| 45 | + |
| 46 | +### Using the Explore Mode |
| 47 | + |
| 48 | +1. Navigate to **Explore** in Perses UI |
| 49 | +2. Select **SQL Explorer** |
| 50 | +3. Choose your SQL datasource |
| 51 | +4. Write your SQL query in the editor |
| 52 | +5. Switch between **Table** and **Graph** views |
| 53 | +6. Click **Run Query** to execute |
| 54 | + |
| 55 | +Example query: |
| 56 | +```sql |
| 57 | +SELECT |
| 58 | + timestamp AS time, |
| 59 | + cpu_percent AS value, |
| 60 | + host |
| 61 | +FROM system_metrics |
| 62 | +WHERE $__timeFilter(timestamp) |
| 63 | +ORDER BY timestamp DESC |
| 64 | +LIMIT 100 |
| 65 | +``` |
| 66 | + |
| 67 | +### Time Series Queries in Dashboards |
| 68 | + |
| 69 | +Add a panel to your dashboard: |
| 70 | + |
| 71 | +1. **Add Panel** |
| 72 | +2. Select **Query Type**: Time Series Query |
| 73 | +3. Select **Plugin**: SQL Time Series Query |
| 74 | +4. Select **Datasource**: Your SQL datasource |
| 75 | +5. Write your query: |
| 76 | + |
| 77 | +```sql |
| 78 | +SELECT |
| 79 | + time_bucket('$__interval seconds', timestamp) AS time, |
| 80 | + host AS label, |
| 81 | + AVG(cpu_percent) AS value |
| 82 | +FROM system_metrics |
| 83 | +WHERE $__timeFilter(timestamp) |
| 84 | +GROUP BY time, host |
| 85 | +ORDER BY time |
| 86 | +``` |
| 87 | + |
| 88 | +### Available SQL Macros |
| 89 | + |
| 90 | +The plugin automatically replaces these macros in your queries: |
| 91 | + |
| 92 | +- **`$__timeFilter(column)`**: Generates `column >= 'start' AND column <= 'end'` |
| 93 | +- **`$__timeFrom`**: Start timestamp of the time range |
| 94 | +- **`$__timeTo`**: End timestamp of the time range |
| 95 | +- **`$__interval`**: Calculated interval in seconds (for time bucketing) |
| 96 | +- **`$__interval_ms`**: Calculated interval in milliseconds |
| 97 | + |
| 98 | +Example with macros: |
| 99 | +```sql |
| 100 | +SELECT |
| 101 | + time_bucket('$__interval seconds', created_at) AS time, |
| 102 | + category, |
| 103 | + COUNT(*) AS count |
| 104 | +FROM transactions |
| 105 | +WHERE $__timeFilter(created_at) |
| 106 | + AND status = 'completed' |
| 107 | +GROUP BY time, category |
| 108 | +ORDER BY time |
| 109 | +``` |
| 110 | + |
| 111 | +Gets transformed to: |
| 112 | +```sql |
| 113 | +SELECT |
| 114 | + time_bucket('300 seconds', created_at) AS time, |
| 115 | + category, |
| 116 | + COUNT(*) AS count |
| 117 | +FROM transactions |
| 118 | +WHERE created_at >= '2026-01-25T10:00:00Z' AND created_at <= '2026-01-25T11:00:00Z' |
| 119 | + AND status = 'completed' |
| 120 | +GROUP BY time, category |
| 121 | +ORDER BY time |
| 122 | +``` |
| 123 | + |
| 124 | +## Query Requirements |
| 125 | + |
| 126 | +### For Time Series Visualization |
| 127 | + |
| 128 | +Your query must return: |
| 129 | +- **Time column**: Named `time` or aliased as `time` (timestamp format) |
| 130 | +- **Value column**: Named `value` or aliased as `value` (numeric) |
| 131 | +- **Label columns** (optional): For series differentiation (e.g., `host`, `sensor_id`) |
| 132 | + |
| 133 | +### Supported Time Formats |
| 134 | + |
| 135 | +- ISO 8601: `2026-01-25T10:00:00Z` |
| 136 | +- Unix timestamp (seconds): `1706176800` |
| 137 | +- Unix timestamp (milliseconds): `1706176800000` |
| 138 | + |
| 139 | +## Database-Specific Features |
| 140 | + |
| 141 | +### PostgreSQL / TimescaleDB |
| 142 | + |
| 143 | +- Supports `time_bucket()` for time aggregation |
| 144 | +- SSL modes: `disable`, `require`, `verify-ca`, `verify-full` |
| 145 | +- Advanced connection options via PostgreSQL config |
| 146 | + |
| 147 | +### MySQL / MariaDB |
| 148 | + |
| 149 | +- Standard SQL queries |
| 150 | +- TLS/SSL support |
| 151 | +- Connection timeout and max connections configuration |
| 152 | + |
| 153 | +## Development |
| 154 | + |
| 155 | +### Prerequisites |
| 156 | + |
| 157 | +- Node.js >= 22 |
| 158 | +- npm >= 10 |
| 159 | +- Docker (for test databases) |
| 160 | + |
| 161 | +### Setup |
| 162 | + |
| 163 | +```bash |
| 164 | +# Install dependencies |
| 165 | +npm install |
| 166 | + |
| 167 | +# Start test databases |
| 168 | +make db-up |
| 169 | + |
| 170 | +# Run tests |
| 171 | +npm test |
| 172 | + |
| 173 | +# Type checking |
| 174 | +npm run type-check |
| 175 | + |
| 176 | +# Start development server |
| 177 | +percli plugin start . |
| 178 | +``` |
| 179 | + |
| 180 | +### Testing |
| 181 | + |
| 182 | +```bash |
| 183 | +# Run all tests |
| 184 | +npm test |
| 185 | + |
| 186 | +# Run with coverage |
| 187 | +npm test -- --coverage |
| 188 | + |
| 189 | +# Start test databases |
| 190 | +make db-up |
| 191 | + |
| 192 | +# Verify test data |
| 193 | +make verify-all |
| 194 | + |
| 195 | +# Stop databases |
| 196 | +make db-down |
| 197 | +``` |
| 198 | + |
| 199 | +## Architecture |
| 200 | + |
| 201 | +- **Frontend** (TypeScript/React): Datasource editor, query editor, explore mode |
| 202 | +- **Backend** (Go): SQL proxy in Perses core (`internal/api/impl/proxy/`) |
| 203 | +- **Schemas** (CUE): Configuration validation |
| 204 | + |
| 205 | +All database connections and query execution happen server-side via the Perses backend proxy for security. |
| 206 | + |
| 207 | +## Examples |
| 208 | + |
| 209 | +See `test-data/` directory for sample database schemas and queries: |
| 210 | +- `test-data/postgres/` - PostgreSQL with TimescaleDB examples |
| 211 | +- `test-data/mysql/` - MySQL examples |
| 212 | +- `test-data/mariadb/` - MariaDB examples |
| 213 | + |
| 214 | +## Contributing |
| 215 | + |
| 216 | +See [CONTRIBUTING.md](CONTRIBUTING.md) for development guidelines. |
| 217 | + |
| 218 | +## License |
| 219 | + |
| 220 | +Apache License 2.0 |
0 commit comments