Skip to content

Big int #847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ArnabChatterjee20k wants to merge 30 commits into
base: main
Choose a base branch
from
Draft

Big int #847

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
af321ee
Implement BIGINT support across database adapters and add getLimitFor…
ArnabChatterjee20k Mar 27, 2026
3137084
Add support for VAR_BIGINT in Database and Attribute classes
ArnabChatterjee20k Mar 27, 2026
b5c7447
Add tests for BIGINT attribute handling and validation limits
ArnabChatterjee20k Mar 27, 2026
bd14576
Update getLimitForBigInt method to return 4294967295 for Mongo and SQ…
ArnabChatterjee20k Mar 27, 2026
acfeae1
updated bigint mongo
ArnabChatterjee20k Mar 27, 2026
07358a5
Enhance BIGINT handling in MariaDB and SQL adapters
ArnabChatterjee20k Mar 27, 2026
909d8a5
Add VAR_BIGINT to Database class attribute types
ArnabChatterjee20k Mar 27, 2026
f2acd8c
Enhance BIGINT size limit validation in Database class
ArnabChatterjee20k Mar 27, 2026
602eefc
Refine BIGINT handling in Structure and Filter validators
ArnabChatterjee20k Mar 27, 2026
14828d9
Refine BIGINT type determination in Postgres adapter
ArnabChatterjee20k Mar 27, 2026
080de0e
Remove BIGINT size limit validation from Database and Validator classes
ArnabChatterjee20k Mar 30, 2026
73e917e
Add support for VAR_BIGINT in Database class and corresponding tests
ArnabChatterjee20k Mar 30, 2026
ef5c779
updated tests
ArnabChatterjee20k Mar 30, 2026
4490a49
big int validator for attribute
ArnabChatterjee20k Mar 30, 2026
4c5aeb1
updated
ArnabChatterjee20k Mar 30, 2026
8137486
updated attribute validator
ArnabChatterjee20k Mar 30, 2026
b87cf3a
updated
ArnabChatterjee20k Mar 30, 2026
ad89dab
Merge remote-tracking branch 'origin/main' into big-init
ArnabChatterjee20k Apr 1, 2026
5ee5d1b
Add support for unsigned big integers in database adapters and valida…
ArnabChatterjee20k Apr 8, 2026
38e31f4
updated
ArnabChatterjee20k Apr 8, 2026
cafb353
updated
ArnabChatterjee20k Apr 8, 2026
dbe3537
Refactor attribute size handling in database adapters to support both…
ArnabChatterjee20k Apr 8, 2026
ba45e22
Enhance attribute validation to support string type for big integers
ArnabChatterjee20k Apr 8, 2026
75a7f28
Refactor attribute size handling to enforce integer type across datab…
ArnabChatterjee20k Apr 8, 2026
946b841
updated
ArnabChatterjee20k Apr 8, 2026
fafacee
updated
ArnabChatterjee20k Apr 8, 2026
fe3bf72
updated
ArnabChatterjee20k Apr 8, 2026
d3282bc
updated
ArnabChatterjee20k Apr 8, 2026
1761511
updated the type for big int validator
ArnabChatterjee20k Apr 9, 2026
0e43fd4
updated
ArnabChatterjee20k Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions src/Database/Adapter.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -880,6 +880,13 @@ abstract public function getLimitForString(): int;
*/
abstract public function getLimitForInt(): int;

/**
* Get max BIGINT limit
*
* @return int
*/
abstract public function getLimitForBigInt(): int;

/**
* Get maximum attributes limit.
*
Expand Down Expand Up @@ -1442,6 +1449,11 @@ abstract public function decodeLinestring(string $wkb): array;
*/
abstract public function decodePolygon(string $wkb): array;

public function getSupportForUnsignedBigInt(): bool
{
return false;
}

/**
* Returns the document after casting
* @param Document $collection
Expand Down
11 changes: 10 additions & 1 deletion src/Database/Adapter/MariaDB.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1734,6 +1734,10 @@ protected function getSQLType(string $type, int $size, bool $signed = true, bool

return 'INT' . $signed;

case Database::VAR_BIGINT:
$signed = ($signed) ? '' : ' UNSIGNED';
return 'BIGINT' . $signed;

case Database::VAR_FLOAT:
$signed = ($signed) ? '' : ' UNSIGNED';
return 'DOUBLE' . $signed;
Expand All @@ -1748,7 +1752,7 @@ protected function getSQLType(string $type, int $size, bool $signed = true, bool
return 'DATETIME(3)';

default:
throw new DatabaseException('Unknown type: ' . $type . '. Must be one of ' . Database::VAR_STRING . ', ' . Database::VAR_VARCHAR . ', ' . Database::VAR_TEXT . ', ' . Database::VAR_MEDIUMTEXT . ', ' . Database::VAR_LONGTEXT . ', ' . Database::VAR_INTEGER . ', ' . Database::VAR_FLOAT . ', ' . Database::VAR_BOOLEAN . ', ' . Database::VAR_DATETIME . ', ' . Database::VAR_RELATIONSHIP . ', ' . Database::VAR_POINT . ', ' . Database::VAR_LINESTRING . ', ' . Database::VAR_POLYGON);
throw new DatabaseException('Unknown type: ' . $type . '. Must be one of ' . Database::VAR_STRING . ', ' . Database::VAR_VARCHAR . ', ' . Database::VAR_TEXT . ', ' . Database::VAR_MEDIUMTEXT . ', ' . Database::VAR_LONGTEXT . ', ' . Database::VAR_INTEGER . ', ' . Database::VAR_BIGINT . ', ' . Database::VAR_FLOAT . ', ' . Database::VAR_BOOLEAN . ', ' . Database::VAR_DATETIME . ', ' . Database::VAR_RELATIONSHIP . ', ' . Database::VAR_POINT . ', ' . Database::VAR_LINESTRING . ', ' . Database::VAR_POLYGON);
}
}

Expand Down Expand Up @@ -2251,6 +2255,11 @@ public function getSupportForObject(): bool
return false;
}

public function getSupportForUnsignedBigInt(): bool
Copy link
Copy Markdown
Contributor Author

@ArnabChatterjee20k ArnabChatterjee20k Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mongodb, postgresql, sqlite doesn't support unsigned bigint

{
return true;
}

/**
* Are object (JSON) indexes supported?
*
Expand Down
12 changes: 12 additions & 0 deletions src/Database/Adapter/Mongo.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1319,6 +1319,7 @@ public function castingAfter(Document $collection, Document $document): Document
foreach ($value as &$node) {
switch ($type) {
case Database::VAR_INTEGER:
case Database::VAR_BIGINT:
$node = (int)$node;
break;
case Database::VAR_DATETIME:
Expand Down Expand Up @@ -2220,6 +2221,7 @@ private function getMongoTypeCode(string $appwriteType): string
Database::VAR_MEDIUMTEXT => 'string',
Database::VAR_LONGTEXT => 'string',
Database::VAR_INTEGER => 'int',
Database::VAR_BIGINT => 'long',
Database::VAR_FLOAT => 'double',
Database::VAR_BOOLEAN => 'bool',
Database::VAR_DATETIME => 'date',
Expand Down Expand Up @@ -3013,6 +3015,16 @@ public function getLimitForInt(): int
return 4294967295;
}

/**
* Get max BIGINT limit
*
* @return int
*/
public function getLimitForBigInt(): int
{
return Database::MAX_BIG_INT;
}

/**
* Get maximum column limit.
* Returns 0 to indicate no limit
Expand Down
5 changes: 5 additions & 0 deletions src/Database/Adapter/Pool.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,11 @@ public function getLimitForInt(): int
return $this->delegate(__FUNCTION__, \func_get_args());
}

public function getLimitForBigInt(): int
{
return $this->delegate(__FUNCTION__, \func_get_args());
}

public function getLimitForAttributes(): int
{
return $this->delegate(__FUNCTION__, \func_get_args());
Expand Down
5 changes: 4 additions & 1 deletion src/Database/Adapter/Postgres.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1972,6 +1972,9 @@ protected function getSQLType(string $type, int $size, bool $signed = true, bool

return 'INTEGER';

case Database::VAR_BIGINT:
return 'BIGINT';

case Database::VAR_FLOAT:
return 'DOUBLE PRECISION';

Expand Down Expand Up @@ -2000,7 +2003,7 @@ protected function getSQLType(string $type, int $size, bool $signed = true, bool
return "VECTOR({$size})";

default:
throw new DatabaseException('Unknown Type: ' . $type . '. Must be one of ' . Database::VAR_STRING . ', ' . Database::VAR_VARCHAR . ', ' . Database::VAR_TEXT . ', ' . Database::VAR_MEDIUMTEXT . ', ' . Database::VAR_LONGTEXT . ', ' . Database::VAR_INTEGER . ', ' . Database::VAR_FLOAT . ', ' . Database::VAR_BOOLEAN . ', ' . Database::VAR_DATETIME . ', ' . Database::VAR_RELATIONSHIP . ', ' . Database::VAR_OBJECT . ', ' . Database::VAR_POINT . ', ' . Database::VAR_LINESTRING . ', ' . Database::VAR_POLYGON);
throw new DatabaseException('Unknown Type: ' . $type . '. Must be one of ' . Database::VAR_STRING . ', ' . Database::VAR_VARCHAR . ', ' . Database::VAR_TEXT . ', ' . Database::VAR_MEDIUMTEXT . ', ' . Database::VAR_LONGTEXT . ', ' . Database::VAR_INTEGER . ', ' . Database::VAR_BIGINT . ', ' . Database::VAR_FLOAT . ', ' . Database::VAR_BOOLEAN . ', ' . Database::VAR_DATETIME . ', ' . Database::VAR_RELATIONSHIP . ', ' . Database::VAR_OBJECT . ', ' . Database::VAR_POINT . ', ' . Database::VAR_LINESTRING . ', ' . Database::VAR_POLYGON);
}
}

Expand Down
14 changes: 14 additions & 0 deletions src/Database/Adapter/SQL.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -896,6 +896,16 @@ public function getLimitForInt(): int
return 4294967295;
}

/**
* Get max BIGINT limit
*
* @return int
*/
public function getLimitForBigInt(): int
{
return Database::MAX_BIG_INT;
Copy link
Copy Markdown
Contributor Author

@ArnabChatterjee20k ArnabChatterjee20k Apr 8, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here going other way around
It is the signed big int limit(supported by all adapter)
For adapters supporting unsigned int, in the attribute validator, we are validating via string comparison

}

/**
* Get maximum column limit.
* https://mariadb.com/kb/en/innodb-limitations/#limitations-on-schema
Expand Down Expand Up @@ -1164,6 +1174,10 @@ public function getAttributeWidth(Document $collection): int
}
break;

case Database::VAR_BIGINT:
$total += 8; // BIGINT 8 bytes
break;

case Database::VAR_FLOAT:
$total += 8; // DOUBLE 8 bytes
break;
Expand Down
5 changes: 5 additions & 0 deletions src/Database/Adapter/SQLite.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1019,6 +1019,11 @@ public function getSupportForObject(): bool
return false;
}

public function getSupportForUnsignedBigInt(): bool
{
return false;
}

/**
* Are object (JSON) indexes supported?
*
Expand Down
64 changes: 52 additions & 12 deletions src/Database/Database.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
use Utopia\Database\Validator\Attribute as AttributeValidator;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\Authorization\Input;
use Utopia\Database\Validator\BigInt as BigIntValidator;
use Utopia\Database\Validator\Index as IndexValidator;
use Utopia\Database\Validator\IndexDependency as IndexDependencyValidator;
use Utopia\Database\Validator\PartialStructure;
Expand All @@ -42,6 +43,7 @@ class Database
// Simple Types
public const VAR_STRING = 'string';
public const VAR_INTEGER = 'integer';
public const VAR_BIGINT = 'bigint';
public const VAR_FLOAT = 'double';
public const VAR_BOOLEAN = 'boolean';
public const VAR_DATETIME = 'datetime';
Expand Down Expand Up @@ -2512,6 +2514,7 @@ private function validateAttribute(
maxStringLength: $this->adapter->getLimitForString(),
maxVarcharLength: $this->adapter->getMaxVarcharLength(),
maxIntLength: $this->adapter->getLimitForInt(),
maxBigIntLength: $this->adapter->getLimitForBigInt(),
supportForSchemaAttributes: $this->adapter->getSupportForSchemaAttributes(),
supportForVectors: $this->adapter->getSupportForVectors(),
supportForSpatialAttributes: $this->adapter->getSupportForSpatialAttributes(),
Expand All @@ -2521,6 +2524,7 @@ private function validateAttribute(
filterCallback: fn ($id) => $this->adapter->filter($id),
isMigrating: $this->isMigrating(),
sharedTables: $this->getSharedTables(),
supportUnsignedBigInt: $this->adapter->getSupportForUnsignedBigInt(),
);

$validator->isValid($attribute);
Expand Down Expand Up @@ -2588,6 +2592,11 @@ protected function validateDefaultTypes(string $type, mixed $default): void
throw new DatabaseException('Default value ' . $default . ' does not match given type ' . $type);
}
break;
case Database::VAR_BIGINT:
if ($defaultType !== 'integer' && $defaultType !== 'string') {
throw new DatabaseException('Default value ' . $default . ' does not match given type ' . $type);
}
break;
case self::VAR_DATETIME:
if ($defaultType !== self::VAR_STRING) {
throw new DatabaseException('Default value ' . $default . ' does not match given type ' . $type);
Expand All @@ -2607,6 +2616,7 @@ protected function validateDefaultTypes(string $type, mixed $default): void
self::VAR_MEDIUMTEXT,
self::VAR_LONGTEXT,
self::VAR_INTEGER,
self::VAR_BIGINT,
self::VAR_FLOAT,
self::VAR_BOOLEAN,
self::VAR_DATETIME,
Expand Down Expand Up @@ -2909,6 +2919,16 @@ public function updateAttribute(string $collection, string $id, ?string $type =
throw new DatabaseException('Max size allowed for int is: ' . number_format($limit));
}
break;
case self::VAR_BIGINT:
$sizeString = BigIntValidator::normalizeUnsignedString((string)$size);
$limit = (!$signed && $this->adapter->getSupportForUnsignedBigInt())
? BigIntValidator::UNSIGNED_MAX
: BigIntValidator::SIGNED_MAX;

if (BigIntValidator::compareUnsignedStrings($sizeString, $limit) > 0) {
throw new DatabaseException('Max size allowed for bigint is: ' . BigIntValidator::formatIntegerString($limit));
}
break;
case self::VAR_FLOAT:
case self::VAR_BOOLEAN:
case self::VAR_DATETIME:
Expand Down Expand Up @@ -2975,6 +2995,7 @@ public function updateAttribute(string $collection, string $id, ?string $type =
self::VAR_MEDIUMTEXT,
self::VAR_LONGTEXT,
self::VAR_INTEGER,
self::VAR_BIGINT,
self::VAR_FLOAT,
self::VAR_BOOLEAN,
self::VAR_DATETIME,
Expand Down Expand Up @@ -3137,7 +3158,7 @@ public function updateAttribute(string $collection, string $id, ?string $type =
$collection,
$newKey ?? $id,
$originalType,
$originalSize,
(int)$originalSize,
$originalSigned,
$originalArray,
$originalKey,
Expand Down Expand Up @@ -5577,7 +5598,9 @@ public function createDocument(string $collection, Document $document): Document
$this->adapter->getIdAttributeType(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
null,
Copy link
Copy Markdown
Member

@abnegate abnegate Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to pass the adapter values here

Copy link
Copy Markdown
Contributor Author

@ArnabChatterjee20k ArnabChatterjee20k Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes cause mongodb, postgres, sqlite doesn't support unsigned bigint
So for the structure validator we needed this
passing null here as current document by default is null

$this->adapter->getSupportForUnsignedBigInt()
);
if (!$structure->isValid($document)) {
throw new StructureException($structure->getDescription());
Expand Down Expand Up @@ -5685,7 +5708,9 @@ public function createDocuments(
$this->adapter->getIdAttributeType(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
null,
$this->adapter->getSupportForUnsignedBigInt()
);
if (!$validator->isValid($document)) {
throw new StructureException($validator->getDescription());
Expand Down Expand Up @@ -6257,7 +6282,8 @@ public function updateDocument(string $collection, string $id, Document $documen
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes(),
$old
$old,
$this->adapter->getSupportForUnsignedBigInt()
);
if (!$structureValidator->isValid($document)) { // Make sure updated structure still apply collection rules (if any)
throw new StructureException($structureValidator->getDescription());
Expand Down Expand Up @@ -6378,7 +6404,8 @@ public function updateDocuments(
$this->adapter->getMaxUIDLength(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
$this->adapter->getSupportForUnsignedBigInt()
);

if (!$validator->isValid($queries)) {
Expand Down Expand Up @@ -6423,7 +6450,8 @@ public function updateDocuments(
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes(),
null // No old document available in bulk updates
null, // No old document available in bulk updates
$this->adapter->getSupportForUnsignedBigInt()
);

if (!$validator->isValid($updates)) {
Expand Down Expand Up @@ -7269,7 +7297,8 @@ public function upsertDocumentsWithIncrease(
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes(),
$old->isEmpty() ? null : $old
$old->isEmpty() ? null : $old,
$this->adapter->getSupportForUnsignedBigInt()
);

if (!$validator->isValid($document)) {
Expand Down Expand Up @@ -7423,6 +7452,7 @@ public function increaseDocumentAttribute(

$whiteList = [
self::VAR_INTEGER,
self::VAR_BIGINT,
self::VAR_FLOAT
];

Expand Down Expand Up @@ -7521,6 +7551,7 @@ public function decreaseDocumentAttribute(

$whiteList = [
self::VAR_INTEGER,
self::VAR_BIGINT,
self::VAR_FLOAT
];

Expand Down Expand Up @@ -8084,7 +8115,8 @@ public function deleteDocuments(
$this->adapter->getMaxUIDLength(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
$this->adapter->getSupportForUnsignedBigInt()
);

if (!$validator->isValid($queries)) {
Expand Down Expand Up @@ -8306,7 +8338,8 @@ public function find(string $collection, array $queries = [], string $forPermiss
$this->adapter->getMaxUIDLength(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
$this->adapter->getSupportForUnsignedBigInt()
);
if (!$validator->isValid($queries)) {
throw new QueryException($validator->getDescription());
Expand Down Expand Up @@ -8562,7 +8595,8 @@ public function count(string $collection, array $queries = [], ?int $max = null)
$this->adapter->getMaxUIDLength(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
$this->adapter->getSupportForUnsignedBigInt()
);
if (!$validator->isValid($queries)) {
throw new QueryException($validator->getDescription());
Expand Down Expand Up @@ -8635,7 +8669,8 @@ public function sum(string $collection, string $attribute, array $queries = [],
$this->adapter->getMaxUIDLength(),
$this->adapter->getMinDateTime(),
$this->adapter->getMaxDateTime(),
$this->adapter->getSupportForAttributes()
$this->adapter->getSupportForAttributes(),
$this->adapter->getSupportForUnsignedBigInt()
);
if (!$validator->isValid($queries)) {
throw new QueryException($validator->getDescription());
Expand Down Expand Up @@ -8900,6 +8935,7 @@ public function casting(Document $collection, Document $document): Document
foreach ($attributes as $attribute) {
$key = $attribute['$id'] ?? '';
$type = $attribute['type'] ?? '';
$signed = $attribute['signed'] ?? true;
$array = $attribute['array'] ?? false;
$value = $document->getAttribute($key, null);
if (is_null($value)) {
Expand Down Expand Up @@ -8932,6 +8968,11 @@ public function casting(Document $collection, Document $document): Document
case self::VAR_INTEGER:
$node = (int)$node;
break;
case self::VAR_BIGINT:
if (\is_string($node) && BigIntValidator::fitsPhpInt($node, $signed)) {
$node = (int)$node;
}
break;
case self::VAR_FLOAT:
$node = (float)$node;
break;
Expand All @@ -8948,7 +8989,6 @@ public function casting(Document $collection, Document $document): Document
return $document;
}


/**
* Encode Attribute
*
Expand Down
Loading
Loading