feat: Token pagination support (#896)

* feat: add new token based pagination support

* feat: add new token based pagination support

* chore: add token pagination

* chore: add token pagination test

* fix: update pagination

* feat: add token pagination support

* feat: add token pagination support

* feat: add null check

---------

Co-authored-by: shrutiburman <87537688+shrutiburman@users.noreply.github.com>

Author: kridai