Custom serializer for step/workflow output

[zihaolam]

Desired Behavior

Currently serializers are using JSON.stringify/JSON.parse. Any plans to support custom serializers to support non serializable json data? Just ran into a gotcha case where Date was serialized to string but still typed as a Date in step output.

Example Usage

const backend = BackendSqlite.connect(dbPath, { serializer: {
    serialize(data) {
        return devalue.stringify(data)
    },
    deserialize(data: string) {
        return devalue.parse(data)
    }
})

If there are no plans to support custom serializers, i guess it might be good to type StepOutputs in a way that makes typescript complain errors when output returned are not serializable

Maybe Output extends JsonValue or something

export interface StepApi {
  run: <Output extends JsonValue>(
    config: Readonly<StepFunctionConfig>,
    fn: StepFunction<Output>,
  ) => Promise<Output>;
}

[jamescmartinez]

Thanks for this. Some possible approaches:

1. Treat all payloads as JSON-only by tightening the types so TS rejects non-JSON values. This is probably the smallest and most predictable fix.

2. Like you suggested, we could allow something like a serializer option on the backend that would support Date and other JS values, but it would have a much larger design surface: replay compatibility, backend portability, migrations if the serializer changes, etc.

My current leaning is #1. What do you think?

[zihaolam]

1. Yep, i agree with 1 as quickfix, to make sure ppl don't step into unpredictable bug like i did (I can submit a PR for this)
2. Agree as well, it is a pretty big change. Not sure whats the best way but what do u think about versioning for serializers, with pattern similar to implementWorkflow versioning