Skip to content

Two way serialization#1838

Open
workaholicAB wants to merge 18 commits into
plotly:devfrom
workaholicAB:two-way-serialization
Open

Two way serialization#1838
workaholicAB wants to merge 18 commits into
plotly:devfrom
workaholicAB:two-way-serialization

Conversation

@workaholicAB

Copy link
Copy Markdown
Contributor

Replaces previous PR with squashed commits

Idea

Provide non-lossy data exchange between server & renderer client in initial layout and callbacks, namely two-way serialization that allows either party sends any object then receive it back in its original type.

CHANGELOG

Added a serialization support of pandas.DataFrame as the very first type, which will be followed by a few more types like numpy.array, DateTime, Plotly figures and more.

Quick preview of how serialization work

- Layout

Before :

xValues = numpy.linspace(0, 3, 4)
yValues = numpy.linspace(4, 7, 4)
df = pandas.DataFrame({'x': xValues, 'y': yValues})
table = DataTable(data=df.to_dict('records'), id='data_table_1')

After :

xValues = numpy.linspace(0, 3, 4)
yValues = numpy.linspace(4, 7, 4)
df = pandas.DataFrame({'x': xValues, 'y': yValues})
table = DataTable(data=df, id='data_table_1')

- Callback

Before :

@callback(Output('data_table_2', 'data'), Input('data_table_1', 'data'))
def update(data): # <-- You would receive array object
	return data

After :

@callback(Output('data_table_2', 'data'), Input('data_table_1', 'data'))
def update(data): # <-- You would receive pandas.DataFrame object as it's what you've passed to data_table_1
	return data

workaholicAB and others added 3 commits November 17, 2021 03:40
* feat: basic serializer implemented for TreeContainer

* fix: added check for serializable component create component

* feat: deserialize values in renderer side

* wip: quick pseudo codes

* chore: working on descriptive comments

* feat: implemented setting props in treeContainer

* feat implemented serializedKeys for component

* feat: serialization basically done up to milestone 4

* chore: some code reset to clean unnecessary changes

* fix: lint fix

* chore: add pandas to requirement

* update serialization

* feat: (wip) add to_parquet serialization

* chore: add some todo comments

* feat: (wip) implemented parquet reader & writer in JS side

* feat: (wip) add 3 different dataframe formats

* fix: avoid try/except as much as possible

* fix: remove PROP_ID, use tempfile for parquet output

* Set up parquetjs-lite (#8)

* feat: clean up codes and prepare for PR review

* fix: bug fix

* feat: changed serialization applying

* chore: minor refactor & clean up

* chore: lint fixes

* feat: add `columns` to serialized dataframe

* fix: deserialize bug

* fix: lint

* Two way serialization reading parquets (#9)

* wip: async deserialization

* feat: finish async deserialize, improve serialization with more additional/autofilled props

* chore: minor refactor

Co-authored-by: workaholicpanda <>

* fix: parquets reader

* fix: async deserialization

* fix: make serialize value async

* fixed asyncronize issue in renderer side

* changed file open to with

* fix: use to_dict for now

* updated package-lock files

* just trying

* Revert "just trying"

This reverts commit f16df3d.

* Revert "updated package-lock files"

This reverts commit 9d072ab.

* trying to disable serialize

* formatted code style to fix testing issue

* update renderer package-lock.json

Co-authored-by: workaholicpanda <>
Co-authored-by: Szabolcs Markó <akksi@akksi.eu>
@gvwilson gvwilson self-assigned this Jul 23, 2024
@gvwilson gvwilson removed their assignment Aug 2, 2024
@gvwilson gvwilson added feature something new P2 considered for next cycle community community contribution labels Aug 13, 2024
@ndrezn

ndrezn commented Aug 22, 2024

Copy link
Copy Markdown
Member

Assigning myself as we are exploring this feature for Dash. It's not clear this is the implementation we will move forward with but this is a great proposed strategy for handling Python objects directly in Dash components!

@ndrezn ndrezn self-assigned this Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community community contribution feature something new P2 considered for next cycle

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants